Compare commits

...

4849 Commits

Author SHA1 Message Date
Nishant Das
5e89dac9f0 pin it (#11958) 2023-02-02 06:55:15 -03:00
Potuz
2579242669 modify slice in place (#11956)
* modify slice in place

* unit test

* comment explaning function

---------

Co-authored-by: terencechain <terence@prysmaticlabs.com>
2023-02-02 00:01:46 +00:00
terencechain
529d48d0c5 Capella: use builder (#11906) 2023-02-01 08:24:51 -08:00
Raul Jordan
b8bbeae740 Introduce Thread Safe Map Data Structure (#11940)
* fix thread safety issue

* gazelle

* push up benchmarks

* rev

* rem keys method

* shallow copy

* fxi
2023-02-01 14:32:01 +00:00
terencechain
59cb9f0dd7 Use head state only for bls messages broadcast (#11949)
* Use head state only for bls messages broadcast

* Fix build

---------

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2023-02-01 06:01:10 +00:00
Radosław Kapka
b10e59a144 Add finalized flag to API responses (#11947)
* server code

(cherry picked from commit fc68e6e78f2a84c66cd6c4db2a482cccdaf487d5)

* tests

* middleware

* more tests

* fix one more test
2023-01-31 17:58:16 -06:00
terencechain
c070283bf1 Call FCU with attribute on non head block (#11919)
* Call FCU with attribute on non head block

* Fix condition

* Filer save head

* Refactor

* Rm

* Add tests

* Fix race test

* Potuz feedback

* Reorder r != currentHeadRoot

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-01-31 19:17:16 +00:00
Nishant Das
3f67818a65 Utilize Read Only Head State (#11932) 2023-01-31 10:41:53 -08:00
Manu NALEPA
847835534d Validator client: Add missing endpoint.DeleteRequest - Fixes #11943 (#11944)
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2023-01-31 18:01:51 +00:00
Potuz
c2194c57b7 Rate limit broadcasting of BLS changes at the fork and at RPC endpoint (#11936) 2023-01-31 09:31:22 -08:00
int88
bd83018e76 delete from s.nodeByPayload as well when pruneFinalizedNodeByRootMap (#11813)
* delete from s.nodeByPayload as well when pruneFinalizedNodeByRootMap

* regresstion test

* add different payload node

---------

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-01-31 10:53:29 +00:00
terencechain
39114e6817 Use copied head state for metrics reporting (#11933)
* Use copied head state

* Move metrics to the end

---------

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-01-31 10:07:27 +00:00
terencechain
67a9701e06 Add spec tests v1.3.0 rc.2 (#11929)
* Update test

* Fix shas

* Fix shas

* Use hotfix path for spectests

* Gazelle

---------

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-01-31 09:42:09 +00:00
Nishant Das
5dc7741b30 Revert Bazel Hash Change (#11941) 2023-01-31 09:17:14 +00:00
james-prysm
53a135adbd update-go-pbs.sh: fixing script for mac users & updating bazel dependency (#11937)
* fixing script for mac users

* updating checksum for bazel tool

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-01-31 01:17:12 +00:00
Raul Jordan
950d0c0282 Update Bazel Grail Toolchain SHA and Protect Digest Map with Lock (#11939)
* lock digest map

* imports

* toolchain update
2023-01-30 22:40:48 +00:00
Nishant Das
e8c25effb0 Cache Fork Digest Computation (#11931) 2023-01-29 16:07:50 +00:00
Potuz
cc454bb42c Cycle the BLS changes pool when falling below a threshold (#11873)
* Cycle the BLS changes pool when falling below a threshold

* move cycle logic within pool

* set threshold at 2000

* fix conflict

* more fixes

---------

Co-authored-by: kasey <489222+kasey@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-01-28 14:42:03 +00:00
Sammy Rosso
9529c73ff1 EIP-4881: Spec implementation (#11720)
* Initial spec rewrite

* Finish adding merkle tree implementation

* Last bits

* Move reverse function

* Add comments

* Add deposit tree snapshot

* Add deposit tree

* Add comments + cleanup

* Fixes

* Add missing errors

* Small fixes

* Add unhandled error

* Cleanup

* Fix unsafe file.Close

* Add missing comments

* Small fixes

* Address some of deepSource' compaints

* Add depositCount check

* Add finalizedDeposit check

* Replace pointer magic with copy()

* Add test for slice reversal

* add back bytes method

* Add package level description

* Remove zerohash gen and add additional checks

* Add additional comments

* Small lint fixes

* Forgot an error

* Small fixes

* Move Uint64ToBytesLittleEndian32 + test

* Fix uint subtraction issue

* Move mixInLength below error handling

* Fix

* Fix deposit root

---------

Co-authored-by: rauljordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-01-27 17:35:25 +00:00
terencechain
eca129d8ff Better log for block that never became head (#11917)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-01-27 11:58:43 -05:00
terencechain
53a4939da4 Use correct attribute if there's a payload ID cache miss (#11918)
* Use correct attribute if there's a payload ID cache miss

* Test

* Default case unknown state version

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-01-27 11:13:47 +00:00
Manu NALEPA
efc1e06c6b Validator client beacon api non functional fixes (#11915)
* Validator Client Beacon API: Use Go idiomatic error handling

* Validator Client Beacon API: Use `buildURL`

`buildURL` sort params by keys.

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-01-27 10:44:23 +00:00
james-prysm
07210d7360 prysmctl: withdrawal ux improvement (#11909)
* adjusting submit bls to not early exit function in partial success

* Update cmd/prysmctl/validator/withdraw.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-01-27 09:57:36 +00:00
Sammy Rosso
9136d30121 Return error when using unsupported mnemonic language (#11805)
* Return error when using unsupported mnemonic language

* Fix failing tests

* More test fixes

* Add MnemonicLanguage to CreateWalletRequest

* Add MnemonicLanguage to remaining tests

* Add language to error

* Add missing (.)

* Set default language + fix renumbered fields

* Replace hardcoded language with default var

* default set

* gaz

* fix proto field setting

* fix up

* Remove WithMnemonicLanguage from tests

* Remove WithMnemonicLanguage from tests

* Fix conflicting information failure

* Gazelle + remove hardcoded english values

* fix

* Fix tests

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-01-26 23:44:38 +00:00
Potuz
56907bb2c6 Randomize bls changes at fork (#11923)
* Rate Limit broadcasting of BLS changes

* Rate limit BLS changes broadcasting

* random

* linting

* mock test

Co-authored-by: terencechain <terence@prysmaticlabs.com>
2023-01-26 23:13:28 +00:00
Radosław Kapka
77d3ccb9ad Clean up state types (#11916)
* Clean up state types

* rename package
2023-01-26 14:40:12 +00:00
Radosław Kapka
a296a0c783 Update builder code to Capella (#11904) 2023-01-26 14:07:44 +01:00
Guillaume Ballet
c6338e3a31 beacon-chain, config: get consensus values from beacon config (#11798)
* beacon-chain, config: get consensus values from beacon config

* review feedback

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-01-25 10:42:03 +00:00
terencechain
0a4b0a68d3 Clean up unused code (#11900)
* Clean up unused code

* More unused

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-01-25 10:09:14 +00:00
james-prysm
f9dd462338 PrepareBeaconProposer: bug fix (#11912)
* fixing bug and adding unit test

* reverting small change

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-01-24 22:47:26 +00:00
Potuz
943eb93a57 save genesis state to hot cache and do not look it up from disk (#11911)
* do not lookup genesis state from disk

* make sure genesis state is saved

* fix tests

Co-authored-by: kasey <489222+kasey@users.noreply.github.com>
2023-01-24 23:14:50 +01:00
Potuz
9619170df1 Return error structures from bls changes endpoint (#11908)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-01-24 15:58:43 +00:00
james-prysm
4da0abace7 validator exit test (#11910)
* adding exit test

* fixing deep source errors
2023-01-24 15:26:58 +00:00
Patrice Vignola
d919f800e3 Add REST implementation for Validator's GetDuties (#11893)
* Add REST implementation for `MultipleValidatorStatus`

* Fix PR comments

* WIP

* Add REST implementation for Validator's SubscribeCommitteeSubnets

* Remove redundant test

* Initialize dutiesProvider

* Address PR comments

* Remove duplicate import

* Fix build break

* WIP

* WIP

* WIP

* Add valid test case for getDutiesForEpoch

* Add failure tests for getDuties

* Revert test change

* Address PR comments

* Address PR comments

Co-authored-by: Emmanuel NALEPA <emmanuel.nalepa@kiln.fi>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2023-01-24 14:33:29 +00:00
james-prysm
38f095d556 prysmctl: changeblstoexecutionaddress ( withdrawals) (#11790)
* adding unit test for withdrawal

* updating to handle api changes

* Update cmd/prysmctl/withdrawal/withdrawal.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update cmd/prysmctl/withdrawal/cmd.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update cmd/prysmctl/withdrawal/withdrawal.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update cmd/prysmctl/withdrawal/cmd.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update cmd/prysmctl/withdrawal/withdrawal.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update cmd/prysmctl/withdrawal/withdrawal.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update cmd/prysmctl/withdrawal/withdrawal.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* updating based on feedback

* rewording prints

* Update cmd/prysmctl/withdrawal/withdrawal.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

* add error handling when status not ok

* addressing some of the feedback, adding new tests

* adding unit test with error return

* adjusting based on review comments

* addressing more comments and fixing unit test

* updating prompts

* migrating some functions to api/client

* removing unused function

* adding review feedback

* removing unused functions

* migrating command, old command under accounts still exist

* reverting tos.go

* fixing bazel lint

* fixing build

* resolving unused parameter caught by deepsource

* fixing unit tests

* Update cmd/prysmctl/validator/cmd.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

* Update cmd/prysmctl/validator/cmd.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

* adding in website

* addressing review comment

* adding new verify only flag

* Update cmd/prysmctl/validator/withdraw.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update cmd/prysmctl/validator/withdraw.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update cmd/prysmctl/validator/withdraw.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update cmd/prysmctl/validator/withdraw.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update api/client/beacon/client.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update api/client/beacon/client.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update api/client/beacon/client.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update api/client/beacon/client.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update cmd/prysmctl/validator/withdraw.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* addressing comments

* fixing unit test

* Update cmd/prysmctl/validator/cmd.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update cmd/prysmctl/validator/cmd.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update cmd/prysmctl/validator/cmd.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update cmd/prysmctl/validator/cmd.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* address comments

* Update cmd/prysmctl/validator/cmd.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* adjusting object for new provided format

* adding omit empty

* fixing unit test, and linting

* adjusting based on cli changes

* fixing test data and metadata property

* addressing backwards compatability

* reverting based on review comment

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-01-24 10:05:55 +00:00
Potuz
04010d45c8 Do not check headstate being Capella for BLS changes on gossip (#11902)
* Do not check headstate being Capella for BLS changes on gossip

* fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-01-24 00:11:26 +00:00
Radosław Kapka
1a048a2f2a Refactor BLS change pool (#11894)
* Refactor BLS change pool

* update mock's signature

* do not return error from `MarkIncluded`

* fix tests

* fix mock's signature

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-01-23 14:11:45 +00:00
terencechain
3df2dedbb2 Exit properly with terminal block hash (#11892) 2023-01-23 14:27:36 +01:00
Potuz
2fee906d25 Use LIFO instead of FIFO when packing BLS changes (#11896) 2023-01-20 16:30:42 +00:00
Sammy Rosso
9f44d6e452 Add flag for custom slasher database directory (#11890)
* Add slasher database directory flag

* Fix up some beacon chain flag uses

* fmt

Co-authored-by: rauljordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-01-18 21:52:53 +00:00
Patrice Vignola
55f311eb73 Add REST implementation for Validator's SubscribeCommitteeSubnets (#11804)
* WIP

* Add REST implementation for Validator's SubscribeCommitteeSubnets

* Remove redundant test

* Initialize dutiesProvider

* Remove duplicate import

* Fix build break

* Address PR comments

* Remove file committed by mistake

* Fix broken test

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-01-18 21:21:07 +00:00
Manu NALEPA
0f90bacac9 Add REST implementation for CheckDoppelGanger (#11835)
* Add REST implementation for `MultipleValidatorStatus`

* Fix PR comments

* Address PR comments

* Add REST implementation for `CheckDoppelGanger`

* Use context

* Fix comments

* Fix PR comments

* Fix PR comments

* remove blank lines

* Fix comments

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2023-01-18 16:13:45 +00:00
Radosław Kapka
30974039f3 Small adjustments to Beacon API (#11874)
* better `fillDBTestBlocks`

* error when submitting unknown block type

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2023-01-18 12:15:25 +00:00
int88
3392ecb3e1 fix TestVotes_CanFindHead (#11821)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2023-01-18 10:54:44 +00:00
Nishant Das
fa2b64f702 Remove Unused Block Setter (#11889)
* remove methods

* remove mock

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-01-18 08:56:44 +00:00
omahs
6f5e35f08a Fix: typos (#11885)
* Fix: typo

Fix: typo

* Fix: typos

Fix: typos

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2023-01-18 15:21:58 +08:00
terencechain
79d6ce45ad Add capella's marshal and unmarshal (#11879)
* Add capella's marshal and unmarshal

* skip test

* Fix TestJsonMarshalUnmarshal/execution_payload_Capella

* Fixing test

* Skip http test

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-01-17 23:44:59 +00:00
terencechain
73cd7df679 Add capella fork version for Sepolia testnet (#11888)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-01-17 19:51:01 +00:00
Nishant Das
d084d5a979 Add Disable Staking Contract Check Flag (#11886)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-01-17 19:00:30 +00:00
Sammy Rosso
db6b1c15c4 Add additional tests to bytesutil (#11877)
* Add missing tests from bytes.go and integers.go

* Fix failing test

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2023-01-17 17:54:43 +00:00
Dhruv Bodani
7c9bff489e Add REST implementation for GetSyncCommitteeContribution (#11875)
* add REST implementation for GetSyncCommitteeContribution

* fix imports

* Update validator/client/beacon-api/sync_committee.go

Co-authored-by: Radosław Kapka <radek@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-01-17 16:43:22 +00:00
Nishant Das
1fca73d761 Delete interop.Dockerfile (#11887)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-01-17 15:09:32 +00:00
Potuz
fbafbdd62c Stream blocks capella (#11883)
* capella blocks stream

* add unit tests

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-01-17 12:17:18 +00:00
Potuz
75d98cf9af Do not log an overflow computing Capella's slot start (#11884)
* Do not log an overflow computing Capella's slot start

* nishant's review

Co-authored-by: Nishant Das <nishdas93@gmail.com>

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2023-01-17 10:59:51 +00:00
Radosław Kapka
96401e734e Implement bls_to_execution_change API event (#11865)
* Implement `bls_to_execution_change` API event

(cherry picked from commit 1c8a2c580b3664e1238380138dc65c30e238cb11)
(cherry picked from commit aabcaac619)

* missing event code

(cherry picked from commit daa4fd2b72)

* Update beacon-chain/core/feed/operation/events.go

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2023-01-17 09:45:20 +00:00
terencechain
5480d607ac Change withdrawal amount unmarshal to uint64 (gwei) (#11866)
* Change withdrawal amount unmarshal to uint64 (gwei)

* Init server

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2023-01-16 21:25:16 +00:00
terencechain
38f0a81526 Clarify circuit breaker logs (#11876)
* Clarify circuit breaker logs

* Revert bad changes

* Fix test
2023-01-16 17:49:23 +00:00
Potuz
ad680d3417 Move broadcast of BLS changes to the forkwatcher (#11878)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-01-16 15:00:11 +00:00
Potuz
047cae5e8b Sign BLS_TO_EXECUTION_CHANGES with GENESIS_FORK_VERSION (#11870)
* Sign BLS_TO_EXECUTION_CHANGES with GENESIS_FORK_VERSION

* update spectests

* extra blank line

* fix consensus_specs_sha

* get rid of LightClients tests

* fix unit tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-01-16 14:29:33 +00:00
terencechain
179252faea Address historical summaries feedback (#11864)
* Address historical summaries feedback

* Fix build

* Fix test

* Move historicalSummaries

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-01-16 12:44:40 +00:00
Potuz
f93b82ee89 broadcast bls changes at the fork (#11872) 2023-01-16 11:51:03 +00:00
int88
508e1ad005 optimize and fix bug of attestations related code (#11869)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2023-01-16 02:19:46 +00:00
terencechain
1b2d917389 New prysm get block RPC (#11834) 2023-01-13 15:45:17 -08:00
kasey
eb6b811071 Remove ShardingForkVersion/Epoch from config (#11868)
* remove ShardingForkVersion/Epoch from config

* update magic value counting config params

* skip the sharding params, which we disavow

we wanted to remove these upstream and may well do some day but right
now it's too big of a mess to untangle.

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-01-13 17:11:33 +00:00
kasey
cf71dbdf97 "Fail" in logs makes FAIL in e2e hard to find (#11867)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2023-01-13 13:03:30 +01:00
kasey
9328e9af1f Capella State Detection (#11862)
* capella state version detection bug fix

* add capella to yaml "template"

* changes for capella state detection

* lint

* don't set capella fork version == altair!!!

* less brittle test for fork schedule rpc

* fix assertions that use wrong field name

* don't test capella/sharding fv against upstream

* hat tip Terence for sanity check

* Update config/params/loader_test.go

Co-authored-by: terencechain <terence@prysmaticlabs.com>

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terencechain <terence@prysmaticlabs.com>
2023-01-13 04:04:37 +00:00
terencechain
4762fd71de Add historical summaries to beacon api state (#11863)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-01-12 17:06:23 +00:00
Nishant Das
769daffb73 Fix State Fetcher (#11820)
* fix fetcher

* kasey's review

Co-authored-by: terencechain <terence@prysmaticlabs.com>
2023-01-12 10:00:12 +00:00
Radosław Kapka
16e6e0de6c Replace FutureForkStub with specific interfaces (#11861)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-01-12 00:42:35 +00:00
Nishant Das
396fc3dc7e Fix Waiting For Bandwidth Issue While Syncing (#11853)
* fix bugs

* fix up

* comment

* raul's review

* comment

* finally fix it
2023-01-12 00:07:09 +00:00
terencechain
7fa3ebfaa8 Add historical summaries to state and processing (#11842)
* Add historical summaries to state and processing

* Process historical roots test

* Passing spec tests

* Fix shas and tests

* Fix mainnet spectest sha

* Fix tests

* Update beacon-chain/core/epoch/epoch_processing.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/core/epoch/epoch_processing_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update proto/prysm/v1alpha1/beacon_state.proto

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/state/state-native/hasher.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Radek's feedback

* Getters error

* Dont return

* Fix else

* Fix tests

* Fix test

* Rm white space

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-01-11 22:20:15 +00:00
Patrice Vignola
81b9eceb50 Add Capella support for Validator's REST API ProposeBeaconBlock and GetBeaconBlock endpoints (#11848)
* Add Capella support for Validator's REST API ProposeBeaconBlock and GetBeaconBlock endpoints

* Fix

* Fix

* Add context to capella tests

* Update validator/client/beacon-api/beacon_block_proto_helpers.go

* Update validator/client/beacon-api/beacon_block_proto_helpers.go

* Update validator/client/beacon-api/beacon_block_proto_helpers_test.go

* Update validator/client/beacon-api/beacon_block_proto_helpers_test.go

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: Radosław Kapka <radek@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-01-11 16:55:09 +00:00
Radosław Kapka
505ff6ea3d Replay state to current slot in StateBySlot (#11858) 2023-01-11 14:53:06 +01:00
Nishant Das
2b5125c7bc Check If Validator Is In Sync Committee (#11860) 2023-01-11 10:19:31 +00:00
Preston Van Loon
1e3a55c6a6 Refactor bytesutil, add support for go1.20 slice to array conversions (#11838)
* Refactor bytes.go and bytes_test.go to smaller files, introduce go1.17 and go1.20 style of array copy

* rename bytes_go17.go to reflect that it works on any version 1.19 and below

* fix PadTo when len is exactly the size

* Add go1.20 style conversions

* Forgot another int method

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-01-10 16:41:01 +00:00
Dhruv Bodani
116f3ac265 Add REST implementation for validator's ProposeAttestation (#11800)
* Add REST implementation for validator's ProposeAttestation

* handle nil attestation

* update propose attestation with context

* fix lint

* add remaining nil testcases

* Update validator/client/beacon-api/propose_attestation_test.go

Co-authored-by: Radosław Kapka <radek@prysmaticlabs.com>

* fix BUILD.bazel

Co-authored-by: Radosław Kapka <radek@prysmaticlabs.com>
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2023-01-10 16:21:29 +00:00
Jacob Shufro
bbe003720c add custom headers to DomainData requests (#11767)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2023-01-10 00:20:45 +00:00
Patrice Vignola
e957edcb12 Add REST implementation for Validator's SubmitSignedContributionAndProof (#11812)
* Add REST implementation for Validator's SubscribeCommitteeSubnets

* Add context

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-01-09 22:54:52 +00:00
Dhruv Bodani
684022fa69 Add REST implementation for validator's SubmitSignedAggregateSelectionProof (#11826)
* add REST endpoint for SubmitSignedAggregateSelectionProof

* fix linter action

* update with context

* fix context import

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-01-09 21:18:42 +00:00
Dhruv Bodani
a7a64632b1 fix TestEndToEnd_MinimalConfig_ValidatorRESTApi test (#11856)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-01-09 21:45:37 +01:00
Dhruv Bodani
c89ab764e3 Add REST implementation for validator's GetSyncMessageBlockRoot (#11824)
* add REST implementation for GetSyncMessageBlockRoot

* improve sync_message_block_root_test.go to handle errors

* include context

* fix unit tests
2023-01-09 19:35:33 +00:00
Manu NALEPA
375a76d6c9 Add REST implementation for SubmitSyncMessage (#11827)
* Add REST implementation for `SubmitSyncMessage`

* Use context

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-01-09 17:05:18 +00:00
terencechain
bad7c6a94e Better reconstruct error msg while EL client is syncing (#11851)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-01-09 15:09:45 +00:00
Potuz
bdc3a6ef11 Capella coverage and json structs (#11854)
* Capella coverage and json structs

* use SetupTestConfig
2023-01-09 14:47:40 +00:00
Manu NALEPA
4c9a0bf772 Add REST implementation for SubmitValidatorRegistrations (#11816)
* Add REST implementation for `SubmitValidatorRegistrations`

* Remove unused context

* Use context

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2023-01-06 19:46:09 +00:00
kasey
1c27b21b5a Prevent pollution between scenario tests (#11850)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-01-06 07:49:42 +00:00
Potuz
c1f00923c1 Implement bls_to_execution_changes beacon endpoint (#11845)
* Initial implementation

* fix withrawal unit tests

* add custom hooks api middleware test

* unit tests for endpoint

* update screwed proto file

* double import

* Raul review

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2023-01-06 05:17:33 +00:00
Dhruv Bodani
700f5fee8c Add context to beacon API REST implementation (#11847)
* add context to beacon APIs

* add TODO to merge GET and POST methods

* fix linter action

Co-authored-by: kasey <489222+kasey@users.noreply.github.com>
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2023-01-06 03:32:13 +00:00
Potuz
66b6177ed3 Log forkchoice weights on reorgs (#11849) 2023-01-05 20:06:40 -03:00
Raul Jordan
0b0373f2fd Remove Lock from Checkpoint Cache as LRU is Already Thread-Safe (#11846)
* lru cache in prysm is already thread safe

* rem import
2023-01-05 16:00:11 +00:00
Nishant Das
b481c91453 Shift Multiclient E2E to Post-Submit (#11811)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2023-01-05 02:39:28 +00:00
Manu NALEPA
e1408deb40 Add REST implementation for MultipleValidatorStatus (#11786)
* Add REST implementation for `MultipleValidatorStatus`

* Fix PR comments

* Address PR comments

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2023-01-04 23:15:23 +00:00
Potuz
51601716dc Rename proposer_exists (#11844) 2023-01-04 14:09:40 +00:00
terencechain
0e77a89208 Better version checks for future proof (#11825)
* Better version checks for future proof

* Revert changes for blob

* Use if else

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-01-04 03:47:20 +00:00
terencechain
6715f10ffc Add canUseBuilder helper method (#11829)
* Add canUseBuilder helper method

* Fix build

* Go fmt

* Check circuit breaker first
2023-01-03 12:46:08 +08:00
terencechain
2906196786 Add test utility helper to generate valid bls-to-exec (#11836)
* Add test util helper to generate bls-to-exec

* Gazelle
2023-01-03 00:34:15 +00:00
Potuz
ff82d9ee8c Add back BLS changes from orphaned blocks (#11822)
* Add back BLS changes from orphaned blocks

* Add full test

* add missing file

* remove duplicate imports

* Update beacon-chain/blockchain/head.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terencechain <terence@prysmaticlabs.com>
2023-01-01 18:22:01 +00:00
int88
42450fb24e fix trace name of function ExecuteStateTransitionNoVerifyAnySig (#11828)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-12-31 12:14:26 +00:00
terencechain
b6060d5a49 Track head slot through head block (#11833)
* Track head slot through head block

* Update beacon-chain/blockchain/head.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-12-29 22:23:56 +00:00
terencechain
cf6617933d Fix BUILD.bazel for v1alpha1 rpcs (#11830)
* Fix build.bazel for v1alpha1 rpcs

* Revert common_deps
2022-12-28 21:38:28 +00:00
terencechain
4e28192541 Proposer RPC: helpers to get exits and slashings (#11808)
* Proposer RPC: helpers to get exits and slashings

* add test

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-12-23 05:50:14 +00:00
Nishant Das
4aa075abac Use Single Log File in E2E (#11810)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-12-23 03:43:00 +00:00
kasey
c1563e40bd attempt to fix WaitForTextInFile log corruption (#11819)
* attempt to fix WaitForTextInFile log corruption

* restore Seek, just in case!

* lint

* handle file close error in all cases with a log

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2022-12-23 01:08:32 +00:00
terencechain
dac555a57c Allow proposer duties one epoch in advance part 2 (#11818)
Co-authored-by: rkapka <rkapka@wp.pl>
2022-12-22 23:00:36 +01:00
Potuz
4d549f572c Check BLS_TO_EXECUTION_CHANGE as if they were from Capella (#11762)
In the event we receive a BLS_TO_EXECUTION_CHANGE and our head state is
before Capella, verify it's signature with the Capella fork version.

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-12-22 14:09:24 -03:00
Patrice Vignola
83f48350b2 Fix a bunch of deepsource warnings (#11814) 2022-12-22 09:20:10 +00:00
kasey
77a9dca9eb Cleaner genesis state population (#11809)
* e2e using dynamic configs; runtime bellatrix setup

* starts from phase0 but fee recipient eval fails

* lower TTD to cross before bellatrix epoch in e2e

* fix debug printf

* fix it

* dynamic ttd and more robust fee_recipient eval

* let multiplication set ttd=0 if bellatrix epoch=0

* refactoring fee recipient test after cognit errors

* lint

* appease deepsource

* deep sourcin

* gazelle

* missed a usage of this when refactoring

* refactoring premine genesis to work for all forks

previous set of functions was a copypasta mess

* gaz post-merge

* gaz

* resolve merge artifact and pr feedback

* back out some unneeded changes

* appease deepsource

* move premine-state next to similar utils

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
2022-12-22 05:38:51 +00:00
Nishant Das
9f1c6b2d18 Fix Multiclient E2E (#11803)
* fix multiclient e2e

* parallelize keystore generation

* fix config startup

* fix GetAggregateAttestation

* regression test

* kasey's review

* fix test

* gaz

* fix tests

* deep source
2022-12-22 02:48:46 +00:00
Radosław Kapka
0c997ede0f Query duties - epoch changes (#11806)
* Allow fetching next epoch attester duties

* better formatting

* fix tests

* use param
2022-12-21 16:28:16 +00:00
Radosław Kapka
2dc56878cd Implement getPoolBLSToExecutionChanges API endpoint (#11801)
* Implement `getPoolBLSToExecutionChanges` API endpoint

* fix

(cherry picked from commit d602c94b7b)

* unify with Capella

* newline
2022-12-21 13:01:04 +00:00
kasey
d916827fb9 E2E from Phase0 or Bellatrix (#11802)
* e2e using dynamic configs; runtime bellatrix setup

* starts from phase0 but fee recipient eval fails

* lower TTD to cross before bellatrix epoch in e2e

* fix debug printf

* fix it

* dynamic ttd and more robust fee_recipient eval

* let multiplication set ttd=0 if bellatrix epoch=0

* refactoring fee recipient test after cognit errors

* lint

* appease deepsource

* deep sourcin

* gazelle

* missed a usage of this when refactoring

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
2022-12-21 04:43:29 +00:00
Ye Ding
e43152102e Identify invalid signature within batch verification (#11582) (#11741)
* Identify invalid signature within batch verification (#11582)

* Fix issues found by linter

* Make deepsource happy

* Add signature description enums

* Make descriptions a mandatory field

* More readable error message of invalid signatures

* Add 'enable-verbose-sig-verification' option

* Fix format

* Move descriptions to package signing

* Add more validation and test cases

* Fix build failure

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-12-20 18:41:47 +08:00
Nishant Das
90d5f6097c lock state (#11796) 2022-12-20 15:39:35 +08:00
james-prysm
ce125b763c Improve validator set change event readability and error handling (#11797) 2022-12-19 22:12:09 +01:00
Patrice Vignola
3ff5895548 Add REST implementation for Validator's ProposeExit (#11785)
* Add REST implementation for Validator's GetBeaconBlock

* Move endpoint into constant

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-12-19 12:58:52 +00:00
Patrice Vignola
4eb4cd9756 Add REST implementation for Validator's PrepareBeaconProposer (#11784)
* Add REST implementation for Validator's PrepareBeaconProposer

* Change fee recipients

* Handle error in a better way

* Move endpoint into constant

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-12-19 12:23:54 +00:00
terencechain
81b049ea02 Remove copy for block setters (#11795) 2022-12-19 10:54:25 +00:00
Radosław Kapka
a2fc57cb03 Improve tests for ProduceBlockV2 (#11789)
* Improve tests for `ProduceBlockV2`

* Revert "Auxiliary commit to revert individual files from 31c606198bdc91bac798b7a446d04d5cce86dad7"

This reverts commit d8b264457168f8f5e03d0473288a56606e38ab39.
2022-12-16 20:54:38 +00:00
Radosław Kapka
b63f649d89 Add finalized metadata field to API block fetching (#11788) 2022-12-16 18:21:08 +01:00
Ye Ding
e627d3c6a4 Fix command to fetch latest stable version (#11782) (#11787) 2022-12-16 16:13:54 +00:00
Potuz
856102cba8 Implement bound on expected withdrawals (#11760)
* Implement bound on expected withdrawals

This PR implements https://github.com/ethereum/consensus-specs/pull/3095

* Terence's suggestion

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* Update spectests

* fix minimal preset

* Clear committee cache during header tests to ensure active validators ain't leaking

* Rm debug log

* Rm bad init

* Update consensus spec sha

* Fix bad tests and remove redundant casting

* Fix build

* MAX_VALIDATORS_PER_WITHDRAWALS_SWEEP typo

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-12-16 06:45:26 +00:00
terencechain
8ff6f7768f Sync committee head state cache for Capella (#11773)
* Handle capella

* Handle capella

* Error

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-12-16 00:42:43 +00:00
terencechain
295cc5e4f0 Engine API: new/get_payload for Capella (#11775)
* Engine API: new/get_payload for Capella

* Fix build

* Use switch for payload type check

* Fix bad tests

* Rm redundant arg

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-12-16 00:25:22 +00:00
Radosław Kapka
8a2c7e4269 Add finalized metadata field to API responses (#11783)
* in progress

* grpc working

* middleware

* fix tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-12-16 00:04:40 +00:00
Radosław Kapka
cb02a6897d Update block Beacon APIs to Capella (#11706)
* proto+ssz

* refactor GetBlindedBlockSSZ

(cherry picked from commit 97483c339f99b0d96bd81846a979383ffd2b0cda)

# Conflicts:
#	beacon-chain/rpc/eth/beacon/blocks.go
(cherry picked from commit 9e4e82d2c5)

# Conflicts:
#	beacon-chain/rpc/eth/beacon/blocks.go

* add Capella version

(cherry picked from commit 5d6fd0bbe663e5dd16df5b2e773f68982bbcd24e)
(cherry picked from commit 82f6ddb693)

* support SSZ lol

(cherry picked from commit 52bc2c8d617ac3e1254c493fa053cdce4a1ebd63)
(cherry picked from commit d7d70bc25b)

* update withdrawals proto

* refactor and test GetBlockV2

(cherry picked from commit c1d4eaa79d)

# Conflicts:
#	beacon-chain/rpc/eth/beacon/blocks.go

* refactor and test GetSSZBlockV2

(cherry picked from commit fbc4e73d31)

# Conflicts:
#	beacon-chain/rpc/eth/beacon/blocks.go

* test other functions

(cherry picked from commit 31d4a4cd11)

* move stuff to blinded_blocks.go

(cherry picked from commit 0a9e1658dd)

# Conflicts:
#	beacon-chain/rpc/eth/beacon/blocks.go

* fix migration code

* add Capella to SubmitBlock

* custom hooks

* missing structs

* fix tests

* fix tests

* review

* fix build issues

* replace ioutil with io

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-12-15 23:42:07 +00:00
Patrice Vignola
e843cafe7d Add REST implementation for Validator's GetBeaconBlock (#11772)
* WIP

* WIP

* WIP

* Remove unused parameter

* Address PR comments

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-12-15 22:34:05 +00:00
Radosław Kapka
1c5b0596c7 More user-friendly message in ErrUndefinedExecutionEngineError (#11777)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-12-15 07:46:17 +00:00
David Theodore
159228b34f SparseMerkleTrie depth fix & regression tests (#11778) 2022-12-15 06:50:12 +00:00
Radosław Kapka
1fbb3f3e51 Reconstruct full Capella block (#11732)
* Merge branch 'reconstruct-capella-block' into capella

(cherry picked from commit b0601580ef)

# Conflicts:
#	beacon-chain/rpc/eth/beacon/blocks.go
#	proto/engine/v1/json_marshal_unmarshal.go

* remove unneeded test

* rename methods

* add doc to interface

* deepsource

(cherry picked from commit 903cab75ee)

# Conflicts:
#	beacon-chain/execution/testing/mock_engine_client.go

* bzl

* fix failing tests

* single ExecutionBlockByHash function

* fix engine mock

* deepsource

* reorder checks

* single execution block type

* update tests

* update doc

* bytes test

* remove toWithdrawalJSON

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-12-14 13:42:48 +00:00
Manu NALEPA
9c21809c50 Add REST implementation for ValidatorStatus (#11757)
* Add REST implementation for `ValidatorStatus`

* Fix review comments
2022-12-14 12:58:36 +00:00
terencechain
62ffed7861 Add capella cases for RPC calls (#11759)
* Add capella cases for rpc calls

* Use if else

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-12-14 05:08:54 +00:00
Victor
bdf063819c Fixed comments of the template (#11652)
The comments of the template were not inside the braces and appeared when creating a new Issue.

Thanks

Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-12-14 02:49:19 +00:00
Radosław Kapka
9efc89f993 Correct duties calculation for old slots (#11723)
* initial code

* update proposalDependentRoot

* convert method to function

* decouple tests

Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-12-14 00:33:30 +00:00
Preston Van Loon
5ed52b1e44 Validator: Add trace span information for AttestationRecord save requests (#11742)
* Add additional span information when saving attestation records

* Remove keymanager.sign span. It's very noisy and not a helpful data point.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-12-14 00:06:32 +00:00
kasey
2e49fdb3d2 Start chain from bellatrix state (#11746)
* WIP trying to start from bellatrix state

* env var to control log path with unique paths

due to flaky test re-run behavior, logs from a failed test run are
overwritten by subsequent retries. This makes it difficult to retrieve
logs after the first failed run. It also takes some squinting through
output to find the location of the log file in the first place. This
flag enables logs to be placed in an arbitrary path. Note that bazel
sandboxing generally will force this path to be in the /tmp tree.

* WIP - grabbing changes from rm-pre-genesis branch

* combine bellatrix state w/ rm-pre-genesis branch

* WIP

* use encoding/detect for genesis state bytes

* WIP more fixes towards start from bellatrix

* remove debug wrapping

* WIP

* multiple bugfixes

* fix fork ordering bug and bellatrix genesis blocks

* send deposits, spam tx to advance, fix miner alloc

* WIP

* WIP mess

* WIP

* Print process ID information for purposes of attaching a debugger

* bugs: genesis body_root and deposit index mismatch

* fix voting period start, skip altair check

* add changes

* make it better

* rm startup FCU, rm logs

* cleanup import grouping&ordering

* restore FCU log, get rid of tmp var

* rm newline

* restore newline

* restore wrapped error

* rm newline

* removing boot node version override

this doesn't seem to matter?

* add issue number to todo comment

* rm commented code

* rm vmdebug geth flag

* unexport values only used with genesis test pkg

and add comments where missing from exported values.

* adding comments to special cases for testnets

* migrate comments from PR to actual code :)

* rm unused test param

* mark e2e spawns exempt from gosec warning

* Fix DeepSource errors in `proposer_bellatrix.go` (#11739)

* Fix DeepSource errors in

* Omit receiver name

* Address PR comments

* Remove unused variable

* Fix more DeepSource errors

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Remove `Test_IsExecutionEnabledCapella` (#11752)

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Add REST implementation for Validator's `ProposeBeaconBlock` (#11731)

* WIP

* WIP

* WIP

* Add tests

* WIP

* Add more tests

* Address DeepSource errors

* Remove unused param

* Add more tests

* Address PR comments

* Address PR comments

* Fix formatting

* Remove unused parameter

* Fix TestLittleEndianBytesToBigInt

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* fix validator client (#11755)

* fix validator client

(cherry picked from commit deb138959a)

* Use signed changes in middleware block

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

* API `finalized` metadata field - update protos (#11749)

* API `finalized` metadata field - update protos

* change nums

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* log breaks unit tests that don't do full arg setup

easiest to just remove it for now

* restore prior behavior of phase0 block for altair

* update unit tests to account for special case

* loosen condition for fork version to match config

we don't know which fork version genesis will start from, so we
shouldn't force it to be a phase0 genesis.

* skip until we can mod configs at runtime

* NewGenesisBlockForState computes state root itself

* rm noisy log

* this log would be noisy in mainnet

* fix format specifier, []byte -> string

* core.Genesis UnmarshalJson has a value receiver :)

* no longer needs to be exported

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
Co-authored-by: Patrice Vignola <vignola.patrice@gmail.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-12-14 07:13:49 +08:00
Preston Van Loon
a01d08b857 Add --stamp to release builds (#11768) 2022-12-13 21:33:51 +00:00
Preston Van Loon
fbe0672fc4 state: Improve replay state logs metadata (#11763)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-12-13 18:46:51 +00:00
Potuz
a15ec49844 Do not remove attestations twice (#11764) 2022-12-13 14:43:42 -03:00
int88
a268255ab1 fix mismatch of code and comment (#11761) 2022-12-13 15:57:52 +00:00
james-prysm
043079dafe Adding exit codes to cli commands (#11735)
* adding exit codes to cmd

* adding in fatal logs to other cli actions

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-12-13 12:13:33 +00:00
terencechain
3fcdd58872 Add block setters for consensus type (#11751)
* Add block getters

* Rm changes

* Rm changes

* Revert "Rm changes"

This reverts commit 1ae5db79da.

* Fix tests

* Set graffiti right place

* Potuz feedback

* Update consensus-types/blocks/setters.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Radek feedback

* Fix comments

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-12-13 00:03:36 +00:00
Radosław Kapka
0798092ca8 API finalized metadata field - update protos (#11749)
* API `finalized` metadata field - update protos

* change nums

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-12-12 16:59:27 +00:00
Radosław Kapka
fb981d29e8 fix validator client (#11755)
* fix validator client

(cherry picked from commit deb138959a)

* Use signed changes in middleware block

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-12-12 15:49:02 +00:00
Patrice Vignola
fa01ee5eba Add REST implementation for Validator's ProposeBeaconBlock (#11731)
* WIP

* WIP

* WIP

* Add tests

* WIP

* Add more tests

* Address DeepSource errors

* Remove unused param

* Add more tests

* Address PR comments

* Address PR comments

* Fix formatting

* Remove unused parameter

* Fix TestLittleEndianBytesToBigInt

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-12-12 10:39:51 +00:00
terencechain
09619c388f Remove Test_IsExecutionEnabledCapella (#11752)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-12-11 22:59:12 +00:00
Patrice Vignola
1531313603 Fix DeepSource errors in proposer_bellatrix.go (#11739)
* Fix DeepSource errors in

* Omit receiver name

* Address PR comments

* Remove unused variable

* Fix more DeepSource errors

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-12-10 10:27:07 +00:00
Potuz
babfc66c5b Check BLS changes when requesting from pool (#11718)
* Check BLS changes when requesting from pool

* Terence's suggestions

* Radek's suggestion

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-12-09 14:41:45 +00:00
terencechain
cb9b5e8f6e Use payload attribute type for engine-API (#11719)
* Add payload attribute type

* Gazelle

* Fix test

* Use new payload attribute type

* Fix test

* Fix test

* Update beacon-chain/execution/engine_client.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Feedbacks

* Fix suggestion

* Update argument, fix test

* Return emptyAttri instead of nil

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-12-08 21:39:45 +00:00
james-prysm
145eb5a8e4 add m1 chip info (#11736) 2022-12-08 15:20:32 -06:00
Preston Van Loon
7510f584cb beacondb: Remove cache look up and lock request from boltdb transaction for state summaries (#11745)
* Remove cache look up from bolt transaction

* remove bogus line, oops

* Remove independent cache lookup entirely and just use HasStateSummary

* Rm newline
2022-12-08 19:32:04 +00:00
Patrice Vignola
dbeb3ee886 Onboard validator's Beacon REST API usage to e2e tests (#11704)
* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* Onboard validator's Beacon REST API usage to e2e tests

* Remove unused variables

* Remove use_beacon_api tags

* Fix DeepSource errors

* Revert unneeded changes

* Revert evaluator changes

* Revert import reordering

* Address PR comments

* Remove all REST API e2e tests except minimal one

* Fix validator pointing to inexisting beacon node port

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-12-08 14:38:56 +00:00
Preston Van Loon
ca2618110f e2e: Print process IDs for debugging. (#11734)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-12-08 04:48:16 +00:00
Nishant Das
642c399b9d Add Back Cross Client Option (#11738)
* add changes

* remove env var

* fix
2022-12-07 22:20:35 -06:00
Manu NALEPA
6eee539425 Add REST implementation for Validator's WaitForActivation (Ethereum Protocol Fellowship) (#11671)
* Implement REST `WaitForActivation`

* Activation: Factorize tests

* Fix PR comments

* `missingPubKeys`: Replace map by slice (no need to have a map here)

* Fix typo

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-12-07 19:20:11 +00:00
james-prysm
bd82eb873c Fix to Post Submit E2E (#11733) 2022-12-07 18:19:44 +01:00
Nishant Das
62455b7bcb Fix Lint and Minor Bugs in E2E (#11730) 2022-12-07 05:31:53 +00:00
Preston Van Loon
3d6d0a12dd Update go to 1.19.4 (#11727)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-12-06 21:20:25 +00:00
Patrice Vignola
05148dbc8f Fix DeepSource errors in the Validator's REST API (#11726) 2022-12-06 20:31:56 +00:00
james-prysm
19af1d2bb0 E2E: beacon APIs Part 1 (#11306)
* adding compare beacon block test

* fixing bazel

* fixing evaluator import

* fixing imports

* changing package name

* fixing bazel

* adding logic to check for checking epoch

* fixing linting

* adding check for attester duties

* handle both blockv1 and blockv2

* making middleware objects public instead

* adding test for block attestations

* fixing typo

* adding blockroot test

* adding test for attestations

* fixing type value

* fixing test

* adding in node endpoints

* fixing bazel

* updating web3signer

* printing beacon blocks on request

* fixing struct

* temp log

* forgot string cast

* adding comparison function

* fixing bazel and evaulators, WIP

* fixing bazel

* changing how to minify json

* trying multiclient

* fixing port problem

* reverting evaluator and making test only for mainnet scenario testing

* removing test data

* fixing linting unused functions
git push

* changed to reflect

* adding in ssz comparison

* fixing tests

* fixing conflict

* fixing tests

* making v2 the standard

* adding better error logging

* fixing type

* adding lighthouse settings and fixing some deepsource items

* testing adding delay to evaluator

* testing without peers check

* changing target peers to try to fix lighthouse peer connections

* temp removing other tests

* fix lint issue

* adding peers connect back in

* adding in state version

* fixing bazel

* fixing path error

* testing changes to state

* fix unmarshal

* simplifying beacon api e2e execution

* fixing missed assertian checks

* improve logging and debugging issue

* trying to fix unmarshal

* still breaking more test edits

* removing fork to test unmarshal

* fixing pathing

* resolving error

* fixing beacon_api

* merging in debug api to beacon_api test

* fixing lint and temp commenting out endpoint

* adding in custom comparison function

* fixing custom evaluator

* adding test for block header data

* fixing header evaluation

* add node apis

* fixing linting,adding tests

* fixing bazel and temp removing unused functions

* fixing deepsource and linting issues

* Update testing/endtoend/evaluators/beaconapi_evaluators/beacon_api.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update testing/endtoend/evaluators/beaconapi_evaluators/beacon_api.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* addressing review

* resolving more review comments

* fixing linting

* removing ssz return value as it's large and possibly not needed

* Update testing/endtoend/evaluators/beaconapi_evaluators/beacon_api.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update testing/endtoend/evaluators/beaconapi_evaluators/beacon_api.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update testing/endtoend/evaluators/beaconapi_evaluators/beacon_api.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update testing/endtoend/evaluators/beaconapi_evaluators/beacon_api.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update testing/endtoend/evaluators/beaconapi_evaluators/beacon_api.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update testing/endtoend/evaluators/beaconapi_evaluators/beacon_api.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* fixing more review comments

* Update testing/endtoend/evaluators/beaconapi_evaluators/beacon_api.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* fixing linting and review iteems

* fixing cognit complexity issue

* fixing linting

* fix log printout

* test build kite only with crossclient

* switching out evaluator to depositedvalidatorsareactive

* removed wrong evaluator switching correct one

* removing skip based on review comments

* fixing pathing issue

* test without participation at epoch

* testing without special lighthouse logic in evaluator

* reducing expected participation when multiclient

* fixing imports

* reducing epochs to see if less flaky

* testing with other tests added back in

* reducing epochs ran further

* testing only cross client again

* testing multi run again

* test reverted scenario for tests

* testing with cross client

* removing commented out function

* testing without peers connect

* adding optimization based on suggestions

* removed the wrong peers connect

* accidently commited something I shouldn't have

* fixing lighthouse flag

* Update testing/endtoend/evaluators/beaconapi_evaluators/beacon_api.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update testing/endtoend/evaluators/beaconapi_evaluators/beacon_api.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update testing/endtoend/evaluators/beaconapi_evaluators/beacon_api.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-12-06 16:01:17 +00:00
Nishant Das
faf16f9e56 Allow Nodes Running Via VPNs To Make Successful Dials (#11599)
* fix it

* fix dialer for now

* fix build

* fix test

* fix to v0.24.0

* fix gaz

* fix build

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-12-06 14:54:45 +00:00
Manu NALEPA
0a5c65e29c Add REST implementation for Validator's ValidatorIndex (#11712)
* Add GetAttestationData

* Add tests

* Add many more tests and refactor

* Fix logic

* Address PR comments

* Address PR comments

* Add jsonRestHandler and decouple http logic from rest of the code

* Add buildURL tests

* Remove handlers_test.go

* Improve tests

* Implement `ValidatorIndex` of `beaconApiValidatorClient` using Beacon API

* Implement getStateValidators

* `validatorIndex`: Use `getStateValidators`

Co-authored-by: Patrice Vignola <vignola.patrice@gmail.com>
2022-12-06 12:27:26 +00:00
Radosław Kapka
7dc966bb3b Update state Beacon APIs to Capella (#11708)
* proto

(cherry picked from commit 24f45e021061782ab4d6c101a95368310aad67b6)

* implementation

(cherry picked from commit bbfa22c2053e8176fc004b13ba9c8d62cc3bd352)

# Conflicts:
#	beacon-chain/rpc/apimiddleware/structs.go

* fix compilation error

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-12-06 00:36:23 +00:00
Radosław Kapka
51d35f36f0 Disallow computing committee assignments for old slots (#11722) 2022-12-05 23:37:08 +01:00
Patrice Vignola
943a0556e9 Add REST implementation for Validator's DomainData (#11711)
* Add REST implementation for Validator's DomainData

* Add missing dependency

* Fix getForkVersion logic

* Remove unused helpers

* Fix deepsource error

* Fix deepsource error

* Address PR comments

* Remove outdated comment

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-12-05 10:27:41 +00:00
terencechain
f7cecf9f8a Add payload attribute type (#11710)
* Add payload attribute type

* Gazelle

* Fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-12-02 18:23:47 +00:00
Nishant Das
8be73a52b1 Add State Check For BLS Execution Change Messages (#11716) 2022-12-02 11:13:21 +00:00
Potuz
bebceb3bfa Handle BLS to execution changes included in blocks (#11713)
* Handle BLS to execution changes included in blocks

* log

* review
2022-12-02 14:04:45 +08:00
Mart1i1n
d541010bf1 Fix Typo (#11670)
* Update ffg_update_test.go

Fix some alignment typos.

* Update justification_finalization.go

Fix typo.

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-11-30 22:25:55 +00:00
terencechain
8cb07e0c2b Spec test: capella and update to v1.3.0-alpha.1 (#11683) 2022-11-30 12:08:04 -08:00
Preston Van Loon
aa2bf0c9c4 Spectests: ensure test directories are not empty (#11709)
* Add an assertion that test folders are not empty

* more assertions

* only run sync tests on bellatrix or later
2022-11-30 17:32:10 +00:00
Ye Ding
e49d8f2162 Fix a race condition during initialization (#11444) (#11698)
* Fix a race condition during initialization (#11444)

* Fix tests

* Add more test cases

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-11-29 18:24:50 +00:00
shana
c6ed4e2089 Do not omit json fields if empty in builder client (#11673)
* Do not omit json fields if empty in builder client

* fix tests

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-11-29 12:12:13 +00:00
int88
0ad902e47d fix help message of metric doublylinkedtree_node_count (#11705) 2022-11-29 11:29:48 +00:00
Preston Van Loon
4f4775f9f9 bazel: Update rules_go and remove extra repos in WORKSPACE (#11703)
* Update rule_go to latest release, remove fuzzit stuff

* Delete another thing
2022-11-28 23:00:11 +00:00
terencechain
679e6bc54a Cont FCU if get payload attribute fails (#11693)
* Cont FCU if get payload attribute fails

* Fix err position

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
2022-11-28 19:42:32 +00:00
Radosław Kapka
c7a3cf8563 GetForkChoice API endpoint (#11680)
* proto

* middleware

* change structure

* fix all issues

* test

* validity field
2022-11-28 19:17:53 +00:00
Radosław Kapka
6c3b75f908 Upgrade getBlindedBlock API endpoint to Capella (#11687)
* proto

(cherry picked from commit 7101910e0fab5a5572795115679fd6f8d8c8379b)

* GetBlindedBlock

(cherry picked from commit e5c269ddf7b0c9e04f72ed28982a82de56fcac55)

* middleware

(cherry picked from commit 1719ce5967b0f74786c596cc921f7256e6b224f3)

* refactor

* Update beacon-chain/rpc/apimiddleware/structs.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

* update error message

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-11-28 18:17:05 +00:00
Patrice Vignola
f276c5d006 Make the validator REST API's WaitForChainStart blocking (#11695) 2022-11-28 11:58:04 +01:00
Potuz
29953cb734 Refactor Sync Committee Rewards (#11696) 2022-11-27 11:06:32 -08:00
Nishant Das
a23a5052bc Add Gossip Handler For BLS To Execution Changes (#11690) 2022-11-26 11:07:05 -08:00
Potuz
f9e0d4b13a Batch capella signatures with the rest of the block (#11689) 2022-11-25 09:45:36 +08:00
Potuz
0aaee51973 Process bls changes (#11684)
* Implement ProcessBLSToExecutionChanges

* Batch process signatures

* gaz

* Change runtime behavior

* Terence's review
2022-11-24 19:36:12 +00:00
terencechain
a0c0706224 Add Capella state changes (#11688)
* Add Capella state changes

* Use params.configs
2022-11-24 14:54:55 -03:00
Potuz
a525fad0ea Add upgrade to Capella to statereplay (#11686) 2022-11-24 14:58:05 +00:00
Patrice Vignola
7ab5851c54 Add a gRPC fallback mode to the validator Beacon REST API (#11679)
* Add a gRPC fallback mode to the validator Beacon REST API

* Remove --beacon_api_grpc_fallback flag

* Add missing bazel dependency

* Reorder dependency per gazelle check

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-11-24 11:09:07 +00:00
Patrice Vignola
e231cfd59d Onboard validator's beacon REST API tests to CI (#11682)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-11-24 11:50:13 +01:00
kasey
0a41b957dc env var to control log path with unique paths (#11681)
due to flaky test re-run behavior, logs from a failed test run are
overwritten by subsequent retries. This makes it difficult to retrieve
logs after the first failed run. It also takes some squinting through
output to find the location of the log file in the first place. This
flag enables logs to be placed in an arbitrary path. Note that bazel
sandboxing generally will force this path to be in the /tmp tree.

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2022-11-23 22:56:40 +00:00
Radosław Kapka
f2399e21e1 GetLiveness API endpoint (#11617)
* proto

* initial version

* middleware + tests

* change request structure

* fix all issues

* review feedback

* simplify out-of-range check
2022-11-23 18:23:22 +00:00
kasey
395e49972e prysmctl support generating non-phase0 genesis.ssz (#11677)
* support generating non-phase0 genesis.ssz

* make default (Value) work for EnumValue + lint

* remove messy punctuation

* Ran gazelle for @kasey

* Fix deps viz

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
2022-11-23 14:22:24 +00:00
Radosław Kapka
ee099d3f03 Operations pool for BLS-to-execution-changes (#11631)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-11-22 18:34:14 +01:00
terencechain
e2ffaf983a Add minimal withdrawal size (#11658)
* Add selector with minimal withdrawal size

* @potuz's feedback, Add PayloadAttribruteV2

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-11-22 16:05:37 +00:00
Patrice Vignola
0d4b98cd0a Add REST implementation for Validator's WaitForChainStart (#11654)
* Add REST implementation for Validator's WaitForChainStart

* Add missing error mapping

* Add missing bazel dependency

* Add missing tests

* Address PR comments

* Replace EventErrorJson with DefaultErrorJson

* Add tests for WaitForChainStart

* Refactor tests

* Address PR comments

* Add gazelle:build_tag use_beacon_api comment in BUILD.bazel

* Address PR comments

* Address PR comments

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-11-22 12:12:55 +00:00
Andrew Davis
fe1281dc1a fix(beacon-chain/rpc): correct block root for block events (#11666) 2022-11-22 10:26:56 +00:00
james-prysm
9761bd0753 correctly assign arm64 arch for Apple M1 (#11675)
* fixing typo

* removing if statement, not needed anymore
2022-11-21 23:11:02 +00:00
Manu NALEPA
0bcddb3009 Update mockgen (#11615)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-11-21 17:26:27 +01:00
terencechain
ee9da3aded Fill missing payload ID should exit early if block has seen (#11669)
* Fill missing payload ID should exit early if block has seen

* Move metric inc to top

* Fix test
2022-11-21 10:34:40 -05:00
kasey
d58d3f2c57 E2E deposit testing overhaul (#11667)
* rewrite/refactor deposit testing code

keep track of sent deposits so that they can be compared in detail with
the validator set retreived from the API.

* fix bugs in evaluator and retry

* lint + deepsource appeasement

* typo s/Sprintf/Printf/

* gosec, more like nosec

* fix gosec number - 204->304

* type switch to get signed block from container

* improve comments

* centralizing constants and adding comments

* lock around Depositor to avoid future races

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2022-11-19 03:40:32 +00:00
Preston Van Loon
4b033f4cc7 Update go to 1.19.3 (#11630)
* Update go to 1.19.3

* Update other items to 1.19

* Update golangci-lint to latest release

* Run gofmt -s with go1.19

* Huge gofmt changes

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-11-18 19:12:19 +00:00
terencechain
07d0a00f88 Add capella slashing parameters (#11660)
* Add capella slashing parameters

* Add capella to epoch precompute

* Revert "Add capella to epoch precompute"

This reverts commit 61a5f3d2bd.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-11-17 17:03:53 +00:00
Nishant Das
08d63a0cd0 Remove Geth Bindings From Prysm (#11586)
* check in changes

* gaz

* preston's review

* comment

* fix up

* remove test

* gaz

* preston's review

* fix it
2022-11-17 17:16:19 +08:00
kasey
f02ad8a68b disable geth nat traversal (#11664)
Co-authored-by: kasey <kasey@users.noreply.github.com>
2022-11-16 00:14:42 +08:00
Potuz
466fd359a2 Fix expected_withdrawals (#11662) 2022-11-14 21:33:40 -05:00
terencechain
b5f8e69b6b Add capella to epoch precompute (#11661) 2022-11-14 10:05:58 -05:00
terencechain
cf466702df Capella: validator log withdrawals (#11657)
* Capella: validator log withdrawals

* Capella: validator log withdrawals
2022-11-13 02:26:49 +00:00
Potuz
de73baa4a7 Update 3068 (#11656)
* rename last_withdrawal_validator_index

* Update Capella methods to Specs #3068

* Add missing renames

* rename minimal state

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-11-12 18:38:21 +00:00
Potuz
d4a3b74bc6 capella payload changes (#11647)
* capella payload transition changes

* fix leak

* add back setters_payload from capella

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-11-12 18:19:51 +00:00
Potuz
216cdd9361 Use finalized hash if payloadID cache misses (#11653) 2022-11-12 17:58:14 +00:00
Potuz
68c3e939b8 Fix withdrawalRoot hasher (#11655) 2022-11-12 09:00:53 -08:00
terencechain
bf7e17db8f Fix get RANDAO endpoint for underflow (#11650)
* Fix get randao end point for underflow

* Fix test
2022-11-11 17:58:25 +00:00
Patrice Vignola
ead9a83d8d Add customizable endpoints for the validator's REST API (#11633)
* WIP

* Refactor to use iface.ValidatorClient instead of ethpb.BeaconNodeValidatorClient

* Add mocks for iface.ValidatorClient

* Fix mocks

* Update update-mockgen.sh

* Fix warnings

* Fix config_setting syntax

* Use custom build settings

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* Fix endpoint address and reduce timeout

* Revert most e2e changes

* Use e2e.TestParams.Ports.PrysmBeaconNodeGatewayPort

* Fix BeaconRESTApiProviderFlag port

* Revert e2e changes
2022-11-11 17:33:48 +00:00
Nishant Das
2fef03414d Fix ENR Serialization (#11648)
* fix it

* fix test
2022-11-10 12:03:55 +00:00
Potuz
7b63d5c08c Operations in Capella are as in Bellatrix (#11646)
* Operations in Capella are as in Bellatrix

* add unit test
2022-11-10 02:04:03 +00:00
Potuz
af6d5e9149 don't change again unecessarily (#11645)
* don't change again unecessarily

* remove blinded blocks from gossip
2022-11-09 23:59:10 +00:00
terencechain
d0d7021c1d Add Capella p2p changes (#11644) 2022-11-09 15:11:46 -08:00
Potuz
4db1a02763 Implement upgrade to capella (#11642)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-11-09 16:48:23 +01:00
Mart1i1n
69d7f7f6ca Update ffg_update_test.go (#11639)
Fix some alignment typos.
2022-11-09 12:27:17 +00:00
Inphi
4e342b8802 Fix prysmctl generate-genesis yaml output file (#11635)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-11-08 14:36:58 +00:00
Nishant Das
02566de74c Copy Bytes Safely When Accessing Withdrawals (#11638)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-11-08 13:40:25 +00:00
Potuz
90ecd23d41 implement process_withdrawals (#11634)
* implement process_withdrawals

* change errors to error.go

* gazelle

* James' review

* use bytes.Equal instead

* Radek's review

* Radek's review #2

* fix test

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-11-08 10:15:26 -03:00
int88
4d68211ad4 fix error message of subscriber (#11622)
Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-11-07 16:44:38 +00:00
terencechain
19f6d3bef6 Capella: Add DB changes (#11624)
* Add Capella DB changes

* Add tests
2022-11-07 15:26:27 +00:00
Potuz
37108e6ed8 Implement get_expected_withdrawals (#11618)
* Implement get_expected_withdrawals

* Fix config test and export method

* Radek's review

* start from a different index in a test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-11-07 14:11:16 +00:00
Patrice Vignola
d33af46c90 Add support for building a Beacon API validator client versus a gRPC one (#11612) 2022-11-07 11:29:27 +01:00
Radosław Kapka
53d4659654 GetRandao Beacon API endpoint (#11609)
* `GetRandao` Beacon API endpoint

* test optimistic execution

* review

* change epoch in test
2022-11-05 23:04:58 +00:00
terencechain
09e117370a Flatten blob to an array (#11614)
* Flatten blob to an array

* Rename data back to blob

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
2022-11-04 15:03:25 +00:00
Radosław Kapka
8ade8afb73 Remove SSZ tags from beacon state (#11613)
* Remove SSZ tags from beacon state

* tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-11-04 11:08:57 +00:00
Inphi
872021f10d Remove unneeded protoarray tests (#11607)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-11-04 09:39:52 +00:00
Potuz
bb09295072 Remove withdrawal Queue (#11610) 2022-11-03 16:55:44 +00:00
terencechain
e4b2b1ea7d Unlock pending block queue if insertion errors (#11600)
* Unlock pending queue if insertion errors

* @saolyn's feedback

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-11-02 18:26:27 +00:00
terencechain
ead329e610 Implement [][] to [][48] helper (#11606) 2022-11-02 18:04:40 +00:00
int88
a0c5669511 RLock() for readonly operation (#11603)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-11-02 16:22:42 +00:00
terencechain
0fd5253915 Remove unused protobuf imports (#11602)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-11-02 13:46:23 +00:00
terencechain
86e855d499 Rename blob.blob to blob.data (#11601) 2022-11-02 14:21:03 +01:00
Sammy Rosso
001ae30a59 EIP-4881: Add merkle tree interface (#11597)
* Add merkle tree interface

* Run gazelle

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-11-01 23:31:32 +00:00
terencechain
d1f0e5dd55 Run ./hack/update-go-pbs.sh (#11598) 2022-11-01 23:08:52 +00:00
terencechain
2142b13a41 Add 4844 containers to protobuf (#11596) 2022-10-31 12:11:40 -07:00
Radosław Kapka
ffac232d89 Capella beacon block (#11566)
* in progress

* done, no tests yet

* fix ToBlinded()

* Revert "Auxiliary commit to revert individual files from 2e356b6f5b15d409ac15e825c744528591c13739"

This reverts commit 081ab74e88fb7d0e3f6a81e00fe5e89483b41f90.

* tests

* fix tests

* one more fix

* and one more

* review

* fix proto_test

* another fix

* do not return error when nil object is wrapped

* allow nil payload in body.Proto()

* correctly assert error

* nil checks in body.Execution()

* simplify PR

* Revert "Auxiliary commit to revert individual files from 5736c1f22f2d2f309b9303c13d0fb6b1679c6ecb"

This reverts commit 1ff3a4c864923f5c180aa015aa087a2814498b42.

* fix slice sizes in cloner tests

* better payload tests

* review

Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-10-31 17:58:30 +00:00
int88
26b46301d2 some minor fixes (#11593)
Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-10-30 02:54:15 +00:00
Mark Ridgwell
fdf913aed9 corrected method name in comment (#11594) 2022-10-28 12:17:56 -04:00
terencechain
9435d10652 Add more buckets for block arrival latency histogram (#11589)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-10-27 15:43:41 +00:00
Radosław Kapka
8fa481cb93 Flag to exit validators without confirmation prompt (#11588) 2022-10-27 17:24:41 +02:00
terencechain
26087d7b2d Add unknown roots in error msgs (#11585)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-10-27 10:38:07 -04:00
Nishant Das
5a3a01090a Increase Block Burst Factor For E2E (#11583)
* test

* set higher block burst factor

* add back space

* remove space
2022-10-27 03:06:52 +00:00
Preston Van Loon
55690de685 bazel: Check in 5.3.0 cross compile clang toolchain (#11577)
* Check in 5.3.0 generated clang cross-toolchain

* Remove tools/cross-toolchain/configs/clang/bazel_5.0.0

* Add manual tags and run gazelle

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-10-27 02:48:23 +00:00
kasey
007c776d8a tool to search db for a key prefix (#11417)
* tool to query db by a key prefix

* cleanup

* lint and fmt

* db/kv public visibility

We've discussed before that Bazel visibility constraints don't
accomplish much in go, so I'm phasing them out in places where they get
in the way.

* derp

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-10-26 21:28:02 +00:00
Sammy Rosso
a15e0797e4 Support non english mnemonics for wallet creation (#11543)
* add option to log rejected gossip message

* add bip39 supported mnemonic languages

* Revert "add option to log rejected gossip message"

This reverts commit 9a3d4486f6.

* Add mnemonic language flag

* Update go.mod

* Simplify language mapping

* Add test for setBip39Lang

* Update go.mod

* Improve language matching

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Run gazelle + fix maligned struct

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-10-26 21:04:00 +00:00
int88
1572c530b5 some minor fixes (#11572)
* some minor fixes

* minor fix

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-10-26 20:44:24 +00:00
Preston Van Loon
652303522f bazel: remove --stamp for builds (#11578) 2022-10-26 20:13:02 +00:00
Sammy Rosso
39a7988e9e Add error if chain-config-file used concurrently with network (#10863)
* Add error if chain-config-file used with network

If the chain-config-file flag is used concurrently with the network
flag then error and exit.
Related to #10822.

* Add config file name to error message with `fmt`

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Add unit test for hasNetworkFlag

* Add list of network flags to features

* Fix imports

* Run gazelle

* retrigger checks

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-10-25 10:48:25 +00:00
james-prysm
648ab9f2c2 revert moving validator-exit command (#11575) 2022-10-25 03:52:30 +00:00
terencechain
43a0b4bb16 Retrieve proposer existence in cache with correct parameter (#11567) 2022-10-24 13:20:41 -05:00
int88
57d7207554 fix typo and log error (#11565)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-10-23 22:01:44 +00:00
Potuz
b7b5b28c5a Fix locks in Capella setters (#11569)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-10-22 23:10:45 +00:00
terencechain
968dc5d1e8 Beacon api: get duties prune payload id cache (#11568)
* Beacon api: get duties prune payload id cache

* Fix complains and bad test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-10-22 22:43:57 +00:00
terencechain
c24016f006 Add validator index with Withdrawal pb (#11563)
* Add validator index with Withdrawal pb

* Update BUILD.bazel

* Fix test

* Better tests
2022-10-22 20:54:50 +00:00
Nishant Das
661cbc45ae Vendor Leaky Bucket Implementation (#11560)
* add changes

* fix tests

* change to minute

* remove dep

* remove

* fix tests

* add test for period

* improve

* linter

* build files

* ci

* make it stricter

* fix tests

* fix

* Update beacon-chain/sync/rate_limiter.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-10-20 16:40:13 -05:00
Nishant Das
4bd4d6392d Reduce Block Burst Factor (#11546)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-10-20 14:20:47 +00:00
Radosław Kapka
5f4c26875b Remove ValidatorCapella protobuf (#11562) 2022-10-19 17:56:50 +00:00
Radosław Kapka
6aab2b2b8d /eth/v1/beacon/blinded_blocks/{block_id} API endpoint (#11538)
* proto + stub

* Add execution_optimistic to SSZ response

* implementation

* proto fix

* api middleware

* tests

* more tests

* switch from Errorf to Error

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-10-19 15:08:30 +00:00
Radosław Kapka
b7a878d011 Resolve remaining native state tasks (#11561)
* remove ToProto and ToProtoUnsafe wrappers

* TestAppendBeyondIndicesLimit

* change type of genesisValidatorsRoot

* fuzz tests

* check type assertion
2022-10-19 10:37:45 -04:00
Raul Jordan
2ae9f1be9e Prysm Web UI Release v2.0.2 (#11559)
Co-authored-by: james-prysm <james-prysm@users.noreply.github.com>
2022-10-19 03:11:22 +00:00
Radosław Kapka
98b9c9e6c9 Handle unaggregated attestation event (#11558) 2022-10-18 10:34:25 -04:00
james-prysm
df694aad71 prysmctl: validator exit (#11515)
* moving voluntary exit command to prysmctl

* fixing linting

* fixing imports

* removing unused import:

* refactoring and adding a new unit test

* rolling back some changes

* fixing parameters

* fixing tests

* adding to main prysm ctl

* refactoring how wallet function works and ux to validator exit

* adding interop support

* fixing bazel build

* fixing if statement

* fixing beaconcha.in printout

* fixing web3signer bug, missing signing slot info

* fixing deep source issues

* fixing bazel package visibility for prysmctl

* gaz

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-10-17 16:04:19 -05:00
terencechain
e8400a0773 Fix complains and bad test (#11555)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-10-17 16:20:26 +00:00
Radosław Kapka
dcba27ffbc add server name to address (#11556) 2022-10-17 10:59:17 -04:00
Radosław Kapka
cafe0bd1f8 Capella beacon state (#11141)
* fork

* types

* cloners

* getters

* remove CapellaBlind from fork

* hasher

* setters

* spec params, config tests

* generate ssz

* executionPayloadHeaderCapella

* proto state

* BeaconStateCapella SSZ

* saving state

* configfork

* BUILD files

* fix RealPosition

* fix hasher

* SetLatestExecutionPayloadHeaderCapella

* fix error message

* reduce complexity of saveStatesEfficientInternal

* add latestExecutionPayloadHeaderCapella to minimal state

* halway done interface

* remove withdrawal methods

* merge setters

* change signatures for v1 and v2

* fixing errors pt. 1

* paylod_test fixes

* fix everything

* remove unused func

* fix tests

* state_trie_test improvements

* in progress...

* hasher test

* fix configs

* simplify hashing

* Revert "fix configs"

This reverts commit bcae2825fc.

* remove capella from config test

* unify locking

* review

* hashing

* fixes

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-10-12 11:39:19 -05:00
Potuz
ce7f042974 Add Withdrawal helpers (#11552)
* Add Withdrawal helpers

* Review

Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-10-12 08:44:41 -05:00
int88
974cf51a8c fix some comment and log error (#11550)
* fix some comment and log error

* Update testing/endtoend/component_handler_test.go

* Apply suggestions from code review

Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-10-11 14:07:47 -04:00
Radosław Kapka
eb49404a75 Expose structs from API Middleware (#11547) 2022-10-07 15:45:26 +00:00
Preston Van Loon
6bea17cb54 Update libp2p to support go 1.19 (#11309)
* Update libp2p to support go 1.19

* gaz

* go mod tidy

* Only update the minimum deps

* go mod tidy

* revert .bazelrc

* Update go-libp2p to v0.22.0 and update import paths (#11440)

* Fix import paths

* Fix go-libp2p-peerstore import

* Bazel updates

* fix

* revert some comments changes

* revert some comment stuff

* fix dependency issues

* vendor problematic library

* use your brain

* remove

* tests

Co-authored-by: Marco Munizaga <marco@marcopolo.io>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-10-07 15:24:51 +08:00
Nishant Das
de8e50d8b6 Migrate Historical States In Separate Routine (#11501)
* add changes

* space

* add test

* space

Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-10-05 19:11:03 +00:00
Sammy Rosso
8049060119 Add flag to enable logging on rejected gossip message (#11524)
* add option to log rejected gossip message

* fix rejectGossipMessage return

* revert test + fix import

* revert all beaconchain/sync/validate_* files

* log object and message if flag is set

* fix failing build

* remove object field

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-10-05 15:18:08 +00:00
Krasimir Georgiev
c1446a35c5 fix panic when the rpc client is not initialized (#11528)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Your Name <you@example.com>
2022-10-05 14:02:01 +02:00
james-prysm
f5efde5ccc Keymanager API: fix fee recipient API and add persistence (#11540)
* fixing bug with fee recipient api

* fixing unit tests

* clarifying logs
2022-10-04 17:05:46 +00:00
Nishant Das
cdcb289693 Handle New Agent Version For Lodestar (#11536)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-10-04 14:39:12 +02:00
Nishant Das
5dca874530 Fix Validator Monitor Registration (#11537)
* fix it

* gaz
2022-10-04 12:44:57 +02:00
Nishant Das
885dd2e327 Revert "More efficient way of computing skip slot cache key (#11441)" (#11535)
This reverts commit 0f0d480dbc.
2022-10-04 14:13:11 +08:00
terencechain
0f0d480dbc More efficient way of computing skip slot cache key (#11441)
* More efficient way of computing skip slot cache key

* Gazelle

* Add defensive check

* Fix test setup

* Disable skip slot cache

* Fix rpc tests for dependent root
2022-10-03 13:53:17 -04:00
terencechain
1696208318 Add CLI flag to define execution engine timout (#11489)
* Add CLI flag to define execution engine timout

* Rm unused

* Fix import

* Fix lint

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-09-30 18:05:05 +00:00
terencechain
7f686a7878 Cache proposer slot index for GetProposerDuties (#11521) 2022-09-30 19:19:40 +02:00
Potuz
2817f8e8d6 update head should go even without attestations (#11503)
* update head should go even without attestations

* add unit tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-09-30 13:29:18 +00:00
Radosław Kapka
db76be2f15 Remove state code owners (#11519) 2022-09-30 13:21:06 +00:00
Potuz
ad65c841c4 Add a justified balance getter to forkchoice (#11513)
* init

* unit test

* DeepSource complains

* gaz

* shutting deepsource down

* change var name and use handler type

* Kasey's name suggestion

* fix stategen

* interface signature

Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-09-30 06:39:07 -03:00
terencechain
7b255f5c9d Don't mark builder status unhealthy if mev-boost status is non 200 (#11506)
* Log error for mev boost / relayer status, not return error

* Add ticker to poll relayer status

* Add ticker to poll relayer status

* Update beacon-chain/builder/service.go

* Rm test

* Check nil before calling status

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-09-29 20:54:24 +00:00
Sammy Rosso
157b1e373c fix validator loggin timeTillDuty as a negative number (#11512)
* fix timeTillDuty log shows a negative number

* Update validator/client/validator.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-09-29 19:48:43 +00:00
terencechain
e3df25f443 Fix attestations update head error logging (#11514) 2022-09-29 12:16:42 -07:00
kasey
805473cb38 Give forkchoice to stategen (#11439)
* add forkchoice to stategen.New, update everywhere

* conflict_1

* Fix proposer_bellatrix test

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-09-28 20:10:27 +00:00
terencechain
a54bb19c82 Set builder get payload timeout to 3s (#11413) 2022-09-28 07:55:44 -07:00
terencechain
14908f639e Remove INTERVALS_PER_SLOT from place holder for test (#11502)
* Rm INTERVALS_PER_SLOT from place holder

* Comment
2022-09-26 15:19:04 +00:00
Potuz
77fc45304b Remove protoarray forkchoice (#11455)
* Remove protoarray forkchoice

* exported errors

* fix spectests

* fix tests

* conflict 1

* Preston's review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-09-26 14:45:21 +00:00
terencechain
722c7fb034 Update consensus layer badge to v1.2.0 (#11492) 2022-09-26 00:27:54 +00:00
Potuz
97ef8dde4d Fix sync tests and update to 1.2.0 (#11498)
* Fix sync tests and update to 1.2.0

* fix repo sha
2022-09-25 23:48:14 +00:00
terencechain
211c5c2c5c Beacon api: produce block should skip mev-boost (#11488)
* Beacon api: produce block should skip mev-boost

* Update comments

* Additional test and comments

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer_bellatrix.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Fix suggestion

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-09-23 18:45:26 +00:00
Yier
2ea66a8467 Remove unwanted wrapper of GRPC status error (#11486)
Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-09-23 14:37:30 +00:00
terencechain
7720d98764 Beacon api: propoerly submit blind block (#11483)
* Beacon api: propoerly submit blind block

* Gazelle

* Fix SubmitBlockSSZ

* Update beacon-chain/rpc/eth/beacon/blocks.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/eth/beacon/blocks.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-09-22 20:13:43 +00:00
james-prysm
20e99fd1f9 Improvement to Fee Recipient UX (#11307)
* updating mock

* adding new internal api

* adding generated code

* converting validator index to pubkey

* adding new API endpoint

* fixing mock related new API

* fixing unit tests

* preventing failure on startup, replacing with warnings

* improving log message

* changing warn log to error log

* fixing error formatting

* improve log on beacon node side on startup

* fixing deepsource issue

* improving logs

* fixing unit tests

* fixing more tests

* adding error check

* adding in new case for fee recipient to avoid conflicts on changing beacon node suggested fee recipient

* adding default fee recipient check for gas limit as well

* adding improved unit tests

* accidently checked in tmp folder

* adding more unit tests

* fixing gas limit unit test

* Update validator/rpc/standard_api_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/rpc/standard_api_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/rpc/standard_api_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/node/config.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/node/config.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update proto/prysm/v1alpha1/validator.proto

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/rpc/standard_api.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/client/runner.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* addressing comments

* updating proto generated files

* fixing linting and addressign review comments

* fixing unit test

* improve error handling

* accidently added tmp folder

* improving function error returns

* realizing i am wrapping error incorrectly

* fixing unit test

* addressing review comment

* fixing linting

* fixing unit test

* improving ux around enable builder

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-09-22 11:35:35 -05:00
Nishant Das
61f6b27548 Change Default Timeout To 30 Seconds (#11487) 2022-09-22 10:43:53 +00:00
terencechain
e1c1d0f864 Don't return out of routine when update head errors out (#11470) 2022-09-20 17:31:59 +00:00
Potuz
7a8d780869 do not return on error during on_tick (#11463) 2022-09-19 08:10:11 -03:00
Potuz
588674f2fd Dont return if proposer boost roots are missing (#11459)
* Dont return if proposer boost roots are missing

* move spectests to doublylinked tree

* unit test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-09-17 12:46:11 +00:00
Radosław Kapka
73443208a1 Remove proto state (#11445)
* Remove native state flag and use native state in spectests

* remove feature from tests

* use e2e config in slasher simulator

* use params.BeaconConfig in testutil

* use correct function

* use minimal config in go_test

* fix TestListValidators

* parameterize sync committee bits and aggregation bits

* Fix TestServer_ListIndexedAttestations_GenesisEpoch

(cherry picked from commit 254ab623dde08ae8886b152facdbbd8889ed79db)

* fix more tests

* fix even more

* moreeee

* aaaand more

* one more fix

* one more

* simplify TestGetAltairDuties_UnknownPubkey

* comment out problematic test

* one more fix

* one more

* aaaand one more

* another

* use fieldparams in HydrateBlindedBeaconBlockBodyBellatrix

* create new package for mainnet tests

* TestServer_GetBellatrixBeaconBlock

* change slashed validator index

* clear cache in reward_test.go

* deprecate flag

* create bazel mainnet target

* move attester mainnet test to mainnet target

* "fix" proposer tests

* use minimal config in TestServer_circuitBreakBuilder

* fix TestProposer_ProposeBlock_OK

* more fixes in validator package

* more fixes

* more fixes

* test code

* move TestProposer_GetBeaconBlock_BellatrixEpoch to minimal

* finally

* remove proposer_bellatrix_mainnet_test.go

* fix TestServer_GetBellatrixBeaconBlock_HappyCase

* fix TestServer_GetBellatrixBeaconBlock_BuilderCase

* Preston needs to fix this!

* Revert "Preston needs to fix this!"

This reverts commit b03d97a16e.

* remove proto state tests

* fix migration tests

* static analysis fix

* review

* remove proto state

* swap state in tests

* fix BUILD file in /proto/testing

* remove metrics test with nil state
2022-09-16 18:17:46 -04:00
Potuz
9c2c665e92 Remove optimistic sync candidate check (#11453)
* Remove optimistic sync candidate check

Since we know we have merged and the chain has advanced 128 slots,
there's no possible forkchoice Poissoning attack anymore, removing the
check and allowing any block to be imported optimistically.

* fix test

* fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-09-16 16:05:30 +00:00
Radosław Kapka
0f0ab1327e Deprecate native state flag (#11268) 2022-09-15 20:47:51 +02:00
Potuz
a27feb4cb2 Implement optimistic sync spectests (#11391)
* builder changes

* New valid status

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-09-15 05:42:20 +00:00
Radosław Kapka
41e95cb274 Release headLock on error (#11412)
Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-09-14 11:44:38 +02:00
terencechain
3b8a3c2276 Remove unused WithTimeout (#11420) 2022-09-13 17:18:30 +02:00
terencechain
5a1d260b9a Update block arrival histogram values (#11424)
* Update block arrival histogram values

* Add 250 and 750

* Add back 1500 and 2000

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-09-12 16:58:15 +00:00
terencechain
53627e9709 Metrics: capture reorg distances and depths (#11435)
* Capture reorg distance and depths

* Use historgrams instead of gauges

* Fix build

* Edit help texts
2022-09-12 15:06:52 +00:00
terencechain
9fefb33cdf Forkchoice: track highest received root (#11434) 2022-09-12 16:29:01 +02:00
terencechain
d860daff75 clean up: godoc comments, redundant castings and more (#11428)
* clean up: Godoc comments, redundant castings and more

* Fix assertion check

* Update beacon-chain/forkchoice/doubly-linked-tree/forkchoice.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/forkchoice/protoarray/store.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-09-12 14:03:20 +00:00
Potuz
6cf4f3c260 Report voted fractions for a given root (#11421)
* Report voted fractions for a given root

* unused parameters

* comment update

* change test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-09-11 19:04:40 +00:00
terencechain
99fbf5d3d8 Cancel context propoerly for validator runner (#11429)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-09-09 18:28:20 +00:00
Preston Van Loon
d57a44b973 Update bazel to 5.3.0 (LTS) (#11427)
* update bazel version to 5.3.0

* Remove duplicated flags for test (inherits from build)
2022-09-09 16:43:15 +00:00
Nishant Das
fbe591c363 Fix Gossipsub Parameter (#11425) 2022-09-09 15:36:43 +00:00
Potuz
fc509cc220 Prune during on_tick (#11387)
* Prune during on_tick

* add unit test

* fix tests

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-09-08 14:41:10 +00:00
terencechain
78cbe4dfe1 Fall back to uncached getPaylod if first time outs (#11404)
* Fall back to uncached getPaylod if first time outs

* Skip only deadline error. Unit test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-09-08 14:20:32 +00:00
Potuz
436fcb8682 flip defensive pull flag (#11419) 2022-09-08 13:42:29 +00:00
Radosław Kapka
b4d2395a38 Tests for builder service (#11214)
* Tests for builder service

* fix glob

* minor changes to mock

* add comments to mock

* Revert "Auxiliary commit to revert individual files from bc62a140347e3cbd8bd82e99cf5e9deb98b74df0"

This reverts commit c4c1016cb597b9340d1c81ab3816e037a6b97f9e.

* correct comment

* Do not init builder for empty endpoint

* revert  merging issues

* better nil checks

* test fix

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-09-06 18:29:44 +00:00
Justin Traglia
0cee01ad55 Throw error if marshaling invalid Uint256 (#11347)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-09-06 20:07:08 +02:00
terencechain
14a6fe3f01 Validate downloaded state slot matches header slot (#11396)
* Ensure downloaded state slot matches header slot

* Regression test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-09-06 16:21:59 +00:00
Radosław Kapka
4d90afe944 Fixed size arrays in block interfaces and structs (#11375)
* Fixed size arrays in block fields

* test fix

* fmt

* fix fetcher test

* fix fuzz tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-09-06 14:30:16 +00:00
terencechain
b98e9019ce Rename invalid count to invalid children count (#11411) 2022-09-06 14:04:32 +00:00
terencechain
45a637a3b0 Move provided time is later from warning to debug (#11398)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-09-06 13:27:32 +00:00
terencechain
17f3d66885 Log ready for merge every min (#11410)
* Log ready for merge every min

* Update beacon-chain/execution/check_transition_config.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-09-06 13:07:48 +00:00
Potuz
29390516b0 fix forkchoice endpoint (#11403)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-09-05 17:57:08 +00:00
Potuz
135c30e912 remove children slice when pruning (#11402)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-09-05 16:44:03 +00:00
Potuz
50f9d2bab8 Better log fee recipient mismatch (#11395)
* Better log fee recipient mismatch

Logs if the returned payload from the engine does not have the fee
recipient set in the validator.

Also warn the user if he's proposing a block with the burner fee
recipient address.

* fix tests
2022-09-05 14:13:51 +00:00
Potuz
8743e6a688 Harden witholding fix (#11397)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-09-05 02:26:10 +00:00
Potuz
08e6274c14 add forkchoice node timestamp to json response (#11394) 2022-09-04 19:52:39 +00:00
kasey
cca9ea6989 move forkchoice init back to node (#11344)
* move forkchoice init back to node

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-09-02 13:56:50 -05:00
Mike Neuder
cbc2153664 Wallet Create CLI manager integration (#11331)
* Wallet recover CLI Manager migration

* bazel run //:gazelle -- fix

* Wallet create CLI Manager migration

* Wallet recover CLI Manager migration (#11278)

* Wallet recover CLI Manager migration

* bazel run //:gazelle -- fix

* fix lint and build errors

* add TODO to remove duplicate code

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Wallet recover CLI Manager migration

* bazel run //:gazelle -- fix

* Wallet recover CLI Manager migration

* bazel run //:gazelle -- fix

* Wallet recover CLI Manager migration

* bazel run //:gazelle -- fix

* fix lint and build errors

* Wallet recover CLI Manager migration (#11278)

* Wallet recover CLI Manager migration

* bazel run //:gazelle -- fix

* fix lint and build errors

* add TODO to remove duplicate code

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* bazel run //:gazelle -- fix

* rename to ConstructCLIManagerOpts

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-09-02 14:56:47 +00:00
terencechain
8627fe72e8 Remove activation/exit queue metrics (#11389)
* Remove activation/exit queue metrics

* Gaz

* Rm unused vars

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-09-01 19:11:25 +00:00
Raul Jordan
65bf3d0fa8 Fix Div By 0 in Small Helper (#11390) 2022-09-01 18:26:28 +00:00
Raul Jordan
a5da9aedd4 Add in P2P Metrics for Mainnet (#11386)
* connected peers gauge vec

* build

* add in gossip metric

* clean
2022-09-01 18:00:54 +00:00
Potuz
e1ab034d25 Defensive pulls protoarray (#11385)
* Defensive pull tips protoarray

* unit test

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-09-01 15:05:44 +00:00
Potuz
84bc8f3d64 Fix fillInMissingBlocks (#11353)
* Fix fillInMissingBlocks

Only check that the chain's parent is in forkchoice, rather than it
being the finalized checkpoint. Forkchoice anyway guarantees that the
chain will be a descendant of the finalized checkpoint.

* ensure root is not zero

* fix tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-09-01 14:40:32 +00:00
Radosław Kapka
c4deb84012 Simplify BeaconBlockIsNil() (#11373)
* Simplify `BeaconBlockIsNil()`

* remove unused code

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-09-01 03:40:20 +00:00
kasey
488e19e428 less ominous --weak-subjectivity-checkpoint warning (#11362)
* fix #11361

* change log level to debug

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-09-01 01:56:56 +00:00
Raul Jordan
bcaae1c440 Performance Metrics for Prysm (#11377)
* atts performance and blocks

* idiomatic observe

* all attestation related errors

* block metrics

* db metrics

* metrics func

* rem old metrics

* naming

* rem metric

* rem unneeded

* fix

* fix up

* rev

* fix

* rem
2022-09-01 01:26:19 +00:00
terencechain
587ba83aca Better batch block processing warning (#11372)
* Better batch block warning

* Fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-09-01 00:57:55 +00:00
terencechain
091f16b26c Don't hard shutdown if mev-boost / relay is not available (#11380)
* Don't hard shtudown if mev-boost / relay is not available

* Add else
2022-08-31 23:58:27 +00:00
Potuz
fb9626fdd7 Feature flag to disregard deposit contract (#11370)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-31 22:35:59 +00:00
terencechain
c638e114db Add new metrics (#11374)
* Better batch block warning

* New metrics

* Revert "Better batch block warning"

This reverts commit e21fcfcebe.

* More metrics

* Add activation and exit queues

* Gaz
2022-08-31 18:05:50 -04:00
terencechain
b1e08307ed Fix time to duty to round slot number (#11371)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-31 19:54:44 +00:00
kasey
cac5d0f234 giving commands more clear names per issue #11287 (#11360)
* giving commands more clear names per issue #11287

* mark the top-level help text for cpt deprecated

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2022-08-31 13:18:35 -05:00
james-prysm
52d48b328f Improve Validator Index RPC Error Handling (#11363)
* adding in nil check for head

* adding changes based on feedback

* Update beacon-chain/rpc/prysm/v1alpha1/validator/server_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-08-31 16:42:49 +00:00
Nishant Das
9729b2ec77 Remove The Header Time Check (#11329)
* remove the check

* remove function and tests

* dead code

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-31 14:54:29 +00:00
terencechain
7aa3776aa6 Log tx count only on payload (#11368) 2022-08-31 14:38:44 +02:00
Potuz
760c71ef77 Only update finalized checkpoint in DB if it's newer (#11356)
* Only update finalized checkpoint in DB if it's newer

Do not save to DB a finalized checkpoint that it's older than the
current one.

* Add a test

* Add more strict condition check

* Revert params.SetupTestConfigCleanupWithLock(t)

* Remove new line

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-08-31 00:16:22 +00:00
james-prysm
6c209db3ca fixing json unmarshalling (#11357)
* fixing json unmarshalling

* adding unit test for no conent

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-30 19:31:23 +00:00
Raul Jordan
0725905797 Informative Errors on Execution Client Connection Issues (#11359)
* add err auth help

* error working

* add err auth fix
2022-08-30 19:09:42 +00:00
terencechain
166f8a1eb6 Log corerct header value (#11354)
* Log corerct header value

* gaz

* Go fmt

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-30 16:23:21 +00:00
Radosław Kapka
85896e994e Explain the purpose of deprecatedBeaconFlags (#11355) 2022-08-30 15:47:36 +00:00
Nishant Das
4a00b295ed Pin Fuzzbuzz to Go 1.18 (#11350) 2022-08-30 10:18:23 +02:00
Potuz
d2b39e9697 Defensive pull tips, doubly-linked-tree (#11175)
* Defensive pull tips, doubly-linked-tree

* feature flag

* gaz

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-30 00:48:25 +00:00
Shem Leong
97dc86e742 Support passing of headers to all Engine API calls (#11330)
* Support passing of headers to all Engine API calls

* Update execution headers example

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-08-29 23:34:29 +00:00
terencechain
cff3b99918 Fix can propose blind block (#11346) 2022-08-29 13:30:28 -07:00
terencechain
be9847f23c Remove unused code (#11345) 2022-08-29 18:03:03 +00:00
Håvard Anda Estensen
4796827d22 Replace deprecated linter deadcode with unused (#11334)
* Replace deprecated linter deadcode with unused

* Ignore unused warnings

* Print filename and line number when linting fails

* Fix path

* Remove unused methods

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-08-29 12:45:25 -04:00
Preston Van Loon
57b7e0b572 db: Wrap errors in db.fetchAncestor to better identify unmarshalling issues (#11342)
* db: Wrap errors in db.fetchAncestor to better identify unmarshalling issues. See #11327

* Wrap genesis state fetch, just in case

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-29 16:08:03 +00:00
terencechain
b5039e9bd9 Better chain start log (#11332)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-29 15:48:23 +00:00
james-prysm
f5d792299f e2e: updating web3signer version (#11339)
* updating version

* reverting change to lighthouse sha
2022-08-29 15:29:40 +00:00
Potuz
9ce922304f Track timestamp in forkchoice (#11333) 2022-08-29 14:49:02 +00:00
Nishant Das
3cbb4aace4 Fix IPC Paths For Windows (#11324)
* return early for windows

* mick's review
2022-08-26 23:05:28 +00:00
terencechain
c94095b609 Accept everything when node is optimistic (#11320) 2022-08-26 21:41:59 +00:00
kasey
ae858bbd0a removing dead code to appease linter (#11326)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2022-08-26 16:06:44 +00:00
Radosław Kapka
30cd158ae5 Move forkchoice dump to eth namespace (#11325)
* protos

* server code

* rename v2 to v1 in endpoint

* middleware

* test fix

* test fix

* oops

* remove duplicated import

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-26 14:54:32 +00:00
Nishant Das
2db22adfe0 Handle Execution Client Failures Better (#11321)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-08-26 14:30:13 +00:00
Nishant Das
161a14d256 Update Lighthouse to v3 in our E2E Runner (#11323)
* update to v3

* fix sha

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-08-26 13:24:28 +00:00
Håvard Anda Estensen
9dee22f7ab Pre-allocate slices (#11317) 2022-08-26 13:49:50 +02:00
Potuz
52271cf0ba Report depth and distance on reorgs (#11315)
* Report depth and distance on reorgs

* rename to CommonAncestor

* change event feed

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-25 23:59:08 +00:00
Potuz
e1f56d403c Restore forkchoice dump endpoint. (#11312)
* Restore forkchoice dump endpoint.

Only working on doubly-linked-tree.

* unit test

* revert proto changes

* protoarray

* Deepsource

* shut up!

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-25 23:37:23 +00:00
terencechain
a2193ee014 Accept attestations when node is optimistic (#11319)
* Accept attestations when node is optimistic

* Fix tests

* Add regression tests

* Fix tests

* Fix more bad tests
2022-08-25 20:15:07 -03:00
james-prysm
762b3df491 Beacon API: api wrongly marked deprecated (#11316)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-08-25 21:02:17 +00:00
terencechain
2b3025828f ErrorContains dont allow empty string (#11314)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-25 19:07:39 +00:00
terencechain
436792fe38 Builder: filter header with 0 bid and empty tx root (#11313)
* Filter header with 0 bid and empty root

* Check nil

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-25 18:24:02 +00:00
terencechain
1d07bffe11 Beacon api: fix get blind block (#11304)
* Beacon api: fix get blind block

* Gaz

* Add back before bellatrix behavior

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-08-25 17:19:17 +00:00
Preston Van Loon
f086535c8a Update llvm to 13.0.1 (#11310)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-25 15:15:14 +00:00
Han Shen
3a4c599a96 Implement delete gaslimit (#11290)
* Implement delete gaslimit.

* Minor comment change.

* Reset gaslimit to DefaultConfig's gaslimt instead of 0.

* After gaslimit deletion, use global gaslimit default value instead of values provided in ProposalConfig.

* After deletion, use config default, if that is not available, use global default gaslimit value.

* Use grpc's codes.NotFound instead of http code "404".

* Updated bazel deps (new imports "google.golang.org/grpc/codes" was added for tests).

* Fix "TestServer_RecoverWallet_Derived" test failure.

Previously "params.BeaconConfig()" (thus the default global value
"BLSSecretKeyLength") was overriden by standard_api_test:TestServer_DeleteGasLimit.
Fixed the problem by retoring the origin global default after the test is done.

* Do not change BeaconConfig object, instead change BeaconConfig.DefaultBuilderGasLimit.

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-08-25 14:43:21 +00:00
Nishant Das
1c6cbc574e Update Geth Version In Prysm (#11308)
* clean up

* clean up
2022-08-25 13:55:01 +00:00
Potuz
2317375983 Add feature flag to treat all blocks as optimistic at startup (#11303)
* Add feature flag to treat all blocks as optimistic at startup

* Terence's review

* remove changed empty lines

* Apply suggestions from code review

* Go fmt sorry

* bad comments

Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-08-25 12:40:29 +00:00
terencechain
6354748b12 Update badges (#11305)
* Update badges

* Update README.md
2022-08-24 22:46:56 +00:00
Nishant Das
e910471784 Add In Duty Logging (#11301)
* add it in

* use time until

* potuz's review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-24 19:50:19 +00:00
Potuz
ab7e97ba63 Fix setNodeAndParentValidated (#11302)
* Fix setNodeAndParentValidated

* fix tests
2022-08-24 19:30:45 +00:00
Mike Neuder
e99de7726d Wallet recover CLI Manager migration (#11278)
* Wallet recover CLI Manager migration

* bazel run //:gazelle -- fix

* fix lint and build errors

* add TODO to remove duplicate code

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-08-24 16:57:03 +00:00
Justin Traglia
606fdd2299 Return copy of deposits instead of internal pointer (#11273)
* Return copy of deposits instead of internal pointer

* Update the comment

* Fix linter warning

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-08-24 15:46:51 +00:00
james-prysm
1eb6025aaa Beacon API: validator registration encoding bug (#11299) 2022-08-24 15:05:43 +00:00
Nishant Das
d431ceee25 Improve Logging When Parsing JWT Secret (#11300)
* remove all references

* remove warning
2022-08-24 13:16:48 +00:00
james-prysm
4597599196 Code Cleanup: remove forkchoicer from beacon node (#11294)
* removing forkchoicestore on beacon node

* fixing linting

* Update beacon-chain/node/node.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

* fixing if statement

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-23 17:47:12 +00:00
james-prysm
0c32eb5c03 Beacon API: skip updating fee recipient if it's the same (#11296)
* adding in redudant check

* adding unit tests

* fixing linting

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-23 17:26:43 +00:00
terencechain
4b1cb6fa80 Fork aware beacon API end points (#11274)
* Make operation RPC fork aware

* Gaz

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-08-23 17:07:11 +00:00
Nishant Das
9cfb823cc6 Simplify List Attestations RPC Method (#11292)
* simplify

* fix tests

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-08-23 12:47:16 -04:00
terencechain
cb502ceb8c Skip updating fee recipient if it's the same (#11295) 2022-08-23 10:54:38 -05:00
Roberto Bayardo
8da4d572d9 fix wrapping of nil errors (#11282)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-08-22 16:57:43 +00:00
terencechain
1c6fa65f7b Add back deprecated flags (#11284)
* Add back deprecated flags

* Add enable-validator-registration as alias

* Clean up

* Add deprecatedEnableLargerGossipHistory

* Rm duplicated gossip batch aggregation
2022-08-22 16:05:15 +00:00
Nishant Das
eaa2566e90 Add Back Fallback Provider Flag (#11281)
* add it back

* remove all references

Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-08-22 11:20:21 -04:00
Nishant Das
6957f0637f Bring Down Error To A Debug Log (#11283) 2022-08-22 12:00:50 +00:00
Nishant Das
01b1f15bdf Add Back Resync Routine (#11280) 2022-08-21 13:31:40 +00:00
Nishant Das
b787fd877a Handle Deprecated Flags Correctly (#11276) 2022-08-20 04:16:14 +00:00
Nishant Das
2c89ce810d Bring back old execution flag as an alias (#11275) 2022-08-20 03:28:22 +00:00
Potuz
e687fff922 Insert attestations on block (#11260)
* Insert attestations from blocks to forkchoice

* do not insert twice

* use OnAttestation instead

* Revert "use OnAttestation instead"

This reverts commit 577c37bbeb.

* unit test

* fix test

* Update beacon-chain/blockchain/receive_block_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-08-19 17:02:17 +00:00
james-prysm
5e2498be7e Gas_Limit as string (#11264)
* setting gas limit as string in json and yaml

* adding more tests

* fixing unit test

* fixing string conversion
2022-08-19 11:36:49 -05:00
Potuz
76f958710f Protoarray deadlocks (#11271)
Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-19 14:46:12 +00:00
Potuz
1775cf89c6 Call on_tick on every skipped slot for spectests (#11262)
New spectests ensure that we call on_tick for every slot between the
last tick and the current one. This PR fixes that

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-19 14:25:35 +00:00
Potuz
8fecfaee48 DoublyLinkedTree double locks (#11269)
* DoublyLinkedTree double locks

* fix updateCheckpoints

* add comment

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-19 14:06:57 +00:00
terencechain
f089405d2f Update spec tests to v1.2.0-rc.3 (#11261)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-19 13:46:41 +00:00
Nishant Das
029c81a2e4 Switch Down Libp2p Logging Level By Default (#11266)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-19 13:09:29 +00:00
Nishant Das
56c48b4971 Tag Gosec To Last Stable Version (#11267) 2022-08-19 13:00:30 +00:00
Raul Jordan
20ed47a107 Add a Generate Genesis State Command to Prysmctl (#11259)
* genesis tool

* done with tool

* delete old tool

* no fatal

* fix up

* Update cmd/prysmctl/testnet/generate_genesis.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* radek feedback

* more feedback

* required

* fix up

* gaz

* radek feedback

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-08-19 09:26:25 +00:00
terencechain
e30471f1a0 Remove inclusion distance & slots (#11265) 2022-08-18 16:31:22 -07:00
terencechain
3b38765a2d Better re-org log (#11253)
* Better re-org log

* Dont' return err

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-18 16:53:17 +00:00
Raul Jordan
b60e508c89 Remove Slasher Deprecated Protos (#11257)
* rem deprecated

* fix slasher
2022-08-18 15:56:05 +00:00
Justin Traglia
a65c670f5e Use function argument in error message instead of return value (#11244)
* Do not use return value in error handling

* Revert changes to EpochFromString & SlotFromString

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-08-18 14:01:24 +00:00
Han Shen
4af7d8230a Implement "set gaslimit" for KeyManager API (#11208)
* This PR is for issue #11155 "Keymanager APIs: gas limit api"'s task "set gas limit".

* This PR is for issue #11155 "Keymanager APIs: gas limit api"'s task "set gas limit".

* Fixed comment string.

* Regenerated key_management proto files.

* Addressed code review comment - explitly set BuildConfig.Enabled to false.

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-08-18 04:49:54 +00:00
Han Shen
27733969f7 Fixed flag usage text typo. (#11254) 2022-08-17 23:50:26 +00:00
Raul Jordan
e70fe1c9fd Support Chain Config File Loading in Genesis State Tool (#11249)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-17 19:44:29 +00:00
Justin Traglia
9b3a834437 Parse slot as uint64 in spectest (#11252)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-08-17 19:34:43 +00:00
terencechain
d815fa8f21 Remove deprecated eth endpoints (#11251)
* Remove deprecated eth endpoints

* Add back blockroot

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-17 19:20:34 +00:00
Justin Traglia
ac3079f8cd Add lock around unsafe append in goroutine (#11247)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-08-17 19:01:01 +00:00
Raul Jordan
cb8f6423e0 Do Not Push Prysmctl Docker Images to Dockerhub (#11250)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-17 18:15:31 +00:00
terencechain
515e7c959f Remove deprecated RPC ListBlocks (#11243)
* Remove deprecated RPC ListBlocks

* Fix test

* Rm mock

* Go imports

* Rm unused

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-17 18:07:28 +00:00
Raul Jordan
82bbfce524 Use GolangCI-Lint for Goimports Checking Instead of Third-Party Image (#11246) 2022-08-17 17:15:08 +00:00
terencechain
95430ddb57 Remove deprecated SubmitValidatorRegistration (#11242)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-17 16:35:41 +00:00
james-prysm
21b7861d37 Keymanager API: enum lowercase (#11194) 2022-08-17 16:11:04 +00:00
Raul Jordan
c1e7afa201 Prysm Web UI Release v2.0.1 (#11240)
Co-authored-by: james-prysm <james-prysm@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-17 15:38:18 +00:00
terencechain
dfa400d4a1 Add mainnet merge epoch and ttd (#11207)
* Add tentative mainnet merge epoch and ttd

* Update minimal

* Update commit

* Update WORKSPACE

* Update spec tests

* Add eip4844 place holders

* Skip lightclient types

* Skip lightclient types

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-17 15:18:43 +00:00
terencechain
b04c28b30c Remove disable-attesting-history-db-cache (#11239) 2022-08-17 14:23:13 +00:00
Taranpreet26311
ed07359573 Enable Gofmt Linting via Golang-CI Lint to Allow for Generic Code in Prysm (#11205)
* converting to generic

* enable gofmt linting

* enable gofmt

Co-authored-by: James He <james@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-17 13:53:46 +00:00
Nishant Das
25d87dd27b Remove Disable DiscoveryV5 Flag (#11237)
* remove flag

* go fmt

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-17 12:45:12 +00:00
Nishant Das
a9ccabf6c9 Remove Bolt Mmap Flag (#11236)
* remove mmap flag

* fix mmap

* fix all build

* gaz

* config

* fix
2022-08-17 12:22:41 +00:00
james-prysm
2377d6d6ea Register validator beacon api (#11225)
* adding in beacon API request Object

* fixed proto generation

* fixing protos

* adding new API logic and fixing linting

* adding work in progress unit test for validator registrations

* fixing unit test

* fixing linting

* fixing function name to match interface

* changing emptypb to empty.Empty

* fixing import complaint

* removing unused import

* fixing import

* Update beacon-chain/rpc/eth/validator/validator.go

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* adding emptypb.Empty back in

* Update beacon-chain/rpc/eth/validator/validator.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/eth/validator/validator.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/eth/validator/validator.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* adding validator registration API to apimiddleware

* fixing import

* fixing import format

* fixing protos

* fixing goimports

* removing duplicate import

* fixing proto

* fixing error message in test

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-08-17 11:41:51 +00:00
Raul Jordan
100ca0ebaf Prysmctl Command to Request Beacon Nodes for Block Ranges Over P2P (#11035)
* first

* attempt p2p connect send tool

* attempt

* stream registration

* trying to register

* attempt

* workinnnn

* begin

* p2p prysmctl tool

* ignore

* fix

* delete deprecated

* p2p smaller iface surface area

* further p2p refactor

* gaz

* better logging

* process

* all functionality

* fix up

* rhandle

* v2 req

* cmd

* send sub

* v1 handle

* show head slot

* cmd

* cmd lib

* gazelle fix

* bazel

* gaz

* work on the handshake items

* prevent dial to self

* add config awareness

* gaz

* inferring host addrs from p2p

* initialize data mappings

* add own mock

* fix up logic

* gaz

* add img

* gaz

* add images

* builds

* builds

* nishant feedback:

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-08-17 06:38:57 +00:00
Nishant Das
49ef0ad284 Remove Gossip History Flag (#11232)
* Remove gossip history flag

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-17 04:49:51 +00:00
Nishant Das
e9ee6e2c9d Remove Backup Webhook In the Beacon Node (#11235)
* remove it

* gaz

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-17 04:29:39 +00:00
Han Shen
a3cc26dd2a Replace deprecated jwt.StandardClaims with jwt.RegisteredClaims. (#11230)
* Replace deprecated StandardClaims with RegisteredClaims.

* Replace deprecated StandardClaims with RegisteredClaims.

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-08-17 04:10:11 +00:00
Nishant Das
5ad36486b0 Remove Skip BLS Verification Feature (#11233)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-17 02:59:13 +00:00
Nishant Das
faa0643b16 Remove Toggle (#11234) 2022-08-17 02:40:12 +00:00
Nishant Das
868377318c Remove Checkpoint Info Flag (#11231) 2022-08-17 02:12:12 +00:00
Radosław Kapka
1c180e5088 Protobuf cleanup (#11096)
* remove v2 from all messages that are not request/responses

* rename StreamBlocksAltair to StreamBlocks

* fix tests

* fix validator module

(cherry picked from commit 7f2263e90e80499a851c2e09605e2ef5dccb0df8)

# Conflicts:
#	testing/mock/beacon_validator_client_mock.go
#	testing/mock/beacon_validator_server_mock.go

* fix mocks

* update validator.pb.go

* Revert "rename StreamBlocksAltair to StreamBlocks"

This reverts commit 9c961c4e64.

# Conflicts:
#	proto/prysm/v1alpha1/validator.pb.go
#	testing/mock/beacon_validator_client_mock.go
#	testing/mock/beacon_validator_server_mock.go

* Revert "Auxiliary commit to revert individual files from a872c9d59dee869da5a9c7236c7ac34fcaf8d54c"

This reverts commit 89f19e4f15006c4a0efe593229abc433491e578e.

* Revert "Auxiliary commit to revert individual files from 9acbf7b0160626dae0d39c58fa0d8a3e48c203e0"

This reverts commit 398ecc9cef460c27e7fc92efe4df3fc9d3dbe566.

* update mocks

* fix build errors

* patch up

* fix

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-17 01:35:27 +00:00
Potuz
b4fb8304fe Revert --enable-vectorized-htr (#11229)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-08-16 20:24:08 +00:00
Raul Jordan
1062f5fe4d Fix Proto Generation Scripts and Regen Protos With V3 Bindings (#11227)
* proto gen fix

* regen protos and ssz bindings

* regen

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-16 19:52:24 +00:00
Radosław Kapka
01614e7bfb Add execution_optimistic to getBlockV2 (#11226)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-16 19:31:59 +00:00
terencechain
8d3f474bfa Misc builder fixes (#11228)
* Fix a few bugs

* Fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-16 19:10:40 +00:00
Radosław Kapka
ccfc09151f Add expected waiting time to pending validator log (#11213)
* Add expected waiting time to pending validator log

* fix TestValidator_HandleKeyReload

* test fix

* fix imports

* test fixes

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-16 18:05:56 +00:00
Potuz
85ad61ea83 Check for SIGILL before using gohashtree (#11224)
* Check for SIGILL before using gohashtree

* gohashtree dep

* check error

* move to config startup

* Kasey's advice

* review #1

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-16 17:43:59 +00:00
terencechain
2728b83f6a Remove Execution Client Fallback and Make Providing an Execution Client Required (#10921)
* Starting

* Done

* Fix more tests

* Fix test

* Fix test

* fix up

* building

* requirement

* gaz

* builds

* rem deadcode

* fix

* fix up

* removed checked method

* Update service_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-08-16 17:22:34 +00:00
james-prysm
e0eee87bf4 Improve beacon chain coverage Part 1 (#11080)
* first node test

* adding in configuration flags for code coverage

* adding line to remove file on unit test

* adding new test for compressed field trie but is currently broken

* changing limit on trie

* adding new trie length coverage

* adding in test for empty copy of trie

* adding more trie tests

* adding new field trie

* adding more field trie tests

* adding clarity to chunking equation

* fixing linting

* clarifying function for limit

* updating native state settings to improve ease of future unit tests

* improving unit test

* fixing unit tests

* adding more tests and fixing linting

* adding more coverage and removing unused file

* increasing node coverage

* adding new test for checking config for booleans

* fixing db test

* fixing linting

* adding signing root test

* fixing linting

* removing accidently created beacondata

* switching not non native state

* reverting back to proto use for spec test

* reverting back to proto for some tests

* turning off native state on some tests

* switching more to proto state

* rolling back disablenativestate

* switching to native state in the state-native package for tests

* fixing linting

* fixing deepsource complaint

* fixing some tests to native state and removing some unused flag checks

* convert to native state

* fixing linting

* issues are being triggered by deleting the db this way so reverting change in hopes of changing this

* rolling back testing util

* rolling back some tests from native state

* rolling back db deletion

* test switching native state off after test runs

* fixing hasher test

* fixing altair and bellatrix hashers for native state

* Update beacon-chain/node/node_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/rpc/auth_token_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* fixing imports

* adding altair proof test

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-08-16 16:19:01 +00:00
Potuz
5c84192606 Revert --enable-forkchoice-doubly-linked-tree (#11212)
* Revert --enable-forkchoice-doubly-linked-tree

* fix tests

* fix parallell test

* fix noviablehead tests

* Terence's suggestion
2022-08-16 10:39:57 -03:00
Raul Jordan
d077483577 Add V3 Suffix to All Prysm Packages (#11083)
* v3 import renamings

* tidy

* fmt

* rev

* Update beacon-chain/core/epoch/precompute/reward_penalty_test.go

* Update beacon-chain/core/helpers/validators_test.go

* Update beacon-chain/db/alias.go

* Update beacon-chain/db/alias.go

* Update beacon-chain/db/alias.go

* Update beacon-chain/db/iface/BUILD.bazel

* Update beacon-chain/db/kv/kv.go

* Update beacon-chain/db/kv/state.go

* Update beacon-chain/rpc/prysm/v1alpha1/validator/attester_test.go

* Update beacon-chain/rpc/prysm/v1alpha1/validator/attester_test.go

* Update beacon-chain/sync/initial-sync/service.go

* fix deps

* fix bad replacements

* fix bad replacements

* change back

* gohashtree version

* fix deps

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-08-16 12:20:13 +00:00
Potuz
155d60dd60 Remove forkchoice debug endpoint (#11105)
* Remove forkchoice debug endpoint

* change protos

Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-08-15 21:04:39 +00:00
james-prysm
63b7376b75 Prysm V3: update signing type (#11099)
* updating proto name

* updating generated files

* updating names on block v2 and v3 signing types

* fixing import format

* fixing unit tests

* fixing unit test

* fixing more conflicts

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-15 18:22:14 +00:00
terencechain
8de5512908 Builder: check payload and header HTR root match (#11223)
* Check roots compliance

* Update BUILD.bazel

* Fix test
2022-08-15 18:03:38 +00:00
Nishant Das
65b5c430d1 Fixes API To Only Return Canonical Blocks (#11221)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-15 15:48:44 +00:00
terencechain
afe9fa71f0 Add circuit breaker for relayer/builder (#11215)
* Add circuit breaker for relayer

* Update mainnet_config.go

* Renamings

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-15 15:28:34 +00:00
terencechain
12b3a0048b Add warning about block delay when outsourcing block cosntruction (#11222)
* Add warning about block delay when outsourcing block constr

* Update beacon-chain/builder/service.go

* Update beacon-chain/builder/service.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-08-15 15:04:11 +00:00
terencechain
aa73250373 Health check: optimistic status (#11204)
* Health check: optimistic status

* Tests

* Update beacon-chain/blockchain/service.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/rpc/service.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/rpc/service.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/blockchain/service.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Use correct import

* Update BUILD.bazel

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-15 14:45:50 +00:00
Nishant Das
9565ee12d7 Batch Reconstruction Of Bellatrix Blocks (#11210)
* batch reconstruction

* add test

* clean up

* terence's comment

* terence's other comment

* try out in e2e

* fix

* fix some more

* fix blinded blocks

* fix it

* remove dev flag

Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-08-15 10:16:20 -04:00
sragss
7d7e6fcd62 deprecate eth http api endpoints; update protos (#10946)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-08-14 23:29:56 +00:00
Nishant Das
e04a8bcfbd Remove Deprecated Feature Flags (#11125)
* remove deprecated feature flags

* remove unused flags

* remove other deprecated flags

* remove native state flag

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-14 12:47:53 +00:00
Potuz
186fbc5f99 release checkpoints lock from store.head (#11217) 2022-08-13 07:19:52 -07:00
Potuz
c79ec31c17 Release checkpoints lock on on_tick (#11216)
There is a double lock race condition when NewSlot acquires the
    checkpoints lock first and the nodes lock later, while calls to Head()
    acquire the nodeslock first and the checkpoints lock later.

    This PR releases the checkpoints lock in NewSlot, to reaquire it later
    in updateUnrealizedCheckpoints after getting the nodes lock
2022-08-13 09:30:08 -03:00
Potuz
c64cf05879 update gohashtree (#11206)
* update gohashtree

* go mod
2022-08-12 12:25:43 +00:00
james-prysm
f7a298ae8f Prysm V3: delete old unused APIs (#11086)
* initial commit

* removing old unusued functions and tests

* updating proto generated files

* fixing unit tests

* fixing linting

* fixing file format

* fixing formatting

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-12 09:48:38 +00:00
Siyuan Han
f3997647ac Fix typos (#11209)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-08-12 11:27:24 +02:00
terencechain
9c6efe4abd Don't insert attestations to forkchoice store (#11201) 2022-08-11 19:35:37 -07:00
Leo Lara
9aaa42156b Add cross compilation for OSX ARM64 (Apple Silicon) (#10981)
* Upgrade Grail Bazel toolchain to latest release v0.7.2

This should allow Prysm to compile on Ubuntu 21.10 and 22.04

* Changes necessary in .bazelrc for new version of com_grail_bazel_toolchain

* Add cross compilation for OSX ARM64 (Apple Silicon)

* Update cross-toolchain README to include osx_arm64

* Add double quotes to shell code

* Rename install_clang_cross10.sh -> install_clang_cross.sh

* Update with prysmatic labs uploaded assets

* Touch README to trigger CI

* Revert "Touch README to trigger CI"

This reverts commit 826ac8a6d8.

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
2022-08-11 21:40:24 +00:00
terencechain
d72e42bbb8 Rename web3 provider flag to execution endpoint (#11133)
* Rename web3 provider flag to execution endpoint

* Fix test

* Add aliase and update default port

* Rm alias
2022-08-11 20:39:55 +00:00
kasey
6b42a0a3a1 decreasing stategen interface surface area (#11132)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-11 20:12:05 +00:00
james-prysm
f8ff36f534 Builder API Error Log Simplification (#11199)
* adding in unit tests and updated error messaging

* returning errors correctly

* Update api/client/builder/client.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* updating based on comments

* fixing unit tests

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-11 19:49:16 +00:00
terencechain
bd9776f510 Remove disable sync flag (#11120)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-11 19:27:53 +00:00
Radosław Kapka
252086a0b2 Issue validator warning when beacon node does not use builder (#11203)
* Issue validator warning when beacon node does not use builder

* typo

* improve log and fix test

* gazelle
2022-08-11 15:05:08 -04:00
terencechain
6dc38ba3a9 Make payload ID key reorg resistant (#11186)
* Validate cached payload before propose

* Return payload

* Fork resistent key

* Fix tests

* Add comments

* Gaz

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-11 14:55:26 +00:00
terencechain
93ebba6dca Remove duty countdown flag (#11121) 2022-08-11 14:33:30 +00:00
terencechain
d08aa6b23f Remove deprecated block RPC (#11106)
* Remove deprecated block RPC

* Sync and fix test

* Update BUILD.bazel

* Fix e2e

* Fix e2e
2022-08-11 14:07:52 +00:00
terencechain
b0f1ea0451 Remove head sync capability (#10787) 2022-08-11 01:05:08 +00:00
Radosław Kapka
0f19beb105 Change proposer cache overwrite logic (#11191)
* Change proposer cache overwrite logic

* improve comment

* Update beacon-chain/cache/payload_id.go

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* review

* do not hardcode

Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-08-10 22:49:35 +00:00
Radosław Kapka
5c580d4a1c Replace version.BellatrixBlind with b.isBlinded (#11159)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-08-11 00:00:10 +02:00
terencechain
d35006b0a3 Validate builder header (#11195)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-08-10 18:31:37 +00:00
Radosław Kapka
f96b47bcc7 Typo fix (#11197) 2022-08-10 12:48:12 -04:00
terencechain
1117118435 Add highest slot and blocks received last epoch tracking (#11146)
* Add highest slot and blocks received last epoch tracking

* Comments

* Potuz feedback

* Fix tesT

* Potuz feedback, use clock time

* Raul's feedback

* Raul's feedback

* Use field params

* Fix minimal value

* Fix constant

* Fix constant

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-10 15:58:52 +00:00
Radosław Kapka
c2e4ecdfb5 Rename statefetcher methods (#11196) 2022-08-10 13:24:56 +00:00
terencechain
9b4d22c48c Remove unused ctxs (#11192)
* Remove unused ctxs

* Check ctx errors
2022-08-09 16:43:29 +00:00
Nishant Das
7d5eac7458 Fix Fuzzbuzz Builds (#11190) 2022-08-09 11:09:53 +00:00
Preston Van Loon
ed1116e578 Logging: strip new lines and other control characters (#11185)
* Logging: strip new lines and other control characters

* Use strings.Map and only sanitize the field value

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-08 18:12:12 +00:00
Han Shen
760d2fcfbc This PR is for issue #11155 "Keymanager APIs: gas limit api" (#11156)
* This PR is for issue #11155 "Keymanager APIs: get_limit api".

This PR implements the first task "get gas limit for the validator key".

* Removed useless import alias.

* Added regenerated key_management.pb.go/pb.gw.go files as well as endpoint_factory.go changes.

* Address James's comments:

1. api path component should be "gas_limit" instead of "gaslimit".
2. ran curl test like:
   - go build -o tmp/validator
   - ./tmp/validator --web --validators-external-signer-url=http://localhost:9000/
   - curl  -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.e30.C-BuVfWhKpr9elB05GTJdEEx_8AzkImxzXL03IqcHR8" 127.0.0.1:7500/eth/v1/validator/0xb4844195ce8ca78d9d4f7ffdf4001cfdb079e059542bcc4f45ddfff2bcec7defb1482db4f9426f92f59972da395dd2b5/gas_limit
   - {"data":{"pubkey":"0xb4844195ce8ca78d9d4f7ffdf4001cfdb079e059542bcc4f45ddfff2bcec7defb1482db4f9426f92f59972da395dd2b5","gas_limit":"30000000"}}

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-08-08 17:52:18 +00:00
Raul Jordan
c1f2c36704 Respond With Empty Payloads Pre-Merge (#11184)
* regression test

* build

* nil check

* fix test

* gazelle

* lint
2022-08-08 13:04:09 -04:00
james-prysm
bd3dfb27f3 BugFix: cli stringSlice bug (#11166)
* fixing stringslice issue and adding one more filter

* fixing linting and improving function based on feedback

* rolling back generic function to check ci

* adding function back in

* testing printing error

* reverting change

* Added gofmt detail flag

* temporary change

* Another flag added

* Revert changes

* Modifed entrypoint

* Another change

* Revert

* Change go image

* Added -e flag

* Switched to alpine image

* Switched go to strech

* Fix typo

* Yet another image added

* Added go alpine

* rolling back generic

* reverting version

* Update go.sum

reverting

* Update go.mod

reverting

* Update Dockerfile

rolling back

* Update entrypoint.sh

rolling back

* Update go.yml

reverting changes

* fixing flag for config

* reverting changes and moving change to another PR

* Update validator/node/node.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* updating based on review

* Update container/slice/slice.go

Co-authored-by: Taranpreet26311 <taran@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-08-08 16:04:34 +00:00
terencechain
4e225fc667 Use correct fork and genesis root to verify bid (#11176) 2022-08-05 18:55:55 +00:00
Nishant Das
9261da7fb1 Update E2E (#11140)
* save progress

* tidy

* Update go.yml

* make CI happy

* gosec

* revert back

* Update go.yml

* change go version

* remove fixed test case

* fix ci

* fix updates

* fix up

* fix race tests

* fix bad mock

* lock it

* fix it

* fix e2e builds

* use gotags

* Revert "use gotags"

This reverts commit 808863f427.

* Revert "fix e2e builds"

This reverts commit eb351e7d31.

* Revert "fix it"

This reverts commit 9e99dac94f.

* Revert "lock it"

This reverts commit 1a3c60ad41.

* different approach

* better

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-08-05 13:01:47 +00:00
Vie
ea93a68818 improve log for public key not found (#11174)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-08-05 11:39:47 +00:00
Preston Van Loon
9d375969d1 Enforce log.WithError(err) static analysis and fix all violations (#11163)
* Use log.WithError static analysis from #11143 and fix all violations

* Fix another log violation after pulling from develop

* Update beacon-chain/sync/pending_blocks_queue.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

* @potuz feedback

* Copy paste fail

* fix tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-08-05 10:52:02 +00:00
David Theodore
1323912625 Node by root mutex fix (#11172) 2022-08-04 19:40:25 -07:00
terencechain
c05c18787e Improve terminal difficulty has not been reached yet logging (#11171)
* Improve terminal difficulty has not been reached yet logging

* Update beacon-chain/execution/check_transition_config.go

Co-authored-by: mick <103775631+symbolpunk@users.noreply.github.com>

Co-authored-by: mick <103775631+symbolpunk@users.noreply.github.com>
2022-08-04 16:51:42 +00:00
james-prysm
0ed739ac5f Validator Client: auth-token path log (#11170) 2022-08-04 15:59:36 +00:00
Nishant Das
82ef9f1e48 Bring Back Mplex (#11169)
* bring back

* fix mod
2022-08-04 15:04:51 +00:00
Nishant Das
e169ce9107 Fix Goodbye Codes In Prysm (#11168)
* fix goodbye codes

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-04 14:11:45 +00:00
Potuz
cef95dd5b2 Revert --experimental-disable-boundary-checks (#11160)
* Revert --experimental-disable-boundary-checks

* add deprecation

* gaz

* spectests

* spectests fix

* fix previous justified checkpoint

* copy previous jcp

* Revert "copy previous jcp"

This reverts commit 6e2b865c23.

* update previous jcp on on_tick

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-04 13:26:41 +00:00
terencechain
a82325615b Can recover from missing state summary in DB (#11167)
* Can recover from missing state summary in DB

* Tests

* fix tests

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-08-04 13:01:07 +00:00
Mike Neuder
8aa6057b60 Wallet edit CLI Manager migration (#11136)
* Wallet edit CLI Manager migration

* TODO for code deduplication

* s/walletEdit/remoteWalletEdit/g

* s/walletEdit/remoteWalletEdit/g

* remove unused struct field

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-08-03 19:44:37 +00:00
terencechain
d4ab08050f Clean up: fix typos (#11165)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-03 18:47:30 +00:00
Preston Van Loon
0c3df40b1f Remove unused proto import (#11164)
* Remove unused proto import

* Update generated go files
2022-08-03 18:26:20 +00:00
Potuz
2349899658 Wrap missing state for checkpoint error (#11162)
* Wrap missing state for checkpoint error

* terence's review
2022-08-03 16:49:11 +00:00
Potuz
19e4cd30cf Forkchoice: expose if no tip is viable (#11153)
* Forkchoice: expose if no tip is viable

This PR changes the behavior on when the node is considered optimistic.
A call to `blockchain.IsOptimistic()` relies solely on forkchoice, if
all tips are invalid, then it's optimistic. If the current headroot is
not in forkchoice then it's optimistic.

A call to `blockchain.IsOptimisticForRoot()` will return true if the
requested root is headroot and it's not found in forkchoice

* update comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-03 13:59:03 +00:00
Radosław Kapka
7f4c694bb3 Native Blocks Ep. 3 - Remove old code (#11158)
* Native Blocks Ep. 3 - Remove old code

* rename back package

* fix errors
2022-08-03 13:33:05 +00:00
terencechain
19bc691cef Update lighthouse prater bootnodes (#11152) 2022-08-02 22:04:00 +00:00
Potuz
a5cab4e9c8 Add unit test for reboot behavior (#11151)
* Add unit test for reboot behavior

* fix conflicts
2022-08-02 17:23:27 +00:00
Radosław Kapka
879e310332 Native Blocks Ep. 2 - Switch usages to new package (#10885)
* panic in SizeSSZ

* moving slowly

* adapt old code to new interfaces

* return interfaces from factory functions

* replace the rest of WrappedSignedBeaconBlock

* WrappedBeaconBlock

* WrappedBeaconBlockBody

* miscellaneous

* Test_BeaconBlockIsNil

* replace usages of BeaconBlockIsNil

* replace usages of mutator

* fix all build errors

* fix some more issues

* mutator changes

* relax assertions when initializing

* revert changes in object_mapping.go

* allow calling Proto on nil

* Revert "allow calling Proto on nil"

This reverts commit ecc84e4553.

* modify Copy and Proto methods

* remove unused var

* fix block batch tests

* correct BUILD file

* Error when initializing nil objects

* one more error fix

* add missing comma

* rename alias to blocktest

* add logging

* error when SignedBeaconBlock is nil

* fix last test

* import fix

* broken

* working

* test fixes

* reduce complexity of processPendingBlocks

* simplified
2022-08-02 15:30:46 +00:00
Potuz
4b46dead2f only viable head is invalid (#11117)
* failing onBlock syncing

* passing merge check

* failing signature verification

* still failing block signature

* mock full bellatrix blocks

* working unit test

* return error from FCU if head fails to update

* move bellatrix block generator

* remove bellatrix signature function

* Add liveness unit tests

* revert removal of sync_aggregate.go

* gaz

* Terence's suggestion

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* go fmt

* Nishant's suggestion

Co-authored-by: Nishant Das <nishdas93@gmail.com>

* Fix build

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-08-02 11:55:05 -03:00
james-prysm
c1f89cc4c8 fixing log parsing of public validator key (#11142)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-01 17:08:09 +00:00
Nishant Das
881b5c8adb Make Batch Gossip Aggregation The Default (#11144)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-01 16:46:24 +00:00
Preston Van Loon
d17826fb9d Static Analyzer: require log.WithError instead of log.Errorf("bad: %v", err) (#11143)
* Add analysis template with failing test

* Works for the most common use case

* Progress on tool and more test cases

* Improvements

* handle nil

* works for the most part

* Fix some TODOs in the tool

* Fix some TODOs in the tool
2022-08-01 16:08:01 +00:00
terencechain
699bfdfdb4 Rename pow to execution (#11135)
* Rename pow to execution

* Fix complain

* Fix complain
2022-08-01 14:43:47 +00:00
terencechain
758d28678c Don't create payload with same timestamp as terminal block (#11129) 2022-07-31 17:17:55 +00:00
terencechain
616cfd3390 Add flag for validator gas limit catch-all (#11134)
* Add flag for validator gas limit catch-all

* Update flags.go
2022-07-29 17:20:56 -05:00
terencechain
5005fbfe71 Validate builder signature for Bid (#11124)
* Validate builder signature for Bid

* Few renamings

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-29 05:16:18 +00:00
terencechain
015cf5ed92 Update teku sepolia bootnode (#11131)
* Update teku sepolia bootnode

* Update teku sepolia bootnode
2022-07-29 12:56:16 +08:00
Potuz
801d5eabe5 Deal with nil LVH (#11127)
* Deal with nil LVH

After https://github.com/ethereum/execution-apis/pull/254 the EL may
return  `nil` for LVH. Our current implementation would treat this as a
zero hash and thus mark all blocks since the merge block as INVALID

* Apply suggestions from Terence

Co-authored-by: terencechain <terence@prysmaticlabs.com>

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-28 16:46:50 +00:00
Radosław Kapka
eb066da5c2 Capella changes to protobufs (#11128) 2022-07-28 16:19:36 +00:00
terencechain
44ae300b04 Use the correct withdrawal (#11123)
* Use the correct withdrawal

* Move withdrawal to engine.proto

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-28 13:56:30 +00:00
Nishant Das
588021f75b Penalize Bad Payloads Properly (#11126)
* penalize peers better

* fix it

* fix test
2022-07-28 21:30:47 +08:00
Preston Van Loon
78c55019e6 Do not print stack traces with log.WithError(err)... (#11116)
* Add a copy of github.com/x-cray/logrus-prefixed-formatter with fixes for our static analysis

* gazelle and add failing test

* fix it

* fix link

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-27 17:20:54 +00:00
Nishant Das
1601972625 Make Peer Scorer The Default (#11115)
* make it the default

* deprecate this

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-27 17:01:13 +00:00
Radosław Kapka
e5ab259ee1 Capella changes to protobufs (#11119)
* manual proto changes

* generated files

* missed comment
2022-07-27 18:39:15 +02:00
Raul Jordan
9e74c3d641 Better Log In Case Contract Code Not Found At Address (#11118) 2022-07-27 15:22:16 +00:00
terencechain
59dcea81c2 Clean up push proposer setting method (#11091)
* Clean up push proposer setting method

* Update validator_test.go

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-26 16:57:23 +00:00
terencechain
9149dc2aae Run ./hack/update-go-pbs.sh (#11107) 2022-07-26 16:45:16 +00:00
Nishant Das
a7c9c76b18 Fix Failures With Prysm Starting Up (#11103) 2022-07-26 13:54:49 +00:00
Leo Lara
5a4edf897f Initialise slasher service using New in slasher test to avoid panics (#11046)
* Initialise slasher service using New in slasher test to avoid panics

* Handle unhanlded error

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-07-26 12:06:05 +00:00
Nishant Das
2d6b157eea Disable Fuzz Targets (#11060)
* add changes

* fix fuzzer again

* is fixed

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-07-26 16:19:30 +08:00
Denys Yaroshenko
32745b5484 Do not send empty block header to slasher in validation (#11071)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-07-25 21:03:48 +00:00
terencechain
bfdaf2ec5a Add feature flags to testnet default (#11098) 2022-07-25 19:22:10 +00:00
Raul Jordan
39c343bcab Enable Only Saving Blinded Beacon Blocks for Prater (#11097) 2022-07-25 17:44:30 +00:00
Potuz
de1ecf2d60 non-canonical IsOptimistic check (#11088)
When requesting with IsOptimistic for a root which is non-canonical and
historic, we should check if it is canonical before returning false.

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-07-25 17:05:32 +00:00
Potuz
7aee67af90 Withdrawals: containers and process BLS to ETH1 changes (#11090)
* add proto for withdrawal containers

* process BLSToExecutionChange

* unit tests

* go fmt

* gaz

* unused error

* Fix import error

* Radek's review

* failed test

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-25 14:10:18 +00:00
terencechain
9830ce43d6 Set invalid roots for bad block (#10982)
* Set invalid roots for bad block

* Update for fcu

* Update for fcu

* Set bad blocks in subscriber

* Update process_block_test.go

* Rename
2022-07-25 13:45:03 +00:00
Raul Jordan
63a8690140 Use More Granular Metrics for P2P Blocks By Range Latency Histogram (#11085)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-22 18:08:21 +00:00
james-prysm
7978a0269b initial commit (#11084) 2022-07-22 13:10:19 -04:00
nixorokish
021df67fdc [README] Add GitPOAP Badge to Display Number of Minted GitPOAPs for Contributors (#11081)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-07-22 15:21:27 +00:00
terencechain
f20e6351f5 Add prater bellatrix fork epoch and ttd (#11072)
* Add prater bellatrix fork epoch and ttd

* Update workspace with tar and fix tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-22 14:16:35 +00:00
Potuz
a2e834a683 Fix optimistic sync status during init sync (#11074)
Co-authored-by: rkapka <rkapka@wp.pl>
2022-07-21 17:59:51 +02:00
Preston Van Loon
da25624b1d Validator gRPC: Add deprecated endpoint to prevent breaking change from v2.1.3 (#11078)
* Add a deprecated gRPC endpoint to prevent breaking changes from prior release

* Fix validator build, tests still fail

* Fix validator tests

* Fix validator tests

* Fix validator tests

* Update mocks and uncomment the endtoend fee recipient check. Also
updates proto generated files.

* Remove unrelated changes

* Remove unrelated changes

* Stop yelling

* Finish renaming
2022-07-20 11:08:13 -05:00
Radosław Kapka
1a5dd879c5 Change Unable to cache headers... log level to warning (#10956)
* Change `Unable to cache headers...` log level to warning

* better solution

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-07-20 13:56:25 +00:00
james-prysm
65a9ede2d3 Validator-Registration: E2E current release fix and better error handling (#11075)
* initial commit

* removing generated auth-token

* addressing feedback

* removing mev from message

* removing unneeded limitation
2022-07-20 09:15:20 -04:00
Potuz
4b083c2ca9 Fix IsExecutionEnabled for blinded blocks (#11076)
* Fix IsExecutionEnabled for blinded blocks

The TransactionsRoot of a blinded block may be non-zero, anyway we do
not insert the header in the state until IsExecutionEnabled is true.
This last check was failing on blinded blocks because it was actively
checking that the root should be zero, when it needn't be.

Superseeds #11068

* fix test

* test case

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-19 20:45:45 +00:00
terencechain
82ae4caca8 Add Goerli as an alias to Prater (#11065)
* Add Goerli as an alias to Parter

* Revert "Add Goerli as an alias to Parter"

This reverts commit 47311ce8b0.

* Preston's feedback
2022-07-19 20:00:16 +00:00
james-prysm
e770f0fe1f Web3signer: type cleanup (#11062)
* switching signing root type to hexutil.byte

* more signing root changes

* more conversions for types

* adding bitfield parsing

* fixing linting

* fixing bitfield parsing
2022-07-19 14:05:49 +00:00
james-prysm
30b8fba2ac Validator Registration- proposer settings renaming (#11057)
* fixing ux on propertynaming, introducing placeholder property

* reverting some refactors

* Update debug.go

* Update debug.go

* rolling back change on file

* adding new unit tests and renaming flags

* renaming variable
2022-07-19 08:38:33 -05:00
Radosław Kapka
5d94fd48ca Revert "Testutil refactor attestations (#10952)" (#11073) 2022-07-19 14:41:15 +02:00
Potuz
176d763091 Allow pcli to pretty print blinded blocks (#11067) 2022-07-18 21:52:58 -03:00
terencechain
97dc9ebbc8 Reset proposer root if the root is getting removed in store (#11053)
* Reset proposer root if the root is getting removed in store

* Potuz feedback

* Fix root

* unit test

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-07-18 15:09:01 +00:00
terencechain
6a74dcf35b Misc improvements to interface with builder (#11059)
* Misc improvements to builder

* James feedback
2022-07-17 23:43:16 -07:00
james-prysm
a775798d89 Web3signer: Support json content type with optional metadata (#11058)
* initial commit

* wanting to trigger ci
2022-07-15 11:01:54 -05:00
james-prysm
e2442bb0a6 Web3signer: support json response signature (#11055)
* initial commit

* initial commit

* fixing linting
2022-07-14 15:54:44 -05:00
james-prysm
2669006694 Web3Signer: Validator Registration (#10964)
* initial commit

* adjusting logic to match web3signer specifications

* making Epoch mandatory

* fixing unit test

* rolling back some changes on slots

* fixing unit tests

* testing enable validator registration

* adding validator registration to configs

* fixing prefix

* fixing shasum

* rolling back e2e validator registration until fully supported

* updating web3signer version

* changing types based on feedback
2022-07-14 15:04:03 -05:00
Preston Van Loon
5722a5793c Tracing: properly overwrite context so that spans can be correctly attributed (#11012)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-14 17:00:33 +00:00
james-prysm
96aba590b5 Web3Signer Flag support List in YML config (#11041)
* initial commit

* fixing linting

* fixing linting

* fixing deepsource

* changing visibility of validator flags

* updating package information

* fixing typo

* fixing another typo

* testing bazel config

* fixing linting

* fixing build

* switching flag to stringslice and adding unit tests

* fixing bazel

* rolling back bazel

* migrating unit test into validator flags

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-14 16:34:59 +00:00
Luca G.F
2162ffb05f Fix data race in monitoring test (#11032)
Signed-off-by: Luca Georges Francois <luca.georges-francois@epitech.eu>

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-07-14 16:04:12 +00:00
Nishant Das
d41805a39c Fuzz Remaining Engine Objects (#11049)
* add new fuzz tests

* add fixes

* remove test

* add more checks

* remove comment
2022-07-14 11:33:32 -04:00
terencechain
819632dfc5 Remove builder status check for block proposal (#11052) 2022-07-13 19:10:08 +00:00
Raul Jordan
10fffa6e7c [Feature] - Store Only Blinded Beacon Blocks Post-Merge (#11010)
* add in flags

* add sync and db items

* bring over all other changes

* enable on

* use feature flag

* powchain

* builds

* fix up tests

* pass iface

* gaz

* enable bellatrix blind in unmarshal only behind flag

* poolside

* pass rpc tests

* rebuilds

* naming

* cleaner func

* check needs resync

* idiomatic

* gaz

* rem

* build

* nicer

* build

* cleaner

* surface error

* wrapping

* unmarshal logs

* fix up

* cleaner

* log

* builds

* Update beacon-chain/blockchain/execution_engine.go

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* terence feedback

* test added for resync

* nil check

* fmt

Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-07-13 17:18:30 +00:00
james-prysm
5cda86bb93 Web3Signer: log for ignored wallet password flag (#11018)
* intial commit

* changing log message

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-13 16:42:08 +00:00
Potuz
f2dcc9a570 more pruning unit tests (#11034)
* unit tests

* Sort the slice before compare

* Gazelle

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-13 16:23:29 +00:00
terencechain
63354b5bb7 Don't log unless there's payload attribute (#11050)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-13 15:56:06 +00:00
terencechain
e48f0aef41 Remove IsMergeTransitionBlockUsingPreStatePayloadHeader (#11036)
* Remove IsMergeTransitionBlockUsingPreStatePayloadHeader

* Conflicts

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-07-13 15:29:41 +00:00
terencechain
ab1defc5de Improve error verbosity when payload_id is nil (#11042)
* Improve error verbosity with payload ID is nil

* Conflict

* Fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-13 12:49:40 +00:00
Nishant Das
b0e15f3c8f Fix FuzzForkChoiceResponse Crash (#11043) 2022-07-13 09:47:14 +00:00
Raul Jordan
e01a898264 Execution Payload / Header Interface Wrappers (#11025)
* exec payload iface

* begin using iface

* use iface more

* build beacon

* builds

* txs field

* fix

* merge test

* pass

* test

* refactor

* fix up builder case

* gaz

* comments

* el test

* build

* no mask

* patch up

* exec wrap

* Terence feedback

* builds

* potuz suggestion

* exec data error

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-13 01:49:38 +00:00
terencechain
7c30533870 Log error string instead of data (#11038)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-12 22:40:10 +00:00
Preston Van Loon
7654ffdcfc e2e: fix scenario test suite (#11039) 2022-07-12 22:31:37 +00:00
Preston Van Loon
d6031ac386 Add test_suites for better CI runs (#11037) 2022-07-12 16:42:44 -05:00
int88
f7d3b5c510 fix some comments (#11017)
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-07-12 19:06:38 +00:00
james-prysm
c0f3946f58 Full Deprecation of Fee Recipient File Flags (#11033) 2022-07-12 18:10:44 +00:00
Luca G.F
c33acde64e Fix counters data races in async tests (#11030)
* Fix counters data races in async/debounce tests

Signed-off-by: Luca Georges Francois <luca.georges-francois@epitech.eu>

* Fix counters data races in async/every tests

Signed-off-by: Luca Georges Francois <luca.georges-francois@epitech.eu>
2022-07-12 14:39:18 +00:00
james-prysm
8310d22a05 Validator Registration: use cached signatures if certain properties don't change. (#11014)
* initial commit

* fixing variable rename

* fixing unit test

* adding based on review comments

* renaming cache

* simplifying logic on signed validator registrationRequest

* adding unit tests

* fixing linting

* using wrong dependency
2022-07-12 04:19:49 +00:00
AH
3060096233 Trivial fix to the warning message about fee recipient config (#11027)
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-07-12 03:11:26 +00:00
terencechain
5d06c14cec Check validator has registration before getting header (#11023)
* Check validator has registration before calling header

* Check validator has registration before calling header

* Update proposer_bellatrix_test.go
2022-07-11 20:57:58 -05:00
Preston Van Loon
57abf02e34 Enforce a 1s timeout for block builder reply (#11021)
* Enforce a 1s timeout for block builder reply

* Specify BUILDER_PROPOSAL_DELAY_TOLERANCE

* clarify error message

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-11 22:18:42 +00:00
Potuz
44218a9c5b add nilcheck for payload ID (#11024)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-11 21:36:52 +00:00
Preston Van Loon
d53f2c7661 builder client: Revert key batching PR #11002 (#11022)
* builder client: Revert key batching PR #11002

* forgot

* gaz
2022-07-11 20:55:08 +00:00
terencechain
5e53d6976e Fix proposer duty RPC to allow next epoch query (#11015)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-07-11 18:46:42 +00:00
terencechain
7fd2c08be3 Propagate FCU invalid error (#10997)
* Wrap fcu error

* Wrap fcu error

* Wrap error better

* More test

* Add else

* Potuz feedback

* Propagate the correct root for fcu

* Always return true for invalid

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-09 20:51:03 +00:00
james-prysm
6695e7c756 Push Proposer Settings: improve warn logs + bug fix (#11013)
* initial commit

* adding unit test to fix bug and test for log inclusion
2022-07-08 16:15:49 -04:00
james-prysm
aeede2165d WEB v.2 (#11007)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-08 19:08:11 +00:00
terencechain
67a15183ca Wrap NewPayload error (#10994)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-08 18:24:48 +00:00
terencechain
208dc80702 Add Capella config (#11003)
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-08 17:40:01 +00:00
terencechain
e80806d455 Check nil before logging "Failed to close response body..." (#11011) 2022-07-08 16:55:12 +00:00
Raul Jordan
80d0a82f9b Engine Client Method to Reconstruct Full Bellatrix Beacon Block (#10998)
* engine reconstructor

* gaz

* powchain pass

* metrics

* deadcode

* prevent nil block

* build

* add test based on recs
2022-07-08 14:10:33 +00:00
Preston Van Loon
f9b3cde005 Batch build API requests for RegisterValidator (#11002)
* Add UnmarshalJSON for SignedValidatorRequest

* add failing test for batch limits

* Add functionality

* gofmt

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-07 23:50:48 +00:00
Raul Jordan
d8f9750387 Only Unmarshal Full Tx Bodies in ExecutionBlock JSON Unmarshaler (#11006)
* full tx unmarshaling fixed

* prefix check
2022-07-07 23:08:44 +00:00
james-prysm
5e8b9dde5b Simplify Push Proposer settings (#11005)
* initial commit

* fixing unit tests

* fixing more unit tests
2022-07-07 22:24:06 +00:00
terencechain
c2caff4230 Minor UX improvement to validator registration (#11004) 2022-07-07 19:32:02 +00:00
Raul Jordan
b67c885995 Major Simplification of JSON Handling for Execution Blocks (#10993)
* no more execution block custom type

* simpler json rpc data unmarshaling

* simplicify

* included hash and txs fix

* all tests

* pass

* build

* mock fix

* attempt build

* builds

* build

* builds

* pass

* pass

* build

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-06 22:06:00 +00:00
james-prysm
60ed488428 changing gaslimit to validator registration (#10992)
* changing gaslimit to validator registration

* adding new flag to enable validator registration for suggested fee recipient

* making sure default gaslimit is set

* Update cmd/validator/flags/flags.go

Co-authored-by: terencechain <terence@prysmaticlabs.com>

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-06 18:42:21 +00:00
Potuz
2f0e2886b4 Do not error if the LVH is bogus (#10996)
* Do not error if the LVH is bogus

* add tests and mark the regression PR

* dead code

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-06 17:37:15 +00:00
terencechain
2d53ae79df Cleanups to pulltips (#10984)
* Minor cleanups to pulltips

* Feedbacks

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-06 16:55:17 +00:00
Nishant Das
ce277cb388 Add Fuzzing For JSON Marshalling/Unmarshalling Methods (#10995)
* modify it

* add gaz

* revert

* deps

* revert change

* fix it
2022-07-06 15:15:14 +00:00
Raul Jordan
c9a366c36a Revert "Move Slasher E2E To Scenario Test" (#10986)
This reverts commit 65900115fc.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-06 13:03:30 +00:00
terencechain
3a957c567f Handle invalid_block_hash error from ee (#10991)
* Handle invalid_block_hash error from ee

* Update beacon-chain/blockchain/error.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Remove invalid block and state

* Revert "Remove invalid block and state"

This reverts commit 9ca011b8ce.

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-07-06 00:22:12 +00:00
Raul Jordan
77a63f871d Included Blinded Beacon Block in V1alpha1 Protobuf (#10989)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-05 21:03:14 +00:00
Raul Jordan
8dd8ccc147 pure funcs (#10988) 2022-07-05 16:24:17 -04:00
Preston Van Loon
3c48bce3a3 Annotate build client requests (#10987)
* Annotate build client requests

* Use named return arguments to annotate errors

* Unhandled error was bad

* Error level is better than warning for this

* Clarifying commentary while i'm here

* delete the pasta
2022-07-05 19:33:33 +00:00
Nishant Das
0ed5007d2e Fix Pubsub Panic In Handling Dead Peers (#10976)
* fix

* fix it

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-04 00:41:33 +00:00
Raul Jordan
65900115fc Move Slasher E2E To Scenario Test (#10973)
* consolidate into slasher scenario test

* pattern

* revert

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-03 23:55:33 +00:00
Potuz
379bed9268 add heuristics for pulltips (#10955)
* add heuristics for pulltips

* gazelle

* add unit test

* fix unit test

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-03 20:27:39 +00:00
Potuz
2dd2e74369 update finalization on onblock (#10980)
* update finalization on onblock

* add unit test

* Minor cleanups

Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-07-03 19:39:31 +00:00
Potuz
73237826d3 ensure there are as many deltas as nodes (#10979)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-03 00:53:33 -03:00
terencechain
af4d0c84c8 Check finalized beyond DB (#10978)
* Check finalized beyond DB

* Unhandle error

* Remove debug log

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-07-02 16:31:05 -03:00
Potuz
c68f1883d6 Save Head after pruning invalid nodes (#10977)
* Save Head after prunning

* fix unit test
2022-07-02 16:38:08 +00:00
terencechain
ae1685d937 Log invalid finalized root (#10975)
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-07-02 10:58:54 +00:00
terencechain
2a5f05bc29 Improve "rasied file descriptor limit..." log (#10970)
* Improvement to raise file descriptor log

* Radek feedback

* Change to debug
2022-07-01 18:05:01 -04:00
Potuz
49e5e73ec0 Default SafeSlotsToImportOptimistically to 128 (#10967)
* Default SafeSlotsToImportOptimistically to 128

* fix tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-07-01 17:04:51 +00:00
Nishant Das
2ecb905ae5 Update Prysm Libp2p Dependencies (#10958)
* add all changes in

* fix issues

* fix build

* remove curve check

* fix tool

* add test

* add tidy

* fmt

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-01 15:34:11 +00:00
Radosław Kapka
d4e7da8200 Change log level to debug in fetchBlocksFromPeer (#10969)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-01 14:49:31 +00:00
Nishant Das
4b042a7103 Fix Multiclient E2E (#10965)
* fix it

* gaz

* fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-01 14:02:01 +00:00
Radosław Kapka
e59859c78f Wrap client-stats flags (#10966) 2022-07-01 12:49:38 +00:00
Potuz
6bcc7d3a5e Do not fill in missing blocks on regular sync (#10957) 2022-07-01 11:03:49 +00:00
terencechain
7b597bb130 More sepolia boot nodes (#10962)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-30 19:49:35 +00:00
Potuz
2c7b273260 do not overwrite log (#10963) 2022-06-30 19:02:23 +00:00
Radosław Kapka
8bedaaf0a8 Log error in fetchBlocksFromPeer (#10959)
* Log error in `fetchBlocksFromPeer`

* update case
2022-06-30 16:22:10 +00:00
james-prysm
69350a6a80 Fee Recipient E2E misscounting deterministic keys leading to flakes (#10960) 2022-06-30 15:01:33 +00:00
terencechain
93e8c749f8 Can get payload header from builder (#10954) 2022-06-29 21:13:25 -07:00
james-prysm
96fecf8c57 E2E: fee-recipient evaluator (#10528)
* testing out fee-recipient evaluator

* fixing bazel linter

* adjusting comparison

* typo on file rolling back

* adding fee recipient is present to minimal e2e

* fixing gofmt

* fixing gofmt

* fixing flag usage name

* adding in log to help debug

* fixing log build

* trying to figure out why suggested fee recipient isn't working in e2e, adding more logging temporarily

* rolling back logs

* making e2e test more dynamic

* fixing deepsource issue

* fixing bazel

* adding in condition for latest release

* duplicate condtion check.

* fixing gofmt

* rolling back changes

* adding fee recipient evaluator in new file

* fixing validator component logic

* testing rpc client addition

* testing fee recipient evaluator

* fixing bazel:

* testing casting

* test casting

* reverting

* testing casting

* testing casting

* testing log

* adding bazel fix

* switching mixed case and adding temp logging

* fixing gofmt

* rolling back changes

* removing fee recipient evaluator when web3signer is used

* test only minimal config

* reverting changes

* adding fee recipient evaluator to mainnet

* current version uses wrong flag name

* optimizing key usage

* making mining address a variable

* moving from global to local variable

* removing unneeded log

* removing redundant check

* make proposer settings mroe deterministic and also have the evaluator compare the wanting values

* fixing err return

* fixing bazel

* checking file too much moving it out

* fixing gosec

* trying to fix gosec error

* trying to fix address

* fixing linting

* trying to gerenate key and random address

* fixing linting

* fixing check for proposer config

* trying with multi config files

* fixing is dir check

* testing for older previous balance

* adding logging to help debug

* changing how i get the block numbers

* fixing missed error check

* adding gasused check

* adding log for current gas used

* taking suggestion to make fee recipient more deterministic

* fixing linting

* fixing check

* fixing the address check

* fixing format error

* logic to differentiate recipients

* fixing linting
2022-06-30 00:24:39 +00:00
Potuz
5d29ca4984 Experimental disable boundary checks (#10936)
* init

* bellatrix + altair tests passing

* Add Phase0 support

* add feature flag

* phase0 test

* restore testvectors

* mod tidy

* state tests

* gaz

* do not call precompute

* fix test

* Fix context

* move to own's method

* remove spectests pulltips

* time import

* remove phase0

* mod tidy

* fix getters

* Update beacon-chain/forkchoice/doubly-linked-tree/types.go

* reviews

* fix workspace

* Recursive rlocks

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-06-29 23:37:21 +00:00
terencechain
43523c0266 RPC adds builder service (#10953)
* RPC adds builder service

* Update beacon-chain/builder/service.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-06-29 18:54:24 +00:00
Sammy Rosso
8ebbde7836 Testutil refactor attestations (#10952)
* Add AttestationUtil receiver

* Modify usage to account for the receiver

* Add missing explanatory comments

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-06-29 16:42:33 +00:00
Radosław Kapka
44c39a0b40 Don't log terminal difficulty has not been reached yet... until Bellatrix (#10951) 2022-06-29 13:53:59 +00:00
Radosław Kapka
f376f3fb9b Integrate better fastssz validation errors into Prysm (#10945)
* update dep

* regenerate SSZ, update test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-29 16:05:56 +08:00
Sammy Rosso
8510743406 Add additional logging fields for post-merge transition blocks (#10944)
Added additional logging information to Bellatrix blocks.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-28 13:55:36 +00:00
Radosław Kapka
b82e2e7d40 Use prysmaticlabs/fastssz as a direct dependency (#10941)
* Update dependency

* Regenerate SSZ files

* fix BUILD files

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-28 13:03:24 +00:00
Raul Jordan
acfafd3f0d Add More Visibility Into Sync Committee Message Participation (#10943)
* build

* granular time message

* timing

* log

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-27 23:01:24 +00:00
Delweng
13001cd000 typo: convert tab to space (#10918)
* typo: convert tab to space

Signed-off-by: Delweng <delweng@gmail.com>

* typo: indent shell scripts with 4spaces

Signed-off-by: Delweng <delweng@gmail.com>

* fix the indent typo again

Signed-off-by: Delweng <delweng@gmail.com>

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-06-27 21:53:32 +00:00
terencechain
5291b9d85b RPC: Add submit registration endpoint (#10942) 2022-06-27 19:54:28 +00:00
terencechain
7747471624 Update sepolia ttd (#10940)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-27 18:27:49 +00:00
Radosław Kapka
31f96a05b3 Update fastssz dependency in deps.bzl (#10939) 2022-06-27 18:32:38 +02:00
Murphy Law
dfe8e54a42 Relinquish peerLock when blockFetcher context is done (#10932)
* Relinquish peerLock when blockFetcher context is done

* Unlock before sending range requests

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-06-27 15:25:33 +00:00
Nishant Das
3a841a8467 Fix Eth1Connection API Panic (#10938)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-27 14:19:44 +00:00
Radosław Kapka
7f56ac6355 Massive code cleanup (#10913)
* Massive code cleanup

* fix test issues

* remove GetGenesis mock expectations

* unused receiver

* rename unused params

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-27 13:34:38 +00:00
Preston Van Loon
9216be7d43 State: add fuzz test for unmarshal ssz (#10935)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-06-27 11:19:39 +00:00
Nishant Das
7c489199bf Fix Builder Service Panic (#10937) 2022-06-27 10:50:15 +00:00
Potuz
f8b4d8c57a Deprecate store in blockchain pkg (#10903)
* Deprecate store WIP

* fix spectests build

* mock right interface

* sync tests build

* more tests builds

* blockchain tests

- TestFinalizedCheckpt_GenesisRootOk
- TestCurrentJustifiedCheckpt_CanRetrieve
- TestJustifiedCheckpt_GenesisRootOk
- TestHeadRoot_CanRetrieve
- TestHeadRoot_UseDB
- TestService_ProcessAttestationsAndUpdateHead
- TestService_VerifyWeakSubjectivityRoot
- TestVerifyFinalizedConsistency_InconsistentRoot_ProtoArray
- TestVerifyFinalizedConsistency_InconsistentRoot_DoublyLinkedTree
- TestVerifyFinalizedConsistency_Ok
- TestStore_OnBlock_ProtoArray
- TestStore_OnBlock_DoublyLinkedTree
- TestStore_OnBlockBatch_ProtoArray
- TestStore_OnBlockBatch_DoublyLinkedTree
- TestStore_OnBlockBatch_NotifyNewPayload
- TestCachedPreState_CanGetFromStateSummary_ProtoArray
- TestCachedPreState_CanGetFromStateSummary_DoublyLinkedTree

* more blockchain tests

- TestStore_OnBlockBatch_PruneOK_Protoarray
- TestFillForkChoiceMissingBlocks_CanSave_ProtoArray
- TestFillForkChoiceMissingBlocks_CanSave_DoublyLinkedTree
- TestFillForkChoiceMissingBlocks_RootsMatch_Protoarray
- TestFillForkChoiceMissingBlocks_RootsMatch_DoublyLinkedTree
- TestFillForkChoiceMissingBlocks_FilterFinalized_ProtoArray
- TestFillForkChoiceMissingBlocks_FilterFinalized_DoublyLinkedTree
- TestVerifyBlkDescendant
- Test_verifyBlkFinalizedSlot_invalidBlock
- TestChainService_SaveHeadNoDB

* update best justified from genesis

* deal with nil head on saveHead

* initialize prev justified checkpoint

* update finalization correctly

* update finalization logic

* update finalization logic

* track the wall clock on forkchoice spectests

* export copies of checkpoints from blockchain package

* do not use forkchoice's head on HeadRoot

* Remove debug remain

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* terence's review

* add forkchoice types deps

* wtf

* debugging

* init-sync: update justified and finalized checkpoints on db

* call updateFinalized instead of only DB

* remove debug symbols

* safe copy headroot

Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-06-25 03:57:52 +00:00
terencechain
b7463d0070 Ignore nil node when saving orphaned atts (#10930)
* Ignore nil node when saving orphaned atts

* Just use ErrUnknownCommonAncestor
2022-06-24 18:07:31 +00:00
Radosław Kapka
2b6e86ec1b Some test improvements (#10928)
* extract DeterministicGenesisStateWithGenesisBlock

(cherry picked from commit a5e3a9c9bbbacb23a644f5c68c92839a315f66a1)

# Conflicts:
#	testing/util/state.go

* part 1

* part 2

* part 3

* fix errors

* db interface public visibility

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-24 17:22:39 +00:00
kasey
4e604ee22b Stop ruining our lives with flaky e2e (#10929)
* merge checkpoint sync test with base minimal test

* move web3signer to scenario test

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-06-24 17:01:12 +00:00
Delweng
3576d2ccfe cmd/beacon-chain: add jwtcommands (#10919)
* cmd/beacon-chain: add jwtcommands

Signed-off-by: Delweng <delweng@gmail.com>

* gazelle

Co-authored-by: mick <103775631+symbolpunk@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
2022-06-24 15:54:08 +00:00
Radosław Kapka
4b009ed813 Rename upgrade_to_altair.go to upgrade_to_bellatrix.go (#10926)
(cherry picked from commit c664e59a2e66a2e52814bdb59840ec3a205273e7)

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-24 12:46:07 +00:00
Radosław Kapka
7faad27f5f Update fastssz dependency (#10927) 2022-06-24 12:18:42 +00:00
terencechain
2d3966bf4f RPC: builder block (#10908)
* Can contruct builder block

* Fix build and tests

* Add nil checks and tests

* Gazelle

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-24 02:41:16 +00:00
Nishant Das
58d10e3ace Deprecate Step Parameter from our Block By Range Requests (#10914)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-23 21:48:28 +00:00
terencechain
88becdc114 Register builder service (#10924) 2022-06-23 13:44:27 -07:00
Potuz
20b4c60bcb Apply Proposer Boost from forkchoice (#10893)
* Apply proposer boost at block insertion

* gaz

* fix tests

* revert time change

* conflict

* change genesis time in forkchoice on spectests

* Terence Review

Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-06-22 13:56:07 +00:00
terencechain
0b50ab7832 Consolidate ExecutionPayloadHeader Protobuf definitions (#10917) 2022-06-22 13:08:06 +02:00
Radosław Kapka
6b55d33ea2 Return error when requesting future state (#10915)
* Return error when requesting future state

* fix genesis test case

* fix test cases
2022-06-21 23:13:19 +00:00
terencechain
7e64a3963d Change submit validator registration to registration(s) (#10907)
* Change submit validator registration to registration(s)

* Fixed a few tests

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-21 18:46:28 +00:00
Potuz
55b4af9f92 track the wall clock on forkchoice spectests (#10916) 2022-06-21 18:12:08 +00:00
terencechain
546bb5ed53 Clean up misc warnings (#10900)
* Clean up misc warnings

* Gazelle

* Rm state assignments

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-21 15:26:56 +00:00
Radosław Kapka
29a25b3f09 Add spec URLs to API docs (#10912) 2022-06-21 14:29:44 +00:00
Jie Hou
56af079aea Change gocognit complextity threshold to 65 (#10906)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-06-20 03:57:39 +00:00
Potuz
385f101b2b Use forkchoice to verify finalized descendant (#10905)
* Use forkchoice to verify finalized descendant

* fix test
2022-06-20 00:55:17 +00:00
Radosław Kapka
b3c3b207c9 Enable fastssz to use vectorized HTR hashing algo (#10819)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-18 12:13:36 +00:00
Jie Hou
e0cd10ed3c Refactor: Reduce cognitive complexity for 5 out of top 10 functions in code base (#10854)
* Refactor waitForActivation

* Finish refactor of runner.go

* Refactor validator/client/metrics.go

* Refactored beacon-chain/sync/pending_attestations_queue.go

* Refactored beacon-chain/powchain/log_processing.go

* Resolve conflicts with develop branch

* Fix Deepsource: Context should be the first param

* Address review comments

* Put headersMap as pass-in function parameter

* Change function signature of processBlockInBatch

* Address nisdas's comment
2022-06-18 10:14:43 +00:00
Potuz
2d0fdf8b4a prune within forkchoice (#10896) 2022-06-17 12:53:17 -03:00
Potuz
3a7117dcbf Do not downcast time in currentslot (#10897)
* Do not downcast time in currentslot

* no magic constants

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-16 22:26:55 +00:00
Radosław Kapka
8888fa4bb3 Revert enable-native-state flag (#10898)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-06-16 20:59:46 +00:00
Radosław Kapka
f065209a3e Small API Middleware upgrades (#10895)
* remove useless comments

* add comment

* fix writing bug

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-16 19:19:05 +00:00
Potuz
e439f4aff6 Forkchoice duplicate store (#10840)
* forkchoice tests

* blockchain tests

* no fatal errors

* use ZeroHash for genesis

* deal with zerohashes at genesis

* avoid nil best justified checkpoint on new store

* unit tests

* Debug log

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* Terence's review

* update capitalization

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* update capitalization

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* update capitalization

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* update capitalization

Co-authored-by: terencechain <terence@prysmaticlabs.com>

Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-06-16 18:21:40 +00:00
terencechain
88f8dbecc8 Compute correct domain for validator registration (#10894)
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-06-16 16:32:04 +00:00
james-prysm
2dfe291cf9 Keymanager APIs: fee recipient api (#10850)
* initial commit wip

* setting protos for fee recipient

* updating proto based on specs

* updating apimiddleware

* generated APIs

* updating proto to fit spec

* fixing naming of fields

* fixing endpoint_factory and associated structs

* fixing imports

* adding in custom http types to grpc gateway

* adding import options

* changing package option

* still testing protos

* adding to bazel

* testing dependency changes

* more tests

* more tests

* more tests

* more tests

* more tests

* more tests

* testing changing repo dep

* testing deps

* testing deps

* testing deps

* testing deps

* testing deps

* testing deps

* reverting

* testing import

* testing bazel

* bazel test

* testing

* testing

* testing import

* updating generated proto code

* wip set fee recipient by pubkey

* adding list fee recipient logic

* fixing thrown error

* fixing bug with API

* fee recipient delete function

* updating generated proto logic

* fixing deposit api and adding postman tests

* fixing proto imports

* fixing unit tests and checksums

* fixing test

* adding unit tests

* fixing bazel

* Update validator/rpc/standard_api.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/rpc/standard_api.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* resolving review comments

* fixing return

* Update config/validator/service/proposer-settings.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/rpc/standard_api.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/rpc/standard_api.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* updating proto

* updating message name

* fixing imports

* updating based on review comments

* adding middleware unit tests

* capitalizing errors

* using error instead of errorf

* fixing missed unit test variable rename

* fixing format variable

* fixing unit test

* Update validator/rpc/standard_api.go

* Update validator/rpc/standard_api.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-16 15:10:23 +00:00
Mike Neuder
a80c15f3a9 Refactor validator accounts import to remove cli context dependency (#10890)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-06-16 14:14:03 +00:00
Nishant Das
4de92bafc4 Improve Field Trie Recomputation (#10884)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-06-16 13:14:29 +00:00
terencechain
69438583e5 Pad Uint256's SSZBytes to length 32 (#10889)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-16 04:29:32 +00:00
Raul Jordan
e81f3fed01 Remove Extraneous BoltDB Logs (#10888) 2022-06-16 01:11:07 +00:00
Raul Jordan
1b2a5fb4a5 Update CODEOWNERS (#10887) 2022-06-15 21:51:44 +00:00
Jie Hou
6c878b1665 Refactor: Continue reducing cognitive complexity (#10862)
* Refactor beacon-chain/db/kv/state.go

* Refactor api/gateway/apimiddleware/process_field.go

* Refactor beacon-chain/sync/initial-sync/blocks_queue.go

* Refactor validator/db/kv/migration_optimal_attester_protection.go

* goimports

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-06-15 18:34:59 +00:00
james-prysm
838963c9f7 validator registration request bug: reusing public keys (#10883)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-15 17:26:05 +00:00
kasey
7b38f8b8fc submit lists of validator registrations (#10882)
* submit lists of validator registrations

* RegisterValidator to take a list

* Gazelle

* Fix go imports

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-15 16:29:56 +00:00
Nishant Das
23e8e695cc Fix Sepolia Testnet Initialization (#10886) 2022-06-15 15:05:44 +00:00
Sammy Rosso
ce9eaae22e Add payload data logging (#10845)
* Add logging of block payload data

Added a new func logBlockPayloadData that includes logging of the
block number and the gas utilized.
Related to #10795.

* Replace Info with Debug + renamed func

Renamed the function to be clearer and replaced Info logging with Debug.
Related to #10795.

* Compute correct value for gas utilized

Related to #10795.

* Round result of gas utilized to 2 decimal places

* Add new error message

* Check if block is an Execution block

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* Fix missing imports

* Undo changes

* Update beacon-chain/blockchain/receive_block.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Added error logging to log statements

Changed the error handling from log statements. Instead of returning the
error we log the error.

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-15 10:03:48 +00:00
Nishant Das
7010e8dec8 Graduate Prune Canonical Attestations Feature (#10623)
* graduate canonical prune feat

* fix test

* fix tests

Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-15 09:05:19 +00:00
Nishant Das
9e4ba75e71 Batch Scenario Runs Into Single Test (#10878)
* batch scenarios

* fix

* fix

* Update testing/endtoend/endtoend_test.go
2022-06-15 08:02:31 +00:00
kasey
044a4ad5a3 Ignore genesis state url and checkpoint sync flags after first run of prysm (#10881)
* ignore remote genesis url flag if present in db

* ignore checkpoint sync flags if initialized

* lint

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-14 23:23:25 +00:00
Radosław Kapka
690084cab6 Enable native state for Sepolia (#10880)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-14 21:51:58 +00:00
james-prysm
88db7117d2 Adding additional checksum check for fee recipient. (#10879)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-14 18:43:37 +00:00
mick
1faa292615 Add is_optimistic to SyncDetails, hydrate via ValidateSync (#10692)
* cache test

* oh

* syntax fix

* error fix

* tinker

* tinker

* newlines?

* no-whitespace?

* feedback

* fix

* comment

* comments

* need to figure out how to lint locally...

* feedback

* fixes

* progress

* progress

* dedupe

* s

* working

* remove empty lines

* update test

* return errors properly

* make helpers publicly visible

* fix tests

Co-authored-by: rkapka <rkapka@wp.pl>
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-06-14 17:47:09 +00:00
terencechain
434018a4b9 Add Sepolia config (#10868) 2022-06-14 14:50:05 +00:00
Nishant Das
54624569bf Fix Fuzzing Failures in Our CI (#10875)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-14 13:12:28 +00:00
Håvard Anda Estensen
b55ddb5a34 Use go:build lines and remove obsolete +build lines (#10704)
* Use go:build lines and remove obsolete +build lines

* Run gazelle

* Update crypto/bls/blst/stub.go

* Update crypto/bls/blst/stub.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-06-14 11:47:27 +00:00
terencechain
a38de90435 Move computeCheckpoints to private (#10874)
* Move computeCheckpoints to private

* Feedback

* Godoc

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-14 09:25:41 +00:00
Michael Blau
d454d30f19 Merge ascii art banner (#10773)
* Add Merge ASCII art banner

* Add merge ASCII art banner

* gofmt

* Go fmt

* Fix go fmt again

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-06-14 08:25:42 +00:00
Jie Hou
b04dd9fe5c Enable gocognit linter (#10867)
* Enable gocognit linter

Currently the gocognit complexity threshold is set to 95 to make
sure no existing files will fail the linter. In future we will
reduce this threshold to a much lower one.

The recommended threshold is usually 30. Our code base has maximum
of 97 right now...But it's better late than never to pay attention
to our code compexity.

* Test to see github complains

* Resume to 97

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-06-14 14:19:34 +08:00
kasey
8140a1a7e0 update info message about ws checkpoint (#10871)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-14 00:55:00 +00:00
terencechain
cab9917317 Fix message typo for ErrorIs (#10873)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-06-14 00:19:04 +00:00
terencechain
4c4fb9f2c0 Fix gosec scan: G112 (CWE-400) Potential Slowloris Attack (#10872) 2022-06-13 22:29:26 +00:00
Mike Neuder
80f4f22401 Refactor validator accounts exit to remove cli context dependency (#10841)
* Refactor validator accounts exit to remove cli context dependency

* bazel run //:gazelle -- fix

* fixing deepsource findings

* fixing broken test

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-06-13 15:17:46 +00:00
terencechain
dd296cbd8a Disallow lower justified epoch to override higher epoch (#10865) 2022-06-11 17:37:37 +00:00
terencechain
f9e3b0a3c2 Active balance: return EFFECTIVE_BALANCE_INCREMENT as min (#10866) 2022-06-11 08:54:33 -07:00
terencechain
a58809597e Sync: don't process pending blocks w/o genesis time (#10750)
* Sync: don't process pending blocks w/o genesis time

* Update pending_blocks_queue.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-06-10 05:02:47 +00:00
Nishant Das
7f443e8387 Add Optimistic Sync Scenario Testing (#10836)
* add latest changes

* fix it

* add multiclient support

* fix tests

* Apply suggestions from code review

* fix test

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-09 23:24:53 +00:00
Potuz
18fc17c903 Forkchoice checkpoints (#10823)
* double_tree_changes

* protoarray changes

* beacon-chain changes

* spec tests and debug rpc fixes

* more conflicts

* more conflicts

* Terence's review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-09 22:28:30 +00:00
Sammy Rosso
d7b01b9d81 Fix default mainnet log when using chain config (#10855)
* Fix default mainnet log when using chain config

Add a log to specify the use of a chain-config-file rather than
defaulting to that of mainnet.
Related to #10821.

* Add more specific log message

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-09 21:31:59 +00:00
terencechain
7ebd9035dd Add ttd metric (#10851)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-09 20:35:38 +00:00
Radosław Kapka
578fea73d7 API's IsOptimistic - update header.StateRoot only when block is not missing (#10852)
* bug fix

* tests

* comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-09 19:35:35 +00:00
terencechain
7fcadbe3ef Add optimistic status to chainhead (#10842)
* Add optimistic status to chainhead

* Fix tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-09 17:37:52 +00:00
Radosław Kapka
fce9e6883d Native Blocks Ep. 1 - New types and functions (#10837)
* types and functions

* partially done tests

* refactor

* remaining Proto() tests

* remaining proto.go tests

* simplify UnmarshalSSZ and move BeaconBlockIsNil

* getters_test

* remove errAssertionFailed

* review feedback

* remove cloning protobuf

* fmt

* change IsNil

* fix tests
2022-06-09 13:13:02 +00:00
terencechain
5ee66a4a68 Update engine api buckets (#10847)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-08 20:12:54 +00:00
kasey
f4c7fb6182 checkpoint sync e2e to use 3m timeout w/ elapsed log (#10849)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2022-06-08 19:14:10 +00:00
Potuz
077dcdc643 enable dev on Ropsten (#10839)
* enable dev on Ropsten

* include only vectorized HTR and doublylinkedtree

* error handling

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-08 13:45:13 +00:00
kasey
1fa864cb1a use slot:block index correctly (#10820)
* adding splitRoots, refactor to use it

* use splitRoots & work in roots only

the most common use case for this method is to get a list of
candidate roots and check if they are canonical. there isn't a great
reason to look up all the non-canonical blocks, because forkchoice
checks based on the root only, so just return roots and defer the
responsibility of resolving those to full blocks.

* update comment

* clean up shadowing

* more clear non-error return

* add test case for single root in index slot

* fmt

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-07 16:47:42 +00:00
Mike Neuder
6357860cc2 Refactor validator accounts backup to remove cli context dependency (#10824)
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-06-07 15:19:12 +00:00
mick
cc1ea81d4a Implement generate-auth-secret on beacon node CLI (#10733)
* s

* s

* typo

* typo

* s

* s

* fixes based on PR feedback

* PR feedback

* reverting log changes

* adding flag per feedback

* conventions

* main fixes

* Update cmd/beacon-chain/jwt/jwt.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/beacon-chain/jwt/jwt.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* s

* tests

* test attempt

* test

* Update cmd/beacon-chain/jwt/jwt.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* err fix

* s

* further simplify

* cleanup

* namefix

* tests pass

* gaz

* rem deadcode

* Gaz

* shorthand

* naming

* test pass

* dedup

* success

* Ignore jwt.hex file

* logrus

* feedback

* junk

* Also check that no file was written

* local run config

* small fix

* jwt

* testfix

* s

* disabling test

* reverting main changes

* main revert

* removing temp folder

* comment

* gaz

* clarity

* rem

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2022-06-07 07:37:12 +00:00
Nishant Das
dd65622441 Efficiently Pack Uint64 Lists (#10830)
* make it more efficient

* radek's review

* review again
2022-06-07 06:34:13 +00:00
Nishant Das
6c39301f33 Integrate Engine Proxy into E2E (#10808)
* add it in

* support jwt secret

* fix it

* fix

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-06 23:35:54 +00:00
terencechain
5b12f5a27d Integrate builder client into builder service (#10825)
* Integrate builder client into builder service

* Do nothing if pubkey is not found

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-06 22:38:49 +00:00
terencechain
105d73d59b Update spec labels (#10833)
* Update label version

* Update label version

* Update label version

* Update label version

* Update label version
2022-06-06 18:13:59 -04:00
james-prysm
db687bf56d Validator client builder support (#10749)
* Startinb builder service and interface

* Get header from builder

* Add get builder block

* Single validator registration

* Add mev-builder http cli flag

* Add method to verify registration signature

* Add builder registration

* Add submit validator registration

* suporting yaml

* fix yaml unmarshaling

* rolling back some changes from unmarshal from file

* adding yaml support

* adding register validator support

* added new validator requests into client/validator

* fixing gofmt

* updating flags and including gas limit, unit tests are still broken

* fixing bazel

* more name changes and fixing unit tests

* fixing unit tests and renaming functions

* fixing unit tests and renaming to match changes

* adding new test for yaml

* fixing bazel linter

* reverting change on validator service proto

* adding clarifying logs

* renaming function name to be more descriptive

* renaming variable

* rolling back some files that will be added from the builder-1 branch

* reverting more

* more reverting

* need placeholder

* need placeholder

* fixing unit test

* fixing unit test

* fixing unit test

* fixing unit test

* fixing more unit tests

* fixing more unit tests

* rolling back mockgen

* fixing bazel

* rolling back changes

* removing duplicate function

* fixing client mock

* removing unused type

* fixing missing brace

* fixing bad field name

* fixing bazel

* updating naming

* fixing bazel

* fixing unit test

* fixing bazel linting

* unhandled err

* fixing gofmt

* simplifying name based on feedback

* using corrected function

* moving default fee recipient and gaslimit to beaconconfig

* missing a few constant changes

* fixing bazel

* fixing more missed default renames

* fixing more constants in tests

* fixing bazel

* adding update proposer setting per epoch

* refactoring to reduce complexity

* adding unit test for proposer settings

* Update validator/client/validator.go

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* trying out renaming based on feedback

* adjusting based on review comments

* making tests more appropriate

* fixing bazel

* updating flag description based on review feedback

* addressing review feedback

* switching to pushing at start of epoch for more time

* adding new unit test and properly throwing error

* switching keys in error to count

* fixing log variable

* resolving conflict

* resolving more conflicts

* adjusting error message

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2022-06-06 19:32:41 +00:00
james-prysm
f0403afb25 Suggested-Fee-Recipient flag should not override (#10804)
* initial commit

* fixing unit test

* fixing gofmt

* updating usage language

* updating flag description

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-06 16:58:52 +00:00
Potuz
3f309968d4 Only prune in newer finalization (#10831)
* Only prune in newer finalization

* add regression test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-06 15:25:22 +00:00
Potuz
52acaceb3f Enforce that every node descends from finalized node (#10784)
* Enforce that every node descends from finalized node

* fix test

* fix conflict

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-06 14:28:49 +00:00
Nishant Das
5216402f66 Clean Up State Finalizer (#10829) 2022-06-06 09:01:58 +00:00
Potuz
2536195be0 Change forkchoice API (#10774)
* Change forkchoice API

doubly-linked-tree changes

* protoarray changes

* blockchain tests

* rebase and fix conflicts

* More blockchain fixes

* blockchain test fixes

* doubly linked tree changes again

* protoarray changes v2

* blockchain packages v2

* Radek's review

* Fix on batch processing

* Terence's review

* fill in at start

* Revert "fill in at start"

This reverts commit 8c11db063a.

* wrap error message

* fill in before mutating the state

* wrap nil node errors

* handle unknown optimistic status on init sync

* rename insert function

* prune on batches only after forkchoice insertion

Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-06-05 17:48:21 +00:00
terencechain
a4b9e006af Fill missing payload ID (#10803) 2022-06-05 05:34:08 +00:00
terencechain
76b941b310 Update ropsten TTD (#10817)
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-04 02:54:51 +00:00
Nishant Das
d099c2790b Add Back Timestamp Related Deposit Processing (#10806)
* use it

* add test

* fix

Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-06-04 01:58:03 +00:00
Potuz
2fc3d41ffa goppelganger -> doppelganger (#10816)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-03 22:53:58 +00:00
Jim McDonald
2e4174bdd6 Set fee recipient validator index as per spec. (#10814)
The beacon API specification states that the `validator_index` field of
the `fee_recipient` structure is a quoted string rather than a bare
integer.  This fixes up the encoding to match the spec.

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-06-03 21:52:17 +00:00
kasey
a170fd4bd6 Revert "Change name and return type of HighestSlotBlocksBelow (#10811)" (#10818)
This reverts commit 0e9dfe04a1.

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2022-06-03 18:36:06 +00:00
terencechain
3e36eacd1e Add registration DB methods (#10812) 2022-06-03 15:13:04 +00:00
kasey
0e9dfe04a1 Change name and return type of HighestSlotBlocksBelow (#10811)
* HighestSlotBlocksBelow to return a single value

* HighestSlotBlocksBelow->HighestBlockBelowSlot

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-03 07:16:34 +00:00
Radosław Kapka
302a5cf00b API SSZ content negotiation (#10760)
* API SSZ content negotiation

* custom code

* use raw string

* code review feedback

* remove duplicated test
2022-06-02 22:13:58 +00:00
kasey
c72f1af951 rewind cursor upon nil value (bucket) (#10802)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-06-02 04:06:19 +00:00
terencechain
2bc3ef29e0 Wrap unknown RPC errors (#10793) 2022-06-01 16:46:07 +00:00
terencechain
bc91d63fcf Add builder service skeleton and flag (#10789)
* Add builder service skeleton and flag

* Fix build
2022-06-01 15:12:15 +00:00
Potuz
8f18920ac7 Update forkchoice justified on init sync (#10801)
* Update forkchoice justified on init sync

* handle err

* add regression test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-01 14:13:47 +00:00
Nishant Das
2cc62cdf40 Fix Fuzzing (#10798) 2022-06-01 13:15:50 +00:00
terencechain
95c140b512 Validator client: add submit registration (#10785)
* Add client registration methods

* Mocken

* Run mockgen

* Fix bad tests

* Fix rest of the tests

* Tests

* Fix time tests

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-01 01:53:25 +00:00
kasey
7563bc0444 HighestSlotBlocksBelow, but in reverse (#10772)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-31 21:10:33 +00:00
Preston Van Loon
4353d39daa Mark e2e test targets as flaky for now (#10778)
* Mark e2e test targets as flaky for now

* Mark e2e test targets as flaky for now

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-31 20:11:47 +00:00
terencechain
2de2000eb7 Add back finalized info in new block log (#10792)
* Add back finalized info in new block log

* Better refactor

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-31 17:44:36 +00:00
terencechain
94f6389ebd Run mockgen (#10775) 2022-05-31 09:43:01 -07:00
Potuz
b582ca27e6 Forkchoice fill chain (#10783)
* Move chain inclusions to forkchoice handling

* gaz

* return early if no blocks are pending

* Terence's review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-31 10:19:48 +00:00
Preston Van Loon
2586a9e667 Add and verify context in AddCommitteeShuffledList (#10786)
* Add and verify context in AddCommitteeShuffledList

* Add and verify context in AddCommitteeShuffledList

* Change unnecessary arg reordering

* fix build, oops

* Regression test

* Regression test

* fix fuzz cache disabled
2022-05-30 21:38:37 -03:00
terencechain
87251d627d Ensure finalized root can't be zeros (#10791) 2022-05-30 18:01:30 +00:00
Potuz
3ff285dda5 Do not fill in blocks that are not in the finalized branch (#10776)
* Do not fill in blocks that are not in the finalized branch

* mark blocks as invalid

* Fix old off-by-ones, do not pass finalized state
2022-05-30 11:55:39 +00:00
terencechain
364ad3fbda Reinsert reorg atts (#10767)
* Add common ancestor root for protoarray

* More efficient algo

* Tests

* Fix linting

* Fix linting

* Fix linting

* Fix linting

* Fix linting

* Fix linting

* Apply suggestions from code review

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

* Feedbacks

* Revert saveHead changes

* Revert "Revert saveHead changes"

This reverts commit a15fddc2e6.

* Fix rest of the tests

* Update beacon-chain/blockchain/head.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-29 19:32:42 +00:00
Potuz
4cbb69602f Clean up onBlockBatch and prune forkchoice on init sync (#10768)
Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-05-28 11:56:58 +00:00
David
64920d719d add mutex for validator.highestValidSlot (#10722)
* add mutex for validator.highestValidSlot

* fixed deadlock issue

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-05-28 00:44:06 +00:00
Potuz
3cf385fe91 Unrealized justification (#10659)
* unrealized justification API

* Add time elapse logging

* add unrealized justification checkpoint

* Use UnrealizedJustificationCheckpoint

* Refactor unrealized checkpoints

* Move logic to state package

* do not use ctx on a sum

* fix ctx

* add tests

* fix conflicts

* unhandled error

* Fix ordering in computing checkpoints

* gaz

* keep finalized checkpoint if nothing justified

* gaz

* copy checkpoint

* fix check for nil

* Add state package tests

* Add tests

* Radek's review

* add more tests

* Update beacon-chain/core/epoch/precompute/justification_finalization.go

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* deduplicate to stateutil

* missing file

* Add stateutil test

* Minor refactor, don't export certain things

* Fix exports in tests

* remove unused error

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-27 16:38:00 +00:00
David
adabd1fa4f service.head race condition fix (#10741)
* added various read mutex locks for service.head

* added RLocks around all calls to s.headRoot()

* added RLocks around all calls to s.headBlock()

* reduce lock surface-> Stop(),handleEpochBoundary()

* refactor Stop() to +performance, -lock_surface

* Apply suggestions from code review

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* fixed indentation

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-05-27 14:24:43 +00:00
Nishant Das
9be2111b7a Write To Only One File For Geth Logs (#10769) 2022-05-27 13:06:30 +00:00
Raul Jordan
1dec9eb912 Remove Unnecessary Stack Traces for Engine Errors (#10765)
* cleaner rpc responses

* clean up unecessary stack trace

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-26 22:09:33 +00:00
terencechain
a2951ec37d Update Ropsten ttd to 100000000000000000000000 (#10762)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-26 21:12:59 +00:00
terencechain
216fdb48cf Add back ttd override flags (#10763)
* Add back ttd override flags

* Add warning log for overrides

* Print has hex

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
2022-05-26 19:12:02 +00:00
Preston Van Loon
61c5e2a443 Fix error message with incorrect flag names (#10761) 2022-05-26 17:47:00 +00:00
Nishant Das
da9f72360d Make Common Dep Set For E2E (#10758) 2022-05-26 09:33:04 +02:00
Preston Van Loon
1be46aa16e Add a fuzz test for fieldtrie (#10757)
* Add a fuzz test for fieldtrie

* gofmt
2022-05-26 13:17:34 +08:00
kasey
a1a12243be Sync from finalized (#10723)
* checkpoint sync use finalized state+block

instead of finding the block at the beginning of the weak subjectivity
epoch.

* happy path test for sync-from-finalized

* gofmt

* functional opts for the minimal e2e

* add TestCheckpointSync option

* wip: pushing for CI

* include conn index in log for debugging

* lint

* block until regular sync test finishes

* restore TestSync->testDoppelGangerProtection link

* update bazel deps for all the test targets

* updating to match current checksum from github

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2022-05-25 22:52:43 +00:00
kasey
a1dd2e6b8c updating to match current checksum from github (#10756)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2022-05-25 18:29:17 +00:00
Nishant Das
ecb605814e Add in Separate Directories per Test (#10753)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-05-25 15:57:13 +00:00
terencechain
61cbe3709b Ignore subset aggregates (#10674)
* Ignore subset aggregates

* Add test

* Update BUILD.bazel

* Update validate_sync_contribution_proof_test.go

* Update validate_sync_contribution_proof_test.go

* Don't utilize pooled objects. Use direct caches

* Handle mainnet/minimal better

* Revert att changes

* Check overlaps before set

* Feedbacks

* Fixed a copy bug

* Fixed the same copy bug in committee indices cache

* Use SafeCopyBytes

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-25 05:40:06 +00:00
Raul Jordan
7039c382bf Update Golang X Tools to Support Generics in Prysm (#10752)
* update tools to enable generics support

* tidy

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-25 05:10:05 +00:00
Nishant Das
4f00984ab1 fix (#10751) 2022-05-24 23:10:51 -04:00
terencechain
edb03328ea Sync: use copied bits for seen cache (#10747)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-24 21:11:33 +00:00
Sammy Rosso
c922eb9bfc Add new DomainType for application usage (#10740)
* Add new DomainType for application usage

* Add DomainApplicationMask to config_test

* Add func to convert uint32 to 4 byte array

We add a convenience function Uint32ToBytes4 that takes a uint32,
converts it to big endian order and return a 4 byte array.

* Add unit test for Uint32ToBytes4

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-05-24 20:16:05 +00:00
terencechain
051a83a83d Engine metrics: help text typos (#10746)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-24 19:17:17 +00:00
terencechain
f46ff626df Service: return errors on nil checkpoints (#10748) 2022-05-24 16:15:04 +00:00
Preston Van Loon
8130ff29bc Update .buildkite-bazelrc: change CI to use toplevel remote caching. (#10744) 2022-05-24 13:22:44 +00:00
Nishant Das
5d4078305a Use Correct Math Library (#10742)
* use correct math lib

* one more case
2022-05-24 07:22:46 +00:00
Preston Van Loon
6910460173 Refactor migrateStateValidators for better readability (#10727)
* Refactor migrateStateValidators for better readability

* pass test

* rev

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-05-23 23:56:02 +00:00
terencechain
7cc291c09f Update proposer boost score and spec tests (#10665)
* Update mainnet_config.go

* Fixed a few tests

* Update rest of the tests

* Consolidate blockchain errors

* Update spec tests to v1.2.0-rc.1

* Fix withdrawal epoch overflows

* add slashings to forkchoice spectests

* Remove unused parameter

* Disable skip slot cache

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-05-23 23:08:24 +00:00
Raul Jordan
76645bccee Move ExecutionPayload Utils to Consensus Types Subpackage (#10732)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-23 21:57:32 +00:00
terencechain
46c0579816 Fix withdrawal epoch overflows (#10739)
* Fix withdrawal epoch overflows

* Fix typo, used epoch instead of churn
2022-05-23 21:10:19 +00:00
kasey
c66d9e9a11 Builder client (#10703)
* builder api client

* unexport error

* thanks, DeepSource!

* replace hexSlice w/ hexutil.Bytes

* use uint256 for BaseFeePerGas

* more confidence in correct endianness

* comment fix per Terence

* fix proto conversion for uint256

* couple more value checks in the http client tests

* TestMarshalBlindedBeaconBlockBodyBellatrix

* appease deepsource

* middleware to log requests

* big int round trip test

* very superficial test to make deepsource happy

* round trip test between proto payloads

* round trip starting from marshaled struct

* deepsource... for you, the moon

* remove unused receiver

* gofmt

* remove test destroying line added while debugging

* handle nil body in logging middleware

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-23 17:30:51 +00:00
Potuz
237807b248 Do not update forkchoice checkpoints on calls to Head (#10702)
* Only update forkchoice checkpoints at epoch transition

* gazelle

* blockchain package changes

* fix node

* spectest package

* rpc package

* fix spec test

* Fix spectests

* fix new_slot.test

* gaz

* fix spectests

* fix conflicts

* Update beacon-chain/blockchain/process_block.go

* regression do not update on newSlot

* revert bd64cab

* terence's review

* fix conflicts

* fix latest conflicts

* gaz

* go mod tidy

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-22 18:37:01 +00:00
terencechain
bcd75adedd Consolidate blockchain errors (#10736)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-22 17:48:54 +00:00
Potuz
0c981e8853 Change synced new block log (#10724)
* Change synced new block log

* Update beacon-chain/blockchain/log.go

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* Fix finalized -> justified

* Update beacon-chain/blockchain/receive_block.go

* fix conflicts

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-22 16:57:48 +00:00
Potuz
e7991b9d7b Track unrealized justification/finalization in forkchoice (#10658)
* Track unrealized justification/finalization in forkchoice

* add missing files

* mod tidy

Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-05-21 10:27:07 -03:00
Raul Jordan
244e670b71 Move BeaconBlockNil Checker Function to Consensus-Types/Wrapper Package (#10731)
* beacon block is nil wrapper

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-20 23:29:16 +00:00
Raul Jordan
dc5fb92b28 Remove Unnecessary State Interfaces (#10707)
* rem other state interfaces

* redundant check

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-20 22:40:03 +00:00
terencechain
a2db03b9e9 Update engine API err codes (#10730)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-05-20 20:08:00 +00:00
terencechain
370cf1a6c8 Chain info: Return err if checkpoint is nil (#10729)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-20 18:41:33 +00:00
Potuz
76f6d74b83 Add new slot tests (#10728)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-20 18:17:53 +00:00
terencechain
addb3cd665 More selective on marking block as bad (#10681)
* Starting, looking for feedbacks

* Update error.go

* Wrap invalid blocks to rest of the processings

* More tests

* More tests

* Fix tests

* Update process_block_test.go

* Update execution_engine_test.go

* Update BUILD.bazel

* Nishant's feedback and Kasey's recommendation

* Add comments on what an invalid block is

* Update beacon-chain/blockchain/error.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

* Update beacon-chain/blockchain/error.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

* Rm faulty invalid conditions

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-20 17:05:39 +00:00
Preston Van Loon
c603d120d7 Refactor high complexity StreamBlocksAltair (#10726)
* Refactor high complexity StreamBlocksAltair

* catch nil data

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-20 16:17:55 +00:00
Raul Jordan
39893bbe30 Encapsulate Fork-Specific Parameters Under BeaconState Interface (#10706)
* encapsulate beacon state specific spec parameters

* edit build file

* code review suggestion

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-20 15:30:30 +00:00
Nishant Das
a984605064 Scenario End To End Testing (#10696)
* add new changes

* fix it all

* add nicer scenario

* some more cleanup

* restructure tests

* godoc

* skip one scenario

* space

* fix test

* clean up

* fix conflicts

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-05-20 09:34:10 +00:00
Nishant Das
f28b47bd87 Raise Soft File Descriptor Limit Up To The Hard Limit (#10650)
* add changes

* comment

* kasey's review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-20 08:12:52 +00:00
kasey
588dea83b7 Config registry (#10683)
* test coverage and updates to config twiddlers

* LoadChainConfigFile error if SetActive conflicts

* lint

* wip working around test issues

* more fixes, mass test updates

* lint

* linting

* thanks deepsource!

* fix undeclared vars

* fixing more undefined

* fix a bug, make a bug, repeat

* gaz

* use stock mainnet in case fork schedule matters

* remove unused KnownConfigs

* post-merge cleanup

* eliminating OverrideBeaconConfig outside tests

* more cleanup of OverrideBeaconConfig outside tests

* config for interop w/ genesis gen support

* improve var name

* API on package instead of exported value

* cleanup remainders of "registry" naming

* Nishant feedback

* add ropstein to configset

* lint

* lint #2

* ✂️

* revert accidental commented line

* check if active is nil (replace called on empty)

* Nishant feedback

* replace OverrideBeaconConfig call

* update interop instructions w/ new flag

* don't let interop replace config set via cli flags

Co-authored-by: kasey <kasey@users.noreply.github.com>
2022-05-20 07:16:53 +00:00
Nishant Das
1012ec1915 Drop Down Expected Sync Participation During Fork (#10708)
* reduce

* skip
2022-05-19 12:28:56 +00:00
Raul Jordan
b74947aa75 Fix Slasher E2E Flakiness (#10715)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-19 08:51:31 +00:00
kasey
e3f69e4fad run e2e for 12 epochs (#10717)
* run for the original number of epochs

* to avoid race, wait 1/2 epoch after 1st ready node

* Update testing/endtoend/minimal_e2e_test.go

* Update testing/endtoend/minimal_e2e_test.go

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-05-19 06:45:58 +00:00
terencechain
092e9e1d19 Clean up various warnings (#10710)
* Clean up various warnings

* Update beacon-chain/rpc/prysm/v1alpha1/debug/state_test.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Fix redundant casting genState

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-05-19 04:38:04 +00:00
Sammy Rosso
1c51f6d1be Fix Tests in sync/validate_beacon_blocks.go to Pass for the Right Reasons (#10711)
* test: better InvalidSignature validation

Added digest to topic so that the test would not fail at decodePubsubMessage.
Generated a valid signature from an invalid key so that it would be considered invalid.
Added a check for the correct error: ErrSigFailedToVerify.
Related to #9791.

* test: better BlockAlreadyPresentInDB validation

Added digest to topic so that the test would not fail at decodePubsubMessage.
Added a check that the block is ignored.
Related to #9791.

* test: better Syncing validation

Replaced error silencing with NoError assertion.
Added check for the expected validation decision: ValidationIgnore.
Related to #9791.

* test: better IgnoreAndQueueBlocksFromNearFuture validation

Rename test to best describe it's function.
Replace error silencing with a check for the expected error.
Related to #9791.

* test: better RejectBlocksFromFuture validation

Added digest to topic so that the test would not fail at decodePubsubMessage.
Replaced error silencing with NoError assertion.
Added check for the expected validation decision: ValidationIgnore.
Related to #9791.

* test: better RejectBlocksFromThePast validation

Added digest to topic so that the test would not fail at decodePubsubMessage.
Replaced error silencing with a check for the expected error.
Added check for the expected validation decision: ValidationIgnore.
Related to #9791.

* test: better SeenProposerSlot validation

Added digest to topic so that the test would not fail at decodePubsubMessage.
Replaced error silencing with NoError assertion.
Added check for the expected validation decision: ValidationIgnore.
Related to #9791.

* test: fix topic and silenced errors

* set chain service slot

* test: better RejectBlocksFromBadParent validation

Set bad parent block.
Fixed expected errors and validation decision.

* revert: correct log msg for unknown parent block

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-19 03:45:45 +00:00
Preston Van Loon
ee6aa4d4ec rpc: Better connection handling (#10714) 2022-05-19 02:53:24 +00:00
terencechain
0d2696ed4e Add bopsten config and cli flag (#10700)
* Add bopsten config and cli flag

* Rename : )

* Update genesis time

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-18 19:29:24 +00:00
mick
03d44d7bfe Weak subjectivity warning copy update, warn -> info (#10699)
* warn -> info

* wrap

* lintfix

* mick-needs-a-better-IDE

* gofmt

* empty commit to trigger cicd

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-05-18 16:20:35 +00:00
james-prysm
b38e4ddc3e Web3signer: Bellatrix Block (#10590)
* initial commit

* fixing bazel

* removing extra space

* adding blindedbeaconblock

* adding unit tests

* fixing bazel build

* switching to switch statement

* fixing function parameters

* fixing ineffectual err error

* deleting unused files

* updating web3signer version to support bellatrix

* adding metrics

* testing longer run

* changing log level to help find errors

* changing logging back to all to better understand the issue

* testing better way to get public keys for web3signer on validator

* fixing bazel

* rolling back changes

* missed rolling back 1 thing

* adding flag back in

* adding comments back in

* testing lower participation

* adding logs to see web3signer keys for comparison

* fix bazel

* adding more logs temporariliy

* switching hex utility function

* web3signer doesn't support deposits, so changing this

* rolling back unintended unit test

* rolling back some changes for debugging

* Update validator/keymanager/remote-web3signer/v1/web3signer_types.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update testing/endtoend/components/web3remotesigner.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update testing/endtoend/endtoend_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update testing/endtoend/endtoend_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* fixing merge conflict

* Update validator/keymanager/remote-web3signer/v1/requests.go

* Update testing/endtoend/endtoend_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/v1/requests.go

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* prysm doesn't currently support deposits for web3signer

* reverting back to old implementation

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-05-18 15:20:20 +00:00
Mike Neuder
9fab9df61e Refactor validator accounts delete to remove cli context dependency (#10686)
* add functional options accounts delete

* bazel run //:gazelle -- fix

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-05-17 23:13:36 +00:00
Radosław Kapka
eedafac822 SSZ responses for block production (#10697)
* Simplify SSZ handling

* fix tests

* add version to SSZ proto and to GetBeaconStateSSZV2

* TestProduceBlockV2SSZ

* rest of tests

* middleware

* use constants

* some middleware changes

* test fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-17 20:13:06 +00:00
terencechain
e9ad7aeff8 Fix a bad forkchoice proposer boost test (#10705)
* Fix a bad forkchoice proposer boost test

* Improve tests

* Add fork regression

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-05-17 16:26:56 -03:00
terencechain
3cfef20938 Add better debugging logs for validate block p2p pipeline (#10698) 2022-05-17 17:27:52 +00:00
Potuz
e90284bc00 Add Error checks for nil inner state (#10701) 2022-05-17 17:38:35 +08:00
terencechain
01e15a033f Improve beacon node doesn't have a parent in db logs (#10689)
* Improve beacon node doesn't have a parent in db logs

* Update round_robin.go

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-16 18:51:27 +00:00
Radosław Kapka
f09b06d6f6 Allow SSZ-serialized blocks in publishBlindedBlock (#10679)
* SubmitBlockSSZ grpc

* SubmitBlockSSZ middleware

* test fixes

* use VersionedUnmarshaller

* use VersionedUnmarshaller

(cherry picked from commit 7388eeb963)

* tests

* fuzz: Add fuzz tests for sparse merkle trie (#10662)

* Add fuzz tests for sparse merkle trie and change HTR signature to return an error

* fix capitalization of error message

* Add engine timeout values (#10645)

* Add timeout values

* Update engine_client.go

* Update engine_client.go

* Update beacon-chain/powchain/engine_client.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/powchain/engine_client.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/powchain/engine_client.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update engine_client.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Cleanup of `stategen` package (#10607)

* powchain and stategen

* revert powchain changes

* rename field to blockRootsOfSavedStates

* rename params to blockRoot

* review feedback

* fix loop

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Process atts and update head before proposing (#10653)

* Process atts and updeate head

* Fix ctx

* New test and old tests

* Update validator_test.go

* Update validator_test.go

* Update service.go

* Rename to UpdateHead

* Update receive_attestation.go

* Update receive_attestation.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Add link to e2e docs in `README` (#10672)

* Improve `ReceiveBlock`'s comment (#10671)

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Call fcu on invalid payload (#10565)

* Starting

* remove finalized root

* Just call fcu

* Review feedbacks

* fix one test

* Fix conflicts

* Update execution_engine_test.go

* Add a test for invalid recursive call

* Add comprehensive recursive test

* dissallow override empty hash

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Cache and use justified and finalized payload block hash (#10657)

* Cache and use justified and finalized payload block hash

* Fix tests

* Use real byte

* Fix conflicts

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* do not export slotFromBlock

* simplify tests

* grpc

* middleware

* extract package-level consts

* Simplify SSZ handling

* fix tests

* test fixes

* test hack

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-05-16 17:59:43 +00:00
james-prysm
16e66ee1b8 fee-recipient: update error log to warn log (#10684)
* initial commit

* updating fee recipient on beacon node

* updating logs

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-05-16 14:40:13 +00:00
terencechain
3d3890205f Remove invalid terminal block error code (#10646)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-15 03:14:44 +00:00
terencechain
a90335b15e Blockchain: Use get_block to retrieve block (#10688)
* Use get block

* Remove unused errNilParentInDB

* Fix TestVerifyBlkDescendant

* Update beacon-chain/blockchain/service.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-05-15 02:23:01 +00:00
Radosław Kapka
8cd43d216f Changes to SSZ handling (#10687)
* Simplify SSZ handling

* fix tests

* add version to SSZ proto and to GetBeaconStateSSZV2

* remove unwanted methods

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-13 20:29:18 +00:00
Radosław Kapka
d25c0ec1a5 Fix bugs in /eth/v1/beacon/blinded_blocks (#10673)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-05-13 18:20:39 +00:00
Radosław Kapka
e1c4427ea5 Allow SSZ-serialized blocks in publishBlock (#10663)
* SubmitBlockSSZ grpc

* SubmitBlockSSZ middleware

* test fixes

* use VersionedUnmarshaller

(cherry picked from commit 7388eeb963)

* tests

* do not export slotFromBlock

* simplify tests

* extract package-level consts

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-13 10:54:45 +00:00
Preston Van Loon
7042791e31 fuzz: Fix off by one error in sparse merkle trie item insertion (#10668)
* fuzz: Fix off by one error in sparse merkle trie item insertion

* remove new line

* Move validation to the proto constructor

* fix build

* Add a unit test because @nisdas is going to ask for it

* fix up

* gaz

* Update container/trie/sparse_merkle.go

* Update container/trie/sparse_merkle_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
2022-05-13 07:02:43 +00:00
terencechain
e771585b77 Fix error.Is ordering (#10685)
* Fix error.is ordering for ErrUndefinedExecutionEngineError

* Update BUILD.bazel
2022-05-12 22:43:34 +00:00
Radosław Kapka
98622a052f Extract OptimisticSyncFetcher interface (#10654)
* Extract `OptimisticSyncFetcher` interface

* extract IsOptimistic

* fix tests

* more test fixes

* even more test fixes

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-12 17:23:45 +00:00
Potuz
61033ebea1 handle failure to update head (#10651)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-12 16:36:46 +00:00
Potuz
e808025b17 regression test off-by-one (#10675)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-12 15:49:37 +00:00
Radosław Kapka
7db0435ee0 Unify WARNING comments (#10678)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-12 15:25:44 +00:00
Nishant Das
1f086e4333 add more fuzz targets (#10682) 2022-05-12 10:47:29 -04:00
james-prysm
184e5be9de Fee recipient: checksum log (#10664)
* adding checksum check at validator client and beacon node

* adding validation and logs on validator client startup

* moving the log and validation

* fixing unit tests

* adding test for back checksum on validator client

* fixing bazel

* addressing comments

* fixing log display

* Update beacon-chain/node/config.go

* Apply suggestions from code review

* breaking up lines

* fixing unit test

* ugh another fix to unit test

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-11 19:36:57 +00:00
terencechain
e33850bf51 Don't return nil with new head (#10680) 2022-05-11 11:33:10 -07:00
Preston Van Loon
cc643ac4cc native-state: Simplify MarshalSSZ (#10677)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-11 14:25:11 +00:00
Nishant Das
abefe1e9d5 Add Sync Block Topic Fuzz Target (#10676)
* add new target

* remove

* Update beacon-chain/sync/BUILD.bazel
2022-05-11 13:18:12 +00:00
Nishant Das
b4e89fb28b Add in State Fuzzing (#10375)
* add it in

* build tags

* gaz

* remove dependency on fast-ssz for now

* copy and comment

* add in native state

* fix build

* no assert on invalid input, just return

* Add failing case

* fix it up

Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
2022-05-11 12:47:54 +00:00
terencechain
4ad1c4df01 Cache and use justified and finalized payload block hash (#10657)
* Cache and use justified and finalized payload block hash

* Fix tests

* Use real byte

* Fix conflicts

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-10 21:20:28 +00:00
terencechain
6a197b47d9 Call fcu on invalid payload (#10565)
* Starting

* remove finalized root

* Just call fcu

* Review feedbacks

* fix one test

* Fix conflicts

* Update execution_engine_test.go

* Add a test for invalid recursive call

* Add comprehensive recursive test

* dissallow override empty hash

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-10 20:02:00 +00:00
Radosław Kapka
d102421a25 Improve ReceiveBlock's comment (#10671)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-10 19:13:28 +00:00
Radosław Kapka
7b1490429c Add link to e2e docs in README (#10672) 2022-05-10 14:16:40 +00:00
terencechain
bbdf19cfd0 Process atts and update head before proposing (#10653)
* Process atts and updeate head

* Fix ctx

* New test and old tests

* Update validator_test.go

* Update validator_test.go

* Update service.go

* Rename to UpdateHead

* Update receive_attestation.go

* Update receive_attestation.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-09 23:12:50 +00:00
Radosław Kapka
5d94030b4f Cleanup of stategen package (#10607)
* powchain and stategen

* revert powchain changes

* rename field to blockRootsOfSavedStates

* rename params to blockRoot

* review feedback

* fix loop

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-09 22:25:47 +00:00
terencechain
16273a2040 Add engine timeout values (#10645)
* Add timeout values

* Update engine_client.go

* Update engine_client.go

* Update beacon-chain/powchain/engine_client.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/powchain/engine_client.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/powchain/engine_client.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update engine_client.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-05-09 20:02:20 +00:00
Preston Van Loon
97663548a1 fuzz: Add fuzz tests for sparse merkle trie (#10662)
* Add fuzz tests for sparse merkle trie and change HTR signature to return an error

* fix capitalization of error message
2022-05-09 16:51:48 +00:00
Radosław Kapka
7d9d8454b1 Improvements to powchain package (#10652)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-09 13:57:34 +00:00
Radosław Kapka
21bdbd548a Deduplicate native state (a.k.a. One State to rule them all) (#10483)
* v0

* getters/setters

* init and copy

* hasher

* all the nice stuff

* make bazel happy

* remove tests for smaller PR

* remove old states

* move files

* import fixes

* custom MarshalSSZ

* fixed deadlock

* copy version when copying state

* correct issues in state_trie

* fix Copy()

* better e2e comment

* add code to minimal state

* spectest test

* Revert "Auxiliary commit to revert individual files from 84154423464e8372f7e0a03367403656ac5cd78e"

This reverts commit 9602599d183081291dfa0ba4f1036430f63a7822.

* native state assert

* always error

* always log

* more native state usage

* cleanup

* remove empty line

* Revert "spectests"

This reverts commit 1c49bed5d1cf6224afaf21e18562bf72fae5d2b6.

# Conflicts:
#	beacon-chain/powchain/service.go
#	beacon-chain/state/v1/state_trie.go
#	beacon-chain/state/v2/state_trie.go
#	beacon-chain/state/v3/state_trie.go
#	testing/spectest/shared/phase0/finality/BUILD.bazel
#	testing/spectest/shared/phase0/finality/runner.go

* dedup field trie

* fix test issues

* cleanup

* use correct field num in FinalizedRootProof

* use existing version constant

* halfway there

* "working" version

* some fixes

* fix field nums in tests

* rename v0types to nativetypes

* Revert "Auxiliary commit to revert individual files from dc549b1cf8e724bd08cee1ecc760ff3771d5592d"

This reverts commit 7254d3070d8693b283fc686a2e01a822ecbac1b3.

* uncomment code

* remove map size

* Revert "Revert "spectests""

This reverts commit 39c271ae6b.

* use reverse map

* Revert "Revert "Revert "spectests"""

This reverts commit 19ba8cf95c.

* finally found the bug

(cherry picked from commit a5414c4be1bdb61a50b391ea5301895e772cc5e9)

* simplify populateFieldIndexes

* fix copy

(cherry picked from commit 7da4fb8cf51557ef931bb781872ea52fc6731af5)

* remove native state from e2e

* remove index map

* unsupported functions

* Use ProtobufBeaconState() from native state

* tests

* typo

* reduce complexity of `SaveStatesEfficient`

* remove unused receiver name

* update doc.go

* fix test assertion

* fix test assertion 2

* Phase0 justification bits

* bring back state tests

* rename fieldIndexRev

* versioning of ToProto

* remove version check from unexported function

* hasher tests

* don't return error from JustificationBits

* extract fieldConvertersNative

* helper error function

* use fieldConvertersNative

* Introduce RealPosition method on FieldIndex

* use RealPosition in hasher

* remove unused fields

* remove TestAppendBeyondIndicesLimit

(cherry picked from commit 3017e700282969c30006b64c95c21ffe6b166f8b)

* simplify RealPosition

* rename field interface

* use helper in proofs.go

* Update beacon-chain/core/altair/upgrade.go

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-05-09 13:02:34 +00:00
Nishant Das
e2caaf972f Fix Multiclient E2E Runs (#10660) 2022-05-09 12:38:11 +02:00
Nishant Das
c5ddc266ae Perform Request Size Limiting When Caching Eth1 Headers (#10649)
* rate limit reqs

* fix issues

* make it better

* final check

* terence's review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-09 03:19:23 +00:00
Nishant Das
74518f0e1b Log Out Missing Validators in Participation Drops (#10624)
* debug failures

* clean up

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-05-09 01:44:47 +00:00
terencechain
122c3f44cc Insert over-the-wire slashing to forkchoice (#10615)
* remove equivocating votes from forkchoice

* shutup deepsource

* Add insertSlashingsToForkChoiceStore

* Fix equality checks, add tests

* Update process_block.go

* Fix equality check for doublylinked

* More checks

* Rm err return

* Consider slashings over the wire

* Update new_slot_test.go

* Fix spec tests

* Add ReceiveAttesterSlashing

* Update mock.go

* Revert spec test changes

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-05-07 15:56:34 +00:00
kasey
b1b13cfca7 simplify config names, use strings (#10656)
* simplify config names, use strings

* lint

Co-authored-by: kasey <kasey@users.noreply.github.com>
2022-05-06 21:42:27 +00:00
Preston Van Loon
495013e832 Update github actions to go 1.18 (#10655) 2022-05-06 17:50:52 +00:00
kasey
ae82c17dc3 run process_slots before caching committee data (#10639)
* demonstrate issue with committee caching in e2e

* lint

* run process_slots before caching committee data

* pass transitioned state to UpdateCommitteeCache

* don't run extra epochs w/ web3signer

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2022-05-06 14:33:09 +08:00
Potuz
8bd1f16223 Remove unused method (#10643) 2022-05-05 21:46:28 +02:00
kasey
58b18c7996 Fix URL handling in beacon node api client (#10632)
* if url.Parse suceeds, don't mess with node urls

* lint

* add test case for basic auth

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2022-05-05 16:03:31 +00:00
terencechain
dda48c452d Move head change attestation log (#10636) 2022-05-05 15:09:58 +00:00
Potuz
025d053fa4 Change log when head changes (#10633) 2022-05-05 11:40:21 +00:00
terencechain
7af89a82c9 Use justified hash as safe block hash (#10627)
* Use justified hash as safe block hash

* Remove debug log

* Feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-05 00:36:09 +00:00
kasey
10e1f04ce4 Checkpoint sync fixes (#10630)
* fixes for checkpoint sync

* improve node version regex; real client test cases

* add test case for checkpoint sync finalization fix

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2022-05-04 23:41:33 +00:00
Preston Van Loon
5817090b59 Remove junk metadata file (#10626) 2022-05-04 17:26:01 +00:00
Potuz
df695346a5 process slashings once (#10621)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-04 14:51:18 +00:00
Nishant Das
f9b4a340a3 Graduate Orphaned Attestation Insertion Feature (#10622)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-04 13:59:41 +00:00
Preston Van Loon
d2dbc13427 fuzz: Fix tests with fuzz tags (#10619)
* some fixes to fuzz building

* use len

* remove comments

* fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-04 06:15:09 +00:00
Nishant Das
e5e4dee629 Update Batch Verification Routine (#10127)
* add changes so far

* a lot of tests

* add flags and comments

* raul's comment

* raul's review

* fix build

* fix

* fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-04 04:47:53 +00:00
Raul Jordan
ab2b0c5c99 Use Single Beacon Block Wrapper Function Across Prysm (#10608)
* remove altair wrappers in favor of generic ones for blocks

* rem deprecated

* body wrapper

* builds

* nil check
2022-05-04 11:55:35 +08:00
terencechain
7c3147ca89 Fix isProcessedBlock order of operation (#10620) 2022-05-04 01:28:08 +00:00
terencechain
03ae8672b6 Refactor and use has_block getter (#10592)
* Refactor and use has block getter

* Update blocks_fetcher_utils.go

* Fix tests

* Fix tests

* Fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-04 00:17:40 +00:00
Preston Van Loon
f763b35494 fuzz: build with -tags=fuzz,blst_disabled (#10618) 2022-05-03 22:19:05 +00:00
Preston Van Loon
51581e9b6e Spectest: refactor forkchoice helper (#10616)
* Seperate forkchoice builder for lower cognitive score and better reusablity

* gaz

* deepsource resolutions

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-03 20:59:07 +00:00
terencechain
e5ee7f7e8b Don't mark undefined ee error block bad for regular processing (#10613)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-03 20:12:58 +00:00
Nishant Das
d0fabd86fb Fix Flaky Sync Committee Evaluator (#10600)
Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-03 19:26:59 +00:00
Raul Jordan
5315c45453 More Useful Log When Auth Fails to Execution Client (#10609)
* more helpful auth message when auth to engine does not work

* better language

* clarity

* check for 401

* fix panic

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-03 18:40:08 +00:00
terencechain
55883876e4 Update consensus spec version to v1.1.10 (#10614) 2022-05-03 17:54:08 +00:00
Radosław Kapka
1d587c0e95 No timeout for API events (#10611)
* no event timeout

* Revert "Auxiliary commit to revert individual files from 1bfe8327ed02104432cc6bde09ac6afc8fb21f62"

This reverts commit 5f7aa10e5bc0bc18a414efdf425ae90cbcf08b99.

* proper solution

* comment fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-03 17:42:06 +00:00
terencechain
d4fa490dec Handle blind block for DB (#10580)
* Handle blind block for DB

* Update blinded_beacon_block_bellatrix_test.go

* Update blocks_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-03 16:55:59 +00:00
Raul Jordan
59041cf868 Warn Users if EE Returns Unexpected Fee Recipient Address (#10604)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-03 15:18:42 +00:00
terencechain
8f1ee146a9 Use notify engine if changed head method (#10606)
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-05-03 14:31:31 +00:00
Radosław Kapka
a5f1dcaa65 Revert "Simplify Initial Sync (#10523)" (#10610)
* Revert "Simplify Initial Sync (#10523)"

This reverts commit 3b69f7a196.

* comment
2022-05-03 11:39:53 +00:00
terencechain
9615484fe9 notifyForkchoiceUpdate can prune invalid blocks (#10510)
* notifyForkchoiceUpdate can prune invalid blocks

* define and use ErrInvalidPayload

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-03 03:48:58 +00:00
terencechain
5ad4f14ffc Use remove equivocating validators (#10603)
* remove equivocating votes from forkchoice

* shutup deepsource

* Add insertSlashingsToForkChoiceStore

* Fix equality checks, add tests

* Update process_block.go

* Fix equality check for doublylinked

* More checks

* Rm err return

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-05-03 00:46:05 +00:00
Potuz
4c869fa587 remove equivocating votes from forkchoice (#10597)
* remove equivocating votes from forkchoice

* shutup deepsource

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-02 19:19:03 +00:00
Raul Jordan
20ab988a4a Rename Block Package in Consensus-Types to Interfaces (#10605)
* interfaces package

* builds

* gaz
2022-05-02 14:32:37 -04:00
Raul Jordan
16bbf5602f Move Consensus Type Wrappers Into Consensus Types Package (#10598)
* builds

* move block to consensus-types

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-02 15:43:40 +00:00
Potuz
f9113dfd06 Forkchoice use parent hash when removing invalid blocks (#10594) 2022-05-02 06:53:22 -07:00
terencechain
0faf7ac01f Debug log when new head is not in db (#10591) 2022-04-29 14:26:39 -07:00
Preston Van Loon
9d2bdfe14d p2p: Fix pubsub fuzz test (#10583)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-04-29 18:20:18 +00:00
Nishant Das
ad03938964 Fix Doppelganger Check (#10582)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-04-29 17:19:43 +00:00
Raul Jordan
84916672c6 Remove Eth2-Types Dependency in Prysm (#10578)
* replace eth2 types

* replace protos

* regen proto

* replace

* gaz

* deps

* amend

* regen proto

* mod

* gaz

* gaz

* ensure build

* ssz

* add dep

* no more eth2 types

* no more eth2

* remg

* all builds

* buidl

* tidy

* clean

* fmt

* val serv

* gaz

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-04-29 10:32:11 -04:00
Nishant Das
2f29bb64f6 Fallback To Rebuilding Deposit Tries (#10562)
* fallback

* fix it

* make it clearer

* terence's review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-04-29 09:33:59 +00:00
terencechain
156e40e886 Clean up optimistic candidate block usages (#10579)
* Clean up optimistic candidate block usages

* More clean ups

* Update godoc comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-04-29 08:43:51 +00:00
terencechain
b7a82d0fd1 Can retrieve cached initial sync block and db block (#10568)
* Save cached initial sync blocks before getting head block

* Add better abstraction to get block

* Move unlock read to a better location

* Feedbacks

* Add head changed logging

* Harder hasBlock requirement

* Update beacon-chain/blockchain/service.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update receive_attestation.go

* Don't process head if the block is unknown

* Use a helper method

* Fix test

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-04-29 07:56:31 +00:00
Raul Jordan
61bfb77120 Max Epoch Helper from Eth2-Types (#10581)
* max epoch helper from eth2 types

* gaz

* godoc

* Update consensus-types/primitives/epoch_test.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-04-29 02:16:02 +00:00
Raul Jordan
7b03d901ee Fix Nogo Lint Ignore for Go 1.18 (#10577)
* builds

* builds

* fatal

* edit
2022-04-28 18:10:43 +00:00
Raul Jordan
7a3df7642b Update Prysm to Go 1.18 (#10576)
* update go

* Update rules_go patch

* Update gazelle

* begin building

* qtls

* gaz

Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
2022-04-28 13:07:29 -04:00
Mike Neuder
231208c977 Refactor validator accounts list to remove cli context dependency (#10554)
* accounts list cleanup

* go.sum

* go mod tidy -compat=1.17

* add comment back

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-04-28 14:46:46 +00:00
Raul Jordan
001f719cc3 Move ETH2 Types Into Prysm (#10534)
* move eth2 types into Prysm

* bazel

* lint

* use existing math helpers

* rem eth2-types dep

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-04-28 13:57:40 +00:00
Nishant Das
58ad800553 Cleanup Discarded Connections Correctly (#10574)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-04-28 13:09:03 +00:00
Radosław Kapka
2060f876b1 Code cleanup in blockchain module (#10566)
* Code cleanup in `blockchain` module

* revert comment fix

* remove test

* forkchoice test fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-04-28 11:34:25 +00:00
james-prysm
e226237590 Fee Recipient: improve logs (#10571)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-04-28 10:48:05 +00:00
Preston Van Loon
314ef8e1bd Add support for blst modern builds on linux amd64 (#10567)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-04-28 10:02:02 +00:00
Nishant Das
a428664eb5 Fix Broken E2E Runs (#10572)
* fix broken e2e

* hack but works

* make it super ugly

* fix

* fix again

* revert it

* dont make everything exclusive

* gaz
2022-04-28 08:45:59 +00:00
Nishant Das
7c3d89b25f Graduate Batch Gossip Verification Feature (#10553)
* grad feat

* fix stuck tests
2022-04-26 08:28:35 -04:00
Radosław Kapka
f4c004085b Add lock around reading eth1data (#10557)
* Add lock around reading eth1data

* fix lock pattern

* more locking

* move lock inside loop

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-04-26 06:19:14 +00:00
terence tsao
c0e207eb47 Add proposer index and graffiti to received block log (#10564)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-04-25 20:51:10 +00:00
terence tsao
7003d97061 Don't mark block as bad with unknown ee error (#10556)
* Don't mark block as bad with unknown ee error

* Some tests

* Comments

* Comments
2022-04-25 18:41:35 +00:00
Radosław Kapka
957d853a2f Blinded block APIs (#10331) 2022-04-25 17:42:58 +02:00
james-prysm
29695d8906 Fee Recipient: API/logs run only after bellatrix (#10543)
* initial commit

* unit test

* Update validator/client/validator.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* updating check based on review comments

* adjusting how logs are used

* adding in git add .

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-04-25 13:34:20 +00:00
Nishant Das
a8ba5e7dd8 Disable Transaction Evaluator (#10563)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-04-25 10:13:11 +00:00
Nishant Das
0ad190c1fb Graduate Balance Trie Feature (#10552) 2022-04-25 11:16:41 +02:00
Raul Jordan
2e056b38da Engine API Proxy Utility for Merge Testing (#10533)
* execution client package renaming

* define interceptors and options

* further clean

* further dedup

* further simplify

* rev

* rem

* more modifications

* further clean

* defines tests

* pass first test

* proper tests

* all tests in

* gaz

* lint

* wait start

* assign on ports

* gaz

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-04-25 05:03:05 +00:00
james-prysm
966de59478 E2E: switching to run time generated yaml for web3signer (#10513)
* switching to run time generated yaml

* fixing bazel

* fixing unit test

* fixing linter problems

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-04-23 08:41:24 +00:00
terence tsao
c8919bd233 Move validate merge transition block outside of notify new payload (#10526)
* Move validate merge transition block

* Update process_block_test.go

* Conflict, use swith err

* Update execution_engine.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-04-23 08:11:15 +00:00
Nishant Das
baa2e2e340 Fix Transaction Pool in E2E (#10561) 2022-04-22 21:13:11 +00:00
Nishant Das
7765d275d1 Add in State Slot for Sync Committee Cache (#10475)
* add changes

* Update beacon-chain/core/helpers/sync_committee_test.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-04-22 08:03:32 +00:00
Håvard Anda Estensen
2fd7c926ed Use the T.TempDir and B.TempDir to create temp dirs for testing (#10560)
* Use t.TempDir and b.TempDir for temporary test dirs

* Remove redundant dir cleanup
2022-04-21 20:45:44 +00:00
moshe-blox
21fc9853ee Encode empty extra_data as "0x" and base_fee_per_gas as Uint256 i… (#10539)
* Encode empty `extra_data` as "0x" and `base_fee_per_gas` as Uint256 in the Beacon API

* Fixed parsing of Uint256 in the API

* Add missing tests for empty hex & Uint256 fields in `apimiddleware`

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-04-21 15:44:38 +00:00
Radosław Kapka
642de455d5 Update Beacon API's Postman collection to 2.2.0 (#10559)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-04-21 15:07:58 +00:00
Radosław Kapka
d371cd6e89 Add lock to PeerStatusScorer.SetHeadSlot (#10558)
(cherry picked from commit e04e4795c81ea939cf8996c632c52598cd69d170)
2022-04-21 15:02:20 +00:00
james-prysm
c8a7f6f0bd Suggested Fee Recipient : bug fix (#10555)
* initial commit

* adding unit tests

* Update validator/client/validator.go

Co-authored-by: Nishant Das <nishdas93@gmail.com>

* fixing unit tests based on suggestions

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-04-21 03:57:19 +00:00
Nishant Das
d7a7fa025d Graduate Active Balance Cache Feature (#10550)
* graduate feature

* gaz

* clear cache
2022-04-21 02:59:52 +00:00
Nishant Das
6b6ac4c3fb Graduate Optimized Balance Update Feature (#10549)
* graduate feat

* stop imports

* gaz
2022-04-20 09:02:33 +00:00
Nishant Das
3b8651cbf4 Graduate Optimized Get Block Feature (#10548) 2022-04-19 22:30:29 -07:00
Preston Van Loon
9d29d2f4bf Update libp2p to latest release (#10423)
* Update libp2p to latest release

* gazelle

* fix panic in tests

* gaz

* fix conflicts

* revert back

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
2022-04-19 12:53:58 +00:00
Nishant Das
3d205a387c Remove Proposer Selection Feature (#10547)
* remove config option

* fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-04-19 08:59:19 +00:00
Nishant Das
269b382229 Fix E2E Failures (#10540)
* fix it

* fix all issues

* gaz

* fix yaml

* Update config/params/testdata/e2e_config.yaml

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update testing/endtoend/minimal_e2e_test.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-04-19 07:53:02 +00:00
Preston Van Loon
80ebbcf03e gocognit: Lower complexity threshold to 100, fix a few complexity issues (#10542)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-04-19 07:12:24 +00:00
terence tsao
32ebe94515 Clean up batch process block (#10520) 2022-04-19 06:44:25 +00:00
Håvard Anda Estensen
d2f4a8cc7c Replace ioutil with io and os (#10541)
* Replace ioutil with io and os

* Fix build errors
2022-04-18 20:42:07 +00:00
Giulio rebuffo
982de94428 misleading comment in receive_block.go (#10538)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-04-18 12:15:28 +00:00
terence tsao
a41025e5ec Rename optimistic_sync.go to execution_engine.go (#10537)
* Rename optimistic_sync.go to execution_engine.go

* Update BUILD.bazel
2022-04-18 05:49:48 +00:00
terence tsao
3ce26ae7e2 Add process payload header (#10529)
* Add process payload header

* Update block.go

* Remove unused receivers

* Use errors vars

* Add some comments

* Add some comments
2022-04-17 07:02:59 +00:00
terence tsao
1acedd5b01 Rename receipt root to receipts root (#10535) 2022-04-16 13:18:07 +00:00
Radosław Kapka
a0679c70d3 Service constructors and Start() - better separation of concerns (#10532)
* move waitForStateInitialization to Start

* remove channel

* handle error in test

* fix service tests

* use fatal log

* deterministic-genesis

* sync

* rpc

* monitor

* validator-client

* test fixes

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-04-16 02:45:35 +00:00
Leo Lara
7f53700306 Remove feature and flag Pyrmont testnet (#10522)
* Remove feature and flag Pyrmont testnet

* Remove unused parameter

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-04-15 15:30:32 +00:00
Radosław Kapka
3b69f7a196 Simplify Initial Sync (#10523)
* move waitForStateInitialization to Start

* remove channel

* handle error in test

* fix service tests

* use fatal log

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-04-15 13:32:31 +00:00
Nishant Das
72562dcf3a Fix Another Off By 1 In Our Finalized Trie (#10524)
* fix everything

* fix more

* fix test

* Update beacon-chain/powchain/service.go

* Update beacon-chain/powchain/service.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-04-15 12:51:56 +00:00
Preston Van Loon
cc23b8311a Static analysis: gocognit (#10527)
* Add gocognit to static analyzers with a very high threshold

* edit readme and sort analyzers
2022-04-15 06:29:07 +00:00
terence tsao
cbe54fe3f9 Handle nil execution block response when logging TTD (#10502) 2022-04-13 19:47:04 -07:00
Nishant Das
1b6adca3ca Handle Finalized Deposit Insertion Better (#10517) 2022-04-13 10:08:19 +02:00
Nishant Das
1651649e5a Update to 1.17.9 (#10518) 2022-04-13 05:25:13 +00:00
Potuz
56187edb98 Use forkchoice first when checking canonical status (#10516) 2022-04-12 21:34:13 -03:00
Preston Van Loon
ecad5bbffc e2e: Provide e2e config yaml to web3signer (#10123)
* e2e: Provide e2e config yaml to web3signer

* fix build for //testing/endtoend:go_default_test

* Update with web3signer with teku fixes

* buildifier

* Add slasher case for web3signer

* Update testing/endtoend/minimal_e2e_test.go

* Update web3signer to 21.10.6

* Revert "Update web3signer to 21.10.6"

This reverts commit bdf3c408f2.

* Remove slasher part of web3signer e2e tests

* Revert "Remove slasher part of web3signer e2e tests"

This reverts commit 24249802ae.

* fix slasher web3signer test

* fixing build

* updating yaml to match testnet_e2e_config.go

* trying a different order to the e2e test and adding a log

* trying different way to kill process

* handling unhandled error

* testing changes to config WIP

* fixing bazel WIP

* fixing build

* ignoring test for now to test

* fixing bazel

* Test only web3signer e2e

* rolling back some commits to test

* fixing bazel

* trying an updated web3signer version

* changing flag to match version

* trying current version of develop for web3signer

* testing not using the --network property

* addressing build error

* testing config change

* reverting to go bakc to using the network file

* testing adding epochs per sync committee period

* rolling back configs

* removing check to test

* adding log to get sync committee duties and changing bellatrix fork epoch to something large and altair to epoch 1

* fixing bazel

* updating epoch in config file

* fixing more descrepencies between the configurations

* removing un used yaml

* removing goland added duplicates

* reverting using network minimal

* fixing bug

* rolling back some changes

* rolling back some changes

* rolling back changes

* making sure web3signer test doesn't make it to bellatrix fork yet

* reverting changes I did not touch

* undo comment

* Update testing/endtoend/deps.bzl

* Apply suggestions from code review

* rm nl

* fix //testing/endtoend:go_mainnet_test

* Remove unnecessary dep

* fix //testing/endtoend:go_mainnet_test

* addressing review comments

* fixing build and internal conflict

* removing web3signer slasher test as it's unneeded due to the interface nature of key signing, the regular slashing test is enough

* fix: The validator we fetch from the binary can only run before altair, if you add that and the web3signer then these things can never run together as the web3signer sets it to before bellatrix

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: James He <james@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-04-12 16:57:46 +00:00
Nishant Das
407182387b fix it all (#10511) 2022-04-12 19:50:29 +08:00
terence tsao
ad0b0b503d Move GetTerminalBlockHash to powchain engine (#10500)
* Move GetTerminalBlockHash to powchain engine

* Update service.go

* Update service.go

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-04-12 10:19:07 +00:00
terence tsao
58f4ba758c Metrics tracking EE VALID/SYNCING/INVALID response counter (#10504)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-04-12 09:51:13 +00:00
james-prysm
64f64f06bf Remote Key Manager API(web3signer) (#10302)
* removing flag requirement, can run web3signer without predefined public keys

* placeholders for remote-keymanager-api

* adding proto and accountschangedfeed

* updating generated code

* fix imports

* fixing interface

* adding work in progress apimiddleware code

* started implementing functions for remote keymanager api

* fixing generted code from proto

* fixing protos

* fixing import format

* fixing proto generation again , didn't fix the first time

* fixing imports again

* continuing on implementing functions

* implementing add function

* implementing delete API function

* handling errors for API

* removing unusedcode and fixing format

* fixing bazel

* wip enable --web when running web3signer

* fixing wallet check for web3signer

* fixing apis

* adding list remote keys unit test

* import remote keys test

* delete pubkeys tests

* moving location of tests

* adding unit tests

* adding placeholder functions

* adding more unit tests

* fixing bazel

* fixing build

* fixing already slice issue with unit test

* fixing linting

* Update validator/client/validator.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/types.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/node/node.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/types.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/validator.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* adding comment on proto based on review

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/rpc/standard_api.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/rpc/standard_api.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/rpc/standard_api.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/rpc/standard_api.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* adding generated code based on review

* updating based on feedback

* fixing imports

* fixing formatting

* Update validator/rpc/standard_api.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* fixing event call

* fixing dependency

* updating bazel

* Update validator/rpc/standard_api.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/rpc/standard_api.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/rpc/standard_api.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/rpc/standard_api.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* addressing comment from review

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-04-11 16:05:40 -04:00
terence tsao
e70055733f Save state to DB after proposer boost (#10509) 2022-04-11 16:51:49 +00:00
Radosław Kapka
36e4f49af0 Bellatrix evaluators (#10506)
* defensive nil check

* separate ExecutionPayload/Header from codegen

* tell bazel about this new file

* Merge: support terminal difficulty override (#9769)

* Fix finding terminal block hash calculation

* Update mainnet_config.go

* Update beacon_block.pb.go

* Various fixes to pass all spec tests for Merge (#9777)

* Proper upgrade altair to merge state

* Use uint64 for ttd

* Correctly upgrade to merge state + object mapping fixes

* Use proper receive block path for initial syncing

* Disable contract lookback

* Disable deposit contract lookback

* Go fmt

* Merge: switch from go bindings to raw rpc calls (#9803)

* Disable genesis ETH1.0 chain header logging

* Update htrutils.go

* all gossip tests passing

* Remove gas validations

* Update penalty params for Merge

* Fix gossip and tx size limits for the merge part 1

* Remove extraneous p2p condition

* Add and use

* Add and use TBH_ACTIVATION_EPOCH

* Update WORKSPACE

* Update Kintsugi engine API (#9865)

* Kintsugi ssz (#9867)

* All spec tests pass

* Update spec test shas

* Update Kintsugi consensus implementations (#9872)

* Remove secp256k1

* Remove unused merge genesis state gen tool

* Manually override nil transaction field. M2 works

* Fix bad hex conversion

* Change Gossip message size and Chunk SIze from 1 MB t0 10MB (#9860)

* change gossip size and chunk size after merge

* change ssz to accomodate both changes

* gofmt config file

* add testcase for merge MsgId

* Update beacon-chain/p2p/message_id.go

Change MB to Mib in comment

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* change function name from altairMsgID to postAltairMsgID

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Sync with develop

* Merge branch 'develop' of github.com:prysmaticlabs/prysm into kintsugi

* Update state_trie.go

* Clean up conflicts

* Fix build

* Update config to devnet1

* Fix state merge

* Handle merge test case for update balance

* Fix build

* State pkg cleanup

* Fix a bug with loading mainnet state

* Fix transactions root

* Add v2 endpoint for merge blocks (#9802)

* Add V2 blocks endpoint for merge blocks

* Update beacon-chain/rpc/apimiddleware/structs.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* go mod

* fix transactions

* Terence's comments

* add missing file

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Sync

* Go mod tidy

* change EP field names

* latest kintusgi execution api

* fix conflicts

* converting base fee to big endian format (#10018)

* ReverseByteOrder function does not mess the input

* sync with develop

* use merge gossip sizes

* correct gossip sizes this time

* visibility

* clean ups

* Sync with develop, fix payload nil check bug

* Speed up syncing, hide cosmetic errors

* Sync with develop

* Clean up after sync

* Update generate_keys.go

* sync with develop

* Update mainnet_config.go

* Clean ups

* Sync optimistically candidate blocks (#10193)

* Revert "Sync optimistically candidate blocks (#10193)"

This reverts commit f99a0419ef.

* Sync optimistically candidate blocks (#10193)

* allow optimistic sync

* Fix merge transition block validation

* Update proposer.go

* Sync with develop

* delete deprecated client, update testnet flag

* Change optimistic logic (#10194)

* Logs and err handling

* Fix build

* Clean ups

* Add back get payload

* c

* Done

* Rm uncommented

* Optimistic sync: prysm validator rpcs (#10200)

* Logs to reproduce

* Use pointers

* Use pointers

* Use pointers

* Update json_marshal_unmarshal.go

* Fix marshal

* Update json_marshal_unmarshal.go

* Log

* string total diff

* str

* marshal un

* set string

* json

* gaz

* Comment out optimistic status

* remove kiln flag here (#10269)

* Sync with devleop

* Sync with develop

* clean ups

* refactor engine calls

* Update process_block.go

* Fix deadlock, uncomment duty opt sync

* Update proposer_execution_payload.go

* Sync with develop

* Rm post state check

* Bypass eth1 data checks

* Update proposer_execution_payload.go

* Return early if ttd is not reached

* Sync with devleop

* Update process_block.go

* Update receive_block.go

* Update bzl

* Revert "Update receive_block.go"

This reverts commit 5b4a87c512.

* Fix run time

* add in all the fixes

* fix evaluator bugs

* latest fixes

* sum

* fix to be configurable

* Update go.mod

* Fix AltairCompatible to account for future state version

* Update proposer_execution_payload.go

* fix broken conditional checks

* fix all issues

* Handle pre state Altair with valid payload

* Handle pre state Altair with valid payload

* Log bellatrix fields

* Update log.go

* Revert "fix broken conditional checks"

This reverts commit e118db6c20.

* LH multiclient working

* Friendly fee recipient log

* Remove extra SetOptimisticToValid

* fix race

* fix test

* Fix base fee per gas

* Fix notifypayload headroot

* tx fuzzer

* clean up with develop branch

* save progress

* 200tx/block

* add LH flags

* Sync with devleop

* cleanup

* cleanup

* hash

* fix build

* fix test

* fix go check

* fmt

* gosec

* Blocked stream

(cherry picked from commit f362af9862db680b6352692217ad5c08d44a1e86)

# Conflicts:
#	proto/prysm/v1alpha1/validator.pb.go

* remove duplicate param

* test

* revert some test changes

* Initial version of EE tx count

* evaluate all txs in epoch

* remove logs

* uncomment tests

* remove unwanted change

* parameterize ExpectedExecEngineTxsThreshold

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Zahoor Mohamed <zahoor@zahoor.in>
Co-authored-by: kasey <489222+kasey@users.noreply.github.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: Zahoor Mohamed <zahoor@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-04-11 13:45:22 +00:00
terence tsao
d98428dec4 Can prune nodes from canonical and payload maps (#10496)
* Can prune nodes from canonical and payload maps

* Update store_test.go

* prune payload hashes, canonical nodes and better testing

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-04-11 11:08:50 +00:00
Nishant Das
00b92e01d3 Fetch Non Finalized Deposits Better (#10505) 2022-04-11 09:59:22 +02:00
Potuz
ca5adbf7e4 Fix optimistic logging (#10503) 2022-04-09 23:09:04 +00:00
Nishant Das
a083b7a0a5 Set Auth Differently In Our Powchain Constructor (#10501) 2022-04-09 11:03:05 +02:00
Raul Jordan
dd5995b665 Proper Connection Management for ETH JSON-RPC Client for Startup and Runtime (#10498)
* begin connection management revamp

* kiln runs

* retry

* reconnect

* add

* rpc connect fix

* remove logging

* logs

* retry

* default value for web3flag

* test pass

* comments

* ensure auth works
2022-04-09 09:28:40 +08:00
Taranpreet26311
6903d52dde Added horusec (#10499)
* Added horusec

* Improving aesthetics

* Restrict branch
2022-04-08 10:52:04 -04:00
Potuz
ac8d27bcf1 Deal with node balance underflow (#10492)
* deal with balance underflow

* fix log format

* preston's review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-04-07 17:38:48 +00:00
Nishant Das
8d6afb3afd Fix Issues From Multiclient E2E (#10486)
* fix it

* fix index addition

* fix sync issues

* make it nicer
2022-04-07 01:52:24 +00:00
james-prysm
d51f716675 web3signer: fixes json altair block (#10490)
* fix

* fixing naming
2022-04-07 08:24:30 +08:00
terence tsao
0411b7eceb Payload ID caching (#10481) 2022-04-06 16:36:52 -07:00
Radosław Kapka
8dfc80187d Fix receipts root field name in API Middleware (#10488)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-04-06 21:56:11 +00:00
terence tsao
3833f78803 Remove invalid nodes (#10399) 2022-04-06 14:24:00 -07:00
Potuz
83a83279d4 Remove synced tips and use last valid hash (#10439)
* Remove synced tips

use last valid hash in removing invalid nodes.

* add test

* Remove unused code

* More unused parameters

* Fix proposer boost

* terence's review #1

* Fix conflicts

* terence's review 2

* rename argument

* terence's review #3

* rename optimistic -> status

* Minor clean up

* revert loop variable change

* do not mark lvh as valid

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-04-06 14:18:30 +00:00
terence tsao
bdab34fd01 Use correct head for notifyForkchoiceUpdate (#10485)
* Use correct head for notifyForkchoiceUpdate

* Fix existing tests

* Update receive_attestation_test.go
2022-04-06 19:05:53 +08:00
kasey
de0143e036 save origin block root before finalize (#10463)
* save origin block root before finalize

* add test for SaveOrigin

* goimports :(

* signature to LoadGenesis changed in a diff PR

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-04-05 17:13:53 +00:00
Nishant Das
2a7a09b112 Add Lock Analyzer (#10430)
* add lock analyzer

* fix locks

* progress

* fix failures

* fix error log

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-04-05 16:39:48 +00:00
terence tsao
984575ed57 Optimistic check: handle zeros check point root (#10464)
* Handle zero roots

* Update chain_info_test.go

* Update chain_info_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-04-05 14:37:25 +00:00
Nishant Das
927e338f9e Enable Bellatrix E2E (#10437)
* Fix finding terminal block hash calculation

* Update mainnet_config.go

* Update beacon_block.pb.go

* Various fixes to pass all spec tests for Merge (#9777)

* Proper upgrade altair to merge state

* Use uint64 for ttd

* Correctly upgrade to merge state + object mapping fixes

* Use proper receive block path for initial syncing

* Disable contract lookback

* Disable deposit contract lookback

* Go fmt

* Merge: switch from go bindings to raw rpc calls (#9803)

* Disable genesis ETH1.0 chain header logging

* Update htrutils.go

* all gossip tests passing

* Remove gas validations

* Update penalty params for Merge

* Fix gossip and tx size limits for the merge part 1

* Remove extraneous p2p condition

* Add and use

* Add and use TBH_ACTIVATION_EPOCH

* Update WORKSPACE

* Update Kintsugi engine API (#9865)

* Kintsugi ssz (#9867)

* All spec tests pass

* Update spec test shas

* Update Kintsugi consensus implementations (#9872)

* Remove secp256k1

* Remove unused merge genesis state gen tool

* Manually override nil transaction field. M2 works

* Fix bad hex conversion

* Change Gossip message size and Chunk SIze from 1 MB t0 10MB (#9860)

* change gossip size and chunk size after merge

* change ssz to accomodate both changes

* gofmt config file

* add testcase for merge MsgId

* Update beacon-chain/p2p/message_id.go

Change MB to Mib in comment

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* change function name from altairMsgID to postAltairMsgID

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Sync with develop

* Merge branch 'develop' of github.com:prysmaticlabs/prysm into kintsugi

* Update state_trie.go

* Clean up conflicts

* Fix build

* Update config to devnet1

* Fix state merge

* Handle merge test case for update balance

* Fix build

* State pkg cleanup

* Fix a bug with loading mainnet state

* Fix transactions root

* Add v2 endpoint for merge blocks (#9802)

* Add V2 blocks endpoint for merge blocks

* Update beacon-chain/rpc/apimiddleware/structs.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* go mod

* fix transactions

* Terence's comments

* add missing file

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Sync

* Go mod tidy

* change EP field names

* latest kintusgi execution api

* fix conflicts

* converting base fee to big endian format (#10018)

* ReverseByteOrder function does not mess the input

* sync with develop

* use merge gossip sizes

* correct gossip sizes this time

* visibility

* clean ups

* Sync with develop, fix payload nil check bug

* Speed up syncing, hide cosmetic errors

* Sync with develop

* Clean up after sync

* Update generate_keys.go

* sync with develop

* Update mainnet_config.go

* Clean ups

* Sync optimistically candidate blocks (#10193)

* Revert "Sync optimistically candidate blocks (#10193)"

This reverts commit f99a0419ef.

* Sync optimistically candidate blocks (#10193)

* allow optimistic sync

* Fix merge transition block validation

* Update proposer.go

* Sync with develop

* delete deprecated client, update testnet flag

* Change optimistic logic (#10194)

* Logs and err handling

* Fix build

* Clean ups

* Add back get payload

* c

* Done

* Rm uncommented

* Optimistic sync: prysm validator rpcs (#10200)

* Logs to reproduce

* Use pointers

* Use pointers

* Use pointers

* Update json_marshal_unmarshal.go

* Fix marshal

* Update json_marshal_unmarshal.go

* Log

* string total diff

* str

* marshal un

* set string

* json

* gaz

* Comment out optimistic status

* remove kiln flag here (#10269)

* Sync with devleop

* Sync with develop

* clean ups

* refactor engine calls

* Update process_block.go

* Fix deadlock, uncomment duty opt sync

* Update proposer_execution_payload.go

* Sync with develop

* Rm post state check

* Bypass eth1 data checks

* Update proposer_execution_payload.go

* Return early if ttd is not reached

* Sync with devleop

* Update process_block.go

* Update receive_block.go

* Update bzl

* Revert "Update receive_block.go"

This reverts commit 5b4a87c512.

* Fix run time

* add in all the fixes

* fix evaluator bugs

* latest fixes

* sum

* fix to be configurable

* Update go.mod

* Fix AltairCompatible to account for future state version

* Update proposer_execution_payload.go

* fix broken conditional checks

* fix all issues

* Handle pre state Altair with valid payload

* Handle pre state Altair with valid payload

* Log bellatrix fields

* Update log.go

* Revert "fix broken conditional checks"

This reverts commit e118db6c20.

* LH multiclient working

* Friendly fee recipient log

* Remove extra SetOptimisticToValid

* fix race

* fix test

* Fix base fee per gas

* Fix notifypayload headroot

* tx fuzzer

* clean up with develop branch

* save progress

* 200tx/block

* add LH flags

* Sync with devleop

* cleanup

* cleanup

* hash

* fix build

* fix test

* fix go check

* fmt

* gosec

* add deps

* cleanup

* fix up

* change gas price

* remove flag

* last fix

* use new LH version

* fix up

* fix finalized block panic

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Zahoor Mohamed <zahoor@zahoor.in>
Co-authored-by: kasey <489222+kasey@users.noreply.github.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Zahoor Mohamed <zahoor@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-04-05 14:02:46 +00:00
terence tsao
f44c99d92a Add blinded beacon block protobufs and wrappers (#10473)
Co-authored-by: rkapka <rkapka@wp.pl>
2022-04-05 14:57:19 +02:00
terence tsao
7d669f23ab Sync: process pending block with optimistic parent (#10480) 2022-04-04 18:46:05 -07:00
terence tsao
9b64c33bd1 RPC: GetValidatorPerformance for Bellatrix (#10482) 2022-04-04 19:50:57 -04:00
Potuz
defa602e50 Adapt Doppelganger to Altair (#9969)
Co-authored-by: rkapka <rkapka@wp.pl>
2022-04-04 15:55:55 +02:00
Radosław Kapka
67c8776f3c Fix execution payload field names in API Middleware (#10479) 2022-04-04 13:19:31 +02:00
Mohamed Zahoor
896d186e3b process the optimistic blocks whose parent are optimistic too (#10350)
* process the optimistic blocks whose parent are optimistic too

* adding unit test

* fix test

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-04-03 23:57:52 -03:00
Raul Jordan
edb98cf499 Move Engine API to Powchain and Consolidate Connection Management (#10438)
* gazelle

* tests passing

* use the same engine client

* pass

* initialize in right place

* erge

* build

* imports

* ensure engine checks work

* pass powchain tests

* powchain tests pass

* deepsource

* fix up node issues

* gaz

* endpoint use

* baz

* b

* conf

* lint

* gaz

* move to start function

* test pass
2022-04-01 14:04:24 -04:00
Potuz
ce15823f8d Save head after FCU (#10466)
* Update head after FCU

* rename function

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-04-01 00:08:01 +00:00
terence tsao
7cdc741b2f Sync: don't set block to bad due to timeout (#10470)
* Add filter error and tests

* Update BUILD.bazel

* Kasey's feedback
2022-03-31 23:29:27 +00:00
terence tsao
5704fb34be Remove duplicated engine mock (#10472)
* Rm mock

* udpate

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-03-31 19:14:09 -04:00
Preston Van Loon
3e2a037d42 github workflows: pin go version to 1.17 (#10471)
* github workflows: pin go version to 1.17

* Update go.yml

* Revert "Update go.yml"

This reverts commit 4a2d36d05d.

* pin golangci-lint

* try go 1.17

* Revert "Revert "Update go.yml""

This reverts commit 8a89663874.

* move and increase version of checkout

* attempt to ignore export path

* try with entrypoint only

* try some rearranging of stuff

* Split up jobs

* Use hack mentioned in https://github.com/securego/gosec/issues/469\#issuecomment-643823092

* Delete dappnode release trigger

* rm id

* try pin golangci-lint version

* try pin golangci-lint version

* Do not provide a specific go version and lets see what happens

* comment checkout, wtf is wrong with github actions

* it works locally...

* trying with some cache key for lint...

* Revert "trying with some cache key for lint..."

This reverts commit c4f5ae4495.

* try tellign it to skip go installation

* revert commented line, do something to satisify deepsource

* do something to satisify deepsource
2022-03-31 22:16:14 +00:00
Potuz
177f9ccab0 Return historical non-canonical blocks as optimistic (#10446)
* Return historical non-canonical blocks as optimistic

* terence's review
2022-03-31 12:51:14 +00:00
Potuz
649dee532f insert block to forkchoice after notifyNewPayload (#10453)
* insert block to forkchoice after notifyNewPayload

* Add optimistic candidate check

* Terence's review

* Apply suggestions from code review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-03-31 09:17:49 -03:00
terence tsao
4b3364ac6b Log terminal total difficulty status (#10457)
* Log terminal total diff status

* Update check_transition_config.go

* Update BUILD.bazel

* Update check_transition_config.go

* Update check_transition_config.go

* Update check_transition_config.go
2022-03-31 03:05:23 +00:00
kasey
0df8d7f0c0 refactor genesis state flag handling, support url (#10449)
* refactor genesis state flag handling, support url

* lint fix

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-03-30 22:23:34 +00:00
Mike Neuder
ade7d705ec Cleanup of Keymanager Deleter interface (#10415)
* Migrating Keymanager account list functionality into each keymanager type

* Addressing review comments

* Adding newline at end of BUILD.bazel

* bazel run //:gazelle -- fix

* account deleter cleanup

* bazel run //:gazelle -- fix

* remove stale logging statement

* adding deleter interface to mock functions

* fixing deepsource findings

* go.sum

* bazel run //:gazelle -- fix

* go mod t-dy -compat=1.17

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
2022-03-30 16:52:54 -05:00
Preston Van Loon
c68894b77d Spectest: Improve test size and timeouts (#10461)
* Adjust test sizes

* Improve the bazel test attributes for forkchoice and other tests

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-03-30 20:04:42 +00:00
terence tsao
fe98d69c0a Clean up blockchain pkg (#10452)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-30 17:45:24 +00:00
terence tsao
7516bc0316 Fix execution block's base fee endianess marshal/unmarshal (#10459) 2022-03-30 10:17:18 -07:00
Radosław Kapka
be6f3892e8 Align Beacon API with 2.2.0 spec (#10455) 2022-03-30 15:17:19 +00:00
Potuz
588605ceeb Remove TODOs that were already taken care of (#10454) 2022-03-30 13:24:02 +00:00
terence tsao
59b9519284 Forkchoice spec test: set boost with deterministic timing (#10428)
* Set boost with deterministic timing

* confs

* gaz

* Update runner.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-03-29 19:48:33 +00:00
Nishant Das
d4038fb752 add rlock (#10444) 2022-03-29 06:55:41 -07:00
Raul Jordan
56ab5872bb Add Histogram Response Buckets to Engine API Methods (#10414)
* gaz

* defer payload func

* amend

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-28 22:17:31 +00:00
Raul Jordan
eb6d68a4b1 Deterministic Proposer Root Boosting in Forkchoice (#10427)
* begin refactor

* fix doubly linked tree tests

* pass

* fix up

* gaz

* buidl

* buidl more

* comment

* args check

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-28 21:34:41 +00:00
kasey
7920528ede Checkpoint Sync 4/5 - enable checkpoint sync to be used by beacon node (#10386)
* enable checkpoint sync in beacon node

* lint fix

* rm unused error

* addressing PR feedback from Radek

* consistent slice -> fixed conversion

Co-authored-by: kasey <kasey@users.noreply.github.com>
2022-03-28 21:01:55 +00:00
Raul Jordan
1af3c07ec5 Stop Checking Transition Configuration Post-Merge (#10413)
* no longer check transition config after the merge

* nil check

* better payload check

* stop checking post merge based on received blocks

* add proper test

* gaz

* fix test

* lint

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-28 17:14:02 +00:00
terence tsao
7279349ae2 Cleanup helpers in beacon-chain/core/blocks/payload.go (#10435)
* less fragile check for execution payload

* gaz

* gaz

* include field

* clean up helpers for checking merge status

* Update beacon-chain/core/blocks/payload.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

* Update payload.go

* Rename

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-03-28 15:25:49 +00:00
Potuz
2d60d04b57 Update LastValidatedCheckpoint on DB (#10412)
* Update LastValidatedCheckpoint on DB

* fix conflicts
2022-03-28 14:48:14 +00:00
Nishant Das
ef8bc97d3e more efficient (#10440) 2022-03-28 20:27:14 +08:00
Nishant Das
071f6de559 Fix Beacon API Responses For Bellatrix (#10436)
* fix bugs

* fix build

* simplify it
2022-03-26 07:34:29 +00:00
Preston Van Loon
b697463da9 wrapper: Remove deprecated fork specific wrapper methods for signed beacon blocks (#10369)
* Remove deprecated wrappers, lots of refactoring

* Revert proto/prysm/v1alpha1/validator.proto

* fix tests

* fix test

* fix conversion in e2e

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-25 23:00:44 +00:00
kasey
bfbf693660 Checkpoint Sync 3/5 - beacon node api client lib and prysmctl cli tool (#10385)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2022-03-25 17:18:03 +00:00
Radosław Kapka
ed07f4bd77 Add execution_optimistic field to API responses (#10389)
* protos

* grpc

* middleware

* event protos

* event backend

* event middleware

* fix tests

* use `IsOptimisticForRoot` with proper block instead of header

* better algorithm

* return proper error

* review

* optimistic return on bellatrix proposal

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-25 16:31:50 +00:00
terence tsao
1922416cac Use correct pre state to call new payload (#10416)
* Update todo strings

* Go fmt

* add merge specific checks when receiving a block from gossip

* Fix beacon chain build

* Interop merge beacon state

* fix finding Transactions size

* Update go commit

* Merge union debugging (#9751)

* changes test cases per ssz changes

* noisy commit, restoring pb field order codegen

* get rid of codegen garbage

* M2 works with Geth  🎉

* Fix bazel build //...

* restoring generated pb field ordering

* defensive nil check

* separate ExecutionPayload/Header from codegen

* tell bazel about this new file

* Merge: support terminal difficulty override (#9769)

* Fix finding terminal block hash calculation

* Update mainnet_config.go

* Update beacon_block.pb.go

* Various fixes to pass all spec tests for Merge (#9777)

* Proper upgrade altair to merge state

* Use uint64 for ttd

* Correctly upgrade to merge state + object mapping fixes

* Use proper receive block path for initial syncing

* Disable contract lookback

* Disable deposit contract lookback

* Go fmt

* Merge: switch from go bindings to raw rpc calls (#9803)

* Disable genesis ETH1.0 chain header logging

* Update htrutils.go

* all gossip tests passing

* Remove gas validations

* Update penalty params for Merge

* Fix gossip and tx size limits for the merge part 1

* Remove extraneous p2p condition

* Add and use

* Add and use TBH_ACTIVATION_EPOCH

* Update WORKSPACE

* Update Kintsugi engine API (#9865)

* Kintsugi ssz (#9867)

* All spec tests pass

* Update spec test shas

* Update Kintsugi consensus implementations (#9872)

* Remove secp256k1

* Remove unused merge genesis state gen tool

* Manually override nil transaction field. M2 works

* Fix bad hex conversion

* Change Gossip message size and Chunk SIze from 1 MB t0 10MB (#9860)

* change gossip size and chunk size after merge

* change ssz to accomodate both changes

* gofmt config file

* add testcase for merge MsgId

* Update beacon-chain/p2p/message_id.go

Change MB to Mib in comment

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* change function name from altairMsgID to postAltairMsgID

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Sync with develop

* Merge branch 'develop' of github.com:prysmaticlabs/prysm into kintsugi

* Update state_trie.go

* Clean up conflicts

* Fix build

* Update config to devnet1

* Fix state merge

* Handle merge test case for update balance

* Fix build

* State pkg cleanup

* Fix a bug with loading mainnet state

* Fix transactions root

* Add v2 endpoint for merge blocks (#9802)

* Add V2 blocks endpoint for merge blocks

* Update beacon-chain/rpc/apimiddleware/structs.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* go mod

* fix transactions

* Terence's comments

* add missing file

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Sync

* Go mod tidy

* change EP field names

* latest kintusgi execution api

* fix conflicts

* converting base fee to big endian format (#10018)

* ReverseByteOrder function does not mess the input

* sync with develop

* use merge gossip sizes

* correct gossip sizes this time

* visibility

* clean ups

* Sync with develop, fix payload nil check bug

* Speed up syncing, hide cosmetic errors

* Sync with develop

* Clean up after sync

* Update generate_keys.go

* sync with develop

* Update mainnet_config.go

* Clean ups

* Sync optimistically candidate blocks (#10193)

* Revert "Sync optimistically candidate blocks (#10193)"

This reverts commit f99a0419ef.

* Sync optimistically candidate blocks (#10193)

* allow optimistic sync

* Fix merge transition block validation

* Update proposer.go

* Sync with develop

* delete deprecated client, update testnet flag

* Change optimistic logic (#10194)

* Logs and err handling

* Fix build

* Clean ups

* Add back get payload

* c

* Done

* Rm uncommented

* Optimistic sync: prysm validator rpcs (#10200)

* Logs to reproduce

* Use pointers

* Use pointers

* Use pointers

* Update json_marshal_unmarshal.go

* Fix marshal

* Update json_marshal_unmarshal.go

* Log

* string total diff

* str

* marshal un

* set string

* json

* gaz

* Comment out optimistic status

* remove kiln flag here (#10269)

* Sync with devleop

* Sync with develop

* clean ups

* refactor engine calls

* Update process_block.go

* Fix deadlock, uncomment duty opt sync

* Update proposer_execution_payload.go

* Sync with develop

* Rm post state check

* Bypass eth1 data checks

* Update proposer_execution_payload.go

* Return early if ttd is not reached

* Sync with devleop

* Update process_block.go

* Update receive_block.go

* Update bzl

* Revert "Update receive_block.go"

This reverts commit 5b4a87c512.

* Fix run time

* Update go.mod

* Fix AltairCompatible to account for future state version

* Update proposer_execution_payload.go

* Handle pre state Altair with valid payload

* Handle pre state Altair with valid payload

* Log bellatrix fields

* Update log.go

* Friendly fee recipient log

* Remove extra SetOptimisticToValid

* Fix base fee per gas

* Fix notifypayload headroot

* clean up with develop branch

* Sync with devleop

* Fix

* Fix

* fix

* Fix

* fix tests

* revert some changes

* fix tests

* Update optimistic_sync_test.go

* Simplify IsExecutionEnabledUsingHeader

Co-authored-by: Zahoor Mohamed <zahoor@prysmaticlabs.com>
Co-authored-by: Zahoor Mohamed <zahoor@zahoor.in>
Co-authored-by: kasey <489222+kasey@users.noreply.github.com>
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-25 16:03:15 +00:00
Potuz
a5dc40393d Allow inner protoarray nodes to become VALID (#10418)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-25 15:35:44 +00:00
Raul Jordan
b238235210 prevent run (#10432) 2022-03-25 11:08:38 -04:00
terence tsao
73a6a85069 Ensure finalized root can't be zeros (#10422) 2022-03-24 23:55:54 +00:00
terence tsao
ac4a30588f Properly initialize replayer builder for rpc (#10426) 2022-03-24 23:12:47 +00:00
Raul Jordan
3579551f15 Use --http-web3provider for Execution Engine Connection (#10307)
* combine endpoints

* initialization in the start function instead

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-03-24 20:53:07 +00:00
Nishant Das
956e7a7563 Update Go-Ethereum (#10417)
* add changes

* fix CI

* wait here

* Fix gazelle with the correct directive for libp2p

Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-24 19:00:53 +00:00
kasey
acd7e62cfb fix issue #10420 (#10421)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2022-03-24 18:32:39 +00:00
Radosław Kapka
165f0667f0 Implement getDebugChainHeadsV2 API endpoint (#10419)
* protos

* grpc

* middleware

* update comments
2022-03-24 13:08:40 +00:00
kasey
49826ebe28 Checkpoint Sync 2/5 - API support for retrieving weak subjectivity data (#10384)
also refactor replayer code for better reuse:
- separately expose stategen's canonical block func
- CanonicalHistory in ws api, NewCanonicalBuilder
- refactor CanonicalHistory into a ReplayerBuilder

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2022-03-23 22:54:07 +00:00
terence tsao
3ad6abd9c0 Add tracings for engine api methods (#10407)
* Add engine call tracings

* Update BUILD.bazel

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-23 22:17:09 +00:00
terence tsao
c61f54176d Ignore attestations voting for the wrong finalized checkpoint (#10295)
* Ignore attestations voting for the wrong finalized checkpoint

* aggregate

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-23 21:48:26 +00:00
Potuz
0165351db3 Setup Optimistic sync when starting the service (#10397)
* Load Last Validated Checkpoint at service startup and initialize
Forkchoice

* fix spectests

* Update service_test.go

* wip

* Fix blockchain tests

* remove unnecessary summary saves

* Terence's suggestion

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-03-23 21:11:36 +00:00
Potuz
8f49167117 return invalid roots from SetOptimisticToInvalid (#10403)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-03-23 15:55:05 -03:00
Nishant Das
b4c27f64de Add Method To Modify Participation Bits (#10409)
* fix up all tests

* Update beacon-chain/state/v2/BUILD.bazel

* add tests
2022-03-23 23:18:02 +08:00
Preston Van Loon
e4ac5e74b7 RPC: Deduplicate block production logic (#10368)
* RPC: Deduplicate block production logic

* Surface error to caller

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-03-23 12:51:20 +00:00
Nishant Das
0b9b635646 Block Processing And State Transition Improvments (#10408)
* cleanup

* clean up more

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-03-23 11:57:48 +00:00
Leo Lara
c8c1d04c07 Complete coverage for in beacon chain state for getters checkpoint (#10373)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-03-23 09:58:45 +00:00
james-prysm
601987faf2 Fee Recipient : ux log fixes and prevent validator from calling beacon node if flags are not provided. (#10406)
* initial commit

* improving logs
2022-03-22 13:13:59 -05:00
james-prysm
a060d765b3 rename prepare beacon proposer to fee recipient (#10404) 2022-03-22 15:00:50 +00:00
Potuz
5550334956 set optimistic status on notify payload (#10398)
* set optimistic status on notify payload

* pre-execution blocks are never optimistic

* add unit test

* add test for lengths

* Update optimistic_sync_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-03-22 03:35:02 +00:00
Mike Neuder
002253bba3 Migrating Keymanager account list functionality into each type (#10382)
* Migrating Keymanager account list functionality into each keymanager type

* Addressing review comments

* Adding newline at end of BUILD.bazel

* bazel run //:gazelle -- fix

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-03-22 03:04:09 +00:00
Radosław Kapka
c055642f79 Return correct header root from API calls (#10401)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-22 01:56:44 +00:00
terence tsao
c7d64c03ac Insert payload hash to forkchoice store nodes (#10400)
* Insert payload hash to forkchoice store nodes

* Fix tests

* Update new_slot_test.go

* Update beacon-chain/blockchain/process_block.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-03-22 01:20:42 +00:00
james-prysm
131fb43ba7 Fee recipient flag rename for beacon node (#10402)
* initial commit

* fixing comment

* fixing bazel
2022-03-21 21:25:33 +00:00
kasey
cf0bd633f0 Checkpoint Sync 1/5 - fork/version detection and unmarshaling support (#10380)
* fork/version detection and unmarshaling support

* Update config/params/config.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update proto/detect/configfork.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* PR feedback

* move ssz initialization into the detect package

* clarify comment

* VersionForEpoch is much simpler/clearer in reverse

* simpler VersionForEpoch; build AllConfigs in init

* use fieldparams for Version

* Update proto/detect/configfork_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* remove custom ForkName type, use runtime/version

* pr cleanup

* random fix from bad gh ui suggestion; privatize

* privatize fieldSpec methods; + unit tests

* Update proto/detect/configfork.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

* fix bad github ui suggestion

* ensure unique versions for simpler config match

* fmt & adding unit test for ByState()

* table-driven unit test for ByState

* TestUnmarshalState

* OrderedSchedule -> network/forks per PR feedback

* goimports

* lint fixes

* move proto/detect -> ssz/encoding/detect

* use typeUndefined in String

* backport config tests from e2e PR

* fix config parity test; make debugging it easier

* lint

* fix fork schedule initialization

* cleanup

* fix build

* fix big ole derp

* anything for you, deep source

* goimportsss

* InitializeForkSchedule in LoadChainConfigFile

* PR feedback

Co-authored-by: kasey <kasey@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-03-21 19:43:41 +00:00
james-prysm
df8da80db8 Implement Fee Recipient: CLI flags and Calling Beacon API (#10312)
* send proposer data to config

* wip implementation with file and url based config import

* improving logic on get validator index

* fix function

* optimizing function for map and address bug

* fixing log

* update cache if it doesn't exist

* updating flags

* initial unit test scaffold

* fixing validator to call rpc call, removed temporary dependency

* adding the API calls for the runner

* fixing broken build

* fixing deepsource

* fixing interface

* fixing fatal

* fixing more deepsource issues

* adding test placeholders

* updating proposer config to add validation

* changing how if statement throws error

* removing unneeded validation, validating in a different way

* wip improving tests

* more unit test work

* fixing unit test

* fixing unit tests and edge cases

* adding unit tests and adjusting how the config is created

* fixing bazel builds

* fixing proto generation

* fixing imports

* fixing unit tests

* Update cmd/validator/flags/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* updating flags based on comments, fixing unit tests

* fixing bazel

* removing unneeded function

* fixing unit tests

* refactors and unit test fixes based on comments

* fixing bazel build

* refactor the cache out fo the fee recipient function

* adding usecase for multiple fee recipient

* refactor burn name

* Update validator/client/validator.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/validator.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/validator.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* fixing bug with validator index based on code review

* edited flag descriptions to better communicate usage

* fixing manual reference to flag name

* fixing code review comments

* fixing linting

* Update validator/client/validator.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* addressing comments and renaming functions

* fixing linting

* Update cmd/validator/flags/flags.go

* Update cmd/validator/flags/flags.go

* Update cmd/validator/flags/flags.go

* improving comments

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-03-21 13:48:02 -05:00
Raul Jordan
dc527a3c80 Remove Visibility Check Script for Bazel (#10391)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-21 17:03:52 +00:00
Potuz
94f80bd208 Call FCU on update head after process attestation (#10394)
* Call FCU on update head after process attestation

* add test

* fix test

* terence review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-03-21 01:50:08 +00:00
terence tsao
c6bfefa0bc Add notifyNewPayload to onBlockBatch (#10392) 2022-03-18 09:36:46 -07:00
Radosław Kapka
3446eaa5f2 Add missing tests to state retrieval (#10390) 2022-03-17 18:40:07 +00:00
terence tsao
193866c731 Add fcu to batch blk processing (#10371) 2022-03-17 07:35:50 -07:00
terence tsao
b90ce1b60f Bellatrix fork choice spec tests (#10381)
* Add bellatrix forkchoice spec tests

* Update service.go

* better conversion
2022-03-17 05:30:44 +00:00
terence tsao
8449d24ed0 Fix BaseFeePerGas endianess (#10376)
* Fix base fee endianess and add more test entropys

* Update json_marshal_unmarshal_test.go

* Update setters_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-16 22:12:08 +00:00
Radosław Kapka
b92b01f379 API middleware support for bellatrix in produceBlockV2 (#10374) 2022-03-16 21:44:15 +00:00
terence tsao
bcd180ee4d Remove deprecated set_optimistic_to_valid calls (#10370) 2022-03-16 21:14:13 +00:00
Raul Jordan
5f10e51a49 Add API Middleware Support for Fee Recipients Endpoint (#10377)
* add in success log

* add in info log

* uint64 for schema

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-16 20:02:26 +00:00
terence tsao
c4f03868ce Update default fee warning (#10378)
* Add fcu to batch blk processing

* Log error only if is burned

* Revert some changes

* Burn address
2022-03-16 19:26:51 +00:00
james-prysm
d55c0d74dd PrepareBeaconProposer API (#10367)
* intial commit implementing prepare beacon proposal

* removing files accidently checked in by goland

* removing files accidently checked in by goland

* Delete BUILD 2

* Delete armeabi_cc_toolchain_config 2.bzl

* Delete builtin_include_directory_paths 2

* Delete cc_toolchain_config 2.bzl

* Delete BUILD 2.bazel

* Delete cc_wrapper 2.sh

* Delete module 2.modulemap

* Delete BUILD 2

* Delete BUILD 2

* Delete WORKSPACE 2

* Delete cc_toolchain_config_linux_arm64 2.bzl

* Delete cc_toolchain_config_osx 2.bzl

* Delete cc_toolchain_config_windows 2.bzl

* Delete 0258716f94e00f9df0da869fd97f9e0d0c6ac83eb528677d918f0ac9be5f4b8d 2

* Delete test-unmarshal-bad 2.json

* Delete test-unmarshal-good 2.json

* adding simple unit tests

* removing duplicate imports

* adding bazel changes

* adding validation and switched to table driven tests

* updating based on comments

* adding to test retrieve from db

* fixing bazel
2022-03-16 11:17:41 -04:00
Potuz
807b71244b Add database methods for optimistic sync (#10357)
* Add database methods for optimistic sync

* Add epoch comparison

* add extra epoch comparison

* Summary instead of block

* fix tests

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-03-15 20:52:59 +00:00
Potuz
2744eba391 Update optimisticCandidateBlock to specs version (#10355)
* Update optimisticCandidateBlock to specs version

* fix double import

* Change error message
2022-03-15 19:12:52 +00:00
Leo Lara
5507558678 Some improvements for the state package testing (#10316)
* Refactor to unify state getters block tests

* Add reference tests to Altair and Bellatrix versions ofthe state

* Fix function naming convetion

* Add state-native/v2/references_test.go and state-native/v3/references_test.go

* Gazelle run

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-03-15 18:32:09 +00:00
terence tsao
8cecd4e8bf Log bellatrix fields (#10365)
* Log bellatrix fields

* Update log.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-15 14:02:39 +00:00
terence tsao
69a9388515 Spec test v1.1.10 (#10298)
* spectest

* uncomment shas

* Fc spec test service shouldn't not spawn attestation routine

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-03-15 13:17:51 +00:00
terence tsao
bd308b6d73 Update default fee recipient error log (#10366) 2022-03-14 20:35:56 -07:00
Radosław Kapka
9abea200a5 Add Bellatrix version to ProduceBlock API (#10361)
* Add Bellatrix version to `ProduceBlock` API

* fix errors

* fix test

* Revert "Auxiliary commit to revert individual files from 8cc6fc9ca3b6ddb67aeafeaddc2a2e1ca6610ded"

This reverts commit 6358ee92d899e8ee6587ad5503cebb40ac92add4.

# Conflicts:
#	beacon-chain/blockchain/optimistic_sync_test.go
#	beacon-chain/blockchain/pow_block_test.go
#	beacon-chain/blockchain/testing/mock_engine.go

* use proper mock

* bzl

* fix build file

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-14 23:08:01 +00:00
terence tsao
730678bf21 Handle pre state Altair with valid payload (#10364)
* Handle pre state Altair with valid payload

* Handle pre state Altair with valid payload
2022-03-14 22:40:08 +00:00
kasey
0b1a777d62 ran gofmt @ v1.17, these are the resulting changes (#10362)
* ran gofmt @ v1.17, these are the resulting changes

* fix some flaky tests that are tripping up this PR

* gofmt

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2022-03-14 15:58:13 -05:00
terence tsao
5461c5b84f Insert stored fee recipient for ForkchoiceUpdated call (#10349)
* Add default

* Add and use fee recipient in db

* Update BUILD.bazel

* Feedback

* Add basic addr check

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-14 18:44:40 +00:00
Preston Van Loon
03e7acbf9f increase bazel java max mem to 4gb (#10363)
* increase bazel java max mem to 4gb

* DEBUG: set nocache and modify nogo to force any disk cache misses. Revert this commit after testing in CI

* Revert "DEBUG: set nocache and modify nogo to force any disk cache misses. Revert this commit after testing in CI"

This reverts commit dc41cce43d.
2022-03-14 17:53:42 +00:00
Potuz
6cc88e6454 enable optimistic sync (#10358)
* enable optimistic sync

* revert computation of HTR in notifyNewPayload

* fix tests

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-03-14 17:25:40 +00:00
terence tsao
65cea17268 Return early if ttd is not reached (#10359) 2022-03-14 16:38:05 +00:00
Preston Van Loon
aa86c94a91 Update bazel to 5.0.0 (#10352)
* Update to bazel 5

* Add new bazel 5 experimental flags to improve remote caching

* regen cross compile for bazel 5

* gazelle

* remove some old flags

* use heremtic builds sandbox

* minimal downloads for better disk usage

* Restore manual tags for darwin builds
2022-03-14 15:05:08 +00:00
Radosław Kapka
75bb25d515 Bellatrix publish block - API Middleware fixes (#10348)
* Bellatrix publish block - API Middleware fixes

* update field name

* correct docs
2022-03-14 13:36:46 +00:00
terence tsao
1e845bc276 Fix sync committee assignment for Bellatrix (#10356)
* Fix AltairCompatible to account for future state version

* Tests

* gaz

* More tests

* Update comments

* Rename to HigherThanAltairVersionAndEpoch

* HigherEqualThanAltairVersionAndEpoch

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-03-14 01:00:33 +00:00
Nishant Das
58f7e942f2 Fix Mnemonic Validation (#10354)
* fix mnemonic validation

* potuz's review

* Add fuzz test for wallet recover

Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
2022-03-14 07:30:11 +08:00
Potuz
3f1e3cf82f remove optimistic control flag (#10340)
* remove optimistic control flag

* Rename

* More

* More

* redundant return

* fix conflict

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-03-12 17:32:04 +00:00
terence tsao
60003c481b Add unlock to IsOptimistic returns (#10341)
* Add unlock to returns

* add regression test

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-11 10:36:47 +00:00
Preston Van Loon
c1197d7881 Add static analysis for unsafe uint casting (#10318)
* Add static analysis for unsafe uint casting

* Fix violations of uintcast

* go mod tidy

* Add exclusion to nogo for darwin build

* Add test for math.Int

* Move some things to const so they are assured not to exceed int64

* Self review

* lint

* fix tests

* fix test

* Add init check for non 64 bit OS

* Move new deps from WORKSPACE to deps.bzl

* fix bazel build for go analysis runs

* Update BUILD.bazel

Remove TODO

* add math.AddInt method

* Add new test casts

* Add case where builtin functions and declared functions are covered

* Fix new findings

* cleanup

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-03-11 09:34:30 +00:00
Nishant Das
693cc79cc9 Minor Cleanup from #10255 (#10337)
* minor cleanup

* kasey's review

* kasey's review
2022-03-11 07:34:15 +00:00
kasey
23778959eb this is safer if a goroutine require fails (#10344)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2022-03-11 05:29:52 +00:00
kasey
92278e2255 fix TestValidator_WaitForKeymanagerInitialization_Web race (#10343)
* TestValidator_WaitForKeymanagerInitialization_Web

* gofmt

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2022-03-10 20:04:27 -06:00
james-prysm
c58ce41b3b Flagutil: utility for json file from directory or url and unmarshal to struct (#10333)
* initial commit

* adding first testcase wip

* fixing test

* adding more unit tests

* adding bazel file

* adding more unit tests and file checks

* addressing comments

* refactoring based on comments

* added bazel

* fixing build
2022-03-10 14:25:26 -06:00
kasey
d3b09d1e9d --curses=no for cleaner logs (no term refresh) (#10334)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-10 18:05:36 +00:00
Michael Neuder
27082e2cd2 Adding ExtractKeystores to the Keymanager interface (#10313)
* adding ExtractKeystores to the Keymanager interface

* adding ExtractKeystore to mockKeymanager struct types

* bazel run //:gazelle -- fix

Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-03-10 17:19:04 +00:00
terence tsao
12080727ea Save and retrieve fee recipients for db (#10336)
* Can save fee recipients in db

* Update BUILD.bazel

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-03-10 15:05:10 +00:00
Potuz
a731b8c0bc Boost proposer later (#10338)
* Boost Proposer after inserting block

* add regression test
2022-03-10 13:13:06 +00:00
Raul Jordan
f6eed74500 PrepareBeaconProposer Protobuf Schema (#10332)
* prepare v1 protos

* add in protos

* prepare beacon proposer

* stubs

* builds

* gaz

* build

* rem

* ssz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-09 21:40:26 +00:00
Potuz
329a4a600c Add USE_PRYSM_MODERN environment variable (#10229)
* Add USE_PRYSM_MODERN environment variable

* fix deepsource

* change naming convention

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-09 21:00:12 +00:00
terence tsao
7ce712bb5e Kiln run time changes (#10321)
* First take

* Update proposer_execution_payload.go

* Update optimistic_sync_test.go

* Add tests

* Update proposer_execution_payload.go

* Fix tests

* Add deprecation

* Fix bad merge

* New test

* Update beacon-chain/core/blocks/payload.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Proposer test can get and compare payload

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-03-09 20:26:23 +00:00
kasey
17a43c1158 Retrieving state by slot to always apply canonical block at slot, when available (#10255)
* new stategen.StateReplayer/ReplayerBuilder to give more fine-grained
  control of replaying state+block history
* all rpc/api methods updated to use the new interface, return post-state

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-03-09 13:33:18 -06:00
Radosław Kapka
77b8b13eff Bellatrix API support for block endpoints (#10324)
* refactor GetBlockV2

* Add bellatrix to GetBlockSSZV2

* Add bellatrix to ListBlockAttestations

* Add bellatrix to SubmitBlock

* gzl

* return error from SubmitBlock

* return nil

* Better code flow when getting blocks

* remove tautology

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-09 17:18:52 +00:00
Potuz
b59484b285 Export SetOptimisticToInvalid (#10330)
* temp

* export remove invalid blocks
2022-03-09 13:50:57 -03:00
Radosław Kapka
1964fb8146 Correct package name in features' README file (#10329) 2022-03-09 14:24:00 +00:00
Nishant Das
63825290cb Handle Port Registration Better (#10325) 2022-03-09 13:52:50 +01:00
kmax.eth
1619d880d4 fix TestSyncHandlers_WaitTillSynced (#10328)
* fix TestLockUnlock_CleansUnused

* fix TestSyncHandlers_WaitTillSynced
The test is failing silently (while go test showing 'PASS') due to panic caused by feed type mismatch. As a result, intended testing logic is not exercised at all. There are a couuple fixes here:
1. fix the feed event type to be pointer
2. add seendCache to avoid nil pointer panic
3. fill block in beaconDB so validateBeaconBlockPubSub gets short cut
4. replace time.Sleep() with deterministic channel waiting

* handle cancel func
2022-03-09 05:42:03 +00:00
kmax.eth
af2b858aa2 fix TestLockUnlock_CleansUnused (#10326) 2022-03-09 04:08:28 +00:00
Potuz
57a323f083 Forkchoice featureflag (#10299)
* Compiling main beacon-chain binary

* Add feature flag

* passing protoarray tests

* passing nodetree tests

* passing blockchain package tests

* passing rpc tests

* go fmt

* re-export forkchoice store from blockchain package

* remove duplicated import

* remove unused var

* add nodetree rpc method

* remove slot from IsOptimisticForRoot

* release lock in IsOptimistic

* change package name

* Revert "change package name"

This reverts commit 679112f9ef.

* rename package

* Update doc

* Fix span names

* Terence + Raul review

* remove go:build flags

* add errors dep

* spec tests

* fix call to IsOptimisticForRoot

* fix test

* Fix conflict

* change name of function

* remove ctx from store.head

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-03-09 03:05:51 +00:00
Nishant Das
738f00129b control test (#10323) 2022-03-08 07:58:19 -08:00
Radosław Kapka
3a03623094 E2E: Cleanup old Eth1 code (#10320) 2022-03-07 23:14:10 +00:00
terence tsao
cde58f6924 Rest of kiln changes (#10319)
* Update configs

* gazelle

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-07 19:47:08 +00:00
Preston Van Loon
78fe712e53 Validator: deduplicate fork logic for block proposals (#10297)
* Deduplicate block proposals

* fix tests, more dedup of tests

* Add godoc to BuildSignedBeaconBlock.

* Rename SignRequest_Object => SignRequestObject

* Fix error messages

* Add tests for new methods

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-07 19:18:54 +00:00
terence tsao
40e5a5d796 RPC: add prepare execution payload (#10311)
* Add prepare execution payload

* Add prepare execution payload

* Add prepare execution payload

* Update client.go

* Update proposer_execution_payload.go

* Update proposer_execution_payload_test.go

* Handle post bellatrix finalied blk

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer_execution_payload.go

* Use BeaconBlockIsNil

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-03-07 18:43:20 +00:00
Radosław Kapka
87507cbfe2 Support Bellatrix state in getStateV2 standard API (#10314) 2022-03-07 17:56:54 +01:00
Mohamed Zahoor
b516cfd998 cherry picked PR of #10233 (#10301)
* fix merge conflicts

* fix more merge conflicts

* fix fmt

* remove unnecessary cfg

* setting bad block if the bellatrix validation fails

* added an edge condition

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-07 13:19:07 +00:00
Radosław Kapka
f98d1ce64b Run multiple go-ethereum nodes in e2e (#10277)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-03-07 13:23:15 +01:00
Mohamed Zahoor
a103dd91c0 cherry picked PR for #10274 (#10300)
* ignore topic messages (except block topic) during optimistic sync

* address review comments

* nit pick fix

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-03-06 08:09:12 +00:00
terence tsao
74fe2cc8d0 Spawn attestation routine at a better place (#10303)
* Spawn attestation routine at a better place

* Revert

* Update service.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-04 22:25:04 +00:00
Nishant Das
a4bbaac262 Add Better Error Logs for Context Deadlines (#10310)
* better log here too

* remove period

* Update beacon-chain/p2p/pubsub.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-03-04 16:21:19 +00:00
Nishant Das
1af11885ee Remove Support for 2d-list Hashers (#10290)
* add changes

* fix logic bug

* fix

* potuz's review

* Update beacon-chain/state/stateutil/eth1_root.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-03-04 15:19:07 +00:00
Nishant Das
1437cb8982 Update GoHashTree Library (#10308) 2022-03-04 14:18:52 +00:00
Nishant Das
0b559afe30 Better Error Log In Context Deadlines (#10309)
* add better error log

* radek's review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-04 09:54:06 +00:00
Radosław Kapka
54915850a2 Refactor E2E port registration (#10306)
* Refactor e2e port registration

* uncomment tests

* explain calculation

* fix things

* change param to pointer

* fix errors

* unit test and constant
2022-03-04 09:26:28 +00:00
terence tsao
69618d157a Bellatrix validator terminal block helpers (#10305)
* Add get transition block

* Update proposer_execution_payload.go
2022-03-03 15:00:11 -05:00
Nishant Das
fa750650ed Disable Vectorized HTR from Our Dev Flag (#10304) 2022-03-03 03:49:26 +00:00
terence tsao
7303985232 Validator RPC: add optimistic status check (#10291)
* Add optimistic status check

* Simplify a bit more

* Update status_test.go

* Add non opt tests

* Update aggregator_test.go

* More tests

* Preston's feedback
2022-03-03 00:16:34 +00:00
Raul Jordan
958dd9d783 Check Engine API Transition Configuration in Background (#10250)
* transition proto

* gen pb

* builds

* impl transition config

* begin tests

* transition config messed up

* amend proto

* use str

* passing

* gaz

* config

* client test

* pb

* set to 0

* rem log

* gaz

* check transition config

* check config differences

* check transition config in background

* gaz

* pass

* redundant

* fix up error handling and healthz

* simplify status

* gazelle

* build

* err config check

* test

* gaz

* Fix run time

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-01 21:38:51 +00:00
Preston Van Loon
a3f8ccd924 validator: Fix flaky test TestServer_RefreshJWTSecretOnFileChange (#10296) 2022-03-01 20:34:05 +00:00
terence tsao
d64f6cb7a8 Add engine methods to block processing (#10285)
* Add notify newPayload and forkchoiceUpdate

* Tests

* Raul's feedback

* Update optimistic_sync.go

* Simplify

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-01 16:43:06 +00:00
terence tsao
a9a75e0004 Release lock before return validatedTips (#10289)
* Release lock before return

* add test

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-03-01 15:36:47 +00:00
Nishant Das
339540274b Integration of Vectorized Sha256 In Prysm (#10166)
* add changes

* fix for vectorize

* fix bug

* add new bench

* use new algorithms

* add latest updates

* save progress

* hack even more

* add more changes

* change library

* go mod

* fix deps

* fix dumb bug

* add flag and remove redundant code

* clean up better

* remove those ones

* clean up benches

* clean up benches

* cleanup

* gaz

* revert change

* potuz's review

* potuz's review

* potuz's review

* gaz

* potuz's review

* remove cyclical import

* revert ide changes

* potuz's review

* return
2022-02-28 21:56:12 +08:00
Raul Jordan
12ba8f3645 Renaming Random in ExecutionPayloads to PrevRandao (#10283)
* rename proto

* p header

* regen

* regen ssz

* fix randao

* random name changes

* bazel builds

* bt

* incorrect prev randao

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-26 03:47:16 +00:00
Raul Jordan
f3a7f399c0 Engine API Client Authentication for the Merge via HTTP (#10236)
* round tripper with claims

* auth

* edit auth

* test out jwt

* passing

* jwt flag

* comment

* passing

* commentary

* fix up jwt parsing

* gaz

* update jwt libs

* tidy

* gaz

* lint

* tidy up

* comment too long

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-02-25 19:08:43 +00:00
james-prysm
6163e091a7 web3signer: fixes for e2e (#10281)
* fixing logs and caught bug in mappers

* Fix schema

* improving logging

* Update validator/keymanager/remote-web3signer/internal/client.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* adding logurus dependency

Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
2022-02-25 02:42:26 +00:00
terence tsao
2fb4ddcbe7 Engine API: add payload status handling and tests (#10282)
* Add status handling and tests

* Update client.go

* Fmt

* Update mock_engine_test.go

* Update client_test.go
2022-02-24 19:35:01 +00:00
Radosław Kapka
1c2e463a30 --api-timeout flag (#10260)
* `--api-timeout` flag

* simplify code

* review feedback

* better error handling

* better docs

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-24 18:01:37 +00:00
james-prysm
01e9125761 web3signer: url parsing bug (#10278)
* adding in fixes for url

* fixing gazelle

* fixing wrong keymanager kind

* adding required scheme to urls

* fixing another unit test

* removing unused file

* adding new commit to retrigger deepsource ci
2022-02-24 10:24:11 -06:00
terence tsao
02a088d93c Add validate_merge_block (#10273)
* Add

* Update pow_block.go

* Update BUILD.bazel

* Update beacon-chain/blockchain/pow_block.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* feedbacks

* Feedbacks

* Fmt

* Update BUILD.bazel

* Update BUILD.bazel

* Update pow_block_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-23 22:41:11 +00:00
terence tsao
8cadb2ac6f Update IsOptimistic to always false (#10276)
* Update  to always false

* Use epoch

* Update chain_info_test.go

* Update chain_info_test.go
2022-02-23 20:16:45 +00:00
Potuz
be722604f7 Fix logarithm of 2 (#10275)
* Fix logarithm of 2

* add regression test

* Update encoding/ssz/merkleize.go

Co-authored-by: Nishant Das <nishdas93@gmail.com>

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-02-23 12:44:04 +00:00
Raul Jordan
3bb2acfc7d Update Web UI to v1.0.3 (#10264)
* update web UI version

* fixing format

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: James He <james@prysmaticlabs.com>
2022-02-22 18:26:15 +00:00
terence tsao
7719356b69 Add opt sync bool to IsOptimistic (#10270)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-22 16:46:42 +00:00
Raul Jordan
75b9bdba7c Small Comment Fix in Exchanging Transition Config (#10271) 2022-02-22 16:16:02 +00:00
Nishant Das
525c818672 Remove SSZ Cache (#10256)
* remove ssz cache

* gaz

* lint

* analyze more

* fix
2022-02-22 17:27:51 +08:00
terence tsao
a55fdf8949 Use type string for total_difficulty (#10265)
* Use string for difficulty

* fix go

* fix test

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-02-21 14:03:12 +00:00
Preston Van Loon
7f41b69281 Wrapper: Update block interface and reorganize fork logic (#10267)
* Add ssz.HashRoot interface composition to BeaconBlock interface, move fork specific logic into it's own files

* Remove needless underscore
2022-02-20 20:53:05 +00:00
Preston Van Loon
c5189a6862 Deduplicate TestProposer_ProposeBlock_OK (#10266)
* Deduplicate tests for TestProposer_ProposeBlock

* remove erroneous save of block

* ensure block root is returned
2022-02-19 22:52:03 +00:00
Leo Lara
b4b976c28b Experimental prototype of Apple M1 processor support (#10192)
* Experimental prototype of Apple M1 processor support

* Enable Apple M1 compilation of herumi MCL by adding a precompiled library

* Renable nogo

* Fix by gazelle

* Update go.mod to reflect go 1.17.6 changes in WORKSPACE

* go mod tidy

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
2022-02-19 18:17:08 +00:00
Nishant Das
ced24892a5 Cleanup Powchain Service (#10259)
* add cleanup

* gaz

* Fix test build
2022-02-18 14:13:31 +00:00
terence tsao
49f989e342 Fix base fee endianness (#10253) 2022-02-16 17:48:55 +00:00
Leo Lara
3003f08770 Remove unnecessary target of third_party/herumi/bls.BUILD (#10252)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-02-16 16:35:19 +00:00
terence tsao
0232b5f8f5 Regen pbs (#10227)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-02-16 10:06:41 -06:00
terence tsao
90a15b2fbe Handle PayloadStatus UnmarshalJSON nil cases (#10249)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-16 01:24:50 +00:00
Raul Jordan
0f58c9a925 Exchange Transition Configuration for Engine API (#10248)
* transition proto

* gen pb

* builds

* impl transition config

* begin tests

* transition config messed up

* amend proto

* use str

* passing

* gaz

* config

* client test

* pb

* set to 0

* rem log

* gaz
2022-02-16 00:56:23 +00:00
terence tsao
6311cfd8ab Fix payloadStatusJSON fields (#10246)
* Use pointers

* Use pointers

* Use pointers

* Update json_marshal_unmarshal.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-15 23:26:58 +00:00
Radosław Kapka
6dcf47675b E2E BellatrixE2EForkEpoch constant (#10240)
* Add `BellatrixE2EForkEpoch` constant

* change epoch

* Config param

* revert e2e update

* disable linter

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-15 21:06:14 +00:00
terence tsao
72aa782849 PayloadStatus reserve 0 enum (#10247)
* Leave 0 as UNKNOWN

* Leave 0 as UNKNOWN
2022-02-15 20:32:39 +00:00
terence tsao
cc637bad4a Fix ForkchoiceUpdatedResponse json tag (#10243) 2022-02-15 15:49:36 +00:00
Potuz
f5719f8c8e Fix updateCanonicalNodes (#10241)
* Fix updateCanonicalNodes

* fix logic

* terence's review
2022-02-14 21:30:26 +00:00
Rootul Patel
f550a964f5 Add path to Keystore (#10152)
* Add failing test

* Add path to keystore

* Assert encoded contains "path"

* Fix lint

Ran `bazel run //:gazelle -- fix`

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-02-14 09:57:15 -06:00
Radosław Kapka
c5c039fd6b Unify GenesisValidator(s)Root throughout the codebase (#10230)
* Unify `GenesisValidator(s)Root` throughout the codebase

* comments and literals

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-02-14 13:34:38 +00:00
Radosław Kapka
4d9947543f Native state feature flag (#10232)
* add flag

* working version

* rename flag

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-14 12:33:02 +00:00
Michael Neuder
7370c42bae Adding nil error check to connection status for ETH1 nodes (#10186)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-02-14 10:21:45 +00:00
Radosław Kapka
3c76cc3af5 Return state interface from native state constructors (#10208)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-14 17:51:22 +08:00
terence tsao
28af5bc601 Fix eth_getBlockByHash call (#10239)
* Update client.go

* Update client_test.go
2022-02-13 16:33:11 +00:00
terence tsao
c9f299b50a Clean up fork choice (#10226)
* Clean up fork choice

* Update beacon-chain/forkchoice/protoarray/proposer_boost_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/forkchoice/protoarray/store_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/forkchoice/protoarray/store_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update store.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-02-12 03:33:46 +00:00
terence tsao
9dfb385160 Pad fields to correct length (#10237)
* Pad fields to correct length

* Use constants

* builds

* Fix test

* Update BUILD.bazel

* tests

* passing

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-12 01:58:09 +00:00
Raul Jordan
b1774efeb7 Fix JSON Marshal/Unmarshal for Execution Block (#10238) 2022-02-12 01:06:28 +00:00
Potuz
4b3a723166 Do not stop on error loading synced tips (#10235)
* Do not stop on error loading synced tips

* Terence's review

* Update beacon-chain/blockchain/optimistic_sync.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-11 21:46:53 +00:00
terence tsao
50f253619e Fix engine client method names (#10234)
* Fix engine client method names

* Typo
2022-02-11 16:12:11 -05:00
james-prysm
753afb4fb2 small fix for web3signer (#10223) 2022-02-11 09:41:31 -05:00
Leo Lara
1835f54197 Refactor for DRY state getters_validator_test and getters_test (#10117)
* Refactor for DRY state getters_validator_test and getters_test

* Add BUILD.bazel to beacon-chain/state/testing

* Handle type assertion error in beacon-chain/state tests

* Fix with gazelle

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-02-11 14:48:29 +08:00
terence tsao
26a2311c82 Clean up outdated bellatrix code (#10225)
* Rm outdated merge code

* Update config_test.go

* Native

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-11 04:40:12 +00:00
Potuz
9f419bee7d Remove unnecessary reassignement (#10224) 2022-02-10 23:51:20 -03:00
james-prysm
d04eaf8fa0 Better DB Logging (#10215)
* initial commit

* initial commit

* reverting change

* reverting change

* Update beacon-chain/node/node.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/db/kv/kv.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-10 23:15:01 +00:00
Potuz
82ceb51548 Load Synced Tips when starting from Finalized State (#10196)
* Load Synced Tips when starting from Finalized State

* Terence's review #1

* Fix tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-10 22:18:42 +00:00
Potuz
6a2ef13b87 add metrics for synced tips (#10221)
* add metrics for synced tips

* Update beacon-chain/forkchoice/protoarray/metrics.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-10 18:48:46 +00:00
Potuz
72a2dd004b Fix integer overflow (#10222)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-02-10 15:20:36 -03:00
Raul Jordan
1a0e16a48b Initialize the Engine API Client in the Beacon Chain's Powchain Service (#10217)
* define engine caller

* gaz

* Update cmd/beacon-chain/flags/base.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-10 16:46:31 +00:00
Nishant Das
bff5c1e1a9 Handle Bellatrix Fork Epochs Correctly (#10219)
* handle it properly

* Update beacon-chain/sync/fork_watcher.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/sync/fork_watcher.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-10 16:15:55 +00:00
Nishant Das
3f5ce0cdca Handle a ZeroHash as our Eth1 Genesis Block Root (#10218) 2022-02-10 11:28:57 +00:00
terence tsao
d51b52d432 Add pow block protobuf (#10205)
* Add pow_block protobuf

* Revert hash changes

This reverts commit 4afb280385.

* Revert "Add pow_block protobuf"

This reverts commit 4afb280385.

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-09 22:27:30 +00:00
Raul Jordan
7b5a821c81 Implement Execution Engine Client (#10185)
* add in fixtures

* handle real fixtures

* marshaling issues

* ensure error handling works

* begin http client test

* custom type marshaling

* gaz

* custom type

* marshalers

* builds

* builds

* add to build file

* regen

* only root types

* bytes hash

* regen

* bytes hash

* regen

* regen

* alias magic

* revert proto

* more alias magic

* revert build files

* regen

* define marshal methods

* tests passing

* hexbytes

* rem

* rem

* comment

* gaz

* data

* fix conf

* http tests

* sync

* client tests passing

* deepsource

* gaz

* item

* exec block by hash

* begin tests

* http tests passing

* payload id bytes

* proper json encode decode for payload id

* gaz

* proper marshal

* merge engine client

* use big ints

* passes

* confirmed passing

* add in functional methods

* deepsource

* pb

* Update beacon-chain/powchain/engine-api-client/v1/client_test.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* latest

* tests passing again

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-09 20:54:56 +00:00
Mohamed Zahoor
f89265f2f7 Adding a new method in HeaderFetcher (#10212)
* Is optimisyic for root

* add the function in mock chainservice also

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-09 20:21:15 +00:00
james-prysm
c731b715ed Keymanager-Kind: fix cli kind parsing and allow for backward compatability (#10211)
* initial commit with unit tests

* reverting and adding in comments
2022-02-09 14:40:52 -05:00
Radosław Kapka
2a68c69d8f Beacon Node API Postman collection (#10213)
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-02-09 18:42:23 +00:00
terence tsao
a82fd7bf68 Fix merge transition configs (#10206)
* Add proper merge configs

* Switch to bytes

* Use string

* Update mainnet_config.go

* Rm unused

* Fix test

* Update config_test.go

* Update beacon-chain/blockchain/pow_block.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update pow_block_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-09 18:15:29 +00:00
james-prysm
9c540627ab default allowList for validator (#10210)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-02-09 17:18:49 +00:00
james-prysm
11f50453fc Keymanager api postman test collection (#10209)
* initial commit

* adding keymanager api repo url

* updated postman collection

* removing unused folder

* removing validator build

* Update hack/keymanager-api/README.md

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update hack/keymanager-api/README.md

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update hack/keymanager-api/README.md

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-02-09 16:48:08 +00:00
Potuz
5f8b01ccda Make UpdateSyncedTips safe for context cancellation (#10174)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-09 14:54:27 +00:00
Nishant Das
d800107927 Add Peer Scorer to Dev Mode (#10207) 2022-02-09 13:51:59 +01:00
Nishant Das
d3d5cfca0b Peer Score Evaluator (#9717)
* eval stuff

* change up

* update dep

* go mod tidy

* gaz
2022-02-09 09:53:25 +00:00
Potuz
18ef760ee5 Add DeleteBlock() (#10203)
* Add DeleteBlock()

* Raul review #1

* Update comments

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-02-08 23:26:26 +00:00
james-prysm
14e3e80df3 Keymanager api alignment (#10176)
* keymanager-api

* import updates to standards

* adding in more unit tests and code fixes to improve api experience

* deleting accidently generated files

* deleting accidently generated files

* adding in more test coverage

* fixing linter issue

* removing incorrect unit tests

* improving logic for accounts import

* linter fix

* addressing review comments

* fixing based on comments

* fixing nonzero root

* fixing protos

* regen protos

* adjusting protos again

* updating api specs

* fixing code and unit tests after specs changed

* fixing imports

* adding in required changes for api middleware also adding unit tests to catch changes for protos

* fixing deepsource issues

* fixing linting

* seeing if using pointers helps

* addressing comments

* updating bazel build

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-02-08 13:13:36 -06:00
Raul Jordan
3bea0e7896 ExecutionBlock Protobuf Type for the Merge (#10201)
* exec block

* exec block

* base fee

* regen

* exec block test

* fmt

* bytes for big ints

* engine pb

* passing

* add size

* bytes

* size

* size bytes

* passing
2022-02-08 18:06:02 +00:00
Radosław Kapka
8455656597 Use beacon state interfaces instead of concrete structs (#10195)
* return interface from testing/util

* remove usages of v1

* return interface from InitializeFromProto

* return interface from InitializeFromProto

* fix test

* fix interface visibility

* more fixes

* use InitializeFromProtoUnsafe in testing/util

* return early error from mock

* v2

* fix tests

* remove unnecessary assertion

* use struct in nil state test

* Revert "Auxiliary commit to revert individual files from 6bb528c2c5df2446ad18450009f63f44318d41a9"

This reverts commit 7d70238a301209f6dbfc8ff1d81b16e33b0bd67d.

* use struct in sync committee test

* v3

* use InitializeFromProtoUnsafe in mock

* use version information

* Revert "Auxiliary commit to revert individual files from 6bb528c2c5df2446ad18450009f63f44318d41a9"

This reverts commit 5d5e6f2884d21caec7530c16ad2a0d0d27c44aa1.

* revert changes to ClearPreGenesisData

* fix build error

* remove error from PreGenesisState

* bzl
2022-02-08 09:30:06 +00:00
terence tsao
af8bafd895 Spectest 1.1.9 (#10199) 2022-02-07 20:50:11 -08:00
terence tsao
c538a6c068 Add propose bellatrix block (#10189)
* Add propose bellatrix block

* Update BUILD.bazel
2022-02-08 01:29:11 +00:00
Raul Jordan
d5eb8392b6 Engine API Client Custom Marshaling (#10188)
* custom type marshaling

* gaz

* custom type

* marshalers

* builds

* builds

* add to build file

* regen

* only root types

* bytes hash

* regen

* bytes hash

* regen

* regen

* alias magic

* revert proto

* more alias magic

* revert build files

* regen

* define marshal methods

* tests passing

* hexbytes

* rem

* rem

* comment

* gaz

* data

* builds

* passing
2022-02-07 17:13:01 +00:00
Potuz
4906a0e6de add isOptimisticCandidateBlock and IsExecutionBlock (#10191)
* add isOptimisticCandidateBlock and IsExecutionBlock

* remove check

* fix tests

* Align with spec, reuse helper `ExecutionBlock`

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-02-07 01:26:20 +00:00
terence tsao
168e06e607 Save sync tips to DB (#10171)
* Save sync tips to DB

* Fix build

* Update process_block_test.go

* Copy map

* Revert back to nil

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-06 18:00:47 +00:00
Potuz
cf18d5dd42 Add SafeSlotsToImportOptimistically beacon configuration flag (#10190)
* Add SafeSlotsToImportOptimistically beacon configuration flag

* fix e2e

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-06 03:14:56 +00:00
Potuz
aeb6940935 Prune synced tips (#10143)
* Prune synced tips

* Terence's minor cleanups

* Update optimistic_sync.go

* go fmt

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-02-05 20:24:02 +00:00
Potuz
4020a603b6 Invalid synced tips (#10154)
* add synced_tips structure

* create optimisticStore type

* Add UpdateSyncedTips

* use context

* update comment

* use better context cancellation

* reinsert removed error from different PR

* Minor clean ups

* remove invalid nodes from forkchoice

* Update beacon-chain/forkchoice/protoarray/optimistic_sync.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Add check for tip status

* Terence's minor cleanups

* Potus' feedbacks

* Update optimistic_sync.go

* Add erorr

* Synced

* update weights

* Add context error check

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-03 22:57:15 +00:00
terence tsao
3a5cfab5f2 Add IsOptimistic chain info getter (#10184)
* Add  chain info getter

* Interface

* Test

* Update mock.go

* Go fmt

* Update beacon-chain/blockchain/testing/mock.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-02-03 21:30:07 +00:00
james-prysm
8e5ba13352 Fix Small Bug in Genesis Validators Root Validation Func (#10183)
* initial commit

* switching to table driven test
2022-02-03 20:58:25 +00:00
Nishant Das
e7e2f7850d Cleanup Multiclient E2E (#10180) 2022-02-03 14:14:26 +01:00
terence tsao
c8748260fd Implement is_valid_terminal_pow_block (#10172)
* add validator terminal block difficulties

* Fix test

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-03 05:28:23 +00:00
Nishant Das
a70febbf56 E2E Config Cleanup (#10178) 2022-02-03 04:53:09 +00:00
Raul Jordan
55ce1ccc33 Engine API Client Skeleton and Configuration (#10177) 2022-02-02 19:46:27 -08:00
Preston Van Loon
28e0dc5d09 Tool to print current eth1data votes (#9335)
* Initial progress on eth1data votes reporting tool

* gofmt

* add pyrmont genesis time to help txt

* Fix eth1voting tool

* correct Prater genesis time

* Update so it builds

* Add a README

* gofmt

Co-authored-by: rkapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-02 20:41:14 +00:00
Nishant Das
2b0f74904e Multiclient E2E With Lighthouse (#10020)
* save work

* current progress

* fix it more

* save progress

* fixes so far

* add signature test

* fix up changes so far

* change to latest

* update lighthouse version

* fix build

* fix again

* do one

* clean up

* fix build

* fix it

* fix test

* change tag

* remove e2e

* Update config/params/testnet_e2e_config.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* update

* Update sha

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
2022-02-02 19:13:52 +00:00
Nishant Das
5cf976e492 Merge State Cleanup (#10011)
* state related cleanup

* remove unused imports

* radek's review

* fix for native state

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-02 17:51:24 +00:00
Nishant Das
80aa4f483c Make Opt Max Cover Default (#10109)
* make default

* radek's review

* clean up

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-02 16:26:48 +00:00
terence tsao
fb92d518f9 Remove merge setting cli flags (#10162)
* Rm merge client setting

* Fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-02 15:33:48 +00:00
Nishant Das
1774188a17 Fix Certain Edge Cases With the Analyzer (#10164)
* fix analyzer

* fix other deadlock

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-02-02 14:11:31 +00:00
Radosław Kapka
ea31f096b5 Native beacon state: fieldtrie package (#10146)
* Native beacon state: `fieldtrie` package

* reuse ProofFromMerkleLayers

* more reuse + fixes

* more reuse + fixes

* move validateIndices to helpers

* remove TestAppendBeyondIndicesLimit

* do not duplicate package

* correct errors

* handle32ByteArrays function

* type switch

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-02-02 12:18:06 +00:00
Nishant Das
ea1698c1b3 Make Next Slot Cache Default (#10110)
* deprecate all of it

* handle empty slice

* fix bad bug

* fix it

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-02 11:40:46 +00:00
Nishant Das
0802c8fe66 add new methods in (#10173) 2022-02-02 14:56:07 +08:00
Raul Jordan
4322185d80 Use Execution Payload Defined in Engine V1 Namespace (#10170)
* payload use

* incl

* cloners

* regen pb

* gen ssz

* use proper payload

* broken imports

* build

* state

* builds

* proper engine

* builds

* fmt

* baz

* ssz

* baz

* fmt

* regen ssz

* receipts root

* regen pb

* ssz

* receipts root

* ci lint

* proto

* test pkg

* bazel
2022-02-02 03:30:35 +00:00
Hugo Masclet
cab0a4c9df Fix typo in CONTRIBUTING.md (#10168)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-02-02 00:01:12 +00:00
Michael Neuder
8b2b109939 Rename imported keymanager to local. (#10155)
* imported -> local

* reverting to the state of the PR at eb1e3c3d1

* use changes from develop

* del

* rem patch

* patch

* rename to local

* gazelle

* add back build

* imported rename

* gaz

* local

* merge fix + remove proto changes

* comment revert

* build

* gofmt and one new reference

* gofmt pt2

* Update validator/accounts/wallet_edit.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update validator/rpc/accounts.go

* rename

* gaz

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-02-01 19:54:19 +00:00
james-prysm
59be30e9f4 Web3Signer: metrics, tracing, and code coverage (#10161)
* initial commit for metrics, tracing, and code coverage

* fixing unit test, logs, and naming conventions

* Update validator/keymanager/remote-web3signer/internal/client.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/internal/client.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* adding required dependency

* adding more error annotations

* adding in negative test

* fixing bazel build

* gaz

* Update validator/client/validator_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/validator_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/service.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/internal/client.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/validator_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/validator_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* addressing review comments

* Update validator/keymanager/remote-web3signer/internal/client_test.go

* reverting channel direction

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-02-01 18:28:41 +00:00
Mohamed Zahoor
b27d535ade Storing synced tips in DB (#10136)
* DB support for storing synced tips

* gazel fix

* changed the synced_tips to map

* import formatting

* added function docs

* fix potuz comments

* fix prestons comment

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-02-01 16:17:28 +00:00
Nishant Das
e83c9d5862 Revert "Replace a Few IntFlags with Uint64Flags (#9959)" (#10163)
* Revert "Replace a Few IntFlags with Uint64Flags (#9959)"

This reverts commit 790bf03123.

* fix

* fix
2022-02-01 08:51:17 +00:00
Raul Jordan
eef2122a9e Engine API Protobuf Definitions (#10150)
* define proto types for engine api

* proto lint

* prettify

* build bazel

* generated pb

* txs field

* regen

* ssz gen marshal

* fastssz

* ssz

* fmt

* json marshal unmarshal custom

* exec json tests

* passing tests

* deepsource

* gaz

* gaz

* deps

* ssz max

* regen proto

* regen ssz
2022-02-01 01:47:34 +00:00
Radosław Kapka
6406afc6cf Native beacon state: v3 (#10139)
* Native beacon state: v3

* nogo_config

* remove duplicated bazel rule

* gzl

* fix unnecessary assignment

* review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-02-01 08:32:39 +08:00
terence tsao
8a01d412f5 Implement fork choice spec tests (#10159)
* starting

* Reimplement store

* begin on proposer boost

* implement fork choice proposer boost algorithm

* boosting

* gaz

* add mutexes and previous root

* comment on compute proposer boost

* safe

* rem todo

* reset and add tests

* unit test for proposer boost score

* boost works

* Can process block

* Basic test case passing

* ex ante

* test

* propoer test

* More progresses

* More fixes

* rm unused pieces

* Refactor, add phase 0

* locks

* vanilla ex-ante attack

* test similar to spec test

* works works works

* boost test working for num votes > proposer boost weight

* commentary fixes

* rem unused

* comments

* Proposer boost use store time

* Reset proposer root

* debugging

* passing

* Rm unused visibility imports

* Move update head to better place

* Fix deepsrc complains

* Fix more complains

* Raul's feedback

* Use correct byte lengths

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-01-31 19:03:48 +00:00
james-prysm
aed6e13498 Web3Signer: CLI implementation (#10056)
* initial commit for cli integration of web3signer

* resolving conflicts and execution

* remove aggregation slot from proto

* rem aggregation slot

* define a sync message block root struct

* fix sync message name

* sync message block root struct

* amend where sync committee block root is used

* altered switch statement to return correct json request by type

* fixing fork data import, types, and unit tests

* reverting unwanted changes

* reverting more unwanted changes

* fixing deepsource issues

* fixing formatting

* more fixes for deepsource and code clean up

* only want to fetch once for fetch validating public keys

* adding more comments

* new unit tests for requests and fixing a mapper issue

* Update validator/client/validator.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/accounts/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* adjusting comment

* adjusting comment

* fixing import organization

* including more unit tests

* adding new cli edit

* adding in checks for wallet initialize

* adding web3signer flags to main.go

* some how resolved files did not save correctly

* adding in check to make sure web flag only works with types imported and derived

* Update validator/client/sync_committee.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/aggregate.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/accounts/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/main.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/flags/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/flags/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* reverting changes that accidently got checked in

* reverting

* reverting

* continuing to revert unintenteded changes

* reverting

* removing more unneeded changes

* addressing review comment

* initial refactor

* adding in more clarifying comments

* fixing mock

* resolving desource issues

* addressing gosec scan for helper go file

* addressing gosec

* trying to fix bazel build

* removal of interface to fix build

* fixing maligned struct

* addressing deepsource

* fixing deepsource

* addressing efficiency of type checking

* fixing bazel test failure

* fixing go linter errors

* gaz

* web changes

* add w3signer

* new kind

* proper use

* align

* adding prysm validator flags to help flags list

* addressing root comment

* ci lint

* fixing standardapi tests

* fixing accounts_test after removal of keymanager from rpc server

* fixing more unit tests

* Update cmd/validator/flags/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/flags/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/service.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/service.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* addressing missed err checks

* fixing mock tests

* fixing gofmt

* unskipping minimal e2e test and removing related TODOs

* Update testing/endtoend/components/validator.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update testing/endtoend/components/validator.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* adding some error wrapers to clarify failure point

* fixing bazel build with new error checks

* taking preston's advice to make test fail faster to understand what's going on with the test

* checking if genesis validators root is not zero hash

* adding check for genesis validators root giving zero hash

* fixing missing dependency

* adding check for wallet

* log all

* fixing errors for http responses

* switching marshal to pretty print

* adding pretty sign request test

* fixing base url setting

* adding in check for web3signer and temporary wallet instead of having to open the wallet

* refactoring web3signer to not require wallet

* bazel build fix

* fixing gazelle build

* adding content type of request

* fixing more bazel

* removing unused code

* removing unused comments

* adding skip test back in

* addressing a validation and error message

* fix parse

* body

* fixing logic for datadir

* improving error handling

* show resp

* fix

* sign resp as str

* point of pointer remove

* sign resp

* unmarshal sig resp

* read body as str

* adding more verbose logging

* removing unused result

* fixing unit test

* reconfiguring files to properly nest code and mocks

* fix build issue

* using context when using client function calls

* fixing based on suggestion

* addressing comments

* gaz

* removing defined max timeout

* reverting json print pretty

* Update validator/accounts/wallet_edit.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* removing unneeded code restrictions

* should not introduce new code that may impact existing key manager types

* adjusting comments

* adding in json validation

* running go mod tidy

* some logging

* more logs

* fixing typo

* remove logs

* testing without byte trim

* fixing order or properties

* gaz

* tidy

* reverting some logs

* removing the confusing comments

* Update validator/client/aggregate.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/aggregate.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* addressing pr comments

* editing bytes test

* Run gazelle update-repos

* run gazelle

* improving unit test coverage

* fixing text

* fixing a potential escaped error

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-01-31 10:44:17 -06:00
Potuz
26ec352bdc Update synced tips (#10129)
* add synced_tips structure

* create optimisticStore type

* Add UpdateSyncedTips

* use context

* update comment

* use better context cancellation

* reinsert removed error from different PR

* Minor clean ups

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-01-31 15:05:05 +00:00
Nishant Das
ce712a1a3e Fix P2P Test (#10157)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-31 12:23:25 +00:00
Preston Van Loon
98470fdb13 Update rules_go to v0.29.0 (#10064)
* Update rules_go, delete patch, fix analyzers, remove old deployContract code, and prune old dependency which was broken in rules_go update

* Undo some WORKSPACE diffs

* Gazelle

* remove nofuzz, fix build issue

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-31 07:06:07 +00:00
Preston Van Loon
ef906cf704 Update fuzzbuzz.yaml (#10151)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-29 16:55:31 +00:00
Raul Jordan
65ec437ce5 Fork Choice Proposer Boost Changes (#10083)
* begin on proposer boost

* implement fork choice proposer boost algorithm

* boosting

* gaz

* add mutexes and previous root

* comment on compute proposer boost

* safe

* rem todo

* reset and add tests

* unit test for proposer boost score

* boost works

* ex ante

* test

* propoer test

* vanilla ex-ante attack

* test similar to spec test

* works works works

* boost test working for num votes > proposer boost weight

* commentary fixes

* rem unused

* comments

* boost necessary to sandwich unit test

* sad path

* gaz

* Update beacon-chain/forkchoice/protoarray/proposer_boost.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/forkchoice/protoarray/proposer_boost.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/forkchoice/protoarray/proposer_boost_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/forkchoice/protoarray/proposer_boost_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/forkchoice/protoarray/proposer_boost_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/forkchoice/protoarray/proposer_boost_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/forkchoice/protoarray/proposer_boost_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/forkchoice/protoarray/proposer_boost_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* gaz

* radek comments

* reset boosted root in on new slot

* pass

* broken tests

* cfg test

* modify e2e config

* Revert e2e changes

* Test

* Test again

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-01-29 16:32:01 +00:00
Preston Van Loon
58733a9f40 go 1.18 fuzz tests: initial testing (#10063)
* Add basic fuzz test

* Add a few tests for HTR utils

* gofmt

* Add requirement for fuzz gotag

* Run gaz and fix tags

* Update pubsub_fuzz_test.go

I'll fix this later

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-29 08:01:58 +00:00
terence tsao
762ea6dce1 State balance cache: handle input zeros (#10144)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-28 17:14:13 +00:00
terence tsao
eef1730f9e Bellatrix sync changes (#10097)
* Add bellatrix sync changes

* Fix tests

* Update validate_beacon_blocks.go

* check state and block versions

* Sync with develop

* Update rpc_chunked_response_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-28 16:26:52 +00:00
Potuz
de7f7d783e Remove unused code (#10142)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-01-28 14:52:30 +00:00
Nishant Das
5eafb80091 fix it (#10145) 2022-01-28 12:07:41 +08:00
Potuz
cb0c382bf3 Fix Fork Choice prunning (#10137)
* Fix Fork Choice prunning

* Use copy node. Update comments and var names.

* rm unused err var

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-01-27 19:17:19 +00:00
Radosław Kapka
15485f50c3 Fix native state imports (#10140) 2022-01-27 11:02:16 +00:00
Radosław Kapka
9ee00f09a1 Native beacon state: v2 (#10121)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-01-27 09:42:32 +01:00
Radosław Kapka
72817a6d0e Unify validator status response in RPC (#10128) 2022-01-26 14:23:53 -05:00
Preston Van Loon
943dec525c Refactor mock test helpers (#10133)
* Delete deploy contract tool. Move mock to its own package as testonly with some helper functions

* gofmt contracts/deposit/mock/mock.go

* move stategen mock.go to its on testonly pkg

* move password_reader_mock.go to mock testonly package

* move mock keymanager to its own testonly package

* move attestations mock

* move voluntaryexits mock

* Move slashings mock to mock pkg

* move the slasher mock

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-26 14:48:20 +00:00
Leo Lara
d9799e6b6c #10036 Replace codename Merge with Bellatrix (2nd step) (#10116)
* Rename BeaconBlockBodyMerge to BeaconBlockBodyBellatrix

* Rename SignedBeaconBlockMerge to SignedBeaconBlockBellatrix

* Rename CopyBeaconBlockMerge to CopyBeaconBlockMerge

* Rename NewBeaconBlockMerge to NewBeaconBlockBellatrix

* Rename BeaconBlockMerge to BeaconBlockBellatrix

* Rename some comments and strings in pkg proto: Merge -> Bellatrix

* Rename PbMergeBlock to PbBellatrixBlock

* Many renames of merge -> bellatrix in proto package

* Rename some Merge -> Bellatrix in beacon chain package

* More names

* Fix formating in config/params/config.go

* Rename Merge -> Bellatrix in proto/prysm/storage

* Several renames and corrections Merge -> Bellatrix

Co-authored-by: Potuz <potuz@potuz.net>

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Potuz <potuz@potuz.net>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-01-26 07:24:47 +00:00
Potuz
f0fd29d367 Is optimistic (#10124)
* add synced_tips structure

* create optimisticStore type

* Add ForkChoice.Optimistic()

Adds the logic implementing whether a synced block is optimistic or not.

* Terence's bug catch

* Reinforce warning

* Preston's review

* add comment on optimistic sync
2022-01-26 01:42:18 +00:00
terence tsao
22982e5221 Add on_tick from spec (#10130)
* Add `on_tick` from spec

* Update BUILD.bazel
2022-01-25 22:07:57 +00:00
terence tsao
5a505509c3 Use store in blockchain service (#10125)
* Use store in blockchain service

* handle nil cases
2022-01-25 18:47:39 +00:00
Radosław Kapka
588658a649 Extract conditional slot processing to new function (#10114)
* Extract conditional slot processing to new function

* rename function
2022-01-25 14:52:18 +00:00
terence tsao
ae31eed013 Add blockchain store for checkpoints (#10118)
* Add store with tests

* Rm genesis time

* Previous finalized

* Go fmt

* Update beacon-chain/blockchain/store/type.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-01-24 19:53:05 +00:00
Nishant Das
e26cde5e09 Make Balance Trie Computation Default (#10112)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-24 11:36:10 +00:00
Radosław Kapka
5cc201288d Native beacon state: v1 (#10086) 2022-01-24 11:24:38 +01:00
Nishant Das
b4fa626aff Use Batch Decompression For Signatures (#10105)
* use batch decompress method

* terence's review

* gosimple

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
2022-01-21 22:57:29 +00:00
Nishant Das
498e5efed1 Make Get Block Optimization Default (#10113)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
2022-01-21 22:22:44 +00:00
Nishant Das
f563d00e79 Filter Duplicates Messages For Sync Contributions (#10106)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-21 18:25:59 +00:00
Nishant Das
ff50448ca2 Make Batch Verification Of Gossip Default (#10111)
* make default

* add it back

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-21 17:26:42 +00:00
Preston Van Loon
26120e26cb Resolve issues with G304 (#10115) 2022-01-21 16:52:31 +00:00
terence tsao
8d610319bc Remove commented code (#10104)
* Update service.go

* go fmt

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-20 18:15:05 +00:00
terence tsao
288b38be8e Bellatrix p2p changes (#10072)
* Add `process_execution_payload` to `process_block`

* bring in bellatrix p2p changes

* lint

* fix testcase to use MaxChunkSize as config to change allowed size instead of config

* add left out config params

* Fix TestForkSchedule_CorrectNumberOfForks

* fixed Nishant's comments

* gofmt

* add switch case

Co-authored-by: Mohamed Zahoor <zahoor@zahoor.in>
Co-authored-by: Zahoor Mohamed <zahoor@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-01-20 22:12:15 +08:00
Nishant Das
af06bb9737 Do Not Copy State During Upgrade (#10102)
* no copy

* no copy

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-20 06:54:21 +00:00
Nishant Das
52f1190b17 Cleanup Up Resources Correctly (#10103)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-20 06:21:06 +00:00
Preston Van Loon
7df01feb58 Remove libfuzzer / afl based builds in favor of go 1.18 fuzz testing (#10065)
* Remove libfuzzer / afl based builds in favor of go 1.18 fuzz testing

* rm beacon-fuzz archive

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-20 04:51:02 +00:00
Nishant Das
49c73e65a1 make it better (#10101) 2022-01-20 10:37:27 +08:00
terence tsao
06c3306ac2 Update stategen for Bellatrix (#10089)
* Update stategen to for Bellatrix

* Update BUILD.bazel

* Update replay.go
2022-01-19 22:18:11 +00:00
Preston Van Loon
c3228cd5a7 e2e: Add web3signer component (#10088)
* Add initial web3signer binary

* Add support for web3signer component in e2e

* revert some changes

* Reference https://github.com/ConsenSys/web3signer/issues/485 in commentary

* gofmt

* Add notice about java 11 requirement

* Sec issue, revert to 0750 permissions

* remove unused param

* remove unused import

* use hexutil

* Fix yaml struct tags. See https://github.com/ConsenSys/web3signer/issues/485\#issuecomment-1015994840

* fmt

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-01-19 16:22:23 +00:00
james-prysm
8da8855ad5 Web3Signer: Sign Method Implementation (#10084)
* breaking up changes from cli pr

* reverting some changes, adding in changes from remote-web3signer

* adding raul's change

* adding fork info to signing calls

* fixing imports

* gaz

* fixing gofmt

* removing unneeded comment

* Update validator/client/aggregate.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update validator/client/sync_committee.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* addressing comments

* revert proto changes

* proto changes

* reserve

* reserve

* switching to passing slot from passing fork, using slot to find the fork

* removing unneeded check

* fixing missed unit test

* optional

* optional

* gaz

* improving some definitions with constants

* improving some definitions with constants

* rem opt

* rem

* gaz

* moving mocks to its own folder

* adding in bazel field to fix

* fixing type check error

* fixing build

* fixing strict imports

* fixing dependencies

* changing bazel build

* changing bazel build

* changing bazel build

* removing testing only dependency

* removing dependency on testing util package

* update bazel build

* Update checktags_test.go

* Update active_balance.go

* Update sync_committee_minimal.go

* Update sync_committee_mainnet.go

* Update active_balance_disabled.go

* Update committee.go

* Update committee_disabled.go

* Update sync_committee.pb.gw.go

* Update powchain.pb.gw.go

* Update proposer_indices.go

* Update proposer_indices_disabled.go

* Update sync_committee.go

* Update mainnet.go

* Update p2p_messages.pb.gw.go

* Update finalized_block_root_container.pb.gw.go

* Update beacon_block.pb.gw.go

* Update attestation.pb.gw.go

* Update secret_key_test.go

* Update beacon_state.pb.gw.go

* Update version.pb.gw.go

* Update sync_committee.pb.gw.go

* Update sync_committee_disabled.go

* Update mainnet_test.go

* Update minimal.go

* Update signature_test.go

* Update gocast.go

* Update cgo_symbolizer.go

* Update validator.pb.gw.go

* Update beacon_state.pb.gw.go

* Update signature.go

* Update public_key_test.go

* Update minimal_test.go

* Update checktags_test.go

* Update bls_benchmark_test.go

* Update public_key.go

* Update secret_key.go

* Update aliases.go

* Update init.go

* Update stub.go

* Update journald_linux.go

* Update attestation.pb.gw.go

* Update config_utils_develop.go

* Update stub.go

* Update stub.go

* Update beacon_block.pb.gw.go

* Update validator.pb.gw.go

* Update node.pb.gw.go

* Update config_utils_prod.go

* Update journald.go

* Update beacon_block.pb.gw.go

* Update beacon_chain.pb.gw.go

* Update beacon_state.pb.gw.go

* Update events.pb.gw.go

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* addressing comments from review

* updating length of comment

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update stub.go

revert changes

* Update validator/client/aggregate_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* addressing final comments

* fixing gofmt

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-01-18 14:31:58 -06:00
Mohamed Zahoor
4ca04dcc4b Add storage wrapper for BeaconState (#9963)
* adding proto wrapper for BeaconState

* remove BUILD.bazel file spelling mistake

* address terence review comments

* remove capitalization

Co-authored-by: Nishant Das <nishdas93@gmail.com>

* renamed as storage, use v1alpha1 ojects

* fix lint in the generated file

* store state as ssz bytes

* added a seperate version for managing storage

* goimports generated files to satisfy lint

* changed proto to oneof

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-01-18 15:21:56 +00:00
Radosław Kapka
68ef8ae99a Remove CanonicalStateChan (#10093)
(cherry picked from commit 69a2ffd7c9)
2022-01-18 14:24:47 +00:00
Leo Lara
64c02c405b Move ErrNilValidatorsInState from one in each state version to a common one (#10074)
To increase DRY and enable DRY in tests and other users of the Beacon Chain state package,
an error that was duplicated unnecessarily in each version of the state is moved to the root
Beacon Chain state package.
2022-01-18 08:19:20 +00:00
terence tsao
33d1ae0792 Eth2api: GetBlockSSZ supports bellatrix block (#10077)
* add rpc and tests

* rename
2022-01-17 02:30:09 +00:00
Preston Van Loon
182bd615ac refactoring: Deduplicate code (#10090)
* Deduplicate sync committee indices from state

* Deduplicate code in listblocks RPC endpoints, which only differ in response object

* Deduplicate test code in config/fieldparams

* Delete stale benchmark target. This is already included in go_default_test

* deduplicate test cases in TestIsSlashableValidator_OK and fix blst
BUILD file

* Deduplicate TestStore_IsFinalizedChildBlock

* Revert crypto/bls/blst/BUILD.bazel

* Deduplicate TestStore_SaveBlock_NoDuplicates

* Use a generic wrapper for beacon blocks to streamline test

* Deduplicate TestStore_BlocksCRUD

* Deduplicate TestStore_BlocksHandleZeroCase

* Deduplicate TestStore_BlocksBatchDelete

* deduplicate TestStore_BlocksHandleInvalidEndSlot

* Deduplicate TestStore_BlocksCRUD_NoCache

* Deduplicate common block test setup. Make TestStore_Blocks_FiltersCorrectly support multiple forks in test

* Deduplicate the rest of these tests

* lint

* Deprecation warning

* Add test for WrappedSignedBeaconBlock

* Test error path
2022-01-16 18:44:42 +00:00
terence tsao
3c54eb1cf6 Add propose block bellatrix (#10085)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-01-14 21:24:22 +00:00
Michael Neuder
9166f40a04 Eth1 connections (#10073)
* Adding Eth1 connections data message to service definition

* PoC implementation for ETH1 connections RPC

* naming revisions

* cleanup

* register powchain info fetcher

* regenerate protos

* gazelle Bazel

* fixing comments

Co-authored-by: Michael Neuder <michaelneuder@Michaels-MacBook-Pro.local>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-01-14 11:13:04 -05:00
Preston Van Loon
98990ba280 p2p: Update libp2p and libp2p-pubsub (#10082)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-14 14:06:20 +00:00
Radosław Kapka
9ace518539 Custom types for the native beacon state (#10080)
* custom-types package

* split types into distinct files
2022-01-14 13:25:41 +00:00
Chase Jeter
0ea4b02b8b Added analyzer for detecting recursive/nested mutex read locks. (#10066)
* Added analyzer for detecting recursive/nested mutex read locks.

* Added type assertion checks and removed unused 'iTypes' directory

* Clean up

* Bazel file fixes

* Cleaned up code and added comments.

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-01-14 17:12:21 +08:00
Radosław Kapka
d73794ce72 Make Radek and Nishant code owners of duplicated state code (#10079)
* Make Radek and Nishant code owners of the native state feature

* newline

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-01-13 21:16:42 +00:00
Preston Van Loon
2ad0ad5877 Spectests: Refactor ssz_static tests to deduplicate code and test custom HTR (#10081)
* Copy over refactoring efforts from PR #9402

* Remove unused type

* godoc commentary

* Better example err msg

* Better example err msg
2022-01-13 19:52:39 +00:00
Radosław Kapka
963acefe12 Split state package into state-proto and state-native (#10069)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: kasey <489222+kasey@users.noreply.github.com>
Co-authored-by: Dan Loewenherz <dloewenherz.adm@gmail.com>
Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
Co-authored-by: Fredrik Svantes <fredrik@ethereum.org>
Co-authored-by: Leo Lara <leolara@users.noreply.github.com>
2022-01-13 12:23:53 +01:00
Radosław Kapka
0ee85bcbab Do not reuse endpoint struct between API Middleware requests (#10076) 2022-01-12 18:58:32 +00:00
Preston Van Loon
5fb0bcfacd tools/deployContract: Remove specific k8s code (#10075) 2022-01-12 18:31:38 +00:00
james-prysm
f52a214cce Web3Signer: Object Mapper and Types (#10061)
* initial commit

* fixing some types and renaming file

* initial map function commit and moving to a v1 folder

* adding in remaining mapper functions and comments for types and initial commit on unit tests

* adding readme

* fixing unit tests and errors caught

* fixed a few missed properties and names

* updating error handling to be more descriptive

* Update validator/keymanager/remote-web3signer/client_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/v1/custom_mappers.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/v1/custom_mappers.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* fixing comments

* fixing comments

* adding more error checks

* Update validator/keymanager/remote-web3signer/client.go

* fmt and bazel build

* fixing unit tests

* lint needed

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-01-11 17:15:40 -05:00
terence tsao
88b94eae18 Export reverse byte order function (#10040)
* Reverse byte order

* Update BUILD.bazel

* Go imports

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-11 18:42:03 +00:00
Potuz
3d02addfe4 Update README.md (#10071)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-01-11 16:32:07 +00:00
Raul Jordan
53f7030871 Initialize Merkle Layers and Recompute Dirty Fields in BeaconState Proofs (#10032)
* handle trie recomputes

* recomp

* state trie

* args

* deep source

* amend test

* proofs test more fixes

* tests pass
2022-01-11 17:36:03 +08:00
terence tsao
84335b0084 Bellatrix spec tests (#10014)
* Add merge spectests

* fix build

* gazelle

* operation

* all tests should pass

* go fmt

* fix test

* fix dependency

* rm unused constants

* rename to bellatrix

* Gaz

* Rm unused files

* Use Bellatrix for test names

* Add more spec tests

* Gaz and fix test

* sync with dev

* Preston's feedback

* Go fmt

* sync with dev

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-10 18:27:08 +00:00
Leo Lara
b1c2454658 #10036 Replace codename Merge with Bellatrix (1st step) (#10044)
* Rename BeaconStateMerge to BeaconStateBellatrix

* Rename version.Merge to version.Bellatrix

* Rename ComputeFieldRootsWithHasherMerge to ComputeFieldRootsWithHasherBellatrix

* Rename test names to Bellatrix

* Rename comments and strings to Bellatrix

* Fix formatting in a few files

* Revert wrong renaming in test name

* Revert renaming to Bellatrix in mainnet_config.go

* Revert renaming of db key without migration

* Regenerate from proto changes

* Rename new use of already renamed symbols

* gofmt and goimports after regenerating protofiles

* revert weird imports

Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
2022-01-10 16:47:30 +00:00
Fredrik Svantes
29a8b6c08f Updating from buster to bullseye (#10025)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-01-10 15:48:43 +00:00
Radosław Kapka
3bce9df382 Allow comma-separated event topics (#10052) 2022-01-07 17:17:29 +00:00
terence tsao
8ee3019954 Can upgrade to Bellatrix in process slots (#10062)
* Add `process_execution_payload` to `process_block`

* Add upgrade bellatrix to process slots

* Add upgrade bellatrix to process slots

* gazelle
2022-01-07 16:47:33 +00:00
terence tsao
ef1227b14c Add process_execution_payload to process_block (#10060) 2022-01-06 19:36:39 +00:00
Preston Van Loon
80b7f691ae p2p: no alloc extract of gossip digest (#10055)
* Extract gossip topic without a full copy via strings.Split

* Add basic benchmark

* gofmt

* Add empty topic test case

* Remove nl

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-01-06 18:17:51 +00:00
terence tsao
c69bce5d84 Use fieldparams for BLS public key (#10042)
* Use fieldparams for pubkey length

* Fix validator tests

* fix more tests

* fix mock validator

* Fix typo

* bunch of typos

* Update bytes.go

* Update BUILD.bazel

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-06 17:33:08 +00:00
Fredrik Svantes
ad06230291 Updating eip3076_spec_tests and eth_networks URLs (#10057)
From old eth2-clients to new eth-clients

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-01-06 16:58:37 +00:00
terence tsao
a1a301ad5d process payload (#10054) 2022-01-06 08:22:38 -08:00
james-prysm
d28ae62d02 Web3signer keymanager: key manager implementation (#10016)
* initial commit for work on web3signer-keymanager

* WIP key manager client

* adding in more valid unit tests

* refactoring some of my code to work better with current implementations

* adding comments, some error situations and fixing unit tests

* updating go.mod

* gaz

* addressing sandbox errors on test

* fixing more sand box debug items

* missed yet another error case on my test

* reverting go.mod

* addressing comments

* missed removal of unit test

* removing some deepsource issues

* adjusting based on review comments

* addressing comments

* adding the missed types

* changing public keys to only pull once

* fixing commented code format

* gaz

* Update validator/keymanager/remote-web3signer/keymanager.go

updating comments

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/client.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* fixing conflicts

* addressing comments

* fixing deepsource issues

* more deep source issue fix

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-01-05 14:32:46 -06:00
terence tsao
d6a02833dc Add HTR for txs (#10030)
* Add htr for txs

* gaz

* minimal

* test pack
2022-01-05 18:36:39 +00:00
terence tsao
fade8aa7a9 Add validate payload helpers (#10028)
* Add merge complete helper

* Update BUILD.bazel

* minimal

* Add validate payload helpers

* gaz

* update tests

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-01-05 15:28:58 +00:00
terence tsao
8604832499 Use prysm require test lib (#10039)
* Use prysm require test lib

* Fix typo

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-05 07:06:51 +00:00
terence tsao
ceabee0198 Update Transactions and ExtraData non empty check (#10049) 2022-01-03 23:53:39 +01:00
terence tsao
ebedf481f8 Update bazel version to 4.2.2 (#10045) 2021-12-27 23:51:23 +00:00
terence tsao
9f27af43cb Spectest 1.1.8 (#10041)
* Update spec test to v1.1.8

* Update names

* Update network repo

* Update WORKSPACE

* Better names

* Update beacon-chain/core/time/slot_epoch.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-12-26 23:18:28 +00:00
terence tsao
09690590f0 Update spec test to v1.1.7 (#10033) 2021-12-23 21:33:41 +00:00
terence tsao
e3c8491177 Add is_execution_enabled helper (#10027) 2021-12-22 15:03:54 -08:00
terence tsao
bfc90e9abb Add is_merge_block helper (#10026)
* Add merge complete helper

* Update BUILD.bazel

* minimal

* Add is_merge_block helper

* rename to `IsMergeBlock`
2021-12-22 15:52:50 +00:00
Raul Jordan
d173a6e695 Merkle Proof Support for BeaconState Finalized Root and Sync Committees (#10029) 2021-12-19 17:02:42 -08:00
Potuz
7f857ae23a Fix sync commitee endpoint (#10015) 2021-12-18 17:04:30 +01:00
terence tsao
23e39d3d64 Fix state alias v3 (#10022) 2021-12-16 22:44:43 +00:00
terence tsao
ec65c442c4 Minor cleanups on monitor service (#10013)
* Minor cleanups on monitor service

* Update tests

* Update process_block_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-15 20:42:20 +00:00
terence tsao
b38b0186b8 Use signature length from fieldparams (#10019)
* Use bls sig length from fieldparams

* fmt

* fix tests

* fix tests

* fix tests

* Update tags_test.go

* fix tests

* Update BUILD.bazel
2021-12-15 15:14:30 -05:00
terence tsao
4bf935928b Use field params root length (#10012)
* Use config length 32

* Add root length to minimal

* Fix build

* Gaz
2021-12-14 18:42:05 +00:00
terence tsao
acc4ee8d9d Add beacon state upgrade to merge version helper function (#9981) 2021-12-14 01:18:27 -08:00
terence tsao
7b7ed87ad8 Add more merge processing (non substrantive) (#10000)
* Add more merge processing

* Update tests

* raul's feedback

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-12-13 18:04:37 +00:00
Nishant Das
5ab88da183 Use Static Sizes for State Fields (#10007) 2021-12-13 10:16:23 +01:00
Raul Jordan
d6338f6042 Update Web UI Version to v1.0.2 (#10009)
* latest web ui

* fmt

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-12-11 11:36:00 -06:00
Antonio Sanso
c84c6ab547 Add Missing spec tests in sss_static.go (#10003)
* Update ssz_static.go

* Update ssz_static.go

* Update testing/spectest/shared/phase0/ssz_static/ssz_static.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Add Missing spec tests in sss_static.go

* Update testing/spectest/shared/altair/ssz_static/ssz_static.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-12-10 15:18:57 -06:00
Radosław Kapka
52d8a1646f Standardize config overriding in tests across repo (#9998)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-10 04:18:47 +00:00
kasey
3c61cc7d8a allow checkpoint or genesis origin; refactoring (#9976)
* allow checkpoint or genesis origin; refactoring

some quick readability improvements and simplifying the logic enforcing
the startup ordering of the attestation processing routine

* address PR feedback

* gofmt

* Update beacon-chain/blockchain/receive_attestation.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Apply suggestions from code review

use log.WithError for aggregation friendliness

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

Co-authored-by: kasey <kasey@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-09 22:23:00 +00:00
Raul Jordan
37ca409cc1 Add More Fields to Remote Signer Sign Request (#10004)
* add in new fields

* sign request added

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-09 21:53:54 +00:00
Raul Jordan
b381ad49b5 Add Justifications for Gosec Ignored (#10005)
* pin gosec

* edit

* go back to master

* justifications

* Update crypto/bls/blst/signature.go

* proper format

* gosec
2021-12-09 19:40:48 +00:00
james-prysm
00c3a7dcaf Web3signer: Add an HTTP Client Wrapper to Interact with Remote Server (#9991)
* initial commit for web3signer code work in progress

* adding more functions for web3signer

* more improvements to unit tests and web3signer functions

* fixing unit test

* removing path construction

* fixing failing unit test

* adding more happy path unit tests

* fixing unit tests

* temp removing keymanagerfiles being wip

* removing some comments

* Update validator/keymanager/remote-web3signer/client.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/client.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* making errors lowercase

* addressing review comments

* missed resolving a conflict

* addressing deepsource issues

* bazel test and gazelle

* no lint

* deadcode

* addressing comments

* fixing comments

* small fix for readability

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-12-09 09:33:53 -06:00
james-prysm
d66edc9670 fix cors middleware (#9999)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-12-08 17:13:28 -05:00
Raul Jordan
4b249607da Return ERROR Status in ImportKeystores standard API endpoint if slashing protection fails (#9995)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-08 19:52:05 +00:00
Radosław Kapka
02483ba89c Add missing config cleanup in tests (#9996)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-08 19:24:16 +00:00
terence tsao
9629c354f1 Use merge configs for processing (#9982) 2021-12-08 11:02:53 -08:00
Raul Jordan
886332c8fa Add Safe Sub64 Method (#9993)
* add sub 64

* add sub

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-08 09:44:54 +00:00
Raul Jordan
424c8f6b46 API Middleware for Keymanager Standard API Endpoints (#9936)
* begin the middleware approach

* attempt middleware

* middleware works in tandem with web ui

* handle delete as well

* delete request

* DELETE working

* tool to perform imports

* functioning

* commentary

* build

* gaz

* smol test

* enable keymanager api use protonames

* edit

* one rule

* rem gw

* Fix custom compiler

(cherry picked from commit 3b1f65919e04ddf7e07c8f60cba1be883a736476)

* gen proto

* imports

* Update validator/node/node.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* remaining comments

* update item

* rpc

* add

* run gateway

* simplify

* rem flag

* deep source

Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-07 20:26:21 +00:00
terence tsao
cee3b626f3 make db merge compatible (#9987)
* make db merge compatible

* util merge.go per @rauljordan

* Go fmt

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-07 18:51:05 +00:00
Radosław Kapka
5569a68452 Code cleanup (#9992)
* Value assigned to a variable is never read before being overwritten

* The result of append is not used anywhere

* Suspicious assignment of range-loop vars detected

* Unused method receiver detected

* Revert "Auxiliary commit to revert individual files from 54edcb445484a2e5d79612e19af8e949b8861253"

This reverts commit bbd1e1beabf7b0c5cfc4f514dcc820062ad6c063.

* Method modifies receiver

* Fix test

* Duplicate imports detected

* Incorrectly formatted error string

* Types of function parameters can be combined

* One more "Unused method receiver detected"

* Unused parameter detected in function
2021-12-07 17:52:39 +00:00
Raul Jordan
1eff00fb33 Display Num Pruned Items in Slasher Only When Actually Prunes (#9989)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-07 14:38:19 +00:00
terence tsao
2bcda2e021 Add more merge related protobufs (#9986)
* Add merge protos

* Update proto/eth/v2/beacon_block.proto

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-12-07 08:08:14 +00:00
Raul Jordan
98fea2e94d Slasher Significant Optimizations (#9833)
* optimizations to slasher runtime

* remove unnecessary code

* test for epoch update

* commentary

* Gaz

* fmt

* amend test

* better logging

* better logs

* log

* div 0

* more logging

* no log

* use map instead

* passing

* comments

* passing

* for select loop wait for init

* sub

* srv

* debug

* fix panic

* gaz

* builds

* sim gen

* ineff

* commentary

* data

* log

* base

* try

* rem logs

* sim logs

* fix wait for sync event

* ev

* init

* init

* Update beacon-chain/slasher/service.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* comments

* elapsed

* Update testing/slasher/simulator/simulator.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* timeout

* inner cancel

* ctx err everywhere

* Add context aware to several potentially long running db operations

* Fix missing param after updating with develop

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-12-06 21:45:38 +00:00
Radosław Kapka
e53be1acbe Do not rely on peer order in TestDebugServer_ListPeers (#9988)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-12-06 19:29:48 +00:00
Nishant Das
c9f5f8dad5 Use Cached Finalized State When Pruning Deposits (#9985)
* use cached finalized state

* Minor grammar edits

* fmt

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-12-06 12:38:23 +00:00
terence tsao
03bf463d0f Remove unused validator proto imports (#9983) 2021-12-04 21:05:54 +01:00
terence tsao
afb49aaeb9 Add and use merge penalty params (#9878)
* Add and use merge penalty params

* add attester slashing test

* Update config_test.go

* Use DeterministicGenesisStateMerge
2021-12-03 22:00:34 +00:00
Potuz
1fd65ad7fd typos (#9979) 2021-12-03 20:18:43 +00:00
Potuz
b1df3b55b0 more typos (#9980) 2021-12-03 16:31:30 -03:00
Preston Van Loon
3767574c77 stateutil: Reduce allocations by specifying slice capacity eager (#9977)
* Reduce allocations by specifying slice capacity eager

* gofmt
2021-12-03 09:06:05 +00:00
Raul Jordan
d3c97da4e1 Ensure Slashing Protection Exports and Keymanager API Work According to Spec (#9938)
* password compliance

* delete keys tests

* changes to slashing protection exports

* export tests pass

* fix up failures

* gaz

* table driven tests for delete keystores

* comment

* rem deletion logic

* look ma, no db

* fix up tests

* ineff

* gaz

* broken test fix

* Update validator/keymanager/imported/delete.go

* rem
2021-12-02 09:58:49 -05:00
Raul Jordan
1d216a8737 Filter Errored Keys from Returned Slashing Protection History in Standard API (#9968)
* add err condition

* naming
2021-12-02 03:32:34 +00:00
Raul Jordan
790bf03123 Replace a Few IntFlags with Uint64Flags (#9959)
* use uints instead of ints

* fix method

* fix

* fix

* builds

* deepsource

* deep source
2021-12-01 23:34:53 +00:00
Preston Van Loon
ab60b1c7b2 Update go-ethereum to v1.10.13 (#9967)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-01 19:31:21 +00:00
Nishant Das
236a5c4167 Cleanup From Deepsource (#9961)
* ds cleanup

* fix

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-01 18:56:07 +00:00
Nishant Das
5e2229ce9d Update Libp2p to v0.15.1 (#9960)
* fix deps

* tidy it all

* fix build

* remove tls patch

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-01 18:09:34 +00:00
Potuz
6ffba5c769 Add v1alpha1_to_v2.go (#9966)
* Add v1alpha1_to_v2.go

* add tests

* gazelle

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-01 17:39:43 +00:00
Potuz
3e61763bd7 fix operation precedence (#9965) 2021-12-01 17:14:08 +00:00
terence tsao
23bdce2354 Fix grpc client connected... logging (#9956)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-12-01 14:45:39 +00:00
Nishant Das
d94bf32dcf Faster Doppelganger Check (#9964)
* faster check

* potuz's review

* potuz's review
2021-12-01 12:37:10 +00:00
Nishant Das
7cbef104b0 Remove Balances Timeout (#9957)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-12-01 04:03:26 +00:00
Potuz
cd6d0d9cf1 Monitor aggregated logs (#9943) 2021-12-01 03:35:55 +00:00
Potuz
afbe02697d Monitor service (#9933)
* Add a service for the monitor

* Do not block service start

* gaz

* move channel subscription outide go routine

* add service start test

* fix panic on node tests

* Radek's first pass

* Radek's take 2

* uncap error messages

* revert reversal

* Terence take 1

* gaz

* Missing locks found by Terence

* Track via bool not empty interface

* Add tests for every function

* fix allocation of slice

* Minor cleanups

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-11-30 19:27:03 -03:00
terence tsao
2c921ec628 Update spec tests to v1.1.6 (#9955)
* Update spec test to v1.1.6

* Update spec test to v1.1.6
2021-11-30 21:21:59 +00:00
terence tsao
0e72938914 Uncap error messages (#9952) 2021-11-30 07:41:07 -08:00
Potuz
71d55d1cff Check for syncstatus before performing a voluntary exit (#9951)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-11-30 10:40:59 +00:00
terence tsao
d8aa0f8827 Alter config filed name to devnet if it's not populated in file (#9949) 2021-11-29 19:27:26 -08:00
Nishant Das
37bc407b56 Refactor States To Allow for Single Cached Hasher (#9922)
* initial changes

* gaz

* unexport and add in godoc

* nocache

* fix edge case

* fix bad implementation

* fix build file

* add it in

* terence's review

* gaz

* fix build

* Apply suggestions from code review

remove assigned ctx

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-11-29 16:30:17 +00:00
Potuz
5983d0a397 Allow requests for next sync committee (#9945)
* Allow requests for next sync committee

* fix deepsource and variable rename

* Minor cleanup

* Potuz's comments

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-11-28 17:34:24 +00:00
terence tsao
85faecf2ca Add test utility merge state (#9944)
* Add test utility merge state

* gaz

* gaz
2021-11-26 15:53:25 +00:00
terence tsao
f42227aa04 Rest of the merge state implementation (#9939)
* Add rest of the state implementations

* Update BUILD.bazel

* Update state_trie_test.go

* fix test

* fix test

* Update beacon-chain/state/v3/state_trie.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

* Update beacon-chain/state/v3/state_trie.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

* add ctx

* go fmt

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2021-11-25 17:41:05 -03:00
terence tsao
c9d5b4ba0e Add merge beacon block wrappers (#9906) 2021-11-24 14:26:17 -08:00
Potuz
fed004686b Add verbosity to aggregation logs (#9937) 2021-11-24 11:35:45 -08:00
Raul Jordan
4ae7513835 Import Keystores Standard API Implementation (#9924)
* begin

* rem deleted code

* delete keystores all tests

* surface errors to user

* add in changes

* del

* tests

* slice

* begin import process

* add import keystores logic

* unit tests for import

* tests for all import keystores keymanager issues

* change proto

* pbs

* renaming works

* use proper request

* pb

* comment

* gaz

* fix up cli cmd

* test

* add gw

* precond

* tests

* radek comments
2021-11-24 10:40:49 -05:00
Nishant Das
1d53fd2fd3 revert change (#9931) 2021-11-24 07:09:15 -08:00
Potuz
a2c1185032 Monitor sync committee (#9923)
* Add sync committeee contributions to monitor

* gaz

* Raul's review

* Added lock around TrackedValidators

* add comment to trackedIndex

* add missing locks because of trackedIndex

* Terence fixes 2

* moved TrackedValidator to service from config

* Terence comment fix

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-11-24 09:56:34 +08:00
terence tsao
448d62d6e3 Add merge beacon chain objects and generate ssz.go (#9929) 2021-11-23 23:34:31 +00:00
terence tsao
4858de7875 Use prysmaticlabs/fastssz (#9928)
* Use prysmaticlabs/fastssz

* Generated code
2021-11-23 21:28:24 +00:00
terence tsao
cd1e3f2b3e Rename coinbase to fee recipient (#9918)
* Rename coinbase to fee recipient

* Fix imports

* Update field name

* Fee receipient

* Fix goimports

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-23 17:49:06 +00:00
Nishant Das
6f20d17d15 Rename To Signature Batch (#9926)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-11-23 17:57:06 +01:00
terence tsao
94fd99f5cd Add getters and setters for beacon state v3 (part 2) (#9916) 2021-11-22 15:56:23 -08:00
Potuz
d4a420ddfd Monitor blocks (#9910)
* Add proposer logging to validator monitor

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-11-22 18:20:31 -03:00
terence tsao
838b19e985 Add getters and setters for beacon state v3 (part 1) (#9915) 2021-11-22 09:37:55 -08:00
Potuz
788338a004 Stop packing deposits early if we reach max allowed (#9806)
* Stop packing deposits early if we reach max allowed

* Add logs to proposals without deposits

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer_deposits.go

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer_deposits.go

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer_deposits.go

* reinsert debug log

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-11-20 14:14:07 +00:00
kasey
39c33b82ad Switch to lazy state balance cache (#9822)
* quick lazy balance cache proof of concept

* WIP refactoring to use lazy cache

* updating tests to use functional opts

* updating the rest of the tests, all passing

* use mock stategen where possible

reduces the number of test cases that require db setup

* rename test opt method for clear link

* Update beacon-chain/blockchain/process_block.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* test assumption that zerohash is in db

* remove unused MockDB (mocking stategen instead)

* fix cache bug, switch to sync.Mutex

* improve test coverage for the state cache

* uncomment failing genesis test for discussion

* gofmt

* remove unused Service struct member

* cleanup unused func input

* combining type declaration in signature

* don't export the state cache constructor

* work around blockchain deps w/ new file

service_test brings in a ton of dependencies that make bazel rules
for blockchain complex, so just sticking these mocks in their own
file simplifies things.

* gofmt

* remove intentionally failing test

this test established that the zero root can't be used to look up the
state, resulting in a change in another PR to update stategen to use the
GenesisState db method instead when the zero root is detected.

* fixed error introduced by develop refresh

* fix import ordering

* appease deepsource

* remove unused function

* godoc comments on new requires/assert

* defensive constructor per terence's PR comment

* more differentiated balance cache metric names

Co-authored-by: kasey <kasey@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-19 15:59:26 +00:00
Potuz
905e0f4c1c Monitor metrics (#9921)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-11-19 15:34:28 +00:00
Nishant Das
0ade1f121d Add Balance Field Trie (#9793)
* save stuff

* fix in v1

* clean up more

* fix bugs

* add comments and clean up

* add flag + test

* add tests

* fmt

* radek's review

* gaz

* kasey's review

* gaz and new conditional

* improve naming
2021-11-19 20:01:15 +08:00
Raul Jordan
ee52f8dff3 Implement Validator Standard Key Manager API Delete Keystores (#9886)
* begin

* implement delete and filter export history

* rem deleted code

* delete keystores all tests

* gaz

* test

* double import fix

* test

* surface errors to user

* add in changes

* edit proto

* edit

* del

* tests

* gaz

* slice

* duplicate key found in request
2021-11-19 04:11:54 +00:00
Potuz
50159c2e48 Monitor attestations (#9901)
Log attestation performance on the validator monitor
2021-11-18 22:14:56 -03:00
Preston Van Loon
cae58bbbd8 Unskip v2 end to end check for prior release (#9920) 2021-11-18 23:17:19 +00:00
Raul Jordan
9b37418761 Warn Users In Case Slashing Protection Exports are Empty (#9919)
* export text

* Update cmd/validator/slashing-protection/export.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2021-11-18 20:49:19 +00:00
Raul Jordan
a78cdf86cc Warn Users if Slashing Protection History is Empty (#9909)
* warning in case not found

* better comment

* fatal
2021-11-17 15:39:43 +00:00
Nishant Das
1c4ea75a18 Prevent Reprocessing of a Block From Our Pending Queue (#9904)
* fix bugs

* test

* raul's review
2021-11-17 10:05:50 -05:00
terence tsao
6f4c80531c Add field roots for beacon state v3 (#9914)
* Add field roots for beacon state

* Update BUILD.bazel

* Adding an exception for state v3

* fix deadcode

Co-authored-by: nisdas <nishdas93@gmail.com>
2021-11-17 09:04:49 +00:00
terence tsao
e3246922eb Add merge state type definitions (#9908) 2021-11-16 08:36:13 -08:00
Nishant Das
5962363847 Add in Deposit Map To Cache (#9885)
* add in map + tests

* tes

* raul's review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-11-16 02:07:42 +00:00
Potuz
5e8cf9cd28 Monitor exits (#9899)
* Validator monitor process slashings

* Preston's comment

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* add test for tracked index

* Prestons requested changes

* Process voluntary exits in validator monitor

* Address Reviewers' comments

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-11-15 19:45:18 +00:00
Raul Jordan
b5ca09bce6 Add Network Flags to Slashing Protection Export Command (#9907) 2021-11-15 19:21:54 +00:00
terence tsao
720ee3f2a4 Add merge state protobuf (#9888)
* Add beacon state protobuf

* Update proto/prysm/v1alpha1/beacon_state.proto

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update proto/prysm/v1alpha1/beacon_state.proto

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Regenerate pbs

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-15 18:53:43 +00:00
Potuz
3d139d35f6 Validator monitor process slashings (#9898)
* Validator monitor process slashings

* Preston's comment

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* add test for tracked index

* Prestons requested changes

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-11-15 17:39:55 +00:00
terence tsao
ea38969af2 Register start-up state when interop mode (#9900)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-15 16:32:54 +00:00
terence tsao
f753ce81cc Add merge block protobuf (#9887)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-15 15:42:18 +00:00
Nishant Das
9d678b0c47 Clean Up Methods In Prysm (#9903)
* clean up

* go simple
2021-11-15 10:13:52 -05:00
Nishant Das
4a4a7e97df handle canceled contexts (#9893)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-11-13 16:03:27 +08:00
Nishant Das
652b1617ed Use Next Slot Cache In More Places (#9884)
* add changes

* terence's review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-11-12 23:46:06 +00:00
Nishant Das
42edc4f8dd Improve RNG Documentation (#9892)
* add comments

* comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-12 20:30:09 +00:00
terence tsao
70d5bc448f Add cli override setting for merge (#9891) 2021-11-12 12:01:14 -08:00
Preston Van Loon
d1159308c8 Update security.txt (#9896)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-12 15:55:45 +00:00
Nishant Das
e01298bd08 Remove Superflous Errors From Parameter Registration (#9894) 2021-11-12 15:28:21 +00:00
terence tsao
2bcb62db28 Remove unused import (#9890) 2021-11-11 22:54:48 +00:00
terence tsao
672fb72a7f Update spec tests to v1.1.5 (#9875)
* Update spec tests to v1.1.5

* Ignore `TERMINAL_BLOCK_HASH_ACTIVATION_EPOCH`

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-10 02:41:16 +00:00
terence tsao
7fbd5b06da time pkg: can upgrade to merge helper (#9879)
* Helper: can upgrade to merge

* Typo
2021-11-09 18:36:50 +00:00
Raul Jordan
0fb91437fc Group Slashing Protection History Packages Idiomatically (#9873)
* rename

* gaz

* gaz

* Gaz

* rename

* edit

* gaz

* gaz

* build

* fix

* build

* fix up

* fix

* gaz

* cli import export

* gaz

* flag

* rev

* comm

* package renames

* radek
2021-11-09 16:49:28 +00:00
Raul Jordan
6e731bdedd Implement List Keystores for Standard API (#9863)
* start api

* keystores list

* gaz

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-11-08 19:08:17 +00:00
Dan Loewenherz
40eb718ba2 Fix typos: repones -> response, attestion -> attestation (#9868)
* Fix typo: repones -> response

* Fix typo: attestion -> attestation
2021-11-07 11:02:01 -06:00
terence tsao
e1840f7523 Share finalized state at start up (#9843)
* Reuse finalized beacon state at startup

* Better logging for replay

* Update tests

* Fix lint

* Add `WithFinalizedStateAtStartup`

* Update service.go

* Remove unused fields

* Update service_test.go

* Update service_test.go

* Update service_test.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-06 13:45:16 +00:00
terence tsao
b07e1ba7a4 GenesisState when zero hash tests (#9866) 2021-11-06 06:57:54 +01:00
Raul Jordan
233171d17c [Service Config Revamp] - Sync Service With Functional Options (#9859)
* sync config refactor

* rem

* rem

* testing

* gaz

* next

* fuzz

* build

* fuzz

* rev

* log

* cfg
2021-11-05 19:08:58 +00:00
kasey
2b0e132201 fix #9851 using GenesisState when zero hash in stategen StateByRoot/StateByRootInitialSync (#9852)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-11-05 18:14:37 +00:00
Raul Jordan
ad9e5331f5 Update Prysm Web to Use v1.0.1 (#9858)
* update web

* site data update

* fmt
2021-11-04 16:12:32 -04:00
terence tsao
341a2f1ea3 Use math.MaxUint64 (#9857)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-04 18:48:09 +00:00
Raul Jordan
7974fe01cd [Service Revamp] - Powchain Service With Functional Options (#9856)
* begin powchain service refactor

* begin refactor

* powchain passes

* options pkg

* gaz

* rev

* rev

* comments

* move to right place

* bazel powchain

* fix test

* log

* contract addr

* happy path and comments

* gaz

* new service
2021-11-04 14:19:44 -04:00
Raul Jordan
ae56f643eb Rename Web UI Performance Endpoint to Summary (#9855)
* rename endpoint

* rename endpoint
2021-11-04 15:13:16 +00:00
terence tsao
2ea09b621e Simplify should update justified (#9837)
* Simplify should update justified

* Update tests

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-03 22:29:03 +00:00
terence tsao
40fedee137 Use prev epoch source naming correctly (#9840)
* Use prev epoch source correctly

* Update epoch_precompute_test.go

* Update type.go

* Update tests

* Gazelle

* Move version to read only

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-11-03 21:50:41 +00:00
kasey
7cdddcb015 Weak subjectivity verification refactor (#9832)
* weak subjectivity verification refactor

This separates weak subjectivity verification into a
distinct type which does not have a dependency on
blockchain.Service

* remove unused variable

* saving enqueued init blocks before ws verify

* remove TODO, handled in previous commit

* accept suggested comment change

start comment w/ name of function NewWeakSubjectivityVerifier

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* update log w/ Raul's suggested language

* explicit zero value for clarity

* add comments clarifying how we adhere to spec

* more clear TODO per Raul's feedback

* gofmt

Co-authored-by: kasey <kasey@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-11-03 15:07:46 +00:00
Raul Jordan
4440ac199f Empty Genesis Validators Root Check in Slashing Protection Export (#9849)
* test for empty genesis validators root

* precod

* fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-03 14:16:57 +00:00
Radosław Kapka
d78428c49e Return proper responses from KeyManagement service (#9846)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-02 15:46:11 +00:00
Raul Jordan
2e45fada34 Validate Password on RPC CreateWallet Request (#9848)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-02 15:20:43 +00:00
Raul Jordan
4c18d291f4 Rename Interop-Cold-Start Package to Deterministic-Genesis (#9841)
* rename interop-cold-start

* rev

* rev

* BUILD

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-02 14:55:36 +00:00
Preston Van Loon
dfe33b0770 stateutil: Remove duplicated MerkleizeTrieLeaves method (#9847) 2021-11-02 14:31:16 +00:00
Raul Jordan
63308239d9 Define Validator Key Management Standard API Schema (#9817)
* begin service for key management

* begin defining schema

* generate bindings

* rev

* add in custom compiler

* use custom plugin with option

* goimports

* fix up proto to take in multiple passwords

* keymanagent proto edit

* rev

* rev

* dev

* builds

* comment

* indent

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-01 17:26:15 +00:00
terence tsao
712cc18ee0 Use BeaconBlockIsNil helper more (#9834)
* Replace manual checks with helper

* Rename to `BeaconBlockIsNil`

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-01 14:13:05 +00:00
Nishant Das
3d318cffa2 Fix Individual Votes RPC Endpoint (#9831)
* fix

* fix tests

* add terence's review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-11-01 18:36:08 +08:00
terence tsao
026207fc16 Remove de-duplication condition for seen aggregates (#9830)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-10-29 21:53:49 +00:00
Raul Jordan
6b7b30ce47 Use GET Request for Slashing Protection Export (#9838) 2021-10-29 18:23:45 +00:00
Yash Bhutwala
105bb70b5e remove extra condition (#9836) 2021-10-29 09:29:33 -07:00
Potuz
9564ab1f7f Fix validator performance logs (#9828)
* Fix validator performance logs

* Add test

* Clean both phase 0 and altair calculations

* Add back total inclusion distance

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-10-28 16:56:27 +00:00
Raul Jordan
0b09e3e955 Refresh Web Backend JWT Secret Upon File Changes With FSNotify (#9810)
* refresh auth file

* refresh auth token from file changes

* gaz

* test for refresh

* rem token

* secret

* refresh

* remove wallet dir on test end
2021-10-28 10:24:39 -04:00
Radosław Kapka
bb319e02e8 Event support for contribution_and_proof and voluntary_exit (#9779)
* Event support for `contribution_and_prrof`

* event test

* fix panic in tests

* fix

* Revert "Auxiliary commit to revert individual files from dc8d01a15f0056c1fb48733219feab6461f71695"

This reverts commit f5f198564079781f80e1a045cefad7c27f89af25.

* remove receiver

* revive test

* move sending events to sync package

* remove receiver

* remove notification test

* build file

* notifier tests

* revert removal of exit event in API

* simplify exit test

* send notification in contribution API method

* test fix

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-10-28 08:56:22 +00:00
terence tsao
53c86429e4 Add altair test for RPC end point (#9829) 2021-10-27 21:00:09 -04:00
Raul Jordan
61172d5007 Add Missing Objects to Keymanager Protobuf (#9827)
* add missing objects

* update keymanager

* fix up missing sign calls

* gaz

* msg block root

* naming
2021-10-27 18:30:53 +00:00
Yash Bhutwala
1507719613 fix altair individual votes endpoint (#9825) 2021-10-27 15:15:51 +00:00
Raul Jordan
4c677e7b40 Remove JWT Expiration (#9813)
* remove expiry from claims

* fix tests

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-10-26 10:24:09 +00:00
Raul Jordan
28f50862cb Goimport All Items in Proto Folder In Bash Script (#9815)
* update goimports to format all, including gw.pb.go files

* update hack

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-10-26 10:07:49 +00:00
Nishant Das
2dfb0696f7 Fix Validator Exit in our V1 Method (#9819) 2021-10-26 10:14:11 +02:00
terence tsao
ae2c883aaf Generate secret key from big number test only (#9816)
* Generate secret key from big number test only

* Gazelle

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-10-25 19:14:19 +00:00
Raul Jordan
ad9ef9d803 Simplify Prysm Backend Password Requirements (#9814)
* simpler reqs

* gaz

* tidy mod
2021-10-25 13:17:47 -05:00
Kirill Fedoseev
b837f90b35 Fix GetDuties (#9811)
* Fix GetDuties

* Add regression test
2021-10-25 09:09:47 -05:00
Håvard Anda Estensen
7f3ec4221f Add errcheck and gosimple linters (#9729)
* Add errcheck linter

* Check unchecked error

* Add gosimple linter

* Remove type assertion to same type

* Omit nil check

len() for nil slices is defined as zero

* Revert "Remove type assertion to same type"

This reverts commit af69ca1ac8.

* Revert "Revert "Remove type assertion to same type""

This reverts commit 5fe8931504.

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-10-22 17:40:03 -05:00
Radosław Kapka
5b3375638a Improve description of datadir flag (#9809)
* Improve description of `datadir` flag

* improve doc
2021-10-22 12:53:13 +00:00
terence tsao
3c721418db Check participation flag offset length (#9784)
* Check participation flag offset length

* Fix test

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-10-21 15:01:09 +00:00
terence tsao
d7cad27cc3 Pool returns empty contribution slice instead of nil (#9808) 2021-10-20 14:45:57 -07:00
terence tsao
290b4273dd InitializePrecomputeValidators check overflows (#9807) 2021-10-20 13:06:43 -05:00
Raul Jordan
a797a7aaac Refactor Web Authentication to Use Auth Token (#9740)
* builds

* initialize auth token from scratch

* change auth

* add improvements

* moar auth fixes

* web auth changes running

* url encode

* tests

* gaz

* navigate auth

* separate line

* auth token

* 304

* persistent auth tokens and jwts

* fixed up test

* auth token path test and integration test

* auth token test

* auth token command

* gaz

* gaz

* Radek feedback

* fix up

* test

* Update validator/rpc/auth_token.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-10-20 16:37:05 +00:00
Raul Jordan
f7c34b0fd6 Validate Keystores Validator Client RPC Endpoint (#9799)
* validate endpoint

* validate keystores proto

* wallet validate

* validate keystores endpoint

* better err message

* added in gaz
2021-10-20 09:23:59 -05:00
Nishant Das
c6874e33f7 fix transitions (#9804) 2021-10-20 19:36:08 +08:00
Radosław Kapka
13ddc171eb Ignore validators without committee assignment when fetching attester duties (#9780)
* Ignore validators without committee assignment when fetching attester duties

* simplify

* fix tests

* slice capacity

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-10-19 17:41:12 +00:00
Radosław Kapka
acde184aa7 Fill out Version for SSZ block (#9801) 2021-10-19 16:45:38 +00:00
Radosław Kapka
5fd6474e56 Allow submitting sync committee subscriptions for next period (#9798)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-10-19 14:04:04 +00:00
Preston Van Loon
65db331eaf Remove bazel-go-ethereum fork, use upstream go-ethereum (#9725)
* Check in go-ethereum crypto/sepc256k1 package with proper build rules

* gaz

* Add karalabe/usb

* viz improvement

* Remove bazel-go-ethereum, use vendored libraries only

* move vendor stuff to third_party so that go mod wont be mad anymore

* fix geth e2e flags

* fix geth e2e flags

* remove old rules_foreign_cc toolchain

* Update cross compile docker image to support os x

* works for geth build

* remove copy of sepc256k1

* revert changes in tools/cross-toolchain

* gaz

* Update go-ethereum to 1.10.10

* Revert "revert changes in tools/cross-toolchain"

This reverts commit 2e8128f7c3.

* revert tags changes

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-10-19 00:00:22 +00:00
Radosław Kapka
7b8aedbfe4 Update Beacon API to v2.1.0 (#9797)
* `Eth-Consensus-Version` header

* rename unused receiver

* omit receiver name

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-10-18 18:55:18 +00:00
terence tsao
cf956c718d Update to Spectest 1.1.3 (#9786)
* Update transition spec test setup

* Update WORKSPACE

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-10-18 18:13:40 +00:00
Raul Jordan
975f0ea1af [Service Config Revamp] - Blockchain Initialization (#9783)
* viz

* unexport config

* builds

* viz

* viz

* register cfg

* fuzz

* blockchain opts

* deepsource

* rename flag opts
2021-10-18 17:48:05 +00:00
terence tsao
a80b1c252a Refactor rpc proposer into smaller files (#9796)
* Refactor rpc proposer into smaller files

* Update BUILD.bazel

* Fix test

* Update BUILD.bazel
2021-10-18 11:35:32 -05:00
Preston Van Loon
1f51e59bfd Bazel: minimal build config alias (#9794) 2021-10-18 14:22:57 +00:00
Nishant Das
bfcb113f78 Fix Doppelganger Protection (#9748)
* add fix

* fix tests

* Update beacon-chain/rpc/prysm/v1alpha1/validator/status.go

* fix tests

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-10-18 13:09:14 +00:00
terence tsao
20c7efda2c Process slashings return error on unknown state (#9778)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-10-18 12:40:47 +00:00
Radosław Kapka
f2990d8fdd Return errors for unknown block/state versions (#9781)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-10-18 12:18:49 +00:00
Nishant Das
d1f3050d20 Ignore G307 for Gosec (#9790) 2021-10-18 14:04:01 +02:00
Raul Jordan
4dbb5d6974 Add WebUI Security Headers (#9775) 2021-10-15 12:40:23 +02:00
Raul Jordan
59547aea66 Deregister Remote Slashing Protection Until Further Notice (#9774)
* avoid registering

* disable endpoints

* remote slasher protection register

* gaz

* fatal on external protection flag call

* radek comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-10-14 19:33:05 +00:00
Radosław Kapka
545424dd09 Fix epoch calculation when fetching sync committee duties (#9767)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-10-14 18:52:26 +00:00
Nishant Das
508b18f1bd Return Error For Batch Verifier (#9766)
* fix it

* make err check happy

* radek's review

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-10-13 23:52:22 +00:00
Radosław Kapka
e644e6b626 Remove special preparation of graffiti (#9770)
* add attestations to migration tests

* remove sepcial preparation of graffiti

* Revert "add attestations to migration tests"

This reverts commit adbe8cf4bf.
2021-10-13 15:32:15 +00:00
Radosław Kapka
280dc4ecf0 Register v1alpha2 endpoints in the gateway (#9768)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-10-12 15:13:52 +00:00
terence tsao
7a825a79ae E2e test: stricter participation check (#9718)
* Stricter participation check

* 0.99 is still better than 0.95...

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-10-12 11:32:36 +00:00
Nishant Das
b81f5fc7a5 Handle Invalid Number Of Leaves (#9761)
* fix bad trie

* handle edge case

* radek's review

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-10-12 10:36:57 +00:00
Nishant Das
06c084ff52 Log out Gossip Handling Errors (#9765) 2021-10-12 07:44:47 +00:00
Preston Van Loon
76e06438e9 Update bazel version to 4.x (#9763)
* Update to bazel 4.0.0

* bazel 4.2.1

* Regenerate crosstool configs

* restore manual tags

* restore manual tags
2021-10-12 06:17:24 +00:00
Nishant Das
f8f037b63d updateInterval (#9764) 2021-10-11 22:12:07 -05:00
terence tsao
f114a47b5b Improve "synced block.." log (#9760) 2021-10-09 15:27:34 +02:00
Nishant Das
65d2df4609 Add in Stronger Length Checks (#9758)
* add changes

* radek's review

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-10-08 17:41:36 +00:00
Radosław Kapka
63349d863b Do not return data for validators without sync duties (#9756)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-10-08 15:20:40 +00:00
Nishant Das
e2a00d3e2e Update to v1.1.2 version of the Spec (#9755) 2021-10-08 15:08:57 +02:00
terence tsao
271ee2ed32 Cleanup slot helpers into intended locations (#9752)
* Refactor slot time helpers to intended locations

* Gazelle

* Update beacon-chain/core/time/slot_epoch.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/core/time/slot_epoch_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Go fmt

* Update transition_fuzz_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-10-07 19:15:35 +00:00
Radosław Kapka
b5f0bd88b0 Add sync details to error messages (#9750) 2021-10-07 20:50:03 +02:00
terence tsao
e7085897ad Use BLS signature length config value (#9746)
* Use `BeaconConfig().BLSSignatureLength`

* Update BUILD.bazel

* Fix build

* Update BUILD.bazel
2021-10-07 10:37:53 -05:00
Raul Jordan
d9b98e9913 Health Endpoints Not Registered in Validator gRPC Gateway Fix (#9747) 2021-10-06 19:07:29 +00:00
terence tsao
a9f9026c78 Minor cleanups (#9743)
* Minor cleanups

* Delete slasher client files

* Revert "Delete slasher client files"

This reverts commit 0c995a1d4a.

* Update slasher_client.go
2021-10-06 13:23:40 -05:00
Nishant Das
b128d446f2 Bring Eth2FastAggregateVerify Inline With the Spec (#9742)
* fix

* tie it closer to the spec
2021-10-06 10:48:56 +08:00
Radosław Kapka
9aa50352b6 Allow fetching sync committee duties for current and next period's epochs (#9728) 2021-10-05 08:46:16 -07:00
Mohamed Zahoor
362dfa691a GetBlock() - Simple optimisations (#9541)
* GetBlock() optimizations

* addressed preston's comments

* fix fmt

* added benchmarks

* fix deepsource

* merge fix

* remove unnesesary check

* fixed terence's comments

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer.go

Co-authored-by: Nishant Das <nishdas93@gmail.com>

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-10-05 17:28:02 +08:00
Raul Jordan
843ed50e0a Register Slashing Protection Client in Validator (#9735) 2021-10-05 00:41:10 +00:00
Preston Van Loon
da58b4e017 Bump github.com/libp2p/go-tcp-transport to v0.2.8 (#9734) 2021-10-04 23:53:22 +00:00
Raul Jordan
800f78e279 Web UI Fix for Prysm V2.0.0 (#9732) 2021-10-04 22:26:55 +00:00
terence tsao
644038ba61 eth2 api: use balance instead of effective balance (#9722)
* Use balance instead of effective balance

* Update test to reflect validator balance

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-10-04 20:57:40 +00:00
Nishant Das
865ef4e948 Pin Base Images (#9727)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-10-04 18:55:56 +00:00
Håvard Anda Estensen
b793d6258f Add Golangci-lint to GitHub Actions and add Deadcode linter (#9597)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-10-04 13:06:21 +02:00
Raul Jordan
f7845afa57 Use Unique Slot Time Tickers for Slasher (#9723) 2021-10-03 07:49:01 +00:00
terence tsao
c21e43e4c5 Refactor: move functions beacon-chain/core/time -> time/slots (#9719)
* Move necessary functions beacon-chain/core/time -> time/slots

* Fix fuzz

* Fix build

* Update slot_epoch.go
2021-10-01 15:17:57 -05:00
Raul Jordan
4f31ba6489 Changes to E2E for Optimized Slasher (#9698)
* remaining slasher e2e changes

* testing gaz

* slash e2e

* deepsource

* gaz

* viz

* revert wait group changes

* comment

* lock around reset cache

* add slashing reason

* revert changes

* is sync

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
2021-09-30 20:28:14 +00:00
terence tsao
2bc3f4bc6a Update sync time error message (#9713)
* Update sync_committee.go

* Update beacon-chain/core/altair/sync_committee.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Fix slot calculation

* Go fmt

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-30 20:03:57 +00:00
Radosław Kapka
601493098b Return error when request JSON contains unknown fields (#9710)
* Return error when request JSON contains unknown fields

* Update api/gateway/apimiddleware/process_request.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-30 19:42:20 +00:00
terence tsao
8219af46e4 Move slot epoch from core to time pkg (#9714)
* Move slot epoch from core to time pkg

* Fix fuzz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-30 19:00:14 +00:00
Raul Jordan
f5234634d6 Prevent Saving Empty Chunks to Disk (#9707)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-30 18:29:40 +00:00
Raul Jordan
26978fcc50 Make Jaeger in E2E Sharding Aware (#9711)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-30 17:35:05 +00:00
Nishant Das
7c67d381f4 Remove Peer Scorer From Dev (#9712)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-30 16:51:13 +00:00
Nishant Das
a196c78bed clean up (#9709) 2021-09-30 08:10:49 -07:00
Nishant Das
3c052d917f Use Scoring Parameters To Enable Global Score (#8794)
* fix score parameters

* add check

* fix deadlock issues

* fix

* fix again

* gaz

* comment

* fix tests and victor's review

* gaz

* clean up

* fix tests

* fix tests

* gate behind flag

* add scorer to dev

Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-30 03:39:53 +00:00
Raul Jordan
7dae66afc9 Highest Attestations Endpoint for Optimized Slasher (#9706)
* begin highest atts

* highest atts

* deep source
2021-09-29 21:33:28 -05:00
Radosław Kapka
62dc74af2f Move head event's epoch transition to epoch start (#9704)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-09-29 17:18:10 -05:00
Raul Jordan
13cdb83591 Validator Changes for Optimized Slasher (#9705)
* val changes

* mock

* mocks

* gomock any

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-29 21:25:45 +00:00
Preston Van Loon
520bc9d955 Update validator reporting logs and metrics for Altair (#9589)
* Mark fields as deprecated due to Altair

* Only print inclusion distance fields before Altair fork

* Report phase0 and altair metrics respectively

* only set phase0 fields in phase0, only set altair fields in altair

* better use of fields

* Update go pbs

* Update individual votes method

* regen go proto files

* formatting

* Feedback from @potuz

* Annotate metrics per @potuz suggestion

* Set previous release e2e to end 1 epoch before altair. Add some out of bounds checks for validator metrics reporting and a panic catch

* gofmt

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-29 20:49:58 +00:00
Raul Jordan
df33ce3309 Remaining Slasher Beacon Node Changes (#9701)
* slasher beacon node changes

* remaining beacon node items

* moar changes

* gaz

* flag fix

* rem slashable

* builds

* imports

* fix up

* pruning faster test

* deepsource

* fix wrong item

* node node feature flags

* broken test

* preston review

* more preston comments

* comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-29 18:17:37 +00:00
terence tsao
86efa87101 Refactor sync and participation field roots (#9703)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-29 17:46:51 +00:00
Potuz
ff625d55df Use target root for pending attestations in tests instead of unrelated one (#9699)
* Use target root for pending attestations instead of unrelated one

* remove unnecessary block saves

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-09-29 12:27:47 -05:00
Mohamed Zahoor
0678e9f718 finalise deposits before we initialise the node (#9639)
* finalize deposits before we initialize the node

* Update beacon-chain/blockchain/service.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Update beacon-chain/blockchain/service.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* moved this when intiializing caches

* added test case

* satisfy deepsource

* fix gazel

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-09-29 15:43:24 +00:00
Nishant Das
10251c4191 increase size (#9702) 2021-09-29 10:01:10 -05:00
Potuz
861c2f5120 remove unnecessary asserts from tests (#9700) 2021-09-29 22:17:06 +08:00
terence tsao
bfc821d03a Can save justified checkpoint to DB (#9697)
* Can save justified checkpoint to DB

* Update beacon-chain/blockchain/process_block_test.go

Co-authored-by: Nishant Das <nishdas93@gmail.com>

* Update beacon-chain/blockchain/process_block.go

Co-authored-by: Nishant Das <nishdas93@gmail.com>

* Update beacon-chain/blockchain/process_block.go

Co-authored-by: Nishant Das <nishdas93@gmail.com>

* Update beacon-chain/blockchain/process_block.go

Co-authored-by: Nishant Das <nishdas93@gmail.com>

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-09-29 03:00:52 +00:00
Raul Jordan
9edba29f64 Slasher Simulator Code for Testing Optimized Slasher Behavior (#9695)
* slashing simulator

* add in necessary items for slasher sim

* sim item

* fix up

* fixed build

* rev

* slasher sim in testing

* testonly

* gaz

* gaz

* fix viz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
2021-09-29 02:27:21 +00:00
Nishant Das
0edb3b9e65 Flatten Attestation Packing (#9683)
* flatten attestation packing

* dedup again

* terence's review

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-29 01:56:55 +00:00
Preston Van Loon
6c5bf70021 CI: fix remote caching (#9696) 2021-09-29 00:18:31 +00:00
Nishant Das
393549ad19 Add in Balance Safety Check (#9419)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-28 22:19:10 +00:00
terence tsao
8f8ccf11e4 Update head more timely before (#9651)
* Move update head closer to transition

* Update process_block.go

* Update tests

* Move savePostStateInfo back

* Update process_block.go

* Update process_block_helpers.go

* Minor clean up for better diff

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-28 21:51:11 +00:00
terence tsao
806bcf1d29 Clean up unused types & function comments (#9691)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-28 21:21:07 +00:00
Raul Jordan
2a2239d937 Detect Slashable Attestations in Optimized Slasher (#9694)
* detect slashable attestations

* Update beacon-chain/slasher/detect_attestations.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-09-28 20:54:17 +00:00
Preston Van Loon
c94ba40db2 config: ensure config matches altair presets (#9690)
* config: ensure config matches altair presets

* gofmt

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-09-28 20:02:12 +00:00
Raul Jordan
1816906bc7 Detect Slashable Blocks in Optimized Slasher (#9693)
* pass

* Update beacon-chain/slasher/detect_blocks.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Update beacon-chain/slasher/detect_blocks.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-28 19:33:45 +00:00
Radosław Kapka
c32090aae5 Allow sending Altair blocks to /eth/v1/beacon/blocks (#9685)
* Allow sending Altar blocks to `/eth/v1/beacon/blocks`

* tests

* add documentation

* fix ineffectual assignment

* change type of sync committee bits

* remove unused import

* fix Altair epoch calculation

* compare slot against slot

* do not publicly export E2E constant

* tests for setInitialPublishBlockPostRequest

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-28 19:07:32 +00:00
Raul Jordan
57f965df50 Include Slasher Receiving Methods (#9692)
* first

* add receive details

* ensure most builds

* add slasherkv changes

* db iface additions

* build

* gaz

* proper todo comment

* terence comments

* sig check

* bad sig checks

* proper lock issue

* fix test

* fix up tests
2021-09-28 18:13:16 +00:00
Radosław Kapka
8a6e2a5c63 Add MIN_SYNC_COMMITTEE_PARTICIPANTS parameter to config (#9689)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-09-28 16:54:12 +00:00
Raul Jordan
6d79f61fda Process Slashings for Newly Optimized Slasher (#9687)
* add process slashings

* verify sig

* add method

* add test

* add in process slashings functionality

* target state for aggregate

* comment

* Radek comments
2021-09-28 10:27:40 -05:00
Nishant Das
bf2d2cf981 make opt max cover default (#9684) 2021-09-28 17:34:29 +08:00
Raul Jordan
f2840c9ffa Slasher Min/Max Chunk Logic (#9673)
* slasher chunks code

* slasher chunks code

* avoid using shared

* testing helper

* slasher gaz

* radek comments

* preston feedback
2021-09-28 02:04:32 +00:00
Preston Van Loon
cbb4361c64 Update spectests to v1.1.0 (#9680)
* Update spectests to v1.1.0

* ignore placeholder fields, fix spec config check
2021-09-28 00:50:06 +00:00
terence tsao
328e3e6caf Update mainnet_config.go (#9678) 2021-09-27 13:39:55 -07:00
Preston Van Loon
ee0a453b7b core: refactor signing and domain methods from helper to core/signing pkg (#9520)
* Move domain function and all signing root functions from beacon-chain/core/helpers to beacon-chain/core

* @terencechain suggestion to put these methods under core/signing
2021-09-27 16:19:20 +00:00
Radosław Kapka
3e640fe79f Remove unused Eth1Data-related code from the proposer (#9670)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-27 14:29:41 +00:00
Radosław Kapka
bf41fd854d Remove fmt.Println from committee cache (#9677) 2021-09-27 11:18:20 +00:00
Nishant Das
6eb158c16a Check Head State For New Head Methods (#9676)
* check head state

* add tests
2021-09-27 08:27:11 +00:00
terence tsao
376d248c22 Add in progress handler to committee cache (#9664)
* Add in progress handler for committee cache

* Remove debug print

* Update validators.go

* Fix all the tests

* More tests

* Update committee_disabled.go

* Update committee_disabled.go

* Update testing util

* Update main.go

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-09-26 23:27:57 +08:00
Nishant Das
6e4c2b4b20 Wrap Gossip Validators With Error (#9660) 2021-09-25 18:06:48 -07:00
Raul Jordan
75936853af Optimized Slasher Docs and Helpers (#9578)
* bring over helpers

* slasher helpers pass tests

* fix dead link

* rem eth2

* gaz

* params

* gaz

* builds

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-09-24 13:38:13 -05:00
terence tsao
ea9ceeff03 Various clean up before v2 (#9672)
* Update package names

* Various clean up

* Gazelle

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-24 17:42:16 +00:00
Radosław Kapka
78ea402d9d Correct the semantics of startEpoch calculation in registerSyncSubnet (#9671) 2021-09-24 16:02:55 +00:00
Radosław Kapka
fae806c73e Small changes in API Middleware (#9666)
* Small changes in API Middleware's custom hooks

* reorder fields in `Endpoint`

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-24 14:34:19 +00:00
Radosław Kapka
45510fafea Fix error handing for block v2 endpoints (#9667)
* Fix error handing for block v2 endpoints

* rename helper func
2021-09-24 13:52:00 +00:00
Radosław Kapka
12480e12b2 Add flags for disabling selected API (#9606)
* Add flags for disabling selected API

* tests

* build file

* Use comma-separated modules

* test fix

* fix gateway tests

* fix import in flag tests
2021-09-24 09:25:42 +00:00
Radosław Kapka
7dd99de69f Restructure API Middleware (#9663)
* Restructure API Middleware

* fix package name in tests

* build file

* gzl

* fix one more test
2021-09-23 20:41:04 +00:00
Raul Jordan
a9a4bb9163 Move Shared/Testutil into Testing (#9659)
* move testutil

* util pkg

* build

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-23 18:53:46 +00:00
Radosław Kapka
2952bb3570 Increase debug gRPC message size to 128MB (#9661)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-23 18:31:24 +00:00
Radosław Kapka
1fbe05292f Fix SubmitPoolSyncCommitteeSignatures API endpoint (#9646)
* Fix `SubmitPoolSyncCommitteeSignatures` API endpoint

* fix test

* fix another test

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-23 18:01:07 +00:00
Radosław Kapka
36b1f322c0 Various bug fixes in Eth API (#9649)
* fix block structure

* correct status code when block is not found

* make `/internal` work with events and SSZ

* test fix

* better block serialize tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-23 17:34:11 +00:00
Nishant Das
5225d97fd4 Remove Update Timely Feature Config (#9655)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-23 17:04:40 +00:00
Nishant Das
57b7bf6ea0 Update Mainnet Bootnodes (#9656)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-23 16:23:01 +00:00
Raul Jordan
29513c804c Create Encoding Bytesutil (#9658)
* bytesutil

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-23 15:23:37 +00:00
Nishant Das
b9ad81db50 bump it up (#9657) 2021-09-23 09:20:19 -05:00
Nishant Das
d694a775d7 Check For Blocks In Our Initial Sync Cache (#9558)
* add change

* test

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-09-23 05:18:19 +00:00
Nishant Das
4f3762f1f7 Clean Up V2 BeaconState (#9648)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-09-23 01:10:25 +00:00
terence tsao
8678610683 Update package name comments (#9653) 2021-09-22 22:38:06 +00:00
Radosław Kapka
8d4cdde07e Clean up code in Eth API (#9650)
* typo fix

* code cleanup

* fix spans

* move state convertion to migration package

* add migration package to v1 visibility
2021-09-22 17:59:06 +00:00
Raul Jordan
191bce3655 Add True Eth2 Deposit Contract, Bytecode, ABI (#9637)
* solidity contract, abi, and deposit util

* gaz

* gaz

* drain contract remove

* build

* fix up deploy

* add readme

* fix e2e

* revert

* revert flag

* fix

* revert test flag

* fix broken test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-22 17:27:13 +00:00
terence tsao
f52f737d2b Review Altair core epoch files with changes (#9556)
* Clean up core epoch for Altair

* Update BUILD.bazel

* Remove check for spec test

* Update validators_test.go

* Update validators_test.go

* Update beacon-chain/core/altair/epoch_precompute.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/core/altair/epoch_precompute_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/core/altair/epoch_spec.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-09-22 15:07:05 +00:00
Preston Van Loon
2cbe10c701 Fix long running e2e altair exception for previous release (#9645)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-22 04:01:43 +00:00
Preston Van Loon
36a7575437 e2e: fix trace_sync http handler registration (#9644)
* Avoid a global http server so that multiple test runs aren't registering global http handlers and causing a panic

* gofmt
2021-09-22 03:34:09 +00:00
terence tsao
8e3e6d0156 Fix package comment typo (#9641)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-21 20:25:12 +00:00
Raul Jordan
f3d6dbcc1e Move Shared/Params Into Config/Params (#9642)
* config params into pkg

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-21 19:59:25 +00:00
Radosław Kapka
26893bcb8c Production version of Altair Eth APIs (#9640)
* internal prefix in proto services

* half-baked inmplementation

* rename v1 to eth

* use router and remove old flag

* uncomment cors

* update v2 methods in proto services

* move adding path prefix after param processing

* remove unneeded code

* remove flag

* fix e2e

* uncomment sync committee e2e

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-21 19:20:57 +00:00
Raul Jordan
eebcd52ee6 Miscellaneous Packages from Shared Into Proper Folders (#9638)
* slashutil

* builds

* interop

* viz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-21 18:11:16 +00:00
Raul Jordan
45bfd82c88 Add Encoding SSZ Package (#9630)
* ssz package

* compile

* htrutils

* rem pkg doc

* fix cloners_test.go

* fix circular dep/build issues

Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-21 15:02:48 +00:00
Nishant Das
b943f7bce5 Add Flag To Parameterize Number of Peers in A Subnet (#9631)
* add flag

* use old value

* gaz

* raul's review
2021-09-21 07:55:52 +00:00
Radosław Kapka
28364d9f33 E2E for API Middleware (#9635)
* rename file

* Enable SSZ-serialization in v2 endpoints

# Conflicts:
#	proto/eth/v2/beacon_state.pb.go

* e2e

* rename file

* improve logging errors

* Revert "improve logging errors"

This reverts commit 796bcf3e97.

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-21 04:31:41 +00:00
deepsource-autofix[bot]
531f05d30d Unused parameter should be replaced by underscore (#9632)
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-09-20 20:51:59 +00:00
Preston Van Loon
728c77cc0c CI: use nostamp for code coverage and fuzzer uploads (#9634)
* Follow up to #9633, nostamp for code coverage

* nostamp for fuzzer uploads too

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-20 18:59:44 +00:00
terence tsao
161a13ac09 Correctly filter sync committee message at period boundary (#9626)
* Add head sync committee indices with slot consideration

* Apply rpc

* Apply sync

* Update tests

* Update mock.go

* Update sync_committee_test.go

* Update validator_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-20 18:06:43 +00:00
Preston Van Loon
cb631360e9 CI: nostamp config for better cache hits (#9633) 2021-09-20 17:29:41 +00:00
terence tsao
2ba29a3cfc Move block and attestation to proto v1alpha1 folder (#9618)
* Add block and attestation to container pkg

* Move aggregation into attestation

* Update attestation_test.go

* Move them to proto

* Gazelle

* fix cycle

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-20 16:17:03 +00:00
Nishant Das
150640305a Support Go Builds For Darwin Arm64 (#9600)
* add changes

* tidy

* stranded dep

* fixed commit

* use real commit

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-18 17:56:05 +00:00
Raul Jordan
11a1f681e0 Move Shared Packages Into Async/ (#9620)
* async packages

* change pkg

* build

* working

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-18 17:26:11 +00:00
Raul Jordan
7dadc780b8 Move Shared Packages into Math/ and IO/ (#9622)
* amend

* building

* build

* userprompt

* imports

* build val

* gaz

* io file

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-17 21:55:24 +00:00
Raul Jordan
d2f74615ab Move Miscellaneous Shared/ Packages Into Semantic Groups (#9624)
* remove shared network and ip util

* forkutil

* gaz

* build

* gaz

* nogo

* genrule

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-17 19:20:50 +00:00
terence tsao
6308678a89 Fail early if block and state are diff versions (#9625) 2021-09-17 13:03:14 -05:00
Radosław Kapka
93a4ebc662 Enable SSZ-serialization in v2 endpoints (#9587) 2021-09-17 14:08:59 +00:00
Nishant Das
4109d4a868 Standardize Metrics To Match Minimal Metric Spec (#9623) 2021-09-17 13:35:12 +02:00
Nishant Das
568797d53d Add MultiLock When Fetching Sync Head State (#9595)
* add lock when fetching sync state

* improvements

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-17 02:03:18 +00:00
Nishant Das
18e1d43360 Update blst to v0.3.5 and add BLS spec tests (#9611)
* add new dep

* add changes so far

* fix all tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-17 01:32:23 +00:00
Raul Jordan
8f0008c45a Move Related Packages Into API/ Pkg (#9619)
* begin on api pkg

* build

* build

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-16 19:55:51 +00:00
terence tsao
f49637b09b Move iputil and httpuil pkgs into network pkg (#9621)
* Add network pkg

* Go fmt

* Update discovery_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-16 19:12:27 +00:00
Raul Jordan
3daa2d8a18 Remove Deprecated Slasher (#9608)
* delete slasher

* rem slasher

* fix build

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-16 18:26:58 +00:00
Raul Jordan
61c24c3870 Move Fuzz/ Into Testing/ (#9617)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-16 17:32:51 +00:00
Raul Jordan
23e2d62c0c Create Container Package for Shared/ Subpackages (#9607)
* ensure run

* amend

* fix initial sync test target

Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-16 12:05:58 -05:00
Raul Jordan
8df96426ef Move Scripts Into Hack/ Directory (#9605)
* ensure build

* moved third party to toplevel:

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-16 16:22:39 +00:00
terence tsao
989daf267a Remove next slot cache usage for fork choice handler (#9601)
* Remove next slot cache usage for fork choice

* Go fmt

* Update process_attestation_helpers.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-16 15:29:59 +00:00
Raul Jordan
495e92ce9f Create Runtime Package for Moving Several Packages in Shared/ (#9588)
* add maxprocs

* add prereqs

* add tos and version pkg

* add in all runtime packages

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-16 09:46:29 +00:00
terence tsao
514d4ef8f5 Invert next slot state cache feature (#9602)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-16 07:48:06 +00:00
Radosław Kapka
d020e1b2d9 Correctly display epoch participation in GetStateV2 (#9598)
* Correctly display epoch participation in `GetStateV2`

* fix file names

* handle null case

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-16 02:51:51 +00:00
terence tsao
ee5d75732d Add pkg crypto (#9603)
* Add pkg crypto

* Update go.yml

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-15 22:55:11 +00:00
Preston Van Loon
fff0208207 Fix long running e2e tests (#9604)
* Revert e2e changes from #9565

* Override ChurnLimitQuotient to reflect minimal config prior to #9565
2021-09-15 22:04:13 +00:00
Radosław Kapka
c47923d20e Properly display sync committees in GetStateV2 (#9599)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-15 17:23:13 +00:00
Raul Jordan
5230af0e0c Move EndtoEnd Tests to Testing/ Folder (#9586)
* endtoend to testing/

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-15 14:42:05 +00:00
Nishant Das
34391fa780 Implement Batch Verifier For Pubsub Objects (#9344)
* add initial commit

* add aggregates in

* fmt

* add new changes

* gaz

* fix copy

* terence's review

* fix broken build
2021-09-15 05:49:50 +00:00
terence tsao
31a78ab22a Create config package for Shared/featureconfig (#9593)
* add config/features

* Gazelle

* Gazelle

* Fix build

* Go mod tidy

* active cache

Co-authored-by: rauljordan <raul@prysmaticlabs.com>
2021-09-15 01:18:39 +00:00
terence tsao
3e71997290 Create time Package for Shared/timeutil, mclock and slotutil (#9594)
* add time pkg

* Go fmt
2021-09-15 00:09:04 +00:00
terence tsao
77de467250 add lru (#9592) 2021-09-14 18:11:25 -05:00
Raul Jordan
9935ca3733 Move Shared/ Subpackages Into Monitoring/ Folder (#9591)
* add tracing

* monitoring pkg

* move prom

* Add client stats

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-14 20:59:51 +00:00
Radosław Kapka
5b37deb1a6 Allow listing attestations for Altair blocks (#9580)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-14 20:03:09 +00:00
Raul Jordan
01a72a9df2 Move Benchutil to Testing/ Folder (#9583)
* benchutil to testing

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-14 19:18:15 +00:00
Radosław Kapka
f98505bfbe Align ProduceBlockV2 to HTTP API spec (#9581)
* Align `ProduceBlockV2` to HTTP API spec

* snake case protos

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-14 17:48:24 +00:00
Raul Jordan
6340e58f36 Move Spectests Into a Testing/ Folder (#9582) 2021-09-14 16:02:58 +00:00
Radosław Kapka
ac3c544e29 Make until_epoch exclusive in PrepareSyncCommitteeSubnets (#9579)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-14 15:14:28 +00:00
Radosław Kapka
25f2ca4159 Align ProduceSyncCommitteeContribution to HTTP API spec (#9572)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-14 14:39:16 +00:00
Radosław Kapka
ae21665b0a Align GetStateV2 to API HTTP spec (#9569) 2021-09-14 09:07:03 +00:00
terence tsao
408392aa06 Enable prune canonical attestation (#9576)
* Enable prune canonical attestation

* Update shared/featureconfig/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-13 20:10:45 +00:00
Radosław Kapka
562e128251 Align GetSyncCommitteeDuties to API HTTP spec (#9570)
* Align `GetSyncCommitteeDuties` to API HTTP spec

* doc

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-13 19:14:05 +00:00
Radosław Kapka
4f86714c2a Align PrepareSyncCommitteeSubnets with API HTTP spec (#9571)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-13 18:16:16 +00:00
Radosław Kapka
1e4063e69c Align PublishContributionAndProofs to HTTP API spec (#9573)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-13 17:47:05 +00:00
terence tsao
e8e62856c8 Enable insert orphaned attestation (#9575)
* Enable insert orphaned att

* Go fmt
2021-09-13 17:05:09 +00:00
Radosław Kapka
ed68dd757f Implement ProduceBlockV2 in the beacon API (#9560)
* Rename service func to ProduceBlockV2

# Conflicts:
#	beacon-chain/rpc/eth/validator/validator.go

* initial implementation

# Conflicts:
#	beacon-chain/rpc/eth/validator/validator.go
#	proto/eth/v2/validator.pb.go

* test first try

* try new signature in test

* Fix test's sync committee signature

* Use 0xAA instead of 0xFF

* properly get altair block

* final implementation

* Revert "Auxiliary commit to revert individual files from 17779d4228b8c65452343fe5c51ef5764766f732"

This reverts commit 3f352a145c70c8edf8dd283f684fecf93f6d6487.

# Conflicts:
#	beacon-chain/rpc/eth/beacon/blocks.go

* revert pb.go changes to block

* add signature to return value

* fix ineffectual assignment

* simplify if/else statement

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-09-13 13:58:42 +00:00
Nishant Das
fd920bb786 Fix Finalized Block Retrieval in Altair (#9561)
* fix

* radek's review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-09-13 06:57:31 +00:00
terence tsao
416840036a Add beta4 spec test (#9565)
* Add beta4 spec tests

* Fix test setup for Altair registry update

* Update WORKSPACE

* Update churn limit calculation for e2e

* Hard set churn limit

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-11 01:33:31 +00:00
Preston Van Loon
1374b6dd4f Enable active balance cache (#9567)
* Enable active balance cache

* gofmt
2021-09-10 21:47:25 +00:00
Raul Jordan
dcc1f7c0ec Move End-to-End Tests for Altair Into Develop (#9564)
* end to end from hf1

* remove duplicate import

* skip sync eval

* conditional sync participation

* altair fork epoch to 6

* preston feedback

* proper fork epoch

* run for 6

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-10 19:59:43 +00:00
Radosław Kapka
2d9ae57378 Align SubmitPoolSyncCommitteeSignatures to API HTTP spec (#9563) 2021-09-10 09:55:56 -07:00
terence tsao
c6e7ab79ef Share ProcessSlashings between upgrades (#9557)
* Reuse process slashing

* Go fmt

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-10 15:57:16 +00:00
Radosław Kapka
006a2620c4 Align GetBlockV2 to API HTTP spec (#9562) 2021-09-10 10:20:52 -05:00
Nishant Das
741e136b22 Run Fork Watcher (#9559) 2021-09-10 11:51:55 +00:00
Nishant Das
1ccbe79209 Add Altair Metadata Changes (#9552)
* add metadata changes

* gaz

* add tests

* remove

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-10 08:10:08 +00:00
Nishant Das
57c2d86da1 Fork Watcher For Sync (#9550)
* add watcher

* raul's review

* preston's and terence's review

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-10 03:52:56 +00:00
Fredrik Svantes
ffd7579476 Add how users should report security vulnerabilities for this repository (#9525)
* Add how users should report security vulnerabilities for this repository

Suggestion to add a notice on how to report security vulnerabilities. This is visible at https://github.com/prysmaticlabs/prysm/security

* Add correct email

Co-authored-by: Nishant Das <nish1993@hotmail.com>

* Update SECURITY.md

Co-authored-by: Nishant Das <nish1993@hotmail.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-09-10 03:13:09 +00:00
Nishant Das
97f6143a43 Add Remaining Gossip Changes (#9553)
* add changes

* add changes here in

* rem duplicate import

* fix topic not being set in test

* terence's review

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-10 02:36:31 +00:00
Radosław Kapka
fc44ecbb16 Align GetEpochSyncCommittees to API HTTP spec (#9555)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-09 21:54:08 +00:00
Kirill Fedoseev
5325558f03 Recompute SqrRootSlotsPerEpoch at startup for custom configs (#9554)
* Recompute SqrRootSlotsPerEpoch at startup for custom configs

* Fix bazel deps

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-09-09 17:03:00 +00:00
Radosław Kapka
a6abfa5dd8 Implement PublishContributionAndProofs in the beacon API (#9526)
* implementation

* tests

* do not reuse pool

* godoc

* Revert "Auxiliary commit to revert individual files from 7f95a794ef9b2e1feac4683e181fc7d0fc289a0a"

This reverts commit 917ed4cc7a457847139d11bfb7c8d163584b48be.

* handle error in test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-09 15:49:11 +00:00
Nishant Das
2faaafb159 add remaining rpc methods (#9551) 2021-09-09 10:12:49 -05:00
Radosław Kapka
ddfa269e4f Add Version field to v2 endpoints (#9546)
* version enum

* get block

* get state

* rename shared to version

* goimports

* Go fmt

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-09-09 13:29:09 +00:00
Radosław Kapka
cb9c1ad0fe Implement Semi-Generic GetBeaconBlock RPC Endpoint for Pre & Post-Fork Blocks (#9517)
* BuildBlockData

* GetBlockAlatair

* build file

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer.go

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer_test.go

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer_test.go

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer_test.go

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer_test.go

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer_test.go

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer_test.go

* review comments

* add alias

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* add get block test

* comment

* implement get beacon block

* commentary

* test setup issues

* Use correct genesis state

* test renames

* pass

* rem commented code

* ctx unused

* proper grpc errors

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-09-09 05:26:49 +00:00
Nishant Das
279cc16c88 Add Sync Subnets Subscribers (#9539)
* add in digest

* add new stuff

* add tests

* terence's review

* Update beacon-chain/sync/subscriber.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* terence's review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-09-09 03:25:02 +00:00
Radosław Kapka
4b90c10cb2 Fix templating in eth protos (#9548)
* 1st try

* working version

* clean up build file

* fix v1 + cleanup

* remove unneeded comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-08 19:37:26 +00:00
Raul Jordan
83a9079700 Implement Generic ProposeBeaconBlock Function for RPC (#9549)
* add in generic propose block method

* propose generic with tests

* ineff assign issue
2021-09-08 18:54:04 +00:00
terence tsao
7c45c5f8cd More misc altair changes (#9547)
* More altair changes

* Gazelle
2021-09-08 17:52:37 +00:00
Nishant Das
90038a6001 Implement Network Encoder using SSZ Interface (#9535)
* add stuff

* unused

* kasey's review

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-08 23:16:53 +08:00
Preston Van Loon
b586d3784b Rename beacon-chain/core/state to beacon-chain/core/transition pkg (#9530)
* Move, refactor, fix

* @rkapka PR feedback: revert irrelevant changes

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-08 10:41:47 +00:00
kasey
bc551b7e30 using upstream fastssz with altair fixes (#9542)
* using upstream fastssz with altair fixes

* update generated ssz code

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-07 19:42:37 +00:00
terence tsao
d7679d2e71 Validator: rest of the Altair changes (#9532)
* Bring in rest of the validator chagnes

* Go fmt

* Deepsource

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-07 18:43:50 +00:00
Radosław Kapka
ddbac85ec4 Allow returning Altair block headers (#9543)
* GetBlockHeader

* ListBlockHeaders
2021-09-07 18:07:30 +00:00
Nishant Das
ee28dc3d4f Add in Fork Digest For Subscribers (#9538)
* add in digest

* radek's review

* raul's review

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-07 16:58:24 +00:00
Raul Jordan
d210dd7691 Maintainable Beacon Block API Endpoints (#9528)
* add new proto endpoints and deprecate old

* regen protos

* regen mocks

* begin fixing tests

* build tests

* e2e build

* go build

* resolve conflicts

* pass tests

* amend generated name

* fix build

* more fixes

* update gateway paths

* radek comments

* fix comment

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-09-07 15:17:50 +00:00
Radosław Kapka
d6416b29b2 Set Content-Length to zero for empty POST reponse (#9540)
* Set `Content-Length` to zero for empty POST reponse

* remove unused function param
2021-09-07 08:54:44 -05:00
Preston Van Loon
dc71f35f4f Remove copyutil pkg, move v1alpha1 copy methods into v1alpha1 pkg (#9533) 2021-09-04 06:55:57 +00:00
Preston Van Loon
4beb352e6f Move Slot and Epoch functions from helpers to core pkg (#9519)
* Move Slot and Epoch functions from helpers to core

* limited viz

* goimports

* fix fuzz build

* fix fuzz build

* fix

* fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-03 20:10:31 +00:00
terence tsao
29b851a2b7 RPC: Stream altair block (#9521)
* Add tests

* Fix tests

* Fix test

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-03 19:11:40 +00:00
terence tsao
ebf3897017 Validator: propose Altair block (#9414)
* Validator can propose block

* Gazelle

* Preston's feedback

* Rename

* Fix deep source

* Fix build

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-09-03 18:26:10 +00:00
Nishant Das
cc790ceb2e Add RPC Changes to Sync For Block Methods (#9497)
* add changes

* radek's review

* kasey's review

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-03 16:00:56 +00:00
Radosław Kapka
e875fe3dfc Implement ProduceSyncCommitteeContribution in the beacon API (#9524)
* implementation

* test

* change func signature

* remove broadcaster from test
2021-09-03 09:47:35 -05:00
Preston Van Loon
265b5feabf p2p: Handle nil topic (#9522)
* fix nil topic

* fmt

Co-authored-by: nisdas <nishdas93@gmail.com>
2021-09-03 14:39:54 +08:00
Nishant Das
30b2adc5d6 Add in Sync Contribution And Proof Pubsub Validator (#9475)
* add contribution handlers

* fix it

* fmt

* fix more

* add clean up

* Apply suggestions from code review

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/sync/validate_sync_contribution_proof.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* use validation pipeline

* deepsource

* fix all tests

* clean up

* comment

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-03 04:06:54 +00:00
terence tsao
924fe20dcd RPC: list block altair end point (#9494)
* Implement list block altair

* Dont export block container

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-02 22:21:33 +00:00
Preston Van Loon
2d41f0885c p2p: Split MsgID into its own file as a public method (#9518)
* p2p: Split MsgID into its own file as a public method

* gofmt

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-02 21:48:14 +00:00
terence tsao
019f0257c2 RPC: sync committee duties (#9501)
* Add get altair duties

* Refactor, clean up, Rahul's feedback

* Gazelle

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-02 21:17:42 +00:00
Kirill Fedoseev
f319535af5 Fix countdown timer (#9484)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-02 20:41:01 +00:00
Radosław Kapka
c45fe5cc1c Implement GetStateV2 in the beacon API (#9506)
* implementation

# Conflicts:
#	beacon-chain/state/v2/BUILD.bazel

* Revert "Auxiliary commit to revert individual files from 2cbe98c88777cac071876fe97f85029fad964e51"

This reverts commit edc4ff52e7796aefd1782e31eaf40231a3134693.

* tests

* fix function call

* make state package visible to migration

* fix span names

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-02 19:28:55 +00:00
terence tsao
1f48accb0e RPC: Extend participation and performance to Altair (#9499)
* Add endpoints and tests

* Update beacon-chain/rpc/prysm/v1alpha1/beacon/validators_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-02 18:42:04 +00:00
terence tsao
b10964514a Implement SubmitPoolSyncCommitteeSignatures in the beacon API (#9464) (#9503)
* Implement `SubmitPoolSyncCommitteeSignatures` in the beacon API (#9464)

* Update server

* remove duplicated field

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-09-02 16:54:53 +00:00
Radosław Kapka
4b9fb1cd1b Add tests to GetBlockV2 (#9486)
* Add tests to `GetBlockV2`

* add missing error assert

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-02 13:08:47 +00:00
Marcin Górzyński
02966e64d8 Feature lru cache wrapper 2 (#9511)
* Add Wrapper to LRU Cache to handle Invalid Parameters #9461

* Regenerate BUILD.bazel and simplify tests using lru.Cache

* Fix: fuzz_exports.go build error

* Fix: block_fuzz.go

* Revert lru.Cache interface

* Remove redundant err check in pending_attestations_queue_test.go

* Add tests for lru wrapper

* Use lru package in prysm/shared instead of lruwrpr

* Fix: goimports

* Fix: BUILD.bazel

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-09-02 18:36:54 +08:00
Nishant Das
43a24e3d8b Add Subscription Handler (#9508)
* use subscriber

* fix up tests

* fix

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-02 06:10:02 +00:00
Nishant Das
59ed552c64 Buffer Errors For Resubscribers (#9513)
* fix blocker

* make test better

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-02 03:33:25 +00:00
terence tsao
95e07963fb Stategen: fix state by slot off by 1 (#9509)
* Fix state by slot off by 1

* Comment

* Go fmt

* Add err

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-02 03:05:08 +00:00
Raul Jordan
2cd2bc87d0 Sync Committee Message P2P Validations (#9481)
* add in handlers

* add topic switch

* begin refactor

* build

* revert change to sub file

* reordered

* add in handlers

* add topic switch

* fmt

* begin refactor

* build

* revert change to sub file

* reordered

* rationale for cache size

* simplify

* deep source

* kasey feedback

* ignore if not in committee

* add more unit tests

* more cov

* code review commentary

* pass tests

* commentary

* terence comments

* terence feedback on key for cache

Co-authored-by: nisdas <nishdas93@gmail.com>
2021-09-02 02:26:38 +00:00
Radosław Kapka
69ed46af0f Add RPC validator sync committee code (#9510) 2021-09-01 23:08:05 +00:00
Radosław Kapka
28e472aa5f Implement PrepareSyncCommitteeSubnets in the beacon API (#9498)
* initial implementation

* final implementation

* review

* fix logic

* refactor duration calculation

* tests

* revert alias change

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-01 17:19:29 +00:00
terence tsao
46b22bb649 Clean up committee helpers (#9504)
* Clean up committee related helper funcs

* Go fmt

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-09-01 15:17:10 +00:00
Nishant Das
44375a3d67 Handle Missing Status Message Better (#9505)
* throw error in status method

* fix up

* fix up

* gaz
2021-09-01 22:25:22 +08:00
Nishant Das
eb690d3737 Add Subscription Topic Handler (#9490)
* add methods

* zahoor's review
2021-09-01 12:07:20 +08:00
Preston Van Loon
dad03ade77 p2p: Return error if no ChainState found for a peer (#9502)
* p2p: Return error if no ChainState found for a peer

* add commentary
2021-08-31 22:25:54 +00:00
terence tsao
6ee0a7e811 Spectest: increase sizes from small to medium (#9500) 2021-08-31 21:28:09 +00:00
Nishant Das
0ea4e9a71c fix trie (#9495) 2021-08-31 08:01:26 -07:00
terence tsao
da5f37fc71 Clean up from hf1 (#9493)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-08-31 10:47:44 +00:00
Preston Van Loon
59e8523272 fix spectest timeouts and sizing (#9492)
* fix spectest timeouts and sizing

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-31 09:36:56 +00:00
Nishant Das
8d4b92962b Add in Remaining Altair Core Tests (#9489)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-08-31 07:18:45 +00:00
terence tsao
8f88d574f5 Stategen: replay altair epoch (#9468)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-30 23:57:08 +00:00
terence tsao
921acc300b Clean up ProcessInactivityScores (#9483) 2021-08-30 22:50:51 +00:00
terence tsao
516401537a Update to spec test beta3 (#9491)
* Update to beta3 spec tests

* `ProcessEpochParticipation` processes only active indices

* Go fmt

* Update epoch_precompute_test.go

* Update WORKSPACE

* Fix current and previous epoch switch up
2021-08-30 19:32:11 +00:00
terence tsao
572498f954 Fix deep source complains (#9485) 2021-08-29 20:57:37 +02:00
Raul Jordan
20a1a58860 Proposer Sync Aggregate RPC Methods (#9479)
* sync agg

* more readable

* pass tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-28 20:47:45 +00:00
Radosław Kapka
822522a3af Implement GetSyncCommitteeDuties in the beacon API (#9478)
* implementation

* tests

* review feedback

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-28 22:00:46 +02:00
Nishant Das
4d02329cd5 Add in P2P Changes (#9390)
* add in initial changes

* add test method

* raul's review

* Update beacon-chain/p2p/gossip_scoring_params.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/p2p/gossip_scoring_params.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* preston's review

* kasey's review

* only 1

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-08-27 09:34:20 +08:00
Raul Jordan
031830baa4 Remove Kafka from Prysm (#9470)
* remove kafka

* gaz

* rem foreign rules

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-26 23:59:00 +00:00
terence tsao
2cc9fc9e0e Add altair spec tests (#9477)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-26 23:08:04 +00:00
terence tsao
1284496648 Increase seen attestation cache sizes (#9476)
* Increase attestation cache sizes

* Update WORKSPACE

* Go fmt

* Fix tests

* Go fmt

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-26 21:37:35 +00:00
Raul Jordan
c1f841d6d4 Allow Sending Interfaces Over Application Event Feeds (#9458)
* add event feed changes

* deep source issues

* omit receiver name

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-08-26 15:21:29 -05:00
Raul Jordan
7d5f30b01e Delete Deprecated Flags and Old Slashing Protection for V2 (#9453)
* delete deprecated flags and old slashing protection schema

* remove toledo mentions

* build

* do not remove migration
2021-08-26 13:48:25 -05:00
Radosław Kapka
89941d4be2 Implement GetBlockV2 in the beacon API (#9433)
* Implement `GetBlockV2` in the beacon API

* fix gateway config test

* Revert "fix gateway config test"

This reverts commit 8179400b2a.

* unregister v2

* review feedback

* improve comment

* reduce duplication

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-26 15:22:06 +00:00
terence tsao
74df479bd3 Use active balance cache (#9469)
* Use active balance cache

* Feature flag and test

* Gazelle

* Fix fuzz

* Fix test

* Mix in validator count

* Update test to reflect validator count

* Add eth1 data block hash

* Fix assign

* Use eth1 data root

* Update active_balance_test.go

* Remove eth1 data root

* Remove eth1 data usage in tests

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-26 14:32:40 +00:00
Nishant Das
d22552944d Update Blockchain Package From Altair (#9422)
* add latest changes

* fix up

* raul's review

* pass cache tests

* Update beacon-chain/blockchain/head_sync_committee_info.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* kasey's review

* rename cache err

* add nil checks and error path checks

* gaz

* fix test

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-08-26 05:01:09 +00:00
terence tsao
2903275e7e Altair process slots (#9455)
* Update process slots

* More test scenarios

* Update transition_test.go

* Add CanUpgradeToAltair

* Go fmt

* Gaze

* Update beacon-chain/core/state/transition.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update transition_test.go

* Update altair fork epoch for e2e

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-26 09:59:23 +08:00
Preston Van Loon
3d6fad3121 Update Prater config to include Altair fork epoch (#9467)
* Update Prater config after https://github.com/eth2-clients/eth2-networks/pull/58. Add tests to enforce compliance.

* gofmt

* spec: true

* fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-25 17:18:29 +00:00
terence tsao
ffbb722777 Add active balance cache (#9456)
* Add active balance cache

* Better comment

* Fuzz

* ,

* go fmt

* Fix err

* Raul's feedback

* A few renames to effective
2021-08-25 16:39:12 +00:00
Radosław Kapka
8b10adc92a Fix struct type used when sending beacon committee subscriptions (#9466) 2021-08-25 15:35:28 +00:00
Nishant Das
0d60863bed Add Stategen Log (#9460)
* add log

* amend it

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-08-25 14:34:09 +00:00
Nishant Das
0208d9b328 Change P2P Logs to Trace Logs (#9459) 2021-08-25 15:35:50 +02:00
Raul Jordan
e150e29710 Add In Generated Mocks for Altair RPC (#9457) 2021-08-24 21:38:44 +00:00
terence tsao
45b6a80a30 Validator: perform sync committee duties (#9411)
* Validator sync committee methods

* Gazelle

* Update visibility

* Add setupWithKey

* Refactor selection proofs

* Fix build

* Refactor compute and sign

* Fix sign request

* Fix test

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-24 17:33:42 +00:00
terence tsao
cd4bb6a9c8 Add altair process epoch (#9449)
* Add altair process epoch

* Verify balance

* Gaz

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-24 16:07:11 +00:00
Nishant Das
114a14a4b6 Patch Initial Sync For Non Finalized Blocks (#9452)
* fix for now

* off by 1

* preston's review
2021-08-24 10:20:45 -05:00
Raul Jordan
a49c0f19ae Add Sync Committee Head State Cache (#9448)
* add sync committee head state cache

* handle err in init

* include file in build file

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-24 07:54:04 +00:00
Preston Van Loon
362212b5ed Fix 'headSlot' log statement in init-sync (#9451)
* Fix 'headSlot' log statement in init-sync

* gofmt -w beacon-chain/sync/initial-sync/round_robin.go
2021-08-24 07:00:18 +00:00
terence tsao
5c96a2713d Prevent Reinsertion of Orphaned Attestations Into Pool (#9442)
* Can re-save orphaned attestation

* Go fmt

* Fix tests

* Go fmt

* Bug fix flag

* Bug fix flag

* Go fmt

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-24 03:05:08 +00:00
terence tsao
6457ec17bf Process epoch: precompute previous and finalized epoch (#9445)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-24 01:42:05 +00:00
terence tsao
d3475a563d Remove Canonical Attestations From Pool Correctly (#9444)
* Remove canonical attestaitons

* Rm debug log

* Remove deleted test

* Tests and fmt

* Tests

* Bug fix flag

* Move feature flag

* Go fmt

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-24 00:33:17 +00:00
Nishant Das
ab101e8a4a Fix Validator Backups (#9446)
* fix val backups

* fix

* debug

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-23 21:46:04 +00:00
Preston Van Loon
5442d0f486 Altair: Merge passing hf1 tests and shared code (#9447)
* Altair: Merge passing hf1 tests and shared code

* Remove weird files

* del commented code

* Add dynamic file fetching, remove hardcoded list of files

* Remove hard coded paths

* remove hardcoded paths from phase0 too

* nosec fixes
2021-08-23 20:09:24 +00:00
Raul Jordan
0ee203fc2e Deduplicate Copy Functions In State Packages (#9437)
* remove duplicated copy functions

* add all unit tests

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-08-23 16:53:50 +00:00
terence tsao
63e0a4de84 Transition: Altair process block (#9430)
* Add altair transition

* Add tests

* refactor exit into operation funcs

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-08-21 00:32:37 +00:00
terence tsao
cac1e94f04 Add altair epoch processing functions (#9404) 2021-08-20 16:55:42 -07:00
Raul Jordan
bdfdba1e23 Remove Unused Trieutil Helpers (#9435)
* remove unused trieutil helpers

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-20 23:00:04 +00:00
Raul Jordan
e2a3dc13fb Less Flakey E2E API Checks (#9439)
* compare list attestations instead

* add query filter

* fix failing

* fix response

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-20 22:30:56 +00:00
Radosław Kapka
af2801f215 Implement GetEpochSyncCommittees in the beacon API (#9434)
* initial implementation

* get committee for epoch

* test

* fix failing test

# Conflicts:
#	beacon-chain/rpc/prysm/v1alpha1/validator/assignments_test.go

* combine function arguments

* review feedback

* use sync committee setter

* fix failing test

* fix build

* remove bad test

* refactor for single responsibilities

* radek suggestions

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-20 21:32:56 +00:00
Raul Jordan
fa4a4225a7 Improve DB.SaveBlocks Performance (#9431)
* improve save blocks performance

* comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-20 20:01:38 +00:00
terence tsao
005ce7e238 Remove float64 usage for fork choice (#9438) 2021-08-20 14:15:18 -05:00
terence tsao
25bc3f2aeb More altair spec tests (#9436)
* Add fork and ssz static spec tests

* Fix build

* Remove minimal tags
2021-08-20 16:47:37 +00:00
terence tsao
67c3ae0117 Add signature verification using current fork (#9405) 2021-08-20 07:38:35 -07:00
Raul Jordan
6368da2716 Remove Nested BoltDB View Within Update Transaction in SaveHeadBlockRoot (#9428)
* no nested bolt txs

* remove the nested view in update calls

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-19 21:42:30 +00:00
terence tsao
3764b7dca3 ValidatorIndexByPubkey length check (#9424)
* `ValidatorIndexByPubkey` length check

* gaz

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-19 19:33:05 +00:00
Raul Jordan
eac542a8ac Change Eth2 Repository Names (#9425)
* eth2 repo name changes

* rem sha

* use consensus spec terminology and pin sha
2021-08-19 13:00:57 -05:00
Nishant Das
9f36116c2f Update Core Helpers (#9393)
* add helpers

* naming

* remove comments

* Terence's feedback

* Revert `UpdateCommitteeCache` changes

* Update beacon-chain/core/helpers/committee.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-08-19 12:18:27 +00:00
terence tsao
d874b9140f Optimize process attestations (#9418) 2021-08-18 15:41:09 -07:00
Preston Van Loon
dfd740414f Altair: operations spectests (#9413)
* Update spectests to v1.1.0-beta.1 from hf1 branch

* fix params loading

* Altair: operations spectests

* gazelle and nosec on trusted variable inputs

* remove legacy statement from spectest/minimal/altair/operations/BUILD.bazel
2021-08-18 22:01:05 +00:00
Raul Jordan
72cfe50eff Move Validator Accounts Protos Into Proper Place (#9416)
* move into proper namespace

* package naming

* add strict deps

* regen protos

* add web api

* regen

* builds

* rename

* build
2021-08-18 21:24:01 +00:00
terence tsao
61acf6f1d1 Slash validator with altair config (#9415) 2021-08-18 18:44:17 +00:00
Radosław Kapka
0d8ceb18b0 Improve v2 protos (#9412)
* Generate SSZ code for `BeaconBlockContainerV2`

* new line at end of file

* remove unneeded V2 suffix

* more fun stuff

# Conflicts:
#	proto/eth/v1/generated.ssz.go

* rename Altair to V2
2021-08-18 15:50:37 +00:00
Nishant Das
5b7b9f9ac9 Refactor Subnet Methods For Altair (#9386)
* change subnets

* add in missing one

* add comment

* Update beacon-chain/p2p/subnets.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* terence's review

* terence's review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-18 04:55:48 +00:00
Nishant Das
8e503d0980 Change Back Read-Only Validator Declaration (#9410)
* remove indirection

* fix
2021-08-18 11:59:55 +08:00
Preston Van Loon
4966300c96 Update spectests to v1.1.0-beta.1 from hf1 branch (#9403)
* Update spectests to v1.1.0-beta.1 from hf1 branch

* fix params loading
2021-08-17 21:23:38 +00:00
Raul Jordan
bdb09ca9ea V1Alpha1 gRPC Gateway End-to-End Test Evaluators (#9375)
* add in initial evaluator for gateway v1alpha1

* add gazelle

* build

* compare

* functional options and tool for local verification

* chain head response compare base64

* compare validators request

* add validators compare

* more compare

* passing validator comparisons

* compare attestations pool

* compare atts

* begin peers test

* e2e verifiers

* correct port

* renaming

* evaluators

* rem api compare tool

* peer compare tests

* deepsource

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-16 20:31:21 +00:00
terence tsao
b28d65dc34 Add pyrmont hard fork config (#9394)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-16 19:51:43 +00:00
Preston Van Loon
643e20c50a p2p: report status failure when no genesis time is set after start (#9395) 2021-08-16 19:15:43 +00:00
Radosław Kapka
bb68c591f2 Remove beacon-chain/rpc/eth/v1 directory (#9392) 2021-08-16 13:11:25 +02:00
Nishant Das
f8355bb241 Add in Database Support For Altair State (#9388)
* add in schema

* add in altair state support

* nil check

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-16 12:39:52 +08:00
Raul Jordan
8122da6c97 Add Gosec Github Action (#9332)
* add gosec security scan

* add gosec ignores first batch

* more nosec for exec

* add filepath clean

* more nosec

* file inclusion nosec

* build

* herumi

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-15 15:24:13 +00:00
Radosław Kapka
1936f991eb Prepare protos for Ethereum API v2.0.0 (#9384)
* Move proto services to a different package

# Conflicts:
#	beacon-chain/rpc/service.go

* fix tests

* goimports

* fix java class name

* without aggregation_bits

* aggregation_bits

* v2 endpoints

* generate new code and define function stubs

* build files

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-14 16:41:03 +00:00
Nishant Das
49440c90ba Add in Database Support For Altair Blocks (#9387)
* add in schema

* gaz

* fix panic in rpc

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-14 15:50:49 +00:00
Nishant Das
47ee6fa17e Add in Altair Message ID Function (#9382)
* add in changes

* add method call

* gaz

* rauls review

* rauls review

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-14 14:36:02 +00:00
Radosław Kapka
de907fc362 Fix java classname in beacon_debug_service.proto (#9380)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-13 17:59:55 +00:00
Nishant Das
a1c0bee397 Add in Object Mapping For Types (#9381)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-13 17:28:26 +00:00
Raul Jordan
b42465a7dd Optimized Slasher Parameters (#9361)
* add in parameters and helpers files

* add in small set of changes

* build file

* viz
2021-08-13 16:53:04 +00:00
Nishant Das
8fe22422a6 add it in (#9383) 2021-08-13 10:53:51 -05:00
Radosław Kapka
f516e71167 Move proto services to a different package (#9379)
* Move proto services to a different package

# Conflicts:
#	beacon-chain/rpc/service.go

* fix tests

* goimports

* fix java class name
2021-08-13 10:55:24 +00:00
Radosław Kapka
ede85c21e2 Clean up v1 protos (#9377)
* Clean up v1 protos

# Conflicts:
#	proto/eth/v1/generated.ssz.go

* A little more cleanup

# Conflicts:
#	proto/eth/v1/generated.ssz.go

* fix java class names
2021-08-13 09:12:18 +00:00
Nishant Das
a9f2170657 Update P2PUtils And Parameters From Altair (#9373)
* add changes

* add p2putils changes

* clean up

* fix build

* remove test

* raul's review
2021-08-13 13:11:11 +08:00
Nishant Das
3a528147a2 Change to Proto Check For Blocks (#9372)
* check inner objects

* add test in here.

* verify better
2021-08-13 00:35:23 +00:00
Nishant Das
c8cffbbf02 Update Dependencies For Prysm (#9374)
* update deps

* tidy
2021-08-12 11:07:13 -05:00
Raul Jordan
1f102c256d Altair Committee Cache Changes (#9366)
* changes to hf1 caches

* suggested

* Update beacon-chain/cache/sync_subnet_ids.go

* add in proper changes

* committee.go

* committee test and disabled

* nil active balance check
2021-08-12 00:04:40 -05:00
terence tsao
e3492698f8 Core: modified process attestation for Altair (#9367)
* Add epoch participation and proposer base reward helpers

* Update beacon-chain/core/altair/attestation.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Check length

* Go fmt

* Add process att

* Refactor and tests

* Update attestation.go

* Gazelle

* Under flow error message

* Update beacon-chain/core/altair/attestation.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-11 21:58:25 -05:00
Jim McDonald
5256751e8b Trim SSE event names. (#9342)
* Trim SSE event names.

When sending SSE events the event name contains trailing whitespace.
This patch removes the whitespace before processing the event, ensuring
it is easily parsed by clients.

* Separate test.

* Test receiveEvents, not writeEvent.

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-08-11 23:00:39 +00:00
terence tsao
225f9a74f6 Add epoch participation and proposer base reward helpers (#9356)
* Add epoch participation and proposer base reward helpers

* Update beacon-chain/core/altair/attestation.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Check length

* Go fmt

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-11 20:54:45 +00:00
terence tsao
d77616f705 Fixes Issues Found by Goland IDE Code Inspect (#9368)
* Ran code inspect

* Update shared/testutil/altair.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-11 20:12:22 +00:00
Raul Jordan
a217d71d08 Sync Protobufs Between Hard Fork Branch and Develop (#9362)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-11 19:14:49 +00:00
Raul Jordan
efa1f29311 Use Different Golang JWT Library (#9360)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-11 18:10:53 +00:00
Raul Jordan
136b157d00 Better Slashing Protection Import/Export User Experience (#9355)
* failure mode if command goes wrong

* improve export

* fix struct orderw

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-11 17:24:24 +00:00
Raul Jordan
c1280867ed Add Tracing Request Sink to E2E (#9341)
* add tracing request sink

* fix struct order

* add in base64 encode and gzip

* add encoding and gzip

* tracing sink and replay tool

* post

* replay

* include latest sink and replay tool

* capture the gzout file instead

* rem time sleep

* handle err

* better handling

* add documentation

* changes

* working sync

* working

* added more logging
2021-08-11 15:38:00 +00:00
Nishant Das
0dd7a8b078 Fixes Altair Bug Introduced For State Representation (#9358)
* add test

* test

* test
2021-08-11 15:02:55 +00:00
terence tsao
41ab27402f Sync committee subnet cache (#9350)
* Add sync subnet id cache

* Update BUILD.bazel

* Update BUILD.bazel

* Update beacon-chain/cache/sync_subnet_ids.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Add comments

* add more comments

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
2021-08-11 00:26:20 +00:00
terence tsao
9668ab0fea Clean up duplicated imports (#9354) 2021-08-10 14:55:24 -05:00
Radosław Kapka
7a9c717ae0 Reduce locking time in recomputeFieldTrie (#9353)
* Reduce locking time in `recomputeFieldTrie`

* add comment

* reuse variable and improve comment
2021-08-10 09:54:35 +00:00
Nishant Das
a5998fab2f Clean Up State Representation (#9339)
* checkpoint all work

* add new fixes

* fixes

* gaz

* terence's review

* terence's review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-09 21:54:24 +00:00
terence tsao
8df6854a74 Core: upgrade state to Altair (#9349)
* Add upgrade to altair

* Update BUILD.bazel

* Update beacon-chain/core/altair/upgrade.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Move getter and setter outside the loop

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-09 21:27:51 +00:00
Jim McDonald
3de168b806 Fix state state transition logging. (#9343)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-09 20:05:46 +00:00
Preston Van Loon
bfdaf8a3f5 Fix config loader tests (#9348)
* don't mutate mainnet config

* add missing minimal config params

* Ensure all yaml fields were set and are correct

* gofmt

* Add sanity check that some empty yaml wasn't given

* Gazelle and deepsource feedback
2021-08-09 18:40:06 +00:00
terence tsao
38787353a2 Attestation: get participation status (#9340)
* Add participation flag indics matching helpers

* Gazelle

* Tests

* Update BUILD.bazel

* Update attestation.go
2021-08-09 16:35:46 +00:00
Nishant Das
170e2e115b Fix Flaky Sync Committee Test (#9337)
* fix flake

* Update beacon-chain/core/altair/sync_committee_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-09 14:49:13 +00:00
terence tsao
be85b7f909 Cleanup: remove test only process block/attestation (#9345)
* Remove test only process block attestation code

* Rm fuzz process block

* Revert "Rm fuzz process block"

This reverts commit 5d1b0f0741.

* Still can test fuzz

* Fix return

* More unused funcs
2021-08-08 22:01:06 -05:00
terence tsao
ad048a8750 Rewards/penalties: epoch processing for Altair (#9334)
* Add epoch precompute methods

* Better name

* Bring expensive computation outside

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-08-06 20:02:35 +00:00
Raul Jordan
90ea158301 Add Some More Traces to Blockchain Service (#9336)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-06 15:02:36 +00:00
Radosław Kapka
ef9eff29b8 Tiny state management improvements (#9338)
* Tiny state management improvements

* fix indices length calculation

* review comments

* revert function rename
2021-08-06 10:47:50 +00:00
terence tsao
4cce4f1ee5 Refactor slash validator func's signature (#9321)
* Refactor slash validator func's signature

* Named arguments

* Update beacon-chain/core/blocks/proposer_slashing.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-06 04:06:49 +00:00
Raul Jordan
ce10be9a16 Add Latest Slasher KV Methods to DB Subpackage (#9328)
* tests pass

* use end slot for epoch pruning

* amended to endslot of epoch

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-08-05 21:27:05 +00:00
Raul Jordan
4f0b00894b More Precise Metrics E2E (#9331)
* metrics test copute slots since genesis

* gazelle

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-05 20:44:56 +00:00
Radosław Kapka
57fac9b74f Improve API Middleware design (#9324)
* remove superfluous WriteHeader call

* move custom handlers outside hooks

* rename OnPostStart hook

* function that returns a default endpoint

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-05 20:21:43 +00:00
Mohamed Zahoor
395feeb35d Cross check db state bucket after migration (#9311)
* added state db check after migration

* satisfy deepsource

* added db compare after migration sub-command

* fomt issue

* point ot bolt issue

* gazel fix

* maxUint84

* revert accidental deletion of WORKSPACE file

* added sorting support

* cleanup debug

* gazel fix

* satisfy deepsource

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-05 18:30:05 +00:00
terence tsao
7bc22e4ddf Cache: sync committee index position (#9317)
* Add sync committee index cache

* Update BUILD.bazel

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-05 17:44:13 +00:00
Radosław Kapka
016beb51c0 Make SubmitAttestations v1 implementation similar to v1alpha1 (#9319)
* Make `SubmitAttestations` v1 implementation similar to v1alpha1

* fix test

* fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-05 08:01:20 +00:00
Radosław Kapka
456b4d463e Implement PrepareBeaconCommitteeSubnet in the validator API (#9318)
* validator migration

* status helpers

* implementation

* fix existing tests

* build file

* tests

* run endpoint through API Middleware

* reduce helper package visibility

* empty caches before each test

* assign all validators to subnets

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-08-05 07:23:52 +00:00
terence tsao
4d33068496 Core helper: validate sync message time (#9325)
* Add validate sync message time

* Update BUILD.bazel
2021-08-05 12:39:32 +08:00
terence tsao
6e9faa3618 Epoch: precompute functions for Altair part 1 (#9309)
* Add epoch precompute methods

* Split into part 1 for better review

* Update epoch_precompute_test.go

* Preston's feedback

* Skip if the valiator is not eligible

* Add regression test for non eligible

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-04 15:13:05 +00:00
Nishant Das
fd42cb712f Fix Incorrect Conditional Check (#9323)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-04 13:50:27 +00:00
Radosław Kapka
c5b68b4f03 Implement PublishAggregateAndProofs in the validator API (#9294)
* grpc implementation

* build file

* full implementation

* remove unused function

* tests

* wrap top-level array

* Update beacon-chain/rpc/eth/v1/validator/validator.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-04 09:01:38 +00:00
Mohamed Zahoor
239d904d27 Fix memory usability (#9310)
* fix cache usability

* fix the name of the prometheus flag

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-04 02:33:28 +00:00
terence tsao
461be99fac Block: process sync aggregate (#9308)
* Add altair base reward funcs

* Update BUILD.bazel

* Add process sync aggregate

* Update BUILD.bazel

* Update block.go

* Update block_test.go

* Feedback, refactor and more tests

* Further refactor sync committee votes

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-08-03 16:14:15 +00:00
Preston Van Loon
9572add171 Fail fast when using old define=ssz=<value> (#9316)
* Fail fast when using old define=ssz=<value>

* remove print statement
2021-08-02 19:04:30 +00:00
ahadda5
17cf2e5377 Fixing .bat file issues (#9266)
* replacing [] with "" for ifs
using !! vs %% for var with delayed expansion
Check first for 404 then download.

* remove the goto after if are fixed

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-08-02 16:09:09 +00:00
Radosław Kapka
e4388c7fce Update grpc-gateway dependency to proper version (#9315)
* logging

* properly update grpc-gateway dependency

* Revert "logging"

This reverts commit 6c59a87ac0.

# Conflicts:
#	go.mod

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-02 14:52:54 +00:00
Mohamed Mansour
47c28e6cd6 Show error in logs if passing invalid flags in yaml (#9033)
* Show error in logs if passing invalid flags in yaml

The YAML configs pass if an invalid flag is set, this causes frustration
because the flag could have been mispelled causing a huge mystery.

This will strictly check if the flags are deserialized, if a flag
doesn't exist, it will error out as a Fatal error with appropriate
reasoning which flag is wrong.

* Fix review comments to make some of them non-fatal

* Make yaml.TypeError a pointer

* Remove UnmarshalStrict from spectests since they use a different YAML package

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-08-01 03:26:24 +00:00
Preston Van Loon
0c1fc65604 Core/deposit: refactor blocks.ProcessDeposit for less complex altair implementation (#9305)
* Refactor blocks.ProcessDeposit to return whether or not the deposit resulted in a new validator entry.

* Only mark validator as new after successfully appending it to the state

* fix merge conflict

* fix merge conflict

* gofmt

* fix merge conflict

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-31 23:42:07 +00:00
terence tsao
5c36d59672 Reward: altair base reward (#9307)
* Add altair base reward funcs

* Update BUILD.bazel

* Better tests

* Update BUILD.bazel
2021-07-30 19:17:20 +00:00
Nishant Das
7576442ef0 Only Create Deposit Proofs Pre-Genesis (#9289)
* remove proof gen

* fix pre-genesis case

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-30 18:33:31 +00:00
terence tsao
d130bf0035 Fix more duplicated import prefix (#9306) 2021-07-30 18:05:38 +00:00
Mohamed Zahoor
0d818a5709 Reduce memory overhead during DB migrations (#9298)
* wip

* migration happening

* wip

* fix memory utilization, add the testcases

* migration issues

* the gazel

* remove debug logging

* goimport

* gazel
2021-07-30 08:38:54 -05:00
terence tsao
560fe69425 Core: sync committee helpers (#9269)
* Add sync committee helpers

* Create BUILD.bazel

* Update BUILD.bazel

* Update BUILD.bazel

* Fix visibility

* Preston's review
2021-07-29 23:19:32 +00:00
terence tsao
012d279663 Fix duplicated imports (#9304)
* Fix duplicated imports

* Fix metrics test
2021-07-29 16:45:17 -05:00
terence tsao
fcd4938ec0 Head: validator index getters (#9303) 2021-07-29 20:05:48 +00:00
Radosław Kapka
4db0dfc4f1 Properly broadcast attestations (#9301)
* better handling of gateway response

* empty response tests

* build file

* broadcast attestations properly

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-07-29 18:33:18 +02:00
terence tsao
cf4ff3d6f9 Clean up phase 0 spec tests (#9299) 2021-07-29 09:01:18 -05:00
Raul Jordan
9145310647 Eliminate Proto V2 Namespace (#9297)
* get rid of v2 in prysm codebase

* replace block2

* builds

* terence comments

* gazelle
2021-07-28 21:23:44 +00:00
Mohamed Zahoor
21c79f4dfa Protect the state objects in memory when saving them (#9291)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-28 18:27:27 +00:00
Raul Jordan
98f7f82b2f Copy Altair Changes Into v1alpha1 Namespace (#9296)
* move the v2 changes into v1alpha1

* v1alpha1 builds

* regenerate pbs

* regenerate ssz

* top level builds
2021-07-28 18:05:07 +00:00
terence tsao
f16b9859e7 Test: add altair test utility (#9287)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-07-28 17:15:23 +00:00
Nishant Das
a372e30c8f Fix Spaced Arguments for Subcommands (#9292)
* fix args for commands

* fix nasty edge case
2021-07-28 14:36:48 +00:00
Nishant Das
415973fca3 remove flag (#9290) 2021-07-28 16:11:22 +08:00
Raul Jordan
2d9915d2d9 Define Unimplemented V2 Services (#9286)
* generate ssz bindings for protos

* define services and unimplemented

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-27 20:42:14 +00:00
Mohamed Zahoor
c6bd7a2f83 Efficient Historical State Representation (#9155)
* added migration in to new schema

* gazel fix

* goimports fix

* construct state with validator entries from a seperate bucket

* save state with validator entry indirection

* fixed save and retreieve issues

* fixed more test cases related to DeepEqual

* added save benchmark

* gazel fix

* organize benchmarks

* add cache and improve state construction  performance

* gazel fix

* check type

* remove cache checking from Has

* fix decoding when cache is off

* fix slice bugs

* add migration testcases

* linter fix

* fix few review comments

* fix review feedback

* gazel fix

* satisfy deepsource

* added the feature flag

* lint fix

* fix usage of featureconfig Inti in testcases

* call resetConfig method

* add feature flag in  migration testcases

* fix formatting

* change bucket name from blockRootValidatorKeysIndexBucket to blockRootValidatorHashesBucket

* remove from cache when state deleted

* fixed few more comments

* added to devModeFlags

* added cache delete under the feature flag

* fix lint

* change cache sizes and improve documentation

* fiexed few more review coments

* not using hash anymore and using a new SaveStates function

* satisfu deepsource

* run gazel

* fix feature flag related stuff

* fixing merge conflict fix

* few goodies

* improve UX and dont swallow error

* merge fix

* import format fix

* fix migrationion when flag not given issue

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-07-27 20:02:53 +00:00
terence tsao
7129af5ccf Core/deposit: altair processing methods and tests (#9227) 2021-07-27 12:10:08 -07:00
Raul Jordan
f700808c8b Generate SSZ Bindings for Latest Protos (#9285) 2021-07-27 17:57:11 +00:00
Raul Jordan
1ea1c69174 Copy V1Alpha1 Protos and API Endpoints To V2 Namespace (#9284)
* v2 builds

* generate go code

* fix build
2021-07-27 16:54:49 +00:00
terence tsao
54772596e0 State/V2: rest of pkg (#9283)
* Add rest of state v2

* Update BUILD.bazel
2021-07-27 15:47:03 +00:00
Nishant Das
a556cf27c4 Use Only Checkpoint Cache for Processing Attestations (#9282)
* use only one cache

* clean up
2021-07-27 07:26:53 -07:00
terence tsao
5b5c8f4d3f State/V2: setters (#9279)
* Add state v2 getters

* Update BUILD.bazel

* Add state v2 setters

* Update BUILD.bazel
2021-07-26 17:27:05 -05:00
terence tsao
245dc23cde State/V2: getters (#9228)
* Add state v2 getters

* Update BUILD.bazel
2021-07-26 20:55:30 +00:00
Radosław Kapka
837cd4eb8f Implement ProduceAttestationData in the validator API (#9271)
* functionality

* test

* build file

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-07-26 19:22:58 +00:00
Radosław Kapka
8c8f1bb9c1 Regression test for #9264 (#9277) 2021-07-26 11:48:51 +00:00
Kaan
a8c49c50ad v1alpha1 validators: catch possible reqState nil case (#9275)
* v1alpha1 validators: catch possible reqState nil case

* Minor format of error log

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-07-26 13:50:40 +08:00
Lorem Ipsum
14d0d9195f Fixing wrong node information in fallback log (#9258) (#9272) 2021-07-25 23:46:54 +02:00
terence tsao
8db0c9a0e6 Move proto/interfaces -> proto/prysm/v2 (#9270)
* Move proto/interfaces -> proto/prysm

* Update paths

* Fix block2

* Update blocks_fetcher_utils.go

* Update BUILD.bazel

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-23 20:10:15 +00:00
Preston Van Loon
938a038c42 Fix //... for darwin (#9273)
* Fixes

* More manual tags

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-23 18:26:49 +00:00
Raul Jordan
30e93f5763 Move Slashing Protos to V2 (#9256)
* proto slashing

* move slashing related protos

* gaz

* include slasher pbs consolidated

* slasher pb fmt imports

* imports

* beacon builds

* build validator

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-23 18:01:01 +00:00
Radosław Kapka
be82c8714f Do not require a handler function in the gateway (#9264)
* Do not require a handler function in the gateway

* test fix

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-23 16:48:22 +00:00
Preston Van Loon
c0076cc7a2 State: move state interfaces into state package (#9268)
* Move interfaces from github.com/prysmaticlabs/prysm/beacon-chain/state/interface to github.com/prysmaticlabs/prysm/beacon-chain/state

* remove/rename state2
2021-07-23 16:11:21 +00:00
Radosław Kapka
a8363405f8 Implement ProduceBlock in the validator API (#9259)
* server-side implementation

* build file

* run through api middleware

* tests

* build file

* gofmt

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-07-23 16:29:38 +02:00
Raul Jordan
2d10bcf179 Move State Protos Into V2 Namespace (#9257)
* move state protos

* regen ssz

* edit v1 code

* fix imports

* building

* beacon chain builds

* validator and shared builds

* fuzz builds

* changes

* spectest builds

* tools build

* remove import cycle

* generate ssz

* pcli

* gaz

* kafka

* gaz
2021-07-23 08:07:40 -05:00
terence tsao
526596a679 State/v1: add atts and block roots getter tests (#9255)
* Add att and block getter tests

* Update BUILD.bazel

* Update getters_attestation_test.go

* Update beacon-chain/state/v1/getters_block_test.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Fix mutation tests

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-07-22 22:31:14 +00:00
Raul Jordan
5864795ca5 Move RPC Protos Into V2 Namespace (#9254)
* moved rpc protos

* gazelle

* change pb

* validator health

* edit name

* naming
2021-07-22 21:00:28 +00:00
Raul Jordan
b7919b3115 Move DB Protos Into V2 Namespace (#9253)
* move db protos into prysm v2

* build

* build

* imports gaz

* v2

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-22 20:05:06 +00:00
Raul Jordan
715aa408e3 Move Web Protos Into Prysm V2 (#9249)
* move web protos

* move web to v2

* update web protos

* gaz

* proto build

* replace mentions of validator v2

* gaz
2021-07-22 19:19:24 +00:00
Raul Jordan
ceb1ec451a Remove Faucet Protos (#9252)
* rem faucet protos

* tidy
2021-07-22 18:27:31 +00:00
Radosław Kapka
b35c3ab2e7 Implement GetAggregateAttestation in the validator API (#9243)
* server-side implementation

* run through api middleware

* tests

* build file

* build file fix

* return attestation with most aggregation bits

* build file

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-22 17:48:32 +00:00
terence tsao
c8685d256c Lint: add v1 as import alias (#9250)
* Add v1 as import alias

* Update helpers.go

* More

* Update skip_slot_cache_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-22 17:13:18 +00:00
Raul Jordan
0fb43aec34 Remove Unused Cluster Protos (#9251)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-22 16:39:27 +00:00
terence tsao
9addb5f7c3 Core/epoch: sync committee period (#9248)
* Combine function parameters

* Add sync committee epoch calculations

* Update BUILD.bazel
2021-07-22 16:13:42 +00:00
Nishant Das
33f7582d25 Add Doppelganger E2E (#9239)
* add current fixes

* fix prev release

* Update endtoend/endtoend_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update endtoend/endtoend_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update endtoend/endtoend_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* review comments for radek and raul

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-07-21 23:00:12 -05:00
Raul Jordan
6dadb80cc4 Move v1alpha1 into Prysm API namespace (#9245) 2021-07-21 21:34:07 +00:00
Radosław Kapka
ae140073e7 Remove proposer_sync_aggregate.go (#9231)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-07-21 19:35:18 +00:00
terence tsao
b667d30d3f Feature/UpdateHeadTimely: cache finalized checkpoint when initial sync (#9244) 2021-07-21 11:55:27 -07:00
terence tsao
e7bdf35721 Fix unused variables (#9234)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-21 16:06:57 +00:00
Nishant Das
c780301096 Remove Validator DB Cache (#9238)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-21 15:26:07 +00:00
terence tsao
e1d543a77b Combine function parameters (#9242) 2021-07-21 14:55:37 +00:00
Nishant Das
2ffe8336fc Fix Parsing of Nested Subcommands (#9236) 2021-07-21 13:01:33 +02:00
Nishant Das
11d9e7da9b Update Libp2p to v0.14.4 (#9108)
* fix

* tidy

* fix deps

* tidy

* fix pubsub
2021-07-20 23:50:47 +00:00
Radosław Kapka
53e02b1601 Log correct warning when disabling flags (#9230)
* Log correct warning when disabling flags

* use interface

Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
2021-07-20 16:54:08 +00:00
Preston Van Loon
3a35a953bf proto/prysm/v2: Introduce Altair pb changes (#9222)
* Copy v2 protos from hf1

* Remove some extra Altair naming

* Build with github.com/ferranbt/fastssz/pull/50

* Regen pb.go

* Update fastssz to master
2021-07-19 20:42:06 +00:00
Preston Van Loon
8d1c9fe1e2 epoch/precompute: Add changes from Altair (#9219)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-19 19:39:23 +00:00
terence tsao
4229b466ae Add sync committee pool pkg (#9203)
* Add sync committee pkg

* Fix err variable namings
2021-07-19 19:06:49 +00:00
Preston Van Loon
412ddbb29e beacon-chain/state/v1: ReadOnlyValidator wrapper constructor method (#9221)
* Introduce changes from Altair hf1 branch

* go pkg viz changes

* Fix test

* goimports for some reason

* Use a more safe method of wrapping validators with regards to nil validators. Add basic tests for this wrapped validator

* Use a more safe method of wrapping validators with regards to nil validators. Add basic tests for this wrapped validator

* Panic fixes

* Fix tests

* remove nil validator test as it is no longer possible

* goimports for some reason

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-19 18:26:39 +00:00
Radosław Kapka
8a7010f5aa Implement GetProposerDuties in the validator API (#9223)
* endpoint implementation

* extract common code

* tests

* remove named return values

* correct test log

* sort duties by slot

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-07-19 18:00:15 +00:00
Nishant Das
40a96bc2b6 Invert Update Head Timely Flag (#9224)
* invert

* Update shared/featureconfig/flags.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-19 17:23:43 +00:00
terence tsao
1beb0071b5 State/v2: Add InitializeFromProto and InitializeFromProtoUnsafe (#9226)
* Add `InitializeFromProto`

* Update BUILD.bazel
2021-07-19 16:47:42 +00:00
terence tsao
2a0c4e0d5f Use block wrapper and interface for propose block (#9218) 2021-07-19 15:51:38 +00:00
Nishant Das
dad205bd04 Invert Optimized Balance Update Feature (#9225) 2021-07-19 14:27:20 +00:00
Preston Van Loon
15bfcf8ff6 Add tests for wrapped Altair objects (#9213)
* start tests for wrapped methods

* Add a few tests and skeletons for interface impl methods

* Add latest interfaces update from hf1 and stub the appropriate tests

* Add a few more tests

* More progress

* Migitate nil pointer issues in construct. Complete tests

* Copy changes from hf1 to v1alpha1 wrapper

* Use testutils from #9201

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-18 19:32:52 +00:00
terence tsao
15704053e1 Add Altair configs (#9150)
* Add altair configs

* Update minimal_config.go

* Update workspace and config tests

* Revert workspace changes

* Update WORKSPACE

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-07-18 16:18:43 +00:00
terence tsao
5be1ccd3cf Hydrate altair block (#9201)
* Hydrate altair blocks

* Update BUILD.bazel

* HTR test

* Update BUILD.bazel

* Comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-16 23:35:37 +00:00
terence tsao
97e6eb14e2 Altair block wrapper: update return to interfaces (#9212) 2021-07-16 23:09:45 +00:00
Mohamed Zahoor
6f126c92c0 Make exportdb tool faster (#9184)
* added prodecer consumer pattern for reading and printing

* make state interface visible in exploredb

* bazel magic for visibility

* fix manual visibilithy marking

* linter fix

* parallelized bucket stats

* add log.WithErr and log.Infof

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-07-16 22:24:16 +00:00
Radosław Kapka
223d5309a0 Implement GetAttesterDuties in the validator API (#9207)
* remove irrelevant comment

* fix proto documentation

* first implementation

* register validator server

* first working version

* add sync checker and test file

* first test

* gzl

* final version + tests

* gzl

* run duties through API Middleware

* extract dependent root getter

* dependentRoot docs

* wrap indices in array

* fix static analysis issues

* modify err nil check

* create local variables in slot processing test

* Update proto/eth/v1/validator_service.proto

* review

* simplify index loops

* better calculation of committees at slot

* comment about impossible comndition

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-16 20:55:22 +00:00
Preston Van Loon
31d2482f9c Add proto wrappers for Altair (#9209)
* Starting attestation test

* Commiting checkpoint progress before dinner

* Add remainder of tests

* Copy hf1 cloners and add tests

* Introduce wrapper changes from hf1

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-16 20:30:26 +00:00
terence tsao
3ef0c5d6e5 Fix deepsource: shadow of builtin detected (#9210)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-07-16 20:04:30 +00:00
Preston Van Loon
187a1ca53d shared/copyutil: Add copy methods for Altair (#9208)
* Starting attestation test

* Commiting checkpoint progress before dinner

* Add remainder of tests

* Copy hf1 cloners and add tests

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-07-16 19:30:48 +00:00
Preston Van Loon
17d7fc492e shared/copyutil: Add tests for copy methods (#9204)
* Starting attestation test

* Commiting checkpoint progress before dinner

* Add remainder of tests

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-07-16 16:56:09 +00:00
Preston Van Loon
f029fdd44a beacon-chain/state/stateutil: Fix benchmark target (#9199)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-16 14:39:14 +00:00
Preston Van Loon
44d266313e testutil: Add NotEmpty method (#9202) 2021-07-15 22:32:10 +00:00
terence tsao
1a9a46d042 Block util: block header from block interface (#9196)
* Add beacon block header from block interface

* Update BUILD.bazel

* Update shared/blockutil/block_utils.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Go fmt

* Remove unused bodyroot

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-07-15 16:03:46 +00:00
Preston Van Loon
71c8c75bc3 shared/queue: Add RetrieveByKey method (#9195)
* Checkout shared/queue changes from hf1 branch

* Remove logic that removes and reinserts an item into the queue at the back

* Add test to ensure the correct thing was returned

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-07-15 10:31:19 -05:00
terence tsao
b22935a724 Rename AttestationNotifier (#9197) 2021-07-14 18:32:03 -07:00
Radosław Kapka
3ff5b95945 Remove database field from v1alpha1 validator server (#9191)
* Remove database field from v1alpha1 validator server

* remove field

* gzl

* remove not needed variable

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-14 13:16:21 +00:00
Nishant Das
e6bd5f31b1 Cleanup Fast Aggregate Verify (#9193) 2021-07-14 11:12:50 +00:00
terence tsao
9a4ab41761 BaseReward test only (#9189)
* Move baseward to test only

* Update BUILD.bazel

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-07-13 20:12:53 +00:00
terence tsao
7f0749c398 Clean up shared interface package (#9172)
* Remove shared interface folder

* Rename file names

* Gazelle

* Move interface to /beacon/p2p/

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-07-13 18:58:43 +00:00
Nishant Das
d73ac71ee6 add std out for beacon node (#9183) 2021-07-13 08:45:26 -05:00
terence tsao
6ba741f251 Calculate epoch outside of loop (#9178) 2021-07-12 12:35:36 -07:00
Radosław Kapka
4ebb008bc9 Protos and stubs for validator API endpoints (#9174)
* match proto definitions with spec

* stubs for grpc server's functions
2021-07-12 10:38:55 -05:00
Nishant Das
a9ee3ee06a Update Go-Ethereum (#9157)
* update

* tidy

* fix

* fix e2e

* fix e2e

* fix bad tx opts
2021-07-09 07:58:43 -05:00
Preston Van Loon
7c23d02c3e Remove cluster-pk-manager tool (#9165)
* Remove cluster-pk-manager tool

* Go mod tidy
2021-07-08 13:31:41 -05:00
Preston Van Loon
9dc3dd04c7 Update go-bitfield (#9162) 2021-07-08 08:31:40 -07:00
ahadda5
72886986ea prysm scripts should not save 404 pages (#9072)
* first .sh attempt, checked curl

* removed echo

* more meaningful msg

* prysm.bat changes

* do not save 404, using -f and grep

* removed extra line

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-07-06 23:34:42 +00:00
terence tsao
1d3a9983cc Move block interface next to generated pb (#9146)
* Move block interface next to pb

* Update fuzz build bazel

* Move interface to /proto/interface and wrapper next to generated pb

* Fix fuzz build bazel

* Add //proto/eth/v1alpha1/wrapper

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-06 15:34:05 +00:00
Mohamed Zahoor
cca4a8c821 exploredb tool to display bucket contents (#9116)
* add state bucket content display test case

* save work

* state db disection methods

* fix go.mod

* gazel fix

* pass the testcase if the db is not present. for CI

* get db path from env. optimize code

* update deps.bzl

* go mod tidy

* revery deps.bzl

* moved bucket content inspection to exploredb tool

* satisfy deepsource and tidy go.mod

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-07-06 15:11:11 +00:00
terence tsao
6c8fd745a4 Better import alias names (#9149) 2021-07-06 09:37:33 -05:00
terence tsao
466546bb40 Add sync committee aggregation_bits tags for minimal and mainnet (#9147)
* Add aggregation_bits tags for minimal and mainnet

* Update go-bitfield

* Go mod tidy
2021-07-03 18:59:35 -05:00
terence tsao
1ff4605398 Add priority queue tests (#9142)
* Add priority tests

* Gazelle

* bazel

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-07-02 21:54:28 +00:00
terence tsao
2ae6452cb5 Add Eth2FastAggregateVerify (#9144)
* Add `Eth2FastAggregateVerify`

* Update propose_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-07-02 19:27:08 +00:00
terence tsao
00907fd7d1 Update log test and ChainInfoFetcher interface (#9143) 2021-07-02 14:01:37 -05:00
terence tsao
1edf1f4c87 Restructure RPC packages to be future compatible (#9136)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-02 15:54:52 +00:00
terence tsao
877cb9f6e6 Add priority queue from hf1 (#9138)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-02 15:31:32 +00:00
Nishant Das
5d65ace970 Implement Doppelganger Check (#9120)
* checkpoint changes

* Update beacon-chain/rpc/validator/status.go

Co-authored-by: Potuz <potuz@potuz.net>

* Update beacon-chain/rpc/validator/status.go

Co-authored-by: Potuz <potuz@potuz.net>

* add in client side tests

* add ordering

* add all new test cases

* gate feature

* handle edge case

* add one more test case

* fatal error

* zahoor's review

* Update validator/client/validator.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update validator/client/validator.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* doppelganger not doppleganger

* preston's review

* add in comment

* change comment

* Fix e2e to only run new flags on the current version

* Fix bug where zero byte public keys were always sent in the request when attestation history existed. Still some tests to fix due to another bug in attester protection AttestationHistoryForPubKey.

* go mod tidy, gazelle

* Increase test size

* fix timeout, change size back to small

Co-authored-by: Potuz <potuz@potuz.net>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-07-02 12:11:09 +08:00
Preston Van Loon
2df024afc6 Return pubkey with AttestationHistoryForPubKey AttestationRecord response (#9135)
* Return pubkey with AttestationHistoryForPubKey AttestationRecord response

* Fix tests
2021-07-02 05:50:56 +08:00
Preston Van Loon
ca7b312761 Bazel: set alias targets to manual only (#9134) 2021-07-01 14:48:09 +00:00
Preston Van Loon
d6bd619429 Bazel: minimal test build transitions (#9122)
* Add build transitions for minimal config based tests

* respect existing gotags

* clean up import

* Add some commentary

* gaz

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: rauljordan <raul@prysmaticlabs.com>
2021-06-30 19:22:09 +00:00
Preston Van Loon
24d17a536c Return an error on unrecognized arguments when running default commands in Prysm (#9129)
* Return an error on unrecognized arguments when running default commands in Prysm

* fix cases for subcommands

* Deepsource suggestion
2021-06-30 16:24:04 +00:00
Raul Jordan
b23f63a064 Beacon State V2 Interface Definition With Semantic Version Paths (#9125)
* v2 state initialize and semantic paths

* ensure build

* gaz changes to ignored build files

* gaz
2021-06-30 15:06:19 +00:00
terence tsao
4fb3e05124 Add search for most profitable sync contribution (#9121)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-29 17:26:01 +00:00
terence tsao
cd3a2e87fe Naive aggregation for SyncCommitteeContribution (#9114)
* Copy CopySyncCommitteeContribution

* Update BUILD.bazel

* Add naive aggregation for sync contribution

* Gazelle

* Update deps.bzl

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-06-29 16:11:11 +00:00
Radosław Kapka
0347b4b4af Retrieve validators through v1 API in e2e (#9117)
* Retrieve validators through v1 API in e2e

* fix comment

* simplify boolean comparison
2021-06-29 09:47:59 -05:00
Preston Van Loon
0cfc16ec3b Reorder WORKSPACE go dependency registration (#9113)
* Reorder registration of dependencies so that Prysm go deps are registered first and everything else is registered after.

* Build fixes for gazelle

* Another nofuzz fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-29 00:04:26 +00:00
terence tsao
ea34af4abf Add proposer sync contribution type (#9106)
* Add `proposerSyncContributions`

* Update BUILD.bazel

* Add dedup and tests

* Update BUILD.bazel

* Revert grpc changes
2021-06-28 13:54:33 -05:00
Mohamed Zahoor
0f2e6fb8d5 Tool for Exploring BoltDB in Human-Readable Formats (#9063)
* exploredb tool

* bucket stats tool

* bucket stats tool

* satisfy deepsource

* format error

* add flag to get the db file

* revert go get og humanize

* gazel fix

* fix merge errors

* remove bucket-stats

* satisfy deepsource

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-28 16:59:20 +00:00
Radosław Kapka
43623e1089 Handle comma-separated query parameters in API requests (#9102)
* return empty array when no peers found

* add missing query params to endpoint factory

* fix validator endpoints in endpoint factory

* peers test

* Move API param handling to separate file

* Handle comma-separated query parameters
2021-06-28 15:37:14 +00:00
Radosław Kapka
940ce0c292 Make Eth APIs flag name implementation agnostic (#9112)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-28 15:12:39 +00:00
Radosław Kapka
71f1f19334 Small API Middleware improvements (#9111)
* extract creating DefaultErrorJson objects

* do not use default http client for proxying

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-28 11:36:12 +00:00
Nishant Das
f75548ad1e Protect Against Unsafe Coefficients (#9109) 2021-06-28 11:53:03 +02:00
terence tsao
442f8d1d3c Remove old protos (#9107) 2021-06-28 07:41:37 +00:00
ahadda5
349f832bd3 No length check in AggregatePublicKeys (#9105)
* check of len and nil

* minor edit

* minor edit

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-06-27 16:13:55 +00:00
Raul Jordan
a860648960 Reduce Usage of Eth2 Terminology in Prysm (#9104)
* remove all mentions

* more changes

* folder by folder

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-26 19:00:33 +00:00
Radosław Kapka
dd0ae1bbef Move API param handling to separate file (#9101)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-26 18:42:54 +00:00
Radosław Kapka
9eb3ff6bdf Fix a few bugs in the API (#9099) 2021-06-26 11:17:45 -07:00
Radosław Kapka
b114247836 Move common gateway registration code to new package (#9092)
* Move common gateway registration code to new package

* no shared inside of beaconchain

* beacon gateway

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-25 11:02:11 +00:00
Raul Jordan
8f5c0833ff Block Stream V2 Protos (#9096) 2021-06-25 00:05:38 +00:00
Radosław Kapka
43655e6c04 Remove verbatim strings with flag names on startup (#9093)
* Remove verbatim strings with flag names on startup

* gzl

* fix visibility

* gaz

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-24 17:20:47 +00:00
Radosław Kapka
ff99a2ed40 Run text/event-stream values returned from grpc-gateway through API Middleware (#9080)
* register events handler in v1

* begin `status.Error` messages with uppercase

* something works

* handle errors

* topic comments

* handle the rest of event types

* gzl

* make code more testable + test

* better error handling

* tests

* gzl

* fix deps

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-24 16:18:28 +00:00
Raul Jordan
860716666b Add Altair Protobuf Changes to Prysm (#9087)
* all altair protobuf additions

* add altair changes to deps
2021-06-24 10:32:30 -05:00
Raul Jordan
1e99ec30cb Revert "Improve performance during CheckAttesterDoubleVotes" (#9085)
This reverts commit a958dd246b.
2021-06-23 21:19:22 +00:00
Preston Van Loon
1f6a031630 Add a test for e2e validator to run against prior release (#9042)
* Add a test for e2e validator to run against prior release

* gofmt

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-23 17:07:28 +00:00
Preston Van Loon
942472fc7e Add goimports check (#9075)
* Add goimports check

* lil fixes

* lil fixes

* trying to use someone elses project instead of grinding on it myself
2021-06-22 19:57:26 +00:00
Preston Van Loon
228369e95f Double java heap size for buildkite (#9073)
(cherry picked from commit 9e28840b95cc96620841053902bf22889d69ea1b)
2021-06-22 18:28:34 +00:00
Nishant Das
9ee086feaf Disable No Freelist Sync In Backups (#9069) 2021-06-21 15:14:55 +02:00
Nishant Das
6e42f1bc5d add lock (#9068) 2021-06-21 16:33:06 +08:00
Raul Jordan
4ec0d60911 Prevent Exporting False Positive Slashing Event Due to Old Prysm DB Schema Bug (#9060)
* regression test added

* better comment

* export edge case

* add case for edge test

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-21 05:15:06 +00:00
David Shoots
2583dc1279 Remove noisy / unneeded log.Error() calls (#9066) 2021-06-20 13:31:27 -07:00
Raul Jordan
a747d151ba Fix Small Type Assertion Bug in Slasher (#9061) 2021-06-18 18:50:20 +00:00
Radosław Kapka
4f3c17cf69 Make the shared/gateway package unaware of consuming services (#9050)
* gateway refactor

* test customizations

* change variable names

* fix BUILD file

* remove unused code

* move mux registration out of gateway

* fix gateway tests

* better name for struct

* fix mock endpoint factory

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-18 13:35:13 +00:00
terence tsao
a4be7c4a46 Testutil: Reset deposit cache internally (#9059)
* Reset testutil cache internally

* Fix import

* Fix test to correct error

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-18 04:05:40 +00:00
Alexander Mollohan
ae7e2764e0 Adding ability to override directory permissions for db backup (#8888)
* Adding ability to override directory permissions for db backup

* adding new flag to usage.go

* Backup now has its own directory handling method to reduce footprint of prospective change

* removing unneeded test

* fixing build error

* switching to url param for override

* stripping flag

* adding in unit tests

* fixing a test

* minor changes for testing

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-18 01:37:58 +00:00
Radosław Kapka
fbed11b380 Do not register API Middleware with validator gateway (#9055)
* Do not register API Middleware with validator gateway

* add nil checker

* revert bool

* nil check

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-18 00:38:15 +00:00
terence tsao
4d1b5f42af Better comments for aggregator functions (#9053)
* Better comments for aggregator functions

* Naming returned value
2021-06-17 17:28:19 +00:00
Mohamed Zahoor
e953804239 Process future Blocks and Attestations (#8975)
* queue future blocks

* process early attestations

* fix formatting

* fix testcases

* fix signature testcase and move verify slot time to original place

* Add testcases to test early blocks and slot processing

* added test case to cover failure case while inserting in block pending queue

* satisfy deepsource

* fix review comments

* remove invalid testcase

* do not queue blocks which are within MAXIMUM_GOSSIP_CLOCK_DISPARITY

* format fix

* added a helper to check of the block slot is within MAXIMUM_GOSSIP_CLOCK_DISPARITY

* add helper to check clock disparity and test case for the same

* deepsource fix

* satisfy deepsoruce

* sysc/service.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-17 00:17:52 +00:00
terence tsao
3c0a9455a2 Rename field ops service to att service (#9046)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-16 21:48:56 +00:00
terence tsao
7d4418e2fe Go imports (#9045)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-16 14:28:04 +00:00
Preston Van Loon
9dc4c49fbd Update web-ui to beta.4 (#9047)
* add site data

* gofmt

* fix embed

* fmt

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-16 13:36:43 +00:00
Preston Van Loon
6971245e05 Revert "Invert attest-timely flag" (#9044)
This reverts commit 5fe98ab260.
2021-06-15 20:48:47 +00:00
Radosław Kapka
12403d249f [Feature] - API Middleware (#8926)
* HTTP proxy server for Eth2 APIs (#8904)

* Implement API HTTP proxy server

* cleanup + more comments

* gateway will no longer be dependent on beaconv1

* handle error during ErrorJson type assertion

* simplify handling of endpoint data

* fix mux v1 route

* use URL encoding for all requests

* comment fieldProcessor

* fix failing test

* change proxy port to not interfere with e2e

* gzl

* simplify conditional expression

* Move appending custom error header to grpcutils package

* add api-middleware-port flag

* fix documentation for processField

* modify e2e port

* change field processing error message

* better error message for field processing

* simplify base64ToHexProcessor

* fix json structs

* Run several new endpoints through API middleware (#8922)

* Implement API HTTP proxy server

* cleanup + more comments

* gateway will no longer be dependent on beaconv1

* handle error during ErrorJson type assertion

* simplify handling of endpoint data

* fix mux v1 route

* use URL encoding for all requests

* comment fieldProcessor

* fix failing test

* change proxy port to not interfere with e2e

* gzl

* simplify conditional expression

* Move appending custom error header to grpcutils package

* add api-middleware-port flag

* fix documentation for processField

* modify e2e port

* change field processing error message

* better error message for field processing

* simplify base64ToHexProcessor

* fix json structs

* /eth/v1/beacon/states/{state_id}/validators

* /eth/v1/beacon/states/{state_id}/validators/{validator_id}

* /eth/v1/beacon/states/{state_id}/validator_balances

* /eth/v1/beacon/states/{state_id}/committees

* allow skipping base64-encoding for query params

* /eth/v1/beacon/pool/attestations

* replace break with continue

* Remove unused functions (#8924)

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Process SSZ-serialized beacon state through API middleware (#8925)

* update field names

* Process SSZ-serialized beacon state through API middleware

* revert changes to go.mod and go.sum

* Revert "Merge branch '__develop' into feature/api-middleware"

This reverts commit 7c739a8fd7, reversing
changes made to 2d0f8e012e.

* update ethereumapis

* update validator field name

* update deps.bzl

* update json tags (#8942)

* Run `/node/syncing` through API Middleware (#8944)

* add IsSyncing field to grpc response

* run /node/syncing through the middleware

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Return HTTP status codes other than 200 and 500 from node and debug endpoints (#8937)

* error codes for node endpoints

* error codes for debug endpoints

* better comment about headers

* gzl

* review comments

* comment on return value

* update fakeChecker used for fuzz tests

* fix failing tests

* Allow to pass URL params literally, without encoding to base64 (#8938)

* Allow to pass URL params literally, without encoding to base64

* fix compile error

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Process SSZ-serialized beacon state through API middleware (#8925)

* update field names

* Process SSZ-serialized beacon state through API middleware

* revert changes to go.mod and go.sum

* Revert "Merge branch '__develop' into feature/api-middleware"

This reverts commit 7c739a8fd7, reversing
changes made to 2d0f8e012e.

* update ethereumapis

* update validator field name

* update deps.bzl

* update json tags (#8942)

* Run `/node/syncing` through API Middleware (#8944)

* add IsSyncing field to grpc response

* run /node/syncing through the middleware

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Return HTTP status codes other than 200 and 500 from node and debug endpoints (#8937)

* error codes for node endpoints

* error codes for debug endpoints

* better comment about headers

* gzl

* review comments

* comment on return value

* update fakeChecker used for fuzz tests

* fix failing tests

* Allow to pass URL params literally, without encoding to base64 (#8938)

* Allow to pass URL params literally, without encoding to base64

* fix compile error

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* unused import

* Return correct status codes from beacon endpoints (#8960)

* Various API Middleware fixes (#8963)

* Return correct status codes from `/states` endpoints

* better error messages in debug and node

* better error messages in state

* returning correct error codes from validator endpoints

* correct error codes for getting a block header

* gzl

* fix err variable name

* fix nil block comparison

* test fixes

* make status enum test better

* fix ineffectual assignment

* make PR unstuck

* return proper status codes

* return uppercase keys from /config/spec

* return lowercase validator status

* convert requested enum values to uppercase

* validator fixes

* Implement `/beacon/headers` endpoint (#8966)

* Refactor API Middleware into more manageable code  (#8984)

* move endpoint registration out of shared package

* divide main function into smaller components

* return early on error

* implement hooks

* implement custom handlers and add documentation

* fix test compile error

* restrict package visibility

* remove redundant error checking

* rename file

* API Middleware unit tests (#8998)

* move endpoint registration out of shared package

* divide main function into smaller components

* return early on error

* implement hooks

* implement custom handlers and add documentation

* fix test compile error

* restrict package visibility

* remove redundant error checking

* rename file

* api_middleware_processing

* endpoints

* gzl

* remove gazelle:ignore

* merge

* Implement SSZ version of `/blocks/{block_id}` (#8970)

* Implement SSZ version of `/blocks/{block_id}`

* add dependencies back

* fix indentation in deps.bzl

* parameterize ssz functions

* get block ssz

* update ethereumapis dependency

* gzl

* Do not reuse `Endpoint` structs between API calls (#9007)

* code refactor

* implement endpoint factory

* fix test

* fmt

* include pbs

* gaz

* test naming fixes

* remove unused code

* radek comments

* revert endpoint test

* bring back bytes test case

* move `signedBeaconBlock` to `migration` package

* change `fmt.Errorf` to `errors.Wrap`

* capitalize SSZ

* capitalize URL

* more review feedback

* rename `handleGetBlockSSZ` to `handleGetBeaconBlockSSZ`

* rename `IndexOutOfRangeError` to `ValidatorIndexOutOfRangeError`

* simplify parameter names

* test header

* more corrections

* properly allocate array capacity

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-06-15 10:28:49 -05:00
kasey
19ff25f171 fixes 9029 + a few related client-stats fixes/improvements (#9039)
- fix missing scrape logic for these metrics:
      - sync_eth1_connected
      - sync_eth1_fallback_configured
      - sync_eth1_fallback_connected
    - update readme to add more detailed source info
      and indicate unsupported metrics.
    - fix documentation bug in prometheus help text
      for powchain_sync_eth1_connected

Co-authored-by: kasey <kasey@users.noreply.github.com>
2021-06-14 23:59:15 +00:00
Raul Jordan
4551bb47fc Add Events Streaming Endpoint for Eth2 Standard API (#9001)
* begin event source support

* add in events server and proto definitions

* fmt

* add in stream support and proper building

* add in the protos

* tidy

* add structure and comments

* fmt

* revert proto changes

* only return events we explicitly request

* capture missing events needed for eth2 apis

* handle underflow

* no hash tree roots

* gazelle

* static

* race tests passing

* simplify

* handle all required events

* begin on stream tests

* correctness and abstracting method for forwarding to state feed

* genesis root

* include missing pieces

* logging

* rem and and handle return

* added mock

* add in mock

* add tests for block events

* added comprehensive tests

* radek comments

* add missing unit tests

* updatemockgen file

* viz

* done

* Update beacon-chain/rpc/eventsv1/events.go

* Update beacon-chain/rpc/eventsv1/events.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-14 16:58:08 +00:00
Radosław Kapka
77bf4bd9f0 Create ValidatorIndexOutOfRangeError (#9035) 2021-06-14 16:38:48 +00:00
terence tsao
dfd2cc751c Refactor helpers for Altair (#9026)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-11 19:29:50 +00:00
Nishant Das
fb08014146 Add Better Zero Key Checks (#9022)
* add better checks

* preston's review
2021-06-11 13:48:57 -05:00
Preston Van Loon
6981b16ecd RPC: Surface underlying error when failing to return a state. (#9024) 2021-06-11 13:50:02 +00:00
Preston Van Loon
71fd747dca Proto mirror script fix (#9020)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-11 03:05:50 +00:00
Nishant Das
7a6ce27497 Change To LRU from FIFO Cache (#8996)
* change to lru

* fix tests

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-10 21:53:34 +00:00
Raul Jordan
6eb006175a Capture ChainReorg, FinalizedCheckpoint, and NewHead Events in Beacon Node (#9011)
* capture missing events needed for eth2 apis

* handle underflow

* no hash tree roots

* gazelle

* static

* race tests passing

* simplify

* correctness and abstracting method for forwarding to state feed

* genesis root

* add in slotutil
2021-06-10 16:13:15 -05:00
Preston Van Loon
8f90e91e99 Update jwt-go library to fix CVE-2020-26160 (#9017)
* Replace github.com/dgrijalva/jwt-go with github.com/form3tech-oss/jwt-go, including security fix

* Fix to include go cast in deps.bzl and go mod

* Add gocast stub for go.mod / deps.bzl trick

* revert some changes

* Remove ignore tag

* gaz

* go mod tidy

* add build exclusion

* Go mod?

* go mod tidy
2021-06-10 13:59:34 -05:00
Alexander Mollohan
644d5bbb1a Graffiti hex support (#8894)
* more or less feature complete, needs unit tests

* Test support and concomitant refactors

* Resolving requested changes on PR

* minor fix before trying to figure out bazel

* bazel change

* minor changes requested in pr

* fixing build errors

* fixing build errors

* fixing PR feedback

* fixing PR feedback

* resolving nitpicks

* resolving nitpicks

* build failures REEEEE

* tests fail if its not this way, so this way it is

* getting the tests to pass

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-06-10 16:54:24 +00:00
Preston Van Loon
0d8dd8d280 Update mirror-ethereumapis.sh to include committer name / email (#9009)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-09 17:09:09 +00:00
Raul Jordan
bb6f00fd87 New Proto Messages Required for Event Streams (#9010)
* add event messages

* define all required event messages

* add new proto event messages

* formatting
2021-06-09 16:49:03 +00:00
Preston Van Loon
f2768c1414 Client-stats: Fail gracefully when metrics are missing (#8976)
* Client-stauts: Fail gracefully when metrics are missing

* gazelle
2021-06-07 20:37:14 +00:00
Raul Jordan
a81c4ed6a0 Protobuf Changes for Eth2 APIs EventSource Support (#8999)
* protobuf changes for event source support

* deps update

* tidy

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-07 19:29:36 +00:00
Preston Van Loon
26c4e7b527 Update prysm.sh to support Mac M1 chip (#9000)
Updated for Apple M1 users.

Co-authored-by: mrabino1 <mrabino1@gmail.com>
2021-06-07 18:51:05 +00:00
terence tsao
aa4c21d6df Run go imports (#8995) 2021-06-06 07:39:16 +02:00
Preston Van Loon
833ac0092c Fix repo name in mirror script (#8992) 2021-06-04 22:50:41 +00:00
Raul Jordan
2ee5bd12cf Improve Mirror Bash Script for EthereumAPIs (#8991)
* amend mirror script

* bash lint
2021-06-04 16:15:19 +00:00
Preston Van Loon
d4aa4bc258 Remove unused ClosestPowerOf2 method (#8990) 2021-06-04 15:03:18 +00:00
Radosław Kapka
1ea042dd6e Do not use proto casing in v1alpha1 API JSON fields (#8988) 2021-06-04 16:03:40 +08:00
Raul Jordan
dd10a3ab5d Fix EthereumAPIs Mirror Script (#8986) 2021-06-03 21:30:24 +00:00
Raul Jordan
8d428606a2 Mirror EthereumAPIs Bash Script (#8980)
* mirror script

* revert

* add in final script

* comments
2021-06-03 21:05:33 +00:00
terence tsao
8433a313f0 Enhance att sig set checks (#8978)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-03 15:30:46 +08:00
Preston Van Loon
693fdf87bb Add clientstats spec as a README.md (#8977)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-03 00:13:31 +00:00
Raul Jordan
5aac06f04e Move EthereumAPIs Into Prysm (#8968)
* begin move

* use same import path

* imports

* regen protos

* regen

* no rename

* generate ssz

* gaz

* fmt

* edit build file

* imports

* modify

* remove generated files

* remove protos

* edit imports in prysm

* beacon chain all builds

* edit script

* add generated pbs

* add replace rules

* license for ethereumapis protos

* change visibility

* fmt

* update build files to gaz ignore

* use proper form

* edit imports

* wrap block

* revert scripts

* revert go mod
2021-06-02 18:49:52 -05:00
Raul Jordan
1d2fc60ba7 Checkin Generated Protobufs and Proto Files for EthereumAPIs (#8973)
* generated

* add ignore

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-02 20:20:11 +00:00
kasey
2b6c7e7b37 use urfave defaulting method for correct help text (#8972)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-06-02 19:47:47 +00:00
Preston Van Loon
fc898d541f Update prysm scripts to support client-stats, deprecate slasher (#8971) 2021-06-02 14:06:59 -05:00
Nishant Das
91bd477f4f Add Metadata V2 Object and Interface (#8962)
* checkpoint changes

* gaz

* add error check

* gaz

* fix tests

* fix tests

* terence's review

* terence's review

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-02 13:44:34 +08:00
Preston Van Loon
ef20f2da7e Altair: new protos/state object (#8907)
* Altair: new protos/state

* Undo changes in proto/migration/migration.go

* Change proto/beacon/p2p/v1/generated.ssz.go to 0644

* Update proto/beacon/p2p/v1/types.proto

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* PR Feedback: rename SyncCommitteeSigningData to SyncAggregatorSelectionData

* 644

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-02 03:07:13 +00:00
Nishant Das
8a6e2a33a2 Fix Interop Mode (#8961) 2021-06-02 02:45:10 +00:00
aliask
eb1d122aec Feature/debian packages (#8872)
* Basic build targets for debian packages

* Add comments to config files

* Harden beacon-chain systemd service

* Add install scripts to set users and folders up

* Rename bazel target, fix service files

* Extra service hardening, cleanup install scripts

* Fix linting issues

* //shared:version_file run local only

* gazelle

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-06-01 15:44:24 +00:00
terence tsao
638e76cbd7 Sort imports (#8959) 2021-05-31 20:47:30 +02:00
Ivan Martinez
bb8625ede5 Remove stack trace from eth1 connection errors (#8957) 2021-05-28 21:25:54 +00:00
Ivan Martinez
40b4079924 Update deprecated function usages (#8953)
* Clear deprecated usages within Prysm

* Remove deprecatd usages where possible

* Update
2021-05-28 08:57:47 +02:00
Nishant Das
6bb3cc45ef Add Nested Nil Checks To Wrapper (#8951)
* add checks

* fix
2021-05-27 16:07:51 +00:00
Ivan Martinez
e97f6cfdbf ETH2 APIs: Handle invalid validator status as error (#8946)
* Handle invalid validator status as error

* Change text

* Fix review comments

* Remove nil status check

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-05-27 06:44:36 +00:00
terence tsao
d5f4385525 Copy to return interface (#8950) 2021-05-26 17:54:06 -07:00
kasey
8094c1c4c5 regenerate ssz types w/ variable-len bounds checks (#8936)
* regenerate ssz types w/ variable-len bounds checks

using the code in this PR:
https://github.com/ferranbt/fastssz/pull/45

* updating with improved bounds checking

* updating pinned fastssz dependency to newest

* removing redundant higher bounds checks

* regenerate again w/ new fastssz
2021-05-26 16:47:06 -05:00
terence tsao
276d03553c Block interface clean up (#8947) 2021-05-26 11:33:46 -07:00
Nishant Das
caf9bdbc6f Use Block Interface Across Prysm (#8918)
* commit initial work

* checkpoint current work

* gaz

* checkpoint

* req/resp changes

* initial-sync

* finally works

* fix error

* fix bugs

* fix issue

* fix issues

* fix refs

* tests

* more text fixes

* more text fixes

* more text fixes

* fix tests

* fix tests

* tests

* finally fix builds

* finally

* comments

* fix fuzz

* share common library

* fix

* fix

* add in more defensive nil checks

* add in more defensive nil checks

* imports

* Apply suggestions from code review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Apply suggestions from code review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Update shared/interfaces/block_interface.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Update shared/interfaces/block_wrapper.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Update shared/interfaces/block_interface.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* imports

* fix bad changes

* fix

* terence's review

* terence's review

* fmt

* Update beacon-chain/rpc/beacon/blocks.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* fix tests

* fix

* fix all tests

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-05-26 16:19:54 +00:00
Preston Van Loon
7b55213e6d Refactor attestationDelta to reuse the sqrt value of active current epoch (#8945)
* Refactor attestationDelta to reuse the sqrt value of active current epoch

* use baseRewardsPerEpoch
2021-05-26 09:37:35 -05:00
Ivan Martinez
d5ad15ed80 ETH2 APIs: Implement Validator Status support (#8940)
* Update protos for new changes

* Begin implementing validator status

* Add support for validator status to state/validators

* Fix nil pointer
2021-05-26 00:24:29 -04:00
terence tsao
2e322bdf6d Add missing finality spec tests (#8939)
* Remove unused functions

* Add finality spec tests

* Gazelle

* Add minimal tag to BUILD.bazel
2021-05-26 08:18:51 +08:00
terence tsao
5fe98ab260 Invert attest-timely flag (#8827) 2021-05-25 07:22:12 -07:00
Zahoor Mohamed
a958dd246b Improve performance during CheckAttesterDoubleVotes (#8927)
* de-normalize attestationDataRootsBucket to improve performace in CheckAttesterDoubleVotes

* parallize processing of every attestation

* fix double vote test case to take care of jumbled double votes due to parallel processing

* remove attestationRecordsBucket totally and take care of pruning

* missed out build file

* fix review comments

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-05-25 13:02:18 +05:30
Ivan Martinez
aa0fb058c6 ETH2 APIs: Update protos for new ethapis changes (#8920)
* Update protos for new changes

* Fix

* Fix deps

* Fix build

* Include newer changes to ethapis

* Fix

* Fix import

* Do not return unconnected peers
2021-05-24 20:02:01 -05:00
Preston Van Loon
0cbdce6bd6 Remove unused sszgen dependency (#8934)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-05-24 17:26:57 +00:00
Radosław Kapka
5ae6e83c7f Update deps.bzl after updating grpc-gateway dependency (#8933) 2021-05-24 16:18:26 +00:00
Radosław Kapka
180b8447f7 Update grpc-gateway dependency (#8929) 2021-05-24 09:27:50 -05:00
Nishant Das
503ec126e1 Fix Panic When Accessing State Frequently (#8928)
* fix issue

* dont allocate for existence checks

* radek's review
2021-05-24 06:52:33 +00:00
Nishant Das
76cdbaca22 Interface Analyzer (#8923) 2021-05-23 21:55:42 -07:00
terence tsao
09d7efe964 Remove unused functions (#8924) 2021-05-23 09:36:46 +02:00
Preston Van Loon
7dc3d7fc60 Write generated ssz files as non-executable 644 (#8909)
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-05-21 16:27:33 +00:00
Preston Van Loon
59897b78ff IntegerSquareRoot improvements (#8910)
* Add slower, but more precise sqrt method

* nofuzz
2021-05-21 11:02:16 -05:00
Ivan Martinez
f4d6e91e72 ETH2 APIs: Add spans and remove unused Server fields (#8916)
* Add spans and remove unused Server fields

* Remove more
2021-05-20 14:56:53 -05:00
Ivan Martinez
213369f5d0 Fix testing for attestation pools endpoint (#8914)
* Fix testing for attestation pools endpoint

* Remove
2021-05-20 18:59:25 +00:00
Ivan Martinez
8aa909de2d ETH2 API: Implement v1/state/validator endpoints (#8896)
* Implement validator endpoints

* Import

* Fix errors

* Add comments and fix naming :wq

* Fix functions to access state once

* Optimization for slots

* Test more

* Fix

* test listing committees for specific epoch

* Fix error retrurn

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: rkapka <rkapka@wp.pl>
2021-05-20 07:41:39 +00:00
Preston Van Loon
36e9edd654 Minor changes from hf1 branch (#8906)
* Copy new slot helpers from hf1 branch

* Make HandleEth1DataSlice public for altair
2021-05-19 16:11:48 +00:00
Ivan Martinez
9db4eba72b ETH2 API: Implement ListPoolAttestations endpoint (#8902)
* Implement ListPoolAttestations

* Spacing

* add lil doc

* Fix testing and rework logic
2021-05-19 09:55:24 -05:00
Radosław Kapka
fe440bc0eb Copy slice when exiting all keys with --exit-all (#8897)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-05-18 17:17:42 +00:00
terence tsao
f68e0846ba Ran go imports (#8898) 2021-05-18 16:11:02 +00:00
terence tsao
f2a2d5d43c Fix typo (#8899) 2021-05-18 08:33:52 -07:00
Preston Van Loon
f2fe1f7683 Use ExecuteStateTransition in onBlock (#8865)
* Refactor ExecuteStateTransition to use ExecuteStateTransitionNoVerifyAnySig and then verify the signatures. This reduces duplicated code paths.

* Use refactored method from PR #8864

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-05-17 19:52:15 +00:00
Nishant Das
35dc8c31fa Add in Context For Req/Resp (#8895)
* Add in context

* use better function signatures

* add in chainservice

* fix rpc requests

* add in test

* fix tests

* add in comments

* Update beacon-chain/sync/rpc_chunked_response.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Update beacon-chain/sync/rpc_send_request.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* rename files

* fmt

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-05-17 19:25:59 +00:00
Radosław Kapka
519e1e4f4a Decode peer ID properly when fetching peer (#8883)
# Conflicts:
#	beacon-chain/rpc/nodev1/node_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-05-17 18:58:13 +00:00
Ivan Martinez
2f10b1c7b1 Change gogoproto compiler to protoc-gen-go-cast (#8697)
* Remove gogoproto compiler

* Remove more gogoproto

* Improvements

* Fix gengo

* More scripts

* Gazelle, fix deps

* Fix version and errors

* Fix gocast for arrays

* Fix ethapis

* Fixes

* Fix compile errors

* fix go.mod

* //proto/... builds

* Update for protov2

* temp fix compilation to move on

* Change everything to emptypb.empty

* Add grpc to proto/slashings

* Fix almost all build failures

* Oher build problems

* FIX THIS FUCKING THING

* gaz literally every .bazel

* Final touches

* Final final touches

* Fix proto

* Begin moving proto.Marshal to native

* Fix site_data

* Fixes

* Fix duplicate gateway

* Fix gateway target

* Fix ethapis

* Fixes from review

* Update

* Fix

* Fix status test

* Fix fuzz

* Add isprotoslice to fun

* Change DeepEqual to DeepSSZEqual for proto arrays

* Fix build

* Fix gaz

* Update go

* Fixes

* Fixes

* Add case for nil validators after copy

* Fix cast

* Fix test

* Fix imports

* Go mod

* Only use extension where needed

* Fixes

* Split gateway from gengo

* gaz

* go mod

* Add back hydrated state

* fix hydrate

* Fix proto.clone

* Fies

* Revert "Split gateway from gengo"

This reverts commit 7298bb2054.

* Revert "gaz"

This reverts commit ca95256570.

* Merge all gateway into one target

* go mod

* Gaz

* Add generate v1_gateway files

* run pb again

* goimports

* gaz

* Fix comments

* Fix protos

* Fix PR

* Fix protos

* Update grpc-gateway and ethapis

* Update ethapis and gen-go-cast

* Go tidy

* Reorder

* Fix ethapis

* fix spec tests

* Fix script

* Remove unused import

* Fix fuzz

* Fix gomod

* Update version

* Error if the cloned result is nil

* Handle optional slots

* ADd more empty checks to clone

* Undo fuzz changes

* Fix build.bazel

* Gaz

* Redo fuzz changes

* Undo some eth1data changes

* Update go.mod

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Undo clone beacon state

* Remove gogo proto more and unused v1_gateway

* Add manual fix for nil vals

* Fix gaz

* tidy

* Tidy again

* Add detailed error

* Revert "Add detailed error"

This reverts commit 59bc053dcd.

* Undo varint changes

* Fix nil validators in deposit test

* Commit

* Undo

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-05-17 18:32:04 +00:00
Radosław Kapka
e254e8edcc Simplify handling hex state ID (#8890)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-05-17 09:56:00 +02:00
Nishant Das
e6a1d5b1b9 Refactor RPC Topic Construction (#8892)
* check in changes

* goimports

* export it

* add new rpc type
2021-05-15 08:58:56 +08:00
terence tsao
bc27a73600 Various code cleanups (#8889) 2021-05-13 08:29:14 +02:00
Nishant Das
4ef5c9ad15 add it in (#8885) 2021-05-12 15:59:26 +08:00
Radosław Kapka
ddab82e52a Handle undiscovered peers when listing peers (#8882)
* Handle undiscovered peers when listing peers

* off by 1 fix
2021-05-11 19:11:34 +02:00
Radosław Kapka
8fe26e7c84 Serialize nil ENR record (#8881) 2021-05-11 08:31:43 +00:00
Radosław Kapka
f97c7ee1da Initialize VoluntaryExitsPool and BlockReceiver in rpc server (#8876)
* Initialize `VoluntaryExitsPool` in rpc server

* initialize `BlockReceiver`
2021-05-11 07:16:13 +00:00
terence tsao
ad4fadb36a Minor clean up on mixing length root (#8875)
* Minor clean up on mixing length root

* Update root size to 32

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-05-10 22:39:53 +00:00
Nishant Das
255354f27e Fallback For Deposit Trie Creation (#8869)
* fallback

* make it an error

* comment

* Update beacon-chain/rpc/validator/proposer.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-05-10 07:34:10 +08:00
Preston Van Loon
050b244fa6 Fix rules_docker dependency. See bazelbuild/rules_docker#1814 (#8867)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-05-07 17:38:22 +00:00
Preston Van Loon
ab6d8486fb Update golang to 1.16.4 (#8868) 2021-05-06 21:55:26 +00:00
Preston Van Loon
b628b5f8ca Refactor ExecuteStateTransition to use ExecuteStateTransitionNoVerifyAnySig (#8864)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-05-06 09:15:37 +00:00
ahadda5
ac4fe2db5a Czech language support from go-bip39 incorporated (#8861)
* added Czech from go-bip39

* go mod tidy

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-05-05 20:51:05 +00:00
Preston Van Loon
2f66d1a46c Use mock web3/powchain for fuzz test (#8866)
* Use mock web3/powchain for fuzz test

* gofmt

* build viz
2021-05-05 20:18:55 +00:00
Nishant Das
60c6cf7438 reduce map copies (#8860) 2021-05-05 16:28:35 +08:00
ahadda5
8dd2915d3c Unify gRPC Gateways into Single, Shared lib (#8799)
* moved validator/rpc/gateway under shared,  validator/rpc/web is not visiable to the new shared gateway

* decided to have one lib with two methods if needed, status and stop are the same, however New and Start are not

* moved beacon's gateway to shared and moved the main func to its own /server folder

* goftm

* gofmt

* removed the extra loggin

* reduce visibility

* tighter visibilty of the shared lib for beacon and validator only for now

* fix patterns , ctx needs to be first param

* fix comments

* gofmt

* added enum for the caller id

* added unit test for gateway

* deprecated .WithDialer to .WithContextDialer

* changed the string callerId to uint8, and rearranged the Gateway struct based on the compiler

* fix 1 based on comment

* iota type

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-05-04 17:20:58 +02:00
terence tsao
3c9b9e7bf5 Add interop write block and state to disk (#8840)
* Add interop write block and state to disk

* Comment why false

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-05-04 03:42:06 +00:00
kasey
a090650446 fixes #8848 -- scrape validator_(total|active) (#8858)
fixes #8848 -- scrape validator_total, validator_active
2021-05-03 17:24:27 -05:00
kasey
dace0f9b10 cli to push metrics to aggregator service (#8835)
* new prometheus metrics for client-stats metrics

* adds client-stats types beacause they are
  used by some of the prometheus collection code.
* new prometheus collector for db disk size
* new prometheus collector for web3 client
  connection status

* adds client-stats api push cli in cmd/client-stats

* adds api metadata to client-stats collector posts

* appease deepsource

* mop up copypasta

* use prysm assert package for testing
2021-05-03 09:57:26 -05:00
terence tsao
a6e1ff0e63 Run go imports (#8851)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-05-03 13:44:58 +00:00
terence tsao
1dde588932 Revert "Invert next-slot-state-cache flag (#8830)" (#8855)
This reverts commit 64bd9dba9b.
2021-05-03 08:04:45 +08:00
Nishant Das
54d4d39ea1 Revert "Simplify Effective Balance Calculation (#8743)" (#8844)
* Revert "Simplify Effective Balance Calculation (#8743)"

This reverts commit 45d2df1af7.

* add flag
2021-05-01 13:07:38 -05:00
kasey
7be6fc1780 new prometheus metrics for client-stats metrics (#8834) 2021-04-30 13:37:38 -07:00
Radosław Kapka
2fdfda2804 Fix remote keymanager's dynamic key reload (#8817)
* Fix remote keymanager's dynamic key reload

* wait for activation test

* runner test

* rename mock creation func

* fix compile error

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-04-30 15:15:22 +00:00
Nishant Das
a063952abe Fix Script (#8842)
* enable script

* add permission in

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-04-30 14:58:29 +00:00
Radosław Kapka
1a00121e60 Correct documentation for DepositsNumberAndRootAtHeight (#8843)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-04-30 14:40:42 +00:00
terence tsao
8fffa6c92a Remove junk files (#8846) 2021-04-30 16:20:14 +02:00
terence tsao
64bd9dba9b Invert next-slot-state-cache flag (#8830)
* Invert `next-slot-state-cache`

* Go fmt

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-04-30 12:47:44 +00:00
Preston Van Loon
f2767f4f8d Update github.com/grailbio/bazel-toolchain. Fixes #8072 (#8839) 2021-04-29 14:57:16 +00:00
Nishant Das
71f53d908f Add Stricter Checks To The Deposit Cache (#8838)
* add stricter checks

* fix tests
2021-04-29 14:24:13 +02:00
terence tsao
7b8e48d80e Invert proposer-atts-selection-using-max-cover flag (#8828)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-04-28 13:44:13 +00:00
Nishant Das
b6c50df435 Add New Method to Fuzz State (#8823) 2021-04-28 10:17:36 +00:00
Preston Van Loon
9b697d2c12 Partially revert PR #8809 (#8829) 2021-04-28 17:02:32 +08:00
Victor Farazdagi
d1a952a6fc Spec tests: improve readability of rewards tests output (#8826)
* Spec tests: improve readability of rewards tests output

* fix minimal tests

* remove redundant var

* simplify
2021-04-27 16:14:11 +00:00
Nishant Das
58a81c704b Remove Missing Logs Method (#8825)
* remove missing logs method

* gaz

* remove unused var
2021-04-27 19:12:47 +08:00
Nishant Das
5c9f361903 validate big num better (#8822) 2021-04-27 14:46:47 +08:00
Nishant Das
ef9f6c5b4d Initialize Data Correctly For Powchain Service (#8812)
* initialize index correctly

* comments

* review comments

* Update beacon-chain/powchain/service.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-04-25 00:11:15 -05:00
Preston Van Loon
424e115331 Independent eth1 voting (#8811) 2021-04-24 18:32:45 -07:00
Radosław Kapka
f1d7c0f2c9 Create a new wallet on accounts import only when necessary (#8801)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-04-24 08:26:41 +00:00
Raul Jordan
3d96e3c142 Amend DepositNumberAndRootAtHeight Function to Fix Off-By-One (#8809)
* amend deposit cache and add comprehensive unit tests to fix off-by-one error

* fix tests

* still need to +1 deposit index
2021-04-24 08:02:34 +00:00
Raul Jordan
8cda50f179 Optimized Slasher Database Methods (#8805)
* slasher kv additions

* gaz

* pass tests
2021-04-23 12:06:13 -05:00
Nishant Das
3574ee177d vendor codecov script (#8803) 2021-04-23 09:50:03 -05:00
Victor Farazdagi
386b69f473 Fix comments (#8802)
* fix incorrect exported name in comments

* add comments to exported methods
2021-04-23 12:06:05 +00:00
Victor Farazdagi
648e360842 Refactor shared/testutil/spectest.go (#8800)
* Refactor shared/testutil/spectest

* update operations

* update epoch_processing

* rename spectest.go
2021-04-22 16:34:00 +00:00
Victor Farazdagi
23ff14c34b Move spectests into a separate package (#8798)
* move general/phase0/bls tests

* phase0/operations tests

* phase0/sanity/blocks

* fix import

* phase0/sanity/slots

* phase0/sanity/slots minimal

* phase0/epoch_processing tests

* phase0/rewards tests

* simplify

* more simplification

* phase0/shuffling tests

* phase0/ssz_static tests

* spectest/utils

* fix build

* readme

* update test sizes

* update readme
2021-04-22 14:53:30 +00:00
Kevin Lu
e52df323f7 Added public/private flags for IP address white/blacklisting (#8648)
* Added public/private flags for IP address white/blacklisting

* Provided reference docs for private IP address ranges

* Formatting changes and inline comments for private ip address specification

* @nisdas feedback, commentary fixes

* Added in inverse case where we allow for private address

* changed helper function to take in multiaddr Action as argument, syntax changes for control logic

* minor refactoring to control flow of configFilter and private addr filter helper function

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-04-21 06:25:30 +00:00
Victor Farazdagi
823f7c99d2 Update spec tests (#8792)
* update path to minimal tests

* update specs testutils

* fix attestation_test

* attester_slashing_test

* fix block_header_test

* block_processing_minimal

* core/blocks minimal spectests

* epoch_processing minimal spectests

* slot_processing minimal spectests

* bls spectests

* update mainnet tests

* fix remaining tests

* gazelle
2021-04-20 23:54:06 +00:00
Raul Jordan
711f527efb Amend E2E Test Path (#8793) 2021-04-20 16:54:38 +00:00
Raul Jordan
af8f444166 Upgrade Bazel Go-Ethereum (#8791)
* upgrade bazel go ethereum

* tidy
2021-04-20 15:28:07 +00:00
Raul Jordan
2634430f67 Update Go-Ethereum Dependency (#8788) 2021-04-20 06:57:26 -07:00
terence tsao
70dc22aa9a Fix redundant err check (#8790) 2021-04-20 02:25:16 +00:00
Everest Stefan Munro-Zeisberger
ad269ee147 Set up Prysm for fuzz testing on Fuzzbuzz (#8789)
* Wrap beacon fuzz tests in go-fuzz compatible test functions

* Add fuzzbuzz.yaml
2021-04-19 19:00:46 +00:00
Nishant Das
cff7dbd015 Add Eth1 Header Timestamp Check (#8771)
* add time check

* unit test

* fix

* Update beacon-chain/powchain/service.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-04-17 13:55:41 +00:00
Preston Van Loon
afc0ace624 POW: remove context from log message (#8782) 2021-04-16 19:16:36 +00:00
terence tsao
400e42cc2d Reuse ProcessBlockForStateRoot (#8781) 2021-04-16 10:18:59 -07:00
Victor Farazdagi
59ee339497 Move TestMain into dedicated file (#8779)
* Move TestMain into dedicated file

* update db_test

* formatting
2021-04-16 16:25:28 +00:00
Radosław Kapka
c53d0e1c5f Codecov: ignore testing directories (#8777)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-04-16 08:44:36 -05:00
Raul Jordan
02a8ae7f36 Validator RPC Accounts Voluntary Exit (#8776)
* begin voluntary exits RPC

* added in exit logic

* complete exit func

* revert

* capitalize

* revert

* import

* rpc voluntary exit test

* fmt
2021-04-16 09:46:38 +02:00
terence tsao
a4ff97d24b Refactor signature function's argument (#8773) 2021-04-15 11:35:53 -07:00
Victor Farazdagi
3d3b9d1217 Spec checker tool (#8722)
* Add specdocs static code analyzer

* docs pulling script

* update content pulling script

* add test

* better parsing of incoming docs

* update test

* implements analyzer

* separate tool

* remove analyzer code

* cleanup

* deep source fixes

* untrack raw specs files

* add back phase0 defs

* update spec texts

* re-arrange code

* updated spec list

* cleanup

* more comments and readme

* add merkle proofs specs

* add extra.md

* mark wrong length issue

* update readme

* update readme

* remove non-def snippets

* update comment

* check numrows

* ignore last empty line

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-04-15 14:54:07 +00:00
terence tsao
169cd78bbd Refactor process operation function arguments (#8767)
* Pass slash validator func as argument

* Refactor ProcessBlockHeaderNoVerify

* Refactor Eth1Data and Randao

* Refactor ProposerSlashing

* Refactor AttesterSlashing

* Refactor VoluntaryExit

* Add VerifyNilBeaconBlock to ProcessBlock

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2021-04-15 13:58:54 +00:00
Nishant Das
405e2a1a03 Thread Context for HTR Methods (#8770)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-04-15 12:17:55 +00:00
Shay Zluf
d77c298ec6 Support authorised access to web 3 providers (#8075)
* jwt access token impl

* use secret or jwt

* rename

* separate method for splitting auth

* usage update

* Update beacon-chain/flags/base.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/node/node.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* make things work

* removed unused code

* better, more flexible authorization

* move types and function to proper packages

* fix checking if endpoint is not set

* fix existing tests

* rename Endpoint field to Url

* Tests for HttpEndpoint

* better bearer auth

* tests for endpoint utils

* fix endpoint registration

* fix test build

* move endpoint parsing to powchain

* Revert "fix existing tests"

This reverts commit ceab192e6a.

* fix field name in tests

* gzl

* add httputils dependency

* remove httputils dependency

* fix compilation issue in blockchain service test

* correct endpoint fallback function and tests

* gzl

* remove pointer from currHttpEndpoint

* allow whitespace in auth string

* endpoint equality

* correct one auth data Equals test case

* remove pointer receiver

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-04-15 11:02:02 +00:00
Radosław Kapka
2c6549b431 Tests for beacon-chain/node/config.go (#8769)
* Tests for `beacon-chain/node/config.go`

* gzl

* change tested value of NetworkID
2021-04-15 10:01:50 +00:00
Radosław Kapka
aba2ec9aac Codecov: ignore all mock files (#8768) 2021-04-15 15:55:12 +08:00
Radosław Kapka
a8716d2949 Implement ListForkChoiceHeads in the debug API (#8675)
* initial implementation

* use ChainHeads to get heads

* API unit tests

* remove unnecessary identifier

* fix formatting

* gzl

* Update chainheads to fork choice scope

* use HeadFetcher

* fix test

* gzl

* remove junk

* remove ChainHeads from forkchoice

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2021-04-14 17:01:24 +00:00
Siddhant N Trivedi
04bc8a85ca Add generated file to DeepSource exclude_patterns (#8765) 2021-04-14 14:06:25 +00:00
Radosław Kapka
a9a0ecd76d Create node's Powchain registration file (#8754)
* extract powchain and remove unused function

* improve "no web3 provider" error log

* add second error log

* remove redundant code from registerPOWChainService

* remove deposit contract log from registerPOWChainService

* gzl

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-04-14 12:54:50 +02:00
Radosław Kapka
902e3f4f53 Fix overriding network config in tests (#8756)
* Fix overriding network config in tests

* copy config in OverrideBeaconNetworkConfig

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-04-14 07:39:20 +00:00
terence tsao
228033f7a3 Pass SlashValidator func as argument to ProcessProposer[Attester]Slashings (#8763)
* Pass slash validator func as argument

* Gazelle
2021-04-13 20:32:49 -05:00
Victor Farazdagi
53ffc67850 Update specs comments (#8759)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-04-13 22:26:39 +00:00
terence tsao
59eb2e60d0 Combine function parameters (#8758)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-04-13 19:56:28 +00:00
Nishant Das
0ea11ac212 Rebuild Trie After Reaching Limit (#8740)
* fix up indices

* add comment

* fix up again

* add test

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-04-13 19:12:56 +00:00
terence tsao
3e686a71e3 Move finality delay calculation (#8757)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-04-13 18:28:46 +00:00
Preston Van Loon
efbefaeb7c CI: Use experimental_remote_asset_api (#8761) 2021-04-13 17:18:47 +00:00
Nishant Das
dd7481e99c Disable Persisting Network Keys (#8735)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-04-13 16:25:35 +00:00
Preston Van Loon
fda003288d CI: minimal remote downloads, strict action environment (#8760) 2021-04-13 15:45:43 +00:00
Nishant Das
131a14ee2f Minor State Management Improvements (#8742)
* add improvement

* change to bool

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-04-13 13:23:06 +00:00
Nishant Das
45d2df1af7 Simplify Effective Balance Calculation (#8743)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-04-13 12:44:33 +00:00
Victor Farazdagi
5f3299e598 Update processJustificationAndFinalization(), per current specs (#8741) 2021-04-13 11:51:04 +00:00
Radosław Kapka
5217081567 Create node's P2P registration file (#8745)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-04-13 07:41:06 +00:00
Radosław Kapka
389bad7d24 Extract node configuration to separate file (#8744)
* Extract node configuration to separate file

* rename WarnIfNotSupported

* rename configuration to config

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-04-13 07:04:34 +00:00
Victor Farazdagi
97b4b86ddf Update NextPowerOf2() and PrevPowerOf2() funcs (#8751)
* Update NextPowerOf2() and PrevPowerOf2() funcs

* gofmt

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-04-13 06:09:56 +00:00
Victor Farazdagi
cc9bec7a19 Update specs comments (#8750)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-04-13 05:35:22 +00:00
terence tsao
8b494fb1bc ProcessRewardsAndPenaltiesPrecompute taking in rewards funcs (#8737)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-04-13 04:55:45 +00:00
Nishant Das
6aa1297829 Enable Gossip Scoring (#8730)
* enable gossip scoring

* fix some tests

* fix  up

* handle too small numbers

* add caching to validator count

* fix up

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-04-13 02:20:13 +00:00
Preston Van Loon
a91f2688f1 Set test timeout on //beacon-chain/rpc/validator:go_default_test. Sometimes exceeds 800s otherwise (#8752) 2021-04-13 00:24:04 +00:00
terence tsao
1c1bf37b33 Refactor EligibleForRewards into its own helper (#8739)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-04-12 20:26:08 +00:00
Preston Van Loon
76e36b95f7 CI: Add remote build caching support (#8748) 2021-04-12 18:46:11 +00:00
terence tsao
74cf2320ad Altair: Split no verify functions in own file (#8747)
* Split transition no verify functions in its own file

* Remove redeclared functions

* Go fmt

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-04-12 16:42:08 +00:00
terence tsao
2a1c880673 Altair: Split state getters and setters into its own file (#8746)
* Categorize getters and setters into its own file

* Go fmt
2021-04-12 14:23:55 +00:00
Nishant Das
c65c6ebc38 Invert BLST Go Tag (#8725)
* invert go tag

* fix rc

* try again

* Revert "try again"

This reverts commit 70ff7658b7.

* fix fuzz

* Update shared/bls/blst/BUILD.bazel

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2021-04-11 11:11:15 +00:00
terence tsao
19a9b4b064 Refactor finality helpers to helpers pkg (#8734)
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2021-04-10 16:12:52 +00:00
Victor Farazdagi
1af88b2c35 Remove redundant MerkleRoot function (#8733)
* Remove redundant MerkleRoot function

* update merle_tree() specs comment
2021-04-10 03:13:07 +00:00
ahadda5
e3149c4f0b Slashing Protection RPC Endpoints for Web Backend (#8723)
* added the removed RPC delete account to accounts.go  and the rpc in proto

* reverted the 3 unit tests namely the failed derived delete, no pub keys provided and the successful imported account with provided public keys; also  brought back the createImportedWalletWithAccounts back in wallet_test.go

* strong password defined elsewhere- removed

* Update validator/rpc/accounts_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* added PublicKeys Nil test case

* changed ss to s. ss was a bad name inthe first place

* goimports -w root

* fixed the goimports before running the proto scripts, also changed the deleterequest variable to PublicKeysToDelete

* removed unneeded comment

* added test case for derived, changed delete account to be for both imported and derived

* gofmt

* unrelated files

* unrelated files

* unrelatedfiles restored

* revert unrelated files

* changed the last ss

* adding slashign endpoints

* adding the rpc export and import funcs, still need more testing and add unit tests

* added import slashing unit test

* clean up

* remove less

* Update proto/validator/accounts/v2/web_api.proto

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update proto/validator/accounts/v2/web_api.proto

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update proto/validator/accounts/v2/web_api.proto

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update proto/validator/accounts/v2/web_api.proto

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* camelCase Proto

* update slashing_protection_json

* update proto

* register in validator/rpc/gateway

* removed the server db creation, the validator cannot begin with a null db

* round trip test

* gofmt

* Update validator/rpc/slashing.go

* Update validator/rpc/slashing.go

* Update validator/rpc/slashing.go

* Update validator/rpc/slashing.go

* Update validator/rpc/slashing.go

* Update validator/rpc/slashing.go

* Update validator/rpc/slashing.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-04-09 19:13:59 +00:00
Victor Farazdagi
0808c02c65 Update spec comments (#8727) 2021-04-08 19:33:54 +00:00
Victor Farazdagi
76390c94af Refactor ProcessFinalUpdates(), per updated specs (#8724) 2021-04-08 14:20:52 +00:00
Nishant Das
f0e2f561d5 Make BLST The Permanent Default (#8710)
* remove herumi

* gaz

* deprecate flag

* remove source builds of herumi

* remove

* Revert "remove source builds of herumi"

This reverts commit ac7dd133ed.

* disable blst

* remove herumi hard requirement from fuzz

* restrict viz, ensure all deps removed from fuzz

* remove source builds

* add back opts

* add back herumi initialization

* Revert "add back opts"

This reverts commit ad9b409b8a.

* Revert "remove source builds"

This reverts commit b78ee30dba.

* Revert "restrict viz, ensure all deps removed from fuzz"

This reverts commit 65d951da93.

* Revert "remove herumi hard requirement from fuzz"

This reverts commit ad92191d81.

* redundant

* add lock for rand generation

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-04-07 10:18:19 -05:00
Victor Farazdagi
74d0134ee3 Weak subjectivity minor refactoring (#8715)
* Add IsWithinWeakSubjectivityPeriod helper method

* Add LatestWeakSubjectivityEpoch method

* ParseWeakSubjectivityInputString helper

* switch to Checkpoint

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-04-06 14:32:49 +00:00
Victor Farazdagi
91df0112c7 Add LatestWeakSubjectivityEpoch() method (#8707)
* Add IsWithinWeakSubjectivityPeriod helper method

* Add LatestWeakSubjectivityEpoch method

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-04-06 09:08:44 +03:00
terence tsao
7998348bcb Fix deep source complains (#8714) 2021-04-05 23:04:26 +00:00
ahadda5
99bd988375 Web Backend Recover Wallet (#8679)
* recover wallet rpc support - first attempt

* removed redundant check

* separate createwallet into imported and derived. Recover is derived

* added unit test for recover. Recover is the createWallet for derived

* so proto does CamelCase!

* fixed issues related to unit testing

* expose ValidateMnemonic from accounts to be used by the rpc module

* added Mnemonic25Support and test to ensure it is not et is not empty

* added skipMnemonic25thword support and unit test

* Update proto/validator/accounts/v2/web_api.proto

Proper naming convention

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* ran goimports,changed variable to SkipMnemonic_25ThWord

* Update validator/rpc/wallet.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Update validator/rpc/wallet.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Update validator/rpc/wallet.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Update validator/rpc/wallet.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Update validator/rpc/wallet.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Update validator/rpc/wallet.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Update validator/rpc/wallet.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* fixed variable and text msgs naming convention as per the review

* added unit test for strong password on recover

* Update proto/validator/accounts/v2/web_api.proto

Co-authored-by: Nishant Das <nish1993@hotmail.com>

* Update validator/rpc/wallet.go

Co-authored-by: Nishant Das <nish1993@hotmail.com>

* goimports -w on root proto

* added comments after exposing ValidateMnemonic to by used by rpc.RecoverWallet

* language should be case insensitive

* need to goimports before update protobuf scripts

* Update validator/rpc/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/rpc/wallet.go

comments need to be consistant

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/rpc/wallet_test.go

consistent comments

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* fix comments

* remove the skipMnemonic from rpc, just check if passphrase is empyt

* defer call to set the writePassword flag on web boarding

* gofmt

* fixed the password write test after recovering a wallet. Needed to have two defers. One to set to true then one back to false

* restore powchain.pb.go and finalized_block_root_container.pb.go

* revert beacon messages.pb.go

* revert unrelated files

* revert unrelated files

* revert unrelated files

* unlreated files

* restored the imports

* Update validator/rpc/wallet_test.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Nishant Das <nish1993@hotmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-04-05 20:42:03 +00:00
Raul Jordan
0a2f3e4d48 Resolve Misc DeepSource Lint Issues (#8711) 2021-04-05 19:37:25 +00:00
Victor Farazdagi
dfe5372db5 Add IsWithinWeakSubjectivityPeriod helper method (#8706)
* Add IsWithinWeakSubjectivityPeriod helper method

* Nishant's suggestion on cache reset

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-04-05 18:26:31 +00:00
terence tsao
f2f509be0e Block deposit: refactor batch verify and validation activation (#8698)
* Block deposit: refactor batch verify and validation activation

* Return beacon state

* Return nil

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-04-05 16:49:17 +00:00
terence tsao
01eb77c834 ProcessPreGenesisDeposits: Remove context.TODO() usage (#8705)
* ProcessPreGenesisDeposits: Remove context.TODO() usage

* apply #8704 change

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-04-05 15:07:56 +00:00
Nishant Das
3cd95b4a6c copy better (#8708) 2021-04-05 08:48:19 -05:00
Preston Van Loon
80ab9201b3 Update blst (#8703)
* Update blst to v0.3.4

* go mod tidy

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-04-04 19:59:32 +08:00
terence tsao
1c6c058bba Genesis beacon state: refactor update eth1 data (#8704) 2021-04-03 23:19:48 +02:00
terence tsao
528cd89616 Move chainheads to blockchain pkg (#8700)
* Move chainheads to block chain's head info

* Fix mock
2021-04-03 11:25:25 +00:00
Preston Van Loon
c179cfb93e BLS: Deprecate usage of AggregateVerify (#8699)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-04-02 15:53:08 +00:00
terence tsao
f67228bacb Change ProcessDeposits argument to use deposits (#8696) 2021-04-02 14:48:41 +00:00
Nishant Das
af3d3e8cd3 Update Gossip Parameters (#8683)
* add in more accurate aggregate parameters

* add more param changes

* more cleanup

* fix order of operations

* comments

* remove redundant declaration

* clean up better

* fix up

* victor's review

* disable mesh scoring

* disable mesh scoring

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2021-04-02 20:21:19 +08:00
ahadda5
878439065c Web Backend Revert delete rpc (#8681)
* added the removed RPC delete account to accounts.go  and the rpc in proto

* reverted the 3 unit tests namely the failed derived delete, no pub keys provided and the successful imported account with provided public keys; also  brought back the createImportedWalletWithAccounts back in wallet_test.go

* strong password defined elsewhere- removed

* Update validator/rpc/accounts_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* added PublicKeys Nil test case

* changed ss to s. ss was a bad name inthe first place

* goimports -w root

* fixed the goimports before running the proto scripts, also changed the deleterequest variable to PublicKeysToDelete

* removed unneeded comment

* added test case for derived, changed delete account to be for both imported and derived

* gofmt

* unrelated files

* unrelated files

* unrelatedfiles restored

* revert unrelated files

* changed the last ss

* restore imports

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-04-02 11:33:42 +02:00
Victor Farazdagi
acf17f9b82 Weak subjectivity period calculating helper (#8695)
* move helpers to their own file

* implement ComputeWeakSubjectivityCheckptEpoch

* remove helper func

* fix unit tests

* rename
2021-04-01 22:50:04 +00:00
Victor Farazdagi
9afa304817 Skip e2e test for previous implementation of slasher (#8693) 2021-03-31 17:22:23 +00:00
Radosław Kapka
847640333a Wrap long lines in validator package (#8691) 2021-03-31 14:32:35 +00:00
Preston Van Loon
28e4a3b7e8 Add a Tool to Split a Series of Mnemonic Keys into Distinct Wallets (#8651)
* Add a tool to split a series of mnemonic keys into distinct wallets

* split func

* keysplit tool

* gaz/viz

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-03-30 20:38:40 +00:00
Raul Jordan
d7103fdef3 HTTP Request Sink Tool (#8689)
* request sink test

* append if not exist
2021-03-30 16:42:50 +00:00
Radosław Kapka
f822f0436e Break long lines in beacon chain package (#8686)
* Break long lines in beacon chain package

* change log formatting

* Revert "change log formatting"

This reverts commit b610fd67ed.

* Revert "Break long lines in beacon chain package"

This reverts commit 53215fdcde.

* wrap lines over 160

* revert go.mod and go.sum
2021-03-30 11:24:46 +00:00
Potuz
0b06c48ed0 Fix typo in comment for LoadGenesis (#8685) 2021-03-30 00:45:14 +00:00
Radosław Kapka
190d862552 Implement GetBeaconState in the debug API (#8631)
* Return status.Errorf instead of plain errors from gRPC functions

* return plain errors from helper functions

* change errors to lowercase in node

* correct test expectations

* extracted StateFetcher

* StateFetcher tests

* extract beacon state creation option and fix state tests

* add comment to StateFetcher

* register the server

* implement grpc function

* test ToProto

* gRPC function test with mock state fetcher

* reduce visibility of packages

* add missing error assertion

* removed unused code

* overwrite config name

* gzl

* Fix service fields

* rename StateFetcher to Provider

* Update beacon-chain/state/stateV0/state_trie.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* adjust code to new v0 interfaces

* interface/struct naming changes

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-03-29 21:04:35 +00:00
terence tsao
a8ab279cb8 Protoarray: add ChainHeads (#8684)
* Add ChainHeads

* Add only has bestChild case

* Update beacon-chain/forkchoice/protoarray/store.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-03-29 17:40:38 +00:00
Victor Farazdagi
feeb59de23 Add a separate unit test for ComputeDomainAndSign (#8600)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-03-29 15:26:50 +00:00
terence tsao
2ec3f79146 Remove unused interface (#8682) 2021-03-28 22:53:34 +02:00
Victor Farazdagi
e26dab84f6 E2E remove redundant test case (#8680) 2021-03-28 15:19:51 +00:00
Nishant Das
4886a4e749 Add Remaining Gossip Scoring Topics (#8678)
* add for topics

* Apply suggestions from code review

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2021-03-28 09:37:25 +08:00
Victor Farazdagi
54cf5f3c7a Add connectivity E2E test (#8607)
* Add connectivity e2e test

* randomized backoff period on discovery

* fix signature

* gazelle

* deterministic randg

* define ComponentRunner

* update ETH1 node

* extrace BootNode

* add logger

* remove refs to t

* gazelle

* update BeaconNode

* parametrize tests

* update log formatting

* update ValidatorNode

* remove redundant test id from params

* decrease delta

* move BootNode

* update BootNode

* revert params

* update SlasherNode

* mask unused param

* update helpers

* update test runner

* go mod tidy

* remove unused fields

* re-arrange tests

* extract sync into its own method

* gazelle

* BeaconNode

* types/types.go

* BootNode

* gofmt

* remove unused argument

* remove redundant comment

* add deprecation comment

* types comment

* remove deprecated method

* BeaconNodes -> BeaconNodeSet

* make sure that slasher is required component

* update StartBeaconNodeSet() and StartValidatorNodeSet()

* update SlasherNodeSet

* gazelle

* update connectivity tests

* Nishant's suggestion

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-03-27 00:33:50 +00:00
terence tsao
5f2f53a0a6 Harden beacon state interface for Altair (#8673)
* State: clean up

* Add error to return signature

* Remove SetCurrentEpochAttestations and SetPreviousEpochAttestations

* Fix tests

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-03-26 18:15:03 +00:00
Nishant Das
7f0c92504f Add DB To P2P Service (#8676)
* add files

* goimports
2021-03-26 09:51:58 -05:00
Victor Farazdagi
961a012502 E2E refactoring (#8643)
* define ComponentRunner

* update ETH1 node

* extrace BootNode

* add logger

* remove refs to t

* gazelle

* update BeaconNode

* parametrize tests

* update log formatting

* update ValidatorNode

* remove redundant test id from params

* decrease delta

* move BootNode

* update BootNode

* revert params

* update SlasherNode

* mask unused param

* update helpers

* update test runner

* go mod tidy

* remove unused fields

* re-arrange tests

* extract sync into its own method

* gazelle

* BeaconNode

* types/types.go

* BootNode

* gofmt

* remove unused argument

* remove redundant comment

* add deprecation comment

* types comment

* remove deprecated method

* BeaconNodes -> BeaconNodeSet

* make sure that slasher is required component

* update StartBeaconNodeSet() and StartValidatorNodeSet()

* update SlasherNodeSet

* gazelle
2021-03-25 22:15:58 -05:00
Victor Farazdagi
139017546d E2E refactoring: SlasherNodeSet and SlasherNode (#8672)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-03-26 01:22:32 +00:00
Preston Van Loon
5d12cc1ded Add beaconState RotateAttestations API method (#8669)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-03-25 22:05:15 +00:00
Preston Van Loon
e32c88e14c Update github.com/dgraph-io/ristretto (#8671) 2021-03-25 19:52:16 +00:00
Victor Farazdagi
6a4d4d7028 E2E refactoring: BeaconNode and BeaconNodeSet (#8667)
* E2E refactoring: BeaconNode and BeaconNodeSet

* unstuck!

* hope the magic has worked!
2021-03-25 17:48:09 +03:00
Victor Farazdagi
e56ab297b1 E2E refactoring: ValidatorNode and ValidatorNodes (#8665)
* E2E refactoring: ValidatorNode and ValidatorNodes

* add deprecation warning

* Radek's suggestion on naming

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-03-25 10:35:06 +00:00
Victor Farazdagi
ad303fb943 E2E refactoring: ETH1 Node (#8663)
* Eth1Node added

* gofmt

* update types

* update types

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-03-25 00:16:11 +00:00
Victor Farazdagi
89da5d1ef5 E2E refactoring: bootnode (#8659)
* BeaconNode

* types/types.go

* BootNode

* gofmt

* remove unused argument

* remove redundant comment

* add deprecation comment

* types comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-03-24 20:55:12 +00:00
Raul Jordan
82f25bacf2 Fix All Deep Source Shell Issues and Go Issues (#8661)
* most issues resolved

* fuzz wrapper fixes

* more deepsource

* more shell

* export

* shell

* combine func params, fix go deep source issues

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-03-24 19:57:27 +00:00
Raul Jordan
b2d9f9a2d8 Fix Secrets Failing Build (#8660)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-03-24 18:01:58 +00:00
Potuz
9cb4eafad4 Change early exit error message (#8657)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-03-24 11:17:52 -05:00
Potuz
ff40a68215 Add prater exit info (#8656)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-03-24 13:50:06 +00:00
Radosław Kapka
2fe50c5edd Rename state_fetcher.go (#8646)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-03-24 09:08:39 +00:00
Radosław Kapka
3e92ae0f48 Use context timeout during gateway service shutdown (#8644)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-03-24 05:00:47 +00:00
Radosław Kapka
c112d27ab5 Extract filling testutil's beacon chain root (#8645)
* Extract filling testutil's beacon chain root

* gzl

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-03-24 03:47:36 +00:00
terence tsao
a539e3d66e Move handleValidatorSlice and handleByteArrays to stateutil pkg (#8653)
* State: clean up

* Share handleArray and handleValdiator root functions
2021-03-24 01:59:23 +00:00
terence tsao
446029c1ba State: rename a few things (#8650)
* State: clean up

* Rename to Uint64ListRootWithRegistryLimit

* Gazelle
2021-03-23 21:17:09 +00:00
Nishant Das
fba56df765 Fix Powchain Genesis (#8647)
* fix powchain genesis

* Setup test with a deposit cache

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-03-23 16:44:57 +00:00
Radosław Kapka
8281634131 Introduce API StateFetcher (#8639)
* Return status.Errorf instead of plain errors from gRPC functions

* return plain errors from helper functions

* change errors to lowercase in node

* correct test expectations

* extracted StateFetcher

* StateFetcher tests

* extract beacon state creation option and fix state tests

* add comment to StateFetcher

* remove empty line

* overwrite config name

* fix field names

* remove FillRootsNaturalOpt helper

* reduce statefetcher package visibility

* gzl

* add missing error assertion

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-03-22 15:19:38 +00:00
Nishant Das
b346cde919 fix deduplication (#8640) 2021-03-22 07:07:58 -07:00
kevlu93
eca67cec4c Embed Config Pattern for Sync Services (#8636)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-03-21 19:07:42 +00:00
kevlu93
14439d2b12 Embed Config Pattern For Slasher, Slashing Protection (#8637)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-03-21 17:53:17 +00:00
kevlu93
4a64d4d133 Embed Config Pattern For Attestation, POW Chain, and RPC Services (#8635) 2021-03-21 16:58:41 +00:00
terence tsao
9421ac13d8 State: various clean up (#8633)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-03-19 20:03:19 +00:00
Raul Jordan
d2b1115f46 Feature Flag for Enabling Slashing Protection Pruning (#8632) 2021-03-19 17:54:47 +00:00
Preston Van Loon
9282a73663 Revert "Ignore Syncing Freelist For Validator DB" (#8629)
This reverts commit 3b6b3f6ef6.

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-03-19 05:30:09 +00:00
terence tsao
799a4d80cd State: refactor common root functions (#8630) 2021-03-18 16:29:06 -07:00
Preston Van Loon
fe6e6909e6 Validator Attestation History Pruning: Use a bolt cursor instead of bucket.ForEach, exit early when applicable (#8627)
* Use a bolt cursor instead of bucket.ForEach, exit early when applicable

* Refactor to reuse the same code

* gofmt

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-03-18 22:39:28 +00:00
Preston Van Loon
e477fdfd6d Embedded mainnet genesis state + load genesis.ssz from file (#8614)
* Update rules_go and fix proto conflicts

* gaz

* Update generated code

* First pass inclusion of using baked states

* more emptypb fixes

* remove testnet genesis files, only embed mainnet

* Refactoring for SaveGenesisData, fix tests that use mainnet config but do not support mainnet genesis values

* a bit more refactoring, load genesis from a file. Needs tests still

* Add method to ensure an embedded genesis file also has the appropriate genesis block

* gofmt

* more clear error

* Check genesis fork version to ensure testnet config matches genesis file

* viz

* test for SaveGenesisData

* More genesis db method tests

* Merge

* Minor tweaks, lint, fmt, etc

* Add more test to genesis db methods

* Revert beacon-chain/state/stateV0/BUILD.bazel

* Update beacon-chain/db/iface/errors.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* PR feedback

* Update beacon-chain/db/kv/genesis.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* fmt.Errorf works better for nil errors

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-03-18 21:00:00 +00:00
terence tsao
a921455836 Remove no bootnode error for Prater (#8628) 2021-03-18 19:38:04 +00:00
terence tsao
beadec32b8 Add prater boot nodes (#8623)
* Add bootnodes

* Typos

* Add sigp's lighthouse node
2021-03-18 14:26:18 +00:00
Nishant Das
693ce7b952 Mask All Debug Logs for Endpoints (#8624) 2021-03-18 09:13:01 +00:00
terence tsao
ce725ceec3 Move state pkg to stateV0 pkg (#8620)
* Move state pkg to stateV0 pkg

* Build.bazel

* Remove unused RootsArrayHashTreeRoot

* Revert "Remove unused RootsArrayHashTreeRoot"

This reverts commit bf0bda30d1.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-03-17 19:49:49 +00:00
kevlu93
ecf25d1284 Embed Config Pattern For Blockchain Service and Update Dependency Names Accordingly (#8618)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-03-17 13:36:56 -05:00
Radosław Kapka
0a73be7389 Return status.Errorf instead of plain errors from gRPC functions (#8619)
* Return status.Errorf instead of plain errors from gRPC functions

* return plain errors from helper functions

* change errors to lowercase in node

* correct test expectations

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-03-17 16:47:44 +00:00
Preston Van Loon
034a28710e Add Prater config (#8613)
* Add Prater config

* Register flag everywhere

* gofmt

* Apply suggestions from code review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Update shared/params/testnet_prater_config.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-03-17 14:14:07 +00:00
terence tsao
7b16601399 State: Return interface{} for pb related methods (#8617)
* Return interface{} instead of *pbp2p.BeaconState

* Comment
2021-03-16 22:26:17 -05:00
terence tsao
50e99fb6c1 State: Move compute field roots functions next to implementation (#8615) 2021-03-16 11:14:26 -07:00
terence tsao
ea4ea3d1c1 Remove unused AttestationDataRoot (#8610)
* Remove unused AttestationDataRoot

* Gazelle

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-03-16 16:12:32 +00:00
Radosław Kapka
1f8171d069 Dynamic key reload for remote keymanager (#8611)
* Dynamic key reload for remote keymanager

* fix failing keymanager test

* keymanager tests

* define RemoteKeymanager interface

* WaitForActivation tests

* gzl

* handle error in test
2021-03-16 10:00:05 -05:00
terence tsao
9fea9816bd BeaconState: Expose MarshalSSZ (#8609) 2021-03-16 00:43:27 +00:00
terence tsao
aa389c82a1 Send attestations over feed at right place (#8605) 2021-03-15 07:47:10 -07:00
Radosław Kapka
c577fbd772 Move attestation's source checkpoint validation to VerifyAttestationNoVerifySignature (#8598)
* Move attestation's source checkpoint validation to VerifyAttestationNoVerifySignature

* change parameter type to ReadOnlyBeaconState

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-03-12 21:43:20 +00:00
Radosław Kapka
dc6dee3f4e Fix issues with dynamic key reload for imported/derived keymanager (#8585)
* implementation of handling reload

* proper implementation

* fix WaitForActivation tests

* HandleKeyReload tests

* runner tests

* do not reuse log hook between tests

* gzl

* Drop unnecessary use of the blank identifier

* move mock_validator to testutil package

* Reorganize validator client package

* reduce package visibility

* revert account changes

* gzl

* describe nil parameter

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-03-12 17:23:56 +00:00
Nishant Das
a3c96c2f44 Prune Better For Larger Buckets (#8599)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-03-12 15:05:20 +00:00
Nishant Das
3b6b3f6ef6 ignore syncing freelist for validator db (#8601) 2021-03-12 08:06:20 -06:00
Preston Van Loon
eb694ab5d5 Update rules_go and fix proto conflicts (#8596)
* Update rules_go and fix proto conflicts

* gaz

* Update generated code

* more emptypb fixes

* gaz

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2021-03-11 18:03:19 -06:00
Victor Farazdagi
fa2084330b Fix race condition in TestService_Initialized (#8597) 2021-03-11 21:16:40 +00:00
Raul Jordan
286444a2ec Shorter Pruning Period for Slashing Protection (#8590)
* begin a simpler pruning process

* shorter pruning

* pass all pruning tests

* passing tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-03-11 15:51:16 +00:00
Victor Farazdagi
55b6134be1 Improves UTs coverage in shared/aggregation (#8593)
* improves shared/aggregation test suite

* improves coverage in shared/aggregation/attestations
2021-03-11 15:01:19 +00:00
Radosław Kapka
5374d07c4d Reorganize validator client package (#8592)
* Reorganize validator client package

* reduce package visibility
2021-03-11 11:57:41 +00:00
terence tsao
b62619ae3a State: Refactor Reference and ValidatorMapHandler to stateutil pkg (#8589)
* Starting

* Fix tests

* Gazelle
2021-03-10 20:57:46 -06:00
terence tsao
dc0fc94c13 Use read lock (#8588) 2021-03-10 15:11:25 -08:00
Victor Farazdagi
548b471b8a Simplify nil assignments (#8587)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-03-10 20:38:16 +00:00
Victor Farazdagi
fa92766095 Fixes flaky TestBlocksFetcher_nonSkippedSlotAfter test (#8586)
* Fixes flaky TestBlocksFetcher_nonSkippedSlotAfter test

* fix conditional
2021-03-10 20:03:26 +00:00
Benoit Perroud
9980ca3b7e Add metrics per keys for next scheduled attestation and proposal (#8583)
* Add metrics per keys for next scheduled attestation and proposal

* Found a better place where to update the counters

* Wrap with emitAccountMetrics flag
2021-03-09 21:13:11 +00:00
terence tsao
72be10f9a5 Ran go imports (#8582) 2021-03-09 19:56:05 +00:00
Raul Jordan
ab301aa4fe Remove Accounts Enable/Disable Code (#8576)
* rem

* remove all enable disable code

* fix broken build

* fix more tests

* fix broken tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-03-09 15:46:50 +00:00
Nishant Das
2bb0a602e4 create the bucket only once (#8579)
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2021-03-09 08:22:52 -06:00
Nishant Das
363771a5c7 Update Go Mod (#8578) 2021-03-09 10:26:06 +00:00
terence tsao
bdf2b2019b Use beacon state interfaces across (#8552) 2021-03-08 14:37:33 -08:00
Victor Farazdagi
f2125e5f64 Proposer attestation selection using max-cover (#8571)
* Proposer attestation selection using max-cover

* better alisgn struct field

* more tests

* cleanup

* simplify expressions

* add benchmarks

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-03-08 18:46:24 +00:00
Victor Farazdagi
294b031fa4 Fixes incorrect usages of gazelle:ignore (#8562)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-03-08 18:09:39 +00:00
Raul Jordan
4a98300c59 Use Path to Deposit Data JSON File in Generate Genesis State (#8575)
* use path to deposit data json

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-03-08 16:57:30 +00:00
Preston Van Loon
0f1d14437c Update blst to v0.3.3 (#8574)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-03-08 15:40:10 +00:00
Radosław Kapka
79754bded2 Implement SubmitAttestations in the beacon API (#8563)
* update ethereumapis deps

* V1AttToV1Alpha1 migration

* Implementation plus happy path test

* fix root variable names

* Invalid attestation test

* gzl

* mod tidy

* use a single append to concatenate two slices

* remove outdated comment from attestation processing

* invoke ProcessAttestationNoVerifySignature when validating attestations

* implement missing PoolMock members

* use new VerifyAttestationNoVerifySignature function
2021-03-08 15:42:05 +01:00
terence tsao
90da16432f Refactor ProcessAttestationNoVerifySignature to VerifyAttestationNoVerifySignature (#8566)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-03-06 14:53:28 +00:00
Preston Van Loon
f074c5ee89 Fix validator activation monitoring with inactive keys (#8558)
* refactor / move waiting for activation updates

* Commentary

* Update test to follow the full code path

* gofmt and goimports

* manual imports fixes

* Apply suggestions from code review

typo fixes

* Remove redundant handleAccountsChanged and chan. Thanks @nisdas

* var sub = to sub :=

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-03-05 18:33:39 +00:00
Radosław Kapka
edd86fd358 Simplify API beacon pool tests (#8564)
* fix comment of FarFutureEpoch

* remove not needed validator configuration
2021-03-05 16:10:48 +00:00
Nishant Das
067a519b37 Add Spans To Attestation Caches (#8556)
* add spans

* preston's comments

* add span
2021-03-05 09:17:27 -06:00
Raul Jordan
32f2f711db Exclude Instead of Ignore With Gazelle (#8559)
* exclude and not ignore

* fix broken test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-03-04 21:01:32 +00:00
terence tsao
c1d4ff6239 Use deep ssz (#8557) 2021-03-04 13:31:39 -06:00
Radosław Kapka
e36c3dd668 Ignore "safe" secrets in DeepsSource analysis (#8555) 2021-03-04 13:52:55 +00:00
terence tsao
1c7c62cf8a Attestation pool: continue if bad (#8550)
* Continue saving attestation when one is bad

* Gazelle

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-03-04 08:14:47 +00:00
Raul Jordan
d215607e55 Add Goland Standard "cmd" Pattern for Slasher Binaries (#8542)
* vis

* gaz

* imports

* slasher img

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2021-03-03 18:37:57 +00:00
Raul Jordan
ff329df808 Add All Other Prysm CLI Entrypoints to "cmd" Folder (#8545)
* other cmd

* comments

* gaz

* gaz ignored file

* build

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-03-03 17:05:37 +00:00
Preston Van Loon
c9858b5e6b Validator: Remove optional and duplicated dial opts for streaming middleware (#8549) 2021-03-03 10:28:13 -06:00
Nishant Das
565d51009d Make DB Backup More Efficient (#8543)
* checkpoint

* add test

* Update beacon-chain/db/kv/backup_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-03-03 15:37:13 +00:00
terence tsao
c6b74b2ba7 Beacon state: Add interfaces (#8544)
* Define Readonly and writeonly

* Comment

* First take at clean up

* Completed interfaces

* Fix build and test

* Reordering interfaces

* Add build.bazel

* Add build.bazel

* Fix visibility
2021-03-03 02:26:24 +00:00
Raul Jordan
090fbbf18c Add Goland Standard "cmd" Pattern for Beacon-Chain Binaries (#8540)
* beacon chain cmd pattern

* imports spacing

* more import fix

* edit build file

* e2e viz

* amend e2e
2021-03-02 13:36:03 -06:00
Raul Jordan
cdea2debc9 Add Goland Standard "cmd" Pattern for Validator Binaries (#8541)
* validator cmd

* imports

* more imports

* e2e viz

* alias

* use native alias

* add actual

* fix macro

* work on fix e2e

* add viz

* gaz

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-03-02 12:58:40 -06:00
terence tsao
4c49d4af7e Clean up state pkg's getter and setter (#8533)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-03-02 12:37:36 +00:00
Panagiotis Georgiadis
46f6bd6d08 A bit of cleanup based on goconst tool (#8529)
* Make 1 occurence of 'foo', 'bar' and 'fizz!'

* Make 1 occurence of 'merkleizing list that is too large, over limit' string

* Limit password occurrences

* Make only 1 occurence of 'strongPass'

* Limit testMnemonic occurrences

* Limit expected epoch error messages

* Rename errors and use constant

* Update bazel dependencies

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-03-02 11:39:54 +00:00
Preston Van Loon
b3dcbfe2a4 Update snappy to fix ARM64 with go.1.16 (Reverts #8538) (#8539)
* Revert "Revert "Update to go 1.16" (#8538)"

This reverts commit a92b20d2bb.

* Update snappy
2021-03-02 16:54:16 +08:00
Nishant Das
a92b20d2bb Revert "Update to go 1.16" (#8538)
This reverts commit f4adc0ea86.
2021-03-02 05:35:08 +00:00
Nishant Das
8a27449595 Increase default peer limit from 30 to 45 (#8530)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-03-01 18:35:27 +00:00
Eduard Klementiev
a3781e2ffc Add prysm version to prometheus (#8527)
* Add prysm version to prometheus

* gofmt

* apply suggestion

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-03-01 17:50:20 +00:00
Radosław Kapka
f973924fbf Implement SubmitVoluntaryExit and SubmitProposerSlashing in the beacon API (#8532)
* SubmitProposerSlashing

* SubmitVoluntaryExit

* rename variables
2021-03-01 16:08:39 +00:00
Radosław Kapka
9547f53e88 Migration package tests (#8524)
* first few tests

* rest of tests

* gzl

* test block hydration

* compare roots

* gzl

* compare exit roots

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-03-01 13:52:07 +00:00
terence tsao
c30ee6c166 Ran go import (#8528)
* Ran go import

* Sort
2021-02-28 13:49:00 +01:00
ahadda5
29d1959b81 Recursively Find Validator Database File In Slashing Protection Commands (#8518)
* issue/feature 8500 find validator.db automatically

* gazelle build

* remove less! why there?

* fixed errors import

* fixed errors import

* unit tested

* adding the find validator.db fileutil func to export and import slashing

* deleted the comment

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-02-26 11:04:18 -06:00
Eduard Klementiev
878bc15229 Add --list-validator-indices flag (#8520)
* Add --list-validator-indices flag

* Fix style issues

* Print pubkeys alongside the validator indexes

* Improve list-indices output

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-02-26 15:00:05 +00:00
Nishant Das
e39ce36f1c Fix Default Transport Option (#8525)
* fix

* mod
2021-02-26 10:26:39 +01:00
Nishant Das
b4648f1df9 Update Test Case in Batching Public Keys (#8523) 2021-02-26 01:44:42 +00:00
Preston Van Loon
f4adc0ea86 Update to go 1.16 (#8521)
* Update to go 1.16

* Also update go.mod

* rm go.sum then run go mod tidy

* go mod tidy with go 1.16

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-02-25 21:38:19 +00:00
terence tsao
658cbf5631 RPC: Proper status code for error (#8519)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-25 20:00:33 +00:00
terence tsao
b400098296 Remove unused code (#8517)
* Remove unused code

* Go mod tidy

* Gazelle

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-25 18:13:13 +00:00
Radosław Kapka
c3f875bf65 Implement SubmitAttesterSlashing in the beacon API (#8515)
* Implement SubmitAttesterSlashing in the beacon API

* gzl

* remove migration test rule

* fix BUILD file formatting

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-02-25 11:14:04 -06:00
pinglamb
5db5ca7056 Use eth2-types SSZUint64 (#8514) 2021-02-25 05:51:26 -08:00
Josh Yudaken
f0eb843138 Graffiti ordered index (#8482)
* Added ordered option to graffiti file

* Updated validator to use Ordered graffiti

* Track graffiti ordered index in db

* Update `ordered` to only emit each graffiti once

Co-authored-by: pinglamb <pinglambs@gmail.com>
2021-02-24 22:50:47 +00:00
Potuz
d44c27ec63 Log block arrival time (#8485)
* Log block arrival time

* Comment formatting

* Update beacon-chain/sync/validate_beacon_blocks.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-02-24 20:35:19 +00:00
Nishant Das
0625a6906c Add Better Peer Pruning (#8501)
* add better pruning

* add test

* gaz

* victor's review

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-24 18:40:06 +00:00
Radosław Kapka
4d28d5e4d2 Listen for account changes only when required (#8503)
* initial implementation

* remove listening for changes from wallet creation

* goimports

* test fix

* more goimports

* listen for changes when initializing wallet through gRPC

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-24 18:05:46 +00:00
Nishant Das
08b938982b Propagate Cancellation To Beacon Node Server (#8512)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-24 17:10:25 +00:00
Raul Jordan
6ee290a9af Analyze Secret Credential Additions in DeepSource (#8511)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-24 16:21:28 +00:00
Radosław Kapka
4da7a7797e Implement ListPoolProposerSlashings and ListPoolVoluntaryExits in the beacon API (#8508)
* Implement ListPoolProposerSlashings in the beacon API

* implement ListPoolAttesterSlashings

* add comments to bool arguments

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-24 15:29:25 +00:00
Preston Van Loon
6e831920bf Fix slice out of bounds error in validator db migration (#8510)
* Fix slice out of bounds error in validator db migration #8509

* Add regression testing

* make it double just in case

* gofmt
2021-02-24 15:02:39 +00:00
terence tsao
eca8d85446 Remove outdated and skipped TestConsistentGenesisState (#8506)
* Rm skipped TestConsistentGenesisState

* Fix build: remove unused imports

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-02-24 13:40:19 +00:00
terence tsao
1db3c86b68 Use DeepSSZEqual to satisfy go tests (#8499)
* Use DeepSSZEqual to satisfy go tests

* New line

* New line

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-23 22:20:11 +00:00
Raul Jordan
0c599521b1 Better Handling of Subscriber Errors in Logs Streams (#8505)
* handle subscriber error and increase buffer sizes

* operation order in unsub and close
2021-02-23 15:33:23 -06:00
Radosław Kapka
e40fba7679 Implement ListPoolAttesterSlashings in the beacon API (#8492)
* pool interface and mock

* implementation

* gofmt

* gzl

* Use migration package for slashing mapping

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-02-23 16:17:07 +01:00
terence tsao
6b82873737 Use validatorIndex instead of validatorID (#8498)
* Use ValidtorIndex across Prysm. Build ok

* First take at fixing tests

* Clean up e2e, fuzz... etc

* Fix new lines

* Update beacon-chain/cache/proposer_indices_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update beacon-chain/core/helpers/rewards_penalties.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update beacon-chain/core/helpers/shuffle.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/graffiti/parse_graffiti_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Raul's feedback

* Fix downcast int -> uint64

* Victor's feedback

* Replace validator id with validator index

* Replace validator id with validator index

* Typo

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-02-22 19:40:58 -06:00
terence tsao
3edfa8cb88 Use Custom Type ValidatorIndex Across Prysm (#8478)
* Use ValidtorIndex across Prysm. Build ok

* First take at fixing tests

* Clean up e2e, fuzz... etc

* Fix new lines

* Update beacon-chain/cache/proposer_indices_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update beacon-chain/core/helpers/rewards_penalties.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update beacon-chain/core/helpers/shuffle.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/graffiti/parse_graffiti_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Raul's feedback

* Fix downcast int -> uint64

* Victor's feedback

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-23 00:14:50 +00:00
Victor Farazdagi
b577869ed6 Fixes import aliases (#8497)
* Fixes import aliases

* another fix

* reset gw files

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-22 23:20:57 +00:00
Ivan Martinez
5c14b4c833 ETH2 API: Add outline for debug endpoints (#8496)
* Add outline for debug endpoints

* Add outline for debug endpoints

* Fix comment

* Fix visibility
2021-02-22 16:48:49 -06:00
Potuz
dd08305aa7 Clean duplicated topic field in debug log (#8489)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-02-22 18:13:26 +00:00
Victor Farazdagi
1395c11c29 Fix duplicate import (#8494) 2021-02-22 17:36:18 +00:00
Potuz
ef48f6a061 Move timings of synced block to debug level (#8491)
* Move timings of synced block to debug level

* go fmt

* Change message text

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-02-22 16:32:23 +00:00
terence tsao
09ddfae1d9 Use DeepSSZEqual for tests (#8484)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-02-22 15:33:55 +00:00
Preston Van Loon
ad9cd1933a Add nil check for validator db migration (#8493)
* Add nil check for validator db migration

* continue if source bucket is nil
2021-02-22 14:28:34 +00:00
Nishant Das
0f515784d8 Update Geth Fork (#8490)
* update to latest

* fix
2021-02-22 21:35:45 +08:00
Radosław Kapka
7d3e53f3e4 Constrain read/write channels (#8479) 2021-02-19 15:39:27 +00:00
terence tsao
dc1bec79ed Use Eth2 type CommitteeIndex (#8477)
* Use types.CommitteeIndex

* Go fmt

* Update validator pkg

* Fix e2e

* Happy fuzz tests

* Sync with upstream ethereumapi

* Go mod tidy

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-02-18 20:11:20 +00:00
terence tsao
d472380fef Hide beacon operation field in log if it's 0 (#8330)
* Hide beacon operation field if it's 0

* Update test

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-02-18 19:08:27 +00:00
terence tsao
eea0160dd4 Reformat healthz message by adding a comma (#8472)
* Add comma

* Update shared/prometheus/service.go

Co-authored-by: Nishant Das <nishdas93@gmail.com>

* Update tests

* Apply suggestions from code review

* Update shared/prometheus/service_test.go

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-18 16:31:16 +00:00
Radosław Kapka
1ae429dc61 Implement GetFinalityCheckpoints in the beacon API (#8476)
* span

* Implement GetFinalityCheckpoints in the beacon API
2021-02-18 09:46:17 -06:00
pinglamb
fc265055df Update bazelbuild/rules_go to 0.24.13 (#8463)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-02-18 12:57:16 +08:00
Nishant Das
917252d7d0 Use Correct Peer Status Method (#8471) 2021-02-18 03:35:44 +00:00
Raul Jordan
4f9752bb3e Stop Early in Validator Surround Vote Local Protection (#8460)
* include migration and logic for stopping early in slashing protection checks

* remove commented code

* extract methods

* migration logic tested up

* migration up and down tests

* Update validator/db/kv/attester_protection.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* added in pruning and batched migrations

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-02-17 19:23:59 +00:00
Nishant Das
6391dec5de Reform Inbound Limit (#8465)
* clean up

* name

* comment

* change back

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-17 17:48:44 +00:00
Raul Jordan
1ba414bd77 Ignore Analyzers Package in DeepSource Config (#8467) 2021-02-17 16:53:45 +00:00
Radosław Kapka
6f2438436c Implement GetStateFork in the beacon API (#8456)
* update ethereumapis dependency

* span

* initial implementation

* introduce stategen Service interface and MockService

* Include AddStateForSlot function in the mock service

* return states from mock

* add GenesisState to POWChain mock

* populate roots in helper state

* initialize Slot when creating helper state

* tests

* code refactor - extract helper functions

* gzl

* use SetSlot in tests

* handle SetSlot error

* use new testutil's NewBeaconState

* gzl

* go mod tidy

* rename Service to StateManager

* move regex check to helper

* implement StateByStateRoot

* initial implementation

* tests

* refactor code into smaller functions

* gzl

* simplify StateByStateRoot and tests

* Nishant's feedback

* gzl

* handle error in test

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-02-17 09:26:39 -06:00
Nishant Das
2515dc4c06 Clean Up Go Sum (#8462) 2021-02-17 09:30:56 +00:00
terence tsao
2c36e6534c Better feature flag logging (#8457)
* Remove unused core functions

* First take

* Apply for validator

* Use constant

* Use constant
2021-02-16 19:57:42 +00:00
Nishant Das
fe27ca359c Deprecate Uneeded Flags (#8455)
* remove disable pruning flag

* deprecate disable majority vote flag

* remove eth1dataVoteCache

* remove outdated methods

* gaz
2021-02-16 15:27:37 +00:00
Nishant Das
bf7425bae4 rmeove methods (#8454) 2021-02-16 17:56:02 +08:00
Preston Van Loon
558b16275d Validator: Annotate attestation error spans (#8451)
* Annotate attestation spans

* Annotate attestation spans

* gaz

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-16 09:06:52 +00:00
Victor Farazdagi
a069738c20 ETH2 Types: Slot (#8408)
* update shared/params

* update eth2-types deps

* update protobufs

* update shared/*

* fix testutil/state

* update beacon-chain/state

* update beacon-chain/db

* update tests

* fix test

* update beacon-chain/core

* update beacon-chain/blockchain

* update beacon-chain/cache

* beacon-chain/forkchoice

* update beacon-chain/operations

* update beacon-chain/p2p

* update beacon-chain/rpc

* update sync/initial-sync

* update deps

* update deps

* go fmt

* update beacon-chain/sync

* update endtoend/

* bazel build //beacon-chain - works w/o issues

* update slasher code

* udpate tools/

* update validator/

* update fastssz

* fix build

* fix test building

* update tests

* update ethereumapis deps

* fix tests

* update state/stategen

* fix build

* fix test

* add FarFutureSlot

* go imports

* Radek's suggestions

* Ivan's suggestions

* type conversions

* Nishant's suggestions

* add more tests to rpc_send_request

* fix test

* clean up

* fix conflicts

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
2021-02-16 07:45:34 +00:00
Preston Van Loon
aef5a7b428 Validator: Choose a reasonable cap for the attested target epochs slice (#8452)
* Choose a reasonable cap for the attested target epochs slice

* revert bazelrc

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-02-16 01:37:35 +00:00
terence tsao
4bed8d4ed4 Remove unused core functions (#8449)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-15 22:50:50 +00:00
ahadda5
f4a6b90e8b Mask credentials for logging (#8429)
* Added MaskCredentialsLogging to logutil, which masks the user info, path and query. It leaves the hostname and port untouched . Making it more secure during logging

* Added MaskCredentialsLogging to logutil, which masks the user info,path and query. It leaves the hostname and port untouched . Making it more secure during logging

* Added newline based on the PR checks

* Update shared/logutil/logutil.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update shared/logutil/logutil.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Gazelle

* Update shared/logutil/logutil.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update shared/logutil/logutil.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* added unit tests

* updated one test case

* added logutil_test.go unit test cases

* Refactor validator subnet subscriptions to be non-blocking (#8319)

* Use response.NextEpochDuties for aggregator subnet subscriptions (credit: @KaanKC PR #8204). Make committee subnet subscriptions method non-blocking call

* Fix test

* Fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Add Ability to Specify All Public Keys When Exiting Validators (#8399)

* add programmatic voluntary exit

* add exit all flag

* test

* lint

* add multiple exits test

* fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Increase Validation Queue (#8431)

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Validator: add a DEBUG log to show batch attestation save duration (#8432)

* Add a debug log to show duration

* Autofix issues in 1 file

Resolved issues in validator/db/kv/attester_protection.go via DeepSource Autofix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>

* Add Mutex and Block Profiling (#8435)

* Implement GetStateRoot in the beacon API (#8402)

* update ethereumapis dependency

* span

* initial implementation

* introduce stategen Service interface and MockService

* Include AddStateForSlot function in the mock service

* return states from mock

* add GenesisState to POWChain mock

* populate roots in helper state

* initialize Slot when creating helper state

* tests

* code refactor - extract helper functions

* gzl

* use SetSlot in tests

* handle SetSlot error

* use new testutil's NewBeaconState

* gzl

* go mod tidy

* rename Service to StateManager

* move regex check to helper

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* On Block Cleanup (#8438)

* Beacon API: update GetStateRoot  (#8437)

* Address various feedbacks

* Gaz

* More nil check

* Update beacon-chain/rpc/beaconv1/state_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/beaconv1/state_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/beaconv1/state_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/beaconv1/state_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/beaconv1/state_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* qualifying my unix user ahaddad - no real changes to the files

* Update shared/logutil/logutil.go

* Update shared/logutil/logutil.go

* Update shared/logutil/logutil.go

* Update shared/logutil/logutil.go

* Update shared/logutil/logutil.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-02-15 21:54:19 +00:00
Raul Jordan
36b6a71af4 Configurable DB Mmap Size for Beacon Node and Validator Client (#8448)
* add flag to beacon and validator

* gaz

* fuzz

* add dep viz

* add to tools
2021-02-15 20:29:47 +00:00
Raul Jordan
7c3827a9a4 add test (#8444) 2021-02-15 11:55:28 -06:00
Potuz
d17f210024 Log Block Processing Time (#8441)
* Log Block Processing Time instead of time in slot

* Restore timeSinceSlotStart

* Renamed fields

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-02-15 10:33:38 -06:00
terence tsao
28631e7791 Verify nil block helper (#8447) 2021-02-15 15:11:25 +00:00
terence tsao
28839fbab2 Use latest block header + slot as skip slot cache key (#8443) 2021-02-13 23:13:20 +00:00
Radosław Kapka
0716519be9 Fix error formatting inside fmt.Errorf (#8439)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-12 23:44:46 +00:00
terence tsao
068f758f49 Code inspect - clean ups (#8445) 2021-02-12 17:04:45 -06:00
Preston Van Loon
e2c5ae53e7 Validator: Safer pending attestation records flushing (#8433)
* Add a debug log to show duration

* merge from dev

* use safe pending attestation records struct

* fix build, use atomic bool

* Add deadline checks to CheckSlashableAttestation

* Go fmt

* Add test for in-progress log

* GoDocs

* Rename pending attestation records to queued attestation records

* rename and add commentary on log

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-12 20:19:01 +00:00
Preston Van Loon
473172ca8b Validator: Make read operation use db.view instead of db.update (#8434)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-12 19:06:58 +00:00
Radosław Kapka
47fdb3b99a Revert "Rename NewService to New (#8337)" (#8440)
* Revert "Rename `NewService` to `New` (#8337)"

This reverts commit d121b19145.

# Conflicts:
#	beacon-chain/sync/initial-sync/round_robin_test.go

* fix name in test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-12 17:45:22 +00:00
terence tsao
143d3a3203 Process attestation: skip if forkchoice attestation count == 0 (#8436)
* Cont. if there's no fork choice attestations

* Comment

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2021-02-12 17:11:42 +00:00
terence tsao
66471c2f13 Beacon API: update GetStateRoot (#8437)
* Address various feedbacks

* Gaz

* More nil check

* Update beacon-chain/rpc/beaconv1/state_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/beaconv1/state_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/beaconv1/state_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/beaconv1/state_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/beaconv1/state_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-02-12 15:38:36 +00:00
Nishant Das
7f6b15271a On Block Cleanup (#8438) 2021-02-12 11:36:53 +00:00
Radosław Kapka
cbb0f1e11d Implement GetStateRoot in the beacon API (#8402)
* update ethereumapis dependency

* span

* initial implementation

* introduce stategen Service interface and MockService

* Include AddStateForSlot function in the mock service

* return states from mock

* add GenesisState to POWChain mock

* populate roots in helper state

* initialize Slot when creating helper state

* tests

* code refactor - extract helper functions

* gzl

* use SetSlot in tests

* handle SetSlot error

* use new testutil's NewBeaconState

* gzl

* go mod tidy

* rename Service to StateManager

* move regex check to helper

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-11 21:08:36 +00:00
Raul Jordan
25caa6d1be Add Mutex and Block Profiling (#8435) 2021-02-11 20:20:19 +00:00
Preston Van Loon
d551c8e1d0 Validator: add a DEBUG log to show batch attestation save duration (#8432)
* Add a debug log to show duration

* Autofix issues in 1 file

Resolved issues in validator/db/kv/attester_protection.go via DeepSource Autofix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
2021-02-11 18:27:35 +00:00
Nishant Das
2fd2bafdfa Increase Validation Queue (#8431)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-11 17:35:52 +00:00
Raul Jordan
e236dedc32 Add Ability to Specify All Public Keys When Exiting Validators (#8399)
* add programmatic voluntary exit

* add exit all flag

* test

* lint

* add multiple exits test

* fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-11 16:50:16 +00:00
Preston Van Loon
7d2f7ae9e1 Refactor validator subnet subscriptions to be non-blocking (#8319)
* Use response.NextEpochDuties for aggregator subnet subscriptions (credit: @KaanKC PR #8204). Make committee subnet subscriptions method non-blocking call

* Fix test

* Fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-02-11 16:09:17 +00:00
Nishant Das
ed9c69ec61 Use a Slim Base Image for our Cross-Build Toolchain Docker Image (#8425)
* change it to slim

* update

* Ran regenerate.sh

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-02-11 03:03:15 +00:00
Ivan Martinez
b6a40094a6 Exclude unexported fields for protos in sszutils.DeepEqual (#8415)
* Fix deep equal

* Fixes

* gaz

* Fix test

* Add UnexportedOnly function

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-10 23:00:18 +00:00
terence tsao
de15d6d2c1 Some improvements to proposer cache (#8424)
* Revert to use input epoch

* Revert back to process epoch

* Use processed state

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-10 22:07:12 +00:00
Nishant Das
143cb142bc Make Individual Validators Immutable (#8397)
* initial POC

* clean up

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-10 20:52:45 +00:00
terence tsao
d3e93dd106 Process attestation: reduce checkpoint copies (#8409)
* Clean up process attestation

* Add matching getters

* Fix tests

* Update tests

* Fix test

* Remove read locks

* Typo

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-10 19:30:11 +00:00
terence tsao
56c5938898 Send feed faster with UpdateHeadTimely feature flag (#8422)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-10 18:26:23 +00:00
terence tsao
d44ab1ace5 Add timeSinceSlotStart field to "Synced new block..." log (#8420)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-02-10 17:24:40 +00:00
Nishant Das
ae028d9c1d Insert Finalized Deposits In Another Routine (#8405)
* chk

* terence's review

* add test

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-02-10 10:35:44 -06:00
Preston Van Loon
cbd01d4ff4 Provide TLS certificate to gRPC gateway (#8418)
* Provide TLS certificate to gRPC gateway

* Provide TLS certificate to gRPC gateway

* Provide TLS certificate to gRPC gateway

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-10 04:23:12 +00:00
Victor Farazdagi
65645face1 Update ethereumapis deps (#8417)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-10 03:33:33 +00:00
Ivan Martinez
cd3851c3d5 Add DeepSSZEqual and DeepNotSSZEqual (#8421) 2021-02-09 20:57:22 +00:00
terence tsao
2f98e6aaaf Update head per slot (#8381)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-02-09 17:24:48 +00:00
terence tsao
9afc9d92d9 Feature flag: update head timely (#8412)
* Feature flag: update head timely

* Move finalized imply justified up

* Fix resolve error

* Use invert feature flag

* Make diff easier to review

* Line

* Typo

* Fix condition

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-02-09 15:35:14 +00:00
Victor Farazdagi
a8e501b3cf ETH2 Types: Epoch (#8373)
* update deps

* update deps

* update protos/*

* update deps

* reset protos

* update protos

* update shared/params/config

* update protos

* update /shared

* update shared/slotutil and shared/testutil

* update beacon-chain/core/helpers

* updates beacon-chain/state

* update beacon-chain/forkchoice

* update beacon-chain/blockchain

* update beacon-chain/cache

* update beacon-chain/core

* update beacon-chain/db

* update beacon-chain/node

* update beacon-chain/p2p

* update beacon-chain/rpc

* update beacon-chain/sync

* go mod tidy

* make sure that beacon-chain build suceeds

* go fmt

* update e2e tests

* update slasher

* remove redundant alias

* update validator

* gazelle

* fix build errors in unit tests

* go fmt

* update deps

* update fuzz/BUILD.bazel

* fix unit tests

* more unit test fixes

* fix blockchain UTs

* more unit test fixes
2021-02-09 10:05:22 +00:00
Preston Van Loon
5727d4eb8a Update Herumi with a fix for older intel chips (#8413)
* Update Herumi with a fix for older intel chips. #8410

* Use the correct version of bls-eth-go-binary
2021-02-09 01:44:27 +00:00
terence tsao
fed65122fe Use EnableNextSlotStateCache few more places (#8398) 2021-02-08 13:11:21 -08:00
Radosław Kapka
86a9d4c6a4 Configurable testutil's BeaconState (#8407)
* Configurable testutil's BeaconState

* fix shared and fuzz tests

* return state copy

* use mainnet config values for default state

* handle error in block fuzz

* goimports

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-02-08 20:00:09 +00:00
Panagiotis Georgiadis
0a180dc662 Make a const instead of 3 'abc' occurences (#8406) 2021-02-08 17:08:29 +01:00
terence tsao
f9303ca2e4 Clean up unused functions (#8403)
* Clean ups

* Gazelle
2021-02-05 18:39:15 +00:00
terence tsao
4c25fe978a Update span names (#8394)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-02-04 20:23:50 +00:00
terence tsao
cf88afb287 Block validator buckets (#8395)
* Block validator buckets

* add more buckets to blocked collector list

Co-authored-by: rauljordan <raul@prysmaticlabs.com>
2021-02-04 03:30:09 +00:00
Victor Farazdagi
c97ea766ca Attestation aggregation: optimizations and benchmarks (#7938)
* profitablity tests

* cleanup benchmark

* fix deduplication function

* dedup: move method to atts list

* proper substring handling

* refactor validate method

* update benchmarks

* prepare proposer test

* remove redundant code

* reset test

* remove dedup from maxcover - moved to proposer

* remove redundant test

* remove lower level check for bit length

* optimize candidate validation on att aggregation

* restore test

* fix test

* fix test

* remove dedup functionality

* add benchmark

* optimize list usage

* Attestation aggregration: remove redundant dedup routine

* fix func call

* experiment with bitset based cover

* add benchmark

* samplem implementation using Bilist64

* add tests

* remove redundant code

* remove tmp comments

* unskip test

* update benchmarks

* gazelle

* process err

* optimized max-cover

* Max-cover: optimized implementation based on Bitlist64

* gazelle

* re-arrange overlaps check

* minor comments

* add Bitlists64WithMultipleBitSet

* update benchmarks

* gazelle

* add TestAggregateAttestations_rearrangeProcessedAttestations

* minor updates to rearrange method

* add link to design doc

* remove redundant methods

* simplify test

* add TestAggregateAttestations_aggregateAttestations

* fix issues

* fix assignment

* use ToBitlist(), ToBitlist64()

* fixes test

* benchmarks

* fix typo

* allow opt_max_cover opt-int flag

* update benchmarks

* reset e2e

* fix test

* enable opt_max_cover in e2e tests

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-04 00:58:33 +00:00
Preston Van Loon
e52a821f73 Validator testing library should be marked as testonly (#8392)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-03 23:16:41 +00:00
Nishant Das
d4f241d875 Add Custom Deadline for Slot Progression (#8388)
* use custom deadline

* revert

* preston's comment

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-03 21:39:54 +00:00
Preston Van Loon
48ae49765e Revert "Change sszutil DeepEqual to ignore unexported fields" (#8391)
* Revert "Change sszutil DeepEqual to ignore unexported (#8336)"

This reverts commit 8d986bd414.

* Add back tests, make tests have equal and non equal check

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-03 20:39:07 +00:00
Raul Jordan
afa5b5e790 Add Spans for Validator DB Methods (#8390)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-03 19:29:20 +00:00
Preston Van Loon
c5551ebebb Revert DEPENDENCIES.md change from #8257 (#8389) 2021-02-03 18:46:46 +00:00
terence tsao
616081fbdd Ran code inspect (#8387) 2021-02-03 10:59:17 -06:00
Victor Farazdagi
842bafb002 Update bitfield dependency (#8385) 2021-02-02 21:33:48 +00:00
Raul Jordan
953cc9733c Update Ethdo Keystore Deps (#8382)
* update ethdo deps

* revert workspace

* tidy
2021-02-02 19:05:47 +00:00
Raul Jordan
caac08df33 Add Batch Method for Reading Validator Proposing Histories (#8378)
* add in batch method

* add in new proposal history methods for efficiency and progress bars

* tests fixed to use the new methods

* add back get slot proposing history method

* add gaz
2021-02-02 15:53:12 +00:00
Radosław Kapka
3fd8c4c046 Implement GetGenesis in the beacon API (#8380)
* span

* initial implementation

* test wip

* set nanos to zero

* more testing

* gzl
2021-02-02 08:44:28 -06:00
Radosław Kapka
b5a82b9075 Display URLs for tracking exited validators (#8361)
* Display URLs for tracking exited validators

* extract displaying info to separate function

* fix URL comment

* extract config names

* gzl

* lowercase config names

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-01 19:00:06 +00:00
Nishant Das
c6e96204e8 Update Herumi BLS Library (#8377)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-01 18:31:48 +00:00
terence tsao
2456e6f34d Add missed feature flag usage (#8376)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-01 17:42:50 +00:00
Radosław Kapka
48ed506487 Ajust constructor names of nodes (#8362)
* clean up constructors for nodes

* resurrect validator client

* fix small naming issues

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-01 17:12:52 +00:00
Shay Zluf
4595789ac8 Make validator stable when beacon node goes offline (#8278)
* Make validator stable POC

* fix feedback raul and nishant

* fix wait till first iteration

* fix imports

* retry tests

* fix init

* test retry receive blocks

* remove redundant return statement

* terence feedback

* terence feedback

* remove log

* to check for context after fist call

* remove fatal

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-02-01 16:29:54 +00:00
Victor Farazdagi
d53fdcf781 Simplify expressions (#8370)
* Simplify expressions

* avoid escaping

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-01 15:14:36 +00:00
Victor Farazdagi
372dc47b64 Deprecate enableSyncBacktracking flag (#8371)
* Deprecate enableSyncBacktracking flag

* keep deprecated flags
2021-02-01 11:18:24 +00:00
pinglamb
82426abf5f Add canonical flag to ListBlocks API response (#8287)
* Add canonical flag to ListBlocks response

* Genesis is always canonical
2021-02-01 04:33:17 +00:00
terence tsao
f5f1284cef Remove unused feature flag config fields (#8363)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-30 15:16:31 +00:00
terence tsao
609418ecd3 Update README.md (#8366)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-30 14:28:27 +00:00
Radosław Kapka
f20c9122e8 Remove unused channel from Powchain Service (#8368)
* Remove unused header channel

* test fix

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-01-30 13:33:52 +00:00
Ivan Martinez
afc3b3168a Add Back Accounts Backup Validator RPC Functionality (#8367)
* Add Back Accounts Backup Validator RPC Functionality

This reverts commit a39db494eb.

* Fix
2021-01-29 18:30:44 -06:00
Victor Farazdagi
902c30e389 Update bitfield dependency (#8364) 2021-01-29 19:58:26 +00:00
terence tsao
3aaa98decf Next slot state caching (#8357)
* Add and use trailing slot state cache

* Remove comment

* Update comments

* Update comments

* Move it to background

* Warn is more appropiate

* Raul's comment

* Same strategy for proposer

* Own helper

* Minor touchups

* Tests

* Feature flag

* Gaz
2021-01-29 16:52:43 +00:00
Ivan Martinez
e592cd7a80 Add wallet file flag to edit-config command (#8360) 2021-01-29 09:32:15 +00:00
Nishant Das
b74dd967af Bind Discovery Using Neutral Network Version (#8359)
* fix it

* terence's review
2021-01-29 15:07:32 +08:00
Victor Farazdagi
d254f24a23 Update bitfield dependency (#8356) 2021-01-29 00:39:29 +00:00
Radosław Kapka
8d505e06bd Implement GetForkSchedule in the config API (#8345)
* span

* update ethereumapis

* implementation + tests

* typo

* fix variable shadowing

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-01-28 13:44:23 -06:00
Victor Farazdagi
09b1e06885 Max-cover: optimized implementation based on Bitlist64 (#8352)
* Max-cover: optimized implementation based on Bitlist64

* gazelle

* re-arrange overlaps check

* minor comments

* add Bitlists64WithMultipleBitSet

* update benchmarks

* gazelle

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-28 15:55:02 +00:00
Radosław Kapka
d9c451d547 Introduce helper function for adding context metadata from gRPC headers (#8354)
* implement helper

* use helper function everywhere

* add unit tests

* small cleanup of the helper

* small fixes

* gazellelelele

* fix helper tests
2021-01-28 08:58:32 -06:00
Radosław Kapka
e677b19d31 Implement GetSpec in the config API (#8328)
* Implement GetSpec in the config API

* extract map creation

* fix deepsource compliant

* Revert "fix deepsource compliant"

This reverts commit 0c99310cc9.

* use struct tags for api map keys

* remove log

* use spec tag

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-01-28 00:30:48 +00:00
terence tsao
2bf03d5cba Use strings builder for attester lock key (#8349)
* Use strings builder

* Handle error
2021-01-27 22:33:28 +00:00
terence tsao
0753636159 Add role based lock for validator (#8347) 2021-01-27 20:59:21 +00:00
terence tsao
b8037b0b50 Wait for one third before grabbing the lock (#8348) 2021-01-27 12:06:18 -06:00
Tushar Shah
2f063d0ddc process grpc header flag for exit (#8334)
* process grpc header flag for exit

* ran bazel //:gazelle

* Add regression test

* Re-run "bazel run //:gazelle"

* whitespace

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* gofmt

Co-authored-by: Danny Joyce <djoyce@bisontrails.co>
Co-authored-by: Danny Joyce <djoyahoy@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-01-27 13:12:48 +00:00
terence tsao
1cfae7e098 More blockchain pkg tests (#8343)
* Add more blockchain pkg tests

* Deepsource feedback

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-01-26 20:27:34 -06:00
terence tsao
91fe32a3d1 Handle subscriber error (#8341) 2021-01-26 13:56:24 -06:00
terence tsao
c0fda583e7 Update seen att slashing cache to use map (#8332)
* Hide beacon operation field if it's 0

* Use map as seen slashing cache

* Revert "Hide beacon operation field if it's 0"

This reverts commit 896fa11a0b.

* Rm unused code

* Gaz

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-01-26 17:24:34 +00:00
Nishant Das
9f62405a81 Update Prombolt Collector (#8339)
* update

* block big buckets

* add more buckets

* patch go.mod

* clean up

* Update beacon-chain/db/kv/kv.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-01-26 15:32:46 +00:00
terence tsao
d121b19145 Rename NewService to New (#8337)
* Hide beacon operation field if it's 0

* Rename NewSerivce to New

* Revert "Hide beacon operation field if it's 0"

This reverts commit 896fa11a0b.

* Fix NewServiceRegistry

* Update slasher/detection/service.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-26 10:26:57 +00:00
Radosław Kapka
a7345c1094 Implement GetDepositContract in the config API (#8316)
* Add funnction to retrieve deposit contract's address from powchain

* change bytes.Equal to assert.DeepEqual

* add DepositContractAddress to mocks

* Extract powchain info to a separate struct

* span

* Revert "Extract powchain info to a separate struct"

This reverts commit e01dd5222b.

* implementation + test

* use the correct hexutil library

* read contract address from configuration

* return ETH1 chain ID instead of fork version

* gzl (I hate you)

* remove unused ChainInfoFetcher

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-01-26 09:37:22 +00:00
Victor Farazdagi
db79481c21 Update bitfield dependency (#8338) 2021-01-26 08:35:16 +00:00
Ivan Martinez
8d986bd414 Change sszutil DeepEqual to ignore unexported (#8336) 2021-01-25 17:52:41 -06:00
Radosław Kapka
c827672a30 Rename non-generated files ending with '_mock' (#8317)
* rename *_mock files

* bzl

* rename faulty_mock_powchain

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-25 22:30:55 +00:00
terence tsao
d5ec248691 Rename getter functions to be idiomatic (#8320)
* Rename getter functions

* Rename new

* Radek's feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-25 21:27:30 +00:00
Nishant Das
b2d6012371 Fix IPV6 binding for Beacon Node (#8326)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-25 20:10:41 +00:00
Radosław Kapka
d21365b882 Remove test assertion that reads a package-level variable (#8329)
* remove checking value of shared variable

* fix deepsource compliant

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-25 18:27:59 +00:00
Nishant Das
1c43ea9e69 Update the Toolchain Registration (#8322)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-25 16:46:27 +00:00
Radosław Kapka
fc8dc21aa2 Cancel node context after StopAll (#8289)
* cancel node context after StopAll

* build fix

* cancel in validator

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-01-25 09:57:21 -06:00
Nishant Das
7f5ffb7dd1 Fix Deadlock in Runtime (#8321)
* fix deadlock

* fix

* Satisfy Deepsource

* fmt
2021-01-25 11:01:55 +08:00
Raul Jordan
92932ae58e [Feature] - Slashing Interchange Support (#8024)
* Change LowestSignedProposal to Also Return a Boolean for Slashing Protection (#8020)

* amend to use bools

* ineff assign

* comment

* Update `LowestSignedTargetEpoch` to include exists (#8004)

* Replace highest with lowerest

* Update validator/db/kv/attestation_history_v2.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/db/kv/attestation_history_v2.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Invert equality for saveLowestSourceTargetToDB

* Add eip checks to ensure epochs cant be lower than db ones

* Should be less than equal to

* Check if epoch exists in DB getters

* Revert run time checks

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Export Attesting History for Slashing Interchange Standard (#8027)

* added in att history checks

* logic for export

* export return nil

* test for export atts

* round trip passes first try!

* rem println

* fix up tests

* pass test

* Validate Proposers Are Not Slashable With Regard to Data Within Slasher Interchange JSON (#8031)

* filter slashable blocks and atts in same json stub

* add filter blocks func

* add test for filtering out the bad public keys

* Export Slashing Protection History Via CLI (#8040)

* include cli entrypoint for history exports

* builds properly

* test to confirm we export the data as expected

* abstract helpers properly

* full test suite

* gaz

* better errors

* marshal ident

* Add the additional eip-3076 attestation checks (#7966)

* Replace highest with lowerest

* Update validator/db/kv/attestation_history_v2.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/db/kv/attestation_history_v2.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Invert equality for saveLowestSourceTargetToDB

* Add eip checks to ensure epochs cant be lower than db ones

* Should be less than equal to

* Check if epoch exists in DB getters

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Add EIP-3076 Invariants for Proposer Slashing Protection (#8067)

* add invariant for proposer protection

* write different test cases

* pass tests

* Add EIP-3076 Interchange JSON CLI command to validator (#7880)

* Import JSON CLI

* CLI impotr

* f

* Begin adding new commands in slashing protection

* Move testing helpers to separate packae

* Add command for importing slashing protection JSONs

* fix import cycle

* fix test

* Undo cleaning changes

* Improvements

* Add better prompts

* Fix prompt

* Fix

* Fix

* Fix

* Fix conflict

* Fix

* Fixes

* Fixes

* Fix exported func

* test func

* Fixes

* fix test

* simplify import and standardize with export

* add round trip test

* true integration test works

* fix up comments

* logrus

* better error

* fix build

* build fix

* Update validator/slashing-protection/cli_export.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update validator/slashing-protection/cli_import.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* fmt

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Filter Slashable Attester Public Keys in Slashing Interchange Import (#8051)

* filter slashable attesters from the same JSON

* builds

* fix up initially broken test

* circular dep

* import fix

* giz

* added in attesting history package

* add test for filter slashable attester keys

* pass tests

* Save Slashable Keys to Disk in the Validator Client (#8082)

* begin db funcs

* add in test and bucket

* gaz

* rem changes to import

* ineff assign

* add godoc

* Properly Handle Duplicate Public Key Entries in Slashing Interchange Imports (#8089)

* Prevent Blacklisted Public Keys from Slashing Protection Imports from Having Duties at Runtime (#8084)

* tests on update duties

* ensure the slashable public keys are filtered out from update duties via test

* begin test

* attempt test

* rename for better context

* pass tests

* deep source

* ensure tests pass

* Check for Signing Root Mismatch When Submitting Proposals and Importing Proposals in Slashing Interchange (#8085)

* flexible signing root

* add test

* add tests

* fix test

* Preston's comments

* res tests

* ensure we consider the case for minimum proposals

* pass test

* tests passing

* rem unused code

* Set Empty Epochs in Between Attestations as FAR_FUTURE_EPOCH in Attesting History (#8113)

* set target data

* all tests passing

* ineff assign

* signing root

* Add Slashing Interchange, EIP-3076, Spec Tests to Prysm (#7858)

* Add interchange test framework

* add checks for attestations

* Import genesis root if necessary

* flexible signing root

* add test

* Sync

* fix up test build

* only 3 failing tests now

* two failing

* attempting to debug problems in conformity tests

* include latest changes

* protect test in validator/client passing

* pass tests

* imports

* spec tests passing with bazel

* gh archive link to spectests using tar.gz suffix

* rev

* rev more comment changes

* fix sha

* godoc

* add back save

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Implement Migration for Unattested Epochs in Attesting History Database (#8121)

* migrate attesting history backbone done

* begin migration logic

* implement migration logic

* migration test

* add test

* migration logic

* bazel

* migration to its own file

* Handle empty blocks and attestations in interchange json and sort interchange json by public key (#8132)

* Handle empty blocks and attestations in interchange json

* add test

* sort json

* easier empty arrays

* pass test

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* builds

* more tests finally build

* Align Slashing Interchange With Optimized Slashing Protection (#8268)

* attestation history should account for multiple targets per source

* attempt at some fixes

* attempt some test fixes

* experimenting with sorting

* only one more failing test

* tests now pass

* slash protect tests passing

* only few tests now failing

* only spec tests failing now

* spec tests passing

* all tests passing

* helper function for verifying double votes

* use helper

* gaz

* deep source

* tests fixed

* expect specific number of times for domain data calls

* final comments

* Batch Save Imported EIP-3076 Attestations (#8304)

* optimize save

* test added

* add test for sad path

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* revert bad find replace

* add comment to db func

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Shay Zluf <thezluf@gmail.com>
2021-01-22 17:12:22 -06:00
Radosław Kapka
8ffb95bd9d Allow dynamic key reloading when having inactive keys (imported & derived) (#8119)
* restart waiting for activation on key change

* test fixes

* wiat for activation comments

* regression test

* log fatal when validator cast fails

* derived keymanager

* review comments

* add buffer to channel

* simplify key refetch logic

* reload keys into empty wallet

* removed warning on wallet creation

* add empty line

* export AccountsKeystoreRepresentation type

* unit test for handleAccountsChanged

* test ctx cancellation

* add missing mockRemoteKeymanager interface function

* gazelle

* gzl

* fix panic inside goroutine during runner tests

* rename error message variables

* Update validator/accounts/accounts_list_test.go

* reorder imports

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-01-22 20:21:34 +00:00
terence tsao
229abed848 Test for processAttestations (#8312)
* Add a test for processAttestations

* Log should not contrain could not process attestation...

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-01-22 18:08:16 +00:00
terence tsao
c5e9b1ec9e Refactor verifyAttestationIndices method (#8309)
* Refactor verifyAttestationIndices

* Conflict

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-01-22 17:15:49 +00:00
Radosław Kapka
7842fd9da6 Replace bytes.Equal with assert.DeepEqual in tests (#8318)
* beacon chain

* format imports

* Update beacon-chain/db/kv/migration_archived_index_test.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

* Update beacon-chain/db/kv/migration_block_slot_index_test.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

* remove unused imports

* Update beacon-chain/core/state/skip_slot_cache_test.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

* Update beacon-chain/core/state/skip_slot_cache_test.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

* Update beacon-chain/core/state/skip_slot_cache_test.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

* Update beacon-chain/core/state/skip_slot_cache_test.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

* Update beacon-chain/db/kv/migration_archived_index_test.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

* Update beacon-chain/db/kv/migration_block_slot_index_test.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2021-01-22 15:15:40 +00:00
Victor Farazdagi
75fc3b045b Max-cover: minor optimizations (#8311)
* Max-cover: minor optimizations

* fix test
2021-01-22 20:10:50 +08:00
Nishant Das
50e5b1b4f5 fix deposit index metrics (#8315) 2021-01-22 19:11:02 +08:00
Preston Van Loon
33e266388f GetDuties: Refactor assignment status to deduplicate status computation (#8313) 2021-01-21 14:22:05 -08:00
terence tsao
2586be29ac Refactor on_attestation return signature (#8310)
* Update func on_attestation's return signature

* Add return
2021-01-21 18:38:51 +00:00
Radosław Kapka
9cc1438ea9 Peer count node api (#8306)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-01-21 16:16:56 +00:00
Victor Farazdagi
6e643aca12 Attestation aggregation: baseline benchmark (#8308)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-21 14:29:17 +00:00
Nishant Das
cc5a847eeb update to v1.15.7 (#8307) 2021-01-21 21:27:57 +08:00
Raul Jordan
241322a7c1 Safe Math for ETH1 Batch Headers Request (#8253)
* batch request underflow checks

* Update beacon-chain/powchain/service.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* return error

* Update beacon-chain/powchain/service.go

Co-authored-by: Nishant Das <nishdas93@gmail.com>

* test

* Apply suggestions from code review

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-21 09:19:44 +00:00
Victor Farazdagi
d697b0b2e5 Update bitfield dependency (#8305) 2021-01-21 08:28:57 +00:00
Preston Van Loon
9ec4f727c6 Remove unnecessary state copy (#8303) 2021-01-20 20:00:01 -06:00
terence tsao
ffcadcf184 Rm forkchoice attestation verification (#8301)
* Rm fork choice attestation sig verification

* Unit test

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-20 21:46:13 +00:00
Ivan Martinez
7c59615ae2 Remove go-ssz entirely from prysm (#8257)
* Add DepositSigningData

* gaz

* Add to ssz tests

* Rename to DepositMessage

* Remove deprecated comment

* Remove return

* Fixes from review

* Fixes

* Remove some of gossz

* Remove go-ssz entirely

* Remove unneeded file

* Fix runtime with ssztypes

* Add back ssz files

* Fix formatting

* tidy

* Remove go-ssz from static

* tidy again

* Add tests

* Change to sig

* Fix test

* fx

* Fix visiblity

* Revert "Remove unneeded file"

This reverts commit d66fcda929.

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-20 21:03:46 +00:00
Preston Van Loon
d6cccc18c3 verifyDepositDataWithDomain: Use helpers.ComputeSigningRoot (#8302) 2021-01-20 20:30:58 +00:00
terence tsao
befe8d88b8 Remove duplicated target root check for fork choice attestation (#8277)
* Rm redundant target root check for fork choice attestation

* Comments

* Revert back verifyBeaconBlock

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-01-20 18:27:10 +00:00
Radosław Kapka
153737803a Implement ListPeers in the node API (#8288)
* add span

* update ethereumapis

* align server with ethereumapis

* benchmark

* naive implementation

* rename two status functions

* new Inbound and Outbound status functions

* tests

* 'enr:' prefix

* refactoring

* gzl

* case when one filter is empty

* empty filter condition fix

* deepsource

* rename getPeer to peerInfo

* bring back correct version of ethereumapis

* go mod tidy
2021-01-20 15:52:33 +00:00
Radosław Kapka
4b14fa4317 Better receiver names in validator and slasher modules (#8296)
* validator

* slasher

* rename db to s for store

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-20 14:39:07 +00:00
Victor Farazdagi
27847ee2fe Update bitfield dependency (#8299) 2021-01-20 13:39:54 +00:00
terence tsao
578dabe27c Remove duplicated LMD FFG check for fork choice attestation (#8276)
* Rm redundant lmd<->ffg check for fork choice attestation

* Comments

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-20 10:19:43 +00:00
Victor Farazdagi
ed2c0a3e5e Update bitfield dependency (#8295) 2021-01-20 06:55:53 +00:00
terence tsao
5d841874f7 Return error on AttestingIndices bitfield length check (#8285)
* Return error on committee len check

* AttestingIndices unit test

* Add equality check

* Fixing more tests

* Fix rest of the tests

* Fix invalid bit length test

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-20 03:00:52 +00:00
terence tsao
ab9d596a5f Update fastssz (#8291)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-19 23:48:22 +00:00
terence tsao
084e5bd020 Add test TestStore_OnAttestation_Ok (#8293)
* Add test TestStore_OnAttestation_Ok

* Handle errors

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-19 23:10:10 +00:00
terence tsao
bc2c206832 Add verify slot target epoch function and apply all (#8273)
* Add verify slot target epoch function and apply all

* Fix TestProcessAttestationsNoVerify_IncorrectSlotTargetEpoch

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-01-19 21:41:44 +00:00
Preston Van Loon
655a7e98c3 Stamp binaries by default (#8292) 2021-01-19 20:43:30 +00:00
Radosław Kapka
f89fd67952 Better receiver names in beacon chain module (#8286) 2021-01-19 13:21:32 +00:00
Nishant Das
20b836d038 Add Initial Support For Gossip Scoring Service (#8275)
* checkpoint progress

* gaz

* fix

* add it in

* Update beacon-chain/p2p/pubsub.go

* fmt

* reformat imports

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2021-01-19 12:13:08 +00:00
Preston Van Loon
b33a8eb59f Revert "Use remote cache in CI" (#8281)
This reverts commit 8b6abcbf0c.
2021-01-19 01:55:00 +00:00
Preston Van Loon
8b6abcbf0c Use remote cache in CI (#8280)
* Use remote cache in CI

* Update .buildkite-bazelrc

* Update .buildkite-bazelrc
2021-01-18 22:22:37 +00:00
Raul Jordan
9b367b36fc add happy/sad tests (#8279) 2021-01-18 11:32:17 -08:00
pinglamb
09a792ded4 isCanonical for slot 0 should return true (#8269)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-01-16 16:14:14 +00:00
terence tsao
cf343be76a Add a helper to validate nil attestation (#8272)
* Add verify nil attestation function and apply all

* Remove invalid attestatione debug log

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-01-15 22:19:17 +00:00
Raul Jordan
4c19e622cd Implement Migration Up/Down Logic for Validator DB (#8271)
* rollback logic

* implement up down logic

* begin down migration tests

* rollback works

* unset test

* remove iface

* gaz

* add comment

* fix ineff assign

* preston comment

* add progress
2021-01-15 15:35:21 -06:00
pinglamb
d7d2c6354b Blocks filtering should return genesis when startSlot=0 and endSlot=0 (#8270)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-01-15 18:36:17 +00:00
terence tsao
b6c4bc197f Add on_block finality test (#8266)
* Can generate at epoch boundary

* Remove extra space

* Add on block finality test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-15 13:42:49 +00:00
Nishant Das
ce397ce797 Prune Excess Peers Better (#8260)
* add method

* add changes

* formatting

* choose

* fix waitgroup

* add

* add debug logs

* gaz

* make it better

* fix

* godoc

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-14 21:28:20 +00:00
Radosław Kapka
2d75b12791 Implement GetIdentity in the node API (#8230)
* initial implementation

* register metadata provider

* final implementation

* tests

* fixed imports

* gazelle

* code review

* small cleanup

* change errors.Wrap to status.Errorf

* gazelle + goimports

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-14 20:48:40 +00:00
terence tsao
40155c9828 Testutil: Can generate attestation at epoch boundary (#8265) 2021-01-14 10:22:53 -08:00
Radosław Kapka
a2d4e3302c Implement GetPeer in the node API (#8264)
* tests

* capitalize error message
2021-01-14 10:26:16 -06:00
Nishant Das
612e6ebdc4 gogo protobuf (#8263) 2021-01-14 17:17:26 +08:00
Raul Jordan
fff6472a04 Allow Multiple Targets Per Source Epoch in Attester Protection (#8262) 2021-01-13 23:23:29 +00:00
terence tsao
daf6da5beb Add hydrate indexed att test helper (#8261)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-01-13 22:05:57 +00:00
pinglamb
9369bb6781 Copied over ssz.DeepEqual as sszutil.DeepEqual from go-ssz (#8258)
* Copied over ssz.DeepEqual as sszutil.DeepEqual from go-ssz

* Added test cases for DeepEqual

* Remove commented code
2021-01-13 20:40:56 +00:00
terence tsao
eeda9f18fe Disallow duties request where req.Epoch > current.Epoch+1 (#8252)
* Disallow request epoch to be out of bound

* Input Slot and epoch checks

* Tests

* Review feedbacks

* Unavailable error code

* Rename genesis time fetcher to time fetcher

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-12 23:27:37 +00:00
Preston Van Loon
e967a65b68 Update prometheus/client_golang (#8256) 2021-01-12 22:56:57 +00:00
Preston Van Loon
c87ef2f0e7 Coverage results upload fix for CI (#8255)
* Check in gocovmerge, add all-in-one coverage.sh script for CI

* annotate script

* Annotate tools/gocovmerge/main.go

* deepsource suggestions
2021-01-12 20:12:09 +00:00
pinglamb
1a9207ba46 Removed some simple go-ssz usage (#8250)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-01-12 19:31:34 +00:00
pinglamb
9f423617cb Added db.BlocksBySlot and db.BlockRootsBySlot (#8184)
* Added blockBySlot and blockRootBySlot

* Changed to BlocksBySlot and BlockRootsBySlot

* Updated to use BlocksBySlot and BlockRootsBySlot

* Added missing passthrough to karfa exporter

* Return hasBlocks/hasBlockRoots in the new getters

* Fixed CI lint

* Replace call to bytes.Compare with bytes.Equal

* Reordered the returns of the new getters

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-01-12 18:31:15 +00:00
Radosław Kapka
015102c2d5 Implement GetSyncStatus in the node API (#8241)
* Implement GetSyncStatus in the node API

* gazelle

* add HeadSlot function to fuzz tests' fakeChecker

* use HeadFetcher to get head slot

* remove useless code

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-12 17:17:20 +00:00
Radosław Kapka
aa69e5edcc Allow to create an empty imported wallet (#8251)
* reload keys into empty wallet

# Conflicts:
#	validator/accounts/accounts.go

* removed warning on wallet creation

* export AccountsKeystoreRepresentation type

* rename error message variable

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-12 16:52:01 +00:00
Radosław Kapka
5dda2ca328 Replace HTTP code with gRPC code in GetHealth API endpoint (#8249)
* Replace HTTP code with gRPC code in GetHealth API endpoint

* gazelle

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-12 16:20:20 +00:00
Raul Jordan
470d5aa491 Cleanup Attester DB Protection Files and Add Back Save Lowest Epochs Functionality (#8232)
* cleanup for att protection

* rename to deprecated

* rem old item

* imports
2021-01-12 09:32:13 -06:00
Raul Jordan
d2bd954a6c Remove Snappy from Validator Slashing Protection DB (#8248)
* no snappy

* Update validator/db/kv/migration_optimal_attester_protection.go

* fmt

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-12 01:10:11 +00:00
Ivan Martinez
e5556db49d Add DepositMessage in preparation to remove go-ssz (#8244)
* Add DepositSigningData

* gaz

* Add to ssz tests

* Rename to DepositMessage

* Remove deprecated comment

* Remove return

* Fixes from review

* Fixes

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-12 00:45:11 +00:00
Raul Jordan
d97596348e Add Back New Attester Protection DB Logic (#8242)
* Revert "Revert New Attester Protection DB Logic (#8237)"

This reverts commit 6738fa3493.

* Batch Attestation Records and Flush All at Once in Validator DB (#8243)

* begin flushing logic

* finalize logic before starting tests

* make code DRY

* better log fields

* gaz

* tweak parameter

* rename

* clarifying comment on error handling in event feed

* comprehensive tests

* more comments

* explain parameters in comments

* renamed consts

* Apply suggestions from code review

* gaz

* simplify

* typo

* comments
2021-01-11 23:59:17 +00:00
terence tsao
323eac6d6c Remove timeout debug log (#8247)
* Disallow request epoch to be out of bound

* Remove debug log

* Revert "Disallow request epoch to be out of bound"

This reverts commit d2dc7db594.
2021-01-11 22:23:13 +00:00
Radosław Kapka
5fd03f8fb0 Unify the pattern of using a package-level logger (#8245)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-01-11 20:03:28 +00:00
terence tsao
18bb86754a Hydrate signed block helper (#8246)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-01-11 19:27:30 +00:00
Preston Van Loon
97320a0a8e Remove snappy compression migration for attestation history (#8238)
* Remove snappy compression migration for attestation history

* gofmt

* remove more snappy stuff

* revert validator/db/kv/historical_attestations.go
2021-01-11 09:28:34 -08:00
terence tsao
9a1866b735 Hydrate header test helper (#8234)
* Hydrate headers and fix tests

* Gazelle

* Fix a slashing test

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-01-09 16:45:45 +00:00
Raul Jordan
6738fa3493 Revert New Attester Protection DB Logic (#8237)
* Revert "Optimize Migration for new Attester Protection DB (#8231)"

This reverts commit c4ab67832f.

* Revert "Integrate New Slashing Protection DB Methods at Runtime (#8219)"

This reverts commit 3858068201.

* Revert "DB Migration for Optimal Local Slashing Protection (#8212)"

This reverts commit dd3ac6c2ed.
2021-01-09 04:00:56 +00:00
terence tsao
35ed01e36c Add timely attest flag to fix #8185 (#8235) 2021-01-08 17:54:46 -08:00
Raul Jordan
c4ab67832f Optimize Migration for new Attester Protection DB (#8231)
* migrate using tx commit

* add improvement

* fixed up test

* rem fmt

* gaz

Co-authored-by: nisdas <nishdas93@gmail.com>
2021-01-08 17:31:54 +00:00
Nishant Das
0ff2a53b2f Add Peer Logger (#8226)
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2021-01-08 11:08:11 +00:00
Preston Van Loon
25bba9f43f Fuzzing: conditional BLS enabled via environment variable (#8229)
* Read whether or not to enable BLS via environment variable

* Read whether or not to enable BLS via environment variable
2021-01-08 05:31:24 +00:00
Raul Jordan
3858068201 Integrate New Slashing Protection DB Methods at Runtime (#8219)
* integrate at runtime and revamp tests

* historical att

* Update validator/db/iface/interface.go

* deepsource

* import

* log the slashing kind

* gaz

* create a slashutils

* integrate new slashutil

* imports
2021-01-07 23:30:25 +00:00
Raul Jordan
0d5e2cfb27 Prune Validator Attester Records Older than Weak Subjectivity Periods (#8221)
* pruning and begin test

* comprehensive pruning tests

* add pruning on startup

* also prune source epochs bucket

* more testing

* greatly simplify pruning function

* pruning logic and comprehensive tests in

* att protection test

* gaz

* fix sneaky change

* rev

* documented and tested helper func
2021-01-07 15:40:37 -06:00
terence tsao
bc650c82b4 Hydrate attestation for tests (#8228) 2021-01-07 21:00:21 +00:00
Nishant Das
a855f282c6 Fix to TLS v1.3 (#8222)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-07 19:56:51 +00:00
Raul Jordan
4253888a36 Update Bitfield Dependency (#8227)
* bitfield updates

* tidy

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-07 18:12:49 +00:00
Nishant Das
fb9f4e828d Update FastSSZ To Latest Commit (#8225)
* update fast-ssz

* fix

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-07 17:08:41 +00:00
terence tsao
9ff825a570 Update variable names that are same as imports (#8220)
* Fix names that are same as import name

* Review feedbacks
2021-01-07 10:42:03 -06:00
Victor Farazdagi
d20065218c Max-cover: remove redundant dedup routine (#8224)
* Attestation aggregration: remove redundant dedup routine

* fix func call

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-07 14:18:29 +00:00
Radosław Kapka
353c1f6387 Multiple discovery addresses (#8203)
* multiple discovery addresses

* ipv6 fix

* make len test more robust

* create enr node for testing

* use local node for test

* use mockListener

* remove unused type alias
2021-01-07 13:35:42 +00:00
pinglamb
1b6a0703e3 Show number of user pubkeys for attesting and proposing in "Next duty" log output (#8187)
* Show attesting and proposing counts in "Next duty" log output

* Use map to store all the counts instead of magic reset

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2021-01-07 10:22:38 +00:00
pinglamb
9135774720 Block Fetching with Graceful Retry (#8182)
* Added tooMuchDataRequestedError func

* Added multiplicative decrease and additive increase

* Code review changes

* Renaming the new constants

* Return a different error in graceful retry

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-01-07 06:21:44 +00:00
Potuz
e52c3d48cf Start from last valid root in loadStateBySlot (#8218)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-01-06 18:41:44 -06:00
terence tsao
ba9b563e6e Fix Attest early if valid block is received before 4 seconds (#8197)
* Better int -> string conversion

* First take

* More tests

* Gazelle

* Remove validator subpackage visiblity

* Remove span

* Update validator/client/attest.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update validator/client/attest.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update stream blocks call to use verified only

* Rename to waitOneThirdOrValidBlock

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-06 21:10:26 +00:00
Raul Jordan
dd3ac6c2ed DB Migration for Optimal Local Slashing Protection (#8212)
* begin migration logic

* wrote migration logic

* begin test file

* test for migration working

* gaz

* progressutil

* migration works even if partial data was written
2021-01-06 20:41:31 +00:00
Radosław Kapka
9b3e1eb643 Implement GetHealth in the node API (#8217)
* Implement GetHealth in the node API

* repair fuzz mock

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-01-06 20:11:20 +00:00
Victor Farazdagi
da59fdd22b Attestation Aggregation: optimize attestation list validation (#8213)
* optimize candidate validation on att aggregation

* restore test

* fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-06 19:26:19 +00:00
Shay Zluf
f014374de2 Move attestation protection call after signing (#8216)
* Move validator protection after signing

* fix tests

* remove unused code
2021-01-06 11:41:00 -06:00
Radosław Kapka
392e61fbee Revert changes to sync service status check (#8215) 2021-01-06 11:19:53 +00:00
Radosław Kapka
7135a8542f Check if initial sync service has been initialized (#8214) 2021-01-06 10:45:22 +00:00
Raul Jordan
c354871762 Optimal Local Slashing Protection DB Schema (#8211)
* add new approach for slashing protection

* benches

* tests passing

* gaz

* all tests passing

* comment
2021-01-06 03:04:46 +00:00
Radosław Kapka
bc2cd29d4b Implement GetVersion in the node API (#8207)
* initial implementation

* implement GetVersion

* remove implementation of GetIdentity

* remove MetadataProvider from server

* gzl
2021-01-05 21:06:51 +00:00
terence tsao
023e258f6a Stream verified block (#8206) 2021-01-05 12:40:11 -08:00
Victor Farazdagi
9d737d60f4 Declare err in loop to limit its scope (#8200)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-05 13:55:23 +00:00
Victor Farazdagi
1abe92fd8b Remove redundant parentheses around nil values (#8199)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-05 13:29:46 +00:00
Victor Farazdagi
e5c69bd387 Add space at the start of comments (#8202)
* add space at the start of comments

* undo case change
2021-01-05 13:09:41 +00:00
Victor Farazdagi
318f83957a Fix data races in tests accessing beacon config concurrently (#8190)
* add mutex to params/config

* split config files into test/prod

* add tags checker

* add regression test

* remove debug info

* update bazel config

* go fmt

* make sure that conditional file is kept by gazelle

* update build tag: test -> develop

* gazelle

* remove redundant import

* fix data race in TestService_ReceiveBlock
2021-01-05 04:51:25 +00:00
Raul Jordan
f67f8dd6df Configurable Interval for Debouncing Keystore Reloading in Validator Client (#8149)
* configurable keystores debounce interval

* use a time duration flag instead

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-05 00:52:12 +00:00
Jeff Widmer
0e5da504f4 Fix for prysm.bat issue 'Validator' is not recognized as an internal or external command (#8097)
Fix for the issue where if either the beacon, validator, or slasher are already up to date, then the message:
"'Validator/Beacon/Slasher' is not recognized as an internal or external command, operable program or batch file."
The issue was that the message  "Validator/Beacon/Slasher is up to date." is not echoed to the console but it is trying to be run as a command.

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-01-04 23:27:38 +00:00
Cipio
f6af79f415 Fix file permission checks for Windows (#8164)
* Does bitwise compare for MKDIR

* Implemented a more global solution to windows permission issues

* Remove unneeded line I added earlier.

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-01-04 22:48:42 +00:00
Radosław Kapka
af2c36ec40 Add names to certain return values in accounts (#8159)
* add names to certain return values

* remove redundant

* rename selected public keys

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-04 21:44:15 +00:00
terence tsao
5dc8eb45d3 Add slot in epoch field for `Attestation schedule..." log (#8166)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-04 21:10:32 +00:00
Victor Farazdagi
b54743edbf Add mutex to params/config (#8160)
* add mutex to params/config

* split config files into test/prod

* add tags checker

* add regression test

* remove debug info

* update bazel config

* go fmt

* make sure that conditional file is kept by gazelle

* update build tag: test -> develop

* gazelle

* remove redundant import

* update deps.md (per Nishant's suggestion)

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-04 20:48:39 +00:00
Victor Farazdagi
04b2e0776d Fix import shadowing of state package (#8191)
* update shadowed var name

* update var names

* remove unnecessary delta

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-04 20:07:12 +00:00
Shay Zluf
70da296a3b Bring back disable slashing broadcast flag (#8141)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-04 19:36:17 +00:00
Mohamed Mansour
2defff0886 Add new GetVersion API to Health Service (#8167)
* Add new GetVersion API to Health Service

This is to support showing version information in the web ui.
Since health.go is built through validator bazel, we can use
`shared.GetVersion` directly.

Backend for: prysmaticlabs/prysm-web-ui#107

* Run goimports and update-go-pbs

* go mod tidy

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-01-04 18:51:52 +00:00
Victor Farazdagi
ee8aacbbbf Remove redundant lambda around cleanup code (#8192)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-04 18:16:18 +00:00
terence tsao
4055841952 Update int -> string conversion to make go test happy (#8183)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-04 17:53:59 +00:00
terence tsao
bf673ecb12 Validator nil duty should not panic (#8171)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-04 17:28:00 +00:00
Victor Farazdagi
7c25d5c852 Compare strings using strings.EqualFold() (#8193) 2021-01-04 16:46:22 +00:00
Raul Jordan
4c6e0c5f46 Update Prysm Web UI to Beta.3 (#8163)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-18 23:25:48 +00:00
Raul Jordan
768994550c Fix Unknown Validator Edge Case in ListValidatorBalances (#8162)
* ensures unknown validators do not mess up rest of api response

* rem old test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-18 22:51:44 +00:00
Victor Farazdagi
f038d782c2 Fix issue with custom chain/network ID configuration (#8147)
* move chain/network id to beacon config

* go fmt

* improve tests

* validator params

* update deposit contract address

* complete tests

* re-arrange comments

* Less mis-leading comment, per Terence's review

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-12-18 22:22:48 +00:00
terence tsao
ff64fdcfb5 Flag to enable duty count down (#8161)
* Add a flag to enable count down

* Add tests
2020-12-18 21:12:57 +00:00
Victor Farazdagi
d8c31b79df Fix import shadowing of db package (#8158)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-18 19:12:30 +00:00
Nishant Das
ea88799585 Revert Inbound Peer Limit (#8155)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-18 18:33:31 +00:00
Raul Jordan
72dc43989f Stream Validator and Beacon Logs via gRPC Streams (#8150)
* implement validator logs stream

* fix test

* tidy

* proto regen

* add logs stream to the beacon node

* beacon logs working

* impl

* pass test

* gaz

* rem lock

* fix space
2020-12-18 18:03:24 +00:00
Victor Farazdagi
e772e8c8c2 Init sync: minor style fixes (#8156)
* simplify FSM calls

* improve calculateHeadAndTargetEpochs API

* simplify

* simplify nil assignment
2020-12-18 14:31:15 +00:00
terence tsao
df93affb4e Move hot state cache to stategen (#8153)
* Move hot state cache to stategen

* Fix build.bazel

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-12-17 20:40:47 +00:00
Raul Jordan
d19c57cdb6 Remove Logout from Validator RPC Authenticated Paths (#8151)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-17 19:46:27 +00:00
terence tsao
756ccbe5e4 Update Attestation schedule... log for clarify (#8148)
* Don't use fraction, use different fields

* Update validator/client/validator.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Go fmt

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-17 19:12:41 +00:00
Nishant Das
46c67f1e9e Fallback To Historical Sync For Powchain (#8146)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-17 18:49:22 +00:00
Nishant Das
44c3adb367 Add Public Method To Retrieve Discovery Address (#8143)
* add method

* fix

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-17 18:03:18 +00:00
Radosław Kapka
25b151ab78 Make TLS mandatory by default when unmarshalling remote wallet options (#8133)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2020-12-17 16:59:27 +00:00
Raul Jordan
f75b8a3be1 Implement Unified Validator API Endpoints for Beacon Chain Information (#8139)
* define all endpoints

* wrapper

* implement required validator endpoints

* begin impl

* implement remaining endpoint

* imports

* add in list validators

* intercepter

* test added
2020-12-17 16:26:32 +00:00
Nishant Das
dfdf77cb95 Add Instead of Subtract in Epoch Boundary Check (#8145)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-12-17 14:31:05 +00:00
Radosław Kapka
d5bf8376c2 Clean up account logs and errors (#8142)
* small fixes in logs and errors

* fix error message in test

* found a period hiding in an error message

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-17 13:37:06 +00:00
Nishant Das
e2d7ec6f97 Refactor Method Signatures For Powchain (#8110)
* checkpoint

* fix tests

* fix visibility

* fix

* victor's review

* remove redundant LF

* remove redundant LF

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-12-17 12:33:34 +00:00
Raul Jordan
d650034734 Lock Only When Needed When Fetching Attesting History (#8140)
* lock when needed in attesting history

* rw
2020-12-17 00:18:38 +00:00
Preston Van Loon
a7cf77fc26 Update rules docker to include https://github.com/bazelbuild/rules_docker/pull/1666, which updates the base images (#8136)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-12-16 14:56:38 -06:00
terence tsao
0dcbf177aa Fix participation query returning balances of orphaned chain (#8137)
* Check if block is canonical

* Add tests and better comments

* Grammar

* Update test for nil state
2020-12-16 13:31:34 -06:00
Nishant Das
82bba593eb Do Not Verify Attestations When Packing Them (#8135)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-16 18:06:49 +00:00
Raul Jordan
148e7fcd59 Consolidate Required Web UI Endpoints Into Single Protobuf File (#8127)
* define required endpoints for unifying backends

* proto definitions

* impl

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-16 17:30:48 +00:00
terence tsao
20dede7532 Move state summary cache to DB (#8101) 2020-12-16 08:56:21 -08:00
Preston Van Loon
3fb49433a1 Limit prometheus requests in flight, set timeout to reasonable 30 seconds (#8130) 2020-12-16 04:57:02 +00:00
Preston Van Loon
4326cbbf08 Validator database: use snappy compression on encoded attestation history (#8129)
* Remove old buckets, rename new buckets to be canonical

* Add migration and test

* Do the migrations

* Gazelle

* Clearify code, code review

* gofmt
2020-12-16 03:33:04 +00:00
terence tsao
dc27cd7a1e Pass context to NewKVStore (#8125)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-15 22:07:01 +00:00
terence tsao
0449cd3450 Allow update cache at last slot of the epoch (#8094)
* Epoch boundary updates at the last slot of the epoch

* Report metric on first slot

* Remove comment

* Add locks to param config

* Remove lock for copy

* Revert "Add locks to param config"

This reverts commit 79d5130b58.

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-12-15 21:19:02 +00:00
terence tsao
6244163770 FC test coverage improvement (#8120)
* Test coverage improvements

* Gazelle

* Update namings

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-12-15 20:44:35 +00:00
Victor Farazdagi
f5c87075f2 Fix issue with roaming data dir on Wins (#8095)
* patch beacon node

* make sure that check is executed before anything else

* fix format

* make sure that check is reusable by other executables

* debug infor

* fix typo

* more debug info

* use copydir

* cleanup

* better explanation

* gazelle

* go fmt

* debug tos permissions

* upadte copydir

* gazelle

* better check of tos acceptance

* expand path

* update validator

* move fixing func

* move fixing method

* make sure that updater works both on main and subcommands

* remove from startnode

* add copydir test

* add DirFiles method and tests

* fix test

* add and test HashDir

* update tests

* fix test

* add datadir removal

* update messages

* further update messages
2020-12-15 14:01:51 -06:00
Preston Van Loon
ad7d3c74cc Validator DB cleanup: remove obsolete buckets (#8122) 2020-12-15 18:24:56 +00:00
Nishant Das
508c5fcf2f More Efficient Validation of Proposer Index (#8107)
* metric

* make it better

* make it better

* gaz

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-12-15 16:27:05 +00:00
Victor Farazdagi
a0c475671c Add extra methods to shared/fileutil (#8117)
* add extra methods to fileutil

* Shay's suggestion

* Update shared/fileutil/fileutil.go

Co-authored-by: Shay Zluf <thezluf@gmail.com>

* Adds reference to the original implementation

Co-authored-by: Shay Zluf <thezluf@gmail.com>
2020-12-15 14:18:15 +00:00
Preston Van Loon
72a92fe708 CI: Disable remote caching of GoStdLib (#8111)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-15 05:58:52 +00:00
terence tsao
6a5589f99e Skip proposer indices cache update if exists (#8096)
* Check if cache is empty before update

* Add tests

* Fix tests

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-15 00:09:30 +00:00
terence tsao
70c0bb106b Beacon node code health improvements (#8109)
* Apply code health fixes after code inspect

* Remove attestationRoot

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-12-14 23:41:24 +00:00
Victor Farazdagi
0f18867f08 Remove duplicate package imports (#8104)
* remove duplicate imports

* remove unused lookupLimit

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-14 22:22:55 +00:00
Cipio
630d57377a Fix Slasher Backup DB panic on call. (#8099) 2020-12-14 21:39:26 +00:00
terence tsao
3e9d721280 Validator code health improvements (#8106) 2020-12-14 21:10:02 +00:00
Steven Allen
2428880058 Update go-libp2p to 0.12.0 (#8015)
* Update go-libp2p to 0.12.0

go-libp2p 0.12.0 made some significant changes to the stream interfaces around
stream closing:

* Close now closes in both directions and frees the stream. However, unlike
FullClose did, it doesn't _wait_ for the remote peer to respond with an EOF.
* To close for writing, call CloseWrite (like one would on a TCP connection, etc.).

This patch:

* Replaces calls to FullClose with Close where appropriate.
* Replaces calls to Close with CloseWrite where appropriate.
* Removes redundant Close calls.
* Calls Reset to where appropriate to indicate that the request/response was
  aborted. Unlike Close, this will not flush and will not cause the remote peer
  to read an EOF. Instead, the remote peer will read an ErrReset error.
* Ensures we always either close or reset streams. Send wasn't closing the
  stream on some error paths.
* Now that stream closing is async, we explicitly wait for a response when
  "hanging up" on a peer (so we don't hang up before they receive our
  response/goodbye message).

* update bazel

* Gazelle

* revert unintentional bazel workspace change

* appease an overzealous linter

* update to latest

* Refactor encoder

* gazelle

* Gazelle

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-12-14 17:22:25 +00:00
Potuz
4d1f01aacc Change block_arrival_latency buckets to exponential format (#8065)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-12-14 16:00:07 +00:00
Victor Farazdagi
b9848dc94f Remove unused vars in tests (#8103)
* update process_block_test

* service_attester_test

* update service and rpc_status tests

* go fmt
2020-12-13 05:23:13 +00:00
Nishant Das
579335f81a Remove Saving Of Target And Source Epoch (#8102)
* remove

* skip
2020-12-13 04:26:18 +00:00
Nishant Das
11bbf06d03 Add Inbound Peer Limit (#7942)
* add changes

* fix up

* fix

* add test

* fix test

* fix again

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-12-12 03:46:55 +00:00
Nishant Das
29804fa572 Refactor Subnet Search (#8048)
* checkpoint progress

* checkpoint

* clean up

* do better

* fix test

* fix

* fix

* preston's review

* fix

* fix

* fix

* add iterator

* go doc

* make it a config parameter

* Apply suggestions from code review

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-12-11 18:40:56 -08:00
Radosław Kapka
4ec396c025 Refetch validating keys if no keys are fetched (#8000)
* refetch validating keys every 30 seconds

* deduplicate error/log messages

* remove redundant break statement

* comment about execution flow

* move code to wait_for_activation.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-12-11 20:55:52 +01:00
Raul Jordan
1fbfd52e52 Simpler and Safer Attester Slashing Protection (#8086)
* att sign validations

* eliminate old cached methods and use a simple approach in the db

* redefined db methods

* db package builds

* add multilock to attest and propose

* gaz

* removed concurrency tests that are no longer relevant

* add cache to db functions for attesting history checks

* passing

* add in feature flag --disable-attesting-history-db-cache

* remove lock

* Revert "remove lock"

This reverts commit b1a65020e4.

* comment

* gaz
2020-12-11 12:31:35 -06:00
Mohamed Mansour
2e18df642d Expand Validator Input Errors (#8090)
The reason of why a validator input was failing was very generic,
this allows the actual implementation to state why it failed by
passing the exception forward.

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-12-11 10:57:47 -06:00
Nishant Das
99b3835f19 Add Fallback Option for Eth1 Nodes (#8062)
* fix

* fix tests and change back

* gaz

* change

* ready again

* Update beacon-chain/flags/base.go

Co-authored-by: Shay Zluf <thezluf@gmail.com>

* radek's review

* Update shared/cmd/helpers.go

* Update shared/cmd/helpers_test.go

* Update shared/cmd/helpers_test.go

* Endpoint/endpoint

Co-authored-by: Shay Zluf <thezluf@gmail.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2020-12-11 19:15:04 +08:00
Victor Farazdagi
46d99fdc00 Attestation aggregation: remove proper set duplicates (#8063)
* fix proper-set issue

* move test into dedicated test file

* move functionality from maxcover -> proposer

* adds dedup to proposer

* fix tests

* update tests

* remove redundant proper subset test

* fix bug with identical expression
2020-12-11 11:13:14 +03:00
Preston Van Loon
923e4d3a5e Attempt to reconnect when waiting for activation (#8057)
* Attempt to reconnect when waiting for activation

* rm fuzz/attestation_fuzz.go

* math.Min, not math.Max

* Gofmt

* resolve RVV-B0003
2020-12-10 19:26:31 -06:00
terence tsao
bb9e2ba12c Validator client logs time left to next duty (#8088) 2020-12-10 10:56:18 -08:00
Preston Van Loon
f44b2a35e4 Update waiting validator methods to be context aware (#8078)
* Update waiting validator methods to be context aware

* Enable debug logging for validator in e2e

* invert logic in deferred function, remove for loop

* return early if wait is 0 or less

* overwrite ctx

* t.Stop() chan closure is hanging. Doesnt make sense, so ignoring that cleanup task. It shouldnt happen at runtime anyway

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-12-10 10:00:48 -06:00
terence tsao
00dacbd00d Remove custom block body root and block root methods (#8069)
* Remove custom block body root and block root methods

* Add nil checks and fix tests

* Fmt

* Typo
2020-12-09 12:11:42 -06:00
Radosław Kapka
92736d0188 warn when creating an imported wallet (#8081) 2020-12-09 16:41:26 +01:00
Preston Van Loon
c96db1a122 Add spans to pre and post att signing updates (#8079)
* Add spans to pre and post att signing updates

* span on db method

* Add span to isNewAttSlashable
2020-12-08 23:47:02 -06:00
terence tsao
c5770a2e56 Add beacon block nil body checks (#8077)
* Add nil checks and fix tests

* Gaz

* Revert one minor typo

* Update test

* Remove extra space
2020-12-08 16:27:58 -06:00
terence tsao
ade3b2f2df Add state field count to config (#8068)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-12-07 21:49:28 -06:00
Radosław Kapka
9d7052796b Restore database CLI command (#8061)
* restore beacon node db

* revert image name

* move restore out of the kv folder

* remove files from kv folder

* go mod tidy

* Remove usage of prometheus testutil

* add yes/no to prompt text

* restore slasher db

* organize imports

* go mod tidy

* restore validator db

* close slasher db

* defer close backup db in tests

* simplify function literal
2020-12-07 21:36:43 +01:00
Shay Zluf
fbbdd94fea Resolve panic on shutdown with offline eth1 node (#8033)
* Resolve panic on shutdown with offline eth1 node

* Move fix tne right place nishant feedback

* fix log message

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2020-12-07 10:16:23 +00:00
Victor Farazdagi
b51aec6981 Init sync: minor fixes (#8060)
* fixes typo

* simplify code
2020-12-07 09:33:06 +00:00
terence tsao
b4437e6cec Load graffiti from file (#8041)
* Pass graffati file and   use it

* Visibility

* Parse test

* More proposal tests

* Gazelle

* Add sequential functionality

* fix length check

* Update priorities. Specified -> random -> default

* Log warn instead return err

* Comment

* E2e test

* Comment

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-12-04 23:15:12 +00:00
Arthur Burkart
8ad328d9b3 fix(grpcHeaders): accept values with "=" symbols (#8047)
* fix(grpcHeaders): accept values with equal signs

# What

Before this commit, it was not possible to pass in base64-encoded
content as a header value if it contained an equals sign. This commit
changes the behavior slightly. Rather than ignore key/value pairs where
the value happens to have an equals sign, we assume the first equals
sign delimits the key from the value and pass in the rest of the value
as-is.

Also, instead of printing the header name along with its value, we
print the name, so there is less risk of leaking information into logs
that shouldn't be there.

# Testing

This has not been tested in the context of the full validator client.
Instead, a small example was made to demonstrate the feasibility. The
example is shown here:

```go
package main

import (
	"log"
	"os"
	"strings"

	"github.com/davecgh/go-spew/spew"
	"github.com/urfave/cli/v2"
)

type Config struct {
	GrpcHeadersFlag string
}

func main() {
	app := &cli.App{
		Action: func(c *cli.Context) error {
			for _, hdr := range strings.Split(c.String("grpc-headers"), ",") {
				if hdr != "" {
					ss := strings.Split(hdr, "=")
					spew.Dump(ss[0])
					spew.Dump(strings.Join(ss[1:], "="))
				}
			}
			return nil
		},
		Flags: []cli.Flag{
			&cli.StringFlag{
				Name: "grpc-headers",
				Usage: "A comma-separated list of key value pairs to pass as gRPC headers for all gRPC " +
					"calls. Example: --grpc-headers=key=value",
			},
		},
	}

	err := app.Run(os.Args)
	if err != nil {
		log.Fatal(err)
	}
}
```

Example invocation:

```command
❯ go run main.go --grpc-headers=key=value,Authorization="Basic $(echo -n hello:world | base64)"
(string) (len=3) "key"
(string) (len=5) "value"
(string) (len=13) "Authorization"
(string) (len=22) "Basic aGVsbG86d29ybGQ="
```

* Adds tests to new gRPC header parsing code

* bazel run //:gazelle
2020-12-04 21:31:15 +00:00
Victor Farazdagi
21ede7634e Attestation aggregation: maxcover vs naive aggregation effectiveness test (#8043)
* maxcover performance test

* gazelle

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-04 17:24:57 +00:00
Victor Farazdagi
57b74283d3 Attestation aggregation: removes redundant code from max-cover benchmarks (#8044)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-04 16:51:31 +00:00
Victor Farazdagi
be078d6a16 Update TestMain(): do not call os.Exit() explicitly (#8046)
* update workspace

* update testmain
2020-12-04 16:10:07 +00:00
Victor Farazdagi
ccba8cfa5a Update rules_go to v0.24.9 and golang to v1.15.6 (#8045) 2020-12-04 15:28:03 +00:00
Victor Farazdagi
afbfaedea4 Unskip fixed init-sync service test (#8042) 2020-12-04 11:57:46 +00:00
terence tsao
3ce96701de Update attestation schedule log with total attester count (#8013)
* Log attesting total

* Use the right library

* Go fmt

* Use fmt

* No space is better

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-04 07:02:58 +00:00
simonatsn
d2ba45aad9 Update blst to v0.3.1 and incorporate subgroup changes (#7971)
* Update blst to v0.3.1 and incorporate subgroup changes

* go mod tidy

* gofmt

* Update bzl blst dependency

* Remove unnecessary check for nil

* Run bazel run //:gazelle -- fix

* Update blst to v0.3.2

* fix sha

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-12-04 06:46:08 +00:00
Raul Jordan
14e1f08208 Add Backup Webhooks to All Prysm Services With DBs (#8025)
* integrate backup webhooks

* pass slasher tests

* fix node

* cmd

* gaz

* read test passes

* radek feedback

* added comment

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-12-03 22:28:57 +00:00
Victor Farazdagi
647b4cf108 Round robin: half open interval in syncToFinalizedEpoch (#8039) 2020-12-03 21:55:42 +00:00
terence tsao
c090c6a1c5 Revert "Logging with PadLevelText to true" (#8036)
This reverts commit c51754fa8a.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-03 20:38:00 +00:00
Victor Farazdagi
7dd0c24fea Skip bogus init-sync test (#8038) 2020-12-03 20:20:04 +00:00
Victor Farazdagi
e1755b6066 Invert enable-sync-backtracking (#8034)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-03 15:43:26 +00:00
Victor Farazdagi
3092f75ec2 Init sync: conditional syncing to finalized slot (#7999)
* extract sync methods

* add unit test

* gazelle

* Update beacon-chain/sync/initial-sync/round_robin.go

Co-authored-by: Shay Zluf <thezluf@gmail.com>

* better set back step

Co-authored-by: Shay Zluf <thezluf@gmail.com>
2020-12-03 14:10:51 +00:00
Shay Zluf
821620c520 Add test for local protection genesis attestation (#7977)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-03 01:02:51 +00:00
Victor Farazdagi
5417e8cf31 Init-sync: enable peer scorer by default (#7974)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-03 00:45:20 +00:00
Radosław Kapka
323769bf1a Make TLS connections to a remote wallet non-mandatory (#7953)
* disable-remote-signer-tls flag

* use flag in edit-config

* send requests without TLS

* change warning message

* fix account list output test

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-03 00:18:15 +00:00
terence tsao
c51754fa8a Logging with PadLevelText to true (#8003)
* Use logrus formatter with pad

* Validator

* Fmt

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-12-02 23:49:13 +00:00
terence tsao
2153a2d7c3 Remove logging deposit inclusion slot (#8023)
* Remove logging deposit inclusion slot

* Remove old tests

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-02 21:17:47 +00:00
Nishant Das
20514cd97f Fix Interop Mode (#7978)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-12-02 20:29:36 +00:00
Victor Farazdagi
32f6bfd0a5 update deprecated multiaddr package (#8022) 2020-12-02 11:21:43 -08:00
Victor Farazdagi
c7f7a29d7e remove redundant start slot (#7991)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-12-02 20:47:39 +03:00
Nishant Das
387f7b28c1 Add Buffer For Inbound Peers (#8018)
* allow inbound peers

* comment

* gaz

* gaz

* Update deps.bzl

Co-authored-by: Shay Zluf <thezluf@gmail.com>

* add test

* re-arrange imports

* fix up

* fix tests

* gaz

* Update beacon-chain/p2p/service_test.go

* fmt

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: Shay Zluf <thezluf@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-12-02 10:45:28 -06:00
terence tsao
cf3181e2de Update README to include branching strategy (#8006)
* Update README.md

* Update README.md

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-12-02 09:48:52 -06:00
Mohamed Mansour
9d2fe80140 Fix missing space in error message in account list (#8009)
When validating account list, there was a missing space. This fixes that nitpick.

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-12-02 16:02:59 +03:00
Shay Zluf
f9c696ed54 Pending block queue lock fixes (#8002)
* Unlock in case of error

* fix comment

* fix comment

* revert one defer

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2020-12-02 11:50:37 +02:00
Raul Jordan
3bd5e58a5c Merge branch 'master' into develop 2020-12-01 08:06:16 -06:00
Raul Jordan
fc7c6776f6 Create Bucket If Not Exists When Fetching Proposal History (#8011)
* create bucket if not exist when fetching proposal history

* adding unit test
2020-12-01 08:00:03 -06:00
Potuz
9a1423d62d Log with field error instead of err (#7998) 2020-12-01 05:38:42 -08:00
Nishant Das
a13de7da11 Remove Exclusion List (#7992)
* remove

* gaz

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-01 02:17:29 +00:00
Raul Jordan
01bf97293f Fix Miscellaneous Deep Source Issues (#8007)
* deep source fixes

* sh fixes

* fix import

* test err

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-11-30 19:55:30 -06:00
terence tsao
645931802f Improve genesis event log and load blocks to fork choice log (#7946)
* Update a few logs

* Go fmt

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-01 00:06:16 +00:00
terence tsao
ea10784a4a Validator logging: return early if no att included (#7979)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-11-30 23:28:12 +00:00
terence tsao
3af7809964 Save cached state summaries on Stop() (#7988)
* Save cached state summaries on Stop()

* Fixed test and added one more test
2020-11-30 17:08:23 -06:00
terence tsao
b150acffca Recover state summary for sync validation (#7994)
* Recover state summary

* Add test

* Fix tests

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-11-29 20:42:45 -06:00
Victor Farazdagi
54a42ce4a8 more robust processing of invalid head slot (#7990) 2020-11-29 15:03:25 -06:00
Raul Jordan
9d174d5927 Add Support for Prysm Web V1.0.0-beta.1 (#7983)
* new web release

* site data update, add /logs to logs endpoints

* tests fixed

* test

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-27 20:30:12 +00:00
terence tsao
1b1b36497f Add validator bolt db metric collector (#7982)
* Validator db: add metrics collector

* Use the right library

* Go fmt

* Fix tests
2020-11-27 20:03:44 +00:00
Raul Jordan
04615cb97b Add Validator RPC Endpoint to Retrieve Beacon + Validator Logs Websocket Endpoints (#7981)
* add logs endpoint

* commands to retrieve logs endpoints

* ensure works at runtime

* add auth
2020-11-27 18:28:45 +00:00
terence tsao
b243665d3e Save lowest source and target epochs in post (#7973)
* Replace highest with lowerest

* Update validator/db/kv/attestation_history_v2.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/db/kv/attestation_history_v2.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Invert equality for saveLowestSourceTargetToDB

* Save lowest epcohs at post signature update

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-11-26 19:58:20 +00:00
pinglamb
654ef1afe5 Added flag for specifying header request limit (#7896)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-11-26 18:38:52 +00:00
Preston Van Loon
0dbf5c4e63 Increase public key cache size for Pyrmont (#7967)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-26 18:04:18 +00:00
terence tsao
c456dcdce8 Replace highest with lowest for signed epoch DB methods (#7965)
* Replace highest with lowerest

* Update validator/db/kv/attestation_history_v2.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/db/kv/attestation_history_v2.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Invert equality for saveLowestSourceTargetToDB

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-11-26 17:35:36 +00:00
Raul Jordan
10857223d0 Delete Dead Code in the Validator Client's Database Package (#7970)
* delete deprecated

* mod

* deep source

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-26 16:50:55 +00:00
Nishant Das
4ef8a0f99e Update Geth Again (#7968)
* fix again

* tidy up
2020-11-26 09:52:27 -06:00
Raul Jordan
af0977b8d0 Implement Export Block Proposals (#7964)
* export funcs for proposals

* add failing test

* round trip test passes

* progress

* deepsource

* Update validator/slashing-protection/local/standard-protection-format/export.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update validator/slashing-protection/local/standard-protection-format/import.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* revert

* empty root

* deep source

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-11-26 04:09:35 +00:00
terence tsao
528272fc66 Add Getters/Setters in ValidatorDB For Highest Signed Source and Target Epochs (#7961)
* Add getters and setters for source and target epochs

* Use the correct ImportStandardProtectionJSON

* Add tests

* Fix buckets

* Fix source to target

* Update validator/slashing-protection/local/standard-protection-format/import.go

* Fix bytesutil.BytesToUint64BigEndian will return 0 if input is less than 8 bytes

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-11-26 02:39:23 +00:00
Raul Jordan
c5c868d0a6 Use Separate Buckets to Store Special Data for Proposal Slashing Protection (#7963)
* ignore keys in the schema

* add test

* use separate buckets for special values where pubkey is the index

* test fix
2020-11-25 23:58:01 +00:00
terence tsao
622ab94465 Move Verified and saved pending attestations to pool to debug (#7928)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-25 22:52:57 +00:00
Raul Jordan
ae8a619775 Allow Optional Signing Roots in Proposal History (#7960) 2020-11-25 22:24:07 +00:00
Raul Jordan
36b1eb66d5 Add Attested and Proposed Public Keys DB Methods (#7959)
* add attested and proposed pubkeys methods

* Update validator/db/kv/attestation_history_v2.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-25 21:37:39 +00:00
Nishant Das
639dcb028c Fixing Validator Config (#7940)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-25 20:40:18 +00:00
Raul Jordan
edb40ddea4 Add ValidatorDB Methods for Highest and Lowest Signed Proposals (#7957)
* add in highest and lowest signed proposal

* begin adding tests

* add in db tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-25 20:04:43 +00:00
Preston Van Loon
0d2e3d978c Delete VERSION (#7958)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-25 19:20:57 +00:00
Preston Van Loon
29c6a0c42c Fix zero genesis check, make processAttestation routine wait for genesis time to be set (#7947)
* Fix zero genesis check, make processAttestation routine wait for genesis time to be set

* Update beacon-chain/blockchain/receive_attestation.go

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-25 18:52:59 +00:00
Nishant Das
aefa3e191a Check Sync Status First (#7895)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-11-25 18:27:27 +00:00
Preston Van Loon
987205afc6 Update security.txt (#7956) 2020-11-25 17:38:35 +00:00
dv8silencer
3ae4b793e6 Improve error log for when database network conflicts with runtime network (#7945)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-11-25 05:15:30 +00:00
terence tsao
600427bdb0 More bootnodes (#7948) 2020-11-25 04:25:47 +00:00
terence tsao
c7dd33431f Add lighthouse bootnodes (#7943)
* Add lighthouse bootnodes

* Remove extra line
2020-11-24 18:03:28 +00:00
Shay Zluf
0cf9800b75 Fix locks and fallback to db read if attestation history map is missing a pub key data (#7937)
* minimal change to handle nil attesterHistoryByPubKey

* Revert "Always Update Attesting History If Not Slashable (#7935)"

This reverts commit 3cc2ebc5d5.

* remove unused functions

* move save before propose

* wait before go func

* move wait into the go routine

* handling map mutation

* remove map handling in this case

* log in case it is still not found

* fix log

* fix locks

* Update validator/client/attest_protect.go

* remove code duplication

* remove method extraction

* move metrics to their appropriate place

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-11-24 11:24:23 -06:00
Raul Jordan
3cc2ebc5d5 Always Update Attesting History If Not Slashable (#7935)
* update attesting history always if not slashable

* initialize empty if no history found

* rem duplicate logic
2020-11-24 02:48:20 +00:00
Raul Jordan
2cb814648a Improve Slashing Protection for V1, More Tests and Observability (#7934)
* tests tests and more tests

* tests all passsss

* log for double vote
2020-11-23 19:03:04 -06:00
Raul Jordan
dc897a2007 Optionally Save Wallet Password on Web Onboarding (#7930)
* persist wallet password to wallet dir if onboarded via web

* add flag

* gaz

* add test

* fmt

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-23 22:11:42 +00:00
terence tsao
a051e684ae Update log levels (#7931) 2020-11-23 13:16:08 -08:00
Radosław Kapka
64be627a6d Make grpc-headers flag work (#7932) 2020-11-23 20:38:32 +00:00
Nishant Das
6f766ed583 Exit Pending Queue Properly (#7927)
* exit properly

* terence's review

* Update beacon-chain/sync/pending_blocks_queue.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/sync/pending_blocks_queue.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* fix tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-11-23 19:34:40 +00:00
Raul Jordan
b0dfc46603 Fix Up READMEs for Mainnet (#7910)
* fix up readmes

* Update README.md

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-23 18:47:55 +00:00
Nishant Das
8c3faaa4c7 Add Back Error/Debug Logs (#7922)
* add back logs

* add back string

* Reformat loggings

Co-authored-by: Terence Tsao <terence@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-11-23 18:29:27 +00:00
Nishant Das
4c0db8bca4 Bake In Mainnet Bootnodes (#7925)
* add in bootnodes

* fix teku's bootnode
2020-11-23 17:19:35 +00:00
Raul Jordan
0c5c246ee7 Prysm Web V1 Release (#7921)
* even more cors

* auth fixes for web v1

* ensure web works

* include web ui v1 release

* new site data

* fmt

* test

* tests pass

* gaz

* build fix

* no ssz

* unused type

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-11-23 06:29:58 +00:00
dv8silencer
f871e1f3ef Give error message if trying to import into non-imported wallet (#7913)
Co-authored-by: dv8silencer <15720668+dv8silencer@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-11-23 04:08:07 +00:00
Preston Van Loon
658dd95313 Add teku's bootnode (#7919)
* Add teku's bootnode

* Ignore mainnet config for TODO check
2020-11-23 01:20:55 +00:00
Preston Van Loon
57fe012bc2 P2P: Increase outbound message queue size to 256 (#7916) 2020-11-22 23:54:58 +00:00
Raul Jordan
d62420b940 More Default CORS Rules (#7915)
* even more cors

* terence feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-22 23:10:21 +00:00
Preston Van Loon
11bbea2562 Use params network config as default bootstrap nodes and deposit contract (#7904)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-22 21:22:44 +00:00
terence tsao
2172cd60a6 Update sync loggings (#7914) 2020-11-22 14:31:55 -06:00
Raul Jordan
2a546cc50b Remove Deprecated Tooling (#7912)
* remove old tools

* tidy and gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-22 19:07:02 +00:00
terence tsao
7d0031ee77 Update boot node to not use pyrmont (#7906)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-22 18:47:23 +00:00
pinglamb
acf49fb38f Fix Alpine Docker Image (#7883)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-11-22 18:04:13 +00:00
Raul Jordan
26658a9f1f Add More Default CORS Domains to the gRPC Gateway (#7901) 2020-11-22 17:12:24 +00:00
Nishant Das
60d99c83eb fix pending queue (#7899) 2020-11-22 08:42:27 -08:00
Alon Muroch
98557e8f5e highest slashing attestation RPC endpoint (#7647)
* highest slashing attestation RPC endpoint

* slasher mock fix

* Update proto/slashing/slashing.proto

Co-authored-by: Shay Zluf <thezluf@gmail.com>

* comments + small fixes

* PR review ctx comments and fixes

Co-authored-by: Shay Zluf <thezluf@gmail.com>
2020-11-22 08:51:20 +00:00
Preston Van Loon
1ba747b3c9 Change log from ERROR to DEBUG (#7892) 2020-11-21 23:14:19 -08:00
Preston Van Loon
71ec919306 RPC: healthz should return an error when the node is syncing (#7890)
* RPC: healthz should return an error when the node is syncing

* fix test

* fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-22 00:29:55 +00:00
Fabrice Cheng
34a26740ff Change attestation signature error to DEBUG level (#7891) 2020-11-22 00:08:51 +00:00
Nishant Das
7e76b02bb7 Make Follow Distance Lookup Simpler (#7884)
* faster eth1 search

* simplify it much more

* Update beacon-chain/powchain/block_reader.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-21 22:03:16 +00:00
pinglamb
519b003fc3 Fix creation time of beacon-node, validator and slasher (#7886)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-11-21 21:37:03 +00:00
Preston Van Loon
9a10462c64 p2p: return error when attempting to connect to a bad peer (#7885)
* return error when attempting to connect to a bad peer

* temporarily skip test
2020-11-21 20:09:07 +00:00
Nishant Das
ac60ff2bc2 Add Test For Earliest Voting Block (#7882) 2020-11-21 12:52:42 +00:00
Ivan Martinez
f8a855d168 Remove outdated code in accounts (#7881)
* Remove outdated test in accounts

* gaz
2020-11-21 11:15:44 +01:00
Preston Van Loon
74c7733abf Fix spec diff with comments. Fixes #7856 (#7872)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2020-11-21 06:12:47 +00:00
terence tsao
f63e89813d Remove chain not started error (#7879)
* Remove chain not started error

* Add genesis state not created error
2020-11-21 01:28:55 +00:00
terence tsao
c021e2e8bc Remove deprecated feature flags (#7877)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-21 00:15:44 +00:00
Preston Van Loon
c3fc40907d Fix potential panic with nil *big.Int (#7874)
* Fix potential panic with nil \*big.Int

* regression test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-20 23:09:02 +00:00
Shay Zluf
3fb78ff575 Verify GenesisValidatorRoot Matches the One in DB on Slashing Protection Import (#7864)
* Add GenValRoot dbs

* Test genvalroot

* Fix names

* Add overwrite rejection

* validate metadata genesis validator root

* remove env

* fix database functions

* fix tests

* raul feedback

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-20 22:33:51 +00:00
Raul Jordan
7bd97546f0 Dynamic Reloading of Keys on Any FSNotify Event (#7873)
* dynamic import

* add tests

* spacing
2020-11-20 22:04:59 +00:00
Ivan Martinez
5140ceec68 Hotfix for WaitForChainStart GenesisValidatorsRoot Check (#7870)
* Hotfix for genesis val root

* Add regression test

* Fix error message

* Remove comments

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-20 20:53:12 +00:00
terence tsao
97ad5cd5fd Reduce no attestation in pool to warn (#7863)
* Reduce no attestation in pool to warn

* Use NotFound

* Update validator/client/aggregate.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update validator/client/aggregate.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-11-20 12:17:26 -08:00
Ivan Martinez
4dc65c5787 Save GenesisValidatorsRoot from WaitForChainStart (#7855)
* Add GenValRoot dbs

* Test genvalroot

* Fix names

* Add overwrite rejection

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-11-20 18:06:12 +00:00
Roy
1b012ccfa5 Various Powershell Fixes (#7854)
* Remove incorrect x64 error message when showing usage description

* Add missing escape characters in usage description

The actual environment variable value would be printed without these
escape characters.

* Add missing quotation marks in usage description

* Also test existence of sha and signature files

For multiple reason the executable could be downloaded, but not the
signature files. Later on the script will error out because these files
are lacking.

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-11-20 17:17:38 +00:00
Nishant Das
60cdd69b05 Update Gossipsub Parameters (#7869)
* add param and flag

* change back
2020-11-20 15:36:02 +00:00
Preston Van Loon
90a66df529 Update eth2 specs version badge in README (#7865) 2020-11-20 03:21:11 +00:00
Nishant Das
c4a1fe4d0d Add Basic Support for IP Tracker (#7844)
* add basic support for ip tracker

* clean up

* check for it

* fix

* Update beacon-chain/p2p/peers/status.go

* fix
2020-11-19 12:54:19 +00:00
Nishant Das
8a256de2dd Check Target Root Better (#7837)
* check better

* bring it down

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-19 11:13:54 +00:00
Nishant Das
c3451a6ce9 Cache ETH1 Headers When Requesting Logs (#7861)
* perform a quick patch

* perform a quick patch

* fix

* fix up

* Update beacon-chain/powchain/service.go

* start caching from here

* remove

* fix
2020-11-19 10:47:31 +00:00
terence tsao
4b6441f626 Pending block queue caching with TTL (#7816)
* Update pending blks queue to ttl one

* Update tests

* Comment

* Gazelle

* Fix fuzz

* More comments

* Fix fuxx import

* Nishant's feedback

* Happy lint

* Return error for len(blks) >= maxBlocksPerSlot

* Ensure proposer time conv

* don't use gcache's default exp time it's 0

* fix TestService_AddPeningBlockToQueueOverMax

* Update beacon-chain/sync/pending_blocks_queue.go

Co-authored-by: Nishant Das <nishdas93@gmail.com>

* Fix time conversion

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2020-11-19 05:15:58 +00:00
Nishant Das
eb7ab16f92 Change Back Metadata Error Check (#7852)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-19 04:17:26 +00:00
Nishant Das
e6ecda5ebe add check and test (#7853)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2020-11-19 11:53:24 +08:00
Victor Farazdagi
095c4d5dd5 Peer status peer scorer (#7480)
* define and enforce minimum scorer interface

* better decoupling of multiple scorers in service

* removes redundant weight

* adds peer_status scorer

* minir re-arrangement

* rely on scorer in peer status service

* gazelle

* updates rpc_status

* fix build

* better interface verifying

* remove unnecessary locks

* mark todo

* simplify service

* remove redundant references

* avoid passing contexts

* remove unused context

* refactor errors to p2p package

* refactor goodbye codes into p2p

* simplify status api

* remove isbad method from peers

* update scoring service

* introduce validation error

* gazelle

* add score

* restore isbad method

* resolve dep cycle

* gazelle

* peer status scorer: test score calculation

* bad responses scorer: bad peer score

* remove redundant type checks

* pass nil config

* add rounding

* test IsBadPeer

* test bad peers list

* more tests

* check validation error on non-existent peer

* max peer slot -> highest peer slot

* remove redundant comment

* combine

* combine

* introduce var

* fix tests

* remove redundant update

* minor fix

* Nishant's suggestion

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-18 15:51:42 +00:00
Nishant Das
59d63087b1 Save Powchain Metadata To Disk On Chainstart (#7850)
* save to disk

* log error
2020-11-18 21:44:06 +08:00
Nishant Das
e1dd532af3 handle correctly (#7851) 2020-11-18 21:12:12 +08:00
Ivan Martinez
cfed4fa1b5 Remove listen for ChainStarted in WaitForChainStart (#7849)
* Remove GenValRoot from ChainStarted and remove ChainStarted from WaitForChainStart

* Fix test and add logs
2020-11-18 05:51:00 +00:00
Victor Farazdagi
7735a083b2 Extract common types from sync (#7843)
* extract common types from sync

* fix tests

* simplify

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-18 04:17:42 +00:00
Ivan Martinez
fec469291e Add GenesisValidatorRoot to ChainStartResponse (#7846)
* Add genesis validator root to chainstartresposne

* Deps

* Tidy

* Fix tests

* Fix test

* Fix test and add to ChainStartedData
2020-11-17 20:15:48 -06:00
Shay Zluf
acb47f2920 Implement Standard Slashing Protection JSON With Importing Logic (#7675)
* Use new attestation protection

* tests fixes

* fix tests

* fix comment

* fix TestSetTargetData

* fix tests

* empty history handling

* fix another test

* mock domain request

* fix empty handling

* use far future epoch

* use far future epoch

* migrate data

* copy byte array to resolve sigbus error

* init validator protection on pre validation

* Import interchange json

* Import interchange json

* reduce visibility

* use return value

* raul feedback

* rename fixes

* import test

* checkout att v2 changes

* define import method for interchange format in its own package

* rename and made operations atomic

* eip comment

* begin amending test file

* finish happy path for import tests

* attempt the interchange import tests

* fixed tests

* happy and sad paths tested

* good error messages

* fix up comment with proper eip link

* tests for helpers

* helpers

* all tests pass

* proper test comment

* terence feedback

* validate metadata func

* versioning check

* begin handling duplicatesz

* handle duplicate public keys with potentially different data, first pass

* better handling of duplicate data

* ensure duplicates are taken care of

* comprehensive tests for deduplication of signed blocks

* tests for deduplication

* Update validator/slashing-protection/local/standard-protection-format/helpers_test.go

Co-authored-by: Shay Zluf <thezluf@gmail.com>

* Update validator/slashing-protection/local/standard-protection-format/helpers_test.go

Co-authored-by: Shay Zluf <thezluf@gmail.com>

* tests for maxuint64 and package level comment

* tests passing

* edge cases pass

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-11-17 22:37:43 +00:00
terence tsao
925fba0570 Validate beacon block in pending queue (#7847) 2020-11-17 13:50:51 -08:00
dv8silencer
1a72733c53 Handle duplicate keystores in import path without error (#7842)
* bug fix

* Add regression test

* improve wording

* improve wording

* fix test

* comments, wording

* Comment

* import hex output

* fix test

* remove unnecessary sprintf

* fix test

Co-authored-by: dv8silencer <15720668+dv8silencer@users.noreply.github.com>
2020-11-17 13:50:23 -06:00
Shay Zluf
2976bf7723 Source lrg target (#7839)
* handle source > target better

* promatheus metric for source > target

* handle source > target well in sig bytes

* Update slasher/detection/attestations/spanner_test.go

* Update slasher/detection/attestations/spanner_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-17 17:17:21 +00:00
terence tsao
7c54cfea3f Hardening unaggregated attestation queue check (#7834)
* Add more checks and tests

* Move VerifyLmdFfgConsistency

* Move VerifyFinalizedConsistency

* Move VerifyFinalizedConsistency higher

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-17 16:31:43 +00:00
Fabrice Cheng
d3f8599d19 Add indicator for disabled accounts in account list (#7819)
* add indicator for disabled accounts in `account list`

* add also the account name in red for disable accounts

* bold disable as well

* Update validator/accounts/accounts_list.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-11-17 09:58:53 -06:00
Victor Farazdagi
2034c662af Refactor scoring service (#7841)
* refactor scoring service

* fix anti-pattern issue

* add block providers bad peers detection tests

* check status when peer scoring is disabled

* more tests
2020-11-17 23:28:13 +08:00
terence tsao
ad5151f25d Hardening aggregated attestation queue check (#7826) 2020-11-17 07:25:18 +00:00
Raul Jordan
f75a8efc0d Remove Keymanageropts Pattern from Wallets and Remove Enable/Disable Feature for V1 CLI (#7831)
* rem opts

* rem more km opts

* more removal of km opts

* removal of km opts

* definition of internal accounts store

* refactor enable/disable

* enable build

* fix rpc

* remove keymanageropts

* fix imported tests

* table driven tests for enable disable

* table driven tests for disable

* comprehensive tests for disable

* tests complete for enable and disable

* pass enable disable tests

* clarify imported

* fix deadlocks

* imported tests pass

* remove enable disable entrypoints

* better derived text

* deep source suggestions

* gaz

* tidy

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-17 06:00:20 +00:00
Nishant Das
39817c0586 Add Back Flag to Subscribe to All Subnets (#7836) 2020-11-17 05:25:35 +00:00
Nishant Das
168cffb0dd Check Sub Group for Herumi and Fix Edge Cases (#7823)
* check for herumi

* clean up

* fix tests

* fix
2020-11-17 04:12:23 +00:00
yorickdowne
194ee7c439 Add --mainnet no-op to validator sub-commands (#7833)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-11-17 03:17:08 +00:00
Ivan Martinez
5889670cc7 Remove WaitForSynced (#7835)
* Remove waitforsynced

* Remove WaitForsynced entirely

* Fix bazel

* tidy
2020-11-16 20:48:16 -06:00
Raul Jordan
7449eba612 Refactor HD Wallets for Enhanced Security (#7821)
* begin hd wallet refactor

* further simplify the new derived keymanager

* make it almost a full wrapper around an imported keymanager

* fix up the EIP test

* deprecated derived

* fixing keymanager tests

* fix up derived tests

* refactor initialize keymanager

* simplify hd

* pass some tests

* pass accounts list test

* gaz

* regenerate protos without create account privilege

* enforce account recovery on wallet create

* allow accounts delete to work

* remove mentions of accounts create

* resolve comments and go mod

* fix up tests

* build fixes

* remove insecure warning

* revert

* fix proto file

* remove create account message

* gaz

* remove account create

* update web api protos

* fix up imports

* change func sig

* tidy

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-11-16 22:26:04 +00:00
Preston Van Loon
d85cf028ef Update go-pbs after v1 changes (#7830) 2020-11-16 21:14:04 +00:00
terence tsao
71c6164c42 Remove a few old metrics (#7825) 2020-11-16 18:27:41 +00:00
Nishant Das
83601245f2 update geth (#7824) 2020-11-16 09:29:08 -06:00
james-rms
758ec96d6d beacon-chain: fix segfault (#7822)
Observed this segfault running all tests on mater, occurring
in around 2-3 out of 10 test runs.

```
FAIL: //beacon-chain/sync:go_default_test (shard 3 of 4, run 1 of 10) (see /home/j/.cache/bazel/_bazel_j/1ba834ca9d49f27aeb8f0bbb6f28fdf3/execroot/prysm/bazel-out/k8-fastbuild/testlogs/beacon-chain/sync/go_default_test/shard_3_of_4_run_1_of_10/test.log)
INFO: From Testing //beacon-chain/sync:go_default_test (shard 3 of 4, run 1 of 10):
==================== Test output for //beacon-chain/sync:go_default_test (shard 3 of 4, run 1 of 10):
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x138eea6]

goroutine 1660 [running]:
github.com/prysmaticlabs/prysm/shared/abool.(*AtomicBool).IsSet(...)
	shared/abool/abool.go:39
github.com/prysmaticlabs/prysm/beacon-chain/sync.(*Service).subscribeStaticWithSubnets.func1(0xc002dd4400, 0xc002990940, 0x17bca26, 0x1e)
	beacon-chain/sync/subscriber.go:207 +0xe6
created by github.com/prysmaticlabs/prysm/beacon-chain/sync.(*Service).subscribeStaticWithSubnets
	beacon-chain/sync/subscriber.go:200 +0x172
================================================================================
```

TestStaticSubnets was testing a Service with an uninitialized
chainStarted value. This commit initializes chainStarted explicitly
in all tests that construct a Service. This reduces the observed flake
rate to 0/10 runs. This was verified with:

```
./bazel.sh test //beacon-chain/sync:go_default_test --runs_per_test 10
```
2020-11-16 12:10:34 +01:00
terence tsao
977e539fe9 Loadblock returns err on invalid range (#7811)
* Return error on invalid range and fix tests

* Uncomment some test codes

* Update comment

* Sync with master, fixed more tests

* Rm error condition, update comments, tests
2020-11-16 01:06:13 +00:00
Victor Farazdagi
f361450e8d Update TestMain() to use os.Exit() (#7814)
* update TestMain

* fix sync/initial-sync test

* restore code in rate limiter

* fix rate_limiter tests
2020-11-13 18:28:14 -08:00
Preston Van Loon
0c9389a438 Fix instances of "The result of append is not used anywhere SCC-SA4010" (#7812) 2020-11-13 22:54:12 +00:00
terence tsao
f200a16418 Update Prymont config (#7808) 2020-11-13 11:16:07 -08:00
Raul Jordan
28ad21c410 Simplify Terms of Service Log (#7809) 2020-11-13 17:25:05 +00:00
Ivan Martinez
da835afbaf Add Partial Deposits in E2E (#7801)
* Partial Deposits in E2E

* Undo changes made to evaluator

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-11-13 16:35:09 +00:00
Fabrice Cheng
16bccf05cf [Feature] enable/disable validator accounts (#7746)
* add --enable --disable flags for validator accounts

* refactor DeleteAccountConfig into AccountConfig to be used for enable and disable feature

* add `disable` flag for validator accounts

* [wip] add method to disable account

* refactor account delete

* add disable & enable with proper filters

* fix keymanager unit tests

* update DisabledPublicKeys to be a string instead of [][]byte

* fix FetchValidatingPrivateKeys to only fetch active keys with new string format

* fix FetchValidationPrivateKeys with new DisabledPublicKeys format (as a string)

* rename file + update AccountsConfig to include Disable, Enable and Delete distinct attributes

* rename accounts_activation -> accounts_enable_disable

* revert changes from using string to [][]byte for DisabledPublicKeys

* add FetchAllValidatingPublicKeys to preserve the functionality for accounts list, backup and delete

* fix unit tests

* convert publickeys from [][]byte to str before passing it to pb message

* add unit tests for disable keys

* add unit tests for EnableAccounts

* revert WORKSPACE LLM for now

* ran gazelle

* move function to convert KeymanagerOpts to Config inside rpc and run gazelle

* add unit tests for FetchAllValidatingPublicKeys

* fix keymanageropts for InteropKey

* Fix mistake for enable accounts

* add docstring to DisableAccountsCli and EnableAccountsCli

* remove previous testnet and add toledo & pyrmont
2020-11-13 10:06:24 -06:00
Nishant Das
8dcdfea2a8 Make Blst the Default Library (#7805)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-11-13 15:25:05 +00:00
Nishant Das
244d9633af Update Go-Ethereum Dependency (#7804)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-13 14:52:27 +00:00
Nishant Das
d281ef9c56 Clean Up GoodByes (#7790)
* clean up

* cleanup

* fix

* fix tests

* change

* deepsource

* fix test
2020-11-13 12:58:13 +00:00
Nishant Das
58fcb52220 Fix Windows Builds For Blst (#7803)
* checkpoint

* fixWindowsBuils add transitive includes to mingw toolchain

* comment

Co-authored-by: SuburbanDad <gts.mobile@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-13 07:17:39 +00:00
Preston Van Loon
8d50fa10e6 Remove testnets prior to spec v1.0.0 (#7802) 2020-11-13 06:42:33 +00:00
Preston Van Loon
21d4c8f3f8 Update rules_go, prune unused go_repositories (#7800) 2020-11-13 04:32:15 +00:00
terence tsao
5fdb916b4f Align to spec v1.0.0 (#7469)
* Update eth1data params to double

* Update spec tests tags and state field for fssz gen

* Update more spec test sha tags

* Update slashing params

* Update slashing precompute to use config instead of hardcoded 3

* Update slashing test values due to config changes

* Update configs for slashedless test

* Go mod tidy

* Add toledo config (#7743)

* Update genesis delay to one week (#7782)

* Add Pyrmont config (#7797)

* Add Pyrmont config

* Fix config

* Update genesis time to the correct value

* Remove TestExecuteStateTransition_FullBlock

* Add back missing comments

* Update spectests to v1.0.0

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-11-13 01:00:05 +00:00
Shay Zluf
18be4a4e3e Immediate Slashing Protection Data Storage (#7789)
* Immediate save of validator protection data

* fix error log

* separate delete from save

* remove logs

* rename delete into reset

* comment fix

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-12 22:07:32 +00:00
Raul Jordan
e9136e9679 Remove Outdated Keystore Cryptography (#7796)
* remove outdated dependency

* fix up eip tests

* tidy
2020-11-12 21:16:41 +00:00
Preston Van Loon
5f9239595b Mitigate potential overflow. ethereum/eth2.0-specs#2129 (#7795) 2020-11-12 20:28:19 +00:00
Shay Zluf
47daedaf11 Warn missing protection db (#7792)
* Warn user for missing protection db

* better warning message

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-12 19:22:11 +00:00
terence tsao
52d850f355 Change connect/disconnect logs to debug (#7794)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-12 18:46:51 +00:00
terence tsao
d1b9f12a1e Fix and tests (#7793) 2020-11-12 12:17:54 -06:00
Nishant Das
56fd535dd5 Add Gossip Scoring For Peers (#7184)
* add gossip scoring

* fix

* clean up

* remove

* add new topics

* clean up gossip scoring

* clean up

* fix

* gaz

* remove true

* comment better

* remove from dev
2020-11-12 08:08:07 +00:00
Victor Farazdagi
79d19ea438 Enable head sync only during period of non-finality (#7784)
* enable head sync only during long period of non-finality

* Terence's suggestion

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-12 02:21:59 +00:00
terence tsao
ec2e677668 Update to not return state (#7786) 2020-11-11 16:40:43 -08:00
Raul Jordan
8e3c6e45ef Add EIP-2333 Conformity Tests (#7783)
* begin spec test for eip

* confirmity tests

* gaz
2020-11-11 21:24:08 +00:00
Ivan Martinez
a21a2c9e95 Add configurable deposit amounts to testutil (#7775)
* Add other functions to deposit helpers for configurable balance

* rename and comment

* Use secret key cache and test

* Gaz

* fmt
2020-11-11 14:47:26 -06:00
Potuz
25118fb8dc Stop early PendingExits (#7772)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-11-11 18:55:52 +00:00
Radosław Kapka
06902c667d Fix readme typo (#7779)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-11 14:40:24 +00:00
Potuz
d3ca9985eb log validator index in verifyExitConditions (#7773)
* log validator index in verifyExitConditions

* Fix missing symbol

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2020-11-11 12:55:33 +01:00
Raul Jordan
bd506bf4e8 Add Go Report Card to Prysm (#7778)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-11 09:03:13 +00:00
terence tsao
1a05fcae3c Use requested epoch for GetValidatorParticipation (#7768)
* Ensure request epoch is used

* Update test

* Comment

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-11 08:17:20 +00:00
Ivan Martinez
3c5bf9bf72 Remove unused chainStartPubKeys logic (#7777)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-11 07:37:34 +00:00
Raul Jordan
24457e1aae Fix Up Exits Pool Logic (#7774)
* Fix up exits logic

* comments

* tests for malformed exits

* comment fix

* add yet another unit test, check pending list with binary search

* simplify

* add test for favoring earlier exit epoch

* gaz

* removal of superficial map check
2020-11-11 06:52:58 +00:00
Raul Jordan
660ed2d9a8 Remove Recursive Read Lock in Shared/Rand (#7776) 2020-11-11 05:59:44 +00:00
Raul Jordan
4290ba416c Fix Prysm Runtime Data Races (#7770)
* handle state trie data races

* race fixes

* added proper locks

* fix gaz

* use thread-safe refs() function
2020-11-10 20:57:07 -06:00
terence tsao
9e9a172248 Add chain info tests (#7771) 2020-11-10 23:45:27 +00:00
Victor Farazdagi
2f11e55869 Use t.TempDir() in tests (#7769)
* use t.TempDir()

* remove redundant delete

* simplify setupDB()

* simplify db/testing/setup_db

* fix tests
2020-11-10 22:45:17 +00:00
Raul Jordan
7f7d18e910 Miscellaneous Keystore Fixes (#7756)
* remove v2 accounts rewrite

* warn users to ensure accounts are deleted

* radek feedback
2020-11-10 22:13:09 +01:00
Potuz
e22dd3758d Attestation performance metrics (#7709)
* call LogValidatorGainsAndLosses at end of epoch

* Reviewer fixes

* Reviewer fixes

* Reviewer fixes

* Export Inclusion Distance to Prometheus

* changed default value to 1

* removed default value

* Added other performance metrics

* add slot

* get rid of inclusion_slot

* Fix fmt test

* Reviewer changes

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-11-10 20:13:36 +00:00
Nishant Das
8638e2c0b5 Fix Blst Build For OSX (#7760)
* blst build

* Update stub.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-10 18:14:09 +00:00
Jim McDonald
0fb465ba07 Honor the --max-msg-size option in the gRPC service. (#7762)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-11-10 17:02:10 +00:00
Victor Farazdagi
09e3f0360e Remove redundant calls to os.exit() in TestMain (#7761)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-10 14:56:47 +00:00
Shay Zluf
7b0ee3adfe Use new attestation protection (#7605)
* Use new attestation protection

* tests fixes

* fix tests

* fix comment

* fix TestSetTargetData

* fix tests

* empty history handling

* fix another test

* mock domain request

* fix empty handling

* use far future epoch

* use far future epoch

* migrate data

* copy byte array to resolve sigbus error

* init validator protection on pre validation

* raul feedback

* rename fixes

* nishant feedback

* map with values

* fix tests

* lock and add test

* add and fix concurrency tests

* added tests error msg

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-11-10 14:14:11 +00:00
Nishant Das
f57bab78aa Don't Terminate Log Processing Early (#7757)
* don't terminate log processing

* fix all test

* add a better test var
2020-11-10 13:21:36 +00:00
Preston Van Loon
ce75b2f684 Add more validation to AllValidatorsAreExited (#7755)
* Add more validation to AllValidatorsAreExited

* gofmt

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2020-11-10 11:07:35 +00:00
Nishant Das
742808c6cf Fix Seen Cache Interval (#7751)
* fix

* var

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-10 05:39:17 +00:00
dv8silencer
b4bce7c726 Correct how AllValidatorsAreExited creates status request (#7758)
* fix and regression test

* address feedback

* gofmt

* improve test -- feedback

Co-authored-by: dv8silencer <15720668+dv8silencer@users.noreply.github.com>
2020-11-10 04:46:28 +00:00
Preston Van Loon
9e9a913069 bazel run //:gazelle -- update-repos -from_file=go.mod -to_macro=deps.bzl%prysm_deps (#7759) 2020-11-09 19:48:21 -08:00
Preston Van Loon
93c11e0e53 Update rules_go (#7202)
* Update rules_go

* go 1.15

* try with v0.24.2

* Update Mac OS X SDK

* gaz

* update SDK in toolchain config

* -I flag

* another -I flag

* Update rules_go, gazelle, bazel version

* regen, update rules_docker

* Revert "another -I flag"

This reverts commit 9255133d99.

* Revert "-I flag"

This reverts commit 2954a41d76.

* giving up

* Use OS X 10.12

* Use OS X 10.12

* Revert "Use OS X 10.12"

This reverts commit 4f60d5cb80.

* Revert "Use OS X 10.12"

This reverts commit a79177fab7.

* osx toolchain tweaks necessary to work with 10.15 mac sdk

* Update docker image, regen

* gaz

* test using custom image

* Revert "test using custom image"

This reverts commit 95b8666810.

* explicit go version

* Clean up docker image rules with new definitions. gazelle

* please the linter

* Update protobuf compiler to 3.13.0, run gazelle

* Update gazelle to fix empty build files. https://github.com/bazelbuild/bazel-gazelle/pull/926

* update skylib

* fix herumi fuzz build

* remove comment from tools/cross-toolchain/regenerate.sh

Co-authored-by: rkapka <rkapka@wp.pl>
Co-authored-by: SuburbanDad <gts.mobile@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-10 03:01:56 +00:00
terence tsao
1b9911ccc3 Batch verify aggregated attestation signatures (#7744)
* First take. Got benchmark numbers

* Remove benchmark test

* Final clean up

* Failing to verify aggregator index should be reject
2020-11-10 00:54:44 +00:00
terence tsao
be40e1a3b9 Update delete state(s) functions (#7754) 2020-11-09 15:37:36 -08:00
Raul Jordan
d4c954648c Prevent Usage of Stdlib File/Dir Writing With Static Analysis (#7685)
* write file and mkdirall analyzers

* include analyzer in build bazel

* comments to the single entrypoint and fix validator references

* enforce 600 for files, 700 for dirs

* pass validator tests

* add to nogo

* remove references

* beaconfuzz

* docker img

* fix up kv issue

* mkdir if not exists

* radek comments

* final comments

* Try to fix file problem

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
2020-11-09 14:27:03 -06:00
Potuz
15706a36cb Allow exiting validators to attest (#7747)
* Allow exiting validators to attest

* Added regression test

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-11-09 18:01:38 +00:00
Nishant Das
5995d2394c Pass By Value Instead Of Reference (#7710)
* change to value from reference

* fix up

* make it a pointer

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-11-09 10:08:08 +00:00
Potuz
1c5d533c93 Fix comment on DisableAccountMetricFlag (#7748) 2020-11-09 08:26:52 +00:00
Nishant Das
8cac198692 Keep Non Finalized States (#7742)
* keep non finalized states

* Update beacon-chain/db/kv/state.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-11-07 18:18:29 +00:00
terence tsao
4dcae8707a Startup routine to clean up dirty states in DB (#7729)
* Clean up dirty states in DB

* Add Passthrough

* Fix finalized check

* Add tests

* Remove unused test vars

* Update beacon-chain/db/kv/state.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

* Use non-strict inequality

* Prevent overflow and test

* Use slotsPerArchivedPoint/3 to loosen criteria

* Update test

* Background

* Handle ctx cancellation

* Update beacon-chain/state/stategen/service.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/db/kv/state.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Handle error

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-11-06 21:05:44 +00:00
Raul Jordan
b8644bdeb4 Minor Log Improvement to gRPC Server (#7722)
* small fix

* fix e2e

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-06 19:18:56 +00:00
yorickdowne
d733f2781a Add --mainnet no-op flag for easier docker tooling (#7737)
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-11-06 18:02:04 +00:00
Raul Jordan
20370e2017 Add DeepSource for Bash Files (#7741) 2020-11-06 17:04:48 +00:00
Victor Farazdagi
51796d77f6 Init-sync: backtracking to alternative forks (#7564)
* cherry-pick from previous commits

* use finalized checpoint instead of head slot

* add helpers

* add dedicated error for missing blocks

* add backtracking options to fetcher

* TestBlocksFetcher_alternativeSlotBefore regression test

* update peer block provider test

* update injected deps

* update round robing deps

* update fetcher

* patch status and stream handlers

* test template

* gazelle

* revert BestNonFinalized changes

* extract waitForBandwidth method

* extend test

* remove redundant code

* add flag

* update queue

* update thresholds

* upd inequality

* minor cleanup

* check for mode

* add backtracking queue test

* merge from master

* remove redundant test

* correct logging level

* fix tests

* remove test template

* Nishant's suggestion to parametrize magic number
2020-11-06 15:54:20 +00:00
Nishant Das
135ec5f247 Change To Mainnet Block Time (#7736) 2020-11-06 04:09:26 +00:00
Raul Jordan
d4b23e6821 Fix Pull Request Template Language (#7738) 2020-11-05 20:46:28 +00:00
Victor Farazdagi
6e21b7a623 Add one more assertion to fetcher test (#7733) 2020-11-05 19:48:49 +00:00
Victor Farazdagi
f6cbfd5e27 Init-sync fetcher: backtracking (#7704)
* fetcher: backtracking

* gazelle

* amend comments

* cleanup

* cleanup

* minor fixes

* simplify

* remove unrelevant test

* add TestBlocksFetcher_findAncestor

* more tests

* Address Nishant's feedback

* optimize backtracking

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-05 07:27:46 +00:00
Nishant Das
bafc7479b0 Add Backoff For Peer Dials (#7726)
* add tests

* Update beacon-chain/p2p/peers/status_test.go

* fix conditional

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-11-05 05:52:49 +00:00
Nishant Das
edf7ed614e Reduce Hashing When Filling In Forkchoice Blocks (#7716)
* reduce hashing

* add in test

* Update beacon-chain/blockchain/process_block_test.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-05 01:33:45 +00:00
terence tsao
882d30c382 Remove duplicated --slots-per-archive-point in help menu (#7731)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-05 00:53:59 +00:00
Preston Van Loon
d2694ee198 Powchain: Minor fix for log statement (#7728)
* Minor fix for log statement

* Update beacon-chain/powchain/service.go
2020-11-05 00:10:37 +00:00
Victor Farazdagi
c5a8363998 Remove redundant error return from filterPeers() (#7730) 2020-11-04 23:29:50 +00:00
Victor Farazdagi
7acd73e1fe Refactor p2p requests: extract BeaconBlocksByRangeRequest and BeaconBlocksByRootRequest (#7718)
* extract SendBeaconBlocksByRangeRequest

* adds extra processing

* extract SendBeaconBlocksByRootRequest

* extract into rpc_send_request.go

* add tests

* more tests

* gazelle

* remove redundant import

* for consistency

* for consistency

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-04 20:09:19 +00:00
Radosław Kapka
3485f3b8b0 Check if all validators are exited on every slot (#7719)
* check if validators are exited in every loop iteration

* regression test

* fix mock validator

* handle context.WithValue key in a more idiomatic way

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-11-04 11:41:47 -08:00
terence tsao
3a06f6e228 Update genesis time and deposit contract (#7727)
* Update genesis time and deposit contract

* Remove medalla as default config

* Update default case to use mainnet

* Update contract deployment block

* Update contract deployment block in flag

* Update shared/featureconfig/config.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-11-04 17:36:31 +00:00
Nishant Das
8661eb356f Fuzz: sigp/beaconfuzz#91 (#7725) 2020-11-04 14:05:48 +00:00
Nishant Das
090b71bec5 clean up better (#7723) 2020-11-04 15:24:29 +08:00
terence tsao
f1e6aba34e Fix GetValidatorParticipation slot to epoch conversion (#7721)
* Fix current slot conversion

* Update tests

* Use a higher epoch number
2020-11-04 10:54:56 +08:00
Potuz
b996824446 call LogValidatorGainsAndLosses at end of epoch (#7708)
* call LogValidatorGainsAndLosses at end of epoch

* Reviewer fixes

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-11-03 23:47:12 +00:00
Raul Jordan
0b0d77dd0c Include Multilock for GetAttPreState With Tests (#7669)
* multilock addition with tests and special clean logic on unlock

* bazel changes

* multilock key string concat

* Update beacon-chain/blockchain/process_attestation_helpers.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

* presto feedback

* revert in prog cache

* defer unlock

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-11-03 21:18:15 +00:00
Potuz
1a03dad6bc Include exiting and slashing as active in metrics (#7713)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-11-03 20:40:25 +00:00
Raul Jordan
5d93ee1843 Use Atomic Booleans in Sync Service (#7682)
* use atomic bool in tests as well

* gaz

* wrapped up tests

* fix test

* shadowing fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-03 19:12:24 +00:00
terence tsao
2da1ec8052 Remove expensive validators only method (#7717)
* Remove validators only method

* Gazelle
2020-11-03 12:47:21 -06:00
Nishant Das
c949913822 start from finalized checkpoint (#7715) 2020-11-03 22:43:06 +08:00
Nishant Das
cd00b6f594 fix herumi (#7706) 2020-11-02 21:28:38 -08:00
terence tsao
d22f48f84d Update which state to use for participation endpoint (#7702)
* Update GetParticipation

* Use head state for prev epoch

* Update test
2020-11-02 20:26:51 -06:00
terence tsao
19ac6782c9 Use batch verify signature for a signle block (#7687) 2020-11-02 15:27:29 -08:00
Raul Jordan
0b5db9d4a1 Update Dependencies for Bug Fixes and OSS Vulnerability Patching (#7646)
* dependency updates for OSS fixes

* tidy

* rem libp2p update

* more deps

* no workspace change

* tidy

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-02 18:50:55 +00:00
Nishant Das
40368bedd3 Allocate Fewer Large Slices (#7698)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-02 18:04:15 +00:00
Raul Jordan
51b39420dc Avoid Setting State to Nil In Process Deposit (#7681)
* avoid setting state to nil

* add other nil check
2020-11-02 17:12:05 +00:00
Raul Jordan
d2ae1b9286 Continue In ProcessPreGenesisDeposits (#7678)
* return error properly

* add in test

* reset

* Apply suggestions from code review

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
2020-11-02 10:34:37 -06:00
Nishant Das
4bc7cb6959 exit better (#7697) 2020-11-02 13:25:44 +08:00
Victor Farazdagi
3584bcba8e Fix param naming in BestNonFinalized (#7693) 2020-11-01 00:03:44 +00:00
Victor Farazdagi
926d3b9b34 Init-sync: more tests + minor refactoring (#7692)
* cherry-pick commits

* re-arrange calls
2020-10-31 21:33:57 +00:00
terence tsao
817c16a2f4 Use a short cut in migration step (#7686)
* Don't generate state if it's already in DB

* Use proper missing root

* Update logic for clarity
2020-10-31 19:01:05 +00:00
terence tsao
92b6e0b6af Add and use HasStateInCache (#7691) 2020-10-31 11:38:01 -07:00
Raul Jordan
b3155a04f5 Revert "Revert "Add In Progress Checker For Checkpoint Cache"" (#7690)
This reverts commit 46c04b98d9.
2020-10-31 15:59:50 +00:00
Nishant Das
df762bbfee cleaner logging (#7689) 2020-10-31 19:10:08 +08:00
terence tsao
f79b168ab2 Update to correct odering (#7688) 2020-10-31 10:08:22 +08:00
Nishant Das
211d9bc0b9 Update BLST And Herumi (#7632)
* fix build from source

* clean up

* update again

* change everything

* workaround for now

* fix versioning

* all passing now

* fix build issues

* clean up

* revert use of MulVerify

* gaz

* stub

* Apply suggestions from code review

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* fix all

* fix test

* todo

* fix stub

* revert back

* make deep source happy

* Update shared/bls/herumi/public_key.go

* Update shared/bls/blst/signature.go

* Update shared/bls/blst/signature_test.go

* imports

* move iface to common, export errors

* rm iface build

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-10-30 19:06:33 +00:00
terence tsao
386bfdd6eb Remove deprecated headstate method in DB (#7680)
* Remove head state in DB

* Clean up tests

* Fix pow tests

* Add stateGen to pow service
2020-10-30 15:11:08 +00:00
Raul Jordan
ddc8dc36f8 remove use of recover (#7683)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2020-10-30 14:40:26 +08:00
Preston Van Loon
99f15943a8 Fuzz: sigp/beaconfuzz#78 (#7684)
* Add failing test to verify https://github.com/sigp/beacon-fuzz/issues/78

* revert beacon-chain/core/blocks/spectest/attestation_test.go

* Describe bug in comments, fix bug

* 1
2020-10-30 02:06:53 +00:00
Raul Jordan
581bed2017 Load in JSON Validators for Genesis State Generation (#7643)
* load in genesis state from JSON validators

* completed feature with tests

* bazel change

* fix docker image

* deep source
2020-10-29 22:37:16 +00:00
Raul Jordan
2d4bfbbe31 Web UI Security Improvements (#7676)
* remove delete accounts

* check if user has not yet signed up
2020-10-29 16:38:47 -05:00
terence tsao
fb2dfec1f4 Use state by root to get finalized state (#7677) 2020-10-29 14:04:06 -05:00
Nishant Das
2e4dee5aeb Stategen Bug Fixes (#7674)
* bug fixes

* fix

* terence's review

* Make comments more explicit

* Update variable names

* Update tests

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-10-29 16:14:57 +00:00
Nishant Das
46c04b98d9 Revert "Add In Progress Checker For Checkpoint Cache" (#7672)
This reverts commit 37bf6617c0.

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-10-29 14:35:30 +00:00
terence tsao
301499d134 Fill in blocks to fork choice store during init (#7665)
* Fill in blocks to fork choice store during init

* Use format option

* Use format option

* Use correct head block. Thanks Nishant

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-29 05:27:30 +00:00
Victor Farazdagi
4fc0a50569 Turn init-sync FSM logs to trace level (#7670)
* trace

* go fmt

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-10-29 04:55:07 +00:00
Preston Van Loon
c1c0b53c25 validateCommitteeIndexBeaconAttestation: Reorder boolean logic to reduce db calls (#7671)
* reorder boolean logic to shortcut

* same issue, in another place

* refactor for the greater good
2020-10-29 04:14:16 +00:00
Nishant Das
37bf6617c0 Add In Progress Checker For Checkpoint Cache (#7659)
* add inprogress checker

* fix test

* fix

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-29 01:21:49 +00:00
Preston Van Loon
149d3b84fa Check attestation target checkpoint epoch matches attestation slot (#7667)
* Check attestation target checkpoint is within current or previous epoch

* reject bad att where slot does not match target

* Add test, reduce redundant check
2020-10-29 00:52:18 +00:00
Preston Van Loon
5092093389 Revert "Remove redundant checks in processAttestation" (#7666)
This reverts commit 3b34954e75.
2020-10-28 22:46:26 +00:00
terence tsao
3b34954e75 Remove continue checks (#7663) 2020-10-28 13:04:28 -07:00
Preston Van Loon
ec5e59e212 remove unused and untested method (#7662)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-10-28 12:03:30 -07:00
Shay Zluf
7d1a1643ee Add validator datadir use (#7660)
* fix empty db prompt

* wording

* fix merge

* prompt only on datadir use

* remove move datadir function

* remove flag

* remove prompt text

* remove added prompt function

* gaz
2020-10-28 12:09:44 -05:00
Preston Van Loon
5f80754013 Revert "Remove Top Jaeger Span Offenders" (#7657)
This reverts commit ec8eab21ae.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-28 06:07:14 +00:00
Raul Jordan
d9e4084d6d Remove Lock Contention in getAttPreState (#7656)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-28 05:42:54 +00:00
Raul Jordan
ec8eab21ae Remove Top Jaeger Span Offenders (#7655) 2020-10-28 05:14:54 +00:00
terence tsao
0cbd8bc03d Use root + slot for skip slot cache key (#7654)
* Update skip slot cache's key

* Add tests

* Refactor into its own function

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-10-28 03:09:05 +00:00
Victor Farazdagi
e57770bd0a Fix slasher ReadOnlyDatabase.BlockHeaders parameter names (#7652)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-28 02:36:02 +00:00
Raul Jordan
8c2fff3a75 Allow for 25th Word Passphrases in Mnemonics (#7645)
* advanced functionality, enable 25th word mnemonic passphrase

* 25th word passphrase

* add test

* added test to ensure differences when using the mnemonic 25th word

* better message

* passing tests

* fix up logic
2020-10-27 20:51:29 +00:00
Nishant Das
4f5726b3af Initialize State Correctly In Powchain (#7648)
* fix bug

* Update beacon-chain/powchain/service.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* return error

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-10-27 17:50:46 +00:00
Preston Van Loon
424488bf8a Deprecate db.HeadState (#7653)
* Deprecate db.HeadState

* Deprecate db.HeadState
2020-10-27 16:25:52 +00:00
Radosław Kapka
21c5ba8ed8 display that all validators are exited (#7651) 2020-10-27 09:29:58 -05:00
Victor Farazdagi
dbbbc7586f Remove redundant attribute in validator test (#7649)
* remove redundant attribute in validator test

* remove other attribs
2020-10-27 12:15:35 +00:00
Preston Van Loon
fcbb168c76 Code health: review map usage (#7635)
* remove unused cache states map

* correct typo

* Remove unused array

* Add lock around deposits cache chainstart pubkeys

* Copy attestation before grabbing lock. This may reduce lock contention time as other callers wanting the lock do not need to wait as long for the lock to become available.

* Copy attestation before grabbing lock. This may reduce lock contention time as other callers wanting the lock do not need to wait as long for the lock to become available.

* Set capacity to 1 since it is known that the slice will be 1 after insertion

* require validatorSlashingPreconditionCheck caller to hold lock

* Add lock for voluntary exits pool HasBeenIncluded

* Require rate limiter retrieveCollector to hold lock

* Add lock requirement assertions in sync

* Remove unused struct

* remove ClearCachedStates API

* field initSyncState is unused

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-10-26 21:17:07 +00:00
Nishant Das
f1bce1001d Restrict Proposer Lookup to the Current Epoch (#7542)
* checkpoint

* add test

* Update beacon-chain/core/helpers/committee_test.go

* preston's review

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-10-26 13:40:38 -07:00
terence tsao
ec77196197 Remove checkpoint info cache and usages (#7642)
* Remove checkpoint info cache and usages

* Gazelle
2020-10-26 14:09:19 -05:00
Raul Jordan
a468a12ef0 Allow Specifying Output Directory When Creating DB Backups (#7630)
* move flags

* backup db output dir flag

* fix build

* fix up broken backup test

* Radek's feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-26 18:20:54 +00:00
Radosław Kapka
b0dff891fc Invert enable-eth1data-majority-vote (#7362)
* make flag opt-out

* fix flags

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-26 17:36:47 +00:00
Raul Jordan
687251fedc keep asking for account pass if fails decrypt (#7631) 2020-10-26 17:56:57 +01:00
Alon Muroch
a04b7c2e4f Slasher highest source target (#7604)
* WIP - slasher highest attestation start

* fixed previous

* highest source and target

* highest attestation cache

* cleanup

* persist + fixes

* PR fixes and cleanup

* slashing proto

* highest att. api

* cleanup + tests

* increased highest att. cache to 300K

* removed highest att. api (for a separate PR)

* fixed linting

* bazel build fix

* highest att. kv test

* slasher highest att. test + purge + fix on eviction persist performance

* cleanup + linting

* linting + test fixes

* bazel gazelle run

* PR fixes

* run goimports

* go mod tidy

* ineffectual assignment fix

* run gazelle

* bazel gazelle run

* test fixes

* linter fix

* Apply suggestions from code review

Co-authored-by: Shay Zluf <thezluf@gmail.com>

* goimports run

* cache tests

* A bunch of small fixes

* gazelle fix + gofmt

* merge fixes

* kv ordering fix

* small typos and text fixes

* capital letter fix

Co-authored-by: Shay Zluf <thezluf@gmail.com>
2020-10-26 14:15:42 +02:00
Victor Farazdagi
e6d688f6d5 Init-sync: re-arrange tests (#7641)
* move tests

* formatting

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-26 08:54:06 +00:00
Victor Farazdagi
8a3b75e9e3 Update init-sync FSM (#7640)
* update fsm

* more tests
2020-10-26 05:26:35 +00:00
terence tsao
a73c539fab Forkchoice: update read only lock to regular lock (#7633) 2020-10-24 16:35:18 +00:00
Nishant Das
92efe64b8a Fix Gossip Message ID (#7624)
* fix snappy errors

* gaz

* Update beacon-chain/p2p/pubsub.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-10-24 03:38:05 +00:00
Raul Jordan
b1c047b9ee Prevent Panics in Field Trie Helpers (#7613)
* add tests to prevent panics

* if > 0

* fix typ
2020-10-23 21:41:45 +00:00
Nishant Das
ff50ea2e0d Better Logging When Using Head Sync (#7626)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-10-23 21:00:53 +00:00
terence tsao
ebb3fa71f1 VerifyFinalizedConsistency - return early when canonical (#7628)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-23 19:02:20 +00:00
Preston Van Loon
9ea69a070e Fix attestation broadcast recovery metric (#7629) 2020-10-23 18:41:17 +00:00
Raul Jordan
c59edb3358 Revert "Use validator protection datadir" (#7627)
This reverts commit ab76bdad15.
2020-10-23 17:19:38 +00:00
Radosław Kapka
7e2112b4ba Fix panic issues on beacon node shutdown (#7625)
* allow service cleanup on node shutdown

* revert error to fatal

* remove unused struct field
2020-10-23 12:43:48 +00:00
bidlocode
cdbbf66027 Fix pregenesis countdown msg wording (#7477)
* fix wording

* fix gofmt

* fix msg

* gofmt

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2020-10-23 05:21:52 +00:00
Victor Farazdagi
e5e51e66e1 Handle voting ties in BestFinalized() (#7622) 2020-10-23 04:43:35 +00:00
Nishant Das
6eb022ffa1 Clean Up Block Retrieval Methods (#7593)
* fix and add tests

* return for 0

* add test case

* fix

* fix fail

* add comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-23 04:23:15 +00:00
Nishant Das
e776eb5409 Fix Powchain Error In Startup (#7621)
* fix error

* add test case
2020-10-23 03:57:19 +00:00
Victor Farazdagi
6a2bb65fe2 Fix BestFinalized method (#7619)
* turn tests to use testing vector

* add regression test case

* complete tests
2020-10-23 02:49:42 +00:00
Preston Van Loon
ecc25d2b8c Release and remove feature flag --enable-att-broadcast-discovery-attempts (#7610)
* Release and remove feature flag --enable-att-broadcast-discovery-attempts

* fix test

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-23 01:53:50 +00:00
terence tsao
e07a12e6b7 Move IsCanonical implementation to forkchoice package (#7602)
* Move IsCanonical implementation to forkchoice pkg

* Remove debug print log

* Add tests

* Fixed current tests

* Fixed a test

* Fixed a test

* Add tests for cov
2020-10-23 01:32:13 +00:00
terence tsao
840ffc84ac Save state to DB during long non-finality (#7597)
* Starting saving state during hot

* Add a log

* Add helpers to turn on/off mode

* Add locks

* Add missing return

* Clean up

* Add logic to migration to handle db roots

* Add tests for on and off

* Add more tests

* Add test for migrate

* @prestonvanloon's feedback

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-23 00:35:30 +00:00
Raul Jordan
ca081e8639 Fatal If No TLS Found in RPC Service (#7614)
* fatal if no tls found rpc

* tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-22 23:58:53 +00:00
Raul Jordan
e54ac48f9d Deep Source Security Issues (#7617)
* deep source issues

* fix broken build

* fix beacon chain build

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-10-22 23:31:52 +00:00
Raul Jordan
075f1458b4 Do Not Recommend --ssl-no-revoke When Downloading Prysm Scripts (#7618)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-22 22:25:22 +00:00
Raul Jordan
be6481e178 Amend Cross Compile README (#7615)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-22 21:41:46 +00:00
Shay Zluf
ab76bdad15 Use validator protection datadir (#7355)
* Add validator protection db flag

* fix nil handling

* reuse datadir

* add datadir default config

* Add handling for moving account dir datafile to new set dir

* naming conditionals

* add tests

* fix test

* fix logic to default to wallet dir

* raul feedback

* nishant feedback

* gaz

* revert site_data changes

* fix formatting

* fix formatting

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-10-22 21:05:08 +00:00
Raul Jordan
e7723c4d1f Remove Deprecated Participation Metrics (#7616)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-22 20:14:58 +00:00
Raul Jordan
a688b9e030 Miscellaneous Docs Improvements (#7606)
* misc docs fixes

* revert readme

* fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-22 19:45:31 +00:00
Raul Jordan
ff15621fe1 Fix Nil Pointer in RPC List Peers (#7612)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-22 18:48:58 +00:00
Raul Jordan
4a78071e41 Explicit Password Requirements (#7607)
* explicit password requirements

* explicit reqs

* explicit reqs

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-22 18:02:30 +00:00
Raul Jordan
113b2cd6cf Clarify No Wallet Found Error Messages (#7609)
* clarify no wallet found error messages

* extract error

* fix tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-22 17:31:03 +00:00
Raul Jordan
13af8a7a37 Remove Unusued Base Parameter in SubscribeWithBase (#7608)
* remove base in sub with base param

* refactor subscribeWithBase signature
2020-10-22 16:56:24 +00:00
Nishant Das
7131cd991c Ignore PreGenesis Messages Instead Of Rejecting Them (#7550)
* ignorePreGenesisMessages

* fix test + handelaar's review

* fix issues

* change back

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-22 15:43:35 +00:00
Radosław Kapka
17a08a75ea Move voluntary exit logging to accounts_exit (#7603)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-10-22 15:01:11 +00:00
Nishant Das
ddbece5988 Fix Range Request Responses (#7531)
* checkpoint

* fix tests

* fix all tests

* imports

* gazelle

* reviews

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-10-22 09:33:01 -05:00
bidlocode
ff658ba641 Fix w3endpoint file relative path (#7557)
* add expandpath

* fix error description

* fix bazel & deepsource

* fix

* fix gofmt

* fix refactor

* add tests

* gofmt

* fix

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-10-22 05:53:03 +00:00
Victor Farazdagi
483f7f8177 Fix InitWithReset - rely on previously updated flags (#7598)
* Fix initWithReset to use previously set configuration

* further cleanup

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-22 05:20:56 +00:00
Preston Van Loon
b4c1c1db9b Add lighthouse p2p Good Bye codes (#7601)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-22 00:35:10 +00:00
Preston Van Loon
70d923cf85 Change log level from panicf to errorf so that all services can be shutdown (#7600) 2020-10-21 23:41:01 +00:00
Raul Jordan
e4e8dd4838 Fix Web UI Integration With Prysm Beta (#7591)
* fix change password endpoint

* attempt same port as gateway

* update site to latest release

* final fixes

* rem gorilla mux

* gaz fix

* gaz fix

* fmt

* fix build

* fix flags
2020-10-21 17:32:00 -05:00
Daniel Pittman
553492e6e9 Update TERMS_OF_SERVICE.md (#7590)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-10-21 20:34:40 +00:00
terence tsao
9554ad3221 Exit early if there's no validator slashing (#7587)
* Add a logic to exit early if there's no slashing

* Apply the same idea to final update

* Add conditions to satisfy fuzz

* Revert final updates changes

* Add comments

* Add a new test

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-10-21 19:12:39 +00:00
Raul Jordan
d2f7240255 Context Cancelation Handling for Hot States (#7596)
* attempt deadline for state info

* cancel check
2020-10-21 17:06:48 +00:00
terence tsao
42b7a37281 Add span for AncestorRoot (#7595)
* Add spans for AncestorRoot

* Add span for ancestor by DB

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-21 16:26:19 +00:00
Potuz
9a6a70e804 Add journald to log-format help (#7572)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-10-21 15:47:31 +00:00
bidlocode
9bd3cede23 Checking ToS cli flag in subcommands (#7588)
* check tos in subcmds

* fix
2020-10-21 10:13:46 -05:00
terence tsao
544dac298a Check ListValidatorBalances response length (#7583)
* Check length

* Add regression test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-20 20:45:38 +00:00
Raul Jordan
78ca8c9265 Add Accept Terms of Use Flags to Accounts Commands (#7584) 2020-10-20 20:12:49 +00:00
terence tsao
a5ce6db38e Update participation metrics (#7582)
* Update participation metrics

* Add unhappy tests
2020-10-20 14:14:24 -05:00
Victor Farazdagi
0b53a89d00 More verbose output on init-sync (#7578)
* more verbose output on init-sync

* log slot

* no error stack

* remove redundant data

* less verbose
2020-10-20 14:59:38 +00:00
Raul Jordan
fdef581e02 Invalidate JWT in Backend Logout (#7574)
* logout by invalidating in backend

* gaz

* regen pb.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-10-20 11:41:19 +00:00
Shay Zluf
7c5ee0a806 prune slasher attestations (#7335)
* prune slasher attestations

* remove unused code

* gaz

* remove unused methods from interface

* fix banchmark

* fix banchmark

* go.mod tidy

* preston feedback

* add test

* goimports

* fix test

* rm

* working version

* revert changes

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-20 14:09:01 +03:00
Nishant Das
81b553a00a Fix Overflow Bugs Introduced (#7579) 2020-10-20 06:18:03 +00:00
Raul Jordan
816eb94adf Validate JWT Signing Type in RPC (#7576)
* validate jwt signing type

* lint

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-10-20 05:37:12 +00:00
Radosław Kapka
ec0af98a9e Cache ETH1 headers on startup for eth1data voting (#7541)
* initial commit

* branch cleanup

* gzl

* underscore for unused parameteres

* caching implemented

* BlockExistsWithCache - tests

* add missing method to mock POWChain

* add missing method to faulty mock POWChain

* fix failing test

* add underflow checks

* change warning to error

* use helper to calculate end block

Co-authored-by: nisdas <nishdas93@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-20 05:10:27 +00:00
Raul Jordan
a39db494eb Remove Accounts Backup Validator RPC Functionality (#7575)
* remove accounts backup via RPC

* fmt

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-20 04:33:22 +00:00
Nishant Das
05678b6724 Update to Discovery V5.1 (#7302)
* discoveryV5.1

* add seed node

* fix up

* checkpoint

* Add workaround for discv5.1 signature curve. Add discv5.1 catdog ENR

* remove dead code

* Add another catdog

* Fix bootnode

* fix docker img

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-10-20 04:05:48 +00:00
Raul Jordan
bec91d348e Use Password Specific to Web UI Instead of Wallet Password (#7569)
* use password specific to web UI

* fix up a few more tests

* tests passing

* gaz

* fix fmt
2020-10-20 02:26:31 +00:00
Raul Jordan
1bc86d2658 Remove Account Creation Privilege For Imported Keymanager (#7555)
* rem create

* remove create account privilege for nonhd wallets

* fix bazel

* radek feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-20 01:22:36 +00:00
terence tsao
9db6c0042b Add LMD FFG consistency check to aggregated attestation (#7573)
* Add lmd and ffg check

* Update tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-19 23:19:44 +00:00
terence tsao
3d70d757a1 Add checking for finalized consistency (#7568)
* Add checking for finalized consistency

* Add tests

* @prestonvanloon's feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-19 21:52:18 +00:00
terence tsao
e25cd08049 Rename unsafe-sync to head-sync (#7570)
* Rename unsafe-sync to head-sync

* Update comments

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-19 20:03:31 +00:00
Radosław Kapka
dccf0992e5 Verify eth1data vote in E2E (#7551)
* add majority vote to e2e

* extract policies to a separate package

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-19 19:35:34 +00:00
Raul Jordan
f6ed3f141a Remove Default Wallet Endpoint (#7571)
* remove default endpoint

* remove from required auth endpoints

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-19 19:05:51 +00:00
Radosław Kapka
88b2a4c905 Invert enable-pruning-deposit-proofs (#7565)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-19 18:22:44 +00:00
Preston Van Loon
ab40a112c5 Print the underlying error when attestation fails to get pre state (#7567)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-19 17:03:27 +00:00
Nishant Das
329fbff814 Use StateRoot as Key for Cache (#7540)
* checkpoint

* review

* comment

* fix bad bug

* fix up
2020-10-19 09:13:47 -07:00
Nishant Das
f31d40cf34 Fix Unsafe Sync Flag (#7563)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-18 22:28:44 +00:00
Nishant Das
49909ce351 Remove checkpoint cache flag from --dev (#7560)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-10-18 21:56:55 +00:00
bidlocode
53ab1dff6d Make ToS non-interactive err message more descriptive (#7562)
* add descriptive err msg

* Update shared/tos/tos.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-10-18 17:08:57 +00:00
Nishant Das
b502876e98 Fix Mismatched BlockRoots In State Replay (#7559)
* fix mismatched roots

* alternate approach

* preston's review

* comment
2020-10-18 09:39:27 -07:00
Radosław Kapka
f474c4b1c5 Don't show stack trace for certain voluntary exit failure scenarios (#7554)
* handle some scenarios more gracefully

* allow accounts access to core/blocks package
2020-10-17 10:08:58 +00:00
bidlocode
78fd3b8a87 Fix import cannot find the file on windows (#7556) 2020-10-17 06:00:13 +00:00
Raul Jordan
7e44d1eec7 Rename Direct Keymanager to Imported (#7549)
* rename direct to imported

* km

* fix more refs

* rename all instances appropriately

* rename instances of nonhd
2020-10-16 13:45:14 -05:00
bidlocode
9a0d579607 Fix voluntary exit --tls-cert flag (#7547)
* fix getting flag values from ctx

* fix

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2020-10-16 14:27:01 +00:00
Preston Van Loon
07e7e030d9 Update PubSub and include topic filter (#7496)
* Update pubsub and fix topicIDs

* WIP filter

* Add suggested code from @bidlocode

* add tests and fix bugs

* more tests

* Wait until state initialized to accept pubsub filtering

* rename for clarity and clarify comment

* fix test builds

* Autofix issues in 2 files

Resolved issues in the following files via DeepSource Autofix:
1. beacon-chain/p2p/pubsub_filter.go
2. beacon-chain/p2p/pubsub_filter_test.go

* @nisdas pr feedback

* pr feedback and fuzz fix

* Update beacon-chain/p2p/pubsub_filter.go

* Must have protocol suffix

* Must have protocol suffix

* gofmt

* rm test, fix panic

* Fix tests

* Add isInitialized check

* Add a few more tests for better coverage

* cache fork digest, make pubsub filter part of the p2p service

* rename service

* gofmt

* Add comment

* fix

Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-16 07:05:40 +00:00
Raul Jordan
a81c863ddb Rename Accounts-V2 to Accounts (#7545)
* rename accounts v2

* rename keymanager and fix imports

* rename accounts-v2 instances

* imports

* build

* build fix

* deepsource

* fix up broken aliases

* imports

* gaz

* Update validator/accounts/accounts_import_test.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>

* fmt

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
2020-10-15 22:31:52 +00:00
bidlocode
7aaefd123e Accept ToS: fix (#7539)
* fix create datadir on fresh run

* add test

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-10-15 21:30:38 +00:00
Raul Jordan
7abc1feaf5 Update Ethdo Keystore Deps (#7537)
* include old crypto code

* update gomod

* delete new util

* add versioning for determining derivation strategy

* add versioning

* tidy and gazelle viz

* fix tests

* fix broken

* workspace fix

* fix workspace

* revert

* fix test

* derived test fix
2020-10-15 15:33:10 -05:00
Shay Zluf
acf201428e Use new proposal protection format (#7518)
* Use new proposal protection format

* Update comments

* Split and merge with new db

* fix tests

* fix test

* optimize domain

* fix validation

* fix validation

* check import error

* fix e2e

* fix old propose tests add ign block test

* constant secret key

* static test for signing

* test domain

* fix testsplit

* gaz

* gaz

* tidy

* raul feedback

* fix tests

* tidy

* added info log for the migration

* gaz

* Update validator/client/propose_protect.go

Co-authored-by: Nishant Das <nishdas93@gmail.com>

* nishant feedback

* import fix

* fix

* remove propose protection flag

* fix block sign test

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-15 19:35:31 +00:00
Ivan Martinez
daf0b51361 Eth2 API: Implement block endpoints (#7433)
* Start Beacon API outline

* Rename to v1

* Add impl

* Change to outline

* Add comments

* Remove unneeded items

* Fix linting

* tidy

* Fix visibility

* go.sum

* Fix deps

* Tidy

* Implement blocks API endpoints

* Add check for interface type and fix pointers

* Fix pointer name

* gaz

* Fix comments

* Fix imports

* Fix analysis

* Add more coverage

* Add coverage and fix errors

* Fix head test

* Fix test remove println

* Fix error text and cleanup

* Change tests to TDD

* Add tests for finalized

* Fix att test

* Fix analysis

* Fix go mo d

* Fix proto

* fix go mod

* Extend testing

* Fix tests

* Move migration to package and test block atts

* Fix migration

* Gaz

* Check for block canonical before returning

* Fix text

* Gaz

* Fix tests

* Fix tests

* Fix canonical

* Fix test again

* Fix tests

* Remove unneeded comment

* Plug in RPC service

* Fix err msg
2020-10-15 18:00:49 +00:00
Potuz
1462b7e57e Add show-private-keys flag to accounts-v2 list (#7487)
* Add show-private-keys flag to accoutns-v2 list

* fix fmt

* added tests and fixed return to 32 bytes

* deferred a lock and removed unused method

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-10-15 16:08:52 +00:00
Nishant Das
ec9c6f30bf Remove Local Protection Flag (#7543) 2020-10-15 14:52:45 +00:00
Nishant Das
0b64a335c8 check chainstart (#7494)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-10-15 22:03:48 +08:00
bidlocode
1caf2ca96f Accept ToS: fix paths for windows (#7535)
* fix paths for windows

* gofmt
2020-10-15 03:58:57 +00:00
bidlocode
5f9ea35b3f Add terms of use acceptance requirement (#7527)
* add accept tos

* fix typos

* add fixes

* check tos after config file loaded

* extend test

* add TERMS_OF_SERVICE.md

* fix

* fix typos

* add accept tos flag to e2e

* add flag to help

* add VerifyTosAcceptedOrPrompt to slasher

* fix gofmt

* fix import

* fix bazel
2020-10-15 02:05:30 +00:00
terence tsao
7076a1ec4a More feature flag deletions (#7533)
* Delete disable state lock and init sync verbose flags

* Delete disable slashing broadcast

* Remove disable wait for sync, noise, eth1 cache, static subnet

* Remove enable broadcast recovery attemp and make it as default

* Remove disable head update on per attestation

* Revert disable att braodcast discovery attempt

* gazelle

* Fixed an anti pattern

* Add enableAttBroadcastDiscoveryAttempts back

* Add back WaitForSync

* Remove extra lines

* Use DisableDynamicCommitteeSubnets path per @prestonvanloon feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-14 23:28:49 +00:00
Raul Jordan
a840fa563d Remove Accounts-V1 (#7532)
* remove accounts-v1

* get all tests to not panic

* all client tests passing

* fix node test

* eliminate old flags

* tidy up
2020-10-14 22:20:20 +00:00
Ivan Martinez
803d7c9bd2 Add functionality to retrieve all pending items from pools (#7530)
* Allow slashing pools to retrieve all items

* Add functionality to exits too to retrieve all exits

* Rename to noLimit

* ndo err

* Fix tests

* Fix test

* Fix test again

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-10-14 21:08:24 +00:00
Potuz
76300cef09 Recommend non-HD wallets (#7524)
* Recommend non-HD wallets

* Fix ordering

* set non-HD wallet as first option

* fix go fmt

* changed keymanager-types-Kind enum

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-10-14 19:55:47 +00:00
terence tsao
e5ed2cd141 Remove feature flags within the scope of pkg (#7511)
* Remove feature flags within the scope of pkg

* Remove DisableForkChoice

* Remove e2e usages

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-14 18:34:49 +00:00
Victor Farazdagi
a005c77b3f Update error usage patterns to go1.13+ (#7529)
* rely on errors.Is

* wrap errors
2020-10-14 17:39:52 +00:00
terence tsao
8f04c555fc More feature flags removal (#7526)
* Remove disable domain cache

* Remove don't verify att sig flag

* Remove batch verify and attester copies

* Remove batch verify usage

* Update tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-14 16:18:40 +00:00
Radosław Kapka
0a007384c8 E2E cleanup (#7519)
* e2e cleanup

* export EpochTicker

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-14 15:17:33 +00:00
Nishant Das
022b6667e5 Use Custom SSZ for P2P Types (#7436)
* checkpoint progress

* add roundtrip tests

* change all

* remove error response

* clean up

* Update beacon-chain/sync/error_test.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* gaz

* fix tests

* fmt

* gaz

* change back

* fix again

* clean up

* deep source

* fix all tests

* add gaz

* fix tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-10-14 07:55:28 +00:00
Ivan Martinez
022b09f2e2 Eth2 API: node endpoints outline (#7522)
* Nodev1 outline

* Gaz

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-14 06:11:11 +00:00
Raul Jordan
2f6f79724f Include Policy Bot Application (#7523)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-14 04:14:38 +00:00
terence tsao
7de161e917 Remove disable ssz cache feature flag (#7513)
* Remove ssz cache flag and usages

* Fix TestBeaconState_ProtoBeaconStateCompatibility

* gazelle

* Add cache flag back for tests

* Revert previous changes

* Revert previous changes

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-14 03:13:59 +00:00
Victor Farazdagi
b9844024b4 Update comments describing init-sync process (#7521)
* updates comments

* fetcher mode from config

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-14 01:44:46 +00:00
terence tsao
5cd6f65a2c Remove skip BLS verify flag (#7516)
* Remove skip BLS verify flag

* Update tests to use correct values

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-14 00:53:17 +00:00
Potuz
88083d1000 Add journald option for logger (#7463)
* Add journald option for logger

Fixes #7353

* fix docker images

* go mod tidy

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-10-14 00:10:57 +00:00
terence tsao
9e712e4598 Add justified block insertion to forkchoice when missed (#7520)
* Add justified block insertion to fork choice when missed

* Add regression test
2020-10-13 18:26:43 +00:00
Radosław Kapka
b742511193 Numeric conversion cleanup (#7465)
* validator count

* fix build errors

* fix test error

* randao mixes

* active indices

* lastReceivedMerkleIndex

* redundant conversions

* eth1FollowDistance

* refs

* fix compile error

* slasher

* validator

* revert changes to lastReceivedMerkleIndex

* remove todo

* fix references test

* fix committee test

* Revert "validator count"

This reverts commit 19b376e399.

# Conflicts:
#	beacon-chain/rpc/beacon/validators.go

* Revert "fix build errors"

This reverts commit f4acd6e977.

* Revert "fix test error"

This reverts commit 2a5c9eec63.

* Revert "randao mixes"

This reverts commit 2863f9c24d.

* Revert "active indices"

This reverts commit 6e8385f395.

* Revert "refs"

This reverts commit c64a153f67.

* Revert "fix references test"

This reverts commit fe773b55a6.

* Revert "fix committee test"

This reverts commit 7a0931c448.

* fix compilation errors

* Revert "slasher"

This reverts commit 8b34137931.

* trieutil

* some int() simplifications

* fix deepsource issues

* removed redundant fmt.Sprint

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-13 12:43:49 +00:00
terence tsao
aaabec5cb7 Remove current deprecated flags (#7512)
* Remove deprecated flags

* Remove usages in config

* Gazelle

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-13 06:38:41 +00:00
terence tsao
e9c23673c5 Remove inital sync don't verify att sig flag (#7517) 2020-10-13 06:08:21 +00:00
Radosław Kapka
06d16a24b9 Reduce deposit memory (#7478)
* remove deposit proof and pending deposits

* wip

* wip

* wip

* clean up logs/comments

* function for pruning deposits and feature flag

* remove log

* add flag to beacon flags

* addressed PR feedback from terence

* add test assertions for errors

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-12 18:10:49 +00:00
Victor Farazdagi
ac1a4a078c Fix reported anti patterns (#7501)
* merge var decl

* single append

* replace bytes.Compare -> bytes.Equal

* remove redundant spritnf

* remove redundant spritnf

* trimprefix

* remove redundant nil check

* remove redundant return

* plain channel or unblock on context closing

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-12 16:12:00 +00:00
Victor Farazdagi
a019a0db4c Combines func params of the same type (#7500)
* combines func params

* update leftovers

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-12 15:43:19 +00:00
terence tsao
db48e12270 Update "Requesting parent block..." to debug level (#7508)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2020-10-12 14:41:59 +00:00
terence tsao
9434d66ad0 Update Zinken deployment block (#7507)
* Simplify comparisons

* Update Zinken deployment block
2020-10-12 13:53:56 +00:00
Shay Zluf
3d0fc8bc64 Import old attestation store (#7466)
* import attestation to new data structure

* add tests

* add failure massages

* added signing root to data

* added signing root to data

* public keys 48 length

* remove redundant loop

* fix proposals

* fix manage dir name

* Omit redundant nil check on slices

* nishant feedback

* add test
2020-10-12 13:43:42 +03:00
Preston Van Loon
7cc32c4dda Various code inspection resolutions (#7438)
* remove unused code

* remove defer use in loop

* Remove unused methods and constants

* gofmt and gaz

* nilness check

* remove unused args

* Add TODO for refactoring subscribeWithBase to remove unused arg. It seems too involved to include in this sweeping PR. https://github.com/prysmaticlabs/prysm/issues/7437

* replace empty slice declaration

* Remove unnecessary type conversions

* remove redundant type declaration

* rename receivers to be consistent

* Remove bootnode query tool. It is now obsolete by discv5

* Remove relay node. It is no longer used or supported

* Revert "Remove relay node. It is no longer used or supported"

This reverts commit 4bd7717334.

* Delete unused test directory

* Delete unsupported gcp startup script

* Delete old k8s script

* build fixes

* fix build

* go mod tidy

* revert slasher/db/kv/block_header.go

* fix build

* remove redundant nil check

* combine func args

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-10-12 08:11:05 +00:00
bidlocode
0214553415 Fix config file flags in subcommands (#7475) 2020-10-11 08:26:59 -07:00
terence tsao
b5a913d862 Simplify comparisons ​(X-Y) == 0 (#7495) 2020-10-10 16:19:59 +00:00
Nishant Das
43765b5cb0 Register Subscribers After Node Is Synced (#7468)
* wait for synced

* fix again

* add test

* fix all

* fixes deepsource reported issue

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-10-10 16:50:28 +08:00
Victor Farazdagi
4c09e59b3b Remove duplicate imports (#7491)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-10 03:54:31 +00:00
Raul Jordan
551b03d6e6 Resolve Web UI Beta Testing Bugs (#7471)
* more descriptive password validation error

* include change wallet password fixes

* balance and jwt improvements

* allow for different wallet dirs specified on startup

* ensure wallet password is always validated

* fix up prysm tests

* gaz

* test pass

* pass balances tests

* wrap up fixes

* radek feedback

* fix up tests

* cors fix

* add tests for validator status

* pass tests

* fix n

* skip content type test

* package level cache and send over feed

* package level cache for derived

* all tests passing

* gofmt

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-10 02:07:28 +00:00
Potuz
c4e64afd07 Report on duplicate validator keys during imports (#7459)
* Fix #7393

* fix go fmt

* Add a test

* Convert to map of strings

* @terencechain fixes

* Update validator/keymanager/v2/direct/import.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-10-10 01:45:26 +00:00
Victor Farazdagi
d98a6dda8f Verify interface compliance using typed nil instead of dereferencing and conversion (#7489)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-10 00:36:48 +00:00
terence tsao
8aaa5b6ad0 Refactor ancestor lookup (#7472)
* Add ancestor root fall back to DB check

* Add a warning message

* Refactor ancestor lookups into seperate functions

* Update error string

* Fix has parent check and add a regression test

* Add a conext error test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-10 00:07:56 +00:00
Victor Farazdagi
f92244d497 Remove redundant err checking (#7488)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-09 23:39:23 +00:00
Preston Van Loon
e91165b0b4 Update validator UI to 0.0.1-alpha (#7486)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-09 22:06:23 +00:00
terence tsao
e15a0b08aa Remove duplicated StartSlot (#7484)
* Remove duplicated StartSlot

* Update existing tests

* Gazelle

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-09 21:23:49 +00:00
Raul Jordan
a3a77ab5a8 Open Web UI in Browser Automatically (#7481)
* open web

* Update shared/browser/browser_test.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update shared/browser/browser.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* add to depz.bazel

* run gazelle

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-09 19:51:46 +00:00
terence tsao
650ec797da Add missing mainet spec test (#7482) 2020-10-09 18:58:19 +00:00
Nishant Das
f629c72107 Tighten Up Snappy Framing (#7479)
* fix framing

* fix up conditions

* fix

* clean up

* change back

* simpler

* no need to cast

* Use math.MaxInt64

* gaz, gofmt

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-09 16:50:18 +00:00
Preston Van Loon
98a20766c9 Enable coverage for deepsource (#7441)
* Enable coverage for deepsource

* Add gocovmerge tool

* go mod tidy
2020-10-09 09:15:55 -07:00
Preston Van Loon
1f707842d2 Serve Prysm Web UI from Validator (#7470)
* Prysm web UI basic idea

* Refactor to use shared.Service interface, add sanity test

* register web server

* Determine mimetype

* Use 4242 as port for web

* Allow localhost or 127.0.0.1 for CORS. More tests, commentary

* Add flags, add site_data.go

* ignore site data

* Add sha

* gofmt

* gofmt

* fix script

* Lints

* fix vis

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-10-09 09:58:30 -05:00
dv8silencer
c944f29c7c Fix --clear-db on Windows (#7474)
* fix with text

* fix also for validator and add test

* handle error

* fix another test

* handle error

* gazelle

* fix bug in slasher as well

* gazelle
2020-10-09 09:28:35 +00:00
terence tsao
6b5265d2d4 Update slashing precompute to use config instead of hardcoded value (#7473) 2020-10-08 19:49:20 -07:00
terence tsao
8f64eb622e Optimize IsValidAttestationIndices unique sorted indices check (#7458)
* Optimize unique sorted indices check

* Add err handling for benchmark

* Gazelle

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-08 23:20:18 +00:00
Preston Van Loon
4ddacd57c6 fix blst build tag (#7443)
* fix blst build tag

* fix fuzz build

* remove buildkite specific bazelrc line

* remove conflicting kafka gotag in CI

* fix again

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-10-08 18:37:12 +00:00
Nishant Das
a66f434236 Listen to all Network Interfaces for discovery (#7434)
* listen to all ifaces for discovery

* fix test

* fix tests

* fix all tests

* Revert "fix all tests"

This reverts commit a5d6fffd45.

* Revert "fix tests"

This reverts commit b7a0cf5609.

* Revert "fix test"

This reverts commit 57e6218c68.

* fix and use loopback

* do properly

* fix

* listen to all ifaces

* fix bootnode

* make it trace

* make it loopback

* swap out

* change back

* fix e2e

* fix

* Update beacon-chain/p2p/discovery.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-08 15:16:18 +00:00
dv8silencer
796c336a29 Fix wallet check for Windows by addresses differences in error message text (#7461)
* fix

* test

Co-authored-by: dv8silencer <15720668+dv8silencer@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-10-08 14:30:08 +00:00
Nishant Das
48fcb08ebc Eth1 Handling Cleanup (#7467)
* clean up

* remove test
2020-10-08 13:47:16 +00:00
Shay Zluf
1315a15d9d Import proposal protection (#7430)
* Import proposals protection db

* Import old proposal protection data structure to new one

* test empty slots

* Update validator/db/kv/new_proposal_history.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

* Update validator/db/kv/new_proposal_history_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-10-08 11:51:50 +00:00
Radosław Kapka
25ebed9a70 Cache deposits only up to finalized block (#7453)
* cache deposits until finalized block

* take deposit index from state instead of block

* use stateByRoot

* use eth1data deposit count

* use depositCount-1

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-08 10:12:21 +00:00
terence tsao
703907bd99 Move ComputeProposerIndexWithValidators to test (#7460)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-08 09:24:17 +00:00
Preston Van Loon
d4e6ce6998 Fix deepsource findings (#7457)
* remove unused code

* Incomplete condition fix

* club append to single call

* anti-pattern fix 'should use for range instead of for { select {} }'

* use strings.ReplaceAll

* replace `len(parts[0]) == 0` with `parts[0] == ""`

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-08 08:36:47 +00:00
Preston Van Loon
390a589afb Disable BES in CI (#7462)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-08 07:17:35 +00:00
terence tsao
d34156bfe6 Add close to the body for e2e test (#7450)
* Add close to the body for e2e test

* Add error handling to defer

* Remove extra line

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-08 05:55:53 +00:00
bidlocode
7c8492e83f Add medalla flag (#7399)
* add --medalla flag for uniformity

* add enforcing testnet flag

* fix

* fix comments

* fix sufficient err msg

* fix shared method

* fix

* fix name

* fix name

* add test

* fix e2e

* fix gofmt

* fix test

* add warning

* fix warning

* fix warning test

* fix

* fix

* fix

* fix

* fix gofmt

* fix refactor to func

* add func description

* fix exported

* fix

* fix

* fix

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-10-07 23:04:19 +00:00
Preston Van Loon
668163d740 Revert PR #7429 to fix building on windows (#7456)
* Revert PR #7429 to fix building on windows

* Revert PR #7429 to fix building on windows

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-07 19:09:40 +00:00
terence tsao
7ad2929f0f Revamp proposer cache to fix lookahead bug (#7442)
* Add and test proposer indices cache

* Remove proposer indices usages from committee cache

* Adopt the new proposer indices cache

* Add comments on why genesis epoch is skipped

* Fix the failing tests

* Update beacon-chain/blockchain/process_block.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/core/helpers/committee.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/core/helpers/slot_epoch.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/core/helpers/slot_epoch.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/core/helpers/slot_epoch_test.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/core/helpers/validators.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Address additional feedbacks from @prestonvanloon

* Add comments on why genesis epoch is skipped

* Refactor EndSlot to use StartSlot within

* Add proposer indices disabled

* Add libfuzz tags

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-10-07 17:25:08 +00:00
Victor Farazdagi
9ce64e2428 Refactor peer scorer into peerdata + scorers (#7452)
* updates comment

* manager -> service

* rename receiver

* refacgtor bad_responses

* refactor store

* update status service

* extends data service

* status service test

* refactor block provider scorer

* misc updates

* fix tests

* data -> peerdata

* gazelle

* peerdata/store test

* limit the visibility scope

* Nishant's suggestion

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-07 13:08:51 +00:00
terence tsao
ae78546323 Add Zinken flag to slasher (#7451)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-07 09:44:50 +00:00
Raul Jordan
23bce8d0c5 Include Deposit Data JSON in Wallet Create RPC Response (#7444)
* return deposit data for hd wallet create

* test added for deposit data json

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-07 02:15:54 +00:00
terence tsao
29137f7b39 Remove unused SlotSignature function (#7447)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-06 23:55:37 +00:00
Raul Jordan
bf4a8dcee9 Fix E2E Readme to Include Bazel Define Flag (#7449)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-06 20:49:29 +00:00
Shay Zluf
7b5f71229e New attestation store for local protection (#7248)
* Update attestation with new marshal unmarshal

* Add db methods to handle new slashing protection schema

* lint

* add tests

* goimports

* Apply suggestions from code review

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>

* const fix

* fix ineffectual assignments

* goimports

* fix import issue

* victor feedback fixes

* victor feedback fixes

* receiver methods

* receiver methods

* Update validator/db/kv/attestation_history_new.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

* Update validator/db/kv/attestation_history_new.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

* Update validator/db/kv/attestation_history_new.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

* Update validator/db/kv/attestation_history_new.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

* Update validator/db/kv/attestation_history_new_test.go

* type change

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-10-06 19:59:36 +00:00
Raul Jordan
4d7797827e Concurrent Websocket Handlers for Log Streaming (#7428)
* concurrent conn with mutex

* add buffer size

* only localhost

* clarify caller needs to hold lock

* Update shared/logutil/stream.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update shared/logutil/stream.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* fix up tests

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-06 13:32:48 +00:00
terence tsao
c0ed43d920 Refactor head info for better usages for lock (#7432)
* Refactor head info for better usages for lock

* Fix headroot test to use [32]byte{}

* go fmt: issue introduced in #7429

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-10-06 10:57:18 +00:00
Victor Farazdagi
842c15856b Apply go fmt + tidy: issue introduced in #7429 (#7446) 2020-10-06 09:05:32 +00:00
Potuz
20ac925ee4 Add a journald log format option. (#7429)
* Add a journald log format option.

Add an accepted value "journald" for the --log-format flag so that
logs are directed to journald with the correct error level.

* fix docker images

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-10-06 04:37:16 +00:00
Raul Jordan
6e8ff10003 deep source toml (#7439) 2020-10-05 14:07:49 -07:00
Victor Farazdagi
419fad07cd Handle nil header properly in powchain service tests (#7435) 2020-10-05 10:10:31 +00:00
Ivan Martinez
cf1c346beb Eth2 API: Add outline for beacon endpoints (#7408)
* Start Beacon API outline

* Rename to v1

* Add impl

* Change to outline

* Add comments

* Remove unneeded items

* Fix linting

* tidy

* Fix visibility

* go.sum

* Fix deps

* Tidy

* Add check for interface type and fix pointers

* Fix pointer name

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-10-05 00:15:27 +00:00
Radosław Kapka
3e0b20529b Miscellaneous code quality improvements (#7414)
* anti-patterns

* performance issues

* handle skipped defer

* gazelle fix

* misc bug risks

* make logging of proposer slashings more robust

* simplify calling span.End()

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-04 15:03:10 +00:00
Raul Jordan
d9ae2073e2 Pagination Added to ListAccounts Validator RPC Call (#7422)
* Pagination tests to list accounts

* fetch all

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-03 17:26:39 -05:00
terence tsao
7b3efcf62b Move read lock after hasHeadState (#7427)
* Move read lock before hasHeadState

* Export the locked version

* Update beacon-chain/blockchain/head.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

* Update beacon-chain/blockchain/chain_info.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

* Update beacon-chain/blockchain/chain_info.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

* Update beacon-chain/blockchain/chain_info.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-10-03 20:53:05 +00:00
terence tsao
c7d01fd73c Add lock around fork choice store votes (#7426)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-03 05:48:34 +00:00
Preston Van Loon
2916d183e8 Pubsub: ensure better TopicIDs validation (#7418)
* Ensure better topicIDs validation, add tests

* go mod tidy

* fix tests

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-03 01:51:58 +00:00
terence tsao
c24fb792cb Add locks around head getters (#7423) 2020-10-03 00:45:59 +00:00
Raul Jordan
70f3fcdbd9 Implement Create Account via RPC (#7400)
* add ssz rules

* regenerate protos to include deposit message

* built to match the deposit cli

* derived test

* fix up nil item

* fix up more nil items

* preston suggestion

* comment

* add missing tests to build file

* fix busted tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-10-02 16:35:28 -05:00
Victor Farazdagi
ecfdb354a7 Add new static analyzer: ineffassign (#7413)
* test defined

* first working impl

* register analyzer

* cleanup

* removes unused code

* secure rand in slasher/detection/testing

* Updates test

* fixes ineffassign type checks

* one more

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-10-02 23:56:18 +03:00
terence tsao
690fa12f1a Check block attestation length for operation pool (#7420)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-10-02 20:08:06 +00:00
Nishant Das
63c1057ae6 add bootnode (#7415)
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-10-02 12:28:09 -07:00
Preston Van Loon
1eee1948bb Fall back to ValidationIgnore in the event of a validation panic (#7419)
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-10-02 11:33:45 -07:00
Victor Farazdagi
e36e9250f1 enable cryptorand analyzer in slasher (#7417) 2020-10-02 19:32:21 +03:00
Nishant Das
b589ddd774 use full hash (#7416) 2020-10-02 23:08:51 +08:00
Nishant Das
ba01abbc8f use valid build tags (#7411) 2020-10-02 12:45:58 +03:00
Nishant Das
fa82b09cec Optimize Fetching of Eth1 Blocks (#7386)
* optimize fetching

* tests and clean up

* improve comment

* more cleanup

* lint

* dont terminate lookup

* shay and radek's review

* fix test

* Update beacon-chain/powchain/block_reader.go

* gofmt

* cleanup

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: rkapka <rkapka@wp.pl>
2020-10-02 15:31:05 +08:00
dv8silencer
7d9a706cfa Warn if user is not using a supported platform (#7381)
* Add prereqs package

* Incorporate prereq check into the clients

* gazelle

* gazelle fix

* linter

* Add tests

* minor change in test

* finish up tests

* gazelle

* error during platform detection does not cause client to fail fast

Co-authored-by: dv8silencer <15720668+dv8silencer@users.noreply.github.com>
2020-10-02 09:45:34 +03:00
Raul Jordan
437bab7df0 Update EthereumAPIs (#7409)
* handle unknown validator

* tests passing

* tidy
2020-10-01 22:04:48 -04:00
bidlocode
ee747ca6d4 add display contract (#7407)
* add display contract
2020-10-01 22:55:49 +00:00
terence tsao
de93551332 Ensure span ends (#7405)
* Ensure span ends
* Merge refs/heads/master into span-ends
* Merge refs/heads/master into span-ends
* Add missed span end
* Merge branch 'span-ends' of github.com:prysmaticlabs/prysm into span-ends
* Merge refs/heads/master into span-ends
2020-10-01 21:13:27 +00:00
terence tsao
0839f10dbc Add lock around feature config (#7406)
* Add lock
2020-10-01 20:35:44 +00:00
Victor Farazdagi
d169b490fa Fix ineffectual assignments (#7403)
* update rpc/beacon
* more fixes to beacon-chain/rpc
* update beacon-chain/sync
* Merge refs/heads/master into fix-ineffectual-assignments
* updates beacon-chain/p2p
* Merge branch 'fix-ineffectual-assignments' of github.com:prysmaticlabs/prysm into fix-ineffectual-assignments
* update beacon-chain/*
* fix imports
* update beacon-chain/blockchain
* more updates
* Merge refs/heads/master into fix-ineffectual-assignments
* Merge branch 'master' into fix-ineffectual-assignments
* Merge refs/heads/master into fix-ineffectual-assignments
* next round of updated
* Merge branch 'fix-ineffectual-assignments' of github.com:prysmaticlabs/prysm into fix-ineffectual-assignments
* wrap up remaining items
2020-10-01 18:53:36 +00:00
terence tsao
490cf9b7ba Remove logging of password if fails (#7404)
* Remove logging of password if fails
2020-10-01 17:41:23 +00:00
Victor Farazdagi
95a5b4945b Fixes incorrect checks for errors in several tests (#7392)
* fixes TestServer_ListAssignments_Pagination_InputOutOfRange
* fixes TestServer_ListValidatorBalances_PaginationOutOfRange
* fix TestServer_ListAttestations_Genesis
* remove redundant TestServer_GetValidatorParticipation_DoesntExist and TestGetDuties_NextEpoch_CantFindValidatorIdx
* Merge branch 'master' into fix-invalid-errcheck-tests
* remove unnecessary import
* fix TestStore_OnAttestation
* fix TestStore_OnAttestationUsingCheckptCache
* fix TestVerifyBlkDescendant
* fix pagination tests
* fix account v2 tests
* fix account v2 tests (remote)
* fix TestServer_JWTInterceptor_BadToken
* Merge refs/heads/master into fix-invalid-errcheck-tests
2020-10-01 14:38:53 +00:00
Victor Farazdagi
4aea039324 Remove unused constant in round robin (#7402)
* remove unused constant
2020-10-01 13:02:47 +00:00
Raul Jordan
23181c8629 Implement Backup and Delete Accounts via RPC (#7397)
* begin with delete accounts impl

* delete tests passing

* started backup, delete tests all passing

* implement backup using zip

* backup implementation done with unzip tests

* full tests done

* used proper error codes

* remove comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-09-30 22:37:38 -05:00
Preston Van Loon
d23b247a82 DB: Return error when saving nil eth1data (#7396)
* Return an error on saving nil eth1data
* gofmt
* Merge branch 'master' into return-error-on-nil
* Merge refs/heads/master into return-error-on-nil
* Merge refs/heads/master into return-error-on-nil
2020-09-30 23:23:22 +00:00
terence tsao
5178474280 Check err for filterAttestationsForBlockInclusion (#7398)
* Check err for
* Merge refs/heads/master into check-err
2020-09-30 22:45:17 +00:00
terence tsao
d5caee35fa Update min genesis time for Zinken (#7395)
* Update min genesis time
* Merge refs/heads/master into update-zinken-config
2020-09-30 22:09:23 +00:00
bidlocode
558ee2678b Optional services flags (#7358)
* add DisableMonitoringFlag(--disable-monitoring) to validator
* add EnableRPCFlag(--rpc) to validator (now rpc and grpc-gateway disabled by default)
* add DisableMonitoringFlag(--disable-monitoring) to slasher
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into optional-services-flags
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into optional-services-flags
* Merge branch 'master' into optional-services-flags
2020-09-30 20:55:56 +00:00
Raul Jordan
e1f8a37710 Import Accounts RPC Implementation (#7389)
* implement the import function

* begin tests for import keystores and add preconditions

* import implementation done

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-09-30 14:03:54 -05:00
terence tsao
e07b71b81d Add Zinken config (#7391) 2020-09-30 11:19:05 -07:00
Victor Farazdagi
db92d90309 Removes unused validator flags (#7388)
* removes unused validator flags
* Merge branch 'master' into remove-unused-validator-flags
2020-09-30 16:55:18 +00:00
terence tsao
e2eb2fb0d8 Ensure genesis state retrieval doesn't use zero hashes root (#7378)
* Cover genesis zero hashes case
* Merge branch 'master' into genesis-zero-hashes
* Cover an edge case pre genesis with fresh db
* Move state by root outside of the if
* Merge refs/heads/master into genesis-zero-hashes
* Clarify comments on why zero hashes used
* Merge branch 'genesis-zero-hashes' of github.com:prysmaticlabs/prysm into genesis-zero-hashes
* Merge refs/heads/master into genesis-zero-hashes
* Merge refs/heads/master into genesis-zero-hashes
* Merge refs/heads/master into genesis-zero-hashes
2020-09-30 15:41:09 +00:00
Shay Zluf
953960c860 Add slasher support for different testnets (#7385)
* Add slasher support for different testnets
* Merge branch 'master' into slasher_testnet_support
2020-09-30 15:08:03 +00:00
dv8silencer
7664eab32d Introduce checks for existing wallets into two edge cases (#7349)
* Add check for preexisting wallet

* Reminder

* Create another check.  Adjust tests

* Wording in comment

* Remove unnecessary logic.  Add another regression test.

* Spacing

* Prepare to merge master

* Refactor wallet checks

* Fixed test

* Revert back to original location of check

* Fix test

* Fix test

* Address linter feedback

* Align CreateWallet() with recent changes

* Align RecoverWallet with recent changes

* Correct test error message

* Refactor test to minimize duplication

* rename to isValid and adjust function code

* refactor

* Improve IsValid()

* Fix tests

* Remove comment

* Fix documentation of IsValid()

* Align behavior of OpenWalletOrElseCli

* Fix tests

* Realigning error msg with prior behavior

* Fix another test

* Correct logic

* small change in logic

* Fix err messages

* Create consts for repeated strings

* Add comments to new constants

* gofmt

* Adjust WalletConfig behavior so it is closer to prior-to-PR.  Adjust test accordingly

* adjust error messages

Co-authored-by: dv8silencer <15720668+dv8silencer@users.noreply.github.com>
2020-09-30 16:13:37 +02:00
Raul Jordan
16cdcf5dec Remove Deposit Command from Validator Client (#7379)
* remove deposit code
* Merge refs/heads/master into remove-deposit
2020-09-30 06:42:16 +00:00
Nishant Das
3b5ef50733 Fix Finalized Epoch Checks (#7364)
* fix up epoch checks

* build

* comp tests

* preston's comment
2020-09-30 08:25:25 +08:00
George Carder
529554f3f9 accommodate systems that use sha256sum (#7370)
* accommodate systems that use sha256sum
* cleaner use of checkSum
* Merge branch 'master' into shell_script_support4_sha256sum
* Merge branch 'master' into shell_script_support4_sha256sum
2020-09-29 12:22:35 +00:00
Radosław Kapka
fe9921457c Fix failing static analyzer tests (#7363)
* add "want" expectations
* add build step with analyzer tests
* add colon to yml
* Merge refs/heads/master into fix-analyzer-test-expectations
* Merge refs/heads/master into fix-analyzer-test-expectations
* Merge refs/heads/master into fix-analyzer-test-expectations
* Merge refs/heads/master into fix-analyzer-test-expectations
* remove tests from CI
* readme file
* change header size
2020-09-29 11:29:40 +00:00
terence tsao
bedb16cfb0 Update badge to v0.12.3 (#7367)
* Update badge to v0.12.3
* Merge refs/heads/master into terencechain-patch-1
* Merge refs/heads/master into terencechain-patch-1
2020-09-29 06:10:11 +00:00
Preston Van Loon
0921c00094 Remove processSlots from slashing check (#7357)
* Remove processSlots from slashing check

* imports

* imports

* PR feedback from @nisdas

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-09-29 13:22:55 +08:00
Raul Jordan
6ef4995329 Remaining Proto Definitions for Validator RPC Methods (#7368)
* import, delete, backup, and create acct
* Merge branch 'master' into rpc-changes
* impl methods
* Merge branch 'rpc-changes' of github.com:prysmaticlabs/prysm into rpc-changes
* wallet import comment
2020-09-29 01:58:14 +00:00
Victor Farazdagi
1e0b4e150e Fixes NotNil assertion (#7365)
* fixes NotNil assertion
* cover case of basic types
* removes redundant line
* Update TestStore_UpdateCheckpointState
* fixes TestPruneHistoryBlkHdr
2020-09-28 21:32:58 +00:00
Radosław Kapka
951c139cff Fix URL to voluntary exit documentation (#7361)
* fix documentation URL
2020-09-27 22:12:28 +00:00
terence tsao
eb3e4944e9 Ensure block root is non zeros for setting head before genesis (#7359)
* Ensure block root is non zeros for setting head
* Merge branch 'master' into can-set-head-genesis
* Merge refs/heads/master into can-set-head-genesis
2020-09-27 21:09:08 +00:00
terence tsao
bb98046608 Fix ListValidators inactive check (#7354)
* Fix ListValidators inactive check
* Updated an existing test
* Merge refs/heads/master into fix-list-validators
2020-09-27 20:30:03 +00:00
Eduard Klementiev
4ac0bbca19 gRPC load balancing (#7133)
* gRPC load balancing
* Fix typo
* Merge branch 'master' into 5132-gRPC-client-load-balancing
* Add doc about load balancer
* Update validator/client/multiple_endpoints_grpc_resolver.go
* Merge branch 'master' into 5132-gRPC-client-load-balancing
* gofmt
* Merge branch 'master' into 5132-gRPC-client-load-balancing
* Merge branch 'master' into 5132-gRPC-client-load-balancing
2020-09-27 18:08:28 +00:00
dv8silencer
c0c34f3d3a Fix panic that happens if you wallet-v2 recover 1 account (#7351)
* Bugfix
* Regression test.
* Remove unnecessary code.
* Merge branch 'master' into dv8-i7350
2020-09-26 10:13:56 +00:00
Nishant Das
77c95f3051 Add New Compiler Flags for BLST (#7352)
* add new c opts
* remove
* add to asm
* fix up benchamrks
* Merge refs/heads/master into addNewOpts
2020-09-26 08:48:02 +00:00
terence tsao
68d0c09daf Add, use and test VerifyWeakSubjectivityRoot (#7344)
* Add, use and test `VerifyWeakSubjectivityRoot`
* Comments
* Make bazel test happy
* Merge branch 'master' into verify-ws
* Add flag to help
* Merge branch 'verify-ws' of github.com:prysmaticlabs/prysm into verify-ws
* Merge refs/heads/master into verify-ws
* Merge refs/heads/master into verify-ws
* Merge refs/heads/master into verify-ws
* Merge branch 'master' of github.com:prysmaticlabs/prysm into verify-ws
* Update beacon-chain/blockchain/service.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Update beacon-chain/blockchain/weak_subjectivity_checks.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Merge branch 'verify-ws' of github.com:prysmaticlabs/prysm into verify-ws
* s/&&/|| for VerifyWeakSubjectivityRoot circuit breaker
* Merge refs/heads/master into verify-ws
* Merge refs/heads/master into verify-ws
2020-09-26 08:04:07 +00:00
Raul Jordan
282f3eec01 Display More Info for Validating Keys on Startup (#7347)
* display more info for wallet
* Merge refs/heads/master into display-info-for-keys
2020-09-25 20:43:12 +00:00
Raul Jordan
4a549ffe2b Warn About Deprecation of Accounts-V1 Within the Next 2 Releases (#7348)
* deprecation of accounts v1 error log
2020-09-25 18:24:31 +00:00
Nishant Das
d138c608bc Disable Libp2p Protocols (#7345)
* disable
2020-09-25 12:33:28 +00:00
Nishant Das
c7ff3a4f22 Better Chainstart Logging (#7336)
* better chainstart logging
* single threaded
* Merge branch 'master' into betterChainstartLogging
* Merge refs/heads/master into betterChainstartLogging
* Merge refs/heads/master into betterChainstartLogging
* Merge refs/heads/master into betterChainstartLogging
* Merge refs/heads/master into betterChainstartLogging
* Merge refs/heads/master into betterChainstartLogging
* Merge refs/heads/master into betterChainstartLogging
* Merge refs/heads/master into betterChainstartLogging
* improve formatting
2020-09-25 04:31:23 +00:00
Nishant Das
7a96412ef2 Reject Messages With Invalid Fields in GossipSub (#7337)
* strict no sign
* Merge branch 'master' into StrictNoSign
2020-09-24 23:53:15 +00:00
Raul Jordan
5821454ac7 Implement Wallet Config and HasWallet Endpoints (#7341)
* fix the wallet config implementation

* implemented wallet config endpoint and tests

* better error messages

* fix nil pointer

* more nil issues fix

* gazelle

* fix interceptor

* revise

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-09-24 15:37:59 -05:00
terence tsao
90978cd22e Reset hot state cache size to 32 (#7340)
* Reset hot state cache size to 32
* Merge refs/heads/master into revert-hot-state-cache-size
2020-09-24 18:10:02 +00:00
Raul Jordan
76a3070fd7 Fix Change Wallet Password Logic (#7324)
* fix change password logic
* gaz
* Merge refs/heads/master into change-password-fix
* Merge refs/heads/master into change-password-fix
* Merge refs/heads/master into change-password-fix
* Merge refs/heads/master into change-password-fix
* Merge refs/heads/master into change-password-fix
* Merge refs/heads/master into change-password-fix
* Merge refs/heads/master into change-password-fix
* Merge refs/heads/master into change-password-fix
* Merge refs/heads/master into change-password-fix
* Merge refs/heads/master into change-password-fix
2020-09-24 16:47:13 +00:00
terence tsao
1bc0cc7049 Skip duplicated aggregated attestation in pending queue (#7326)
* Skip duplicated aggregated atts
* Test
* Fixed a regression test
* Merge branch 'master' of github.com:prysmaticlabs/prysm into skip-dup-aggregator
* Go fmt
* Merge refs/heads/master into skip-dup-aggregator
2020-09-24 15:47:03 +00:00
terence tsao
ff69375fbd Revert 7242: Opt out --use-check-point-cache (#7338)
* revert bb37436
2020-09-24 15:03:35 +00:00
Victor Farazdagi
5b814009fa Improves test coverage of init-sync queue (#7320)
* basic invariants for all state transition handlers

* TestBlocksQueue_onCheckStaleEvent extended

* TestBlocksQueue_onScheduleEvent extended

* TestBlocksQueue_onDataReceivedEvent extended

* removes redundant ctx setting

* TestBlocksQueue_onReadyToSendEvent extended

* TestBlocksQueue_onProcessSkippedEvent extended

* test to highest slot update

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-09-24 21:39:46 +08:00
Radosław Kapka
99164761f5 Make voluntary exits release-ready (#7305)
* phrase validation utility
* use passphrase from docs portal
* enable voluntary exits
* Merge branch 'master' into enable-exits
* rename 'voluntary-exit' to 'exit'
* change passphrase
* gazelle
* change constant to error variable
* Merge branch 'master' into enable-exits
* rename error variable
* build fix
* Merge refs/heads/master into enable-exits
* do not remove whitespace in the middle
* Merge refs/heads/master into enable-exits
* Merge refs/heads/master into enable-exits
* Merge refs/heads/master into enable-exits
* Merge refs/heads/master into enable-exits
* code review changes
* Merge refs/heads/master into enable-exits
* Merge refs/heads/master into enable-exits
2020-09-24 11:00:44 +00:00
Radosław Kapka
e1cd9143e0 Package-level comments for static analyzers (#7332)
* package-level comments
* Merge refs/heads/master into analyzer-package-comments
2020-09-24 10:25:55 +00:00
Nishant Das
4d232feda8 Add Spadina Bootnodes (#7330)
* add spadina bootnodes
* add in bootnode
2020-09-24 09:49:24 +00:00
Nishant Das
d687270a89 Update BLST to Latest Commit (#7328)
* update blst
* Merge branch 'master' into updateBLST
2020-09-24 08:34:09 +00:00
Nishant Das
6d8207801c Update Bootnode With New Flags (#7327)
* update bootnode
* gaz
* Update BUILD.bazel
2020-09-24 07:02:36 +00:00
Preston Van Loon
6ad117b175 Return an error when any of the eth1 clients are nil (#7323)
* Return an error when any of the eth1 clients are nil
* return error if eth1DataFetcher is nil
* Annotate span
* gaz
* fix build and add tests
* dear linter, please accept this commit as tribute and let my PR pass
* Merge refs/heads/master into fix-7322
2020-09-23 23:16:06 +00:00
terence tsao
f9e062407f Fix an edge case when loading replay start state by slot (#7313)
* Ensure start state is the parent of the first block
* Regression test
* Merge branch 'master' into state-by-slot-state-state
* Add back the line
* Merge branch 'state-by-slot-state-state' of github.com:prysmaticlabs/prysm into state-by-slot-state-state
* Add a test for lower slot block condition
2020-09-23 21:39:16 +00:00
Radosław Kapka
dca93ce641 Unnecessary Slice-to-Slice Conversion analyzer (#7321)
* analyzer with tests
* fix bazel file
* modify analyzer to fix build issues
* add analyzer to tool chain
* remove arrays from inspections
* fix redundant [:] operator
* Merge branch 'master' into use-slice-directly
* Merge branch 'master' into use-slice-directly
* fix another inspection
* add package-level comment
2020-09-23 16:14:34 +00:00
Nishant Das
347aa14a28 Fix Bazel Reference For Go-Ethereum (#7318)
* fix api change
2020-09-23 10:20:08 +00:00
Victor Farazdagi
3621b2ff25 Better naming for constructor funcs (#7316)
* sync/initial-sync
* NewBeaconClientService -> NewService
* NewSlashingProtectionService -> NewService
* NewPrometheusService -> NewService
* NewColdStartService -> NewService
* NewRegularSync -> NewService
* NewDetectionService -> NewService
* NewWallet -> New
* NewKeystore -> New
* Merge branch 'master' into better-naming
* Merge branch 'master' into better-naming
* NewDepositCache -> New
2020-09-23 08:59:49 +00:00
Nishant Das
49ae42c249 Fix Validator For Spadina (#7314)
* fix commands
* Merge refs/heads/master into fixSpadina
2020-09-23 08:03:13 +00:00
Victor Farazdagi
1ce7cd5f50 Init sync: minor refactoring and more tests for Service (#7286)
* removes redundant code
* removes redundant code
* removes redundant db
* removes redundant p2p
* servive - test future genesis
* more tests for service.Start()
* service - already synced test
* Merge branch 'master' into init-sync-more-service-tests
* Merge branch 'master' into init-sync-more-service-tests
* NewInitialSync to New (avoid stutter)
* TestService_markSynced
* more assertions
* TestService_Resync
* renames test
* Merge branch 'master' into init-sync-more-service-tests
* Merge refs/heads/master into init-sync-more-service-tests
* Merge branch 'master' into init-sync-more-service-tests
* Merge refs/heads/master into init-sync-more-service-tests
* Merge branch 'master' into init-sync-more-service-tests
* fixes e2e test
* Merge branch 'init-sync-more-service-tests' of github.com:prysmaticlabs/prysm into init-sync-more-service-tests
* Merge refs/heads/master into init-sync-more-service-tests
* Merge refs/heads/master into init-sync-more-service-tests
* Merge refs/heads/master into init-sync-more-service-tests
* Merge refs/heads/master into init-sync-more-service-tests
* New -> NewInitialSync
2020-09-23 07:24:33 +00:00
terence tsao
305fdd2e1b Add spadina network name (#7309)
* Update spadina network name
* Merge refs/heads/master into update-config
* Merge refs/heads/master into update-config
* Merge refs/heads/master into update-config
* Merge refs/heads/master into update-config
2020-09-22 23:20:20 +00:00
terence tsao
984cb38680 Remove long deprecated historical regen states in DB (#7308)
* Remove long deprecated db state gen
* Remove long deprecated db state gen
* gazelle
* Delete pass through
* Merge refs/heads/master into rm-deprecated-regen
* Merge refs/heads/master into rm-deprecated-regen
* Merge refs/heads/master into rm-deprecated-regen
2020-09-22 22:57:50 +00:00
terence tsao
bbdd20e898 Log grpc error chain (#7307)
* Log error chain
* Merge refs/heads/master into error-msgs
* Merge refs/heads/master into error-msgs
2020-09-22 22:23:01 +00:00
Victor Farazdagi
568cd3c9ec Fixes e2e "chain started" check (#7311)
* fixes e2e test
2020-09-22 21:34:38 +00:00
Raul Jordan
b09b1f3fa5 Disallow Creation of Multiple Wallets at Wallet Path (#7290)
* disallow multiple wallet creation
* imports
* Merge branch 'master' into disallow-multiple-wallets
* resolve conflicting test
* Merge branch 'disallow-multiple-wallets' of github.com:prysmaticlabs/prysm into disallow-multiple-wallets
* test passes
* Merge refs/heads/master into disallow-multiple-wallets
* Merge refs/heads/master into disallow-multiple-wallets
* Merge refs/heads/master into disallow-multiple-wallets
* Merge refs/heads/master into disallow-multiple-wallets
* Merge refs/heads/master into disallow-multiple-wallets
* Merge refs/heads/master into disallow-multiple-wallets
* radek feedback
* Merge branch 'master' into disallow-multiple-wallets
2020-09-22 19:02:13 +00:00
Nishant Das
22bcfd2c34 Update Message ID (#7304)
* shorten message id
* Merge refs/heads/master into updateMessageID
* Merge refs/heads/master into updateMessageID
2020-09-22 15:23:33 +00:00
Raul Jordan
ba440abe2d Store Hashed Password to Wallet Path (#7295)
* store wallet pass on creation
* store wallet hash
* save hash to wallet dir
* initialize wallet hash
* gaz
* imports spacing
* remove mentions of hashed pass from db schema
* simplify the wallet hash code
* comment removal
* Merge branch 'master' into fix-password-override
* pass tests
* test fix
* Merge branch 'fix-password-override' of github.com:prysmaticlabs/prysm into fix-password-override
* remove extraneous printfs
* tests passing
* require auth for create wallet
* gaz
* fix signup logic
* Merge refs/heads/master into fix-password-override
* Merge refs/heads/master into fix-password-override
* Merge refs/heads/master into fix-password-override
* Merge refs/heads/master into fix-password-override
* Merge branch 'master' into fix-password-override
2020-09-22 14:49:07 +00:00
Nishant Das
09640ae22d Update Go-Ethereum Fork (#7303)
* update
* tidy
* Merge refs/heads/master into updateGeth
* remove metaData
* remove cloudflare
* Revert "remove cloudflare"

This reverts commit 687210f70f.
2020-09-22 13:23:45 +00:00
Nishant Das
719e99ffd9 Replace Roughtime With Timeutils Across Prysm (#7301)
* remove roughtime

* change all references

* rename

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-09-22 19:49:58 +08:00
Shay Zluf
1a4129f5a6 Fix prompt default handling (#7300)
* Fix prompt default handling
* Merge branch 'master' of github.com:prysmaticlabs/prysm into default_handling
* added test
* Merge branch 'master' of github.com:prysmaticlabs/prysm into default_handling
* Merge branch 'master' of github.com:prysmaticlabs/prysm into default_handling
2020-09-22 06:35:15 +00:00
Nishant Das
de3f112a05 Remove Roughtime Usage From Prysm (#7299)
* remove roughtime
* Merge branch 'master' into removeRoughtime
* Merge refs/heads/master into removeRoughtime
2020-09-22 05:47:01 +00:00
terence tsao
3734bfacce Add target epoch and slot consistency check (#7288)
* Add target epoch and slot consistency
* Remove debug logs
* Merge refs/heads/master into att-epoch-consistency
* Merge refs/heads/master into att-epoch-consistency
* Merge refs/heads/master into att-epoch-consistency
2020-09-22 05:13:12 +00:00
Shay Zluf
d5e2b51d66 Add slasher grpc logs (#7289)
* Add grpc logs
* gaz
* Merge refs/heads/master into add-connection-error-handling
* try to restart connection on status unavailable
* Merge branch 'add-connection-error-handling' of github.com:prysmaticlabs/prysm into add-connection-error-handling
* add retry on dial
* goimports
* Merge refs/heads/master into add-connection-error-handling
* gaz
* Merge branch 'add-connection-error-handling' of github.com:prysmaticlabs/prysm into add-connection-error-handling
* Merge branch 'master' of github.com:prysmaticlabs/prysm into add-connection-error-handling
2020-09-22 04:39:34 +00:00
terence tsao
cdd28abc4b Remove renovate.json (#7291)
* Remove renovate.json
* Merge refs/heads/master into rm-renovate
2020-09-21 22:54:24 +00:00
Preston Van Loon
8c8f59e242 Fix sha after v0.12.3 tarballs were re-uploaded. Unskip test to resolve #7277. (#7292)
* Fix sha after v0.12.3 tarballs were re-uploaded. Unskip test to resolve #7277.
2020-09-21 22:22:19 +00:00
Raul Jordan
b1f9f97062 Implement ChangeWalletPassword API Endpoint (#7273)
* allow changing wallet password

* wallet change password logic

* tests for wallet password changes

* more unit tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-09-21 14:44:43 -05:00
Victor Farazdagi
7545d3f2b3 Init sync refactor state initialization check + tests (#7285)
* refactor state initialization into waitForStateInitialization
* more tests
* redundant blank line
* Merge branch 'master' into init-sync-refactor-state-init
* moves comment up
* Nishant's suggestions + bit more refactoring
* Explicitly process error (Nishant's suggestion)
2020-09-21 12:20:26 +00:00
dv8silencer
bdf8bf7be2 Fix accounts-v2 backup so that it can backup HD wallet accounts (#7284)
* Bugfix

* Added test
2020-09-21 10:50:05 +02:00
terence tsao
b928e9531c Two more gossip validations for unaggregated atts subnet (#7278)
* Add two more gossip validations for unaggregated attestations

* Removed a typo log

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-09-20 08:31:10 +08:00
terence tsao
1f6afa8547 Add config PROPORTIONAL_SLASHING_MULTIPLIER (#7275)
* Add config
* Merge refs/heads/master into prop-slashing-mul
* Fixed a typo reported by Radek
* Merge branch 'prop-slashing-mul' of github.com:prysmaticlabs/prysm into prop-slashing-mul
2020-09-19 22:04:24 +00:00
terence tsao
303edbde58 Update spec tests to v0.12.3 (#7276)
* Update urls
* Update sha tags
* Fix paths
* Delete removed test
* Skipping  with an issue
2020-09-19 18:00:28 +00:00
dv8silencer
208ea56a9c Improve error message when creating a wallet in place of one that already exists (#7268)
* Tell user if content already exists at wallet create destination
* Added test
* changed wording
* capitalization
* Merge branch 'master' into dv8-i7264
* Merge branch 'master' into dv8-i7264
* Address PR comments

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Address PR comments

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Address PR comments

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Address PR comments

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Address PR comments

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Minor fix
* Improve test by addressing PR feedback.  Change variable name
* address PR comments
2020-09-18 21:01:46 +00:00
Victor Farazdagi
c8b91ba7b0 Attestation proposing: more tests (#7267)
* refactors test
* Merge branch 'master' into optimize-proposals
* more filtering and sorting proposer atts
* Merge branch 'master' into optimize-att-proposer
* Merge branch 'master' into optimize-att-proposer
* revert
* Terence's suggestion on naming
2020-09-18 17:16:31 +00:00
terence tsao
268df90ddd Update weak subjectivity check point computation (#7251) 2020-09-18 08:06:00 -07:00
Nishant Das
749aba8d09 Fix Blst Signature Copy Test (#7272)
* fix signature test
* Merge branch 'master' into fixLastTest
2020-09-18 12:04:39 +00:00
Shay Zluf
7920afb12e Add missing error handling in slasher (#7270)
* Add missing error handling in slasher
* Merge branch 'master' into missing_error
* Merge refs/heads/master into missing_error
2020-09-18 10:09:29 +00:00
Radosław Kapka
cc147c7097 Align eth1data Majority Vote with the spec (#7200)
* align voting with the spec
* Merge branch 'master' into eth1vote-spec-align
* remove redundant else statements
* add comment to exported variable
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
* Merge branch 'master' into eth1vote-spec-align

# Conflicts:
#	beacon-chain/rpc/validator/proposer_test.go
* refactor tests
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
* fix mock POWChain
* move last block's time check and add more test cases
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
* Merge refs/heads/master into eth1vote-spec-align
2020-09-18 09:46:31 +00:00
Preston Van Loon
852082cdb4 Check that the headers are equal while ignoring the signature for proposer slashings (#7252)
* Check that the headers are equal while ignoring the signature for proposer slashings. Resolves https://github.com/sigp/beacon-fuzz/issues/74

* fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-09-18 17:10:14 +08:00
Preston Van Loon
e927a3d170 Streaming logs: add backfill of messages for new connections" (#7265)
* add backfill of logs

* Add test and gofmt
2020-09-17 19:34:03 -05:00
Raul Jordan
7c2096f209 Websocket Connection Fixes (#7263)
* socket improvements
* Merge branch 'master' into socket-fix
* Merge refs/heads/master into socket-fix
2020-09-17 19:55:34 +00:00
terence tsao
dcdf5d0eac Add and verify weak-subjectivity-checkpoint flag (#7256)
* Handle weak subjectivity input

* Typo

* Feedbacks

* Better error string @rauljordan

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-09-17 14:32:40 -05:00
terence tsao
3db6784990 Update Spadina fork version to 0x02 (#7261)
* Update Spadina fork version to 0x02
2020-09-17 18:50:01 +00:00
terence tsao
d51ead76eb Implement and test GetWeakSubjectivityCheckpoint (#7216)
* Implement and test GetWeakSubjectivityCheckpoint

* Go mod tidy

* Add mock

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-09-17 12:50:31 -05:00
Radosław Kapka
a2cf235687 No-op Instruction analyzer (#7249)
* analyzer, tests and fixes
* error check
* gazelle
* add more asserts
* Merge branch 'master' into nop-analyzer
* Merge refs/heads/master into nop-analyzer
* fix no-op in blst package
* Merge refs/heads/master into nop-analyzer
* add assert after mutation
* mark test as TODO
* add issue number to TODO
2020-09-17 16:18:19 +00:00
Victor Farazdagi
e1e233a6d0 Fix gazelle warning when fuzzing_enabled is not provided (#7260)
* adds fuzzing_enabled configurable files to gazelle ignore list
* more fine grained
2020-09-17 12:58:03 +00:00
Victor Farazdagi
76bac74562 Disables resync when init-sync is disabled (#7259)
* disables resync when init-sync is disabled
2020-09-17 11:42:57 +00:00
Nishant Das
a335bbbb61 Add Flag To Disable Initial Sync (#7258)
* add new flag
* terence's review
2020-09-17 06:11:21 +00:00
Nishant Das
913e4aa538 Fix IPV6 Address Retrieval (#7257)
* fix
2020-09-17 03:02:20 +00:00
Raul Jordan
4b4641bae3 Restructure Prysm Wallet V2 Package to Prevent Import Cycles (#7247)
* fix up tests

* fix test build

* tests pass

* imports and remove shadowing

* lint

* visibility

* lint fix

* wallet init

* remove v22

* fix imports

* fixed broken password test

* Update validator/accounts/v2/wallet_create.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>

* Update validator/rpc/server.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>

* Update validator/node/node.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>

* fmt

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
2020-09-16 20:34:42 -05:00
Raul Jordan
6bfb2b1e46 Use Bufio Large Buffer Instead of Scanner for Large Files in E2E (#7254)
* use readline
* Fix
* max file buffer size
2020-09-16 23:41:41 +00:00
dv8silencer
a97de0d474 Use passphrase for HD-wallets (#7222)
* Use wallet password
* Fix test
* Address another test
* Address linter's feedback.
* Merge branch 'master' into dv8-i7213
* Merge branch 'master' into dv8-i7213
* Delete temp file
* Merge branch 'master' into dv8-i7213
* Merge branch 'master' into dv8-i7213
* Merge branch 'master' into dv8-i7213
* Merge branch 'dv8-i7213' of https://github.com/dv8silencer/prysm into dv8-i7213
* Merge branch 'master' into dv8-i7213
* Add regression test.
* Skip mnemonic confirmation
* Merge branch 'master' into dv8-i7213
* Merge branch 'master' into dv8-i7213
* Merge branch 'master' into dv8-i7213
* Merge branch 'master' into dv8-i7213
2020-09-16 20:12:16 +00:00
terence tsao
f6d3c28ae1 Save cached blocks before Stop() in blockchains service (#7245)
* Save cached blocks before Stop()
* Merge refs/heads/master into save-cached-blocks
* Merge refs/heads/master into save-cached-blocks
* Merge refs/heads/master into save-cached-blocks
* Merge refs/heads/master into save-cached-blocks
* Merge refs/heads/master into save-cached-blocks
* Regression test
* Merge branch 'save-cached-blocks' of github.com:prysmaticlabs/prysm into save-cached-blocks
2020-09-16 16:55:14 +00:00
Raul Jordan
3197748240 Streaming Logs via Websocket Server (#7240)
* Log streaming proof of concept

* fix broken imports

* imports

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-09-16 09:03:52 -05:00
Preston Van Loon
14dbc2b74d Add blst for BLS (#6539)
* Add blst third party dep

* initial build

* add init

* blst passing tests

* add feature flag

* blst and herumi for spec tests

* maybe this works for mac

* Actually set feature flag

* Add stub for VerifyMultipleSignatures for blst

* verifyCompressed

* use correct cores sizes

* aggregate public keys

* add multi-sig verification

* encode not hash

* revert back

* go mod tidy

* update blst to latest commit

* add batch decompress

* fix

* add test

* gofmt

* update blst

* go mod tidy

* remove kubesec, fix

* mod tidy

* disable some remote cache

* disable some remote cache

* disable some remote cache

* disable some remote cache

* Switch to -D__ADX__

* update

* tidy

* fix build

* Make blst for only linux,amd64

* gofmt

* lint

* lint

* gazelle

* fix build tag

* more stub methods

* shift adx instructions to x86

* fix arm64

* Revert "fix arm64"

This reverts commit 4d34ac21b7.

* add one more in

* Revert "Revert "fix arm64""

This reverts commit 1c8ae24ad1.

* try darwin now

* Revert "try darwin now"

This reverts commit 6f884714b8.

* Add sha256

* remove TODO

* checkpoint

* finally builds

* fix up

* add tag

* try again

* explicit disabling

* remove

* select properly

* fix

* better

* make CI happy too

* Update .bazelrc

* Update .bazelrc

* fix tests

* revert back

* Update shared/bls/blst/public_key.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

* Update shared/bls/blst/public_key.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

* clean up tests

* more clean up

* clean up

* add

* Update shared/bls/blst/signature.go

* Update shared/bls/blst/signature.go

* Update .buildkite-bazelrc

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* try again

* remove go tag

* revert change

* gaz

* gazelle ignore

Co-authored-by: nisdas <nishdas93@gmail.com>
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-09-16 21:28:28 +08:00
terence tsao
b0917db4c7 Core pkg: properly return errs (#7246)
* Return errs instead of nils

* Update a few more tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-09-15 22:55:57 -05:00
terence tsao
ee4ebe4c38 Opt out batch verify (#7244) 2020-09-15 18:53:51 -07:00
Nishant Das
56d6e05196 QSP-16 Fix Support for IPv6 (#6363)
* checkpoint changes

* new changes

* comment

* clean up

* set

* fix

* fix more

* more stuff

* clean up

* clean up

* remove

* clean up

* clean up

* comment

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-09-16 08:17:22 +08:00
terence tsao
bb374362e4 Opt out --use-check-point-cache (#7242)
* Make check point info cache an opt-out
* Typo
* Merge branch 'master' of github.com:prysmaticlabs/prysm into cp-cache-opt-out
2020-09-15 17:14:04 +00:00
dv8silencer
c1114fa6be Fix issue with direct wallet's password being overwritten by incorrect password during import. (#7212)
* Maintain working password instead of overwriting it.

* Check for error first

* fix test

* Address linter

* TEMP, INCOMPLETE;  Working on regression test

* WORK IN PROGRESS - changed promptutil to create a mock password reader

* Test working now

* Address testing bugs

* Spacing

* Comments.  Fixed counter bug.

* gazelle fix

Co-authored-by: dv8silencer <15720668+dv8silencer@users.noreply.github.com>
2020-09-15 09:52:07 -05:00
terence tsao
36c921c601 Add spadina config (#7235) 2020-09-14 21:21:46 -07:00
dv8silencer
f31f49582b Fix accounts-v2 deposit and related bugs (#7219) 2020-09-14 16:04:22 -07:00
Preston Van Loon
cebb62997d Add beacon state unmarshal fuzzer, afl support (#6625)
* Add AFL third_party libraries

* add beacon state fuzzing, add afl fuzz bundle

* rm fuzzing engine

* fix and lint

* Check for array out of bounds when calculating proposer delta

* failing test

* fix

* Checkpoint progress

* Add requirement that inclusion distance is not zero, add regression test

* No need for HTR since that is covered in process slots

* Removing some fuzzit logic, old fuzz tests

* Add ssz encoder test and fix

* Fuzzing checkpoint, adding fuzzing to the p2p layer

* ignore some libfuzzer files

* Full testing of p2p processing of blocks, with some mocked stuff

* use tmpdir and always process blocks

* use checkptr

* Update ethereumapis

* go mod tidy

* benchmarks for ferran's fast ssz hash tree root

* Update fastssz

* fmt

* gaz

* goimports

* Fix

* fix ethereumapis

* fix again

* kafka

* fix gen file

* fix compute signing root

* gofmt

* checkpoint progress

* progress

* checkpoint

* updates

* updates

* merge fix

* WIP

* merge

* fix build

* fix merge related issues

* cleanup

* revert unrelated

* lint

* lint

* lint

* manual tags for fuzz

* Commentary on upload script

* some import fixes, but not all

* fix //fuzz:fuzz_tests

* rm unused test

* update generated ssz

* Set // +build libfuzzer

* remove debug code

* A bit of refactoring ot explain why there is a committee_disabled file

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-09-14 11:42:08 -07:00
Victor Farazdagi
e477df321c Updates proposer tests namings (#7234)
* adds proposer_ namespacing
* remove redundant string
* Merge branch 'master' into proposer-test-fix-naming
* Merge refs/heads/master into proposer-test-fix-naming
2020-09-14 12:55:20 +00:00
Radosław Kapka
828156b9d4 fix exit comment (#7233)
* fix exit comment
* Merge branch 'master' into fix-exit-comment
* Merge refs/heads/master into fix-exit-comment
2020-09-14 11:44:50 +00:00
Radosław Kapka
aaa3abf630 Shadowed Predeclared Indentifier analyzer (#7215)
* wip

* working for some ast nodes

* works for all but assignment

* works for all but mixed assignments

* change test name

* plug in the analyzer

* remove `copy` from predeclared list

* rename few shadowing names

* rename `len` to `length`

* add one more test case

* rename `panic` to `panicResult`

* replace `panic` with `panicResult` in error message

* remove test case not covered by the tool

* add `byte` to predeclared list

* additional test cases

* rename `copy` to `copyHandler`

* exclude functions with receivers

* revert to good names

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-09-14 12:49:15 +02:00
terence tsao
e1aa920fc6 Fix out of bound check in AncestorRoot (#7226)
* Move out of bound check to the correct scope
* Merge refs/heads/master into store-out-of-bound
2020-09-14 09:10:52 +00:00
Shay Zluf
fcfd828725 Local protection proposal schema update (#7197)
* Local protection schema update
* Merge branch 'master' of github.com:prysmaticlabs/prysm into proposal_protection_schema_update
* fix interface issue
* fix lint
* gaz
* Merge refs/heads/master into proposal_protection_schema_update
* Merge refs/heads/master into proposal_protection_schema_update
* Merge refs/heads/master into proposal_protection_schema_update
* Merge refs/heads/master into proposal_protection_schema_update
* Merge refs/heads/master into proposal_protection_schema_update
* Merge refs/heads/master into proposal_protection_schema_update
* Merge refs/heads/master into proposal_protection_schema_update
* Merge refs/heads/master into proposal_protection_schema_update
* Merge refs/heads/master into proposal_protection_schema_update
* Merge refs/heads/master into proposal_protection_schema_update
* Merge refs/heads/master into proposal_protection_schema_update
* Update validator/db/kv/new_proposal_history_test.go
2020-09-14 01:55:33 +00:00
terence tsao
3374a06f63 Skip initial roughtime result when EnableRoughtime is disabled (#7221) 2020-09-13 08:53:14 -07:00
Jim McDonald
0b497e57e4 Reverse error condition. (#7217)
* Reverse error condition.
2020-09-11 20:21:50 +00:00
Shay Zluf
af46fc7707 Fix propose protect error handling (#7188)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-09-11 09:53:53 -05:00
Radosław Kapka
b1e2238df9 Invert enable-finalized-deposits-cache (#7214)
* make flag opt-out
2020-09-11 09:49:24 +00:00
Shay Zluf
acddb6035a Fix rename issue (#7205)
* Fix rename issue

* nishant feedback

* add test

* Update shared/fileutil/fileutil.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* preston feedback

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-09-11 00:56:58 +03:00
terence tsao
3147a5ee01 Check attestation slot before broadcast (#7209) 2020-09-10 13:16:35 -07:00
Victor Farazdagi
572227d25e Removes attestation pool pruning from UnaggregatedAttestations getter (#7204)
* removes att pool changing in getter
* test
* add prunning
* Merge refs/heads/master into no-pruning-in-unagg-att-getter
* Merge refs/heads/master into no-pruning-in-unagg-att-getter
* Merge refs/heads/master into no-pruning-in-unagg-att-getter
* Merge refs/heads/master into no-pruning-in-unagg-att-getter
2020-09-10 18:35:57 +00:00
terence tsao
84273e9a34 Implement helper method WeakSubjectivityCheckptEpoch (#7201)
* Implement and test WeakSubjectivityCheckptEpoch
* Merge refs/heads/master into get-wss
* Merge refs/heads/master into get-wss
* Merge refs/heads/master into get-wss
* Merge refs/heads/master into get-wss
* Update beacon-chain/core/helpers/slot_epoch.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-09-10 17:37:52 +00:00
Victor Farazdagi
114fac5149 Att aggregator filter by slot and index (#7206)
* Adds AggregateUnaggregatedAttestationsBySlotIndex method
* tests
* filter attestations by slot/index for aggregator to process
* Merge branch 'master' into att-aggregator-filter-by-slot-and-index
* Merge refs/heads/master into att-aggregator-filter-by-slot-and-index
* Merge refs/heads/master into att-aggregator-filter-by-slot-and-index
2020-09-10 15:55:45 +00:00
Victor Farazdagi
2d9fe5f2cf Disallows bitlist duplicates in seen_bits cache (#7203)
* disallows bitlist duplicates in seen_bits cache
* Merge refs/heads/master into upd-has-seen
2020-09-10 15:24:23 +00:00
Radosław Kapka
6daf45cb8f Correct accounts and validator error messages (#7207)
* incorrect account password
* misleading exit error
2020-09-10 14:49:13 +00:00
terence tsao
c8e93f8789 Verify indices in no verify method (#7199) 2020-09-09 13:38:19 -07:00
Radosław Kapka
94fa046ce1 Voluntary exit CLI/Client integration (#7162)
* rewrite of ProposeExit

# Conflicts:
#	validator/accounts/v2/accounts_exit.go

* translate todos to English

* use node client to calculate current epoch

* resolve dependency cycle

* add missing dependencies

* changed required epochs error message

* fix ProposeExit tests

* remove unused metrics

* simplify public-keys flag name

* organize CLI function

* fix incorrect list of exited keys

* fix formatting of non-exited keys

* add exit root to span attributes

* use errors instead of logs

* log info when all exits failed

* do not log stack trace for non-critical errors

* modify test asserts

* use standard way of displaying errors

* remove todo

* add missing function to wallet mock

* gazelle

* gazelle after stash

* move creating clients to separate function

* gazelle
2020-09-09 13:29:17 -05:00
Preston Van Loon
a74cf5de90 Replace context.Background() with more appropriate context (#7136)
* Replace context.Background() with more appropriate context
* replace a few context.TODO
* Merge refs/heads/master into fix-ctx
* Update validator/accounts/v2/accounts_create.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Fix tests
* fix stream tests
* gofmt
* Merge refs/heads/master into fix-ctx
* Merge refs/heads/master into fix-ctx
* Merge refs/heads/master into fix-ctx
* Merge refs/heads/master into fix-ctx
* Merge refs/heads/master into fix-ctx
* Merge refs/heads/master into fix-ctx
* Merge refs/heads/master into fix-ctx
* Merge refs/heads/master into fix-ctx
* fix conflicts and remove ctx background uses
* fix broken test
* Merge branch 'master' into fix-ctx
* imports
* Merge branch 'fix-ctx' of github.com:prysmaticlabs/prysm into fix-ctx
* Merge refs/heads/master into fix-ctx
* Merge refs/heads/master into fix-ctx
* fix conflicts
* Merge refs/heads/master into fix-ctx
* fmt
* Merge refs/heads/master into fix-ctx
* Merge refs/heads/master into fix-ctx
* Merge refs/heads/master into fix-ctx
* Merge refs/heads/master into fix-ctx
* Merge refs/heads/master into fix-ctx
* fixes tests
2020-09-09 09:48:52 +00:00
Raul Jordan
6d83770534 Fix Small Issue in Validator RPC Health Endpoint (#7195)
* amend node connection response
* imports
* fix err
* fix conf
* err
* test fix
* Merge branch 'master' into fix-health-endpoint
2020-09-08 22:35:31 +00:00
terence tsao
593442a0fa Fix span names in transition.go (#7192)
* Fix span names

* Go fmt

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-09-08 14:36:02 -07:00
Raul Jordan
bd46abc71d Add Chain Genesis Info to Node Connection Response (#7194)
* amend node connection response
* imports
2020-09-08 20:54:56 +00:00
Raul Jordan
cb1f44872d Fix Health Reporting for Validator Client Node Connection (#7181)
* fix health reporting
* import
* Merge refs/heads/master into fix-health
* account test
* Merge branch 'fix-health' of github.com:prysmaticlabs/prysm into fix-health
* Merge refs/heads/master into fix-health
* Merge refs/heads/master into fix-health
* Merge refs/heads/master into fix-health
* Merge refs/heads/master into fix-health
* Merge branch 'fix-health' of github.com:prysmaticlabs/prysm into fix-health
* Merge refs/heads/master into fix-health
* Merge branch 'fix-health' of github.com:prysmaticlabs/prysm into fix-health
* Merge refs/heads/master into fix-health
* fmt
* Merge refs/heads/master into fix-health
2020-09-08 20:02:12 +00:00
Victor Farazdagi
8baa22f065 Fix unclosed tickers/timers (#7190)
* fix resource leak
* fixes leak in blocks fetcher
* client/validator release ticker resorces
* powchain, more straightforward ticker closing
* adds missing ticker.stop() calls
* more straightforward ticker closing
* Merge refs/heads/master into fix-unclosed-tickers-timers
* Merge refs/heads/master into fix-unclosed-tickers-timers
* Merge refs/heads/master into fix-unclosed-tickers-timers
* gofmt issues introduced in https://github.com/prysmaticlabs/prysm/pull/7176
2020-09-08 18:05:38 +00:00
Shay Zluf
f4848e46d4 Safer validator protection db dir (#7176)
* Safer validator protection db dir
* handle old account structure
* Merge refs/heads/master into move_validator_protection_db
* Merge refs/heads/master into move_validator_protection_db
* Merge refs/heads/master into move_validator_protection_db
* Merge refs/heads/master into move_validator_protection_db
* fix e2e issue
* separate function
* Merge branch 'move_validator_protection_db' of github.com:prysmaticlabs/prysm into move_validator_protection_db
* Merge branch 'master' of github.com:prysmaticlabs/prysm into move_validator_protection_db
* Merge refs/heads/master into move_validator_protection_db
* Merge refs/heads/master into move_validator_protection_db
* Merge refs/heads/master into move_validator_protection_db
* Update shared/fileutil/fileutil.go
* Merge refs/heads/master into move_validator_protection_db
2020-09-08 16:42:06 +00:00
terence tsao
f04fffb5fe Revert "Pipelines for ProcessBlockNoVerifyAttSigs and ProcessBlockStateRoot" (#7191)
* Revert "Pipelines for `ProcessBlockNoVerifyAttSigs` and `ProcessBlockStateRoot` (#7180)"

This reverts commit 9219dc77ff.
2020-09-08 15:45:20 +00:00
Victor Farazdagi
51f2cc18e5 Improves asyncutils/Debounce (#7189)
* improves asyncutils/debounce
* better handling of timeout test
* more tests
2020-09-08 14:49:28 +00:00
terence tsao
3de626f0d0 Prevent zero hashes sigs get submitted (#7187) 2020-09-07 12:53:30 -07:00
Nishant Das
a81214219d Hash Pubsub Messages Correctly (#7177)
* check message id

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-09-07 02:05:16 +08:00
terence tsao
245c18784e Ensure aggregateAndProofSig returns when err'ing (#7182)
* Return if sign fails
2020-09-06 06:56:17 +00:00
terence tsao
9219dc77ff Pipelines for ProcessBlockNoVerifyAttSigs and ProcessBlockStateRoot (#7180) 2020-09-04 16:24:43 -07:00
Raul Jordan
1de230110d Add Beacon Gateway Flag to Validator Client (#7179)
* add in cors for web ui in val client and gateway flag
* Merge branch 'master' into add-default-beacon-gateway
* gateway cors value
* Merge branch 'add-default-beacon-gateway' of github.com:prysmaticlabs/prysm into add-default-beacon-gateway
* fix tests
* Merge refs/heads/master into add-default-beacon-gateway
2020-09-04 19:03:18 +00:00
Victor Farazdagi
b6607fac25 Fixes timer resources leak in CountdownToGenesis (#7175)
* fixes timer leak in CountdownToGenesis
* Merge branch 'master' into fix-unclosed-timer-in-countdown
* Merge refs/heads/master into fix-unclosed-timer-in-countdown
2020-09-04 18:07:50 +00:00
terence tsao
e6277ec536 Revert 6966 (#7178)
* revert 6966

* Fix build
2020-09-04 12:25:42 -05:00
Raul Jordan
0961fef727 Implement Create and List Accounts Validator RPC (#7172)
* impl of accounts RPCs
* create and list accounts impls
* impl create account and list accounts
* tests pass
* imports
* fix test
2020-09-04 00:58:36 +00:00
Raul Jordan
366b98ac83 Add Node Connection Endpoint to Validator Client (#7171)
* add sync status checker
* register healthz
* simplify
2020-09-03 23:25:56 +00:00
terence tsao
c2425e81d7 Remove deprecated health end points (#7170)
* Remove deprecated health end points

* Remove fields
2020-09-03 17:39:30 -05:00
terence tsao
8f2950d374 Prevent far future target epoch for attestation state (#7169)
* Prevent far future target epoch

* Regression tests
2020-09-03 13:18:38 -05:00
Raul Jordan
f4a6864343 Prepare Validator Client for Web Usage (#7158)
* prepare validator for web usage
* wallet exists func in rpc server
* only initialize wallet if feed exists and wallet also exists
* enabling web via a feed
* wallet creation works but needs to be able to recover from mnemonic via api
* ensure logic works
* Merge branch 'master' into validator-web
* fix up tests
* test file
* wallet rpc tests
* add all tests to create wallet
* Merge branch 'master' into validator-web
* imports
* Merge branch 'validator-web' of github.com:prysmaticlabs/prysm into validator-web
* Merge refs/heads/master into validator-web
* Merge refs/heads/master into validator-web
2020-09-03 15:11:17 +00:00
Shay Zluf
c1a7c65e05 Deprecate beacon chain slasher flags (#7031)
* Deprecate beacon chain slasher flags
* Merge branch 'master' into deprecate_beacon_slasher_flags
* remove comments
* Merge branch 'deprecate_beacon_slasher_flags' of github.com:prysmaticlabs/prysm into deprecate_beacon_slasher_flags
* hidden
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* remove start slasher client
* de-depracate an in use flag in validator
* Merge branch 'master' of github.com:prysmaticlabs/prysm into deprecate_beacon_slasher_flags

# Conflicts:
#	shared/featureconfig/flags.go
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
* Merge refs/heads/master into deprecate_beacon_slasher_flags
2020-09-03 14:12:43 +00:00
Victor Farazdagi
7fd2536d54 Hardens block validation in init-sync (#7168)
* assert response invariants in beaconBlocksByRange
* Merge branch 'master' into assert-invariants-in-beaconblocksbyrange
* enforce specs invariant in requestBlocks
* blocks queue: onDataReceivedEvent - part1
* Merge branch 'master' into init-sync-block-validation
* blocks queue: onDataReceivedEvent - part2
* Merge branch 'master' into init-sync-block-validation
* block fetcher: test returned blocks validation
* gazelle
* Nishant's suggestion to not loop more than req.count of times
* combine checks
* enforce slot ordering
* enforce skip increments
2020-09-03 07:07:42 +00:00
terence tsao
0e6797d80d Set head as finalized root (instead of archived root) on start (#7167)
* Set head to finalized
* Merge branch 'master' into set-head-to-finalized
* Fixed testtttts
* Merge branch 'set-head-to-finalized' of github.com:prysmaticlabs/prysm into set-head-to-finalized
* Merge refs/heads/master into set-head-to-finalized
2020-09-02 22:43:40 +00:00
Victor Farazdagi
b2b4c2660d Asserts response invariants in beaconBlocksByRange (#7166)
* assert response invariants in beaconBlocksByRange
* Merge branch 'master' into assert-invariants-in-beaconblocksbyrange
2020-09-02 21:59:57 +00:00
Nishant Das
787857c38b Verify Attestation Time Correctly (#7160)
* add changes
* add unit test
* Merge refs/heads/master into fixVerifyAttTime
2020-09-02 19:21:52 +00:00
terence tsao
1cc21ed3c4 Check finalized ancestor (#7165) 2020-09-02 13:38:03 -05:00
terence tsao
7588e491ab Add overflow check for StartSlot (#7149)
* Add overflow checks
* More fixes
* Typos
* Merge branch 'master' into fix-overflow
* One more regression test
* Merge branch 'fix-overflow' of github.com:prysmaticlabs/prysm into fix-overflow
* Gazelle
* Merge branch 'master' into fix-overflow
* Nishant's feedback
* Merge refs/heads/master into fix-overflow
2020-09-02 02:52:36 +00:00
Raul Jordan
afce363e00 Update Protos for Wallet Creation (#7157)
* web api changes
* Merge branch 'master' into wallet-creation-protos
* protos checked in
* Merge branch 'wallet-creation-protos' of github.com:prysmaticlabs/prysm into wallet-creation-protos
2020-09-02 02:13:42 +00:00
Nishant Das
7de3ce0b31 Fix Snappy Framed Reader (#7152)
* add checks
* Merge refs/heads/master into fixFramedSnappy
* Merge refs/heads/master into fixFramedSnappy
* Merge refs/heads/master into fixFramedSnappy
* Merge refs/heads/master into fixFramedSnappy
2020-09-02 01:29:46 +00:00
terence tsao
6e6b871cc1 Use state slot to verify exit signature (#7155)
* Use state slot
* TestValidateVoluntaryExit_InvalidExitSlot
* Fixed TestProposeExit_NoPanic
* Merge refs/heads/master into use-correct-slot
2020-09-02 00:57:52 +00:00
Nishant Das
2349012bd0 Handle Zero Root Requests (#7151)
* handle zero root requests
* clean up
* Merge refs/heads/master into handleZeroRoots
* Merge refs/heads/master into handleZeroRoots
* Merge branch 'master' into handleZeroRoots
2020-09-01 19:09:09 +00:00
RoBiK75
b4c0a89d49 Fixes incorrect output produced by the ListAccounts function (#6976) (#7095)
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-09-01 13:13:44 -05:00
Nishant Das
d368156a08 Remove Deprecated Request Type (#7153)
* remove deprecated type
* fix build and tests
* update log
* Merge refs/heads/master into removeBlocksByRootType
2020-09-01 13:35:30 +00:00
Victor Farazdagi
63149a3dc3 Removes redundant SignedAggregateAndProof declaration (#7154)
* removes redundant SignedAggregateAndProof declaration
2020-09-01 13:11:05 +00:00
terence tsao
fbe088625a Update ProcessBlock to use a pipeline (#7148) 2020-08-31 18:29:27 -07:00
Raul Jordan
ecbab20bad Begin Dynamic Rescan of Validating Keys (#6963)
* begin on dynamic key rescan
* Merge branch 'master' into dynamic-rescan
* begin dynamic rescan
* fsnotify to listen for rescan dir changes
* recheck for slashing protection
* lint
* Merge branch 'master' into dynamic-rescan
* less aggressive recheck interval
* imports
* Merge branch 'dynamic-rescan' of github.com:prysmaticlabs/prysm into dynamic-rescan
* resolve confs
* listen for file changes for accounts file
* reload accounts from keystore
* begin fixing rescan test
* add event feed
* fix confs
* fix conf
* fix broken tests
* Merge branch 'master' into dynamic-rescan
* simplify lines
* do nothing if no subscribers
* Merge branch 'dynamic-rescan' of github.com:prysmaticlabs/prysm into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* fix tests
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* gaz
* Merge branch 'dynamic-rescan' of github.com:prysmaticlabs/prysm into dynamic-rescan
* ident
* Update WORKSPACE
* gaz
* Merge branch 'dynamic-rescan' of github.com:prysmaticlabs/prysm into dynamic-rescan
* add keys on service start
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* ensure debounce util works
* Merge branch 'dynamic-rescan' of github.com:prysmaticlabs/prysm into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* complete refresh, debounce test, and ensure works at runtime
* Merge branch 'dynamic-rescan' of github.com:prysmaticlabs/prysm into dynamic-rescan
* imports and remove log
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* resolve confs
* fix up e2e tests
* Merge branch 'dynamic-rescan' of github.com:prysmaticlabs/prysm into dynamic-rescan
* fix up e2e
* Merge refs/heads/master into dynamic-rescan
2020-08-31 23:38:20 +00:00
Michael Huang
381b5be0fc Fix partially deposited status bug (#7138)
* Fix paritally deposited bug
* Replace conditional logic with depositStatus
* Add test
* Merge branch 'master' into fix-partial-dep-bug
* Merge branch 'master' into fix-partial-dep-bug
* Merge branch 'master' into fix-partial-dep-bug
* Merge branch 'master' into fix-partial-dep-bug
* Merge branch 'master' into fix-partial-dep-bug
2020-08-31 22:45:20 +00:00
Radosław Kapka
6803f3308a Implement exit proposal in validator client (#7125)
* Create mocks for updated ethereumapis

* Revert "Create mocks for updated ethereumapis"

This reverts commit 0da06d5158.

* update ethereumapis

* new metrics

* ProposeExit implementation and tests

* fix comments for metrics

* assert errors

* added one more error assert

* remove redundant comments

* make metric names compliant with best practices

* change root calculation

* fix test assertion

* fix test assertion

* check if domain is not nil

* extract error messages

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-08-31 15:18:22 -05:00
Raul Jordan
60558b7970 Refactor Accounts and Keymanager Methods to Not Rely on CLI (#7135)
* keymanagers no longer use cli ctx
* rename important values to keymanageropts
* further refactor accounts methods to reduce cli dependency
* separating cli vs non cli methods for various accounts functions
* recover wallet cli vs non-cli mode
* ensure half of tests build
* accounts v2 package now builds
* full revamp wallet creation or opening wallets
* everything builds
* tests pass finally
* Merge branch 'master' into no-cli-keymanagers
* ensure commands work as expected
* Merge branch 'no-cli-keymanagers' of github.com:prysmaticlabs/prysm into no-cli-keymanagers
* further fix build
* account creation comments
* fix imports and comments
* fix up failing test
* fix build
* Update derived.go
* Update direct.go
* Update remote.go
* Fixed variable
* fix up red tests
* use Cli instead of CLI, fix tests, and address Radek's feedback
* Merge refs/heads/master into no-cli-keymanagers
* Merge refs/heads/master into no-cli-keymanagers
* Merge refs/heads/master into no-cli-keymanagers
* Merge refs/heads/master into no-cli-keymanagers
* Merge refs/heads/master into no-cli-keymanagers
* Merge refs/heads/master into no-cli-keymanagers
* Merge refs/heads/master into no-cli-keymanagers
* Merge refs/heads/master into no-cli-keymanagers
2020-08-31 19:46:45 +00:00
Ivan Martinez
7854b91ae0 E2E: Change metrics detection to use regex (#7145)
* Change metrics detection to use regex

* Fixes, remove println

* Forgot a println

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-08-31 13:35:26 -05:00
Victor Farazdagi
c9c7cc7b93 Fixes comments in params/config (#7147)
* fixes comments in params/config
* Merge refs/heads/master into params-doc-fix
* Merge refs/heads/master into params-doc-fix
2020-08-31 15:25:05 +00:00
terence tsao
b538f5073d Add metrics (#7142)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-08-31 07:51:47 -07:00
Nishant Das
3ed7b23ed7 Handle Saved Genesis Interop States (#7146)
* handle interop states
* lint
* Merge refs/heads/master into handleInteropCase
* fix test
* Merge branch 'handleInteropCase' of https://github.com/prysmaticlabs/geth-sharding into handleInteropCase
2020-08-31 13:45:03 +00:00
Nishant Das
c2b94d04ed Deprecate DisableNewStateMgmt Flag (#7124)
* deprecate flag
* gaz
* Merge refs/heads/master into deprecateFlag
* Merge refs/heads/master into deprecateFlag
* Merge refs/heads/master into deprecateFlag
* Merge refs/heads/master into deprecateFlag
* Merge refs/heads/master into deprecateFlag
* Merge refs/heads/master into deprecateFlag
* Merge refs/heads/master into deprecateFlag
* Merge refs/heads/master into deprecateFlag
* Merge refs/heads/master into deprecateFlag
* Merge refs/heads/master into deprecateFlag
* Merge refs/heads/master into deprecateFlag
* Merge refs/heads/master into deprecateFlag
* Merge refs/heads/master into deprecateFlag
* Merge refs/heads/master into deprecateFlag
* add check back
2020-08-31 06:59:13 +00:00
Nishant Das
3316516d22 Harden Slot Time Verification (#7139)
* checkpoint

* lint

* terence's review

* fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-08-31 12:51:15 +08:00
Ivan Martinez
12c1daaf2b Verify deposits made and show network deposit is for (#6996)
* Verify deposits made and show network deposit is for
* Merge branch 'master' of github.com:prysmaticlabs/prysm into verify-deposit-netowrk
* Fix formatting
* gofmt
* Merge branch 'master' of github.com:prysmaticlabs/prysm into verify-deposit-netowrk
* Improve deposit sig failure error
* Merge branch 'master' of github.com:prysmaticlabs/prysm into verify-deposit-netowrk
* Fix fmt
* Add "verified for" text
* Update shared/params/config.go
* Merge branch 'master' into verify-deposit-netowrk
* Merge branch 'master' into verify-deposit-netowrk
2020-08-30 16:39:53 +00:00
terence tsao
f09620c9f6 Don't include orphaned atts (#7134)
* Don't include orphaned atts
* gazelle
* Merge refs/heads/master into orphaned-atts
2020-08-30 15:13:21 +00:00
Radosław Kapka
e47e7067c4 Update rpc validator exit tests (#7122)
* Create mocks for updated ethereumapis

* Revert "Create mocks for updated ethereumapis"

This reverts commit 0da06d5158.

* update ethereumapis

* change root calculation

* fix test assertion

* fix test assertion
2020-08-28 15:52:40 -05:00
Nishant Das
4edcf92140 Validate Beacon Block By Range Requests Better (#7132)
* checkpoint
* clean up
* new changes
* clean up
* victor's review
* victor's review
* fix test
2020-08-28 13:50:38 +00:00
terence tsao
7463f2cffd Update participation response (#7129)
* Sync with ethapi
* Implementation and test
* Handle mock
* Fix ProposeExit for e2e
* Merge refs/heads/master into participation
* Preston's feedback
* Merge branch 'participation' of github.com:prysmaticlabs/prysm into participation
2020-08-27 23:57:11 +00:00
terence tsao
fea2cc9e2f Cleanup stategen pkg (#7127)
* Clean up stategen
* Sync with master
* Fix tests
* Merge refs/heads/master into cleanup-newstate
* Merge refs/heads/master into cleanup-newstate
* Rename file
* Merge branch 'cleanup-newstate' of github.com:prysmaticlabs/prysm into cleanup-newstate
* gaz
* Forgot to add metrics page back
2020-08-27 22:29:59 +00:00
Victor Farazdagi
bd489e345a Fixes docs.go -> doc.go file name (#7128)
* fixes doc file name
* Merge refs/heads/master into fix-doc-file-name
2020-08-27 20:28:54 +00:00
Michael Huang
b98b9b740b Add PARTIALLY_DEPOSITED status (#7071)
* Update ethereumapis

* Add PARTIALLY_DEPOSITED status

* go mod tidy

* Fix bug

* Fixup

* Fixup

* Fix tests

* Revert fast ssz bump

* Pending status if deposit amount is 0

* go mod tidy

* More tests

* Trigger Buildkite

* Fix TestMultipleValidatorStatus_Indices

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-08-27 14:41:32 -05:00
Preston Van Loon
b7175b3482 Update fastssz: Attempt 2 (#7115)
* Revert "Revert "Update fastssz" (#7100)"

This reverts commit b954db9704.
* Preston's patch
* Merge branch 'master' of github.com:prysmaticlabs/prysm into revert-7100-revert-6760-update-fssz
* Update fssz, add regression test case
* more HTR with fssz
* fix some tests
* only one test left
* Make it so that HTR will work
* gofmt, imports
* gofmt, imports
* fix
* Merge branch 'master' of github.com:prysmaticlabs/prysm into revert-7100-revert-6760-update-fssz
* fix
* Merge branch 'master' into revert-7100-revert-6760-update-fssz
* Merge refs/heads/master into revert-7100-revert-6760-update-fssz
* gaz
* Merge branch 'revert-7100-revert-6760-update-fssz' of github.com:prysmaticlabs/prysm into revert-7100-revert-6760-update-fssz
* Merge refs/heads/master into revert-7100-revert-6760-update-fssz
* fix test
* Merge branch 'revert-7100-revert-6760-update-fssz' of github.com:prysmaticlabs/prysm into revert-7100-revert-6760-update-fssz
* Merge refs/heads/master into revert-7100-revert-6760-update-fssz
2020-08-27 18:13:32 +00:00
Preston Van Loon
7d28146e1e Return an error if no eth1 endpoint defined and we do not have genesis state (#7126)
* Return an error if no eth1 endpoint defined and we do not have genesis state
* Merge refs/heads/master into fail-without-genesis-state-and-no-eth1-node
2020-08-27 17:18:17 +00:00
Nishant Das
5bfc457904 Request Headers From ETH1 Node Instead of Blocks (#7121)
* request headers instead of blocks
* Merge branch 'master' into changeToHeader
* Apply suggestions from code review

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
* change references
* one more change
* last one
* Merge refs/heads/master into changeToHeader
2020-08-27 16:27:08 +00:00
Nishant Das
13324674ba disable default goerli endpoint (#7120)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-08-27 10:17:21 -05:00
Nishant Das
c78bfab69d Fix Go Builds (#7123)
* change to ssz DeepEqual
* gaz
2020-08-27 13:12:49 +00:00
terence tsao
2ee025ee88 Participation: count atts from orphaned blocks (#7117)
* Cound attestations from orphaned blocks

* Use next epoch's end slot

* Fixed a test

* Fixed a test

* Nishant's feedback

* Typo

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-08-27 16:19:55 +08:00
Radosław Kapka
0410b14260 Add deposit cache and majority vote to dev mode flags (#6768)
* enableFinalizedDepositsCache and enableEth1DataMajorityVote flags
* Merge branch 'master' into add-dev-mode-flags
* Merge branch 'master' into add-dev-mode-flags
* Merge branch 'master' into add-dev-mode-flags
2020-08-27 04:10:36 +00:00
terence tsao
2c3558e449 Remove new state mgmt (#7119) 2020-08-26 21:39:47 -05:00
Nishant Das
e95393f671 fix non existent validators (#7114) 2020-08-26 06:30:30 -07:00
Nishant Das
73cb6daf46 Listens For State Initialized Event (#7112)
* listen for state initialized event
* reg test
* fmt
* Merge refs/heads/master into handleNilHeadState
2020-08-26 10:54:33 +00:00
Nishant Das
8a8edf3743 Clean Up Metadata logs (#7109)
* clean errors

* cleaner

* gazelle

* go check

* Update beacon-chain/sync/error.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-08-26 17:59:16 +08:00
Fabrice Cheng
900e7114da Update VerifyMerkleBranch to match ETH2.0 spec (#6767)
* update VerifyMerkleBranch to match ETH2.0 spec
* Merge branch 'master' into 6705-refactor-verify-merkle-branch
* Make depth a parameter of VerifyMerkleBranch
* Merge branch '6705-refactor-verify-merkle-branch' of github.com-fabdarice:fabdarice/prysm into 6705-refactor-verify-merkle-branch
* Merge branch 'master' into 6705-refactor-verify-merkle-branch
* loop instead of appending
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into 6705-refactor-verify-merkle-branch
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into 6705-refactor-verify-merkle-branch
* fix int conversion
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into 6705-refactor-verify-merkle-branch
* Merge branch '6705-refactor-verify-merkle-branch' of github.com-fabdarice:fabdarice/prysm into 6705-refactor-verify-merkle-branch
* Merge branch 'master' into 6705-refactor-verify-merkle-branch
* fix keymanagerv2 tests
* revert print
* merge conflict
* fix bug during merge
* Merge branch 'master' into 6705-refactor-verify-merkle-branch
* Merge branch 'master' into 6705-refactor-verify-merkle-branch
2020-08-25 20:57:41 +00:00
Ivan Martinez
ba00c55f95 Accounts-V2: Fix wallet-v2 recover and wallet-v2 create for derived (#7104)
* Fix derived to create seed on wallet-v2 create
* Fix derived wallet recover
* Fix comment
* Merge branch 'master' of github.com:prysmaticlabs/prysm into fix-derived-phrase
* Merge refs/heads/master into fix-derived-phrase
* Merge refs/heads/master into fix-derived-phrase
* Merge refs/heads/master into fix-derived-phrase
* Merge refs/heads/master into fix-derived-phrase
* Merge refs/heads/master into fix-derived-phrase
* Update validator/accounts/v2/wallet_create.go
* Merge refs/heads/master into fix-derived-phrase
2020-08-25 19:30:26 +00:00
Victor Farazdagi
8240eb0416 updates bazel-toolchains 3.1.1 -> 3.2.0 (#7107)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-08-25 13:28:37 -05:00
Victor Farazdagi
ba07ccb484 Apply testutils assertions: final cleanup (#7003)
* slasher/beaconclient tests
* slasher/db/kv tests
* Merge branch 'master' into apply-testutils-assertions-to-slasher
* fix build
* slasher/detection tests
* rest of the tests
* misc tests
* tools tests
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge branch 'master' into apply-testutils-assertions-misc
* agg tests
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge branch 'master' into apply-testutils-assertions-misc
* Merge branch 'apply-testutils-assertions-misc' of github.com:prysmaticlabs/prysm into apply-testutils-assertions-misc
* updates aggregated_test
* beacon-chain/operations/attestations/kv/* tests updated
* beacon-chain/operations/attestations tests updated
* beacon-chain/operations/slashings tests updated
* Merge branch 'master' into apply-testutils-assertions-misc
* gazelle
* beacon-chain/core tests updated
* fixes test
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge branch 'apply-testutils-assertions-misc' of github.com:prysmaticlabs/prysm into apply-testutils-assertions-misc
* beacon-chain/rpc tests updated
* beacon-chain/sync/initial-sync tests
* misc tests
* optimizes error message parsing in testutils
* Merge branch 'assertutils-optimize-processing' into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* endtoend tests
* Merge branch 'apply-testutils-assertions-misc' of github.com:prysmaticlabs/prysm into apply-testutils-assertions-misc
* gazelle
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* beacon-chain/blockchain tests updated
* Merge branch 'apply-testutils-assertions-misc' of github.com:prysmaticlabs/prysm into apply-testutils-assertions-misc
* beacon-chain/state/stategen tests updated
* beacon-chain all left-over tests are done
* Merge refs/heads/master into apply-testutils-assertions-misc
* validator tests updated
* slasher tests
* Merge branch 'master' into apply-testutils-assertions-misc
* gofmt
* gazelle
* Merge refs/heads/master into apply-testutils-assertions-misc
* shared upd
* end2end tests deps fixed
* Merge branch 'apply-testutils-assertions-misc' of github.com:prysmaticlabs/prysm into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* misc
* all tests are updated
* Merge branch 'apply-testutils-assertions-misc' of github.com:prysmaticlabs/prysm into apply-testutils-assertions-misc
2020-08-25 15:23:06 +00:00
rkapka
3d9cde3e1c Modify majority vote rules (#6818)
* add deposit guard clauses to majority vote
* Merge branch 'origin-master' into majority-vote-deposit-count
* Merge branch 'master' into majority-vote-deposit-count
* uncommented e2e flag
* fix deposit comparison
* Merge branch 'origin-master' into majority-vote-deposit-count
* changed block height used for 0 deposits
* Merge branch 'master' into majority-vote-deposit-count
* exclude previous period's initial block
* rename first and last blocks
* renamed inRangeVotes arguments
* reorder arguments
* Comments explaining valid range calculation
* Merge branch 'master' into majority-vote-deposit-count
* Comments explaining deposit check
* logging
* process slots before voting for eth1data
* Merge branch 'origin-master' into majority-vote-deposit-count

# Conflicts:
#	beacon-chain/rpc/validator/proposer.go
* Merge branch 'master' into majority-vote-deposit-count
* Merge branch 'master' into majority-vote-deposit-count
* rollback changes to eth1data and randomETH1DataVote
* Merge branch 'master' into majority-vote-deposit-count
* Merge branch 'master' into majority-vote-deposit-count
2020-08-25 14:10:04 +00:00
Nishant Das
b291eb7953 Unset From,Sequence Number and Signature For Pubsub Messages (#7093)
* add new options
* Merge refs/heads/master into removePubsubFields
* Merge refs/heads/master into removePubsubFields
* Merge refs/heads/master into removePubsubFields
* Merge refs/heads/master into removePubsubFields
* Merge refs/heads/master into removePubsubFields
* Merge refs/heads/master into removePubsubFields
* Merge refs/heads/master into removePubsubFields
* Merge refs/heads/master into removePubsubFields
* Merge refs/heads/master into removePubsubFields
* Merge refs/heads/master into removePubsubFields
* Merge refs/heads/master into removePubsubFields
* Merge refs/heads/master into removePubsubFields
2020-08-25 11:08:42 +00:00
terence tsao
f61f02e59b First take of applying assertion funcs to shared tests (#6666)
* cmd tests
* deposit util tests
* feature config tests
* hashutil tests
* htr util tests
* interop tests
* ip util tests
* Update BLS
* Update cmd
* Update bytesutil and depositutil
* Update event
* Update keystore
* Update mathutil
* Update mputil
* Update pagination
* Update params
* Upate prome
* Update testutil
* Update trieutil
* Merge branch 'master' of github.com:prysmaticlabs/prysm into testutil-shared
* Sync with master
* Mod
* Typo
* Revert
* gazelle
* Merge refs/heads/master into testutil-shared
* Merge refs/heads/master into testutil-shared
* Merge refs/heads/master into testutil-shared
* Merge refs/heads/master into testutil-shared
* Gaz
* Merge refs/heads/master into testutil-shared
* Merge refs/heads/master into testutil-shared
* Merge refs/heads/master into testutil-shared
* Merge refs/heads/master into testutil-shared
* Merge refs/heads/master into testutil-shared
* Merge refs/heads/master into testutil-shared
* Merge refs/heads/master into testutil-shared
* fixes build error
* Merge refs/heads/master into testutil-shared
2020-08-25 10:18:29 +00:00
Nishant Das
48fd40e35b fix interop (#7106) 2020-08-25 17:17:06 +08:00
Ivan Martinez
d8ea41ce35 Make Interop keys functional for accounts-v2 (#7103)
* Make Interop functional for accounts v2
* Merge branch 'master' into e2e-accounts-v2
* Fix e2e and account name fetch
* Merge branch 'e2e-accounts-v2' of github.com:prysmaticlabs/prysm into e2e-accounts-v2
* Enable metrics check
* Fix lint
* Merge branch 'master' into e2e-accounts-v2
2020-08-25 01:37:25 +00:00
Nishant Das
e556ac348a Add Shared Field Metrics (#7081)
* add metrics
* Merge branch 'master' into sharedFieldMetrics
* Merge branch 'master' into sharedFieldMetrics
* Merge branch 'master' into sharedFieldMetrics
* Merge branch 'master' into sharedFieldMetrics
2020-08-24 23:03:23 +00:00
terence tsao
c764099231 Use checkpoint info cache (#7098)
* Use checkpoint cache

* Add it to dev

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-08-24 17:11:45 -05:00
terence tsao
b954db9704 Revert "Update fastssz" (#7100)
* Revert "Update fastssz (#6760)"

This reverts commit 78a25f99c3.
* Merge refs/heads/master into revert-6760-update-fssz
2020-08-24 20:06:28 +00:00
Raul Jordan
c9c4cd9f87 Create Validator Accounts V2 Send Deposit (#7080)
* add in deposit logic
* create the deposit functionality
* import formatting and create deposit config
* proceed with retrieving user input for calling the deposit func
* actually send our the deposits
* better handling and comments
* programmatically send all
* lint
* gaz
* add progress bar
* deposit test
* better error handling
* Update validator/accounts/v2/accounts_deposit.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/master into send-deposit
* Merge refs/heads/master into send-deposit
* Merge refs/heads/master into send-deposit
* Merge refs/heads/master into send-deposit
* Merge refs/heads/master into send-deposit
* Merge refs/heads/master into send-deposit
* Merge refs/heads/master into send-deposit
* Merge refs/heads/master into send-deposit
2020-08-24 19:19:21 +00:00
Victor Farazdagi
880298d844 Optimizes error message parsing in testutils (#7096)
* optimizes error message parsing in testutils
* Merge refs/heads/master into assertutils-optimize-processing
* Merge refs/heads/master into assertutils-optimize-processing
2020-08-24 15:15:39 +00:00
Nishant Das
21a56d5419 Handle Nil Signature in Pending Queue (#7097)
* add fix and test

* Update beacon-chain/sync/pending_attestations_queue_test.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-08-24 22:54:58 +08:00
Victor Farazdagi
5c9830f0c3 More conservative selection of non-finalized peers (#7086)
* more conservative selection of non-finalized peers
* Merge refs/heads/master into init-sync-non-finalized-loop
* Nishant's suggestion on possible overflow
* Merge branch 'init-sync-non-finalized-loop' of github.com:prysmaticlabs/prysm into init-sync-non-finalized-loop
* Merge refs/heads/master into init-sync-non-finalized-loop
* Merge refs/heads/master into init-sync-non-finalized-loop
2020-08-24 14:07:21 +00:00
Victor Farazdagi
5d0f6c5b16 Fixes errNoPeersWithFinalizedBlocks naming (#7087)
* fixes errNoPeersWithFinalizedBlocks naming
* Merge refs/heads/master into init-sync-queue-fix-err-naming
* Merge refs/heads/master into init-sync-queue-fix-err-naming
2020-08-24 10:48:52 +00:00
Victor Farazdagi
5cee10f28e Adds DeepNotEqual assertion (#7092)
* adds deepnotequal func
2020-08-24 10:14:13 +00:00
Preston Van Loon
78a25f99c3 Update fastssz (#6760)
* Update fastssz
* Merge branch 'master' of github.com:prysmaticlabs/prysm into update-fssz
* fmt
* gaz
* Merge refs/heads/master into update-fssz
* goimports
* Merge refs/heads/master into update-fssz
* Merge refs/heads/master into update-fssz
* Merge refs/heads/master into update-fssz
* Merge refs/heads/master into update-fssz
* Merge refs/heads/master into update-fssz
* Fix
* fix ethereumapis
* fix again
* kafka
* fix gen file
* fix compute signing root
* gofmt
* checkpoint progress
* progress
* checkpoint
* progress
* Fix build
* checkpoint
* helpers
* Another test fixed
* gaz
* another test fix
* gofmt
* some fixes
* Merge branch 'master' of github.com:prysmaticlabs/prysm into update-fssz
* fix one test
* Merge branch 'master' of github.com:prysmaticlabs/prysm into update-fssz
* fill empty checkpoint roots
* more padding
* more padding
* Fix //beacon-chain/rpc/debug:go_default_test
* fix //beacon-chain/core/state:go_default_test
* fix //beacon-chain/core/state:go_default_test
* fix some htr errors
* fix //slasher/rpc:go_default_test
* Progress on //beacon-chain/core/blocks:go_default_test
* Progress on //beacon-chain/core/blocks:go_default_test
* Progress on //beacon-chain/core/blocks:go_default_test
* fix //slasher/db/kv:go_default_test
* progress
* fix //beacon-chain/sync/initial-sync:go_raceon_test
* gofmt and gaz
* fix one more test, taking a break
* Fix //beacon-chain/core/blocks:go_default_test
* Complete beacon-chain/powchain
* Do most of beacon-chain/rpc/beacon/
* Do most of beacon-chain/blockchain
* fix //beacon-chain/operations/attestations/kv:go_default_test
* Fix //beacon-chain/cache/depositcache:go_default_test
* Fix //slasher/detection:go_default_test
* Progress
* fix //beacon-chain/rpc/validator:go_default_test
* gofmt
* fix //validator/client:go_default_test
* fix
* fix //beacon-chain/blockchain:go_raceoff_test
* fix //beacon-chain/rpc/beacon:go_default_test
* fix 1 of 4 shards in //beacon-chain/sync:go_default_test
* Fix //beacon-chain/sync:go_default_test and gofmt
* prevent panic
* fix //beacon-chain/state/stategen:go_default_test
* fix
* Merge branch 'master' of github.com:prysmaticlabs/prysm into update-fssz
* fix most tests
* Self review, go mod tidy, run regen scripts
* fix slasher
* Update ethereumapis
* disable spawn strategy override
* Merge refs/heads/master into update-fssz
* Merge refs/heads/master into update-fssz
* Remove extra line in imports
* Remove extra line in imports
* Gofmt
* PR feedback from @nisdas
2020-08-24 01:46:17 +00:00
Preston Van Loon
706f375aed Healthz: Move header before write. Fixes #7029 (#7084)
* Healthz: Move header before write. Fixes #7029
* Merge refs/heads/master into fix-7029
2020-08-22 21:12:17 +00:00
terence tsao
5977343a0d UnaggregatedAttestations use helper hasSeenBit (#7083) 2020-08-22 13:42:55 -07:00
terence tsao
f2afeed9da Pool: Use a TTL cache for seen bits (#7015) 2020-08-21 16:27:51 -07:00
Raul Jordan
e69ed7c778 Accounts V2: Implement Backup for Derived Keymanager (#7067)
* backup for derived complete
* Merge refs/heads/master into derived-backup
* Merge refs/heads/master into derived-backup
* Merge refs/heads/master into derived-backup
* Merge refs/heads/master into derived-backup
* Merge refs/heads/master into derived-backup
* Merge refs/heads/master into derived-backup
* Merge refs/heads/master into derived-backup
* Merge refs/heads/master into derived-backup
* Merge refs/heads/master into derived-backup
2020-08-21 19:43:03 +00:00
Raul Jordan
c9caf5dfc5 Set Healthz to Return Error if Chain Has Not Yet Started (#7077)
* healthz in sync when chain not started
* Merge refs/heads/master into fix-healthz-when-process-deposits
* Merge refs/heads/master into fix-healthz-when-process-deposits
* Merge refs/heads/master into fix-healthz-when-process-deposits
2020-08-21 19:09:52 +00:00
Raul Jordan
b4c7a14759 Do Not Unset Rate Limiter Before Node Shutdown (#7078)
* reset rate limiter before shutdown
* Merge refs/heads/master into empty-rate-limit
* Merge refs/heads/master into empty-rate-limit
* Update beacon-chain/sync/service.go
2020-08-21 18:48:43 +00:00
Ivan Martinez
92e1a996cc Slasher: Optimize span marshal and unmarshal (#7062)
* Optimize marshal and unmarshal
* Merge branch 'master' into slasher-optimize-span
* Merge branch 'master' into slasher-optimize-span
* Add tests and comments
* Merge branch 'slasher-optimize-span' of github.com:prysmaticlabs/prysm into slasher-optimize-span
* Merge branch 'master' of github.com:prysmaticlabs/prysm into slasher-optimize-span
* Merge refs/heads/master into slasher-optimize-span
* Merge refs/heads/master into slasher-optimize-span
* Merge refs/heads/master into slasher-optimize-span
2020-08-21 17:29:11 +00:00
rkapka
6228b3cd9f Identical expression comparison analyzer (#7066)
* static analyzer with tests
* Merge branch 'origin-master' into identical-expression-comparison-analyzer
* resolve analyzer errors
* Merge branch 'origin-master' into identical-expression-comparison-analyzer
* change excluded file in nogo_config
* remove tests from bazel file
* exclude test file explicitly
* extracted common code to helper file
* Revert "extracted common code to helper file"

This reverts commit e9ccea7360.
* Merge refs/heads/master into identical-expression-comparison-analyzer
* Merge refs/heads/master into identical-expression-comparison-analyzer
2020-08-21 16:48:27 +00:00
terence tsao
8686a81304 Implement checkpt info cache (#7070) 2020-08-21 08:50:21 -07:00
Victor Farazdagi
05d6dc7a10 Improves block deduplication on batch processing (#7075)
* fixes block deduplication on batch processing
* Nishant's suggestions
2020-08-21 10:51:27 +00:00
terence tsao
ec1dd85c44 Fix identical expressions (#7069)
* Fix ident exps
* Merge refs/heads/master into fix-ident-exp
2020-08-20 22:34:52 +00:00
Roy
7b0b8ee126 Add prysm.sh1 for native Windows support via PowerShell (#6829)
* Add prysm.sh1 for native Windows support via PowerShell
* Remove unused variable
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge branch 'master' into master
2020-08-20 20:52:59 +00:00
terence tsao
55074bcc6c Tidying up a few core functions (#6995)
* DB: add block roots test
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Pool: add seen atts map
* Pool: use seen atts map
* Pool: clear seen map
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Core: clean up unused noverify and tidy up namings
* Gaz
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Core: Add no verify back. Better namings
* Merge branch 'rm-no-verify' of github.com:prysmaticlabs/prysm into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Core: fixed a test
* Merge branch 'rm-no-verify' of github.com:prysmaticlabs/prysm into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
2020-08-20 19:53:22 +00:00
rkapka
89e279f9c8 Account commands cleanup (#7065)
* initial implementation of command
* Add second confirmation step
* Merge branch 'origin-master' into voluntary-exit-command
* fix variable name
* added unit tests
* Merge branch 'origin-master' into voluntary-exit-command
* add comment about stdin
* remove backup-related code from accounts_delete_test
* fix comments
* move filterPublicKeysFromUserInput to new account helper
* remove SkipMnemonicConfirmFlag
* Merge refs/heads/master into voluntary-exit-command
* Merge refs/heads/master into voluntary-exit-command
* Merge refs/heads/master into voluntary-exit-command
* Merge refs/heads/master into voluntary-exit-command
* Merge refs/heads/master into voluntary-exit-command
* Merge refs/heads/master into voluntary-exit-command
* Merge refs/heads/master into voluntary-exit-command
* code review fixes
* removed commented-out code
* Merge branch 'voluntary-exit-command' into account-commands-cleanup

# Conflicts:
#	shared/promptutil/prompt.go
* Merge branch 'origin-master' into account-commands-cleanup

# Conflicts:
#	shared/promptutil/prompt.go
#	validator/accounts/v2/BUILD.bazel
#	validator/accounts/v2/accounts_exit.go
#	validator/accounts/v2/accounts_exit_test.go
* Merge refs/heads/master into account-commands-cleanup
2020-08-20 19:14:03 +00:00
terence tsao
40db6cb0cf Add slot in epoch (#7064)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-08-20 13:43:03 -05:00
rkapka
45e4ed25a3 Voluntary exit CLI command (#7046)
* initial implementation of command

* Add second confirmation step

* fix variable name

* added unit tests

* add comment about stdin

* code review fixes

* removed commented-out code

* added log assert

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-08-20 19:53:09 +02:00
terence tsao
e93d31c60e Use hot state cache for replay (#7063) 2020-08-20 08:53:10 -07:00
terence tsao
8bdf9db147 Add replay block count metric (#7060) 2020-08-20 06:20:26 -07:00
Shay Zluf
2bf1332e5c Slasher improvements (#7061)
* Slasher loging and span removal

* merge and reconnect fix

* ivan feedback
2020-08-20 10:31:16 +03:00
Nishant Das
7744c3ae47 Batch Block Roots Requesting (#7027)
* checkpoint
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into batchRootReq
* test
* gaz
* fix test
* comment
* Merge refs/heads/master into multipleBranchProcessing
* Merge refs/heads/master into multipleBranchProcessing
* Merge refs/heads/master into multipleBranchProcessing
* Merge refs/heads/master into multipleBranchProcessing
* Merge refs/heads/master into multipleBranchProcessing
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into batchRootReq
* terence's review
* Merge branch 'multipleBranchProcessing' of https://github.com/prysmaticlabs/geth-sharding into batchRootReq
2020-08-20 04:50:14 +00:00
Nishant Das
ba5da21026 Shift Sync Warn and Error Messages To Debug (#7059)
* change

* change
2020-08-20 11:30:41 +08:00
Preston Van Loon
395ea76f44 Save finalized checkpoint before migrate to cold (#7055)
* Save finalized checkpoint before migrate to cold
* Save finalized checkpoint before migrate to cold
* Merge branch 'master' into save-checkpoint-first
2020-08-19 21:11:44 +00:00
Preston Van Loon
3aa95b9c16 Use correct context for IsFinalizedBlock call (#7053)
* Use correct context for IsFinalizedBlock call
* Merge branch 'master' into use-correct-ctx
2020-08-19 19:55:39 +00:00
Ivan Martinez
bc20591b72 Accounts-V2: Change ListDirs() to Exists(), fix bug for derived (#7038)
* Change ListDirs() to Exists()
* Merge branch 'master' into fix-derived-pass
* Make function more accurate
* Merge branch 'fix-derived-pass' of github.com:prysmaticlabs/prysm into fix-derived-pass
* Merge refs/heads/master into fix-derived-pass
* Merge refs/heads/master into fix-derived-pass
* Merge refs/heads/master into fix-derived-pass
* Merge refs/heads/master into fix-derived-pass
* Merge refs/heads/master into fix-derived-pass
* Merge refs/heads/master into fix-derived-pass
* Merge refs/heads/master into fix-derived-pass
* Merge refs/heads/master into fix-derived-pass
* Merge refs/heads/master into fix-derived-pass
* Merge refs/heads/master into fix-derived-pass
2020-08-19 16:46:26 +00:00
Nishant Das
85a040bfd2 Default To Mainnet Config When Parsing Chain Config (#7052)
* add reg test
* Merge branch 'master' into defaultToMainnet
2020-08-19 15:42:24 +00:00
terence tsao
399dc85737 Use map to reject seen pending block (#7045) 2020-08-18 09:21:25 -07:00
Victor Farazdagi
9bf80219c9 Applies assertion funcs to slasher/* tests (#6998)
* slasher/beaconclient tests
* slasher/db/kv tests
* Merge branch 'master' into apply-testutils-assertions-to-slasher
* fix build
* slasher/detection tests
* rest of the tests
* misc tests
* Merge refs/heads/master into apply-testutils-assertions-to-slasher
* Merge refs/heads/master into apply-testutils-assertions-to-slasher
* Merge refs/heads/master into apply-testutils-assertions-to-slasher
* Merge refs/heads/master into apply-testutils-assertions-to-slasher
* Merge refs/heads/master into apply-testutils-assertions-to-slasher
* Merge refs/heads/master into apply-testutils-assertions-to-slasher
* Merge refs/heads/master into apply-testutils-assertions-to-slasher
* Merge refs/heads/master into apply-testutils-assertions-to-slasher
* Merge refs/heads/master into apply-testutils-assertions-to-slasher
* Merge refs/heads/master into apply-testutils-assertions-to-slasher
* Merge refs/heads/master into apply-testutils-assertions-to-slasher
* Merge refs/heads/master into apply-testutils-assertions-to-slasher
* Merge refs/heads/master into apply-testutils-assertions-to-slasher
* Merge refs/heads/master into apply-testutils-assertions-to-slasher
* Merge refs/heads/master into apply-testutils-assertions-to-slasher
* Merge refs/heads/master into apply-testutils-assertions-to-slasher
* Merge refs/heads/master into apply-testutils-assertions-to-slasher
* Merge refs/heads/master into apply-testutils-assertions-to-slasher
* Update slasher/db/kv/spanner_test.go

Co-authored-by: Shay Zluf <thezluf@gmail.com>
* Update slasher/node/node_test.go

Co-authored-by: Shay Zluf <thezluf@gmail.com>
* Merge refs/heads/master into apply-testutils-assertions-to-slasher
* Update slasher/db/kv/spanner_test.go
* Merge refs/heads/master into apply-testutils-assertions-to-slasher
2020-08-18 12:41:25 +00:00
Shay Zluf
14cd25f9dc Slasher span cache size flag (#7043)
* Span cache size by using flag
* Merge refs/heads/master into set_span_cache_size
2020-08-18 11:18:33 +00:00
Nishant Das
d81c9ffcfc Reject Duplicate Blocks From Being Saved (#7044)
* check for duplicates

* add test
2020-08-18 18:47:44 +08:00
Shay Zluf
27c8402ae0 Validator recheck slasher client status (#7041)
* Validator recheck slasher client status
* Merge branch 'master' into slasher_status_fix
* Merge branch 'master' into slasher_status_fix
* Merge refs/heads/master into slasher_status_fix
* Merge refs/heads/master into slasher_status_fix
2020-08-18 03:04:39 +00:00
Victor Farazdagi
0a5ec502b0 Updates resyncIfBehind() functionality (#7039)
* defines BestNonFinalized
* updates blocks_fetcher
* updates blocks_queue
* Merge branch 'master' into fix-resync
* Nishant's suggestion
* Merge branch 'fix-resync' of github.com:prysmaticlabs/prysm into fix-resync
* unit test
* Merge branch 'master' into fix-resync
* fixes test
* Merge refs/heads/master into fix-resync
2020-08-18 02:10:45 +00:00
Preston Van Loon
4d463c4a85 Don't ban blocks for context deadlines (#7040)
* Don't ban blocks for context deadlines
* Don't ban blocks for context deadlines
* Add test
* Merge branch 'master' into dont-ban-ctx-deadline2
* fmt
2020-08-18 01:21:10 +00:00
terence tsao
9caa92cae4 Pending queue to handle same slot blocks (#7036)
* Pending queue: rough draft
* Removed extra RUnlock
* apply diff
* Update pending queue tests
* Merge branch 'fix-pending-blocks' of github.com:prysmaticlabs/prysm into fix-pending-blocks
* Merge branch 'master' into fix-pending-blocks
* Comments
* Merge branch 'fix-pending-blocks' of github.com:prysmaticlabs/prysm into fix-pending-blocks
2020-08-18 00:01:32 +00:00
terence tsao
dbd1e8c247 Only cache check point state if it hasn't existed in other cache (#7037)
* Only cache epoch boundary state in the event of skipped epoch boundary slot

* checkpoint

* remove lods

* checkppoint again

* Cache: fill if empty

* Don't include the sync fixes

* Update sync utils

* Avoid extra process slots

Co-authored-by: nisdas <nishdas93@gmail.com>
2020-08-17 16:20:34 -05:00
Victor Farazdagi
bde3073bd6 Wait interval, before finalized peers re-checked (#7032)
* wait on no finalized peers
* Merge branch 'master' into better-no-finalized-peers-handling
2020-08-17 16:01:16 +00:00
Nishant Das
f193dc2f54 Do Not Retrieve From DB For Block Roots (#7028)
* don't use db
2020-08-17 08:41:45 +00:00
terence tsao
a2f781522f Update committee cache size to 32 (#7018)
* Update committee cache size to 32
* Merge branch 'master' into update-committee-cache-size
* Cache: fixed rotating test
* Merge branch 'update-committee-cache-size' of github.com:prysmaticlabs/prysm into update-committee-cache-size
* Merge refs/heads/master into update-committee-cache-size
* Merge refs/heads/master into update-committee-cache-size
* Merge refs/heads/master into update-committee-cache-size
2020-08-17 03:17:46 +00:00
Preston Van Loon
d66f72939e Ignore subscription cancelled warning (#7020)
* Resolve TODO #6449
* Merge refs/heads/master into fix-6449
2020-08-17 01:58:30 +00:00
Preston Van Loon
c3725d11f2 Revert "Sync: Dont ban block roots for context.DeadlineExceeded" (#7017)
* Revert "Sync: Dont ban block roots for context.DeadlineExceeded (#7016)"

This reverts commit a4bbd82a45.
* Merge refs/heads/master into revert-7016-dont-ban-ctx-deadline
2020-08-17 01:18:38 +00:00
terence tsao
715cd40f56 Sync: check blocks and roots lengths (#7019) 2020-08-16 16:48:39 -07:00
Preston Van Loon
a4bbd82a45 Sync: Dont ban block roots for context.DeadlineExceeded (#7016)
* Dont set bad block root if it was context deadline with processing
2020-08-16 19:24:34 +00:00
Nishant Das
0be1957c28 Use Stage 1 For Non Finalized Sync (#7012)
* checkpoint
* dont cancel
* remove
* sync mode
* fixes build
* cap max retries when no finalized peers are found
* use max peers
* change it
* fixes TestService_roundRobinSync/Multiple_peers_with_different_finalized_epoch
* fixes blocks fetcher tests
* fixes blocks queue tests
* fixes TestService_blockProviderScoring test
* gofmt
* Merge branch 'master' into useStage1
* Update round_robin.go

preston's review
* Preston's suggestions
* Merge branch 'useStage1' of github.com:prysmaticlabs/prysm into useStage1
* fixes test
2020-08-16 17:51:14 +00:00
terence tsao
9ebd90c324 Use LRU for checkpoint state cache (#7010)
* DB: add block roots test

* Pool: add seen atts map

* Pool: use seen atts map

* Pool: clear seen map

* Core: optimize InitiateValidatorExit

* Cache: checkpoint state uses LRU

* Cache: use it

* Cache: update tests

* Cache: typo

* Gaz
2020-08-16 07:36:37 -07:00
terence tsao
03c3f06288 Core: optimize InitiateValidatorExit and ProcessAttesterSlashings (#7008)
* Core: optimize InitiateValidatorExit

* Core: optimize

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-08-15 14:01:46 -05:00
terence tsao
c47d18aa0b Featureflag test: reset init (#7007)
* Featureflag: properly init test
* Merge refs/heads/master into fix-feature-test
2020-08-15 15:48:12 +00:00
Nishant Das
17b810f24e Fix 2nd Stage of Sync (#7006)
* handle 2nd stage better
* 2 stage sort
* add warning
* better processing
* victor's review
* fix one test
* Update beacon-chain/blockchain/receive_block.go
* fixes TestService_roundRobinSync/Multiple_peers_with_many_skipped_slots
* Merge branch 'fixSync' of github.com:prysmaticlabs/prysm into fixSync
* fixes TestService_process
* fixes TestService_blockProviderScoring
* Preston's suggestion
* increase number of available slots, allowing 2nd phase to wrap up
* more concise limits
2020-08-15 15:14:59 +00:00
Nishant Das
ee2ba721ab Initialize Config Correctly (#7004)
* initialize correctly
2020-08-15 01:37:57 +00:00
Noah Citron
d24f99d66d added flag to enable roughtime syncing, with it disabled by default (#6898)
* added flag to enable roughtime syncing, with it disabled by defaults

* gofmt

* switched to feature flag

* enable-roughtime works correctly

* enable-roughtime works correctly

* Gaz

* Remove conflict

* fix roughtime enable

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: rauljordan <raul@prysmaticlabs.com>
2020-08-14 14:22:58 -05:00
Raul Jordan
3c776f4e76 Fix Validator RPC JWT Response (#7001)
* fix jwt response
* fix jwt parse
* ken naming
* gaz
2020-08-14 16:30:11 +00:00
terence tsao
b981442022 Register health APIs handler (#6999)
* DB: add block roots test
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Pool: add seen atts map
* Pool: use seen atts map
* Pool: clear seen map
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* RPC: register health endpoint
* Gazelle
2020-08-14 15:18:04 +00:00
Victor Farazdagi
6db448a81d Uncomments tests in kv/unaggregated_test (#6997)
* uncomments tests in kv/unaggregated_test
2020-08-14 07:23:42 +00:00
Raul Jordan
f5a1a8b4ed Integrate Validator RPC Server Into Runtime + Add Gateway (#6994) 2020-08-13 19:49:57 -07:00
terence tsao
3df2980cba Add and utilize seen atts map (#6993)
* DB: add block roots test

* Pool: add seen atts map

* Pool: use seen atts map

* Pool: clear seen map

* Gazelle

* Fixed an existing test

Co-authored-by: rauljordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-08-13 16:52:27 -05:00
Raul Jordan
16c34b627f Add Authentication Functions to Validator RPC (#6968)
* define auth endpoints
* add intercepter with tests
* auth functions
* fix up the auth functions
* add functions for storing and saving the hashed password from the validator db
* validate strong password input and simplify jwt claims
* tests for db funcs
* comments for db funcs
* wrap up the authentication tests
* register auth srv
* use proper db iface package and check if existing password
* fix broken tests and add new test to check if password already exists
* use roughtime
* rlock to check the auth paths
* Merge refs/heads/master into auth-rpc
* Merge refs/heads/master into auth-rpc
* Merge refs/heads/master into auth-rpc
* leave out the stream interceptor
* resolve confs
* Merge branch 'master' into auth-rpc
* confs
* Merge branch 'auth-rpc' of github.com:prysmaticlabs/prysm into auth-rpc
* Merge refs/heads/master into auth-rpc
* Merge refs/heads/master into auth-rpc
* Merge refs/heads/master into auth-rpc
* Merge refs/heads/master into auth-rpc
2020-08-13 20:27:42 +00:00
Ivan Martinez
6ed0539723 Acconuts-V2: Allow importing from any valid keystore, regardless of name (#6992)
* Allow importing from any file

* Fix log

* Add test

* Fix test
2020-08-13 14:45:56 -05:00
Victor Farazdagi
4de0b9fe69 Peer scoring: init sync (#6709)
* refactors redundant part of varname
* introduces score_block_providers
* Merge branch 'master' into init-sync-peer-scoring-slow-peers
* gazelle
* adds comment
* Merge branch 'master' into init-sync-peer-scoring-slow-peers
* removes redundant checks
* add block provider decay test
* Merge branch 'master' into init-sync-peer-scoring-slow-peers
* adds case
* penalize inactive peers
* adds scorebadresponses test
* inroduces no-activity penalty
* gazelle
* gofmt
* Merge branch 'master' into init-sync-peer-scoring-slow-peers
* expanded tests
* implement SortBlockProviders
* change -> update
* updates block fetcher peer filter
* fixes test
* allows to keep track of peer id
* updates scoring coefficients
* fixes test
* block fetcher update
* Merge branch 'master' into init-sync-peer-scoring-slow-peers
* disables empty batch penalty
* Merge branch 'master' into init-sync-peer-scoring-slow-peers
* Merge branch 'master' into init-sync-peer-scoring-slow-peers
* removes outdated code
* update filterPeers
* gazelle
* updates var
* revert changes to var name
* updates blocks_fetcher
* minor fix to import name
* Merge branch 'master' into init-sync-peer-scoring-slow-peers
* add max processed blocks cap
* impoves scoring of stale peers
* Merge branch 'master' into init-sync-peer-scoring-slow-peers
* fixes test
* adds weight sorting to scored peers
* return pid when fetching batches
* updates round robin
* gazelle
* Merge branch 'master' into init-sync-peer-scoring-slow-peers
* updates block provider decay count
* go tidy
* cherry pick
* fixes test
* go tidy
* Merge branch 'peer-scorer-weighted-sorter' into init-sync-peer-scoring-slow-peers
* Merge branch 'master' into init-sync-peer-scoring-slow-peers
* Merge branch 'master' into init-sync-peer-scoring-slow-peers
* refactors blocks fetcher: repackage methods
* Merge branch 'refactor-blocks-fetcher' into init-sync-peer-scoring-slow-peers
* minor fixes
* Merge branch 'master' into init-sync-peer-scoring-slow-peers
* Merge branch 'master' into init-sync-peer-scoring-slow-peers
* allow scores in range (0;1) in weighted filter
* filterScoredPeers improve test suite
* puts feature behind the flag
* Merge branch 'master' into init-sync-peer-scoring-slow-peers
* Merge refs/heads/master into init-sync-peer-scoring-slow-peers
* fixes tests
* Merge branch 'init-sync-peer-scoring-slow-peers' of github.com:prysmaticlabs/prysm into init-sync-peer-scoring-slow-peers
* Merge refs/heads/master into init-sync-peer-scoring-slow-peers
* Merge branch 'master' into init-sync-peer-scoring-slow-peers
* Merge refs/heads/master into init-sync-peer-scoring-slow-peers
* Update beacon-chain/sync/initial-sync/blocks_fetcher_test.go

Co-authored-by: Shay Zluf <thezluf@gmail.com>
* Nishant's suggestion on peer limit variable
* better explanation of non-blocking peer scoring
* Shay's sugession on peer naming
* Merge refs/heads/master into init-sync-peer-scoring-slow-peers
* Merge refs/heads/master into init-sync-peer-scoring-slow-peers
* Update beacon-chain/sync/initial-sync/blocks_fetcher.go

Co-authored-by: Nishant Das <nishdas93@gmail.com>
* Merge refs/heads/master into init-sync-peer-scoring-slow-peers
* gofmt
* Merge refs/heads/master into init-sync-peer-scoring-slow-peers
* Merge refs/heads/master into init-sync-peer-scoring-slow-peers
* Merge refs/heads/master into init-sync-peer-scoring-slow-peers
2020-08-13 17:33:57 +00:00
Nishant Das
282398fd13 Add in Attestation Validity Check (#6983)
* fix bad bug
* add test
* Merge refs/heads/master into fixAttestationNoVerify
* Merge refs/heads/master into fixAttestationNoVerify
* target
* Merge branch 'fixAttestationNoVerify' of https://github.com/prysmaticlabs/geth-sharding into fixAttestationNoVerify
* add cheap check
* Merge refs/heads/master into fixAttestationNoVerify
* Merge refs/heads/master into fixAttestationNoVerify
* Merge refs/heads/master into fixAttestationNoVerify
* Merge refs/heads/master into fixAttestationNoVerify
* Merge refs/heads/master into fixAttestationNoVerify
2020-08-13 16:58:28 +00:00
Victor Farazdagi
74101612ce Moves AssertLogsContains to assertions package (#6985)
* first working implementation
* assertions tests
* adds to requires
* merges assert and require tests into a single suite
* gazelle
* Merge branch 'merge-assert-require-tests' into assert-logs-contains-move-to-assertions
* gazelle
* updates references
* fixes build issue
* Merge branch 'master' into assert-logs-contains-move-to-assertions
* Merge refs/heads/master into assert-logs-contains-move-to-assertions
* Merge branch 'master' into assert-logs-contains-move-to-assertions
* fixes build issue
* Merge branch 'assert-logs-contains-move-to-assertions' of github.com:prysmaticlabs/prysm into assert-logs-contains-move-to-assertions
* Merge refs/heads/master into assert-logs-contains-move-to-assertions
* Merge refs/heads/master into assert-logs-contains-move-to-assertions
2020-08-13 16:22:25 +00:00
terence tsao
308dbc4272 ListStatuses for validator RPC (#6987)
* DB: add block roots test
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Client: support ValidatorPubkeysToStatuses getter
* RPC: implement ListStatuses
* RPC: tests
* Merge branch 'master' into health-apis
* Update an existing test
* Merge branch 'health-apis' of github.com:prysmaticlabs/prysm into health-apis
* Merge refs/heads/master into health-apis
2020-08-13 15:52:56 +00:00
Ivan Martinez
5705a9a1f9 Accounts-V2: Add test for DeleteAccounts (#6982)
* Add test for DeleteAccounts
* attempt to fix
* Try to fix test
* Remove
* fixed Ivan's test
* Merge branch 'master' into add-delete-test
2020-08-13 15:23:08 +00:00
Nishant Das
72bbc33a65 Fix Validator Exits (#6986)
* add change
* Update beacon-chain/core/blocks/exit.go

Co-authored-by: Shay Zluf <thezluf@gmail.com>
* Merge refs/heads/master into fixExits
2020-08-13 14:37:38 +00:00
Victor Farazdagi
dc752fb945 Merges assert and require tests into a single suite (#6984)
* merges assert and require tests into a single suite
* gazelle
2020-08-13 13:57:29 +00:00
Victor Farazdagi
0c94298847 Peer scorer: fix weighted sorting (#6981)
* peer scorer: fix weighted sorting
* Merge refs/heads/master into minor-fixes-to-peer-scorer
* Merge refs/heads/master into minor-fixes-to-peer-scorer
2020-08-13 09:36:50 +00:00
Victor Farazdagi
7611f6d84a injects peer.ID into requested/queued/fetched batches (#6979)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-08-13 16:42:21 +08:00
terence tsao
49549f4687 ListBalances for validator RPC (#6975)
* DB: add block roots test

* Node: register RPC service

* RPC: access to client service

* Client: add balances and pubkeys -> indices getters

* Update validator/client/metrics.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Lint and feedbacks

* Fix lock to prevent a run time panic

* Fixes existing tests

* Validator: add mock for testing

* Validator: Add pubkeys to indices getter

* RPC: tests tests tests

* Lint

* Lint

* Lint

* Lint

* Lint

* Gazelle

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-08-12 18:33:15 -05:00
terence tsao
0d7ea33298 DB: add block roots test (#6974) 2020-08-12 11:28:52 -05:00
Javed Khan
2400536a8e Update sendDepositTx tool (#6972)
* Update sendDepositTx contract address

* Use Medalla config in sendDepositTx

* Update sendDepositTx  - geth not necessary
2020-08-12 22:15:37 +08:00
Victor Farazdagi
da4e96047b go syntax representation of the value in DeepEqual assertions (#6970) 2020-08-12 17:46:56 +08:00
terence tsao
f4d808a542 Validator web RPC skeleton (#6967)
* Proto: remove unused ethapi imports
* Validator: web rpc
* Lint
* Sync with master
* Merge refs/heads/master into web-rpc
* rename
2020-08-12 01:38:01 +00:00
Raul Jordan
86b2bd5edd Implement Accounts-V2 Backup for Direct Keymanager (#6936)
* progress on export feature
* Merge branch 'master' into impl-export
* rename func
* imports
* confs
* export single works
* impl export
* Merge branch 'master' into impl-export
* cleanup exports code
* rename to backup
* rename to backup
* Merge branch 'master' into impl-export
* full test for backup
* Merge branch 'impl-export' of github.com:prysmaticlabs/prysm into impl-export
* gazelle fix
* terence feedback
* merge confs
* Merge branch 'master' into impl-export
* refactor into single select accounts function
* gaz
* Merge refs/heads/master into impl-export
* victor's feedback
* validate yes or no
* confs
* close the zip file in defer
* Merge refs/heads/master into impl-export
* Merge refs/heads/master into impl-export
2020-08-11 23:15:06 +00:00
terence tsao
682401c6b0 Health related web api protos (#6965)
* Proto: first attempt of account related web api protos
* Proto: health related web api protos
* Merge branch 'master' of github.com:prysmaticlabs/prysm into validator-protos-1
* Merge branch 'master' of github.com:prysmaticlabs/prysm into validator-protos-1
* Sync with master
* @rauljordan's feedback
2020-08-11 22:33:03 +00:00
terence tsao
4499fb2ed0 First attempt of account related web api protos (#6964) 2020-08-11 13:23:16 -07:00
terence tsao
46cb3abe2e Add and use genesis time in config (#6966)
* Config: add genesis time and update medalla

* Helper: add GenesisTime getter

* Rest: use the helper
2020-08-11 14:36:32 -05:00
Raul Jordan
11c9699129 Allow Importing One-Off, Private Keys Into Accounts-v2 (#6929)
* import privkey done
* rem print
* tidy
* Merge refs/heads/master into import-priv-key
* Merge refs/heads/master into import-priv-key
* Merge branch 'master' into import-priv-key
* tests pass
* Merge refs/heads/master into import-priv-key
* Merge refs/heads/master into import-priv-key
* resolve confs
* Merge refs/heads/master into import-priv-key
* confs
* Merge refs/heads/master into import-priv-key
* better feedback mechanism
2020-08-11 16:32:05 +00:00
Victor Farazdagi
c24666e02e refactors blocks fetcher: repackage methods (#6962) 2020-08-11 10:15:18 -05:00
Victor Farazdagi
a279f374bc Move to opt-out attestation-aggregation-strategy flag (#6960)
* inverts attestation-aggregation-strategy flag
* Merge refs/heads/master into att-aggregation-opt-out-flag
* marks deprected flag as hidden
* Merge branch 'att-aggregation-opt-out-flag' of github.com:prysmaticlabs/prysm into att-aggregation-opt-out-flag
* Merge refs/heads/master into att-aggregation-opt-out-flag
2020-08-11 12:42:08 +00:00
rkapka
a5f0fd29fc Fix antipatterns (#6961)
* Merge variable declaration and assignment
* Use result of type assertion to simplify cases
* Replace call to bytes.Compare with bytes.Equal
* Drop unnecessary use of the blank identifier
* Replace x.Sub(time.Now()) with time.Until(x)
* Function literal can be simplified
* Use a single append to concatenate two slices
* Replace time.Now().Sub(x) with time.Since(x)
* Omit comparison with boolean constant
* Omit redundant nil check on slices
* Nested if can be replaced with else-if
* Function call can be replaced with helper function
* Omit redundant control flow
* Use plain channel send or receive
* Simplify returning boolean expression
* Merge branch 'origin-master' into fix-antipatterns
* Merge branch 'master' into fix-antipatterns
* Merge branch 'origin-master' into revert-single-channel-antipattern
* Revert "Use plain channel send or receive"

This reverts commit b3108fdc3a.
2020-08-11 12:14:28 +00:00
Nishant Das
b32ccce23d Remove Checked In Metadata File (#6959)
* Delete metaData
2020-08-11 11:26:59 +00:00
Nishant Das
3893ee43f9 support domains in discoveryV5 (#6958) 2020-08-11 18:33:27 +08:00
Ivan Martinez
ea9bb88d13 Accounts-V2: Add command to delete accounts (#6941)
* Add accounts-v2 delete
* Merge branch 'master' of github.com:prysmaticlabs/prysm into add-delete-command
* Gaz
* Add confirmation
* Fix build
* Fix test
* Merge branch 'master' into add-delete-command
* Add more cases for exit
* Merge branch 'master' of github.com:prysmaticlabs/prysm into add-delete-command
* Fix usage
* Fix logs
* Change to DeleteAccounts
* Remove flag
* Fix comment
2020-08-11 03:54:37 +00:00
terence tsao
537fe3f721 Fix loading historical blocks to replay cold state (#6956)
* Replay: cold state only uses finalized blocks
* Replay: regression test
* Merge refs/heads/master into fix-finalized-replay
2020-08-10 23:35:28 +00:00
Ivan Martinez
8b9138a76a Accounts V2: Log and UX cleanup (#6955)
* Clena up password entry for importing
* Merge branch 'master' of github.com:prysmaticlabs/prysm into accounts-v2-cleanup
* Fix password entry
* Clean
* Merge branch 'master' of github.com:prysmaticlabs/prysm into accounts-v2-cleanup
* Change color
* Merge branch 'master' of github.com:prysmaticlabs/prysm into accounts-v2-cleanup
* Fix
* Merge branch 'master' of github.com:prysmaticlabs/prysm into accounts-v2-cleanup
* Fix
* Merge branch 'master' of github.com:prysmaticlabs/prysm into accounts-v2-cleanup
* Cleanup
* Clean go.mod
* Fix
2020-08-10 22:35:30 +00:00
terence tsao
ceb11f432e Proposer: log graffiti (#6954)
* Proposer: log graffiti
* Merge refs/heads/master into log-graffiti
2020-08-10 19:17:59 +00:00
Raul Jordan
cbc27e0f2e Refactor Wallet Password Handling, Preventing Wallet Corruption (#6933)
* wallet should no longer deal with account passwords
* ensure tests are fixed
* Merge branch 'master' into corrupted-pass
* move mnemonic logic into right place
* rem fmts
* add fileutil
* gazelle
* imports
* move seed logic to derived
* fix tests
* imports
* gaz
* Merge refs/heads/master into corrupted-pass
* merge confs
* Merge refs/heads/master into corrupted-pass
* ivan's feedback
* Merge branch 'corrupted-pass' of github.com:prysmaticlabs/prysm into corrupted-pass
* gaz
* fix shared test
* Merge refs/heads/master into corrupted-pass
* resolve conflicts
* fix test build
2020-08-10 18:54:40 +00:00
Raul Jordan
a7f4293eb3 Resolve Accounts-v2 Feedback (#6953)
* Radek feedback
* Merge refs/heads/master into final-accounts-feedback
* Apply suggestions from code review
* Merge refs/heads/master into final-accounts-feedback
* Merge refs/heads/master into final-accounts-feedback
2020-08-10 18:14:10 +00:00
Shay Zluf
335d31688f Wallet file lock (#6952)
* Wallet file lock

* lock wallet config file only in validator instance

* runtime fixes

* added test

* radek feedback

* fix test
2020-08-10 12:48:41 -05:00
Raul Jordan
95fd399dbe Change Log Level of Genesis Time Info Log in RPC (#6920)
* lvl debug
* Merge refs/heads/master into change-log-level
* Merge refs/heads/master into change-log-level
* tidy
* Merge refs/heads/master into change-log-level
* Merge refs/heads/master into change-log-level
* Merge refs/heads/master into change-log-level
* Merge refs/heads/master into change-log-level
* Merge refs/heads/master into change-log-level
* Merge refs/heads/master into change-log-level
* Merge refs/heads/master into change-log-level
* Merge refs/heads/master into change-log-level
* Merge refs/heads/master into change-log-level
* Merge refs/heads/master into change-log-level
* Merge refs/heads/master into change-log-level
* Merge refs/heads/master into change-log-level
* Merge refs/heads/master into change-log-level
* Merge refs/heads/master into change-log-level
* Merge refs/heads/master into change-log-level
* fix log level error
* victor feedback
* Merge refs/heads/master into change-log-level
2020-08-10 16:52:49 +00:00
rkapka
1f21e196b6 Fix multiple anti-patterns from DeepSource analysis (#6951)
* Merge variable declaration and assignment
* Use result of type assertion to simplify cases
* Replace call to bytes.Compare with bytes.Equal
* Drop unnecessary use of the blank identifier
* Replace x.Sub(time.Now()) with time.Until(x)
* Function literal can be simplified
* Use a single append to concatenate two slices
* Replace time.Now().Sub(x) with time.Since(x)
* Omit comparison with boolean constant
* Omit redundant nil check on slices
* Nested if can be replaced with else-if
* Function call can be replaced with helper function
* Omit redundant control flow
* Use plain channel send or receive
* Simplify returning boolean expression
* Merge branch 'origin-master' into fix-antipatterns
* Merge branch 'master' into fix-antipatterns
2020-08-10 16:16:45 +00:00
Preston Van Loon
18b3203f31 Wait for subnet peers before broadcasting onto attestation subnet topic (#6893)
* Initial pass

* Add metric to measure success

* Use a subnet RWLock to prevent duplicate requests, give up after 3 attempts

* push latest commented code

* try with non-blocking broadcast

* Add feature flag, ignore parent deadline if any

* Add slot as metadata

* add tests

* gaz

Co-authored-by: nisdas <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-08-10 10:27:50 -05:00
rkapka
33105f1bbc Simplify make call by omitting redundant arguments (#6949)
* removed redundant args
* Merge refs/heads/master into omit-redundant-make-args
2020-08-10 11:06:53 +00:00
Victor Farazdagi
44f9d07434 Applies assertion funcs to beacon-chain/core/* tests (#6940)
* beacon-chain/core/blocks/spectest tests updated
* beacon-chain/core/blocks tests updated
* beacon-chain/core/epoch tests updated
* Merge branch 'master' into apply-testutils-assertions-to-core
* beacon-chain/core/helpers tests updated
* beacon-chain/core/validators tests updated
* beacon-chain/core/state/spectest tests updated
* beacon-chain/core/state tests updated
* Merge refs/heads/master into apply-testutils-assertions-to-core
* Merge refs/heads/master into apply-testutils-assertions-to-core
* Merge refs/heads/master into apply-testutils-assertions-to-core
* Merge refs/heads/master into apply-testutils-assertions-to-core
* Merge refs/heads/master into apply-testutils-assertions-to-core
2020-08-10 10:34:33 +00:00
Nishant Das
649bfb58e5 Invert State Locks Flag (#6922)
* invert flag

* Update shared/featureconfig/flags.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-08-10 13:10:41 +08:00
terence tsao
6d5b7b0439 Validator/node: update warn log to error (#6945) 2020-08-09 17:27:23 -07:00
Victor Farazdagi
b241fafd13 Peer scorer: weighted sorting, requests capping, stale peers handling. (#6928)
* cherry pick
* fixes test
* go tidy
* Merge refs/heads/master into peer-scorer-weighted-sorter
* Merge refs/heads/master into peer-scorer-weighted-sorter
* Merge refs/heads/master into peer-scorer-weighted-sorter
* Merge refs/heads/master into peer-scorer-weighted-sorter
* Merge refs/heads/master into peer-scorer-weighted-sorter
* Merge refs/heads/master into peer-scorer-weighted-sorter
* Merge refs/heads/master into peer-scorer-weighted-sorter
* Merge refs/heads/master into peer-scorer-weighted-sorter
* Merge refs/heads/master into peer-scorer-weighted-sorter
2020-08-09 20:26:09 +00:00
Nishant Das
b1946ead23 Make Initial Sync Failures Debug (#6930)
* shift to debug logs
* Merge branch 'master' into makeFailuresDebug
* go sum
* Merge branch 'master' into makeFailuresDebug
* Merge branch 'master' into makeFailuresDebug
* Merge refs/heads/master into makeFailuresDebug
2020-08-09 19:47:52 +00:00
terence tsao
7591240366 Revert "Subscriber: use target epoch for hasSeen cache" (#6944)
* Revert "Subscriber: use target epoch for hasSeen cache (#6908)"

This reverts commit 3275a86ece.
2020-08-09 18:52:37 +00:00
Victor Farazdagi
f575a81afd Applies assertion funcs to beacon-chain/core/blocks tests (#6939)
* beacon-chain/core/blocks/spectest tests updated
* beacon-chain/core/blocks tests updated
* Merge refs/heads/master into apply-testutils-assertions-to-core-blocks
* Merge refs/heads/master into apply-testutils-assertions-to-core-blocks
* Merge refs/heads/master into apply-testutils-assertions-to-core-blocks
2020-08-08 19:06:04 +00:00
Victor Farazdagi
fecbec2342 Applies assertion funcs to beacon-chain/db tests (#6938)
* part1
* go tidy
* Merge branch 'go-tidy' into apply-testutils-assertions-to-beaconchain-db
* updated beacon-chain/db/kv tests
* Merge branch 'master' into apply-testutils-assertions-to-beaconchain-db
* Merge refs/heads/master into apply-testutils-assertions-to-beaconchain-db
* Merge refs/heads/master into apply-testutils-assertions-to-beaconchain-db
2020-08-08 18:39:01 +00:00
terence tsao
9a59c4baff Proposer: aggregated unaggregated attestations for block (#6919) 2020-08-08 10:49:02 -07:00
terence tsao
cca439847d Unexport protoarray forkchoice types (#6937)
* Protoarray: un-export all the types

* Protoarray: add proper getters

* Protoarray: update implementations

* Protoarray: update tests

* Protoarray: lint

* Protoarray: lint
2020-08-07 18:37:08 -07:00
Raul Jordan
10f38662e5 Standardize File Permissions in Accounts-V2 (#6934)
* standardize file permissions
* Gaz
* no petname
* Merge refs/heads/master into file-perms
2020-08-07 22:14:07 +00:00
terence tsao
3275a86ece Subscriber: use target epoch for hasSeen cache (#6908)
* Subscriber: use target epoch for hasSeen cache
* Merge refs/heads/master into use-target
* Merge refs/heads/master into use-target
* Merge refs/heads/master into use-target
* Merge refs/heads/master into use-target
* Merge refs/heads/master into use-target
* Merge refs/heads/master into use-target
* Merge refs/heads/master into use-target
* Merge refs/heads/master into use-target
* Merge refs/heads/master into use-target
* Merge refs/heads/master into use-target
* Merge refs/heads/master into use-target
* Merge refs/heads/master into use-target
2020-08-07 20:55:42 +00:00
Victor Farazdagi
43b646a9dd go mod tidy (#6932)
* go tidy
2020-08-07 17:53:40 +00:00
Raul Jordan
1d9374d5c9 Update Prometheus Client (#6921)
* update prom
* Merge refs/heads/master into update-prom
2020-08-07 10:52:50 +00:00
Nishant Das
8100261e70 Log Error Logs Less Frequently (#6923)
* dont log so much

* cleanup

* custom error
2020-08-07 18:06:53 +08:00
Nishant Das
78ec783604 Add New Validator Map Handler (#6910)
* add new method
* Merge branch 'master' into addMapHandler
* comments
* Merge branch 'addMapHandler' of https://github.com/prysmaticlabs/geth-sharding into addMapHandler
* Update beacon-chain/state/types.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Merge refs/heads/master into addMapHandler
* Merge refs/heads/master into addMapHandler
* Merge refs/heads/master into addMapHandler
* Merge refs/heads/master into addMapHandler
* Merge refs/heads/master into addMapHandler
* Merge refs/heads/master into addMapHandler
* Merge refs/heads/master into addMapHandler
* Merge refs/heads/master into addMapHandler
* gofmt
2020-08-07 01:42:50 +00:00
Ivan Martinez
6351d187ef Accounts V2: Remove unused passwords-dir and deprecate flag (#6916)
* Remove passwords dir
* Deprecate flag
* Merge branch 'master' of github.com:prysmaticlabs/prysm into remove-passwords-dir
* Properly deprecate flag
* Merge branch 'master' into remove-passwords-dir
* Merge refs/heads/master into remove-passwords-dir
* Merge refs/heads/master into remove-passwords-dir
2020-08-07 00:02:30 +00:00
Raul Jordan
2502b812fc Sort by Derivation Path in File Name When Importing (#6918)
* add sort by derivation in file name

* Update validator/accounts/v2/accounts_import_test.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>

* imports

* fixed var

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-08-06 18:40:49 -05:00
terence tsao
ceee7489dd Update SignRequest for key manager (#6917)
* Proto: update sign request fields

* Validator client: aggregate signing

* Validator client: proposal signing

* Validator client: attestation signing

* Validator client: update generic sign

* Proto: use ranges

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-08-06 18:18:38 -05:00
Raul Jordan
6b4e4bcf96 Fail Immediately if Importing Accounts from Empty Directory (#6915)
* fail on import from empty directory
* Merge refs/heads/master into fail-on-empty-dir
2020-08-06 22:40:14 +00:00
Raul Jordan
96fcb3a927 Fix Pubkey Not Specified Issues in External Keystore Imports (#6914)
* utilize pubkey from privkey if missing pubkey field in keystore
* change tool as well
* Merge refs/heads/master into strange-keystore
* Merge refs/heads/master into strange-keystore
2020-08-06 21:56:25 +00:00
terence tsao
f0de09d2ad Check before saving attestation to pool (#6912)
* Pool: check before save atts
* Pool: test
* Pool: update metrics
* Merge branch 'master' into check-before-save
* Merge refs/heads/master into check-before-save
2020-08-06 21:02:58 +00:00
Raul Jordan
ac82308e03 Allow for Different Language Mnemonic Recovery (#6913)
* allow for different language mnemonics

* imports
2020-08-06 15:33:28 -05:00
terence tsao
5a216de6d2 RPC/proposer: remove dup vs.deleteAttsInPool (#6911) 2020-08-06 11:18:26 -07:00
terence tsao
dc0f2140b3 Rpc service: log error if slasher is not up (#6907)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-08-06 10:13:09 -07:00
rkapka
1d679c9533 Manual code cleanup (#6906)
* replace fmt.Sprintf with String()

* remove unconditional loop termination

* reapply `break` statements

* replace for loop if statement
2020-08-06 11:29:00 -05:00
Nishant Das
5b75f4e6be add feature config support (#6904) 2020-08-06 20:49:41 +08:00
Preston Van Loon
40b68da486 Add trace spans to GetBlock methods for a better understanding of block proposal delays (#6903)
* Add trace spans to GetBlock methods for a better understanding of block proposal delays
2020-08-06 08:43:46 +00:00
Raul Jordan
ec21316efd Tool for Decrypting/Encrypting EIP-2335 Keystores (#6900)
* begin keystore tool

* add bazel

* keystores tool

* fully decrypt

* implement encrypt

* complete encrypt and decrypt implementations

* finalize tool

* gaz

* fix viz

* help

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-08-06 00:26:52 -05:00
terence tsao
89bd5f3a5a Add debug endpoint to get inclusion slot (#6895)
* Proto: add GetInclusionSlot

* Proto: build

* Service: add beacon db to struct

* Debug: Implement GetInclusionSlot

* Debug: prevent underflow

* Debug: add tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-08-05 22:55:39 -05:00
Nishant Das
64ba3c076b Revert "Use Less Intensive Search" (#6899)
* Revert "Use Less Intensive Search (#6848)"

This reverts commit 69e0e302b3.
2020-08-06 02:40:28 +00:00
Noah Citron
440d3288d5 Chain id and netork id flags (#6894)
* added network-id and chain-id flags
* updated usage.go to include new flags
* Merge pull request #1 from prysmaticlabs/master

rebase with master
* gofmt
* Merge branch 'ChainID-and-NetorkID-flags' of github.com:ncitron/prysm into ChainID-and-NetorkID-flags
2020-08-05 22:03:25 +00:00
Shay Zluf
446bfca4f3 Slasher redial (#6889)
* Restart streams on internal error
* debug instead of fatal on retry
* Merge branch 'master' of github.com:prysmaticlabs/prysm into slasher_redial
* goimports
* conn status fix
* Merge branch 'master' into slasher_redial
2020-08-05 21:17:00 +00:00
Ivan Martinez
492944db01 E2E: Add cpu profile and fix logs and sync time (#6872)
* Add cpu profile for beacon node to e2e

* Fix

* Fix again

* Properly kill beacon nodes

* Fix logs

* Try 100%

* Handle cp profile from pprof

* Fix

* Bring expected participation to 98%

* Only enable pprof in non-local running

* Modify sync time

* Fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-08-05 09:58:50 -05:00
Preston Van Loon
0d118df034 Check roughtime more frequently when there is a large clock offset (#6879)
* recheck roughtime more frequently if we think our clock is far off
* Merge refs/heads/master into faster-roughtime-corrections
* gaz
* Merge refs/heads/master into faster-roughtime-corrections
* Merge refs/heads/master into faster-roughtime-corrections
* Merge refs/heads/master into faster-roughtime-corrections
2020-08-05 12:31:13 +00:00
rkapka
e492343ffd Code quality issues - Remove unused code (#6864)
* remove unused code

* gofmt

* remove junk file

* bring back metrics-related functions

* bring back keymanager's code
2020-08-05 13:59:46 +02:00
Preston Van Loon
de8d3ec612 Warn users of upcoming breaking change to default value for eth1 provider flag (#6885)
* Increase WARN to ERR. Mention that this flag will have a breaking change in the next release
2020-08-05 08:41:54 +00:00
terence tsao
016b95385a Remove Fuzzit (#6881) 2020-08-04 21:26:13 -07:00
Raul Jordan
ac0237e9cf Return Err in Wallet.HasDir function (#6878)
* fix panic in has dir
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* return err
2020-08-05 02:07:41 +00:00
Ivan Martinez
c860553306 Handle empty local slashing protection data (#6877)
* Add handlers for empty protection data
* Remove copy
* Merge refs/heads/master into fix-isnewattslashable
* Add regression tests
* Merge refs/heads/master into fix-isnewattslashable
* Merge branch 'fix-isnewattslashable' of github.com:prysmaticlabs/prysm into fix-isnewattslashable
* Fix
* Add debug log
2020-08-05 01:03:40 +00:00
Raul Jordan
1f9d155f0e Fix Panic in Wallet HasDir() (#6857)
* fix panic in has dir
* Merge refs/heads/master into has-dir-panic
* Merge refs/heads/master into has-dir-panic
* Merge refs/heads/master into has-dir-panic
* Merge refs/heads/master into has-dir-panic
* Merge refs/heads/master into has-dir-panic
2020-08-04 23:23:15 +00:00
Preston Van Loon
bc883aab9e Return an error when wallet is nil rather than panic (#6875) 2020-08-04 15:53:34 -07:00
Nishant Das
859e59b6f7 Change Sync Error Logs To Debug Logs (#6865)
* switch over to debug logs
* change back
* Change e2e to debug logging
* Merge refs/heads/master into changeToDebug-2
* Merge refs/heads/master into changeToDebug-2
2020-08-04 20:30:40 +00:00
Shay Zluf
0d6fb72496 Slasher protection race condition fix (#6871)
* Slasher lock on update
* gaz
* Merge branch 'master' of github.com:prysmaticlabs/prysm into slasher_lock_on_update
* goimports
* line space
* ivan feedback
* Merge refs/heads/master into slasher_lock_on_update
2020-08-04 19:50:23 +00:00
terence tsao
ef69ac4496 Validate attestation: fix error log (#6874)
* attestation: fix  error log
2020-08-04 18:37:35 +00:00
Nishant Das
daedb7411c Fix Justified and Finalized Retrieval in Genesis (#6861)
* fix
* fix tests
* add test
2020-08-04 03:34:17 +00:00
Raul Jordan
6af950f001 Add Back Displaying Deposit Data + Misc Bugs (#6856)
* show deposit data tx
* display
* improve logic for derived
* further improve experience
* imports
* gaz
2020-08-03 20:17:34 +00:00
Raul Jordan
700d82236b Fix Critical Account Keystores Bugs (#6854)
* no need for pass in new account create
* disallow duplicates
* deduplicate and test
* gaz
* fix up broken tests
* unit test for no duplicates
* Merge branch 'master' into fix-account-create
* fix import
* Merge refs/heads/master into fix-account-create
* Merge branch 'fix-account-create' of github.com:prysmaticlabs/prysm into fix-account-create
2020-08-03 18:58:04 +00:00
terence tsao
ee7da0d451 ForceCheckpoint return early when zero hashes (#6852)
* Setter: return early with zero hashes
* Test: zero hashes don't panic
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge refs/heads/master into zero-hashes
* Merge refs/heads/master into zero-hashes
* Merge refs/heads/master into zero-hashes
2020-08-03 18:34:54 +00:00
Nishant Das
69e0e302b3 Use Less Intensive Search (#6848)
* use less intensive search
* Add mock method to implement Listener
* Merge refs/heads/master into lessIntensiveSearch
* Merge refs/heads/master into lessIntensiveSearch
2020-08-03 18:10:23 +00:00
terence tsao
d06dcd2eec Update DepositContractFlag default to Medalla (#6853)
* Update medalla deposit contract address
2020-08-03 17:40:05 +00:00
Raul Jordan
593d3daf84 Better Password Input Experience (#6849)
* more sane passwords
2020-08-03 17:05:30 +00:00
Shay Zluf
6582014b80 Fix deposit data copy (#6847)
* fix deposit data copy

* goimports
2020-08-03 16:25:19 +03:00
Nishant Das
e5dde1c6ef fix enrs (#6846) 2020-08-03 20:05:36 +08:00
terence tsao
d3a9b47643 Update medalla bootnodes (#6836)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-08-02 18:05:08 -05:00
Ivan Martinez
7903ec0b12 Fix CMD Config and modify --historical-slasher-node (#6730)
* Fix CMD config overwriting
* Rename and change historical slasher node
* Merge branch 'master' of github.com:prysmaticlabs/prysm into fix-cmd-config
* Fix
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Add storage warning
* Merge branch 'fix-cmd-config' of github.com:prysmaticlabs/prysm into fix-cmd-config
* Remove period
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Add default values test
* Merge branch 'fix-cmd-config' of github.com:prysmaticlabs/prysm into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Gaz
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Fix inits
* Merge branch 'fix-cmd-config' of github.com:prysmaticlabs/prysm into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Fix
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
* Merge branch 'master' of github.com:prysmaticlabs/prysm into fix-cmd-config
* Fix
* Merge refs/heads/master into fix-cmd-config
* Merge refs/heads/master into fix-cmd-config
2020-08-02 22:09:10 +00:00
Ivan Martinez
22e2278812 Add heap file to E2E artifacts and fix miscalculation for deposit processing (#6657)
* Add heap file to e2e output
* Add comment
* Change to curl
* Gaz
* Merge branch 'master' of github.com:prysmaticlabs/prysm into add-heap-e2e
* Run heap at start and end
* Reorder defers
* Merge branch 'master' of github.com:prysmaticlabs/prysm into add-heap-e2e
* Fix
* Fix
* Change to logpath
* Attempt
* Merge branch 'master' of github.com:prysmaticlabs/prysm into add-heap-e2e
* Fix heap file
* Fix
* Add more beaocn nodes
* Final fixes
* Merge branch 'master' of github.com:prysmaticlabs/prysm into add-heap-e2e
* Fix slasher
* Fixes
* Fix
* Fix comment
* Merge branch 'master' into add-heap-e2e
* Merge refs/heads/master into add-heap-e2e
* Merge branch 'master' of github.com:prysmaticlabs/prysm into add-heap-e2e
* Fix comment
* Merge branch 'master' into add-heap-e2e
* Merge refs/heads/master into add-heap-e2e
* Merge branch 'master' of github.com:prysmaticlabs/prysm into add-heap-e2e
* Fix
* Change memusage to 2gb
* Merge refs/heads/master into add-heap-e2e
* Update endtoend/evaluators/metrics.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-08-02 21:30:59 +00:00
Raul Jordan
eef0d058ec Refactor Direct Keymanager to One-Shot Decrypt (#6823) 2020-08-02 13:02:04 -07:00
Ivan Martinez
37343930dd Change E2E config to test deposits in PR (#6824)
* Change config to allow deposit in PR e2e

* Fix comment

* Fix

* Fix

* Fix

* Fix

* Fix

* Remove pid handling

* Fix build.bazel

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-08-01 12:22:53 -05:00
Nishant Das
002bbf53d0 Remove Custom Genesis Delay (#6827)
* remove custom delay
* gaz
* Merge branch 'master' into removeCustomGenesisDelay
2020-08-01 16:11:36 +00:00
Victor Farazdagi
bf59530d93 Adds block provider scorer (#6756)
* prepares peer scorer collection

* decouples scoring service into separate object

* updates references

* renames package

* removes redundant test init

* gazelle

* gofmt

* updates comment

* fix build

* adds block provider scorer

* score rounding factor constant (per Nishant's suggestion)

* updates penalty applying

* updates score block provider tests

* updates scorer tests

* expand test suite

* get rid of penalties + counters for requested/returned blocks

* removes start score

* fixes provider test

* fixes scorer manager tests

* updates comments

* moves roundScore test function

* maxscore tests

* improves test coverage

* Update beacon-chain/p2p/peers/score_block_providers.go

Co-authored-by: Nishant Das <nishdas93@gmail.com>

* renames var to make it less ambigous - per Nishant's suggestion

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2020-08-01 12:20:05 +03:00
Nishant Das
0e1fb65476 add bootnodes (#6807)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-07-31 20:34:44 -05:00
terence tsao
c7c1c660ab Load config after Configure config (#6821)
* Node: load config after configure
* Merge refs/heads/master into load-config
2020-08-01 00:46:55 +00:00
terence tsao
eb0ad0669f Fix empty wallet check (#6819) 2020-07-31 17:08:40 -07:00
Raul Jordan
4dfe02d919 Fix Accounts List Show Deposit Data When Coming From Eth2.0 Deposit CLI (#6811)
* amend accounts list when coming from launchpad
* Merge refs/heads/master into fix-accounts-list
* Merge refs/heads/master into fix-accounts-list
* Merge refs/heads/master into fix-accounts-list
* Merge refs/heads/master into fix-accounts-list
* Merge refs/heads/master into fix-accounts-list
2020-07-31 22:08:13 +00:00
Preston Van Loon
821f2ec029 Accounts V2: Allow user to override password-dir set in the wallet config (#6817)
* Accounts V2: Log a warning when a user specifies --password-dir when it has already been set.
* Override wallet config, as user intended
* gofmt
* Merge refs/heads/master into av2-flag-improvement
2020-07-31 21:20:56 +00:00
Preston Van Loon
dbd2d9a0bb Accounts V2: improve progressbar feedback (#6814)
* Accounts V2: improve progressbar feedback
* gofmt
* Merge refs/heads/master into pb-improvements
2020-07-31 20:46:03 +00:00
Preston Van Loon
3f983e2a96 Accounts V2: Allow for user to specify single file rather than directory for keystore import (#6813)
* Fix public key formatting
* Accounts V2: Allow for user to specify single file rather than directory
* Merge branch 'master' into fix-pubkey-formatting
* Merge refs/heads/master into fix-pubkey-formatting
2020-07-31 20:17:39 +00:00
Raul Jordan
7a1e5b1dd9 Allow spaces in keystore account passwords (#6810)
* allow for spaces in pass
* handle spaces with tests
* Merge refs/heads/master into handle-spaces
2020-07-31 19:50:44 +00:00
Preston Van Loon
bf82683d4f Fix public key formatting (#6809)
* Fix public key formatting
2020-07-31 19:23:08 +00:00
Preston Van Loon
323bf70b85 Add a script to tag docker images as stable or by their git tag version (#6808)
* Add a script to tag docker images as stable or by their git tag version
* Apply suggestions from code review

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
* Always use HEAD
2020-07-31 18:25:14 +00:00
Nishant Das
f414cf36b9 Revert "Reduce E2E idle time to expedite completion (#6762)" (#6793)
* Revert "Reduce E2E idle time to expedite completion (#6762)"

This reverts commit 187cae8290.
* Merge branch 'master' into revertE2EChanges
* fix
* Merge refs/heads/master into revertE2EChanges
* Change size
2020-07-31 02:42:18 +00:00
terence tsao
f7c8ae5aa6 Config flag: onyx (#6789)
* Config/flags: add onyx back
* Sync with master
* Merge refs/heads/master into onyx
* Merge refs/heads/master into onyx
* onyx flag
* Merge refs/heads/master into onyx
* Merge refs/heads/master into onyx
2020-07-31 01:23:37 +00:00
terence tsao
0cced6e41d Roughtime: use debug instead of info log (#6792)
* Roughtime: use debug instead of info log
* Merge refs/heads/master into roughtime-debug
* Merge refs/heads/master into roughtime-debug
2020-07-31 00:00:24 +00:00
Raul Jordan
18f8578d73 Derived Keymanager Has No Need to Store Accounts to Disk (#6790)
* tests pass
* Merge branch 'master' into derive-no-store
* Merge refs/heads/master into derive-no-store
* gaz
* Merge branch 'derive-no-store' of github.com:prysmaticlabs/prysm into derive-no-store
* Merge refs/heads/master into derive-no-store
2020-07-30 23:23:18 +00:00
Raul Jordan
b2ae4243fd Concurrent Decryption of Direct Keymanager Accounts With Progress Bar (#6788)
* progress bar add

* Update validator/keymanager/v2/direct/direct.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update validator/keymanager/v2/direct/direct.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* fix test

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-07-30 17:54:15 -05:00
terence tsao
0b77f4f84c Update badge to v0.12.2 (#6740)
* Update badge to v0.12.2
* Merge refs/heads/master into terencechain-patch-1
* Merge refs/heads/master into terencechain-patch-1
* Merge refs/heads/master into terencechain-patch-1
* Merge refs/heads/master into terencechain-patch-1
* Merge refs/heads/master into terencechain-patch-1
* Merge refs/heads/master into terencechain-patch-1
2020-07-30 21:56:36 +00:00
terence tsao
a116502c76 Accounts v2 as default (#6784)
* Featureconfig: invert accounts v2
* Test: fixing validator node
* Test: Fixed node test
* Merge refs/heads/master into account-v2-default
* Merge refs/heads/master into account-v2-default
* Merge refs/heads/master into account-v2-default
* Merge refs/heads/master into account-v2-default
* Merge refs/heads/master into account-v2-default
* Merge refs/heads/master into account-v2-default
2020-07-30 21:00:02 +00:00
Nishant Das
2011896005 Fix Local IP Address (#6781)
* fix local ip
* Merge refs/heads/master into fixLocalIP
* Merge refs/heads/master into fixLocalIP
* Merge refs/heads/master into fixLocalIP
* Merge refs/heads/master into fixLocalIP
* Merge refs/heads/master into fixLocalIP
* Merge refs/heads/master into fixLocalIP
* Merge refs/heads/master into fixLocalIP
2020-07-30 20:23:30 +00:00
Raul Jordan
bd7046c790 Improve Speed of Accounts Import (#6785)
* improve speed of import
* better format
* imports
* Merge branch 'master' into fast-import
* Merge refs/heads/master into fast-import
* Merge refs/heads/master into fast-import
* Merge refs/heads/master into fast-import
* Merge refs/heads/master into fast-import
2020-07-30 19:42:16 +00:00
terence tsao
ec4de10001 Add helper ComputeDomainAndSign (#6773)
* Starting
* Merge branch 'master' of github.com:prysmaticlabs/prysm into domain-and-sign
* core blocks tests: updated
* core state tests: updated
* rpc validator tests updated
* sync tests updated
* testutil updated
* Beacon-chain: update tests
* Shared: update tests
* Rest of the tests
* Merge branch 'master' of github.com:prysmaticlabs/prysm into domain-and-sign
* Merge refs/heads/master into domain-and-sign
* Merge refs/heads/master into domain-and-sign
* Merge refs/heads/master into domain-and-sign
* Merge refs/heads/master into domain-and-sign
* Sync with master
* Merge branch 'domain-and-sign' of github.com:prysmaticlabs/prysm into domain-and-sign
* Merge refs/heads/master into domain-and-sign
2020-07-30 19:11:22 +00:00
Nishant Das
edc9fcd0d6 Harden Against DOS Attacks (#6780)
* harden
* gaz
* trace log
* Update beacon-chain/p2p/service.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
* yield go-routine
* Merge refs/heads/master into hardenAgainstDOS
* Merge branch 'hardenAgainstDOS' of https://github.com/prysmaticlabs/geth-sharding into hardenAgainstDOS
* Merge refs/heads/master into hardenAgainstDOS
* Merge refs/heads/master into hardenAgainstDOS
* Merge refs/heads/master into hardenAgainstDOS
* Merge refs/heads/master into hardenAgainstDOS
2020-07-30 18:42:22 +00:00
Preston Van Loon
dec779dc1e Disable --enable-eth1-data-majority-vote in e2e (#6787)
* Disable --enable-eth1-data-majority-vote in e2e. See https://github.com/prysmaticlabs/prysm/issues/6786
* Merge refs/heads/master into disable-vote-with-majority
2020-07-30 18:14:07 +00:00
terence tsao
b56f473800 Unskip TestSetupInitialDeposits_1024Entries (#6775)
* Unskip TestSetupInitialDeposits_1024Entries
* Merge refs/heads/master into unskip-init-deposits
* Merge refs/heads/master into unskip-init-deposits
* Merge branch 'master' of github.com:prysmaticlabs/prysm into unskip-init-deposits
* Sync with master
* Merge branch 'unskip-init-deposits' of github.com:prysmaticlabs/prysm into unskip-init-deposits
* Merge refs/heads/master into unskip-init-deposits
* Merge refs/heads/master into unskip-init-deposits
* Merge refs/heads/master into unskip-init-deposits
* Merge refs/heads/master into unskip-init-deposits
2020-07-30 17:38:34 +00:00
Nishant Das
fd0383d17e Update Libp2p Deps (#6782)
* update libp2p deps
* Merge branch 'master' into updateDeps
* tidy
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into updateDeps
* Merge branch 'updateDeps' of https://github.com/prysmaticlabs/geth-sharding into updateDeps
* revert grpc
2020-07-30 16:39:57 +00:00
Ivan Martinez
48bfffbb3e Accounts-V2: Cleanup password entry and disable export (#6778)
* Cleanup prompts
* Merge branch 'master' of github.com:prysmaticlabs/prysm into cleanup-accounts-v2
* Fix
* add todo
* Merge branch 'master' of github.com:prysmaticlabs/prysm into cleanup-accounts-v2
* Clean
* Merge refs/heads/master into cleanup-accounts-v2
* Merge refs/heads/master into cleanup-accounts-v2
* Change with error
* Merge branch 'cleanup-accounts-v2' of github.com:prysmaticlabs/prysm into cleanup-accounts-v2
* Merge refs/heads/master into cleanup-accounts-v2
* cleanup and remove deposit tx file
* display deposit data
2020-07-30 15:30:19 +00:00
terence tsao
bcea2c1b22 Config/flag: medalla as default (#6770)
* config: use medalla as default

* Tests: align with medalla spec

* Config/flag: deprecate medalla feature flag

* Update shared/featureconfig/flags.go

* Spectest: set genesis forkversion to 0,0,0,0

* Config: set medalla as defat in Configure

* Extra space

* Shard: use copy

* Revert a p2p fork test

* Update default medalla for tools

* Use medalla config for e2e

* Fix E2E

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
2020-07-30 08:45:00 -05:00
Ivan Martinez
187cae8290 Reduce E2E idle time to expedite completion (#6762)
* Make changes to E2E to finish faster

* rename

* Fix

* reduce slots per epoch

* Remove process ID

* Fix

* Fix

* Fix

* Remove pid handling

* Fix build

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-07-29 21:58:30 -05:00
Nishant Das
353bec600f Fix Handshake (#6776)
* remove security issue
* Merge branch 'master' into fixHandshake
2020-07-30 01:04:35 +00:00
Raul Jordan
bde18f0329 Resolve Remaining Accounts Bugs (#6769)
* fix cases
* fix up imports
* ask until successful
* fixed imports flow
* fully utilize progress bar
* tidy
* grpc
* add round trip test fix
* properly use seeds
* update deps
* update deps
* tidy
* rem print
* fix bazel tests
* rem extraneous log
* gaz
* go mod tidy
* grpc version
* fix deps
* Merge branch 'master' into empty-or-nonexistent
* deps
* tidy
* option show bytes
* Merge refs/heads/master into empty-or-nonexistent
2020-07-29 23:54:23 +00:00
Raul Jordan
62c92af247 Remove Deprecated SSZ Decode Edge Case for BeaconBlocksByRoots (#6771)
* remove temp unmarshal edge case
* removed deprecated func
* final err fix
* Merge refs/heads/master into fix-exploit
2020-07-29 23:15:14 +00:00
Raul Jordan
f9a08dd5d2 Revert "Update Ethereumapis" (#6772)
* Revert "Update Ethereumapis (#6761)"

This reverts commit e94e62b98f.
2020-07-29 22:23:13 +00:00
Nishant Das
ecf197c11e Update Pubsub Parameters (#6744)
* initial commit

* fix

* lint

* clean up

* fix bug
2020-07-29 11:14:15 -05:00
Victor Farazdagi
84697a0a59 Adds peer scorer manager (#6752)
* prepares peer scorer collection
* decouples scoring service into separate object
* updates references
* renames package
* removes redundant test init
* gazelle
* gofmt
* Merge branch 'master' into peer-scorer-manager
* updates comment
* fix build
* Merge refs/heads/master into peer-scorer-manager
* Merge refs/heads/master into peer-scorer-manager
* Merge refs/heads/master into peer-scorer-manager
* Merge refs/heads/master into peer-scorer-manager
* Merge refs/heads/master into peer-scorer-manager
* Merge refs/heads/master into peer-scorer-manager
* Merge refs/heads/master into peer-scorer-manager
* Merge refs/heads/master into peer-scorer-manager
* Merge refs/heads/master into peer-scorer-manager
* Merge refs/heads/master into peer-scorer-manager
2020-07-29 09:26:46 +00:00
Raul Jordan
e5f3a30226 Allow Relative Imports in Accounts-v2 (#6758)
* add abs imports tests

* relative imports now work

* misc

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-07-29 11:33:28 +03:00
Preston Van Loon
e94e62b98f Update Ethereumapis (#6761)
* Update Ethereumapis
* Merge refs/heads/master into update-ethereumapis-1
* Merge refs/heads/master into update-ethereumapis-1
2020-07-29 06:03:52 +00:00
terence tsao
d6eb6d3f41 Move ValidateAttestationTime earlier for sync (#6755)
* Move time checks to the start
* Merge branch 'master' of github.com:prysmaticlabs/prysm into check-time-first
* Merge refs/heads/master into check-time-first
* Merge refs/heads/master into check-time-first
* Merge refs/heads/master into check-time-first
* Merge refs/heads/master into check-time-first
* Merge refs/heads/master into check-time-first
2020-07-29 05:31:50 +00:00
Ivan Martinez
fac5e19a17 Accounts-V2: Replace promptui with shared/promptutil (#6759)
* Add promptutil to shared/promptutil
* fixes
* comment
* Implement promptutil into accounts-v2
* gaz
* Merge branch 'master' of github.com:prysmaticlabs/prysm into implement-promptutil
* Apply suggestions from code review
* gofmt and fix all scanned input
* Merge refs/heads/master into implement-promptutil
2020-07-29 04:55:26 +00:00
Fabrice Cheng
cbd731152e Return INVALID validator status if the BLS signature is invalid (#6576)
* return INVALID status if BLS signature is invalid

* add unit tests for VerifyDepositSignature function

* fix lint

* lint fix

* gazelle run run

* update ethereumapis dependencies in go module

* tidy go mod

* update VerifyDepositSignature function to use the proper way of generating sig

* update tests

* update gazelle

* add ValidatorStatus test

* update to Warn

* reorder impot

* merge verifyDepositDataSigningRoot with VerifyDepositSignature

* refactor one last time

* align with specs for computing Domain

* add warn error logs
2020-07-28 23:19:07 -05:00
Ivan Martinez
fc0585f724 Add promptutil to shared/promptuil (#6757)
* Add promptutil to shared/promptutil
* fixes
* comment
2020-07-29 03:04:14 +00:00
Raul Jordan
9d08ba49de Accounts Revamp Fixes: "Overall" Wallet Improvements (#6736)
* change default wallet dir path to not be hidden

* gaz + pass wallet dir

* gaz + move const to flags

* move to flags

* move to flags

* use filepath join in order to create a valid dir name

* add wallet dir

* return err no wallet found issues

* fix up edit remote

* all tests passing

* fix test

* create or open wallet

* ivan feedback

* enter password for account with pubkey

* Update validator/accounts/v2/accounts_create.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>

* works

* preston feedback

* nothing to export

* fmt

* test for create or open

* gaz

Co-authored-by: shayzluf <thezluf@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
2020-07-28 20:20:13 -05:00
Preston Van Loon
1a1c1bb813 Return 503 on healthz with error (#6754)
* Return 503 on healthz with error
* fix other test
* Update shared/prometheus/service_test.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-07-29 00:31:21 +00:00
Ivan Martinez
3a609f44b9 Accounts-V2: Minor fixes for logging (#6751)
* Fixes for accounts-v2
* Merge branch 'master' into fixes
* Undo wallet changes
* Merge branch 'fixes' of github.com:prysmaticlabs/prysm into fixes
* Merge refs/heads/master into fixes
2020-07-28 22:58:11 +00:00
terence tsao
c0d6a231bf Faster ancestor root look up via fork choice store (#6753)
* forkchoice: add `HasParent` and `AncestorRoot` getters
* Tests
* Interfaces
* process block: use getters in `ancestor`
* Update tests
* Merge branch 'master' of github.com:prysmaticlabs/prysm into faster-ancestor-block
* feedback from @rauljordan
2020-07-28 22:29:34 +00:00
rkapka
becb0b19e2 Voting With The Majority (#6644)
* eth1DataMajorityVote draft
* Merge branch 'origin-master' into eth1-majority-voting
* small test change
* move minimal config's HeadOfVotingPeriodLength closer to sqroot(32)
* remove todo
* remove head period and simplify equality check
* extract private functions from eth1DataMajorityVote
* feature flag
* Merge branch 'origin-master' into eth1-majority-voting
* add bytes import to tests
* Merge branch 'master' into eth1-majority-voting
* re-run build
* Merge branch 'master' into eth1-majority-voting
* Merge branch 'master' into eth1-majority-voting
* Merge branch 'master' into eth1-majority-voting
2020-07-28 20:29:12 +00:00
Ivan Martinez
ee1adddd24 Accounts V2: Fix issues reported by rkapka (#6725)
* Fix accounts v2 runtime
* Accounts V2: Fix issues found by rkapka
* Add to help
* Merge refs/heads/master into fix-v2-runtime
* Merge refs/heads/master into fix-v2-runtime
* Merge refs/heads/master into fix-v2-runtime
* Merge refs/heads/master into fix-v2-runtime
* Fixes
* add back error
* Make new wallet error cleaner
* Fix text
* Fix log
* Add case for normal error
* Update validator/flags/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Merge branch 'master' of github.com:prysmaticlabs/prysm into fix-v2-runtime
* Merge refs/heads/master into fix-v2-runtime
* Fix comment
* Merge refs/heads/master into fix-v2-runtime
* fix comment
* Fix test
* Merge branch 'master' into fix-v2-runtime
* Merge refs/heads/master into fix-v2-runtime
* Merge refs/heads/master into fix-v2-runtime
* Merge refs/heads/master into fix-v2-runtime
* Merge refs/heads/master into fix-v2-runtime
2020-07-28 19:23:53 +00:00
Preston Van Loon
06ee5695fb Save finalized checkpoint on beacon chain close (#6431)
* Save finalized checkpoint on save

* gofmt and goimport

* remove head.root arg

* test

* fixes

* remove comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-07-28 13:35:17 -05:00
terence tsao
027ee4c18c Add a ComputeDomainVerifySigningRoot helper (#6743)
* Add ComputeDomainVerifySigningRoot
* Update validate aggregate proof signatures to use it
* Update verify proposer slashing to use it
* Update test to use it
* Merge branch 'master' into domain-and-signingroot
* Quick fix on test
* Merge branch 'domain-and-signingroot' of github.com:prysmaticlabs/prysm into domain-and-signingroot
* Merge branch 'master' of github.com:prysmaticlabs/prysm into domain-and-signingroot
* Merge refs/heads/master into domain-and-signingroot
* Merge refs/heads/master into domain-and-signingroot
2020-07-28 16:49:47 +00:00
Nishant Das
8a69d34b96 Fix Accounts Creation Error (#6748)
* add check
* Merge refs/heads/master into fixAccountsError
* Merge refs/heads/master into fixAccountsError
2020-07-28 16:06:18 +00:00
Nishant Das
7861005990 Allow Multiple Selection For Exports (#6749)
* add changes
* Merge branch 'master' into fixExports
* last request
* Merge branch 'fixExports' of https://github.com/prysmaticlabs/geth-sharding into fixExports
2020-07-28 15:31:37 +00:00
Ivan Martinez
607d5fdf4e Accounts-V2: Change accounts-v2 import to use directory of keystores (#6742)
* Finish implementation of importing dir of keystores

* Fix

* Fix importing

* Finish testing import account

* Update validator/accounts/v2/accounts_import.go

* Fix

* Fix

* fix

* Fix prompt

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-07-28 09:18:22 -05:00
Ivan Martinez
0cd80bb970 Accounts-V2: Change keystore filename to include timestamp (#6741)
* Change keystore filename to include timestamp
* Fix test
* Merge branch 'master' into change-keystore-filename
* Fix comment
* Merge branch 'change-keystore-filename' of github.com:prysmaticlabs/prysm into change-keystore-filename
* Request comments
* Comment
* Add comment
* Fix test
* Fix test
* Add test for AccountTimestamp
2020-07-28 00:02:29 +00:00
Shay Zluf
a867f6175b Make external slasher protection fail on load if slasher is not reachable (#6704)
* external slasher protection fails loudly
* Merge refs/heads/master into external-slasher-fail-loud
* Merge refs/heads/master into external-slasher-fail-loud
* Merge refs/heads/master into external-slasher-fail-loud
* Merge refs/heads/master into external-slasher-fail-loud
* Merge refs/heads/master into external-slasher-fail-loud
* Merge refs/heads/master into external-slasher-fail-loud
* Merge refs/heads/master into external-slasher-fail-loud
* Merge refs/heads/master into external-slasher-fail-loud
* Merge refs/heads/master into external-slasher-fail-loud
* Merge refs/heads/master into external-slasher-fail-loud
* Merge refs/heads/master into external-slasher-fail-loud
* Merge refs/heads/master into external-slasher-fail-loud
* Merge refs/heads/master into external-slasher-fail-loud
* Merge refs/heads/master into external-slasher-fail-loud
* Merge refs/heads/master into external-slasher-fail-loud
* Merge refs/heads/master into external-slasher-fail-loud
* Merge refs/heads/master into external-slasher-fail-loud
* Merge refs/heads/master into external-slasher-fail-loud
* Merge refs/heads/master into external-slasher-fail-loud
* Merge refs/heads/master into external-slasher-fail-loud
* Merge refs/heads/master into external-slasher-fail-loud
* Merge branch 'master' of github.com:prysmaticlabs/prysm into external-slasher-fail-loud
* Merge refs/heads/master into external-slasher-fail-loud
* Merge refs/heads/master into external-slasher-fail-loud
* Merge refs/heads/master into external-slasher-fail-loud
* ivan feedback
* Merge branch 'external-slasher-fail-loud' of github.com:prysmaticlabs/prysm into external-slasher-fail-loud
* lint fix
* ivan feedback
2020-07-27 20:06:49 +00:00
terence tsao
905a57cbd7 Fix checkForChainStart pre-genesis state and 0 validator (#6739) 2020-07-27 10:35:01 -07:00
Ivan Martinez
7fff4ec411 Accounts V2: Add --num-accounts to HD wallet commands (#6721)
* Add num-accounts to wallet recover and account create for derived wallets

* Fix

* Test derived accounts

* Stuff

* Gaz

* Fix

* Fix

* Fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-07-27 09:03:30 -05:00
Nishant Das
664349e553 Account for Bad Blocks in Gossip (#6726)
* add bad block cache

* terence's review

* terence's review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-07-27 19:57:19 +08:00
terence tsao
3a852bc032 eth1 chain ID and network checks (#6727)
* Add config
* Update config values
* Add checks
* Merge refs/heads/master into eth1network
* Merge refs/heads/master into eth1network
* Update e2e test config for devnet
* Merge branch 'eth1network' of github.com:prysmaticlabs/prysm into eth1network
2020-07-27 04:48:36 +00:00
Victor Farazdagi
630cbfb1b6 Fixes issue with concurrent map writes in gossipsub (#6731)
* Fixes issue with concurrent map writes in gossipsub
* Merge refs/heads/master into fix-concurrent-map-write-in-pubsub
2020-07-27 00:56:55 +00:00
terence tsao
ca04621849 saveHeadNoDB does not invalidate hot state cache (#6729) 2020-07-26 17:10:35 -07:00
Nishant Das
c8761ffb18 Revert "Update rules go to v0.22.9 (#6712)" (#6724)
This reverts commit efc24f6aab.
2020-07-26 12:28:42 +08:00
Nishant Das
2cd359bde5 preset memory (#6722) 2020-07-25 11:54:54 -07:00
Nishant Das
7a12fc6d75 Fix Metadata Sequence Checks (#6711)
* checkpoint

* Update beacon-chain/sync/rpc_test.go

* change conditional

* fix test

* fmt

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-07-25 11:53:41 -05:00
Nishant Das
ada351d087 Applies Assertion And Require Test Functions to Powchain (#6652)
* change all tests
* save
* fix tests
* Update deposit_test.go
* Update deposit_test.go
* Update log_processing_test.go
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* fix failing test
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
* Merge refs/heads/master into powchainAssert
2020-07-25 06:20:21 +00:00
terence tsao
d114c6c6b7 Update comments for get_committee_count_per_slot (#6717)
* Update comment
* Merge branch 'master' into epochwise_committee_count_per_slot
* Merge refs/heads/master into epochwise_committee_count_per_slot
* Merge refs/heads/master into epochwise_committee_count_per_slot
* Merge refs/heads/master into epochwise_committee_count_per_slot
* Merge refs/heads/master into epochwise_committee_count_per_slot
* Merge refs/heads/master into epochwise_committee_count_per_slot
* Merge refs/heads/master into epochwise_committee_count_per_slot
2020-07-25 05:29:30 +00:00
terence tsao
01f3a6e6db Update on block/attestation comments (#6720)
* Update on att comments
* Update on block comments
* Merge refs/heads/master into forkchoice-comments
* Merge refs/heads/master into forkchoice-comments
2020-07-25 04:55:42 +00:00
Preston Van Loon
efc24f6aab Update rules go to v0.22.9 (#6712)
* bump rules_go

* reapply patch, remove commented code

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-07-25 12:14:17 +08:00
Preston Van Loon
715a33532d Update ethereumapis (#6719)
* Update ethereumapis
* go mod tidy
* Merge refs/heads/master into update-ethereumapis
2020-07-25 01:03:59 +00:00
terence tsao
366e53c416 Add committee ID check to process attestation (#6718) 2020-07-24 17:12:15 -07:00
Preston Van Loon
e1f33b196d Update spectests to v0.12.2 (#6713)
* Update spectests to v0.12.2
* Update readme to v0.12.2
* Revert "Update readme to v0.12.2"

This reverts commit 34461a1d05.
* Merge branch 'master' into update-spectests-v0.12.2
* fix config path
* Merge refs/heads/master into update-spectests-v0.12.2
2020-07-24 22:25:31 +00:00
Raul Jordan
55009bfa66 Fix Validator Accounts-V2 Runtime (#6714)
* fix accounts v2 runtime
2020-07-24 21:36:28 +00:00
terence tsao
0b9212c685 Revert "Optimize AppendValidator for beacon state" (#6710)
* Revert "Optimize `AppendValidator` for beacon state (#6659)"

This reverts commit c41e382255.
* Test. H/t to @prestonvanloon
* Merge branch 'master' of github.com:prysmaticlabs/prysm into revert-6659-deposit
* Merge branch 'master' into revert-6659-deposit
2020-07-24 05:43:15 +00:00
terence tsao
403dab43f5 Use config DepositContractTreeDepth (#6708) 2020-07-23 21:30:28 -07:00
Ivan Martinez
dfdff37a3e Accounts V2: Fix import command (#6707)
* Fix import
* Merge branch 'master' into fix-import-v2
* Fix
* Merge branch 'fix-import-v2' of github.com:prysmaticlabs/prysm into fix-import-v2
* Fix for comments
* Merge refs/heads/master into fix-import-v2
* Merge branch 'master' into fix-import-v2
2020-07-24 03:19:28 +00:00
Ivan Martinez
3974104a20 Accounts V2: Rename new->create, rename files and usage text fixes (#6695)
* Accounts V2: Renames and usage text fixes

* Fix
2020-07-23 21:18:36 -05:00
Raul Jordan
a5b408769a Accounts V2: Resolve Remaining Keymanager Bugs (#6706)
* v2 fix bugs
* better doc
* include wallet build fix
* fixed broken list test
* add round trip recover seed unit test
* imports
* implement list with tests
* add altona flags
* tests for unicode
* added is valid unicode tests
* fixed up tests to ensure wallet is persisted after everything works
* resolve confs and integrate medalla testnet
* fix build
* add medalla
* fixed import spacing
2020-07-24 00:43:01 +00:00
terence tsao
cc773a1641 Rm unused HashTreeRootState (#6697)
* Remove hash tree root implementations

* Delete benchmark tests

* Update existing tests

* Gaz

* Remove outdated ssz cache fuzz test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-07-23 17:19:05 -05:00
Jim McDonald
415cb9ff8d Add medalla testnet configuration. (#6700) 2020-07-23 14:56:39 -05:00
Preston Van Loon
2c11fcb242 Stategen: always invalidate hot cache state in StateByRootInitialSync (#6698)
* Stategen: always invalidate hot cache state in StateByRootInitialSync
* Sanity test. @nisdas feedback
* Merge refs/heads/master into defer-invalidate
2020-07-23 04:25:50 +00:00
Ivan Martinez
c72db6f96a Accounts V2: Refactor prompts, deduplicate into directory and password (#6674)
* Refactor input code, deduplicate into directory and password
* Change to filepath
* Rename
* Make consts visible to whole validator package
* Fix
* Try commands and fix
* Fix defaults
* Fixes
* Merge branch 'master' into refactor-input-prompts
* Merge refs/heads/master into refactor-input-prompts
* Merge refs/heads/master into refactor-input-prompts
* Merge refs/heads/master into refactor-input-prompts
* Merge refs/heads/master into refactor-input-prompts
* Remove consts and add comment
* Merge branch 'refactor-input-prompts' of github.com:prysmaticlabs/prysm into refactor-input-prompts
* Undo
* Fixes
* Update validator/accounts/v2/prompt.go
* Merge refs/heads/master into refactor-input-prompts
* gofmt
* Merge refs/heads/master into refactor-input-prompts
* Merge refs/heads/master into refactor-input-prompts
* Merge refs/heads/master into refactor-input-prompts
* Merge refs/heads/master into refactor-input-prompts
* Merge branch 'master' into refactor-input-prompts
* Merge refs/heads/master into refactor-input-prompts
* Fix
* Merge refs/heads/master into refactor-input-prompts
* Merge refs/heads/master into refactor-input-prompts
* Accounts V2: Simplify Wallet Save/Read To and From Disk Functions (#6686)

* simplify wallet functions

* fix build

* futher simplify wallet

* simplify read/write methods

* move direct functions to direct keymanager

* further move direct km specific funcs

* cleanup

* simplify the direct tests

* fixed tests

* lint

* further simplify

* tidy

* fix config write

* fixed test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* conflict resolution
* trim
2020-07-23 03:10:23 +00:00
Raul Jordan
7c52ef8c2b Accounts V2: Simplify Wallet Save/Read To and From Disk Functions (#6686)
* simplify wallet functions

* fix build

* futher simplify wallet

* simplify read/write methods

* move direct functions to direct keymanager

* further move direct km specific funcs

* cleanup

* simplify the direct tests

* fixed tests

* lint

* further simplify

* tidy

* fix config write

* fixed test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-07-22 21:12:51 -05:00
Victor Farazdagi
0006377aec Applies assertion funcs to validator tests (#6694)
* Applies assertion funcs to validator/keymanager/v1 tests
* gazelle
* Applies assertion funcs to validator/keymanager/v2 tests
* Applies assertion funcs to validator/misc tests
* Applies assertion funcs to validator/misc tests
* Merge branch 'master' into validator-apply-testutils-assertions
* gazelle
* Merge branch 'master' into validator-apply-testutils-assertions
* Merge branch 'master' into validator-apply-testutils-assertions
* Merge refs/heads/master into validator-apply-testutils-assertions
2020-07-23 01:13:52 +00:00
Preston Van Loon
208659d7f5 Enable local slashing protection by default (#6687)
* Enable local slashing protection by default
* Merge refs/heads/master into enable-local-protection-by-default
* Merge refs/heads/master into enable-local-protection-by-default
* Merge refs/heads/master into enable-local-protection-by-default
* Merge refs/heads/master into enable-local-protection-by-default
* Merge refs/heads/master into enable-local-protection-by-default
* Merge refs/heads/master into enable-local-protection-by-default
2020-07-23 00:47:19 +00:00
Ivan Martinez
cf57db910c Accounts V2: Remove traces from command errors (#6692)
* Remove traces from errors for accounts command
* Merge refs/heads/master into remove-stack
* Merge refs/heads/master into remove-stack
* Merge refs/heads/master into remove-stack
* Merge refs/heads/master into remove-stack
* Merge refs/heads/master into remove-stack
2020-07-23 00:11:00 +00:00
Ivan Martinez
73a9429f1e Accounts V2: Remove CanUnlockAccounts from Wallet (#6693)
* Remove CanUnlockAccounts from Wallet
* Merge refs/heads/master into remve-canunlockaccounts
* Merge refs/heads/master into remve-canunlockaccounts
* Merge refs/heads/master into remve-canunlockaccounts
2020-07-22 23:26:39 +00:00
Raul Jordan
cab89c37f1 Accounts V2: Remove Mnemonic Confirm From Direct Keymanager (#6696)
* remove mnemonic from direct
* Merge refs/heads/master into remove-mnemonic-direct
2020-07-22 22:52:31 +00:00
Ivan Martinez
7d80415089 Move deposit utils to depositutil package (#6690)
* Move deposit utils to depositutil package
* gofmt
* Gaz
* Merge refs/heads/master into move-deposit-utils
* Merge refs/heads/master into move-deposit-utils
2020-07-22 22:21:41 +00:00
Preston Van Loon
f19770a393 Revert "Add Github Superlinter" (#6691)
* Revert "Add Github Superlinter (#6677)"

This reverts commit 6270347fdb.
* Merge refs/heads/master into revert-6677-rauljordan-patch-1
2020-07-22 21:48:04 +00:00
rkapka
95efcb6410 Applies assertion funcs to forkchoice tests (#6685)
* refactored 3 files
* refactored the rest of files
* Merge branch 'origin-master' into refactor-forkchoice-tests
* gazelle fix
* Merge branch 'master' into refactor-forkchoice-tests
* Merge branch 'master' into refactor-forkchoice-tests
* Merge branch 'master' into refactor-forkchoice-tests
2020-07-22 21:11:03 +00:00
Raul Jordan
6270347fdb Add Github Superlinter (#6677)
* Add Github Superlinter

This PR adds the Github superlinter to our codebase as a Github action. Resolves #6308.
* Apply suggestions from code review
* Merge refs/heads/master into rauljordan-patch-1
* fix action
* Merge refs/heads/master into rauljordan-patch-1
* Merge refs/heads/master into rauljordan-patch-1
* Merge refs/heads/master into rauljordan-patch-1
* Merge refs/heads/master into rauljordan-patch-1
* Merge refs/heads/master into rauljordan-patch-1
2020-07-22 20:22:26 +00:00
Victor Farazdagi
e5b7c3f3e6 Applies assertion funcs to validator/keymanager tests (#6683)
* Applies assertion funcs to validator/keymanager/v1 tests
* gazelle
* Applies assertion funcs to validator/keymanager/v2 tests
* Merge refs/heads/master into validator-keymanager-apply-testutils-assertions
* Merge refs/heads/master into validator-keymanager-apply-testutils-assertions
2020-07-22 19:42:43 +00:00
terence tsao
6c408d31df Remove unused protos and operations (#6684)
* Remove attestation containers

* Remove unused last votes

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-07-22 14:05:41 -05:00
Victor Farazdagi
4400321081 Handles case when no peers with finalized blocks are found (#6679)
* Handles case when no peers with finalized blocks are found
* more tests
* Merge branch 'master' into handle-zero-block-init-sync
* Merge branch 'master' into handle-zero-block-init-sync
2020-07-22 18:14:24 +00:00
terence tsao
17f845dcb0 Update head before send feed (#6681)
* Update head before state feed
* Test
* Merge branch 'master' of github.com:prysmaticlabs/prysm into update-head
2020-07-22 17:05:17 +00:00
Preston Van Loon
62ae22e3f9 Use correct context for chainHeadRetrieval (#6678)
* Use correct context for chainHeadRetrieval
* Merge refs/heads/master into fix-context
2020-07-22 16:20:20 +00:00
Nishant Das
647599e87f fix for virtual networks (#6676) 2020-07-22 10:44:17 -05:00
Raul Jordan
4017743f7f Accounts V2: Refactor Wallet Open / Close and Keymanager Initialization (#6668)
* refactoring create account

* dep

* much easier, create a derived account by simply unlocking wallet

* revert changes to new

* make open wallet smarter and utilize cli ctx

* remove the wallet config

* successfully build

* simplify ctx creation for tests

* tests should pass individually

* tests pass

* fixed up to allow for wallet password file input

* fix broken tests

* formatting

* fmt

* simplify recover

* fixed up tests

* implicit use of default wallet path working

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-07-21 23:49:04 -05:00
Preston Van Loon
28096a846e Better gRPC retry including stream retry (#6670)
* Add gRPC retry delay, apply retry and delay to streams

* gofmt, fix flag

* lint

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-07-21 22:45:52 -05:00
Ivan Martinez
e0d11ff1d2 Accounts V2: Wallet Recover (#6661)
* Implement derived wallet recover
* Fix log
* Fix tests
* Comments
* Merge branch 'master' of github.com:prysmaticlabs/prysm into derived-recover
* Remove log
* Comments
* Fix fatal
* add comment
* Merge refs/heads/master into derived-recover
* Merge refs/heads/master into derived-recover
* Merge refs/heads/master into derived-recover
* Merge refs/heads/master into derived-recover
2020-07-22 02:41:39 +00:00
Ivan Martinez
8a65af168d Accounts V2: Change fatal logs to return errors (#6672)
* Change logging out errors to returns
* Merge branch 'master' of github.com:prysmaticlabs/prysm into change-logs-to-return
* Lint
* Merge branch 'master' into change-logs-to-return
2020-07-22 02:04:08 +00:00
terence tsao
d1a828ad87 Rm archival service db usages and proto definitions (#6671) 2020-07-21 17:04:46 -07:00
Preston Van Loon
f1cfb6a222 Update go-ethereum (#6660)
* Update go-ethereum
* fix go-ethereum
* Merge branch 'master' into update-go-ethereum
* Merge refs/heads/master into update-go-ethereum
* Merge refs/heads/master into update-go-ethereum
* Merge refs/heads/master into update-go-ethereum
2020-07-21 21:52:19 +00:00
Raul Jordan
3023f5dbd3 Accounts V2: Derived Keymanager Sign (#6667)
* amend derived with secret keys cache
* all tests for sign
* Merge branch 'master' into sign-derived
* formatting
* Merge branch 'sign-derived' of github.com:prysmaticlabs/prysm into sign-derived
* initialize
* use seed
* fix build
* Merge refs/heads/master into sign-derived
* Merge refs/heads/master into sign-derived
* Update validator/keymanager/v2/derived/derived.go
2020-07-21 21:15:47 +00:00
terence tsao
c41e382255 Optimize AppendValidator for beacon state (#6659) 2020-07-21 12:58:30 -07:00
Preston Van Loon
784f4169ef Check gossip message decoded length before decoding (#6663)
* Check gossip message decoded length before decoding
* gofmt
* Merge refs/heads/master into snappy-decodedlen-check
* Merge refs/heads/master into snappy-decodedlen-check
* Merge refs/heads/master into snappy-decodedlen-check
* @nisdas feedback
2020-07-21 19:14:34 +00:00
terence tsao
367738e83b Remove unused attestation operations in DB (#6664)
* Remove attestation usages in DB
* Merge refs/heads/master into rm-unused
* Merge refs/heads/master into rm-unused
* Fixed export wrapper
* Merge branch 'rm-unused' of github.com:prysmaticlabs/prysm into rm-unused
2020-07-21 18:17:33 +00:00
Victor Farazdagi
ca54c1d480 Applies assertion funcs to validator/client tests (#6654)
* Applies assertion funcs to validator/client tests
* gazelle
* Merge branch 'master' into validator-client-apply-testutils-assertions
* Merge refs/heads/master into validator-client-apply-testutils-assertions
* Merge refs/heads/master into validator-client-apply-testutils-assertions
* Merge refs/heads/master into validator-client-apply-testutils-assertions
* Merge refs/heads/master into validator-client-apply-testutils-assertions
* Merge refs/heads/master into validator-client-apply-testutils-assertions
* Merge refs/heads/master into validator-client-apply-testutils-assertions
2020-07-21 13:02:21 +00:00
Raul Jordan
e97d310453 Accounts V2: Derived Keymanager List Accounts (#6662)
* begin on list derived accounts
* begin on list derived accounts
* continued work on listing derived accounts
* deterministic pet name
* petnames
* finished list
* display deposit data
* lint
* use direct km
* tidy
* begin on revising tests for accounts list
* fix direct list test
* gaz
* tests all passing for derived list
* gaz
2020-07-21 05:06:11 +00:00
Raul Jordan
bc16fa9f50 Accounts V2: Derived Keymanager, Wallet & Account Creation (#6624)
* initialize derived wallet
* derived wallet + account creation
* initialize wallet seed
* encrypted seed file creation
* generate next acct
* create seed from pass
* properly creating derived accounts
* fix up formatting
* prep for review
* start tests for derived create account
* add derived test
* linter
* gaz
* derived keymanager create account test complete
* Merge branch 'master' into derived-keymanager
* tests pass
* gaz
* fix list test
* Merge refs/heads/master into derived-keymanager
* ivan feedback
* skip mnemonic confirm
* Merge branch 'derived-keymanager' of github.com:prysmaticlabs/prysm into derived-keymanager
* comment
* tidy
* fmt
* organize
* test interface conformity
* Update validator/accounts/v2/iface/wallet.go
* ivan comments
* Merge branch 'derived-keymanager' of github.com:prysmaticlabs/prysm into derived-keymanager
* Merge refs/heads/master into derived-keymanager
* Merge branch 'master' of github.com:prysmaticlabs/prysm into derived-keymanager
* Fix
* Fix test
* Merge refs/heads/master into derived-keymanager
* fix errs
* imports
* Gaz
2020-07-21 02:05:23 +00:00
Victor Farazdagi
9deeb592e6 Adds type info into equality assertions (#6656)
* Adds type info into equality assertions
* tests
* reorder got/want
* more tests
* Merge refs/heads/master into assertions-better-error-reporting
* Merge refs/heads/master into assertions-better-error-reporting
* Merge refs/heads/master into assertions-better-error-reporting
2020-07-21 00:47:53 +00:00
Ivan Martinez
93bbb86a30 Accounts-V2: Add tests for exporting and importing (#6615)
* Add test for exporting and importing
* Add test for non-interactive
* Test import cli
* Merge branch 'master' of github.com:prysmaticlabs/prysm into test-export
* Merge branch 'master' into test-export
* Cleanup tests a bit
* Change to assert
* Fix tests
* Merge branch 'master' of github.com:prysmaticlabs/prysm into test-export
* Fix comment, change constants
* Fixes
* Change to require
* Fix broken test
* Merge refs/heads/master into test-export
* Merge refs/heads/master into test-export
* Merge refs/heads/master into test-export
* Merge refs/heads/master into test-export
* Merge refs/heads/master into test-export
* Merge refs/heads/master into test-export
* Merge refs/heads/master into test-export
* Merge refs/heads/master into test-export
* Merge refs/heads/master into test-export
* Merge refs/heads/master into test-export
* Merge refs/heads/master into test-export
* Merge refs/heads/master into test-export
* Merge refs/heads/master into test-export
* Merge refs/heads/master into test-export
* Merge refs/heads/master into test-export
* Merge refs/heads/master into test-export
* Merge refs/heads/master into test-export
2020-07-20 21:12:46 +00:00
Ivan Martinez
d54663a553 Add --historical-slasher-node flag to simplify slasher usage (#6627)
* Add historical-slasher-node flag
* Fix build
* Remove useless flag
* Merge refs/heads/master into add-flasher-flag
* Merge refs/heads/master into add-flasher-flag
* Merge refs/heads/master into add-flasher-flag
* Fix flag
* Merge branch 'add-flasher-flag' of github.com:prysmaticlabs/prysm into add-flasher-flag
* Add flags to beacon chain
* Merge refs/heads/master into add-flasher-flag
* Merge refs/heads/master into add-flasher-flag
* Merge refs/heads/master into add-flasher-flag
* Merge refs/heads/master into add-flasher-flag
* Merge refs/heads/master into add-flasher-flag
* Merge refs/heads/master into add-flasher-flag
* Merge refs/heads/master into add-flasher-flag
* Merge refs/heads/master into add-flasher-flag
* Merge refs/heads/master into add-flasher-flag
* Merge refs/heads/master into add-flasher-flag
* Merge refs/heads/master into add-flasher-flag
* Merge refs/heads/master into add-flasher-flag
* Merge refs/heads/master into add-flasher-flag
2020-07-20 20:23:48 +00:00
terence tsao
0488c9869e Init sync batch block clean up (#6599) 2020-07-20 12:35:30 -07:00
Victor Farazdagi
f0ffd5af03 Introduces peer scorer service (#6579)
* introduces peer scorer service
* gazelle
* Merge branch 'master' into peer-scoring-service
* fixes comment
* fix build error
* linter suggestions
* Merge branch 'master' into peer-scoring-service
* updates tests
* Merge branch 'master' into peer-scoring-service
* badresponses scorer tests
* gazelle
* Merge branch 'master' into peer-scoring-service
* Merge branch 'master' into peer-scoring-service
* adds scorer_test
* gazelle
* updates bad response default penalty
* more comments
* Merge branch 'master' into peer-scoring-service
* Merge branch 'master' into peer-scoring-service
* Merge branch 'master' into peer-scoring-service
* Merge refs/heads/master into peer-scoring-service
* Merge refs/heads/master into peer-scoring-service
* Merge refs/heads/master into peer-scoring-service
* Merge refs/heads/master into peer-scoring-service
* Merge refs/heads/master into peer-scoring-service
* Merge refs/heads/master into peer-scoring-service
* Merge refs/heads/master into peer-scoring-service
* Merge refs/heads/master into peer-scoring-service
* introduces peerdatastore into status
* makes sure that commong peer data store is used
* Merge branch 'peer-scoring-service' of github.com:prysmaticlabs/prysm into peer-scoring-service
* linter
* gazelle
* updates tests
* Merge branch 'master' into peer-scoring-service
* fixes tests
* Nishant's suggestions
* Merge refs/heads/master into peer-scoring-service
* Merge refs/heads/master into peer-scoring-service
* Merge refs/heads/master into peer-scoring-service
* Merge refs/heads/master into peer-scoring-service
* Merge refs/heads/master into peer-scoring-service
* Merge refs/heads/master into peer-scoring-service
* Merge refs/heads/master into peer-scoring-service
* Merge branch 'master' into peer-scoring-service
* gofmt
* Nishant's suggestion to use isBadPeer
* Merge refs/heads/master into peer-scoring-service
2020-07-20 15:12:59 +00:00
Victor Farazdagi
22141db319 Applies assertion funcs to validator/accounts tests (#6653)
* Applies assertion funcs to validator/accounts tests
* Applies assertion funcs to validator/accounts tests
2020-07-20 10:56:45 +00:00
Victor Farazdagi
10d3275638 Applies assertion funcs to rpc tests (#6641)
* applies assertion funcs to beacon-chain/rpc tests
* Merge branch 'master' into rpc-apply-testutils-assertions
* applies assertion funcs to beacon-chain/rpc/beacon tests
* Merge branch 'master' into rpc-apply-testutils-assertions
* applies assertion funcs to beacon-chain/rpc/debug tests
* applies assertion funcs to beacon-chain/rpc/validator tests
* Merge branch 'master' into rpc-apply-testutils-assertions
* Merge refs/heads/master into rpc-apply-testutils-assertions
2020-07-20 02:15:51 +00:00
Victor Farazdagi
1d0fe105fc Extends assertions to accept fmt params (#6645)
* Extends assertions to accept fmt params
2020-07-20 01:36:28 +00:00
terence tsao
0544dd1f8e Applies assertion funcs to shared tests (#6643)
* cmd tests
* deposit util tests
* feature config tests
* hashutil tests
* htr util tests
* interop tests
* ip util tests
2020-07-19 21:08:29 +00:00
Preston Van Loon
b60a965981 ProcessAttestations: Add requirement that inclusion distance is not zero (#6642)
* Add requirement that inclusion distance is not zero, add regression test
* fix test input data
* Merge refs/heads/master into fix-panic
* Merge refs/heads/master into fix-panic
2020-07-19 16:38:45 +00:00
terence tsao
38c8393e52 Revert "Fixes issue with vals order in assert.Equal" (#6640)
* Revert "Fixes issue with vals order in assert.Equal (#6631)"

This reverts commit d535cb95d8.
* Merge branch 'master' into revert-6631-assertions-less-assuming-error-msg
2020-07-19 04:45:04 +00:00
Victor Farazdagi
568238009e Extracts peer data store in p2p/peers status (#6639)
* introduces peer data store to p2p/peers
* Merge refs/heads/master into p2p-extract-data-store
* Merge refs/heads/master into p2p-extract-data-store
* refactores status service
* Merge branch 'p2p-extract-data-store' of github.com:prysmaticlabs/prysm into p2p-extract-data-store
* updates tests
* gofmt
* rever decay test
2020-07-19 03:45:08 +00:00
Preston Van Loon
c419e4ed8f Improved cold state checkpoints: migrate database index and usage (#6461)
* Add database migrations, still need to update the API usage...
* gofmt goimports
* progress
* Merge branch 'master' of github.com:prysmaticlabs/prysm into index-migration
* use slot instead of index
* rename LastArchivedIndex to LastArchivedSlot
* rename LastArchivedIndexRoot to LastArchivedRoot
* remove unused HighestSlotStates method
* deprecate old key, include in migration
* deprecate old key, include in migration
* remove blocks index in migration
* rename bucket variable
* fix code to pass tests
* Merge branch 'master' of github.com:prysmaticlabs/prysm into index-migration
* gofmt, goimports
* fix
* Add state slot index
* progress
* lint
* fix build
* Merge branch 'master' of github.com:prysmaticlabs/prysm into index-migration
* kafka
* Merge refs/heads/master into index-migration
* Merge refs/heads/master into index-migration
* Merge refs/heads/master into index-migration
* remove SaveArchivedPointRoot, a few other big changes
* Merge branch 'index-migration' of github.com:prysmaticlabs/prysm into index-migration
* fix tests and lint
* lint again
* Merge refs/heads/master into index-migration
* Merge refs/heads/master into index-migration
* Merge refs/heads/master into index-migration
* block migration, some renaming
* gaz, gofmt
* add tests
* change index to uint bytes
* Merge branch 'index-migration' of github.com:prysmaticlabs/prysm into index-migration
* rm method notes
* stop if the bucket doesn't exist
* Merge refs/heads/master into index-migration
* Merge refs/heads/master into index-migration
* Merge refs/heads/master into index-migration
* Merge refs/heads/master into index-migration
* Merge refs/heads/master into index-migration
* Merge refs/heads/master into index-migration
* Merge refs/heads/master into index-migration
* Merge refs/heads/master into index-migration
* @rauljordan pr feedback
* Simplify
* Merge refs/heads/master into index-migration
* Remove unused method, add roundtrip test
* gofmt
* Merge refs/heads/master into index-migration
* Merge refs/heads/master into index-migration
* Merge refs/heads/master into index-migration
* Merge refs/heads/master into index-migration
* Merge refs/heads/master into index-migration
* Merge refs/heads/master into index-migration
* Merge refs/heads/master into index-migration
* Merge branch 'master' of github.com:prysmaticlabs/prysm into index-migration
2020-07-18 18:05:04 +00:00
terence tsao
d53539499c Applies assertion funcs to shared tests (part 1) (#6626)
* Update kv aggregated_test.go
* Update block_test.go
* Update forkchoice_test.go
* Update unaggregated_test.go
* Update prepare_forkchoice_test.go
* Update prune_expired_test.go
* Update atts service_test.go
* Update service_attester_test.go
* Update service_proposer_test.go
* Upate exit service_test.go
* Gaz
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Move averageBalance from log.go to info.go
* Move avg balance from log.go to info.go
* Add info test
* Remove unused logEpochData in log.go
* Gaz
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* aggregation tests
* attestation util tests
* bench util tests
* block util tests
* herumi tests
* bytesutil tests
* Merge refs/heads/master into testutil-shared
* Merge refs/heads/master into testutil-shared
* Merge refs/heads/master into testutil-shared
* Merge refs/heads/master into testutil-shared
* Merge refs/heads/master into testutil-shared
* Update shared/aggregation/attestations/attestations_test.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
* Merge branch 'master' of github.com:prysmaticlabs/prysm into testutil-shared
* Fixed ordering
* Merge branch 'testutil-shared' of github.com:prysmaticlabs/prysm into testutil-shared
* Update shared/aggregation/attestations/attestations_test.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
* Update shared/bytesutil/bytes_test.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-07-18 16:31:42 +00:00
Victor Farazdagi
d535cb95d8 Fixes issue with vals order in assert.Equal (#6631)
* fixes issue with vals order in assert.Equal
2020-07-18 11:14:50 +00:00
Victor Farazdagi
7ba500c417 Applies assertion funcs to state tests (#6623)
* applies assertion funcs to beacon-chain/state tests
* applies assertion funcs to beacon-chain/state/stategen tests
* applies assertion funcs to beacon-chain/state/stateutil tests
* gazelle
* Merge refs/heads/master into state-apply-testutils-assertions
* Merge refs/heads/master into state-apply-testutils-assertions
* Merge refs/heads/master into state-apply-testutils-assertions
2020-07-18 07:56:48 +00:00
Preston Van Loon
c573306621 Check for array out of bounds when calculating proposer delta -- follow up (#6630)
* failing test
* fix
2020-07-18 06:39:43 +00:00
Preston Van Loon
5336a167af Check for array out of bounds when calculating proposer delta (#6629)
* Check for array out of bounds when calculating proposer delta
* Merge refs/heads/master into proposer-delta-bug
2020-07-18 04:31:03 +00:00
Preston Van Loon
0e4cb68249 Add regression test found by fuzzer (#6628)
* Add regression test found by fuzzer
2020-07-18 03:47:03 +00:00
Nishant Das
cc3c3a0c54 QSP 29: Add Rate Limiter For All Topics (#6606)
* initial commit
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into rateLimiter
* finally
* add mutex
* rate limiter for all rpc reqs
* remove recursive readlocks
* spelling
* Merge refs/heads/master into rateLimiter
* Merge refs/heads/master into rateLimiter
* Merge refs/heads/master into rateLimiter
* fix all tests
* fix all tests
* Merge refs/heads/master into rateLimiter
* Merge refs/heads/master into rateLimiter
* Merge refs/heads/master into rateLimiter
* Merge refs/heads/master into rateLimiter
* Merge refs/heads/master into rateLimiter
* Merge refs/heads/master into rateLimiter
* Merge refs/heads/master into rateLimiter
* Merge refs/heads/master into rateLimiter
* Merge refs/heads/master into rateLimiter
* Merge refs/heads/master into rateLimiter
2020-07-17 08:58:51 +00:00
Raul Jordan
9ad6277852 Accounts V2: Create Wallet Non-Interactively + Add Test Coverage (#6611)
* create wallet noninteractively
* tests for wallet create
* kapol feedback
* Merge refs/heads/master into create-wallet-tests
* Merge refs/heads/master into create-wallet-tests
2020-07-17 08:21:16 +00:00
terence tsao
ebd05fba01 Improve test coverage info.go (#6618)
* Update kv aggregated_test.go
* Update block_test.go
* Update forkchoice_test.go
* Update unaggregated_test.go
* Update prepare_forkchoice_test.go
* Update prune_expired_test.go
* Update atts service_test.go
* Update service_attester_test.go
* Update service_proposer_test.go
* Upate exit service_test.go
* Gaz
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Move averageBalance from log.go to info.go
* Move avg balance from log.go to info.go
* Add info test
* Remove unused logEpochData in log.go
* Gaz
* Merge branch 'master' into info-cov
* gaz
* Merge refs/heads/master into info-cov
2020-07-16 22:44:43 +00:00
rkapka
eb0d70814a Fixed nil pointer error in DepositCache (#6596) 2020-07-16 14:48:36 -07:00
rkapka
77607c6fdb Applies assertion funcs to cache tests (#6617)
* testutils for cache
* Merge branch 'master' into cache-test-refactor
* removed some empty lines
* Merge branch 'origin-master' into cache-test-refactor
* Merge remote-tracking branch 'rkapka/cache-test-refactor' into cache-test-refactor
* revert package names
2020-07-16 19:34:08 +00:00
Nishant Das
cf2bbec6a8 Use Faster Hash Function (#6616)
* change to proto hash

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-07-16 10:19:14 -05:00
terence tsao
69a2dc2716 Improve test coverage receive_block.go (#6613)
* Update kv aggregated_test.go
* Update block_test.go
* Update forkchoice_test.go
* Update unaggregated_test.go
* Update prepare_forkchoice_test.go
* Update prune_expired_test.go
* Update atts service_test.go
* Update service_attester_test.go
* Update service_proposer_test.go
* Upate exit service_test.go
* Gaz
* TestService_ReceiveBlockInitialSync
* TestService_ReceiveBlockBatch
* TestService_HasInitSyncBlock
* Merge branch 'master' of github.com:prysmaticlabs/prysm into rcv-blk-cov
* Merge branch 'master' of github.com:prysmaticlabs/prysm into rcv-blk-cov
* Merge branch 'rcv-blk-cov' of github.com:prysmaticlabs/prysm into rcv-blk-cov
* Gaz
* Merge refs/heads/master into rcv-blk-cov
* Merge refs/heads/master into rcv-blk-cov
* Merge refs/heads/master into rcv-blk-cov
* Merge refs/heads/master into rcv-blk-cov
2020-07-16 13:34:34 +00:00
Raul Jordan
d897640625 Add YAML Struct Tag to Eth1FollowDistance Parameter (#6612)
* add yaml struct tag to eth1 follow distance
* more config values missing yaml tags
* Merge refs/heads/master into follow-distance-struct-tag
* Merge refs/heads/master into follow-distance-struct-tag
* Merge refs/heads/master into follow-distance-struct-tag
2020-07-16 13:04:59 +00:00
Victor Farazdagi
deb025f57c Applies assertion funcs to blockchain tests (#6605)
* applies assertion funcs to blockchain tests
* Merge branch 'master' into blockchain-apply-testutils-assertions
* gofmt
* Merge refs/heads/master into blockchain-apply-testutils-assertions
* Merge refs/heads/master into blockchain-apply-testutils-assertions
* Merge refs/heads/master into blockchain-apply-testutils-assertions
* Merge refs/heads/master into blockchain-apply-testutils-assertions
* Merge refs/heads/master into blockchain-apply-testutils-assertions
2020-07-16 12:11:39 +00:00
Nishant Das
f6756bb591 QSP 23: Prune Peers From Peer Handler (#6614)
* checkpoint
* Merge refs/heads/master into prunePeers
2020-07-16 10:09:51 +00:00
Raul Jordan
df73851749 Accounts Revamp: Wallet Edit-Config (#6607)
* edit remote config
* gets messed up when writing to file again
* proper editing
* noninteractive mode
* test for edit wallet
* Merge branch 'master' into edit-wallet
* wallet edit test done
* imports
* Merge refs/heads/master into edit-wallet
* Merge refs/heads/master into edit-wallet
2020-07-16 05:08:16 +00:00
Ivan Martinez
fbeba94a92 Change accounts-v2 import and export to be non-interactive (#6609)
* Change import and export to be non-interactive
* Fix
* fix lint
* Merge branch 'master' into make-export-import-nonint
* Comments
* Merge branch 'make-export-import-nonint' of github.com:prysmaticlabs/prysm into make-export-import-nonint
* Merge refs/heads/master into make-export-import-nonint
2020-07-15 23:00:00 +00:00
Victor Farazdagi
a0e5754464 Resolves possible import cycle in testutils assertions (#6610)
* resolves possible import cycle in testutils assertions
* gazelle
* linter
* linter
2020-07-15 20:10:54 +00:00
terence tsao
c309ba6a10 Applies assertion funcs to operation tests (#6608)
* Update kv aggregated_test.go
* Update block_test.go
* Update forkchoice_test.go
* Update unaggregated_test.go
* Update prepare_forkchoice_test.go
* Update prune_expired_test.go
* Update atts service_test.go
* Update service_attester_test.go
* Update service_proposer_test.go
* Upate exit service_test.go
* Gaz
2020-07-15 18:37:51 +00:00
Victor Farazdagi
c2615168d9 Applies assertion funcs to sync tests (#6603)
* applies assertion funcs to sync/initial-sync tests
* applies assertion funcs to sync/initial-sync tests
* gazelle
* Merge branch 'master' into sync-apply-testutils-assertions
* gazelle
* applies assertion funcs to sync/initial-sync tests
* applies assertion funcs to sync/initial-sync tests
* applies assertion funcs to sync/initial-sync tests
* applies assertion funcs to sync/initial-sync tests
* applies assertion funcs to sync/initial-sync tests
* Merge branch 'master' into sync-apply-testutils-assertions
2020-07-15 04:41:11 +00:00
Raul Jordan
f12f75224e Accounts V2: Create Wallet via CLI (#6543)
* begin on the proto definitions

* define remote signer service protos

* basic implementation

* remote keymanager docs

* finalize remote client doc

* amend response

* fix proto defs

* test new and begin test sign

* test sign done

* remote oneof

* rename

* fix build

* viz

* Update validator/keymanager/v2/remote/remote_test.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>

* Update validator/accounts/v2/wallet.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>

* fmt

* move the input check to accounts new

* begin on requesting cli input for remote keymanager config

* move the input check to accounts new

* begin on requesting cli input for remote keymanager config

* define wallet v2

* create wallet most logic done

* init remote wallet

* create wallet moving nicely

* ensure wallet create works

* reenable import export list

* further refactor

* improve handling of input wallet dir

* add all validation to cert path checks

* lint

* list test

* new wallet lint

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
2020-07-14 23:05:21 -05:00
terence tsao
fe14c5086a Fix duplicated selectV1Keymanager (#6604)
* Fix duplicated selectV1Keymanager
2020-07-15 01:05:33 +00:00
Ivan Martinez
d7bcea7906 Allow creating new accounts to be non-interactive (#6602)
* Allow accounts to be made non-interactively
* Merge branch 'master' of github.com:prysmaticlabs/prysm into make-accounts-noninteractive
* Update validator/node/node.go
* Update validator/accounts/v2/list.go
2020-07-14 23:00:58 +00:00
terence tsao
eef9a760ec Fix inconsistent pubkey formatting in logs (#6600)
* Use Trunc for pub key
* Gaz
* Merge refs/heads/master into fix-pub-key
2020-07-14 20:37:01 +00:00
Ivan Martinez
5278b75c02 E2E Improvements (#6587)
* Change long-running e2e to run minimally
* Merge branch 'master' into e2e-config-change
* Remove deposits flag
* Merge branch 'e2e-config-change' of github.com:prysmaticlabs/prysm into e2e-config-change
* Merge branch 'master' of github.com:prysmaticlabs/prysm into e2e-config-change
* Reduce secondsperslot
* Add todo
* Merge branch 'master' of github.com:prysmaticlabs/prysm into e2e-config-change
* Merge refs/heads/master into e2e-config-change
* Merge refs/heads/master into e2e-config-change
* Merge refs/heads/master into e2e-config-change
* Merge refs/heads/master into e2e-config-change
* Merge refs/heads/master into e2e-config-change
* Merge refs/heads/master into e2e-config-change
2020-07-14 18:08:17 +00:00
Victor Farazdagi
d9fd2521af Applies assertion funcs to p2p tests (#6597)
* applies assertion funcs to p2p/encoder tests
* applies assertion funcs to p2p/peers tests
* addr_factory_test + broadcaster_test updated
* connection_gater_test updated
* applies assertion funcs to p2p/service tests
* Merge branch 'master' into p2p-apply-testutils-assertions
* minor fixes
* Merge branch 'master' into p2p-apply-testutils-assertions
* Merge refs/heads/master into p2p-apply-testutils-assertions
2020-07-14 16:51:39 +00:00
terence tsao
ea32af7bf7 Update justified point for batch sync (#6594)
* Update justified points
* Add the same check to on block init sync
* Add comments
* Add test
* Gaz
* Merge branch 'master' into fix-batch-sync
* Merge refs/heads/master into fix-batch-sync
2020-07-14 16:20:25 +00:00
terence tsao
8fda48409c Remove batch blocks save at getAttPreState (#6584)
* Check if block exists in cache
* Consider init sync blocks
* Fixed test
* Merge refs/heads/master into get-block
* Merge refs/heads/master into get-block
* Merge refs/heads/master into get-block
* Merge refs/heads/master into get-block
* Merge refs/heads/master into get-block
* Merge refs/heads/master into get-block
* Merge refs/heads/master into get-block
* Merge refs/heads/master into get-block
* Merge refs/heads/master into get-block
* Merge refs/heads/master into get-block
2020-07-14 15:46:00 +00:00
Shay Zluf
e0c803abfc Fast list validators (#6580)
* Fast list validators
* fix tests
* Merge branch 'master' of github.com:prysmaticlabs/prysm into fast_list_validators
* add small test
* Merge branch 'master' into fast_list_validators
* remove unneeded fatcher
* Merge branch 'fast_list_validators' of github.com:prysmaticlabs/prysm into fast_list_validators
* nishant feedback
* package name
2020-07-14 12:21:55 +00:00
Preston Van Loon
b099cab9b1 automaxprocs: Log error to DEBUG instead of panic (#6592)
* Log error to DEBUG instead of panic. Fixes #6591
* Merge branch 'master' into fix-6591
2020-07-14 01:51:07 +00:00
Ivan Martinez
52e9155df3 Change validator accounts-v2 to validator wallet-v2 accounts (#6589)
* Change `validator accounts-v2` to `validator wallet-v2 accounts-v2

* Change to accounts

* Fix cmd

* Rename cmd.go to cmd_accounts.go
2020-07-13 19:58:06 -05:00
Preston Van Loon
774b4b7eef SubmitAggregateAndProof now prefers its own validator attestations (#6566)
* SubmitAggregateAndProof now prefers its own validator attestations
* only do aggregate bits count comparsion when best also contains the validator index
* gofmt
* better test
* Merge refs/heads/master into prefer-own-attestation
* Merge refs/heads/master into prefer-own-attestation
* Merge refs/heads/master into prefer-own-attestation
* Merge refs/heads/master into prefer-own-attestation
* Merge refs/heads/master into prefer-own-attestation
* Merge refs/heads/master into prefer-own-attestation
* Merge refs/heads/master into prefer-own-attestation
* @terencechain feedback
* Merge refs/heads/master into prefer-own-attestation
* update comment
* Merge branch 'prefer-own-attestation' of github.com:prysmaticlabs/prysm into prefer-own-attestation
* Merge refs/heads/master into prefer-own-attestation
* Merge refs/heads/master into prefer-own-attestation
* Merge refs/heads/master into prefer-own-attestation
* Merge refs/heads/master into prefer-own-attestation
* Merge refs/heads/master into prefer-own-attestation
* Merge refs/heads/master into prefer-own-attestation
2020-07-13 22:02:01 +00:00
Ivan Martinez
cd2ea868ff Direct Keymanager: Add Export and Import commands (#6528)
* add in configs
* ask for enable accounts v2
* begin integration of v2 keymanager
* refactor wallet opening
* include significant refactoring of how opening a wallet works, making it easy to include at runtime
* ensure build with keymanager v2
* further improving runtime integration
* default pass paths
* finally running v2 at runtime
* import spacing
* Merge branch 'master' into v2-accounts-feature
* Merge refs/heads/master into v2-accounts-feature
* Merge refs/heads/master into v2-accounts-feature
* Merge refs/heads/master into v2-accounts-feature
* Merge refs/heads/master into v2-accounts-feature
* Get started on export and import commands
* Work more on import
* Complete functionality
* Merge branch 'master' of github.com:prysmaticlabs/prysm into accounts-import
* Undo
* Merge branch 'master' of github.com:prysmaticlabs/prysm into accounts-import
* cleanup
* Extract code to functions
* Merge branch 'master' of github.com:prysmaticlabs/prysm into accounts-import
* Add comments
* Fix ocmments
* Improvements
* fix
* Merge branch 'master' of github.com:prysmaticlabs/prysm into accounts-import
* Remove GetSigningKeyForAccount
* Progress
* Fix build
* Fix name
* Merge branch 'master' into accounts-import
* Add logging
* Merge branch 'accounts-import' of github.com:prysmaticlabs/prysm into accounts-import
* Merge refs/heads/master into accounts-import
* Merge refs/heads/master into accounts-import
* Merge refs/heads/master into accounts-import
* Merge refs/heads/master into accounts-import
* Merge refs/heads/master into accounts-import
* Merge refs/heads/master into accounts-import
* Merge refs/heads/master into accounts-import
* Merge refs/heads/master into accounts-import
* Merge refs/heads/master into accounts-import
* Merge refs/heads/master into accounts-import
* Merge refs/heads/master into accounts-import
* Merge refs/heads/master into accounts-import
* Merge refs/heads/master into accounts-import
2020-07-13 21:37:18 +00:00
terence tsao
2bbae15194 Handle roughtime response error (#6586)
* Handle roughtime response error
* Update shared/roughtime/roughtime.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Merge branch 'master' into fix-roughtime-logs
* Update shared/roughtime/roughtime.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/master into fix-roughtime-logs
* Merge refs/heads/master into fix-roughtime-logs
* Merge refs/heads/master into fix-roughtime-logs
2020-07-13 21:00:43 +00:00
terence tsao
18f28e2840 VerifyBlkDescendant - fixed a nil return bug and added a test (#6553)
* Refactor verifyBlkDescendant to be public and omit the slot argument, which was only used for error messages
* impl in mock
* gofmt
* Merge refs/heads/master into refactor-verifyBlkDescendant-signature
* Better errors
* Tests. Yay!
* Comments
* Merge refs/heads/master into test-blk-des
* Merge refs/heads/master into test-blk-des
* Fixed receive blck test
* Merge branch 'test-blk-des' of github.com:prysmaticlabs/prysm into test-blk-des
* Merge refs/heads/master into test-blk-des
* Merge refs/heads/master into test-blk-des
* Merge refs/heads/master into test-blk-des
* Merge refs/heads/master into test-blk-des
* Merge refs/heads/master into test-blk-des
* Ensure finalized root not zeros
* Merge branch 'test-blk-des' of github.com:prysmaticlabs/prysm into test-blk-des
* Merge refs/heads/master into test-blk-des
* Merge refs/heads/master into test-blk-des
* Merge refs/heads/master into test-blk-des
* Merge refs/heads/master into test-blk-des
* Merge refs/heads/master into test-blk-des
* Merge refs/heads/master into test-blk-des
* Merge refs/heads/master into test-blk-des
* Merge refs/heads/master into test-blk-des
* Merge refs/heads/master into test-blk-des
* Ensure non zero hashes when compare
* Merge branch 'test-blk-des' of github.com:prysmaticlabs/prysm into test-blk-des
* Merge refs/heads/master into test-blk-des
* Merge refs/heads/master into test-blk-des
2020-07-13 20:14:36 +00:00
Nishant Das
27577bc324 QSP 32: Add Appropriate Stream Deadlines for RPC Requests (#6583)
* add no deadlines
* Merge branch 'master' into fixDeadlines
* nogo
* Merge refs/heads/master into fixDeadlines
* Merge refs/heads/master into fixDeadlines
2020-07-13 19:40:12 +00:00
Nishant Das
79fbaaea0b QSP18: Add Varint Header Validation (#6577)
* add header validation

* Update beacon-chain/p2p/encoder/varint_test.go

* Update beacon-chain/p2p/encoder/varint_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-07-13 14:02:03 -05:00
Victor Farazdagi
d1e754f011 Fixes display issue in testutils/assert (#6585)
* fixes minor display issue in testutils/assert
2020-07-13 17:20:18 +00:00
Victor Farazdagi
77d1a6c698 Adds testutils/assert and testutils/require (#6563)
* testutils/assert

* adds assertion tb mock

* testutil/require

* gazelle

* comment public types

* set TestOnly attribute

* fixes package name

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-07-13 10:19:52 -05:00
Nishant Das
f2b4f91419 save blocks (#6578) 2020-07-13 06:33:45 -07:00
Justin
ec800bac7c Remove redundant check in exit.go (#6559)
* Remove redundant check in exit.go

The check `exit.Exit.ValidatorIndex >= uint64(beaconState.NumValidators())` is done in `beaconState.ValidatorAtIndexReadOnly(exit.Exit.ValidatorIndex)`.
* Merge branch 'master' into patch-1
* Merge branch 'master' into patch-1
* Merge branch 'master' into patch-1
* Merge branch 'master' into patch-1
* Merge branch 'master' into patch-1
2020-07-13 09:36:40 +00:00
Nishant Das
63cb99b359 add logs (#6575) 2020-07-12 22:36:03 -07:00
Nishant Das
b5bd1260d0 Clean Up P2P Service (#6574)
* clean up
* clean up
* gaz
* preston's review
* preston's review
* Merge refs/heads/master into CleanUpP2P
2020-07-13 04:16:24 +00:00
terence tsao
62df4995e6 Clean up block chain pkg initial sync (#6562) 2020-07-12 20:44:06 -07:00
Victor Farazdagi
c35bdf2649 Refactors calls to deprecated libp2p methods (#6554)
* refactors calls to deprecated IDB58Decode()
* Merge branch 'master' into p2p-refactor-deprecated
* updated packages
* gazelle
* mod tidy
* refactors publish()/subscribe() deprecated methods
* gofmt
* test update join/leave topic methods
* re-arrange imports
* Merge branch 'master' into p2p-refactor-deprecated
* Merge refs/heads/master into p2p-refactor-deprecated
2020-07-13 02:28:40 +00:00
Nishant Das
d4c3546434 Validate RPC Topics Before Sending Requests (#6558)
* clean up

* fix panic

* add test mapping

* add schema version change

* fix test

* fix another test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-07-13 09:20:53 +08:00
Ivan Martinez
10c0d5b569 Remove unused functions in validator DB (#6557)
* Remove unused functions in validator DB
* Iface
* Merge branch 'master' into reomve-unused-val-db
* Merge branch 'master' into reomve-unused-val-db
* Merge refs/heads/master into reomve-unused-val-db
2020-07-11 17:23:30 +00:00
Ivan Martinez
18c00ab25d Add BoltTimeout to IO config (#6555)
* Make bolt timeout a config value
* Merge refs/heads/master into best-practices0timeout
* Merge refs/heads/master into best-practices0timeout
* Merge refs/heads/master into best-practices0timeout
2020-07-11 16:43:26 +00:00
Nishant Das
b82defddb2 add timeouts (#6560) 2020-07-11 08:18:25 -07:00
Preston Van Loon
fa85d93a19 Verify roughtime results before accepting time offset (#6556)
* Verify roughtime results before accepting time offset
2020-07-11 04:42:26 +00:00
Preston Van Loon
1f35384578 Reject a block which its parent is not part of the finalized chain (#6549)
* Reject a block which its parent is not part of the finalized chain
* Refactor verifyBlkDescendant to be public and omit the slot argument, which was only used for error messages
* Merge branch 'refactor-verifyBlkDescendant-signature' into reject-bad-block
* fix
* impl in mock
* impl in mock
* gofmt
* Merge refs/heads/master into refactor-verifyBlkDescendant-signature
* Merge branch 'refactor-verifyBlkDescendant-signature' into reject-bad-block
* fix test
* add test
* Merge branch 'refactor-verifyBlkDescendant-signature' into reject-bad-block
* gofmt
* Merge branch 'master' of github.com:prysmaticlabs/prysm into reject-bad-block
* move up in the validation pipeline, @terencechain offline feedback
* Merge refs/heads/master into reject-bad-block
2020-07-11 03:57:42 +00:00
Preston Van Loon
c9ca5857f8 Avoid automaxprocs default logger to printf (#6524)
* Avoid automaxprocs default logger to printf
* Merge branch 'master' into maxprocs
* gofmt, goimports
* Merge refs/heads/master into maxprocs
* Merge refs/heads/master into maxprocs
* gazelle for docker images
* Merge refs/heads/master into maxprocs
* Merge refs/heads/master into maxprocs
* Merge refs/heads/master into maxprocs
* Merge refs/heads/master into maxprocs
* Merge refs/heads/master into maxprocs
* Merge refs/heads/master into maxprocs
* Merge refs/heads/master into maxprocs
* Merge refs/heads/master into maxprocs
* Merge refs/heads/master into maxprocs
* Merge refs/heads/master into maxprocs
* Merge refs/heads/master into maxprocs
* Merge refs/heads/master into maxprocs
* Merge refs/heads/master into maxprocs
* fix docker
* Merge refs/heads/master into maxprocs
* Merge refs/heads/master into maxprocs
2020-07-11 00:57:43 +00:00
Preston Van Loon
6c7965e82a Refactor verifyBlkDescendant to be public in chainservice API (#6552)
* Refactor verifyBlkDescendant to be public and omit the slot argument, which was only used for error messages
* impl in mock
* gofmt
* Merge refs/heads/master into refactor-verifyBlkDescendant-signature
* fix test
2020-07-11 00:16:52 +00:00
Preston Van Loon
29317ca8da Attestation cache: check bitlist length before checking contains (#6551)
* Attestation cache: check bitlist length before checking contains
2020-07-10 21:05:03 +00:00
Preston Van Loon
dbc9686d15 Add roughtime offset histogram metric, log on large offsets (#6546)
* Add roughtime_offset_nsec metric. Log a warning if offset is greater than 2 seconds
* gofmt
* use math.abs, add help text
* gofmt
2020-07-10 17:07:49 +00:00
Nishant Das
cc8b3e349d Save Boundary States (#6542) 2020-07-10 09:00:41 -07:00
Raul Jordan
2c9474ab7f Remote Keymanager: Proto Definitions and Basic Client Implementation (#6526)
* begin on the proto definitions
* define remote signer service protos
* basic implementation
* remote keymanager docs
* finalize remote client doc
* amend response
* fix proto defs
* test new and begin test sign
* test sign done
* Merge branch 'master' into remote-keymanager-v2
* remote oneof
* rename
* Merge branch 'remote-keymanager-v2' of github.com:prysmaticlabs/prysm into remote-keymanager-v2
* fix build
* Merge refs/heads/master into remote-keymanager-v2
* viz
* Merge branch 'remote-keymanager-v2' of github.com:prysmaticlabs/prysm into remote-keymanager-v2
* Merge refs/heads/master into remote-keymanager-v2
* Merge refs/heads/master into remote-keymanager-v2
* Update validator/keymanager/v2/remote/remote_test.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update validator/accounts/v2/wallet.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
* fmt
2020-07-10 05:49:56 +00:00
terence tsao
ac79819077 Fix genesis validator count (#6540)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-07-09 19:54:24 -05:00
terence tsao
d54cefbe42 Part 1 of blockchain pkg clean up (#6537)
* Fix span names

* Clean up on block func

* Add updateFinalized helper

* Deprecate ReceiveBlockNoPubsub for ReceiveBlock

* Proposer to broadcast block

* Update migrate interface

* Replace ReceiveBlock for all

* Go fmt
2020-07-09 18:50:48 -05:00
rkapka
f7088e037c Finalized deposits cache (#6391)
* Renamed beforeBlk to untilBlk
* finalized deposits cache logic
* Merge branch 'master' into deposit-trie-cache

# Conflicts:
#	beacon-chain/blockchain/process_block.go
* simplified returning of finalized deposits
* added comment to FinalizedDeposits struct
* fixes after code review
* Merge branch 'master' into deposit-trie-cache
* fixed variable declaration
* renamed deposit cache test file
* fixed type casting
* compilation error fix
* deposit cache tests
* proposer test
* Merge branch 'master' into deposit-trie-cache
* added missing methods to interop beacon-chain service
* gofmt
* Merge branch 'master' into deposit-trie-cache
* gazelle
* cache deposit with eth1DepositIndex
* Merge branch 'master' into deposit-trie-cache
* fixed MerkleTrieIndex name
* Merge branch 'master' into deposit-trie-cache
* Merge branch 'master' into deposit-trie-cache
* Merge branch 'master' into deposit-trie-cache
* code review adjustments
* feature flag
* Merge branch 'master' into deposit-trie-cache

# Conflicts:
#	beacon-chain/cache/depositcache/pending_deposits.go
#	shared/featureconfig/config.go
#	shared/featureconfig/flags.go
* merge fix
* gazelle
* Merge branch 'master' into deposit-trie-cache
* return a copy of deposits trie from the cache
* move more logic under feature flag
* Merge branch 'master' into deposit-trie-cache
* added missing beacon chain flag
* Merge branch 'master' into deposit-trie-cache
* use helper function for trie copying
* Merge branch 'master' into deposit-trie-cache
* Merge branch 'master' into deposit-trie-cache
* Merge branch 'master' into deposit-trie-cache
* Merge branch 'master' into deposit-trie-cache
* Merge branch 'master' into deposit-trie-cache
2020-07-09 20:24:40 +00:00
Ivan Martinez
b052ab7087 Make E2E less flaky (#6536) 2020-07-09 14:08:21 -05:00
Ivan Martinez
322998f7f1 Move subnet topics to global vars (#6525)
* Deduplicate subnet topic definitions
* Move topics to global file
* Gaz
* Merge branch 'master' into move-topics
* Fix
* Merge refs/heads/master into move-topics
* Bazel
* Merge branch 'move-topics' of github.com:prysmaticlabs/prysm into move-topics
* Fix tests
* Fix
* Undo e2e changes
* Revert "Undo e2e changes"

This reverts commit 3037bb3590.
* Fix
* Fix
* Merge refs/heads/master into move-topics
* Merge refs/heads/master into move-topics
* Merge refs/heads/master into move-topics
* Comments
* Merge refs/heads/master into move-topics
* Merge refs/heads/master into move-topics
2020-07-09 17:38:15 +00:00
terence tsao
47cbfbf437 Remove participation link when list accounts (#6535)
* Remove url
* Merge refs/heads/master into rm-link
2020-07-09 16:50:19 +00:00
Raul Jordan
15d660d8eb Direct Keymanager: Proper Keystore.json Formatting (#6527)
* ensure keystore file has all fields
* fix tests
* fix build
* Merge branch 'master' into full-keystore-json
* define wallet.CanUnlockAccounts() to determine secret keys cache
initialization
* Merge branch 'full-keystore-json' of github.com:prysmaticlabs/prysm into full-keystore-json
* add can unlock accounts items
* Merge refs/heads/master into full-keystore-json
* Merge refs/heads/master into full-keystore-json
* Merge refs/heads/master into full-keystore-json
* Merge refs/heads/master into full-keystore-json
2020-07-09 16:24:06 +00:00
Raul Jordan
e3de674c77 Direct Keymanager: Proper Keystore.json Formatting (#6527)
* ensure keystore file has all fields
* fix tests
* fix build
* Merge branch 'master' into full-keystore-json
* define wallet.CanUnlockAccounts() to determine secret keys cache
initialization
* Merge branch 'full-keystore-json' of github.com:prysmaticlabs/prysm into full-keystore-json
* add can unlock accounts items
* Merge refs/heads/master into full-keystore-json
* Merge refs/heads/master into full-keystore-json
* Merge refs/heads/master into full-keystore-json
* Merge refs/heads/master into full-keystore-json
2020-07-09 16:23:53 +00:00
Preston Van Loon
fd80f73286 Improve make() capacity allocations (#6517)
* attestationutil.AttestingIndices: Minor improvement on indices array allocation
* progress
* more progress on makes
* progress
* more progress
* Merge branch 'master' of github.com:prysmaticlabs/prysm into memory1
* gaz
* fmt and imports
* Merge branch 'master' into memory2
* Min()
* remove spaces
* Merge branch 'master' of github.com:prysmaticlabs/prysm into memory2
* revert beacon-chain/operations/attestations/kv/block.go
* partially revert beacon-chain/operations/attestations/kv/aggregated.go
* Merge branch 'master' into memory2
* Merge branch 'master' into memory2
2020-07-09 15:50:58 +00:00
Nishant Das
c804347fc4 add fix and test (#6533)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-07-09 09:46:35 -05:00
Jim McDonald
b00c235586 Add separate permission for directories. (#6532) 2020-07-09 22:05:43 +08:00
Preston Van Loon
c2deab8948 State: use a constant for field count (#6513)
* State: use a constant for field count
* one more field count for alloc
* one more field count for alloc
* Merge branch 'master' into field-const-21
* Merge branch 'master' into field-const-21
* Merge branch 'master' into field-const-21
* Merge branch 'master' into field-const-21
2020-07-09 03:37:08 +00:00
terence tsao
8da0246702 Close buf writer in event of an error (#6523)
* Close buf writer in event of an error
* Merge refs/heads/master into close-bufwriter
* Merge refs/heads/master into close-bufwriter
2020-07-08 20:03:03 +00:00
Raul Jordan
132ad5beb8 Implement Accounts-v2 List: Direct Keymanager (#6510)
* begin list accounts impl
* colorize
* show deposit data
* separate responsibility
* comprehensive test for list complete
* gaz viz
* print account creation timestamp
* handle errs
* ask for wallet and pass dir
* Merge refs/heads/master into implement-v2-list
2020-07-08 19:21:54 +00:00
Ivan Martinez
ddf494f7b7 Use same permissions for all files in Prysm (#6522)
* Unify all perm prysm wide

* Gaz

* Imports
2020-07-08 12:30:22 -05:00
Victor Farazdagi
af4dfd4c36 Tiny optimization in state refs lock/unlock (#6520)
* tiny optimization in state refs count
2020-07-08 12:53:36 +00:00
Preston Van Loon
f2f2677070 attestationutil.AttestingIndices: Minor improvement on indices array allocation (#6508)
* attestationutil.AttestingIndices: Minor improvement on indices array allocation
* Merge branch 'master' of github.com:prysmaticlabs/prysm into HEAD
* Merge refs/heads/master into memory1
* Merge refs/heads/master into memory1
* Merge refs/heads/master into memory1
* Merge refs/heads/master into memory1
* Merge refs/heads/master into memory1
* Merge refs/heads/master into memory1
* Merge refs/heads/master into memory1
* Merge refs/heads/master into memory1
* Merge refs/heads/master into memory1
* Merge refs/heads/master into memory1
2020-07-08 09:57:50 +00:00
Preston Van Loon
a279f18461 Change from int64 to int for all flags so they load properly from config file. (#6498)
* Change from int64 to int for monitoring port so that the monitoring port is correctly ready from config file.
* Merge branch 'master' into fix-monitoring-port
* Merge branch 'master' into fix-monitoring-port
* Merge branch 'master' into fix-monitoring-port
* replace all other usages of int64 flag. @nisdas feedback
* Merge branch 'master' of github.com:prysmaticlabs/prysm into fix-monitoring-port
* Merge branch 'master' into fix-monitoring-port
* Merge branch 'master' into fix-monitoring-port
* Merge branch 'master' of github.com:prysmaticlabs/prysm into fix-monitoring-port
* revert tools/sendDepositTx
* fix build
* Merge branch 'master' into fix-monitoring-port
* Merge branch 'master' into fix-monitoring-port
* Merge branch 'master' into fix-monitoring-port
2020-07-08 08:21:06 +00:00
Preston Van Loon
c1ccadae55 Delete unused helpers.AttestingIndices (#6509)
* Delete unused helpers.AttestingIndices
* Merge branch 'master' into rm-AttestingIndices
* Merge branch 'master' into rm-AttestingIndices
* Merge branch 'master' into rm-AttestingIndices
* Merge branch 'master' into rm-AttestingIndices
2020-07-08 07:30:29 +00:00
Preston Van Loon
a02553815f Allocate the appropriate memory for retrieveIndicesFromBitfield (#6507)
* Allocate the appropriate memory for retrieveIndicesFromBitfield
* Merge refs/heads/master into memory0
* Merge refs/heads/master into memory0
* Merge refs/heads/master into memory0
* Merge refs/heads/master into memory0
* Merge refs/heads/master into memory0
* Merge refs/heads/master into memory0
* Merge refs/heads/master into memory0
2020-07-08 05:47:26 +00:00
Raul Jordan
fd9003f822 Integrate Accounts v2 Keymanager Into Validator Client (#6489)
* add in configs
* ask for enable accounts v2
* begin integration of v2 keymanager
* refactor wallet opening
* include significant refactoring of how opening a wallet works, making it easy to include at runtime
* ensure build with keymanager v2
* further improving runtime integration
* default pass paths
* finally running v2 at runtime
* import spacing
* Merge branch 'master' into v2-accounts-feature
* Merge refs/heads/master into v2-accounts-feature
* Merge refs/heads/master into v2-accounts-feature
* Merge refs/heads/master into v2-accounts-feature
* Merge refs/heads/master into v2-accounts-feature
* confs
* rem e2e val flag
* Merge branch 'master' into v2-accounts-feature
* Merge refs/heads/master into v2-accounts-feature
* Merge refs/heads/master into v2-accounts-feature
* Merge refs/heads/master into v2-accounts-feature
* Merge refs/heads/master into v2-accounts-feature
* Merge refs/heads/master into v2-accounts-feature
* Merge refs/heads/master into v2-accounts-feature
2020-07-08 05:01:09 +00:00
terence tsao
fe13f1f856 Add init-sync-verbose feature flag (#6515)
* Add flag
* Use flag
* Merge refs/heads/master into init-sync-verbose
* Merge refs/heads/master into init-sync-verbose
* Merge refs/heads/master into init-sync-verbose
* Merge refs/heads/master into init-sync-verbose
* Merge refs/heads/master into init-sync-verbose
2020-07-08 04:19:58 +00:00
Nishant Das
074e3c9aa7 Use Custom Block Hashing Method (#6501)
* blockroot
* gaz
* Merge refs/heads/master into useCustomMethod
* Merge refs/heads/master into useCustomMethod
* Merge refs/heads/master into useCustomMethod
* Merge refs/heads/master into useCustomMethod
* Merge refs/heads/master into useCustomMethod
* Merge refs/heads/master into useCustomMethod
* Merge refs/heads/master into useCustomMethod
* Merge refs/heads/master into useCustomMethod
2020-07-08 03:53:16 +00:00
terence tsao
d42d685f78 Update verifyBlkPreState to verify only (#6506)
* Verify should not return the state
* Update tests
* Merge branch 'master' of github.com:prysmaticlabs/prysm into refactor
* Sync with master
* Merge refs/heads/master into refactor
* Fixed error msgs. Thanks @rauljordan
* Merge branch 'master' of github.com:prysmaticlabs/prysm into refactor
* Merge branch 'refactor' of github.com:prysmaticlabs/prysm into refactor
* Merge refs/heads/master into refactor
* Merge refs/heads/master into refactor
* Merge refs/heads/master into refactor
2020-07-08 02:54:55 +00:00
Preston Van Loon
1067800430 p2p.AddConnectionHandler: Remove unused arg for goodbyeFunc (#6511)
* p2p.AddConnectionHandler: Remove unused arg for goodbyeFunc
* accidental rename reqFunc
* update test
* Merge branch 'master' into rm-unused-param
* Merge branch 'master' into rm-unused-param
2020-07-08 02:04:15 +00:00
terence tsao
d53ab16004 Move skip-regen-historical-states to deprecated (#6512)
* Remove unused test
* Move flag to be deprecated
* Remove usages
* Update BUILD.bazel
* Merge refs/heads/master into rm-skip-stategen-usages
2020-07-08 00:15:45 +00:00
Preston Van Loon
90b8b76ae8 Remove unused EpochAttestationsRoot method (#6514)
* Remove unused EpochAttestationsRoot method
2020-07-07 23:39:11 +00:00
dv8silencer
c77296ff82 Show overall validator results in addition to epoch specific results (#6492)
* Add aggregate stats struct to validator struct.  Log aggregate voting summary
from beginning of node startup.
* Refactoring out to function UpdateLogAggregateStats which updates and logs aggregate statistics
* Changes to spelling.  More refactoring so that the rest of aggregate stats are moved to UpdateLogAggregateStats
* Bugfix - moved startBalance initialization out of if.
* Remove a print debug.  Spelling adjustment.
* Minor refactoring/grammar changes
* Renaming variables for consistency.
* Address linter feedback:  omit 2nd value from range over map as values were not being used.
* Address feedback from PR re comments.

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Addressing feedback from terencechain in PR. (name changes)
* Addressing feedback from terencechain in PR.
* Create test for UpdateLogAggregateStats()
* Changed comment regarding "numberOfEpochs" log
* go fmt metrics_test.go
* bazel run //:gazelle -- fix
* fix typos
* Improved the code to detect the last iteration of the range responses loop
* Merge branch 'master' into tempIssue4987
* For both previous Epoch and total runtime statistics, do not count non-included attestations for "correctly" statistics.
* Update wording in since-launch logging

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Store first logged epoch in startEpoch and appropriately adjust calculations.  Initialize startEpoch to ^uint64(0).
* Merge branch 'master' into tempIssue4987
* Update validator/client/metrics.go
* Update validator/client/metrics.go
* Update validator/client/validator.go
* Update validator/client/metrics_test.go
* Merge branch 'master' into tempIssue4987
* Merge branch 'master' into tempIssue4987
* Merge branch 'master' into tempIssue4987
* Merge branch 'master' into tempIssue4987
* Merge branch 'master' into tempIssue4987
* Merge branch 'master' into tempIssue4987
* Merge branch 'master' into tempIssue4987
* Merge branch 'master' into tempIssue4987
* Merge branch 'master' into tempIssue4987
* Merge branch 'master' into tempIssue4987
2020-07-07 19:23:56 +00:00
Shay Zluf
87c18d4e0d Slasher readiness status response (#6504)
* Slasher readiness status response
* add comments
* Merge refs/heads/master into slasher_status
* Merge refs/heads/master into slasher_status
2020-07-07 17:11:07 +00:00
Ivan Martinez
d9fea7cb8c [QSP] Deduplicate functions for audit best practices (#6476)
* Deduplicate DB functions and getters

* Rename function

* Reduce to prestons suggestions

* Fix bug

* Fix

* Start changing state db funcs

* Fix tests

* Change return

* Add checks

* Fix test

* Fix

* Nishant comment

* Fix

* Fix

* Fix

* Fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-07-07 11:29:25 -05:00
Nishant Das
b868dfa481 Save Archived State (#6503) 2020-07-07 08:04:55 -07:00
Preston Van Loon
46b82eb15d Sync: Verify all block and attestations signatures on finalized blocks by default (#6499)
* Make verifying all signatures the default operation
* Merge branch 'master' into flip-verify-all
* Merge branch 'master' into flip-verify-all
* Merge branch 'master' into flip-verify-all
* Merge branch 'master' into flip-verify-all
2020-07-07 05:13:45 +00:00
Nishant Das
64fa474434 Batch Verify Blocks (#6469)
* add everything so far
* checkpoint progress
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into fastBLS
* fix
* checkpoint
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into fastBLS
* checkpoint again
* checkpoint again
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into fastBLS
* commenting
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into fastBLS
* bls cleanup
* revert this back
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into fastBLS
* revert core changes
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into fastBLS
* add flag
* add test
* add one more test
* clean up
* comment
* lint
* terence's review
* Merge refs/heads/master into fastBLS
* Merge refs/heads/master into fastBLS
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into fastBLS
* Merge refs/heads/master into fastBLS
* remove additional method
* Merge branch 'fastBLS' of https://github.com/prysmaticlabs/geth-sharding into fastBLS
* fix
* Merge refs/heads/master into fastBLS
* copy
* Merge branch 'fastBLS' of https://github.com/prysmaticlabs/geth-sharding into fastBLS
2020-07-07 04:16:12 +00:00
Shay Zluf
8ddfde41e3 Historical detection fix (#6455)
* Historical detection before realtime detection

* comment fixes

* remove logs

* remove logs

* remove logs

* gaz

* handle underflow

* add regressiion test

* update test name

* gaz

* gofmt

* fix comment

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-07-07 05:57:40 +03:00
terence tsao
7f741e48e0 Cleanup process block/operation names (#6500) 2020-07-06 18:24:30 -07:00
Preston Van Loon
e96e1f0569 Add a placeholder constant for ignoring cancelled subscription errors from libp2p pubsub. (#6496)
* Add a placeholder constant for ignoring cancelled subscription errors
from libp2p pubsub.

This change temporarily resolves
https://github.com/prysmaticlabs/prysm/issues/6449.

This change should be revisited after
https://github.com/libp2p/go-libp2p-pubsub/pull/356 merges and the prysm
libp2p pubsub dependency is updated.
* Merge branch 'master' into issue-6449
* Merge branch 'master' into issue-6449
* Merge branch 'master' into issue-6449
2020-07-06 23:55:55 +00:00
terence tsao
1a6c55c637 Split block_operations.go into smaller files (#6495) 2020-07-06 15:27:42 -07:00
Jim McDonald
7bb0ee78af Update beacon state locks (#6326)
* Rationalise state locking
* Rationalise state locking
* Merge branch 'state-locks' of github.com:mcdee/prysm into state-locks
* Merge branch 'master' into state-locks
* Add feature flag
* Merge
* Merge branch 'master' into state-locks
* Merge branch 'master' into state-locks
* Update locks
* Merge branch 'master' into state-locks
* Gazelle
* Tidy-ups
* Merge branch 'master' into state-locks
* Remove commentary to a docs.go file for better presentation on godocs
* Add newBeaconStateLocks as a --dev flag
* gofmt
* Merge branch 'master' into state-locks
2020-07-06 20:52:53 +00:00
Preston Van Loon
005a9dde48 CI: Remove GCP remote cache (#6497)
* Remove GCP remote cache as it has been sunset on GCP
2020-07-06 19:25:35 +00:00
terence tsao
def7b602e3 Hot states use no DB (#6488)
* Add cache to service struct
* Update hot getters/setters to use cache
* Update migration
* Update other services to adapt
* Fix initial sync get state
* Update getter related tests
* Update hot related tests
* Update migrate related tests
* New awesome tests for migration
* Clean up rest of the tests
* Merge refs/heads/master into hot-state-no-db
* Fix block chain head tests
* Fix block chain processor tests
* Fixed RPC tests
* Update cold getter and test
* Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db
* Fix sync tests
* Short cut if state is already in DB
* Remove uneeded saves
* Update beacon-chain/state/stategen/hot_test.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Update beacon-chain/state/stategen/getter_test.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Update beacon-chain/state/stategen/getter_test.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Update beacon-chain/state/stategen/service.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Update beacon-chain/state/stategen/setter_test.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Preston's feedback
* Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db
* Return a copy of cache states
* Remove hot state caches check in StateByRoot
* Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db
* Merge refs/heads/master into hot-state-no-db
* Raul's feedback
* Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db
2020-07-06 17:22:12 +00:00
Nishant Das
fa3da9a6a2 Add In New Core Methods for Batch Verify (#6482)
* add in new core methods

* clean up

* add verification method

* add tests

* add 1 more test

* lint

* Update beacon-chain/core/blocks/block_operations.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Update beacon-chain/core/blocks/block_operations.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Update beacon-chain/core/state/transition.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* terences review

* Update beacon-chain/core/state/transition.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* fmt

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-07-06 11:09:29 +08:00
Ivan Martinez
359b9bef49 Merge separated local slashing protection flags (#6487)
* Merge split-up local protection flags

* Fix deprecated flags

* Fix text
2020-07-03 21:01:54 -05:00
Raul Jordan
d447bb6458 Direct Keymanager: Implement Sign (#6477)
* implement keymanager proto

* utilize proto message and properly comment

* add in godoc

* doc

* add in doc

* sign implementation done

* add documentation for direct keymanager and add lock to signing

* sign method tests

* update godoc

* concrete sign

* fix build

* proper iface
2020-07-03 18:00:02 -05:00
Raul Jordan
8c258278d0 Direct Keymanager: Implement FetchValidatingPublicKeys() (#6475)
* begin on fetch direct
* fetch validating public keys impl
* test for fetch validating keys
* fetch validating public keys done
* helper function and benchmark
* rename package
* viz
* Update validator/accounts/v2/testing/BUILD.bazel

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* gaz
* add lock
* comment
* Merge refs/heads/master into fetch-direct-keys
2020-07-03 20:26:00 +00:00
Ivan Martinez
f0fcebccc4 Move slashing protection code to separate files for proposing and attesting (#6406)
* Get started on cleaning up slashing protection
* Merge branch 'master' of github.com:prysmaticlabs/prysm into cleanup-protection
* Move protection functions to own files
* Lint
* Merge branch 'master' of github.com:prysmaticlabs/prysm into cleanup-protection
* Begin adding test for proposal protection
* Merge branch 'master' of github.com:prysmaticlabs/prysm into cleanup-protection
* Fix build
* Fix tests
* Fix tst
* Fix tests
* Fix proposal tests
* Merge branch 'master' into cleanup-protection
* Merge branch 'master' into cleanup-protection
* Merge branch 'master' into cleanup-protection
* Merge branch 'master' of github.com:prysmaticlabs/prysm into cleanup-protection
* Reorder protections
* Change lock
* Fix test
* Merge branch 'master' into cleanup-protection
* Merge branch 'master' into cleanup-protection
* Merge branch 'master' into cleanup-protection
* Change log
* Merge branch 'cleanup-protection' of github.com:prysmaticlabs/prysm into cleanup-protection
* Merge branch 'master' into cleanup-protection
2020-07-03 19:54:42 +00:00
Raul Jordan
9d979de4ed Direct Keymanager: Implement Account Creation (#6466)
* implementation using petname and keystore
* writing new account to disk along with password
* more logic for properly writing accounts
* print out mnemonic
* save deposit data rlp
* write deposit tx and ssz deposit data to account path
* wrap up account creation
* fix prompt
* generate deposit tx
* direct account creation test
* fix up formatting
* lint
* match formatting
* more sustainable approach towards unmarshaling config file
* resolve feedback
* fix broken import
* comprehensive tests for create account
* tests pass
* Merge branch 'master' into direct-keys
* tidy
* Merge branch 'direct-keys' of github.com:prysmaticlabs/prysm into direct-keys
* Merge refs/heads/master into direct-keys
* gaz
* Merge branch 'direct-keys' of github.com:prysmaticlabs/prysm into direct-keys
* nondeterministic names
* comment
* gaz
* better error wrap
* Merge refs/heads/master into direct-keys
* docker deps
* Merge branch 'direct-keys' of github.com:prysmaticlabs/prysm into direct-keys
* Merge refs/heads/master into direct-keys
* Merge refs/heads/master into direct-keys
* Merge refs/heads/master into direct-keys
* Merge refs/heads/master into direct-keys
* Merge refs/heads/master into direct-keys
* Merge refs/heads/master into direct-keys
* Merge refs/heads/master into direct-keys
* Merge refs/heads/master into direct-keys
* Merge refs/heads/master into direct-keys
* ivan feedback
* Merge refs/heads/master into direct-keys
* Update validator/accounts/v2/wallet.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* fixed tests and comments
* Merge refs/heads/master into direct-keys
2020-07-03 18:49:16 +00:00
terence tsao
8bb5532377 Epoch boundary cache (#6486)
* Add epoch boundary state cache
* Clean up epoch boundary state cache
* Starting tests
* More tests
* Merge branch 'master' of github.com:prysmaticlabs/prysm into epoch-boundary-cache
* Merge refs/heads/master into epoch-boundary-cache
2020-07-03 17:29:30 +00:00
Victor Farazdagi
688f0d7114 Refactors attestation pool pruning (#6485) 2020-07-03 09:43:56 -07:00
Ivan Martinez
c69b3f568e Move hard-coded constants to global consts (#6467)
* Move hard-coded constants to global consts
* Fixes
* Merge branch 'master' into remove-hard-constants
* Merge refs/heads/master into remove-hard-constants
* Merge refs/heads/master into remove-hard-constants
* Merge refs/heads/master into remove-hard-constants
* Merge refs/heads/master into remove-hard-constants
* Comments
* Merge refs/heads/master into remove-hard-constants
* Merge refs/heads/master into remove-hard-constants
* Merge refs/heads/master into remove-hard-constants
* Merge refs/heads/master into remove-hard-constants
* Merge refs/heads/master into remove-hard-constants
* Merge branch 'master' of github.com:prysmaticlabs/prysm into remove-hard-constants
2020-07-03 15:25:32 +00:00
Victor Farazdagi
d1f1628478 Restores att-aggregation benchmarks (#6484)
* restores att-aggregation benchmarks
2020-07-03 14:37:54 +00:00
Nishant Das
7896e087e5 Add in Better Logging for Requests (#6483)
* add in better logging
* gaz
* last one
* mess about
* check strings
* fix
2020-07-03 10:08:45 +00:00
Nishant Das
ac9bc5d98b Reset Streams at the End of the Request (#6481)
* close all incoming streams

* use proper helper

* shift to debug

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-07-03 16:21:18 +08:00
Nishant Das
353229d50a Add New Secure Batch Signature Verification Method (#6479)
* add more tests
* gaz
* Merge refs/heads/master into addBLSMethods
* Merge refs/heads/master into addBLSMethods
* Merge refs/heads/master into addBLSMethods
* Merge refs/heads/master into addBLSMethods
2020-07-03 07:38:13 +00:00
terence tsao
d62e989a67 Minor clean up on state gen package (#6480)
* Remove unused fields and function
* Rename splitInfo to finalizedInfo
* Merge branch 'master' into stategen-cleanup
* Fmt
* Merge branch 'stategen-cleanup' of github.com:prysmaticlabs/prysm into stategen-cleanup
* Merge refs/heads/master into stategen-cleanup
2020-07-03 06:33:58 +00:00
Preston Van Loon
6b9d9e5d3a Add a few test cases to ReceiveBlockNoPubsub with race detection (#6298)
* Add a few test cases to ReceiveBlockNoPubsub with race detection, fix data race in state
* Expose voluntary exits included API. More race condition fixes in state. More tests
* add tests to norace as well for coverage
* Merge branch 'master' of github.com:prysmaticlabs/prysm into add-test-coverage-on-receive-block
* PR feedback
* Fixed can't save head in DB by:
1.) Remove new state mgmt condition in SaveHeadBlockRoot
2.) Reuse state summary cache between db and service
* Merge branch 'master' of github.com:prysmaticlabs/prysm into add-test-coverage-on-receive-block
* passing tests
* gofmt and goimports
* Merge branch 'master' of github.com:prysmaticlabs/prysm into add-test-coverage-on-receive-block
* lint
* move import and goimport again
* move import and goimport again
* Merge refs/heads/master into add-test-coverage-on-receive-block
* Merge refs/heads/master into add-test-coverage-on-receive-block
* Add bool to selectively record events
* Merge refs/heads/master into add-test-coverage-on-receive-block
* Merge refs/heads/master into add-test-coverage-on-receive-block
* Merge refs/heads/master into add-test-coverage-on-receive-block
* Merge refs/heads/master into add-test-coverage-on-receive-block
2020-07-03 05:46:53 +00:00
dv8silencer
0d2bce8200 Improve consistency in peer connect/disconnect logging (#6409)
* Improve consistency in peer connect/disconnect logging
* Merge branch 'master' into dv8s-iss6328
* Merge branch 'master' into dv8s-iss6328
* Merge branch 'master' into dv8s-iss6328
* Merge branch 'master' into dv8s-iss6328
* Merge branch 'master' into dv8s-iss6328
* Corrected capitalization.  Refactored out duplicated logic
* Merge branch 'master' into dv8s-iss6328
* Merge branch 'master' into dv8s-iss6328
* Merge branch 'master' into dv8s-iss6328
* Merge branch 'master' into dv8s-iss6328
* Merge branch 'master' into dv8s-iss6328
* Merge branch 'master' into dv8s-iss6328
* Merge branch 'master' into dv8s-iss6328
* Merge branch 'master' into dv8s-iss6328
* Merge branch 'master' into dv8s-iss6328
* Merge branch 'master' into dv8s-iss6328
* Merge branch 'master' into dv8s-iss6328
* Merge branch 'master' into dv8s-iss6328
* Merge branch 'master' into dv8s-iss6328
* Merge branch 'master' into dv8s-iss6328
* Merge branch 'master' into dv8s-iss6328
* Merge branch 'master' into dv8s-iss6328
2020-07-03 04:44:57 +00:00
Raul Jordan
5b708b54d9 QSP-13 Only Allow Snappy P2P Encoding (#6415)
* enforce only snappy

* fix up tests

* replace with ssz snappy in tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-07-03 11:24:30 +08:00
Preston Van Loon
aa52e693ce Sync: do not remove attestations from pool on failed block processing (#6474)
* Add test and return error on failed block processing
* use atts in body
* Merge branch 'master' into p2p-potential-error-block
2020-07-02 20:24:30 +00:00
Raul Jordan
091525deb8 Revert Stream Duties (#6473)
* ensure gazelle
* fix up propose test
* build
* comment
2020-07-02 17:50:05 +00:00
Victor Farazdagi
f279cf09b8 Removes excessive concurrency from init-sync queue (#6471)
* Removes excessive concurrency from init-sync queue
* update buffer size
2020-07-02 13:06:12 +00:00
Victor Farazdagi
92c5c651f7 Refactors init-sync: queue streams blocks in batches (#6470)
* init-sync: queue streams blocks in batches
2020-07-02 11:19:07 +00:00
Raul Jordan
2d6f4ebf18 Part 1: Implement Accounts-V2 New, Wallet Creation (#6451)
* begin accounts-v2 new

* password validation

* validator accounts new with eip-2335 keystore

* select different wallet type based on enum

* clean up code significantly

* more robust code structure

* check if wallet exists

* define read and create wallet methods

* fmt

* go mod and comment

* comment

* redundant name

* satify gofmt

* add instructions with keymanager opts

* wrap up create and read wallet functionality

* prep for readiness

* doc improvements

* tests for create and read wallet

* update deps

* tidy

* visibility

* gaz

* fix up

* refactor for proper usage, with wallet and keymanager ifaces

* Update validator/flags/flags.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>

* import

* improve structure

* wrap up all comments

* simplify

* lint

* Update validator/accounts/v2/cmd.go

* viz check

* add interface methods as needed

* fix build

* lint

* nishant feedback

* simplify structure

* add tests for strong password check

* all feedback done

* ivan feedback

* ivan feedback

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-07-01 16:30:01 -05:00
terence tsao
8c8cc144f1 Archival RPC usages removal (#6453)
* Starting to delete archived rpc usages
* Fixed rest of the tests
* Gaz
* Merge branch 'master' into deprecate-archival-rpc
* Merge branch 'master' into deprecate-archival-rpc
* Merge refs/heads/master into deprecate-archival-rpc
* Merge refs/heads/master into deprecate-archival-rpc
* Merge refs/heads/master into deprecate-archival-rpc
2020-07-01 20:04:02 +00:00
Martin Linkhorst
aeb0b4bf75 Allow to start slasher RPC server with TLS (#6465)
* add warning if slasher server isn't using tls
* pass on tls key and cert to the slasher rpc server
* Merge branch 'master' into slasher-tls
2020-07-01 19:31:29 +00:00
Victor Farazdagi
270a64f059 QSP-BestPractice: Refactors singular/plural KV-store functions (#6462)
* updates SaveBlock
* updates DeleteAttestation
* updates DeleteState
* updates SaveAttestation
* updates SaveState
* updates SaveStateSummary
* Merge refs/heads/master into kv-refactor-singular-pl-methods
* Merge refs/heads/master into kv-refactor-singular-pl-methods
* Merge refs/heads/master into kv-refactor-singular-pl-methods
2020-07-01 18:33:05 +00:00
Preston Van Loon
165cb0d7b4 E2E: Add a check for cold state storage (#6457)
* Add a check for cold state storage
* amend comment
* gaz
* Merge branch 'master' into coldstate-e2e
* Merge refs/heads/master into coldstate-e2e
* Merge refs/heads/master into coldstate-e2e
* PR feedback
* Merge branch 'coldstate-e2e' of github.com:prysmaticlabs/prysm into coldstate-e2e
* Update endtoend/evaluators/data.go
* Merge refs/heads/master into coldstate-e2e
* Merge refs/heads/master into coldstate-e2e
* Merge refs/heads/master into coldstate-e2e
2020-07-01 17:35:05 +00:00
terence tsao
1f3d9873cd Archival service removal (#6448) 2020-07-01 10:04:06 -07:00
Victor Farazdagi
1b0cfb408e QSP-BestPractice: Refactors inline ttfb/resp time constants (#6463)
* refactors inline ttfb/resp time constants
2020-07-01 09:47:59 +00:00
terence tsao
4e180e452d Add spans to db kv's encode and decode (#6458) 2020-06-30 21:00:49 -07:00
terence tsao
8d8da76aa0 API: Fix finalization/justification slot representations (#6443)
* Fix finalized/justified slots representations
* Update tests
* Merge refs/heads/master into use-epochs
* Merge refs/heads/master into use-epochs
* Merge refs/heads/master into use-epochs
* Merge refs/heads/master into use-epochs
* Go fmt
* Merge branch 'use-epochs' of github.com:prysmaticlabs/prysm into use-epochs
* Merge refs/heads/master into use-epochs
* Merge refs/heads/master into use-epochs
* Fixed TestServer_StreamChainHead_OnHeadUpdated
* Merge branch 'use-epochs' of github.com:prysmaticlabs/prysm into use-epochs
* Merge refs/heads/master into use-epochs
2020-06-30 15:17:59 +00:00
Raul Jordan
a0c38c8d0a QSP-33 Check Max Response For All Topics (#6424)
* remove max len funcs

* fix up tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-30 09:50:28 -05:00
Nishant Das
21ead0a025 Record Peer Latency (#6456)
* add latency
* Merge branch 'master' into addLatency
* Merge refs/heads/master into addLatency
2020-06-30 03:53:14 +00:00
Preston Van Loon
98e26d1dcc Bulk verify block deposits with BLS (#6454)
* First pass at adding bulk signature verification for block deposits

* checkpoint

* some refactoring to allow for bulk deposit signature verification

* nil check

* fix goimports

* fmt

* rename ProcessPreGenesisDeposit -> ProcessPreGenesisDeposits
2020-06-29 20:52:19 -05:00
terence tsao
1f93a1f4c3 Remove unused prune old states (#6450) 2020-06-29 15:02:31 -07:00
Raul Jordan
6c7131cb54 Fix Concurrent Map Write in gRPC Client Connected Logger (#6446)
* fix up concurrent map write
* fix import
* Merge refs/heads/master into concurrent-write-fix
2020-06-29 15:52:10 +00:00
Victor Farazdagi
196f4c6222 QSP-BestPractice Refactors failover peer selection in init-sync (#6442)
* refactors failover peer selection in init-sync
* Merge refs/heads/master into init-sync-more-effective-failover
2020-06-29 15:23:04 +00:00
Ivan Martinez
e7b24323e6 Move MaxRPCPageSize flag from beacon node to shared/cmd (#6297)
* Allow other clients to set MaxRPCPageSize

* Fixes

* Remove default

* Add init

* Fix default

* Set default

* Fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-29 09:17:24 -05:00
Preston Van Loon
047880caf2 Change the aggregatedSlotCommitteeIDCacheLock to release the lock immediately after r/w (#6437)
* Change the aggregatedSlotCommitteeIDCacheLock to release the lock immediately after r/w
* Merge branch 'master' into better-validator-lock
* Merge branch 'master' into better-validator-lock
2020-06-29 06:14:21 +00:00
Nishant Das
2bce9c884a Add More Altona Bootnodes (#6441)
* enable noise
* add secio
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into makeNoiseDefault
* add new bootnodes
2020-06-29 04:02:20 +00:00
Nishant Das
adf9bf7094 QSP 50, QSP 51: Make Noise Default (#6440)
* enable noise

* add secio
2020-06-29 11:19:25 +08:00
Preston Van Loon
bfb6e5e4a0 Use spec attestation time verification in gRPC requests (#6429)
* Re-use attestation time verification

* lint

* fix imports

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-28 14:37:17 -07:00
Ivan Martinez
d973c00c6c Slasher detection improvements (#6432)
* Slashing fixes

* Fixess

* Change fatals to error

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-28 02:34:21 -04:00
Raul Jordan
417bb45398 Begin Scaffold of Validator Accounts V2 (#6310)
* v2 refactor
* fix up imports
* builds
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge branch 'master' into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* imports keymanager v1
* Update validator/keymanager/v1/remote.go
* Update validator/client/polling/propose.go
* Update validator/keymanager/v1/wallet.go
* fmt
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* Merge refs/heads/master into accounts-revamp-p1
* fix conf
* fix build
* gaz
2020-06-27 15:48:36 +00:00
terence tsao
2a997828a3 Audit best practice feedback - validator DB test folder (#6426)
* Refactor validators db

* Lint

* Lint

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-26 21:37:43 -05:00
Preston Van Loon
a9bbbae19a Refactor params/config to separate files for clarity (#6427)
* split mainnet/onyx/altona/e2e configs into different files
* set onyx as the default network
* lint and fix comment
* more commentary
* gofmt
* Merge branch 'master' of github.com:prysmaticlabs/prysm into onyx-config
* gofmt
2020-06-27 00:20:24 +00:00
terence tsao
43b1bcb84f Use CopyValidator for ValidatorAtIndex (#6430)
* Use CopyValidator
* Merge refs/heads/master into validator-at-index
2020-06-26 23:21:11 +00:00
Preston Van Loon
c6179307b3 Add spans for head state access (#6418)
* Add a span for HeadState access
* more spans
* Merge refs/heads/master into add-span
* Merge refs/heads/master into add-span
* Merge refs/heads/master into add-span
* Merge refs/heads/master into add-span
* Merge refs/heads/master into add-span
* Merge refs/heads/master into add-span
* Merge refs/heads/master into add-span
* Merge refs/heads/master into add-span
2020-06-26 22:52:11 +00:00
Nishant Das
5d53aa3cac Update Config for Altona (#6414) 2020-06-26 15:11:40 -07:00
Preston Van Loon
67b8a88b65 Add span for updateRecentCanonicalBlocks (#6421)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-26 15:08:53 -05:00
Preston Van Loon
3ba3133657 Cleanup validateAggregateAttTime with a better error message, more tests, and more clear inline comments (#6423)
* Add span for updateRecentCanonicalBlocks
* Cleanup validateAggregateAttTime with a better error message, more tests, and more clear inline comments
* Merge branch 'master' into cleanup-validateAggregateAttTime
* revert beacon-chain/blockchain/head.go
* Merge branch 'cleanup-validateAggregateAttTime' of github.com:prysmaticlabs/prysm into cleanup-validateAggregateAttTime
* Merge refs/heads/master into cleanup-validateAggregateAttTime
* Merge refs/heads/master into cleanup-validateAggregateAttTime
2020-06-26 19:36:44 +00:00
terence tsao
01b97ffd5e Fix ListValidators retrieval for new state (#6417)
* Retrieve the correct state for new state
* Test
* Gaz
* Merge branch 'master' into fix-list-validators
* Merge refs/heads/master into fix-list-validators
* Merge refs/heads/master into fix-list-validators
* Merge refs/heads/master into fix-list-validators
* Merge refs/heads/master into fix-list-validators
2020-06-26 19:01:46 +00:00
Preston Van Loon
886afb054c Remove duplicated randao mix copy (#6420)
* remove duplicated randao mix copy
* Merge refs/heads/master into rm-dup-copy
* Merge refs/heads/master into rm-dup-copy
* Merge refs/heads/master into rm-dup-copy
2020-06-26 18:27:48 +00:00
Nishant Das
98757e759c Update Geth (#6422)
* update geth
* Merge refs/heads/master into removeSlowdown
2020-06-26 17:59:02 +00:00
Raul Jordan
001547f215 QSP-55 Remove Support for Other Pubsub Protocols (#6419)
* remove nongossip protocols
* Merge refs/heads/master into remove-other-protocols
2020-06-26 17:33:18 +00:00
Nishant Das
f1c0b2e0da Fix Invalid SSZ in Block Roots RPC Request (#6405)
* fix block roots error
* remove comment
* fix
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into fixBlockRootsError
* change back
* add stream deadlines
* Merge branch 'master' into fixBlockRootsError
* add todos
* Merge branch 'fixBlockRootsError' of https://github.com/prysmaticlabs/geth-sharding into fixBlockRootsError
* Update beacon-chain/sync/rpc_beacon_blocks_by_root.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* gofmt
* fix test
* fix failing test
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into fixBlockRootsError
* fix all
* Merge branch 'master' into fixBlockRootsError
* Merge branch 'master' into fixBlockRootsError
* Merge refs/heads/master into fixBlockRootsError
2020-06-26 17:05:04 +00:00
Raul Jordan
ab89053597 QSP-45 Add Check for No Connected Peers BestFinalized (#6402)
* no error returned
* no target root
* fix build
* Merge refs/heads/master into best-finalized
2020-06-26 16:35:12 +00:00
Raul Jordan
252f758baa QSP-9 Prevent Casting to Int if Possible (#6349)
* no cast to int

* fix up significant casting issues

* more casting

* even more casting fixes

* more casts

* fix subnets

* back to ints

* final touches

* broken test fix

* add in blocks test fix

* unskip

* revert bytes fixes

* casting fixes

* Update beacon-chain/db/kv/state.go

* Update beacon-chain/db/kv/blocks.go

* fmt

* slash:

* fix val tests

* fix up conf

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-06-26 11:07:00 -05:00
Victor Farazdagi
78465e2549 QSP-6: Enforces crypto-secure PRNGs (#6401)
* adds cryptorand analyzer

* better naming

* rely on suffix

* sync/pending_* use crypto/rand

* define shared/rand

* updates fetcher

* fixes rand issue in sync package

* gofmt

* shared/rand: more docs + add exclusion nogo_config.json

* updates validator/assignments

* updates comment

* fixes remaning cases

* re-arranges comments

* fixes tests

* renames in shared/rand API

* adds simple no-panic test

* gazelle

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-26 09:58:47 -05:00
Nishant Das
73f3a653bf Clean Up Dead Peers (#6413)
* clean up dead peers
* Merge refs/heads/master into clearDeadPeers
* comment
* Merge branch 'clearDeadPeers' of https://github.com/prysmaticlabs/geth-sharding into clearDeadPeers
2020-06-26 12:24:55 +00:00
dv8silencer
812a663671 Add flag to allow reading bootstrap nodes from a file (#6351)
* parent c837dfb2f2
author dv8silencer <dv8silencer+github@gmail.com> 1592805180 -0500
committer dv8silencer <dv8silencer+github@gmail.com> 1592855989 -0500
Create a bootstrap-node-file flag which reads nodes from a YAML file and make the flag mutually exclusive to the bootstrap-node flag
* Merge branch 'master' into dv8s-iss6316
* Refactor so that boot node reading is in its own function.  Added test
* Added period

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Merge branch 'master' into dv8s-iss6316
* Edit Bazel to include go-yaml dep
* Help Bazel build successfully
* Merge branch 'master' into dv8s-iss6316
* bazel run //:gazella -- fix
* Handle error from YAML parser to address build checks
* Merge branch 'master' into dv8s-iss6316
* Merge branch 'master' into dv8s-iss6316
* Updated flags.go to refine the Usage for the BootStrapNodeFile flag

Co-authored-by: Nishant Das <nish1993@hotmail.com>
* Merge branch 'master' into dv8s-iss6316
* Utilize stdlib TempFile to avoid polluting
* Merge branch 'master' into dv8s-iss6316
* Merge branch 'master' into dv8s-iss6316
* Merge branch 'master' into dv8s-iss6316
* Merge branch 'master' into dv8s-iss6316
* Merge branch 'master' into dv8s-iss6316
* Merge branch 'master' into dv8s-iss6316
* Merge branch 'master' into dv8s-iss6316
* Merge branch 'master' into dv8s-iss6316
* Merge branch 'master' into dv8s-iss6316
* Merge branch 'master' into dv8s-iss6316
* Merge branch 'master' into dv8s-iss6316
* merge master into this branch
* Merge branch 'master' into dv8s-iss6316
* Merge branch 'master' into dv8s-iss6316
* Merge branch 'master' into dv8s-iss6316
* Merge branch 'master' into dv8s-iss6316
* Merge branch 'master' into dv8s-iss6316
* Merge branch 'master' into dv8s-iss6316
* Merge branch 'master' into dv8s-iss6316
* Merge branch 'master' into dv8s-iss6316
* Merge branch 'master' into dv8s-iss6316
* Changed bootstrap-node to StringSlice flag to allow multiple nodes to be passed.  Each value can be .enr file which will be YAML parsed to extract nodes
* Refactored to create separate readbootNodes function and added a test for it.
* More cleaning up
* Changed wording in the cli help for --bootstrap-node
* Merge branch 'master' into dv8s-iss6316
* Since we are taking YAML files, got rid of .enr check and instead we check for the string prefix of a enr record or not to determine if file.
* Merge with dv8silencer/tempissue6316
* Correct spacing

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Changed how we check for YAML file vs CLI-direct ENR
* Minor:  Changed to 1 string from concatenating 2
* Merge branch 'master' into dv8s-iss6316
2020-06-26 11:23:38 +00:00
terence tsao
461bc4694f QST40 QST42 QST44 - Move deletes to test scope only (#6410)
* Rm delete from interface
* Rm passthroughs
* Unexport
* Lint
* Merge branch 'master' into unexport-delete
* Merge refs/heads/master into unexport-delete
* Merge refs/heads/master into unexport-delete
* Merge refs/heads/master into unexport-delete
2020-06-26 04:29:13 +00:00
Preston Van Loon
806a465117 Remove unused p2p encoding method (#6411)
* Remove unused p2p encoding method
2020-06-26 04:02:09 +00:00
Raul Jordan
1b430e0c17 QSP-62 Corrupted Clock In Stream Deadlines (#6404)
* deadline warn
* add proper godoc information
* clarify
* Merge refs/heads/master into corrupted-clock
* fix broken build
* Merge refs/heads/master into corrupted-clock
* Merge refs/heads/master into corrupted-clock
2020-06-25 22:36:18 +00:00
Raul Jordan
6b27452947 QSP-39 AreEth1DataEqual Should Return True If Both Nil (#6372)
* resolve small core issue

* add tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-25 18:01:58 -04:00
terence tsao
ac69dbc2f8 Deprecates --disable-init-sync-batch-save-blocks flag (#6403)
* Deprecate flags in feature config

* Deprecate usages

* Update test

* Go fmt

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-25 15:55:43 -05:00
Raul Jordan
45e1c0bb8c QSP-54 Handle Default Datadir Empty String (#6394)
* default datadir
* Merge refs/heads/master into default-data-dir
* Merge refs/heads/master into default-data-dir
* Merge refs/heads/master into default-data-dir
* Merge refs/heads/master into default-data-dir
* Update node.go
* Merge refs/heads/master into default-data-dir
* Merge refs/heads/master into default-data-dir
* Merge refs/heads/master into default-data-dir
* Merge refs/heads/master into default-data-dir
* Merge refs/heads/master into default-data-dir
* Merge refs/heads/master into default-data-dir
2020-06-25 18:02:07 +00:00
Shay Zluf
9103ec98cb QSP-14 consistent file permissions (#6378)
* QSP-14 consistent file permissions
* Merge refs/heads/master into consistent_permissions
* Merge refs/heads/master into consistent_permissions
* default io file permissions
* Merge branch 'consistent_permissions' of github.com:prysmaticlabs/prysm into consistent_permissions
* fix comments
* Merge refs/heads/master into consistent_permissions
* Merge refs/heads/master into consistent_permissions
* gaz
* Merge branch 'consistent_permissions' of github.com:prysmaticlabs/prysm into consistent_permissions
* Merge refs/heads/master into consistent_permissions
* Rename IoConfig.go to io_config.go
* Update shared/params/BUILD.bazel
* Merge refs/heads/master into consistent_permissions
* raul feedback
* Merge branch 'consistent_permissions' of github.com:prysmaticlabs/prysm into consistent_permissions
* Merge refs/heads/master into consistent_permissions
* Merge refs/heads/master into consistent_permissions
* Merge refs/heads/master into consistent_permissions
* Merge refs/heads/master into consistent_permissions
* Merge refs/heads/master into consistent_permissions
* Merge refs/heads/master into consistent_permissions
* gaz
* Merge branch 'consistent_permissions' of github.com:prysmaticlabs/prysm into consistent_permissions
* Merge refs/heads/master into consistent_permissions
* Merge refs/heads/master into consistent_permissions
* Merge refs/heads/master into consistent_permissions
* Merge refs/heads/master into consistent_permissions
2020-06-25 16:12:59 +00:00
Victor Farazdagi
00f24f5729 Deprecates --disable-init-sync-wrr flag (#6400)
* deprecates wrr-related flag

* gofmt + gazelle

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-25 10:26:30 -05:00
Raul Jordan
90bfc9a395 QSP-49 Recalibrate Roughtime (#6393)
* roughtime recalibrate

* recalibrate roughtime

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-25 09:02:54 -05:00
Nishant Das
5fdf6310f7 add back handler (#6398)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-25 13:07:31 +03:00
Raul Jordan
662c5cd302 QSP-47 Remove Meaningless Relay Addr Check (#6388)
* add relay factory fix
* Merge refs/heads/master into qsp-47
* Merge refs/heads/master into qsp-47
* Merge refs/heads/master into qsp-47
* Merge refs/heads/master into qsp-47
* Merge refs/heads/master into qsp-47
* Merge refs/heads/master into qsp-47
* Merge refs/heads/master into qsp-47
* Merge refs/heads/master into qsp-47
* Merge refs/heads/master into qsp-47
2020-06-25 09:16:30 +00:00
Raul Jordan
26e27bf052 QSP-41 Clarify Block Filters Passthrough In DB (#6397)
* clarify filters
* Merge refs/heads/master into blocks-filter-criteria
* Merge refs/heads/master into blocks-filter-criteria
2020-06-25 08:31:07 +00:00
terence tsao
9a156bd73a Best practice follow up - remove outdated todos (#6392)
* Addressed 30.

* Remove outdated todos

* Go fmt

* Go fmt

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-24 23:14:11 -05:00
Preston Van Loon
412ea2dca1 Update CONTRIBUTING.md to include link to docs portal (#6396)
* Update CONTRIBUTING.md
* Merge branch 'master' into prestonvanloon-patch-1
2020-06-25 02:59:49 +00:00
Raul Jordan
0b178b446a QSP-64 Conditional for Rebuilding BlockRoots Trie in State Setters (#6390)
* rebuild trie for block roots
* Merge refs/heads/master into rebuild-trie-fix
* Merge refs/heads/master into rebuild-trie-fix
2020-06-25 02:03:18 +00:00
Preston Van Loon
fb8be4d555 Refactor BLS (#6395)
* refactor BLS to expose an interface rather than a single implementation
* Split up tests, gofmt, goimports, docs
* godoc for signature.Copy()
* more godocs
* revert gomod / deps changes
* rm newline
* revert proto changes
* rename bls12 to herumi for clarity
2020-06-25 00:47:51 +00:00
terence tsao
d0e2e0e979 Best practice feedback - part 1 (#6385)
* log error for metric
* reuse epoch e
* Better node comments
* Remove not needed breaks
* Use j over i
* Merge branch 'master' of github.com:prysmaticlabs/prysm into best-practice-pt1
* Descendent/Descendant
* Merge refs/heads/master into best-practice-pt1
* Update beacon-chain/forkchoice/protoarray/nodes.go
* Merge refs/heads/master into best-practice-pt1
* Merge refs/heads/master into best-practice-pt1
2020-06-24 21:09:47 +00:00
Raul Jordan
ac77a5c054 QSP-25 Only Enable Relay if Flag Provided (#6386)
* only add relay if flag is on
* Merge branch 'master' into qsp-25
* Merge refs/heads/master into qsp-25
* Merge refs/heads/master into qsp-25
2020-06-24 19:30:40 +00:00
terence tsao
a9c1d25a35 Best practice feedback - part 2 (#6389)
* Feedback 10. Rename to highest epoch
* Feedback 11. Rename to pidepoch
* Feedback 39. Swap length
* Typo
* Merge refs/heads/master into best-practice-pt2
2020-06-24 18:06:19 +00:00
Ivan Martinez
e27ed8174b Remove validator filename prefix from validator/client (#6387)
* Remove validator prefix from files
* Gaz
* Merge branch 'master' of github.com:prysmaticlabs/prysm into val-rename-files
2020-06-24 17:43:24 +00:00
Raul Jordan
bfce4c7f63 QSP-65 Clarify Why UnshuffleList is Used (#6381)
* clarify why unshuffle list
* Update beacon-chain/core/helpers/committee.go
* Merge branch 'master' into qsp-65
2020-06-24 16:57:09 +00:00
Nishant Das
045a983032 Add Altona Config (#6380) 2020-06-24 07:03:16 -07:00
Shay Zluf
b8e1bf0445 Fix proposer slashing context timeout test (#6379)
* Fix proposer slashing context timeout test
* goimports
* revert slashing change
* Merge branch 'master' into fix_proposer_slashing_test
2020-06-24 12:10:40 +00:00
Shay Zluf
edb08bd209 QPS-20 implement removeDisconnectedPeerStatus (#6370)
* QPS-20 implement removeDisconnectedPeerStatus
* Merge refs/heads/master into remove_disconnected_peer_status
* Merge refs/heads/master into remove_disconnected_peer_status
* Merge refs/heads/master into remove_disconnected_peer_status
* Merge refs/heads/master into remove_disconnected_peer_status
* Merge refs/heads/master into remove_disconnected_peer_status
* Merge refs/heads/master into remove_disconnected_peer_status
* Merge refs/heads/master into remove_disconnected_peer_status
* Merge refs/heads/master into remove_disconnected_peer_status
* Merge refs/heads/master into remove_disconnected_peer_status
* Merge refs/heads/master into remove_disconnected_peer_status
* Merge refs/heads/master into remove_disconnected_peer_status
* Merge refs/heads/master into remove_disconnected_peer_status
* remove function
* Merge branch 'remove_disconnected_peer_status' of github.com:prysmaticlabs/prysm into remove_disconnected_peer_status
* Merge refs/heads/master into remove_disconnected_peer_status
* Merge branch 'remove_disconnected_peer_status' of github.com:prysmaticlabs/prysm into remove_disconnected_peer_status
2020-06-24 10:40:33 +00:00
Nishant Das
e45a06cbf1 Revert "QSP-42 Remove Double Unsubscribe in Initial Sync" (#6376)
* Revert "QSP-42 Remove Double Unsubscribe in Initial Sync (#6368)"

This reverts commit 637354f037.

* comment

* add go fmt
2020-06-24 17:01:48 +08:00
Ivan Martinez
e9a597b43c [QSP-6] Change usages of math/rand to crypto/rand (#6362)
* Change math/rand to crypto/rand
* Change length
* Merge branch 'master' of github.com:prysmaticlabs/prysm into change-to-crypto-rand
* Change assignments to true random
* Merge branch 'master' into change-to-crypto-rand
* Change randomness
* Merge branch 'change-to-crypto-rand' of github.com:prysmaticlabs/prysm into change-to-crypto-rand
* Remove comments
* Test
* Fix test
* Merge refs/heads/master into change-to-crypto-rand
* Merge refs/heads/master into change-to-crypto-rand
* Merge refs/heads/master into change-to-crypto-rand
* Merge refs/heads/master into change-to-crypto-rand
* Merge refs/heads/master into change-to-crypto-rand
* Merge refs/heads/master into change-to-crypto-rand
* Merge refs/heads/master into change-to-crypto-rand
* Merge refs/heads/master into change-to-crypto-rand
* Merge refs/heads/master into change-to-crypto-rand
* Merge refs/heads/master into change-to-crypto-rand
* Merge refs/heads/master into change-to-crypto-rand
2020-06-24 05:47:51 +00:00
terence tsao
a4cfd147ff Address spec adherence feedbacks (#6365)
* Addressed 1
* Address 2.
* Addressed 3. and 4.
* Addressed 6.
* Addressed 7
* Addressed 8
* Addressed 9.
* Addressed 10.
* Addressed 11.
* Addressed 12.
* Addressed 13.
* Delete old test
* Merge refs/heads/master into spec-feedbacks
* Change comment "pure" to "stateless"
* Fmt
* Typo
* Merge refs/heads/master into spec-feedbacks
* Merge refs/heads/master into spec-feedbacks
* Merge refs/heads/master into spec-feedbacks
* Merge refs/heads/master into spec-feedbacks
* Merge refs/heads/master into spec-feedbacks
* Merge refs/heads/master into spec-feedbacks
* Merge refs/heads/master into spec-feedbacks
* Merge refs/heads/master into spec-feedbacks
* Merge refs/heads/master into spec-feedbacks
* Merge refs/heads/master into spec-feedbacks
2020-06-24 03:14:44 +00:00
Preston Van Loon
7662045cc6 Disable --enable-stream-duties for e2e tests (#6374)
* Disable --enable-stream-duties for e2e tests
* Merge branch 'master' into fix-e2e
* Merge branch 'master' into fix-e2e
* Merge branch 'master' into fix-e2e
2020-06-24 02:42:38 +00:00
Ivan Martinez
9fadb4f334 Address best practices in validator client (#6375)
* Best practices validator

* Add ctx

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-24 09:53:01 +08:00
Nishant Das
e9057185a5 Fix Peering Issues in Discovery (#6346)
* add current changes
* comment
* Merge branch 'master' into fixPeeringIssues
* Merge refs/heads/master into fixPeeringIssues
* Merge refs/heads/master into fixPeeringIssues
* Merge refs/heads/master into fixPeeringIssues
* clean up
* add comment
* Merge refs/heads/master into fixPeeringIssues
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into fixPeeringIssues
* Merge refs/heads/master into fixPeeringIssues
* Merge refs/heads/master into fixPeeringIssues
* Merge refs/heads/master into fixPeeringIssues
* Merge refs/heads/master into fixPeeringIssues
* Merge refs/heads/master into fixPeeringIssues
* Merge refs/heads/master into fixPeeringIssues
* Merge refs/heads/master into fixPeeringIssues
* Merge refs/heads/master into fixPeeringIssues
* Merge refs/heads/master into fixPeeringIssues
* Merge refs/heads/master into fixPeeringIssues
* Merge refs/heads/master into fixPeeringIssues
* Merge refs/heads/master into fixPeeringIssues
* Merge refs/heads/master into fixPeeringIssues
* Merge refs/heads/master into fixPeeringIssues
* Merge refs/heads/master into fixPeeringIssues
* Merge refs/heads/master into fixPeeringIssues
* Merge refs/heads/master into fixPeeringIssues
2020-06-24 01:09:09 +00:00
Nishant Das
24d09f083f Release State Reference Copy (#6341)
* release flag
* add comment
* improve
* Merge refs/heads/master into releaseStateRef
* Merge refs/heads/master into releaseStateRef
* Merge branch 'master' into releaseStateRef
* Merge refs/heads/master into releaseStateRef
* Merge refs/heads/master into releaseStateRef
* Merge refs/heads/master into releaseStateRef
* Merge refs/heads/master into releaseStateRef
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into releaseStateRef
* remove old flags
* gaz
* Merge refs/heads/master into releaseStateRef
* Merge refs/heads/master into releaseStateRef
* Merge refs/heads/master into releaseStateRef
* Merge refs/heads/master into releaseStateRef
* Merge refs/heads/master into releaseStateRef
* Merge refs/heads/master into releaseStateRef
* Merge refs/heads/master into releaseStateRef
2020-06-24 00:24:59 +00:00
Raul Jordan
637354f037 QSP-42 Remove Double Unsubscribe in Initial Sync (#6368)
* rem double unsub
* Update beacon-chain/sync/initial-sync/service.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/master into qsp-unsub
* Merge refs/heads/master into qsp-unsub
* Merge refs/heads/master into qsp-unsub
* Merge refs/heads/master into qsp-unsub
* Merge refs/heads/master into qsp-unsub
2020-06-23 23:50:52 +00:00
terence tsao
c3adde3b32 QSP10, QSP11 - Add proper locking (#6350)
* Proper locking based on audit feedbacks
* Merge refs/heads/master into qsp10-11
* Merge refs/heads/master into qsp10-11
* Preston's feedback
* Merge branch 'qsp10-11' of github.com:prysmaticlabs/prysm into qsp10-11
* Unlock in invalid index branch
* Merge refs/heads/master into qsp10-11
* Merge refs/heads/master into qsp10-11
* Merge refs/heads/master into qsp10-11
* Merge refs/heads/master into qsp10-11
* Merge refs/heads/master into qsp10-11
* Merge refs/heads/master into qsp10-11
* Merge refs/heads/master into qsp10-11
* Merge refs/heads/master into qsp10-11
* Merge refs/heads/master into qsp10-11
* Merge refs/heads/master into qsp10-11
* Merge refs/heads/master into qsp10-11
* Merge refs/heads/master into qsp10-11
* Merge branch 'master' into qsp10-11
* Merge refs/heads/master into qsp10-11
* Merge refs/heads/master into qsp10-11
* Merge refs/heads/master into qsp10-11
* Merge refs/heads/master into qsp10-11
2020-06-23 23:08:08 +00:00
Raul Jordan
2bdb825c04 QSP-61 Clarify Why Process Pending Attestations May Exit Early (#6371)
* clarify with comment
* Merge refs/heads/master into qsp-61
* Merge refs/heads/master into qsp-61
* Merge refs/heads/master into qsp-61
2020-06-23 22:34:31 +00:00
Raul Jordan
7ba26aa433 Add Mutex to Hot State Cache (#6366)
* mutex to hot state cache
* Merge branch 'master' into add-hotstate-lock
* Merge refs/heads/master into add-hotstate-lock
* Merge refs/heads/master into add-hotstate-lock
2020-06-23 22:06:47 +00:00
terence tsao
374a84d577 Remove new state mgmt within db package (#6373)
* Remove new state mgmt within db package
* Merge refs/heads/master into rm-new-state-db
2020-06-23 21:36:21 +00:00
Raul Jordan
e1310adb4c QSP-Docs All Documentation Improvements from Quantstamp Audit Report (#6367)
* docs improvements so far
* all misc docs items
* finalized
* Merge branch 'master' into qsp-docs
* fix broken test
* Merge branch 'master' into qsp-docs
2020-06-23 21:11:20 +00:00
Preston Van Loon
f9038674d1 Refactor setupDB to return the state summary cache used by the database (#6369) 2020-06-23 13:40:55 -07:00
Victor Farazdagi
33b2636bf1 Enables new aggregation implementation (#6344)
* adds feature flag
* aggregations/attestation package
* better tests
* bazel visibility issues
* removes redundant code
* Merge branch 'master' into aggregation-attestations-package
* updates previous implementation
* Merge branch 'master' into aggregation-enable-new-aggregation
* resolve merge conflict
* resolve merge conflict (more)
* Merge branch 'master' into aggregation-enable-new-aggregation
* force max_cover aggregation on --dev
* Merge branch 'aggregation-enable-new-aggregation' of github.com:prysmaticlabs/prysm into aggregation-enable-new-aggregation
* Merge branch 'master' into aggregation-enable-new-aggregation
* Merge refs/heads/master into aggregation-enable-new-aggregation
* Merge refs/heads/master into aggregation-enable-new-aggregation
* Merge refs/heads/master into aggregation-enable-new-aggregation
* Merge refs/heads/master into aggregation-enable-new-aggregation
* Merge refs/heads/master into aggregation-enable-new-aggregation
2020-06-23 19:13:55 +00:00
Shay Zluf
96a110a193 External slashing protection not requiring signature (#6252)
* validation without signature
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* validation and update funcs
* Merge branch 'slashing_protection_no_sign' of github.com:prysmaticlabs/prysm into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge branch 'master' of github.com:prysmaticlabs/prysm into slashing_protection_no_sign
* Merge branch 'slashing_protection_no_sign' of github.com:prysmaticlabs/prysm into slashing_protection_no_sign
* change order
error handling
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* ivan feedback
* Merge branch 'slashing_protection_no_sign' of github.com:prysmaticlabs/prysm into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* add tests to blocks utils
* terence feedback
* reduce visibility
* Merge branch 'master' of github.com:prysmaticlabs/prysm into slashing_protection_no_sign

# Conflicts:
#	validator/client/polling/validator_attest.go
#	validator/client/polling/validator_propose.go
* fix metrics
* fix error
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* copy behaviour to streaming
* Merge branch 'slashing_protection_no_sign' of github.com:prysmaticlabs/prysm into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
* Merge refs/heads/master into slashing_protection_no_sign
2020-06-23 16:46:48 +00:00
terence tsao
c417b00675 Fix cold state replay to start at correct slot (#6361)
* Fix off by 1
* Regression tests
* Merge branch 'master' of github.com:prysmaticlabs/prysm into fix-offset
* Go fmt
* Merge refs/heads/master into fix-offset
* Merge refs/heads/master into fix-offset
2020-06-23 16:04:42 +00:00
Ivan Martinez
81786159e9 Move permanent globally needed flags to shared/cmd and makes them accessible (#6312)
* Move permanent flags to shared/cmd and make them accessible
* Merge branch 'master' of github.com:prysmaticlabs/prysm into move-flags-to-cmd
* Add test
* Add comment
* Fix gaz
* Replace with new config
* Gaz
* replace
* gaz
* Merge branch 'master' into move-flags-to-cmd
* Merge refs/heads/master into move-flags-to-cmd
* Prestons comments
* Nishant comments
* Merge branch 'move-flags-to-cmd' of github.com:prysmaticlabs/prysm into move-flags-to-cmd
* Merge refs/heads/master into move-flags-to-cmd
* Merge refs/heads/master into move-flags-to-cmd
* Merge refs/heads/master into move-flags-to-cmd
* Merge refs/heads/master into move-flags-to-cmd
* Merge refs/heads/master into move-flags-to-cmd
* Merge refs/heads/master into move-flags-to-cmd
* Merge refs/heads/master into move-flags-to-cmd
* Merge refs/heads/master into move-flags-to-cmd
* Merge refs/heads/master into move-flags-to-cmd
* Merge refs/heads/master into move-flags-to-cmd
* Merge refs/heads/master into move-flags-to-cmd
* Merge branch 'master' into move-flags-to-cmd
* Merge refs/heads/master into move-flags-to-cmd
* fmt
2020-06-23 15:41:20 +00:00
Anton
fe6cf9f33b SubscribeCommitteeSubnets fields length check (#6359)
* SubscribeCommitteeSubnets fields length check

A case where `len(req.Slots) != len(req.CommitteeIds)` but `len(req.CommitteeIds) == len(req.IsAggregator)`
should also throw an error. Without this fix only 2 length inequalities will throw the error.

Greetings from PwC Switzerland
* Added diff length SubscribeCommitteeSubnets test
* Merge branch 'master' into patch-1
2020-06-23 14:50:46 +00:00
Victor Farazdagi
078c157f56 Resolves issues from the best practices in beacon-chain/db (#6358)
* fixes 51
* fixes 52
* fixes 53
* fixes 54
* fixes 56
* Merge branch 'master' into db-best-practices
* fixes 62
* fixes 64
* fixes 65
* fixes 68
* fixes 69
2020-06-23 13:51:00 +00:00
Victor Farazdagi
22b3123ebe Resolves issues from the best practices in sync package (#6357)
* removes redundant locks
* fixes 37
* fixes 38
* reverts lock removal
* Merge branch 'master' into sync-best-practices-updates
* Merge refs/heads/master into sync-best-practices-updates
2020-06-23 12:00:29 +00:00
Victor Farazdagi
64b0f775d3 Fixes receiver name in db/kv (#6356)
* Fixes receinver name in db/kv
* Merge refs/heads/master into fix-receiver-name
* Merge refs/heads/master into fix-receiver-name
2020-06-23 10:15:01 +00:00
Victor Farazdagi
89ef3cfdc6 Fixes error message wording (#6355)
* fixes error message wording
* Merge refs/heads/master into fix-error-msg-wording
2020-06-23 09:44:39 +00:00
Nishant Das
36c82b26e4 Minor Improvements (#6353)
* minor improvements
* Update beacon-chain/sync/deadlines.go
* go fmt
2020-06-23 09:00:09 +00:00
Victor Farazdagi
5c90038007 Adds aggregation/attestations package (#6343)
* adds feature flag

* aggregations/attestation package

* better tests

* bazel visibility issues

* removes redundant code

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-22 23:00:38 -05:00
Raul Jordan
71a6c73e93 Remove All Instances of Kademlia from Prysm (#6323)
* fully remove kad
* Merge branch 'master' into rem-kad
* Update beacon-chain/p2p/service.go
* no workspace kad
* Merge branch 'rem-kad' of github.com:prysmaticlabs/prysm into rem-kad
* mod tidy
* Merge refs/heads/master into rem-kad
* Merge refs/heads/master into rem-kad
* Merge refs/heads/master into rem-kad
* add back bootnode query tool
* Merge branch 'rem-kad' of github.com:prysmaticlabs/prysm into rem-kad
* fix up multiaddr tests
* fix build
* fix up go build
* fix go build
* update dep
* Merge refs/heads/master into rem-kad
* Merge refs/heads/master into rem-kad
* Merge refs/heads/master into rem-kad
* Merge refs/heads/master into rem-kad
2020-06-23 02:53:11 +00:00
terence tsao
5c8da7a1c4 QSP58, QSP59 - Fix DB spans (#6352)
* Fix spans
* Merge refs/heads/master into qsp58-59
2020-06-23 02:22:22 +00:00
terence tsao
368af7e53f Clean up old block processing paths (#6311)
* Remove old block processing paths

* Go fmt

* Go fmt again

* Fix tests

* Using the wright receiver

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-06-22 17:19:33 -07:00
Victor Farazdagi
41c8ff4c1b Renames receiver in sync service (#6348)
* updates coinciding vars
* renames receiver in sync service
* Merge branch 'master' into sync-rename-receiver
2020-06-22 20:37:48 +00:00
terence tsao
c837dfb2f2 ActivationEligibilityEpoch nil case (#6347) 2020-06-22 14:41:21 -05:00
Nishant Das
3b2dc8346d Release State Field Trie (#6340)
* remove flag and references
* fix flag
* Merge refs/heads/master into releaseStateFieldTrie
* Merge refs/heads/master into releaseStateFieldTrie
2020-06-22 15:25:55 +00:00
Ivan Martinez
97315c8837 Add flat spans to slasher runtime (#6287)
* Add flat spans to runtime

* Fix tests

* Remove normal span cache from runtime

* Uncomment

* beyond lookback to db

* Remove heavy span

* better cache handling for disable lookback

* Fix lint

* Fix lint again

* Add cache back for now

* Update slasher/detection/attestations/spanner_test.go

* Fix imports

Co-authored-by: shayzluf <thezluf@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-06-22 09:55:52 -05:00
Victor Farazdagi
6f8349cdb4 Reclaims leakybucket resources in sync service (#6339)
* Reclaims leakybucket resources
* move calls to defer
* do not reuse queue, after stopping
2020-06-22 11:23:23 +00:00
Preston Van Loon
d4545233cd Fix race condition issues in beacon state (#6322)
* Fix race condition issues in beacon state
* Merge branch 'master' of github.com:prysmaticlabs/prysm into race-condition-fixes
* Checkout beacon-chain/state/stateutil/BUILD.bazel from master
* @nisdas PR feedback. defer unlock
2020-06-20 00:27:23 +00:00
rkapka
4f4846794a renamed variable (#6319) 2020-06-19 13:46:18 -07:00
Nishant Das
7e1a61f8eb Add Debug Endpoints For Peers (#6304)
* add debug endpoints

* gaz

* add more stuff

* add tests

* clean up

* Update beacon-chain/rpc/debug/server.go

* fmt

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-19 13:25:47 -05:00
Patrick
624209eba7 Fix Pagination Bug When Total Size % Page Size == 0 (#6266)
* Add failing test for pagination

* Fix failing test for pagination

* Fix other tests for pagination

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-06-19 09:39:12 -05:00
terence tsao
394f3ab53a Revert "Fetch deposit data only up to canonical Eth1 height when proposing (#6309)" (#6313)
This reverts commit fb26b0b65c.
2020-06-18 16:56:38 -07:00
rkapka
fb26b0b65c Fetch deposit data only up to canonical Eth1 height when proposing (#6309)
* get data up to canonical block height

* gofmt

* simplified calculation of canonical height
2020-06-18 17:31:26 -05:00
Victor Farazdagi
90542c21dc Implements max k-coverage greedy algorithm (#6305)
* implements max k-coverage greedy algorithm
* updates go-bitfield dependency
* gazelle
* update base aggregate
* re-arrange to shared
* clean references to atts in max cover
* max_cover: updates visibility
* fixes tests
* attestations related methods
* Merge branch 'master' into attaggregation-max-cover
* better op order
* fix comments
* removes debug stringer methods
* Merge refs/heads/master into attaggregation-max-cover
* log random seed
* Merge branch 'attaggregation-max-cover' of github.com:prysmaticlabs/prysm into attaggregation-max-cover
* Merge refs/heads/master into attaggregation-max-cover
* adds more comments
* Merge branch 'attaggregation-max-cover' of github.com:prysmaticlabs/prysm into attaggregation-max-cover
* fixes typo
2020-06-18 20:56:23 +00:00
Preston Van Loon
6926592065 Add a debug log statement when enabling dev flags (#6306)
* Add a debug log statement when enabling dev flags
* Merge refs/heads/master into add-dbg-log-for-dev-flags
2020-06-18 20:25:34 +00:00
Preston Van Loon
addc3897c4 Use STABLE_ prefix for git tag to force a relink of version (#6307)
* Use STABLE_ prefix for git tag to force a relink of version
* Add comment about STABLE_
2020-06-18 19:59:47 +00:00
Raul Jordan
7067c84c69 Stream Duties Client Implementation (#5867)
* include validator client stream

* Update validator/client/validator_attest.go

* gazelle

* rem extraneous logs

* fixing tests

* resolve most tests

* gaz

* add lock

* ivan feedback

* pass tests for update protect

* gaz

* duties gaz

* no need for canonical head slot

* fix ctx leak

* fmt

* add in feature flag

* add streaming subpackage

* add polling/streaming separation

* able to build

* fix duplicate package names

* fix polling

* imports

* confirm it works

* fixed up comment

* go lint comments

* gaz

* build

* Update validator/client/streaming/service_test.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* tidy

* fmt

* add stream duties to e2e

* add stream duties to e2e flags

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-06-18 13:30:05 -05:00
Jim McDonald
10af753f59 Unlock mutex on return (#6303)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-06-18 11:04:08 -05:00
Nishant Das
5201403750 update herumi (#6301)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-18 09:54:55 -05:00
Jim McDonald
86e9b0ae51 Ping handler timeout (#6302)
* Fix goroutine using cancelled context.

* Only close stream when finished

* Simplify function structure
2020-06-18 19:59:18 +08:00
Nishant Das
340ddf20cb Update Node RPC Endpoints (#6289)
* checkpoint progress
* some more stuff
* fix tests
* fix tests
* lint
* fix test and initialization
* gaz
* Merge branch 'master' into updateP2PRPC
* Merge refs/heads/master into updateP2PRPC
* Merge refs/heads/master into updateP2PRPC
* Merge refs/heads/master into updateP2PRPC
* Merge refs/heads/master into updateP2PRPC
* Merge refs/heads/master into updateP2PRPC
* Merge refs/heads/master into updateP2PRPC
* Merge refs/heads/master into updateP2PRPC
* Merge refs/heads/master into updateP2PRPC
* Merge refs/heads/master into updateP2PRPC
2020-06-18 03:53:46 +00:00
Preston Van Loon
da5ab499da Add @prysmaticlabs/core-team as codeowner for deps.bzl (#6296)
* Add @prysmaticlabs/core-team as codeowner for deps.bzl
* Update CODEOWNERS
2020-06-18 03:38:02 +00:00
Paul Nelson
c1c3b75867 Move generic HashTreeRoot funcs to //shared (#6167)
* Functions that are to be moved to //shared/htrutils captured in separate file.

* Move files into new //shared/htutils package

hash_function.go renamed to hashers.go

* Refactor to reference moved methods in new package

- Added import for htrutils to state and stateutil packages
- Update references to imported funcs (append "htrutils.")
- Updated funcs in htrutils to be exported where necessary

* Add tests
2020-06-18 10:15:13 +08:00
terence tsao
640bba8a6c Avoid active validator count cache for genesis (#6292)
* Add helper to prevent zero hashes

* Test

* Don't use cache if it's genesis

* Regression test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-17 19:48:10 -05:00
terence tsao
6781bd643b Fix getting att pre state in sync (#6295) 2020-06-17 16:44:43 -07:00
terence tsao
802bdf5561 Fix GetValidatorPerformance before/after balances (#6293)
* Add helper to prevent zero hashes

* Test

* Fix precompute validators to have before/after balances

* Update tests to use correct values

* Even better test setups

* Gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-17 13:46:42 -05:00
terence tsao
523fc621d9 First part of block chain service cleanup (#6286)
* Add helper to prevent zero hashes
* Test
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Remove ReceiveBlockNoPubsubForkchoice and rename ReceiveBlockInitialSync
* Apply feature flag to process_block.go
* Update mock and tests
* Update beacon-chain/blockchain/process_block.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/master into bc-pkg-cleanup
* Merge refs/heads/master into bc-pkg-cleanup
* Fix mock ReceiveBlockInitialSync. Thanks Victor!
* Merge branch 'bc-pkg-cleanup' of github.com:prysmaticlabs/prysm into bc-pkg-cleanup
* Merge refs/heads/master into bc-pkg-cleanup
* Merge refs/heads/master into bc-pkg-cleanup
2020-06-17 17:11:21 +00:00
Shay Zluf
472fbd3949 Less restrictive block stream for slashing detection (#6269)
* verify sig in stream

* state cache used

* fix import

* fix import

* fix tests

* add log for proposer slashing

* fix e2e

* use chain head to verify sig

* revert fix

* remove extra line

* add line

* revert extra log

* fix tests

* terence feedback

* fix

* fix test

* remove extra setup

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-17 12:38:07 -04:00
Ivan Martinez
cd439adfc7 Implement flat spans cache into DB (#6248)
* Implement flat spans cache DB

* Fix interface

* Fix db func

* Gaz

* Update slasher/detection/attestations/types/epoch_store_test.go

* Fix tests

* Fix

* Fix for comments

* Fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-06-17 10:26:08 -05:00
Raul Jordan
0bfa1ecd03 Clarify Insecure gRPC Connection Logs (#6276)
* clarify insecure conn
* fix up broken test
* Merge refs/heads/master into amend-grpc-instructions
* Merge refs/heads/master into amend-grpc-instructions
* Merge refs/heads/master into amend-grpc-instructions
* Merge refs/heads/master into amend-grpc-instructions
* Merge refs/heads/master into amend-grpc-instructions
* Merge refs/heads/master into amend-grpc-instructions
* Merge refs/heads/master into amend-grpc-instructions
2020-06-17 14:32:13 +00:00
terence tsao
af3122a9e8 Fork choice avoids redundant call to get_ancestor (#6282)
* Add helper to prevent zero hashes

* Test

* Add cacheJustifiedStateBalances

* Cache justifed balances

* Use cached justified balances

* Remove unncessary justification checks during initial syncing

* Test

* Proper locking

* Use getter

* Add CheckPointIsEqual helper

* Test CheckPointIsEqual helper

* Update finalizedImpliesNewJustified

* Better tests for finalizedImpliesNewJustified

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-17 08:50:10 -05:00
Preston Van Loon
4740f7ed57 pre-allocate memory for AggregateVerify (#6285)
* pre-allocate memory for AggregateVerify
2020-06-16 23:21:31 +00:00
Preston Van Loon
213be4cfbe Verify many block attestations at once (#6277) 2020-06-16 14:19:10 -07:00
Preston Van Loon
5fa3bbb991 Add --initial-sync-verify-all-signat as a dev flag. Issue #5176. (#6283) 2020-06-16 11:06:32 -07:00
terence tsao
fe2f8a1c04 Use justified balances for fork choice (#6272) 2020-06-16 08:10:34 -07:00
Nishant Das
ed883c9fff More Fixes For Powchain (#6278)
* fix issues

* add check

* Update beacon-chain/powchain/service.go

* Update beacon-chain/powchain/service.go

* Update beacon-chain/powchain/service.go

* Update beacon-chain/powchain/service.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-06-15 21:07:20 -05:00
terence tsao
4c143f4ed4 Fix underflow in resyncIfBehind (#6274) 2020-06-15 16:46:36 -07:00
Preston Van Loon
8097eea607 Fix underflow in SlotsSinceGenesis (#6275)
* Fix an underflow in genesis time calculation
* goimports
2020-06-15 21:56:11 +00:00
Ivan Martinez
4b4a7459d9 Fix windows DB path error and make errors more concise (#6270)
* Make DB errors more verbose, fix windows path error
* Merge branch 'master' into windows-err
* Merge refs/heads/master into windows-err
* Merge refs/heads/master into windows-err
2020-06-15 20:00:35 +00:00
Ivan Martinez
bf548d1670 Fix validator creation path (#6271)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-15 12:27:45 -07:00
terence tsao
e59ee2f203 Fix validator summary reporting (#6267)
* Add helper to prevent zero hashes
* Test
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Update validator summary
* Update existing tests to work with new indices
* Remove extra log
* Gazelle
* Merge branch 'master' into fix-validatory-summary
* Merge refs/heads/master into fix-validatory-summary
* Merge refs/heads/master into fix-validatory-summary
2020-06-15 17:25:58 +00:00
Raul Jordan
83d1aa8fa2 Default BeaconRPCProvider for Validator Should Match Beacon Node's Host (#6268)
* default bind to localhost for validator
* Merge branch 'master' into validator-localhost
2020-06-15 16:52:15 +00:00
Victor Farazdagi
9d42148b93 Go mod tidy (#6264)
* updates go.sum
* go mod tidy checker
* reverts go.sum
* updates apk command
* fixes path
* show git status
* more debug info
* more debug info
* minor test
* get rid of git
* multiple checks
* test reset
* change order of checkers
* mod verify
* broken sum
* Merge refs/heads/master into go-mod-tidy
* better info
* Merge branch 'go-mod-tidy' of github.com:prysmaticlabs/prysm into go-mod-tidy
* better info
* reset
* run go mod tidy to fix
2020-06-15 16:15:54 +00:00
Nishant Das
87084ec9f1 fix issues (#6265) 2020-06-15 10:43:45 -05:00
Victor Farazdagi
ebd45ba336 updates go.sum (#6262) 2020-06-15 06:37:07 -07:00
Nishant Das
9d8420b20c Fix Discovery Panic (#6261)
* fix dv5 panic
* add back
2020-06-15 04:22:29 +00:00
terence tsao
a9070ad725 ActiveValidatorCount to use cache (#6254)
* Add helper to prevent zero hashes

* Test

* Add ActiveIndicesCount getter for committee cache

* ActiveIndicesCount test

* ActiveValidatorCount to use cache

* Update cache on miss

* Update tests
2020-06-14 15:24:49 -07:00
terence tsao
f4e9e2f49c Minor clean up to 6249 (#6253)
* Add helper to prevent zero hashes

* Test

* Comments

* Comments
2020-06-14 10:43:25 -07:00
Nishant Das
1dfeb645b6 Filter Peers Properly With Connection Gater (#6251)
* add new filtering
* add new tests
* gaz
* shay's review
* Merge branch 'master' into fixFiltering
2020-06-14 14:47:58 +00:00
Nishant Das
933ab6808b mark peer as bad (#6250) 2020-06-14 07:10:18 -07:00
Nishant Das
f9d80e4a28 Fix Genesis Justified Checkpoints (#6249)
* handle zero hash
2020-06-14 09:02:56 +00:00
Ivan Martinez
f6ecf66d1a Fix pubkey used in validator metrics (#6246)
* Fix validator metrics pubkey
* Merge branch 'master' into val-metrics
2020-06-14 08:28:26 +00:00
Nishant Das
caf61bd824 Use Connection Gater to Manage Peer Connections (#6243)
* use gater
* Merge branch 'master' into connGater
* Update beacon-chain/p2p/testing/p2p.go
* fmt
* gaz
2020-06-14 07:35:05 +00:00
terence tsao
87ba5a5993 Fix ancestor to return most recent root in the case of skip slot (#6242)
* Revert "remove excess copies (#6142)"

This reverts commit c956905cf0.
* Update ancestor function to handle skip slot
* Test
* Revert "Revert "remove excess copies (#6142)""

This reverts commit f1222b5e77.
* Merge refs/heads/master into fix-ancestor
2020-06-14 06:58:36 +00:00
Nishant Das
0067e52b1e Revert "Remove Excess State Copies" (#6240)
* Revert "remove excess copies (#6142)"

This reverts commit c956905cf0.
2020-06-14 06:34:33 +00:00
Shay Zluf
4c66edf2c2 Add cache for flan spans (#6199)
* flat span cache

* Plug cache into kv

* Add comments

* Fix

* Add doc.go

* Gaz

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-13 22:51:54 -04:00
terence tsao
c29cccf78e Don't regen parent state for genesis state (#6237) 2020-06-13 11:04:04 -07:00
Raul Jordan
298955c92b Add a Log for Newly Connected gRPC Clients in the Beacon Node (#6233)
* interceptor disable logging via feature flag
* Merge branch 'master' into validator-connection-logs
2020-06-12 20:41:05 +00:00
Raul Jordan
de45a54991 Do Not Ignore Max Gateway Size gRPC (#6232)
* do not ignore max message size in gateway
* fix build
2020-06-12 20:02:20 +00:00
Raul Jordan
87ca73d605 Countdown Until Genesis (#6231)
* added in logic
* latest countdown
* fixed up formatting and add tests
* ready for review
* Merge branch 'master' into countdown-genesis
* no log if after genesis
* Merge branch 'countdown-genesis' of github.com:prysmaticlabs/prysm into countdown-genesis
* countdown
* smarter logging
* added buffer period for countdown
* Merge refs/heads/master into countdown-genesis
2020-06-12 18:47:18 +00:00
terence tsao
bc76c95d62 Skip proposer duty calculation for slot 0 (#6228)
* Skip proposer assignment for slot 0

* Test

* Clear cache before test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-12 11:20:50 -07:00
Preston Van Loon
c461c72777 Add dockerhub image endpoints (#6230)
* Add dockerhub image endpoints
2020-06-12 17:06:12 +00:00
Nishant Das
c725a9c8f2 Sort Duplicate Blocks In Response (#6222)
* sort out new blocks
* add unit test
* Merge refs/heads/master into sortDuplicates
* victor's review
* Merge branch 'sortDuplicates' of https://github.com/prysmaticlabs/geth-sharding into sortDuplicates
* Merge refs/heads/master into sortDuplicates
2020-06-12 13:50:07 +00:00
Nishant Das
412da8e7cc Follow Up to #6225 (#6226)
* don't wait
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into dontWait
* revert some changes
* bring back
2020-06-12 13:23:29 +00:00
Nishant Das
82f4d6a14a Do Not Wait Till Genesis (#6225)
* don't wait
2020-06-12 11:44:04 +00:00
Preston Van Loon
89762492c8 Reduce JSON-RPC queries (#6221)
* Delete unused client property
* Only update the logs every 5 minutes
* Merge branch 'master' into reduce-json-rpc-calls
* Merge refs/heads/master into reduce-json-rpc-calls
* Merge refs/heads/master into reduce-json-rpc-calls
* PR feedback from @nisdas
* gofmt
* Merge branch 'reduce-json-rpc-calls' of github.com:prysmaticlabs/prysm into reduce-json-rpc-calls
2020-06-12 04:05:40 +00:00
Nishant Das
1a6edbbaa2 Fix Proposer Bug in Prysm (#6213)
* advance slots instead of setting slots
* Merge branch 'master' into fixProposerBug
* comment
* Merge branch 'fixProposerBug' of https://github.com/prysmaticlabs/geth-sharding into fixProposerBug
* Refactor `packAttestations` to utilize same advanced state
* Merge branch 'master' into fixProposerBug
* Merge branch 'master' into fixProposerBug
* Merge branch 'master' into fixProposerBug
* Merge refs/heads/master into fixProposerBug
* Merge refs/heads/master into fixProposerBug
* Update beacon-chain/rpc/validator/proposer.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/master into fixProposerBug
2020-06-11 23:05:10 +00:00
SjonHortensius
409b167899 Provide some feedback to user when creating an account (#6212)
* Provide some feedback to user when creating an account

on my 1y old server the first step took 2 minutes, the second took 4
minutes. Provide at lease _some_ feedback we are doing something
intensive
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge branch 'master' into master
2020-06-11 22:36:08 +00:00
Raul Jordan
abec538fda Prevent Whistleblower Index Bug (#6219)
* resolve proposer panic

* proposer is whistleblower

* import

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-11 15:08:26 -05:00
Shay Zluf
898cd8b42b No sig slasher rpc (#6174)
* detect slashable attestations and blocks without db update
* lint fixes
* fix mock
* Merge refs/heads/master into no_sig_slasher_rpc
* Merge refs/heads/master into no_sig_slasher_rpc
* preston feedback
* Merge branch 'no_sig_slasher_rpc' of github.com:prysmaticlabs/prysm into no_sig_slasher_rpc
* Merge refs/heads/master into no_sig_slasher_rpc
* Merge refs/heads/master into no_sig_slasher_rpc
* Merge refs/heads/master into no_sig_slasher_rpc
* Merge refs/heads/master into no_sig_slasher_rpc
* Merge refs/heads/master into no_sig_slasher_rpc
* Merge refs/heads/master into no_sig_slasher_rpc
* Merge refs/heads/master into no_sig_slasher_rpc
* Merge refs/heads/master into no_sig_slasher_rpc
* Merge refs/heads/master into no_sig_slasher_rpc
* Merge refs/heads/master into no_sig_slasher_rpc
* Merge refs/heads/master into no_sig_slasher_rpc
* Merge refs/heads/master into no_sig_slasher_rpc
* Merge refs/heads/master into no_sig_slasher_rpc
* Merge refs/heads/master into no_sig_slasher_rpc
* Merge refs/heads/master into no_sig_slasher_rpc
* Merge refs/heads/master into no_sig_slasher_rpc
* Merge refs/heads/master into no_sig_slasher_rpc
* Merge refs/heads/master into no_sig_slasher_rpc
* Merge refs/heads/master into no_sig_slasher_rpc
* Merge refs/heads/master into no_sig_slasher_rpc
* Merge refs/heads/master into no_sig_slasher_rpc
* Merge refs/heads/master into no_sig_slasher_rpc
* Merge refs/heads/master into no_sig_slasher_rpc
* nishant and ivan feedback
* Merge branch 'no_sig_slasher_rpc' of github.com:prysmaticlabs/prysm into no_sig_slasher_rpc
* Merge refs/heads/master into no_sig_slasher_rpc
* Merge refs/heads/master into no_sig_slasher_rpc
* Apply suggestions from code review
2020-06-11 18:50:12 +00:00
Ivan Martinez
ec6309a928 Cleanup flat spans implementation (#6150)
* Flat spanner improvements

* Fix tests, add more to ES type

* Move types to detection/types

* Fix

* Fix comments

* Fixes

* Use SlotTickerWithOffset for StreamIndexedAttestations (#5999)

* Change streamindexed to slot ticker with offset
* Make 2/3rds
* Add test
* Merge branch 'master' of github.com:prysmaticlabs/prysm into ticker-offset
* Add check for offset
* Merge branch 'master' of github.com:prysmaticlabs/prysm into ticker-offset
* Fix test
* Merge refs/heads/master into ticker-offset

* Fix long running E2E after v0.12 changes (#6008)

* Fix deposits in long run e2e

* Fix participation

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Small fixes for slasher  (#6021)

* Small fixes for slasher

* Remove useless log

* Fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Add voluntary exit processing to E2E (#6016)

* Add voluntary exit to E2E

* Fix long urnning e2e

* Fix for comments

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Change slashing pool to use spec code  (#6030)

* Change slashing pool to use more spec code

* Fix test

* Undo unneeded changes

* Make sure to catch regression

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Fix long-running E2E (#6047)

* Fixes for long running E2E
* Merge branch 'master' of github.com:prysmaticlabs/prysm into fix-e2e
* Move metrics check up
* Merge refs/heads/master into fix-e2e

* E2E Improvements (#6091)

* Some fixes
* Merge branch 'master' into e2e-fixes
* Add another small delay
* Merge branch 'e2e-fixes' of github.com:prysmaticlabs/prysm into e2e-fixes
* Remove genesis test, make normal e2e run longer
* Gaz
* more fixes
* Merge branch 'master' into e2e-fixes
* Merge refs/heads/master into e2e-fixes
* Fix comment
* Merge refs/heads/master into e2e-fixes

* Begin changing tests

* Start work on tests

* Redo tests for EpochStore

* Add test for highest index

* Fixes

* Gaz

* Fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-11 14:17:44 -04:00
Raul Jordan
2fb485598b Resolve Nil Validator Panic (#6218)
* resolve proposer panic
* Merge refs/heads/master into panic-proposer-slashed
2020-06-11 17:38:03 +00:00
terence tsao
cf3260b948 Revert "Send blocks that fails validation to slasher" (#6217)
* Revert "Send blocks that fail p2p validation to slasher (#6164)"

This reverts commit f40a7575de.
* Merge branch 'master' into revert-6164-slasherP2P
2020-06-11 17:11:28 +00:00
terence tsao
83242466f4 Fix proposer calculation in sync (#6214) 2020-06-11 09:31:01 -07:00
Nishant Das
7f3dc5a2dd Adds a User Agent for Libp2p (#6211)
* Add the latest git tag as part of the version string.

Example:

version=Prysm/v1.0.0-alpha.10/28e61fa40c7d16774b3b1c18d8382c64537bfa84. Built at: 2020-06-11 04:29:18+00:00

* add user agent

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-11 10:34:34 -05:00
Ivan Martinez
260847e92f Fix long running E2E deposits testing (#6208)
* Fix long running e2e deposit processing
* Merge branch 'master' of github.com:prysmaticlabs/prysm into fix-long-e2e
* Remove logs
* Uncomment
* typing
* Reorder
* Add const for node count
* Merge refs/heads/master into fix-long-e2e
* Add trace logging
* Merge branch 'fix-long-e2e' of github.com:prysmaticlabs/prysm into fix-long-e2e
2020-06-11 06:38:15 +00:00
Preston Van Loon
8815f6ced8 Add the latest git tag as part of the version string. (#6210)
Example:

version=Prysm/v1.0.0-alpha.10/28e61fa40c7d16774b3b1c18d8382c64537bfa84. Built at: 2020-06-11 04:29:18+00:00
2020-06-11 13:45:31 +08:00
Fabrice Cheng
28e61fa40c Verify External IP is reachable or log a warning if not (#6155)
* Only build with cgosymbolizer when explicitly desired. This fixes go builds for mac and others

* Try to dial external IP set and display a warning message is not reachable

* fix dialTimeout adding the port

* add connectivity check for UDP as well

* add connectivy verification for both UDP and TCP and move function in p2p/utils.go

* return if Resolving UDP fails

* move constant in utils

* add missing time dependencies

* leverage dial for both TCP/UDP (even thoguh UDP is not supported & add a unit tests for `verifyConnectivity`

* handle err from conn.Close()

* add utils_test go p2p gazelle

* Update beacon-chain/p2p/utils.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>

* Update beacon-chain/p2p/utils.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/p2p/utils.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* do not defer closing the connection

* leverage util helper testutil.AssertLogsContain and testutil.AssertLogsDoNotContain

* remove old code

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
2020-06-10 18:41:11 -05:00
terence tsao
edeb359ded Use SSZ HTR as baseline (#6207)
* Use ssz htr as baseline to compare
* Merge branch 'master' into fix-test
* One more. Fix for TestBlockBodyRoot_NilIsSameAsEmpty
* Merge branch 'fix-test' of github.com:prysmaticlabs/prysm into fix-test
2020-06-10 17:51:06 +00:00
Shay Zluf
f40a7575de Send blocks that fail p2p validation to slasher (#6164)
* send blocks that fails validation to block feed for slasher. in order to enable slashing proposers

* add sig validation

* fix imports

* Update shared/featureconfig/config.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>

* Update shared/featureconfig/config.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>

* Update shared/featureconfig/flags.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>

* remove extra validation path for slasher

* ivan feedback

* notify all blocks that pass minimal validation to block stream

* notify all blocks that pass minimal validation to block stream

* add to e2e flags

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-10 13:24:05 -04:00
Nishant Das
a9e3ea3ccc Add Host Flags for All Servers running in Prysm (#6202)
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into removeHardLimit

# Conflicts:
add flags for all hosts
* lint
* Merge refs/heads/master into addFlagsForHosts
* Merge refs/heads/master into addFlagsForHosts
* Merge refs/heads/master into addFlagsForHosts
* use single flag
2020-06-10 16:04:32 +00:00
Jim McDonald
61855bc596 Add block number to deposit log (#6200)
* Add block number to deposit log
* Update beacon-chain/powchain/log_processing.go

Co-authored-by: Nishant Das <nish1993@hotmail.com>
* Merge branch 'master' into log-deposit-block
* gofmt
* Merge branch 'master' into log-deposit-block
2020-06-10 14:31:58 +00:00
Nishant Das
d66570c72b change gateway port (#6203) 2020-06-10 06:56:39 -07:00
Victor Farazdagi
1ffd13c4f5 Disallows empty string as validator password (#6201)
* allows empty string as validator password
* disallow empty pass on validator creation
* reset
2020-06-10 13:12:19 +00:00
Preston Van Loon
89e1b0f6bb Add --random flag for unencrypted keygen (#6194)
* Add --random flag for unencrypted keygen
* gofmt
2020-06-10 07:13:47 +00:00
Preston Van Loon
ae4a8b3aeb Update RBE toolchain and image to have libgmp-dev with bazel 3.2.0 (#6193)
* Update RBE toolchain and image to have libgmp-dev with bazel 3.2.0
* Merge branch 'master' into update-cross-compile
* some fixes for herumi cross compile
* gaz
2020-06-10 04:08:02 +00:00
Preston Van Loon
1f20cbb3f0 Update bootnode ENR for Onyx (#6192)
* Update bootnode ENR for Onyx
* Merge refs/heads/master into update-bootnode
2020-06-10 01:38:27 +00:00
Nishant Das
8188fc0f9e Use Updated Connection Manager (#6184)
* introduce updated connection manager
* Merge branch 'master' into removeHardLimit
* remove forked connection manager
* Merge branch 'removeHardLimit' of https://github.com/prysmaticlabs/geth-sharding into removeHardLimit
* Merge branch 'master' into removeHardLimit
* Merge branch 'master' into removeHardLimit
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into removeHardLimit
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into removeHardLimit

# Conflicts:
tidy up
* Merge branch 'removeHardLimit' of https://github.com/prysmaticlabs/geth-sharding into removeHardLimit
2020-06-10 00:22:45 +00:00
Nishant Das
d1a1043ef1 Update to v0.12 (#5614) 2020-06-09 15:40:48 -07:00
Preston Van Loon
47dacb88f7 Make a simple json-rpc call to ensure we are connected to a real ETH1 node (#6191)
* Make a simple json-rpc call to ensure we are connected to a real node
* Merge refs/heads/master into check-connection
2020-06-09 21:26:15 +00:00
Victor Farazdagi
50af507407 Improves operations/attestations/kv test coverage (#6181)
* fixes span names
* TestKV_SaveUnaggregatedAttestation tests added
* TestKV_SaveUnaggregatedAttestations test
* adds count tests
* TestKV_Unaggregated_CanDelete
* fixes naming
* Merge branch 'master' into att-pool-kv-cleanup
* better naming
* Merge branch 'att-pool-kv-cleanup' of github.com:prysmaticlabs/prysm into att-pool-kv-cleanup
* naming
* TestKV_Aggregated_SaveUnaggregatedAttestation added
* Merge branch 'master' into att-pool-kv-cleanup
* naming
* updates TestKV_Aggregated_HasAggregatedAttestation
* updates delete tests
* fix order
* Merge branch 'master' into att-pool-kv-cleanup
2020-06-09 21:00:36 +00:00
Victor Farazdagi
187e08d38d Inverts enable-init-sync-wrr flag (#6187)
* Inverts enable-init-sync-wrr flag
2020-06-09 19:16:59 +00:00
terence tsao
568d8c7825 Invert reduce attester state copies (#6185)
* Invert cfg.ReduceAttesterStateCopy flag

* Fix build
2020-06-09 11:47:23 -05:00
Nishant Das
c956905cf0 remove excess copies (#6142)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-09 10:11:11 -05:00
Victor Farazdagi
bea2b3420e Fixes span naming in rpc/validator (#6182)
* fixes span names
2020-06-09 12:28:16 +00:00
terence tsao
0387641662 Add debug endpoint for individual vote status (#6166)
* Add IndividualVotesRequest protos

* Inplement GetIndividualVotes

* Tests

* Build

* Move from debug server to beacon server

* Remove protos

* Remove gen code

* Typo and build

* Gazelle

* Better mock

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-08 21:37:56 -05:00
Victor Farazdagi
ba63186c92 Init-sync queue assert order and uniqueness of blocks emitted (#6146)
* removes duplicate code
* Merge branch 'master' into init-sync-queue-dedup-blocks
* Merge refs/heads/master into init-sync-queue-dedup-blocks
* Merge refs/heads/master into init-sync-queue-dedup-blocks
* Merge refs/heads/master into init-sync-queue-dedup-blocks
* Merge refs/heads/master into init-sync-queue-dedup-blocks
* Merge refs/heads/master into init-sync-queue-dedup-blocks
* Merge refs/heads/master into init-sync-queue-dedup-blocks
* Merge refs/heads/master into init-sync-queue-dedup-blocks
* Merge refs/heads/master into init-sync-queue-dedup-blocks
* Merge branch 'master' into init-sync-queue-dedup-blocks
* removes notification
* Merge branch 'init-sync-queue-dedup-blocks' of github.com:prysmaticlabs/prysm into init-sync-queue-dedup-blocks
* adds test
* updates service
* get rid of deepequal for err check
* Merge refs/heads/master into init-sync-queue-dedup-blocks
2020-06-08 14:52:43 +00:00
Ivan Martinez
cb2813fa05 Force manual validator account creation (#6165)
* Force manual validator account creation

* Fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-08 09:03:14 -05:00
Preston Van Loon
8ffe9858ec Rename white/blacklist to allow/deny list (#6173)
* Rename white/blacklist to allow/deny list
* Deprecate flag properly
2020-06-08 04:35:15 +00:00
Preston Van Loon
323a272a80 Move dependencies to the deps.bzl file and out of the WORKSPACE (#6168)
* Move dependencies to the deps.bzl file and out of the WORKSPACE
2020-06-07 21:48:30 +00:00
terence tsao
113a49b61e Verify historical slots per archived point (#6159)
* Add check for slots point
* Test
* Gaz
2020-06-06 21:47:10 +00:00
Nishant Das
0b70c3ea85 Clean Up Discovery Filtering of Peers (#6128)
* clean up

* fix test

* rename

* context

* preston's review

* remove dep

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-05 22:46:36 -05:00
Nishant Das
a1e3fc9500 Fix Gossip Clock Disparity (#6141)
* fix disparity
* Merge refs/heads/master into fixGossipDisparity
* Merge refs/heads/master into fixGossipDisparity
* Merge refs/heads/master into fixGossipDisparity
* Merge refs/heads/master into fixGossipDisparity
* Merge refs/heads/master into fixGossipDisparity
* Merge refs/heads/master into fixGossipDisparity
2020-06-06 01:54:44 +00:00
Nishant Das
c06d61c3d0 fix issue (#6152) 2020-06-06 09:16:46 +08:00
terence tsao
6af7664a56 Check attestation has pre state before retrieving it (#6149)
* Use state util to get block root

* Move prestate to below

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-05 18:23:35 -05:00
Preston Van Loon
6c0fbcc622 Only build with cgosymbolizer when explicitly requested (#6151)
* Only build with cgosymbolizer when explicitly desired. This fixes go builds for mac and others
2020-06-05 22:21:43 +00:00
Raul Jordan
302b0f8c80 Use FastSSZ Everywhere Applicable (#6135)
* use fast ssz anywhere applicable
* use fastssz clearly
* Merge branch 'master' into use-fastssz
* Apply suggestions from code review
* imports
* Update beacon-chain/p2p/fork.go
* Merge branch 'master' into use-fastssz
* update go-ssz
* update go-ssz
* Merge refs/heads/master into use-fastssz
* Merge refs/heads/master into use-fastssz
2020-06-05 13:48:40 +00:00
Ivan Martinez
e37c9d1334 Log commit hash on slasher startup (#6138)
* Add version log to slasher
* Add slasher version log on startup
* Remove duplicate log
* Merge refs/heads/master into slasher-commit
* Fix tet
* Merge branch 'slasher-commit' of github.com:prysmaticlabs/prysm into slasher-commit
* Undo
* Fix log in rpc
* Merge refs/heads/master into slasher-commit
* Merge refs/heads/master into slasher-commit
* Merge refs/heads/master into slasher-commit
* Merge refs/heads/master into slasher-commit
2020-06-05 12:04:12 +00:00
Victor Farazdagi
fd3cb0be55 Re-enables multi-failure test case in round robin (#6143)
* re-enables multi-failure test case in round robin
2020-06-05 11:26:10 +00:00
Ivan Martinez
92f23df368 Fix inaccurate validator balance metrics (#6134)
* Fix balance metrics
* Reorder
* Add comment
* Merge refs/heads/master into fix-balance-metrics
* Add underflow check
* Merge refs/heads/master into fix-balance-metrics
* Merge refs/heads/master into fix-balance-metrics
* Merge refs/heads/master into fix-balance-metrics
2020-06-05 03:31:20 +00:00
Ivan Martinez
a4f4ab2b1a Use stateutil AttestationDataRoot instead of go-ssz (#6136)
* Change usages of ssz.HashTreeRoot(att.Data) to stateutil
* Gaz
* Tests
* Imports
* Merge branch 'master' into use-stateutil-attdata
* Fix tests
* Merge branch 'use-stateutil-attdata' of github.com:prysmaticlabs/prysm into use-stateutil-attdata
* Merge refs/heads/master into use-stateutil-attdata
* Merge refs/heads/master into use-stateutil-attdata
2020-06-05 01:32:08 +00:00
Ivan Martinez
edbeb9022b Remove unused bazel test target (#6140)
* Remove benc htarget
2020-06-05 01:04:11 +00:00
terence tsao
d3494045a2 Fix aggregator with unaggregated attestation (#6137)
* Use state util to get block root
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Add UnaggregatedAttestationsBySlotIndex
* Tests
* New TestSubmitAggregateAndProof_UnaggregateOk test
* Merge branch 'master' of github.com:prysmaticlabs/prysm into fix-aggregator-broadcast
2020-06-05 00:06:45 +00:00
Shay Zluf
1372391b77 Pretty pcli (#6099)
* new spanner db structure

* pretty print ssz structures in pcli

* nishant feedback

* remove from workspace

* dependency

* update names

* add to go.mod

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-04 16:08:24 -05:00
Jeevan
c3ff5ecb23 Notify users' with warning about using default ETH 1.0 RPC (#6126)
* added warning for default flag; http-web3provider
* Merge branch 'master' into warn-default-providers
* Update beacon-chain/node/node.go
* Merge branch 'master' into warn-default-providers
2020-06-04 19:40:29 +00:00
Ivan Martinez
1022d9dba8 Add AttestationDataRoot to stateutil (#6119)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-04 13:59:41 -05:00
Victor Farazdagi
53b0c0f7ab Adds gofmt checker action to GitHub workflow (#6132)
* adds gofmt checker action to GitHub workflow
* gofmt applied
2020-06-04 16:16:45 +00:00
Victor Farazdagi
84ae840b5d Re-enable previous fetching algorithm testing (#6125)
* re-enable previous fetching algorithm testing
2020-06-04 11:26:02 +00:00
Ivan Martinez
24d315c6d1 Add public keys to response for GetValidatorPerformance (#6115)
* Add public keys to response for GetValidatorPerformance
* Merge branch 'master' into add-pubkey-resp
* Merge branch 'master' into add-pubkey-resp
* Fixed all the broken tests (#6117)
* Merge branch 'add-pubkey-resp' of github.com:prysmaticlabs/prysm into add-pubkey-resp
* Fix skip
* Merge branch 'master' into add-pubkey-resp
* Fix ethapis
* Merge branch 'master' into add-pubkey-resp
2020-06-04 04:55:29 +00:00
rkapka
9c13e78dbd Simplified validator flags (#6108)
* simplified validator flags
* corrected typo
* Merge branch 'master' into simplify-validator-flags
* Merge branch 'master' into simplify-validator-flags
* Merge branch 'master' into simplify-validator-flags
* Merge branch 'master' into simplify-validator-flags
* Merge branch 'master' into simplify-validator-flags
2020-06-04 02:23:17 +00:00
terence tsao
a8c13dcaf8 Add NewBeaconBlock to testutil (#6118)
* Use state util to get block root

* Add NewBeaconBlock
2020-06-03 20:21:36 -05:00
Ivan Martinez
2eff8704a5 Add indice to certain validator logs (#6092)
* Add indice to activation and accounts list command

* Add to more logs

* Fix wrong !

* Fix err

* Fix tests
2020-06-03 19:12:18 -04:00
Victor Farazdagi
c94511223b Assert gfind is available on OSX, for protobufs updates (#6112)
* Assert gfind is available on OSX, for protobufs updates
* wording
* Merge branch 'master' into fix-protobuf-upd-scripts
2020-06-03 21:15:37 +00:00
Raul Jordan
b2b78c8a79 Remove Websockets for ETH1 From Prysm (#6055)
* mark websockets flag as deprecated, begin removing uses
* powchain package now builds
* Merge branch 'master' into no-websocket
* use seconds per eth1 block
* fix tests
* Merge branch 'master' into no-websocket
* rem websocket req
* Merge branch 'no-websocket' of github.com:prysmaticlabs/prysm into no-websocket
* Merge refs/heads/master into no-websocket
* Merge refs/heads/master into no-websocket
* fix broken test
* Merge refs/heads/master into no-websocket
* Merge refs/heads/master into no-websocket
* Merge refs/heads/master into no-websocket
* Merge refs/heads/master into no-websocket
* Merge refs/heads/master into no-websocket
* Merge refs/heads/master into no-websocket
2020-06-03 20:28:16 +00:00
terence tsao
c4afb8e2e7 Save unaggregated att in beacon_aggregate_and_proof subscriber (#6109)
* Use state util to get block root
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Also save unaggregated att
* Merge refs/heads/master into fix-save-agg
* Handle error
* Merge branch 'fix-save-agg' of github.com:prysmaticlabs/prysm into fix-save-agg
* Test
* Merge refs/heads/master into fix-save-agg
2020-06-03 19:49:43 +00:00
Preston Van Loon
d2337d0ec1 Disable some static analysis checks for coverage builds (#6105)
* Disable some static analysis checks for coverage builds
* Merge branch 'master' into fix-coverage-builds
* disable lostcancel for third_party code
* Merge branch 'fix-coverage-builds' of github.com:prysmaticlabs/prysm into fix-coverage-builds
* Merge refs/heads/master into fix-coverage-builds
* A few lostcancel fixes
* Merge branch 'fix-coverage-builds' of github.com:prysmaticlabs/prysm into fix-coverage-builds
* Merge refs/heads/master into fix-coverage-builds
2020-06-03 19:22:48 +00:00
Ivan Martinez
4e96cbeae7 E2E Improvements (#6091)
* Some fixes
* Merge branch 'master' into e2e-fixes
* Add another small delay
* Merge branch 'e2e-fixes' of github.com:prysmaticlabs/prysm into e2e-fixes
* Remove genesis test, make normal e2e run longer
* Gaz
* more fixes
* Merge branch 'master' into e2e-fixes
* Merge refs/heads/master into e2e-fixes
* Fix comment
* Merge refs/heads/master into e2e-fixes
2020-06-03 18:44:13 +00:00
Raul Jordan
80539d9028 Enable JSON-HTTP Gateway by Default on Port 3000 (#6090)
* default port 3000 http gateway
* Merge refs/heads/master into default-gateway
* Merge refs/heads/master into default-gateway
* fix broken node test
* disable grpc gateway
* Merge refs/heads/master into default-gateway
* Merge refs/heads/master into default-gateway
2020-06-03 18:11:43 +00:00
Raul Jordan
75aaa5e07b fix bad root (#6096) 2020-06-03 11:26:07 -05:00
Victor Farazdagi
7429221f3f go fmt (#6101) 2020-06-03 06:42:22 -07:00
Preston Van Loon
e79ba096d6 Rollback libp2p kad dht dependency update (#6084)
* go get github.com/libp2p/go-libp2p-kad-dht@v0.2.1 github.com/libp2p/go-libp2p-kbucket@v0.2.3
* revert API changes
* one more API fix
* Merge refs/heads/master into rollback-kad-dht
* Merge refs/heads/master into rollback-kad-dht
2020-06-02 23:01:42 +00:00
rkapka
ac138eae7b Split validator databases (#6048)
* initial working implementation with a basic test
* merge validator enhancements
* added test dependency to db's build file
* Merge branch 'master' into merge-validators-enhancements
* changed formatting of public key
* Merge branch 'master' into merge-validators-enhancements
* Merge branch 'master' into merge-validators-enhancements
* removed unused import
* Merge branch 'merge-validators-enhancements' into split-validator-databases

# Conflicts:
#	validator/db/BUILD.bazel
#	validator/db/manage.go
#	validator/db/manage_test.go
* tests and small fixes
* extracted common functionality
* Merge branch 'master' into split-validator-databases

# Conflicts:
#	validator/accounts/account_test.go
#	validator/db/manage.go
#	validator/db/manage_test.go
* added missing test dependency to build file
* added missing flags to main.go
* applied code review suggestions
* renamed flags to avoid duplication
* Merge branch 'master' into split-validator-databases
* Merge branch 'master' into split-validator-databases
* removed redundant parenthesis
* Merge branch 'master' into split-validator-databases
* Merge branch 'master' into split-validator-databases
* Merge branch 'master' into split-validator-databases
* Merge branch 'master' into split-validator-databases
* Merge branch 'master' into split-validator-databases
* extracted defer errors to package-level variables
* Merge branch 'master' into split-validator-databases
* comply with error naming convention
* removed incorrect import
* Merge branch 'master' into split-validator-databases
2020-06-02 22:04:16 +00:00
Nishant Das
469499873b Add Sync Pool For Snappy (#6085)
* add pool for snappy

* Update beacon-chain/p2p/encoder/ssz.go

* comment added

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-06-02 14:02:21 -05:00
Shay Zluf
fd19fd10a9 New spanner db structure (#6061)
* new spanner db structure

* lint fixes

* go mod fix

* fix iface

* remove unused

* remove extra line

* change from db

* exported field

* exported field

* revert to original

* fix

* ivan feedback

* ivan feedback

* ivan feedback

* revert mod changes

* fix db impl

* gaz

* import fix

* Try to fix tests

* ivan feedback

* new epoch store

* added comment

* fix error

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
2020-06-02 10:41:21 -04:00
terence tsao
d152b48815 use stateutil.BlockRoot instead of ssz for forkchoice (#6075)
* Use state util to get block root
* Merge refs/heads/master into use-stateutil1
2020-06-01 23:45:57 +00:00
Nishant Das
afbe494e55 Update Geth Fork (#5865)
* update workspace
* new updates
* new commit
* fixes
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into updateGeth
* test fixes
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into fixCreateconfig

# Conflicts:
fix test
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into updateGeth
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into fixCreateconfig

# Conflicts:
update commit
* use geth's dep
* Merge branch 'master' into updateGeth
* update workspace
* fix again
* Merge branch 'updateGeth' of https://github.com/prysmaticlabs/geth-sharding into updateGeth
* Merge branch 'master' into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* fix osx builds
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* add execption and disable nilness
* remove redundant deps
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into updateGeth
* clean up
* go mod
* tidy
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Add patch to resolve panic until https://github.com/bazelbuild/rules_go/pull/2450 can be addressed
* Merge branch 'updateGeth' of github.com:prysmaticlabs/prysm into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
* Merge refs/heads/master into updateGeth
2020-06-01 22:03:33 +00:00
Nishant Das
e715339aab Fix Status Messages During Genesis (#6046)
* fix status messages
* fix finalized checkpoint
* remove code
* Merge refs/heads/master into fixStatusMessages
* Update beacon-chain/blockchain/service.go
* Merge refs/heads/master into fixStatusMessages
* Merge refs/heads/master into fixStatusMessages
* Merge refs/heads/master into fixStatusMessages
* fix
* fix test
* fix again
* fix
* Merge refs/heads/master into fixStatusMessages
* Merge refs/heads/master into fixStatusMessages
* Merge refs/heads/master into fixStatusMessages
* Merge refs/heads/master into fixStatusMessages
* Merge refs/heads/master into fixStatusMessages
* Merge refs/heads/master into fixStatusMessages
* Merge refs/heads/master into fixStatusMessages
* Merge refs/heads/master into fixStatusMessages
* Merge refs/heads/master into fixStatusMessages
* Merge refs/heads/master into fixStatusMessages
2020-06-01 20:26:42 +00:00
Victor Farazdagi
4ed0e432e3 Fixes subsequence matching in deleteValueForIndices (#6071)
* fixes subsequence matching in deleteValueForIndices
* Merge branch 'master' into fix-delete-value-for-indices
* Merge refs/heads/master into fix-delete-value-for-indices
2020-06-01 19:00:47 +00:00
terence tsao
db9e02d7d1 Cache genesis block root (#6069)
* Don't forget to set genesis blockroot
* Typo
* Merge branch 'master' into set-genesis-blk-root
2020-06-01 17:39:09 +00:00
Raul Jordan
31ef6befde Add More Tests to CommitteeAssignments (#6028)
* add test
* Merge refs/heads/master into comm-tests
* Merge refs/heads/master into comm-tests
* Merge refs/heads/master into comm-tests
* Merge refs/heads/master into comm-tests
* Merge refs/heads/master into comm-tests
* Merge refs/heads/master into comm-tests
* Merge refs/heads/master into comm-tests
* Merge refs/heads/master into comm-tests
* pass comm tests
* Merge branch 'comm-tests' of github.com:prysmaticlabs/prysm into comm-tests
* Merge refs/heads/master into comm-tests
* Merge refs/heads/master into comm-tests
* Merge refs/heads/master into comm-tests
* Merge refs/heads/master into comm-tests
* Merge refs/heads/master into comm-tests
* Merge refs/heads/master into comm-tests
* Merge refs/heads/master into comm-tests
* Merge refs/heads/master into comm-tests
* Merge refs/heads/master into comm-tests
2020-06-01 14:50:48 +00:00
Raul Jordan
c23ac8547f Ensure Consistent ListValidators RPC Results (#6054)
* run process slots on list vals call
* fix behavior
* Merge branch 'master' into bad-list-vals
* Merge refs/heads/master into bad-list-vals
* Merge refs/heads/master into bad-list-vals
* Merge refs/heads/master into bad-list-vals
* Merge refs/heads/master into bad-list-vals
* Merge refs/heads/master into bad-list-vals
* Merge refs/heads/master into bad-list-vals
2020-06-01 14:16:40 +00:00
Victor Farazdagi
261a343ea5 Fixes intersection functions (uint64, int64, []byte) (#6067) 2020-06-01 06:35:29 -07:00
Nishant Das
ceaceeb33c Reset Streams Properly (#6066)
* reset streams

* fix build
2020-06-01 16:14:47 +08:00
terence tsao
6598c38625 Forkchoice start using proper root (#6062)
* Skip update head if justiifed root is zero hashes

* Start with genesis root instead

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-01 11:32:34 +08:00
terence tsao
6e514b96fd Remove unused code (#6065)
* Ran code cleanup from goland

* Typo

* Remove unused code

* Gazelle

* Delete comments
2020-05-31 15:47:15 -07:00
terence tsao
6e3b78b99e Ran code cleanup from goland (#6064)
* Ran code cleanup from goland

* Typo
2020-05-31 15:08:36 -05:00
Preston Van Loon
49a0d3caf0 Refactor dependencies, make Prysm "go gettable" (#6053)
* Fix a few deps to work with go.mod, check in generated files

* Update Gossipsub to 1.1 (#5998)

* update libs

* add new validators

* add new deps

* new set of deps

* tls

* further fix gossip update

* get everything to build

* clean up

* gaz

* fix build

* fix all tests

* add deps to images

* imports

Co-authored-by: rauljordan <raul@prysmaticlabs.com>

* Beacon chain builds with go build

* fix bazel

* fix dep

* lint

* Add github action for testing go

* on PR for any branch

* fix libp2p test failure

* Fix TestProcessBlock_PassesProcessingConditions by updating the proposer index in test

* Revert "Fix TestProcessBlock_PassesProcessingConditions by updating the proposer index in test"

This reverts commit 43676894ab.

* Compute and set proposer index instead of hard code

* Add back go mod/sum, fix deps

* go build ./...

* Temporarily skip two tests

* Fix kafka confluent patch

* Fix kafka confluent patch

* fix kafka build

* fix kafka

* Add info in DEPENDENCIES. Added a stub link for Why Bazel? until https://github.com/prysmaticlabs/documentation/issues/138

* Update fuzz ssz files as well

* Update fuzz ssz files as well

* getting closer

* rollback rules_go and gazelle

* fix gogo protobuf

* install librdkafka-dev as part of github actions

* Update kafka to a recent version where librkafkfa is not required for go modules

* clarify comment

* fix kafka build

* disable go tests

* comment

* Fix geth dependencies for end to end

* rename word

* lint

* fix docker

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: rauljordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-05-31 14:44:34 +08:00
Ivan Martinez
d35531cb17 Add index to activationStatus and MultipleValidatorStatus response (#6026)
* Cleanup val status

* More cleanup

* Fix tests

* Fix tests again

* Change to nonExistentIndice

* Fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-29 13:17:27 -05:00
Ivan Martinez
213f8ef088 Fix long-running E2E (#6047)
* Fixes for long running E2E
* Merge branch 'master' of github.com:prysmaticlabs/prysm into fix-e2e
* Move metrics check up
* Merge refs/heads/master into fix-e2e
2020-05-29 16:52:38 +00:00
rkapka
d1e2901b95 Merge validators enhancements (#6027)
* merge validator enhancements
* added test dependency to db's build file
* Merge branch 'master' into merge-validators-enhancements
* changed formatting of public key
* Merge branch 'master' into merge-validators-enhancements
* Merge branch 'master' into merge-validators-enhancements
* removed unused import
2020-05-29 15:57:10 +00:00
Victor Farazdagi
0243bdcf37 Fixes overflow issue in sync/pending_blocks_queue (#6039)
* fixes overflow issue
* Merge refs/heads/master into fix-overflow
* Merge refs/heads/master into fix-overflow
2020-05-29 13:50:41 +00:00
Victor Farazdagi
a0bf8cba52 Removes redundant type conversions (#6041)
* removes redundant type conversions
* remove redundant type in struct decl
* removes redundant parens
* Merge refs/heads/master into redundant-type-conversions
2020-05-29 12:42:39 +00:00
Victor Farazdagi
0eaf0cf27b Properly handles possible nil values (#6042)
* handles possible nil values
2020-05-29 12:09:04 +00:00
terence tsao
ac862d1e7e Increase hot state cache size to 32 (#6033)
* Increase hot state cache size to 16
* Merge refs/heads/master into hot-state-cache-size
2020-05-29 05:01:40 +00:00
Ivan Martinez
cde127354c Change slashing pool to use spec code (#6030)
* Change slashing pool to use more spec code

* Fix test

* Undo unneeded changes

* Make sure to catch regression

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-29 00:26:09 -04:00
Ivan Martinez
0b64db5e21 Add voluntary exit processing to E2E (#6016)
* Add voluntary exit to E2E

* Fix long urnning e2e

* Fix for comments

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-28 22:51:00 -04:00
Raul Jordan
88aaee36bf Fix Weird Pre-Genesis Timestamp (#6031)
* ensure test passes

* Apply suggestions from code review

* comment

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
2020-05-28 21:57:03 -04:00
terence tsao
3fe47c0043 Reduce attester state copies (#6025)
* Test

* Fix

* Removed extra line

* Feature flag

* Gaz
2020-05-28 13:33:08 -04:00
Ivan Martinez
7dd187c2ae Small fixes for slasher (#6021)
* Small fixes for slasher

* Remove useless log

* Fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-28 10:24:54 -04:00
terence tsao
e3180cf977 Invert state field trie (#6013)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-28 08:24:26 -05:00
Victor Farazdagi
9e8eafda6a Fixes problem in YAML front-matter of the new flag tracking issue template (#6023)
* fixes probleim in YAML front-matter
* Merge refs/heads/master into fix-feature-tracking-issue-template
2020-05-28 10:45:18 +00:00
Raul Jordan
9a2c684628 Recommend Users Blocked By Antivirus a Way to Curl Prysm.sh (#6019)
* recommend no revoke
* Merge refs/heads/master into antivirus
2020-05-28 10:00:19 +00:00
Nishant Das
979c0074c7 Register RPC And Pubsub Handlers After Genesis is Determined (#6020)
* fix
* fix test
2020-05-28 05:39:40 +00:00
Preston Van Loon
a139b75dc5 Add documentation and issue template for feature flags (#6018)
* Add details around feature flag usage and recommend filing tracking issues for feature flags.

* 🦄

* 🦄

* labels

* more comments
2020-05-27 20:37:37 -05:00
terence tsao
4c2ed410b0 Invert state ref copy flag (#6015)
* Revert "Release state-ref-copy flag (#6012)"

This reverts commit ebf7122d6f.

* Invert state ref copy flag
2020-05-27 14:30:54 -05:00
terence tsao
ebf7122d6f Release state-ref-copy flag (#6012)
* Release state ref copy flag
* Merge refs/heads/master into invert-flag1
2020-05-27 18:08:12 +00:00
terence tsao
b7c856af6f Add skip to log (#6014) 2020-05-27 12:35:54 -05:00
Nishant Das
7be8942079 Fix Fork Data in Local Node (#5997) 2020-05-27 08:07:40 -07:00
Ivan Martinez
29e4594783 Fix long running E2E after v0.12 changes (#6008)
* Fix deposits in long run e2e

* Fix participation

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-27 08:28:01 -05:00
Victor Farazdagi
f4778048ac Init sync arbitrary batch length + full bandwidth utilization (#5978)
* allow to jump over 50K skipped slots
* more tests
* removes debug logs
* minor fixes
* re-arrage pid updates
* optimizes queue and fsm
* Merge branch 'master' into init-sync-arbitrary-batch-len
* fixes benchmark test
* Merge branch 'master' into init-sync-arbitrary-batch-len
* queue cleanup
* updates polling interval
* fsm test: allMachinesInState
* fsm test: highestStartBlock
* rest of fsm tests
* Merge branch 'master' into init-sync-arbitrary-batch-len
* Merge refs/heads/master into init-sync-arbitrary-batch-len
* Merge refs/heads/master into init-sync-arbitrary-batch-len
* Merge refs/heads/master into init-sync-arbitrary-batch-len
* Merge refs/heads/master into init-sync-arbitrary-batch-len
* Merge refs/heads/master into init-sync-arbitrary-batch-len
* parse machines in order
* Merge branch 'init-sync-arbitrary-batch-len' of github.com:prysmaticlabs/prysm into init-sync-arbitrary-batch-len
2020-05-27 09:13:41 +00:00
terence tsao
0c3586a8ea Add debug proto array fork choice endpoint (#6003)
* Add proto array fork choice object to RPC
* Add pb
* Add pb
* Expose proto array store object
* Test
* Merge branch 'forkchoice-endpoint' of github.com:prysmaticlabs/prysm into forkchoice-endpoint
* s/Nodes/nodes
* Remove proto from gitignore
* More implementations of GetProtoArrayForkChoice
* Comments
* Use hex
* Gazelle
* GetForkChoice Test
* Remove pb.go
* Merge branch 'master' into forkchoice-endpoint
* Typo, thanks Raul!
* Merge branch 'forkchoice-endpoint' of github.com:prysmaticlabs/prysm into forkchoice-endpoint
2020-05-26 23:24:38 +00:00
Ivan Martinez
5afee4ba3a Use SlotTickerWithOffset for StreamIndexedAttestations (#5999)
* Change streamindexed to slot ticker with offset
* Make 2/3rds
* Add test
* Merge branch 'master' of github.com:prysmaticlabs/prysm into ticker-offset
* Add check for offset
* Merge branch 'master' of github.com:prysmaticlabs/prysm into ticker-offset
* Fix test
* Merge refs/heads/master into ticker-offset
2020-05-26 19:34:09 +00:00
Ivan Martinez
0f7cf212a2 Prepare E2E for v0.12 (#5991)
* Prepare e2e for v0.12

* Fix e2e

* Comments

* Attempt 100% participation

* Fixes

* Fix

* Update readme

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-26 14:04:42 -05:00
Ivan Martinez
ecea979b60 Change StreamIndexedAttestations to use target root for state regen (#5971) 2020-05-26 09:21:17 -07:00
Raul Jordan
ad1d8074d0 Improve Deposit Data Account Creation Prompt (#5985)
* improve validator accounts create text
* Merge refs/heads/master into improve-deposit-data-promt
* Merge refs/heads/master into improve-deposit-data-promt
* Merge refs/heads/master into improve-deposit-data-promt
* Merge refs/heads/master into improve-deposit-data-promt
2020-05-26 13:24:46 +00:00
Victor Farazdagi
c70c52b40a Runs go fmt and updates validator files (#5995)
* updates validator files
2020-05-26 11:13:12 +00:00
Martin Linkhorst
b724b20828 Add missing TLS-related flags to Beacon Chain and Slasher (#5984)
* beacon-chain: add missing slasher flags
* slasher: add missing tls-cert flag
* Merge branch 'master' into tls
* Merge branch 'master' into tls
2020-05-26 02:08:15 +00:00
rkapka
b3f005fe94 Fixed deferring of closing dbs (#5982)
* fixed deferring of closing dbs
* removed unnecessary slice pointer
* Merge branch 'master' into merge-defer-fix
* Merge branch 'master' into merge-defer-fix
2020-05-26 00:38:41 +00:00
terence tsao
3b65cb6d0b Consistent BLS function inputs (#5969)
* Make function input consistent

* Change back

* Typo

* Fixed test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-25 16:08:12 -05:00
Raul Jordan
497307decd Remove Warning on Using Default Mainnet Config (#5983)
* removed mainnet warn
2020-05-25 19:43:10 +00:00
Victor Farazdagi
f81c8c6825 Allow to jump over 50K+ skipped slots in init-sync (#5975)
* allow to jump over 50K skipped slots
* more tests
* removes debug logs
* minor fixes
* re-arrage pid updates
* Merge refs/heads/master into init-sync-50K-skipped-slots
2020-05-25 16:43:59 +00:00
rkapka
277fa3300a Merge validator databases (#5968)
* merge functionality

* merge testing

* log info when merge completed successfully

* extracted private methods

* fixed linter errors

* fixed compilation errors

* build files clean-up

* fixed failing test

* corrected command description

* close sources inside defer

* corrected documentation of NewKVStore
2020-05-25 11:20:35 -05:00
Ivan Martinez
b810243f41 Reverse historical detection flag for slasher (#5958)
* Reverse historical detection flag
* Merge branch 'master' of github.com:prysmaticlabs/prysm into reverse-historical-flag
* Merge branch 'master' of github.com:prysmaticlabs/prysm into reverse-historical-flag
* Fix build
* Merge refs/heads/master into reverse-historical-flag
* Fix test
* Merge branch 'reverse-historical-flag' of github.com:prysmaticlabs/prysm into reverse-historical-flag
* Merge refs/heads/master into reverse-historical-flag
2020-05-23 06:40:15 +00:00
Preston Van Loon
5a3114cdc9 Update BLS to allow for spec v0.12 requirements (#5914)
* update bls

* Update BLS, has a minor issue with blsVerifyAggregatedHashWithDomainCast. See https://github.com/herumi/bls-eth-go-binary/issues/17

* update

* fix build
2020-05-23 10:50:02 +08:00
Ivan Martinez
f9d847991c Add RemoveOldestFromCache and save span cache on exit (#5953)
* Add RemoveOldestFromCache
* Add log
* comment
* gaz
* Merge branch 'master' into prune-spancache-historical
* Merge refs/heads/master into prune-spancache-historical
* Merge refs/heads/master into prune-spancache-historical
* Fix span defer
* Merge branch 'prune-spancache-historical' of github.com:prysmaticlabs/prysm into prune-spancache-historical
* Merge refs/heads/master into prune-spancache-historical
* Update slasher/db/kv/kv.go
2020-05-22 19:44:48 +00:00
Ivan Martinez
6aabea2d80 Reverse broadcast-slashing flag to disable-broadcast-slashings (#5952)
* Reverse broadcast-slashings to disable-broadcast-slashings
* Fix tests
* Merge branch 'master' of github.com:prysmaticlabs/prysm into toggle-broadcast-flag
* Merge refs/heads/master into toggle-broadcast-flag
* Fix for comments
* Merge branch 'toggle-broadcast-flag' of github.com:prysmaticlabs/prysm into toggle-broadcast-flag
* Merge refs/heads/master into toggle-broadcast-flag
2020-05-22 17:19:45 +00:00
Ivan Martinez
230659467d Only log attester slashing found for valid slashings (#5954)
* Improve logging for slasher

* Add case for sig verify

* Log out indices for failed slashings

* gaz

* Fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-22 10:19:10 -05:00
Victor Farazdagi
c875dd458b Handles issue with empty dirs on validator accounts create (#5940)
* handles issue with empty dirs on validator accounts create
* better comments
* Merge refs/heads/master into fix-validator-accounts-create
* Merge refs/heads/master into fix-validator-accounts-create
* Terence's review
2020-05-22 11:00:41 +00:00
Nishant Das
6a9112bfae remove byte mempool (#5951) 2020-05-21 20:39:06 -07:00
terence tsao
8eb7c7a0bc Recover skipped archived point (#5950) 2020-05-21 18:15:16 -07:00
Ivan Martinez
4fb3307de3 Add Slasher double block detection to E2E (#5936)
* WIP double block slashings e2e
* Add double block detection to E2E
* Remove unused function
* Respond to feedback
* Remove unneeded util
* Merge branch 'master' of github.com:prysmaticlabs/prysm into e2e-double-block
* Add comments for clarity on conns
2020-05-21 06:02:40 +00:00
Raul Jordan
881ad77c52 Add Debug Block Endpoint + Return SSZ-Encoded (#5926)
* add block endpoint

* handle err

* check for nil block

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-20 20:36:05 -05:00
terence tsao
52988a0805 Return err instead of 0 for unavailable participation (#5934)
* Return err for unavailable participation

* Update     test
2020-05-20 18:47:40 -04:00
Ivan Martinez
5267fe607c Fix slasher pool to not ignore valid slashings (#5927)
* Improve slashing pool handling to not reject valid slashings
* Undo testing stuff
* Merge refs/heads/master into fix-slasher-pool
* Add regression test for pool changes
* Merge branch 'fix-slasher-pool' of github.com:prysmaticlabs/prysm into fix-slasher-pool
* Merge refs/heads/master into fix-slasher-pool
* Merge refs/heads/master into fix-slasher-pool
* Merge refs/heads/master into fix-slasher-pool
* Merge refs/heads/master into fix-slasher-pool
* Merge refs/heads/master into fix-slasher-pool
2020-05-20 20:27:23 +00:00
Ivan Martinez
9fc6be8b74 Change ListIndexedAttestations to be more slasher friendly (#5930)
* Improve slasher beacon comms
* Fixes
* Add case for normal errors
* remove baz
* Merge branch 'master' into improve-slasher-beacon
* Merge refs/heads/master into improve-slasher-beacon
* Merge refs/heads/master into improve-slasher-beacon
* Feedback
* Merge branch 'improve-slasher-beacon' of github.com:prysmaticlabs/prysm into improve-slasher-beacon
* Merge refs/heads/master into improve-slasher-beacon
2020-05-20 19:52:18 +00:00
terence tsao
5f72d28057 Blockchain service provides IsCanonical getter (#5932)
* Export `NonExistentNode` and `Root`
* Add mapping `recentCanonicalBlocks`
* Add `updateRecentCanonicalBlocks`
* Use `updateRecentCanonicalBlocks`
* New tests
* s/Root/root
* Lint
* Merge branch 'master' into is-canonical
* Merge refs/heads/master into is-canonical
2020-05-20 19:16:56 +00:00
Preston Van Loon
020c481a46 Enable domain data cache by default (#5931)
* Enable domain data cache by default
* Merge refs/heads/master into enable-domain-data-cache-by-default
2020-05-20 18:46:03 +00:00
Nishant Das
e3d07489dc Fix Flags in Create Subcommand (#5928)
* fix flag parsing
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into fixCreateconfig
* Merge refs/heads/master into fixCreateconfig
2020-05-20 18:19:30 +00:00
Nishant Das
49edcb099b Log Active Validator Count (#5923)
* add more verbose deposit count
* Merge branch 'master' into addBetterCount
* Merge refs/heads/master into addBetterCount
* Update beacon-chain/powchain/log_processing.go

Co-authored-by: Shay Zluf <thezluf@gmail.com>
2020-05-20 16:24:51 +00:00
Shay Zluf
17169e5a2d External slashing protection (#5895)
* slasher grpc client

* do not export

* slasher on a different package

* fix featureconfig

* change to rough time

* revert roughtime

* remove extra comma

* revert order change

* goimports

* fix comments and tests

* fix package name

* revert reorder

* comment for start

* service

* fix visibility

* external slasher validator protection implementation

* gaz

* fix comment

* add comments

* nishant feedback

* raul feedback

* preston feedback

* fix flags

* fix imports

* fix imports

* port 4002

* added tests

* fix log

* fix imports

* fix imports name

* raul feedback

* gaz

* terence comment

* change name

* runtime fixes

* add flag check

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-20 10:23:22 -05:00
Nishant Das
2de4f86277 Change To Debug Logs (#5917)
* debug
2020-05-20 04:57:24 +00:00
Raul Jordan
ea02edba29 Ensure We Use Request.Epoch in the Validator Duties Server Implementation (#5913)
* resolve bad epoch in duties
2020-05-19 21:26:14 +00:00
Victor Farazdagi
8241b7e20d Re-arranges fetcher locks -> per peer level (#5906)
* fetcher locks -> per peer level
* typo
* removes redundant setting to nil
* Merge refs/heads/master into init-sync-rearrange-locks
2020-05-19 17:48:32 +00:00
Ivan Martinez
aa6190235a Fix prysm.bat internet warning (#5908)
* Fix bat
* Remove %
* Merge branch 'master' into fix-bat
2020-05-19 17:24:47 +00:00
Preston Van Loon
5e6a5902ae Invert --enable-account-metrics to remove user confusion. See #5883 (#5904) 2020-05-19 08:44:54 -05:00
Michael Huang
62b617fa9a Command to fetch validator statuses + MultipleValidatorStatus (#5784)
* Add method to fetch account statuses
* Add status command
* gofmt
* Add comment about sorting
* Close conneciton when finished
* Fix error
* Refactor FetchAccountStatuses
* Add status_test.go
* Move sorting out of FetchAccountStatuses
* Remove unnecessary casting
* Expect ValidatorStatus to be called
* Wrap long comment
* Comment out sorting
* Add all necessary dial options
* Close connection before evaluating error from fetch
* Small changes
* Fix lint issues
* Merge branch 'master' into validator-client-account-statuses
* Update dependencies for docker images
* Rename multipleValidatorStatus to activationStatus
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into validator-client-account-statuses
* Update commit hash for ethereumapis
* Implement MultipleValidatorStatus
* Tests for MultipleValidatorStatus
* Fix bugs
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into validator-client-account-statuses
* Add export comment for MultipleValidatorStatus
* Run `go fmt` in prysm (#5815)

* go fmt
* Add fix for nil state in InitializeFromProto (#5817)

* Add nil check and tests
* Add unsafe test
* Update tools/genesis-state-gen/main.go
* Undo genesis state gen changes
* Merge branch 'proto-fuzz-fix' of github.com:prysmaticlabs/prysm into proto-fuzz-fix
* gaz
* Fix
* Add export comment for MultipleValidatorStatus
* Clean up comments
* Update mock files for beacon_node_validator_service
* Merge branch 'validator-client-account-statuses' of https://github.com/michaelhly/prysm into validator-client-account-statuses
* Run gazelle
* Fix mock issues
* Fetch statuses in batches
* Simplify public key generation for status_test
* Sort validator statuses by status type
* Format validator statuses and print to console
* Fix lint issues
* Delimit with commas
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into validator-client-account-statuses
* Rename otheropts to extraopts
* Merge branch 'master' into validator-client-account-statuses
* Merge branch 'validator-client-account-statuses' of https://github.com/michaelhly/prysm into validator-client-account-statuses
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into validator-client-account-statuses
* Clean up MultipleValidatorStatus tests
* Add sync checker to MultipleValidatorStatus
* Update formatting
* Prepend 0x to validator keys
* Check number of status blocks recieved in status_test
* Move sorting to goroutine
* Capitalize constants
* Fix typo
* Use mock reponses in sort test
* Remove byteutils
* Fix ugly format
* Add comment on MultipleValidatorStatus test
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into validator-client-account-statuses
* Create entrypoint to run status command, and make unexported functions internal
* Move merge step into FetchAccountStatuses
* Revert service.go
* Remove responseToSortedMetadata
* Simplify mergeTwo
* Replace fmt output with logrus
* Fix typo
* Merge branch 'master' into validator-client-account-statuses
* Update comment
* Merge branch 'validator-client-account-statuses' of https://github.com/michaelhly/prysm into validator-client-account-statuses
* Return error on bad credentials
* Merge branch 'master' into validator-client-account-statuses
* Merge branch 'master' into validator-client-account-statuses
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into validator-client-account-statuses
* Skip merge step on error
* Fix conflicts
* Fix mock paths
* Add comments
* Convert some sprintfs to wrapfs
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into validator-client-account-statuses
* Rename ExtractPublicKeys to ExtractPublicKeysFromKeyStore and move to account.go
* Add support for keymanager
* Add supported flags to flags list
* Log warning on intermediary errors
* Update output
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into validator-client-account-statuses
* Merge branch 'master' into validator-client-account-statuses
* Fix conflicts
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into validator-client-account-statuses
* Merge branch 'validator-client-account-statuses' of https://github.com/michaelhly/prysm into validator-client-account-statuses
* Set context timeout for FetchAccountStatuses
* Remove deprecated grpc.WithTimeout
* gofmt
* Remove getters
* Remove parallel stuff
* Move grpc dialing out of status.go
* Update logging based on feedback
* Update validator/accounts/status.go
2020-05-19 05:13:37 +00:00
Ivan Martinez
aad1b93f70 Optimize slasher handling of duplicate attestations (#5886)
* Move data saving to listener service

* Map result data to attestations to prevent duplicate DB reads

* Gaz

* Fix tests

* Revert "Fix tests"

This reverts commit 499b205a88.

* Revert "Move data saving to listener service"

This reverts commit fd52938dba.

* Gaz

* Undo changes

* Imports

* Reorder parameters

* Fix error log

* Add test for mapReusltsToAtts

* Fix errorf

* Change to safer crypto

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-18 20:50:35 -04:00
Jim McDonald
fe7a123bd9 Allow wallet passphrases to be environment variables (#5879)
* Allow wallet passphrases to be environment variables
* Merge branch 'master' into wallet-env-passphrases
* Namespace environment variables and clear them at the end of the test.
* Merge branch 'master' into wallet-env-passphrases
* Merge branch 'master' into wallet-env-passphrases
* Merge branch 'master' into wallet-env-passphrases
* Merge branch 'master' into wallet-env-passphrases
* Merge branch 'master' into wallet-env-passphrases
* Merge branch 'master' into wallet-env-passphrases
* Merge branch 'master' into wallet-env-passphrases
* Merge branch 'master' into wallet-env-passphrases
* Merge branch 'master' into wallet-env-passphrases
2020-05-18 22:25:36 +00:00
rkapka
2a1a0ce43f Change validator password (#5869)
* Created merge command
* Key merging
* Merge branch 'master' into db-manage-validators
* flag comments
* fixed compilation errors
* added comment to Merge function
* Tested that merging copies keys
* Created command to change password for keys
* Merge branch 'master' into change-validator-password
* Merge branch 'master' into change-validator-password
* cleanup after code review
* Merge remote-tracking branch 'rkapka/change-validator-password' into change-validator-password
* Merge branch 'master' into change-validator-password
* Merge branch 'master' into change-validator-password
* Merge branch 'master' into change-validator-password
* Merge branch 'master' into change-validator-password
2020-05-18 21:35:17 +00:00
terence tsao
c07f1600e9 Prevent parent block with same slot (#5842)
* Block must have a higher slot than parent state
* New tests and existing test
* Typo
* Fixed more existing test....
* Merge branch 'master' into sec0
* Merge branch 'master' into sec0
* Merge branch 'master' of github.com:prysmaticlabs/prysm into sec0
* Update spec tests to v0.11.3
* Merge branch 'sec0' of github.com:prysmaticlabs/prysm into sec0
* Update badge to v0.11.3
* Update badge to v0.11.3
* Merge refs/heads/master into sec0
2020-05-18 20:26:34 +00:00
terence tsao
678347e64b Handle missing archived index (#5899)
* Make `archivedRoot` handle missing root case
* Regression tests
* Fixed existing tests
* Removed debug log
* Merge refs/heads/master into handle-missing-archived-index
* Merge refs/heads/master into handle-missing-archived-index
* More comments on the look back
* Merge branch 'handle-missing-archived-index' of github.com:prysmaticlabs/prysm into handle-missing-archived-index
* Merge refs/heads/master into handle-missing-archived-index
2020-05-18 19:46:06 +00:00
Victor Farazdagi
ca26745720 Implements WRR in init-sync, full bandwidth utilization (#5887)
* implements weighted round robin in init-sync
* protection against evil peer
* reshuffle
* shorten flag
* deterministic order of wrr
* Merge branch 'master' into init-sync-wrr
* Merge refs/heads/master into init-sync-wrr
* Merge refs/heads/master into init-sync-wrr
* allow peers with the same capacity to pe selected at random
* Merge branch 'init-sync-wrr' of github.com:prysmaticlabs/prysm into init-sync-wrr
* adds wrr to e2e tests
* Merge refs/heads/master into init-sync-wrr
* Merge refs/heads/master into init-sync-wrr
* Merge refs/heads/master into init-sync-wrr
2020-05-18 18:59:03 +00:00
Ivan Martinez
52105e6083 Fix silent internet error in prysm.sh (#5877)
* Add check for internet in prysm.sh
* Make script fail
* End script early
* Merge branch 'master' into add-internet-check
* Fix order of ops
* Merge branch 'add-internet-check' of github.com:prysmaticlabs/prysm into add-internet-check
* Merge refs/heads/master into add-internet-check
* Merge refs/heads/master into add-internet-check
* Merge refs/heads/master into add-internet-check
* Merge refs/heads/master into add-internet-check
* Merge refs/heads/master into add-internet-check
* Merge refs/heads/master into add-internet-check
* Merge refs/heads/master into add-internet-check
* Add check for internet to bat script
* Merge branch 'add-internet-check' of github.com:prysmaticlabs/prysm into add-internet-check
* Merge refs/heads/master into add-internet-check
2020-05-18 18:30:19 +00:00
Nishant Das
eb77f56d8a Sort Blocks Before Returning them To Peer (#5894)
* add tests and check
* lint
* Merge refs/heads/master into sortBlocks
2020-05-18 17:58:20 +00:00
Nishant Das
e16f384286 fix (#5893)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-18 20:16:27 +08:00
Ivan Martinez
5d8b5a3c30 Remove DB check for receiving p2p slashings (#5892)
* Removes DB check for receiving p2p slashings
* Fix bad logic
* Merge refs/heads/master into p2p-remove-db-check
* Whoops
* Merge branch 'p2p-remove-db-check' of github.com:prysmaticlabs/prysm into p2p-remove-db-check
2020-05-18 07:36:37 +00:00
Nishant Das
5bb9278a88 make them debug (#5891) 2020-05-18 02:27:18 -04:00
terence tsao
343bee35bc Make SlotsPerArchivedPoint configurable (#5888)
* Make slots per archived point configurable

* 256 -> 2048

* Use beaconconfig
2020-05-17 16:40:41 -04:00
Nishant Das
590016085d More Verbose Logs for Incoming Blocks (#5884)
* add logging
* truncate root
2020-05-17 09:07:21 +00:00
Victor Farazdagi
d79950ab3b Optimizes FSM event triggering (#5881)
* Optimizes machine triggering
* updates queue
* Merge branch 'master' into init-sync-fsm-optimizations
* gazelle
* Merge branch 'init-sync-fsm-optimizations' of github.com:prysmaticlabs/prysm into init-sync-fsm-optimizations
2020-05-16 20:26:00 +00:00
terence tsao
337ae6941f Enhancing block tree viewer (#5880) 2020-05-16 12:29:21 -07:00
Nishant Das
150f8beccd Revert "Add cache to Eth1 RPC functions (#5347)" (#5876)
* Revert "Add cache to Eth1 RPC functions (#5347)"

This reverts commit 159ef3d702.
2020-05-16 02:56:09 +00:00
Raul Jordan
8b168aa450 Add RPC Debug Server, Toggle Logging via gRPC (#5873)
* add in rpc debug server
* sets the valid debug level
* builds
* make post
* Merge branch 'master' into debuglog
* Merge refs/heads/master into debuglog
* caps
* Merge branch 'debuglog' of github.com:prysmaticlabs/prysm into debuglog
* comment
2020-05-15 21:54:53 +00:00
Victor Farazdagi
1cc7d67025 Adds rate limiting tests to beaconBlocksByRangeRPCHandler (#5866)
* overflow/nooverflow tests
* formatting
* check peer disconnecting
* more tests
* Merge branch 'master' into sync-rate-overflow-tests
* formatting
* Merge branch 'sync-rate-overflow-tests' of github.com:prysmaticlabs/prysm into sync-rate-overflow-tests
2020-05-15 20:51:23 +00:00
Raul Jordan
272f794868 Add Suggestion to Paste in Deposit Data into Prylabs/Participate (#5872)
* added instructions for account
2020-05-15 20:24:25 +00:00
terence tsao
5515c20ffd Add skip-regen-historical-states flag (#5871)
* Add feature flag

* Comments to clairfy a few things

* Use feature flag

* Update state gen resume

* More comments

* Remove from dv
2020-05-15 14:49:13 -05:00
Ivan Martinez
9333ede9a1 Unify generated mocks into shared/mock (#5862)
* Add validator mocks to script

* Add more mocks

* Generate all the mocks for Ivan

* Change script to use relative paths and gofmt

* bazel

* Remove duplicates

* Remove duplicated interfaces

* change to shared/mock

* Fix script and build

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-05-15 12:27:46 -05:00
Nishant Das
320bb46f35 More Handshake Cleanup (#5863)
* change to debug log

* disconnect on wrong fork

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-15 11:08:42 -05:00
Raul Jordan
007a98b711 Fix Stream Duties Server + Get Duties Before ChainStart (#5849)
* beacon node stream duties fixes

* comment

* fix critical seconds bug

* back to 12

* config changes

* Update beacon-chain/sync/initial-sync/service.go

* revert config

* Update beacon-chain/rpc/validator/assignments.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
2020-05-15 10:25:43 -05:00
Victor Farazdagi
f461f586a1 Refactors redundant global variables from sync (#5860)
* refactors global variables from sync
* gazelle
* Merge branch 'master' into sync-remove-globals
* better spacing
* Merge branch 'sync-remove-globals' of github.com:prysmaticlabs/prysm into sync-remove-globals
* Merge branch 'master' into sync-remove-globals
2020-05-15 08:53:19 +00:00
Ivan Martinez
52c302307a Remove out of date scripts and fix mockgen script (#5859) 2020-05-14 17:11:52 -05:00
Michael Huang
a7b441270d Update GetValidatorPerformance to support validator indices (#5687)
* add test for stream duties
* rem gomock
* Merge branch 'master' into stream-duties
* context cancelation test
* use interface for epoch ticker in duties
* fix build
* compute duties on first call
* pass tests for streams
* gaz
* ss stream duties
* Merge branch 'master' into stream-duties
* ensure only stream across epoch-wide reorgs
* regen mock for validator
* Update beacon-chain/blockchain/head.go
* Update RPC method GetValidatorPerformance to use validator indices
* Merge refs/heads/master into stream-duties
* Merge refs/heads/master into stream-duties
* Update beacon-chain/rpc/validator/server.go
* Merge branch 'master' into stream-duties
* Update commit hash for com_github_prysmaticlabs_ethereumapis
* Aggregate validator indices before appending performance summaries
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into validator-performance-by-indices
* Merge stream-duties
* Make function more test friendly
* Add indices tests
* fixup
* Fix conflicts
* Update gateway max resv size to 16MB (#5756)

* Update max resv size to 16MB
* Merge refs/heads/master into 5752
* Use GrpcMaxCallRecvMsgSizeFlag for beacon node
* Merge branch '5752' of github.com:prysmaticlabs/prysm into 5752
* Typo
* Fix server
* Merge refs/heads/master into 5752
* Merge refs/heads/master into 5752
* Merge refs/heads/master into 5752
* Merge refs/heads/master into 5752
* Update beacon-chain/gateway/server/main.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/master into 5752
* Rebase to origin/master
* Rebase to origin/master
* Merge remote-tracking branch 'origin/master' into validator-performance-by-indices
* Apply suggestions from code review
* Merge branch 'master' into validator-performance-by-indices
* Fix conflict
* fixup
* Fix tests
* Revert workspace file
* gofmt
* Revert executable permissions on beacon-chain/rpc/beacon/validators.go
* Merge branch 'master' into validator-performance-by-indices
* Merge branch 'master' into validator-performance-by-indices
* Merge branch 'master' into validator-performance-by-indices
* Merge branch 'master' into validator-performance-by-indices
* Add SyncChecker to tests
* Merge branch 'master' into validator-performance-by-indices
* Merge branch 'master' into validator-performance-by-indices
* Merge branch 'master' into validator-performance-by-indices
* Merge branch 'master' into validator-performance-by-indices
* Update beacon-chain/rpc/beacon/validators_test.go
* Update beacon-chain/rpc/beacon/validators_test.go
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into validator-performance-by-indices
* Add comment
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into validator-performance-by-indices
* Merge branch 'master' into validator-performance-by-indices
* Merge branch 'master' into validator-performance-by-indices
* Merge branch 'master' into validator-performance-by-indices
2020-05-14 20:52:42 +00:00
terence tsao
2f27966b44 Fix state gen migration (#5858)
* Remove useless `currentSplitSlot == 0`
* Increase slots per archived point to 2048
* Don't delete last archived state
* Revert back to 256
* Merge branch 'master' of github.com:prysmaticlabs/prysm into fix-migration-delete-latest-state
* Don't delete last archived state
* Regression test
* Revert
2020-05-14 20:03:14 +00:00
Ivan Martinez
7a8e85af9e Add testing for EnableFieldTrie in state_trie.go (#5820)
* Add testing for EnableFieldTrie
* Merge branch 'master' into test-field-trie
* Merge branch 'master' into test-field-trie
* Merge refs/heads/master into test-field-trie
* goimports
* Merge refs/heads/master into test-field-trie
* Merge refs/heads/master into test-field-trie
* Merge refs/heads/master into test-field-trie
* Merge refs/heads/master into test-field-trie
* Merge refs/heads/master into test-field-trie
* Merge refs/heads/master into test-field-trie
* Merge refs/heads/master into test-field-trie
* Merge refs/heads/master into test-field-trie
* Merge refs/heads/master into test-field-trie
* Merge refs/heads/master into test-field-trie
* Merge refs/heads/master into test-field-trie
* Merge refs/heads/master into test-field-trie
* Add tests for generic HTR
* Merge refs/heads/master into test-field-trie
2020-05-14 17:38:34 +00:00
Ivan Martinez
6e9839f987 Validate slashings upon pool exit (#5848)
* Validate slashings upon pool exit
* Change in implementation
* Fix test
* Merge branch 'master' into validate-slashing
* Merge refs/heads/master into validate-slashing
* Merge refs/heads/master into validate-slashing
* Merge refs/heads/master into validate-slashing
* Undo e2e changes
* Merge branch 'validate-slashing' of github.com:prysmaticlabs/prysm into validate-slashing
* Merge refs/heads/master into validate-slashing
2020-05-14 17:11:28 +00:00
terence tsao
f6d8633965 Revert "Check Whether Block Is Already Processed" (#5856)
* Revert "Check Whether Block Is Already Processed (#5850)"

This reverts commit 7630f1852f.
2020-05-14 16:37:58 +00:00
Nishant Das
20c2d78731 add new methods and tests (#5852)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-14 22:44:03 +08:00
Nishant Das
7630f1852f Check Whether Block Is Already Processed (#5850)
* check for missing blocks
* error log
* add test
* fix lint
* fix test
* terence's review
* do not mess with on block
* Merge refs/heads/master into addCheck
2020-05-14 14:11:35 +00:00
Victor Farazdagi
471779e4fc Fixes bug with excessive rate limiting (#5853)
* fixes bug with excessive rate limiting
2020-05-14 13:43:36 +00:00
Nishant Das
34c02ffd34 Allow a Larger Allowance for Peer Faults (#5847)
* allow larger allowance
* change to 5
* Merge branch 'master' into allowLargerAllowance
* test
2020-05-14 03:08:58 +00:00
Preston Van Loon
f831a799c5 Ensure proper block slot for process block header (#5846)
* Ensure proper block slot
* Merge refs/heads/master into extra-validation
2020-05-14 00:31:58 +00:00
terence tsao
dc20f3d72b Regen historical using ArchivedPointRoot then State (#5845)
* Regen from ArchivedPointRoot and State
* Merge branch 'master' into fix-regen-historical-states
2020-05-14 00:10:36 +00:00
terence tsao
70c6dbeb68 Delay atts until subsequent slot (#5833) 2020-05-13 15:46:39 -07:00
terence tsao
8ce01acbcf Check seed before shuffle (#5841) 2020-05-13 14:25:36 -07:00
Shay Zluf
ffa08f5a85 Stream p2p agg attestation (#5809)
* stream aggreagted attestations from p2p network to indexed attestation stream

* remove excessive log

* fix test

* handle nil attestation as well

* Update beacon-chain/sync/subscriber_beacon_aggregate_proof.go

* Update beacon-chain/sync/subscriber_beacon_aggregate_proof.go

* Update beacon-chain/sync/subscriber_beacon_aggregate_proof_test.go

* terence feedback

* sort imports

* sort imports

* Change to received buffer

* preston feedback

* error log

* raul feedback

* more logging changes

* fix duplicate package name

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
2020-05-13 12:21:53 -05:00
Nishant Das
08762af5a2 Revert "Use A Single State in Sync (#5764)" (#5837)
This reverts commit d5f43a8214.
2020-05-13 19:57:59 +08:00
Ivan Martinez
ca7e057ca7 Move cheaper operations earlier into code path (#5835)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-13 14:37:24 +08:00
Raul Jordan
578703fa05 Properly Divide SecondsPerSlot Into Milliseconds (#5821)
* use util for dividing slot durations

* fix places with dividing slot duration

* more dividers fixed

* format import

* fix formatting in validator test

* fix slot test tolerance

* nishant suggestions

* imports

* resolve e2e

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-13 11:28:17 +08:00
Nishant Das
d5f43a8214 Use A Single State in Sync (#5764)
* change this stuff

* Update beacon-chain/blockchain/process_block_helpers.go

* Update beacon-chain/blockchain/process_block_helpers.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-05-13 07:49:15 +08:00
Ivan Martinez
6488ab4a26 Add fast function for comparing attestation data (#5834)
* Add function for comparing attestation datas

* Fix imports

* Apply beneficial spots

* Gaz
2020-05-12 16:55:17 -04:00
Ivan Martinez
cd1a4f18f4 Fix historical detection for the slasher (#5831)
* Fix historical detection for the slasher
* More fixes
* Fix
* Run historical detection sequentially
* Add context err handler
* Add return
* Fixes
* Merge branch 'master' into slasher-fixes
2020-05-12 18:32:42 +00:00
terence tsao
9ffc6b71d6 Update spec tests to v0.11.2 (#5832) 2020-05-12 12:44:17 -05:00
Connor Stein
4242581f1e Test EnterPassword function in shared/cmd (#5822)
* test

* fix golint

* more lint

* gazelle

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-05-12 10:01:02 -05:00
Nishant Das
ab23fadc5e Revert Change Added in #5811 (#5828)
* revert

* remove
2020-05-12 21:39:36 +08:00
Victor Farazdagi
baf9ec60eb Adds BlockBatchLimitBurstFactor flag (#5825)
* adds BlockBatchLimitBurstFactor flag
* Merge branch 'master' into burst-factor-flag
2020-05-12 11:31:10 +00:00
Victor Farazdagi
79ce15df63 Removes deprecated flag from config (#5824)
* removes deprecated flag from config
2020-05-12 10:53:24 +00:00
terence tsao
96219fc9fa Add metric for reorg count (#5819)
* Add reorg counter metric
* Merge branch 'master' into reorg-metric
* Merge branch 'master' into reorg-metric
2020-05-12 05:48:16 +00:00
Nishant Das
edfb32021b Validate Finalized Root in Status RPC Requests (#5811)
* additional status validation

* changes

* clean up

* Update beacon-chain/db/kv/finalized_block_roots.go

* Update beacon-chain/sync/rpc_status.go

* Update beacon-chain/sync/rpc_status.go

* Update beacon-chain/sync/rpc_status.go

* terence's review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-12 11:22:01 +08:00
Preston Van Loon
40591924c2 Add attestation inclusion histogram (#5818)
* Add attestation inclusion histogram
* Merge branch 'master' into attestaiton-inclusion-histogram
2020-05-12 02:00:42 +00:00
Ivan Martinez
946a2caafd Add fix for nil state in InitializeFromProto (#5817)
* Add nil check and tests
* Add unsafe test
* Update tools/genesis-state-gen/main.go
* Undo genesis state gen changes
* Merge branch 'proto-fuzz-fix' of github.com:prysmaticlabs/prysm into proto-fuzz-fix
* gaz
* Fix
2020-05-12 00:23:05 +00:00
terence tsao
b5e7f9dd6d Run go fmt in prysm (#5815)
* go fmt
2020-05-11 22:03:15 +00:00
terence tsao
4c9ff7012c Reduce state copies for SubmitAggregateSelectionProof (#5813)
* `GetValidatorPerformance` return `Unavailable` when syncing
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Reduce state copies for SubmitAggregateAndProof
* Merge branch 'master' into reduce-state-copies
2020-05-11 16:21:34 +00:00
Raul Jordan
42278359d9 Add Docker Image for Genesis State Generator (#5697)
* add test for stream duties
* rem gomock
* Merge branch 'master' into stream-duties
* context cancelation test
* use interface for epoch ticker in duties
* fix build
* compute duties on first call
* pass tests for streams
* gaz
* ss stream duties
* Merge branch 'master' into stream-duties
* ensure only stream across epoch-wide reorgs
* regen mock for validator
* Update beacon-chain/blockchain/head.go
* Merge refs/heads/master into stream-duties
* Merge refs/heads/master into stream-duties
* Update beacon-chain/rpc/validator/server.go
* Merge branch 'master' into stream-duties
* add docker
* gaz
* update workspac
* Merge branch 'stream-duties' into state-gen-docker
* pure off
* Merge branch 'state-gen-docker' of github.com:prysmaticlabs/prysm into state-gen-docker
* gen
* base img
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* fix confs
* fix up confs
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
* Merge refs/heads/master into state-gen-docker
2020-05-11 12:00:42 +00:00
Nishant Das
1110a5caab minor fix (#5810) 2020-05-11 19:36:42 +08:00
Nishant Das
6840afb45e Fix Issues With Peer Handshakes (#5799)
* add check

* make handshake better

* fix a lot of things

* Update beacon-chain/p2p/handshake.go

* Update beacon-chain/p2p/handshake.go

* fix things

* fix

* fix handshake

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-11 12:29:23 +08:00
terence tsao
34f28a31b1 No magic number for BlockBodyRoot (#5803)
* `GetValidatorPerformance` return `Unavailable` when syncing

* Use proper config values for BlockBodyRoot

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-11 11:45:31 +08:00
Preston Van Loon
bfe694c4f3 Check for pending deposits and exit early, if possible (#5802)
* Check for pending deposits and exit early, if possible
2020-05-11 02:38:41 +00:00
terence tsao
c000509bdb GetValidatorPerformance return Unavailable when syncing (#5795) 2020-05-10 13:08:45 -07:00
Victor Farazdagi
e53452a437 Moves connectPeers to common test setup file (#5800)
* moves connectPeers
2020-05-10 14:47:05 +00:00
Nishant Das
aeb6dc13ac Add Ability to Restrict Outbound Dials (#5794)
* blacklist cidr
* add test
* Merge refs/heads/master into p2pBlacklist
2020-05-09 22:18:21 +00:00
Ivan Martinez
c82f2d5657 Fix test setup and change participation min back to 100% (#5791)
* Increase time for test to start

* Try anothre fix

* Set minimum participation back to 100%

* Change bnode count to 2

* Give extra wiggle room for metrics

* Remove comment
2020-05-09 15:11:10 -05:00
terence tsao
419343123a Fix e2e flakynes for new state mgmt (#5790) 2020-05-08 21:56:51 -07:00
Nishant Das
a25354adf0 Remove ETH1 Data Vote Cache (#5792)
* remove cache
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into removeETH1DataCache
* remove everything
2020-05-09 04:08:24 +00:00
Ivan Martinez
26eead0d1c Gate disable lookback behind featureflag (#5782)
* Gate disable lookback behind featureflag
* Merge branch 'master' into slasher-lookback-flag
* Update shared/featureconfig/config.go
* Merge branch 'master' into slasher-lookback-flag
* Add context timeout error handling
* Merge branch 'slasher-lookback-flag' of github.com:prysmaticlabs/prysm into slasher-lookback-flag
* gaz
* Merge branch 'master' into slasher-lookback-flag
* Merge branch 'master' into slasher-lookback-flag
* Merge refs/heads/master into slasher-lookback-flag
* Merge refs/heads/master into slasher-lookback-flag
* Merge refs/heads/master into slasher-lookback-flag
* Merge refs/heads/master into slasher-lookback-flag
2020-05-09 02:07:37 +00:00
Preston Van Loon
c6cac96a3b Duplicate init sync test target for better coverage reporting (#5789)
* duplicate target for better coverage testing
* Merge refs/heads/master into init-sync-raceoff
2020-05-09 01:40:14 +00:00
Preston Van Loon
87e0fe5928 Refactor ReceiveBlock to receive the block root. (#5785)
* refactor ReceiveBlockNoPubsub to receive block root
* Refactor ReceiveBlock to receive block root
* A few other minor refactoring to reduce block HTR
* use arg, remove HTR
* more slight refactoring, comments
* fix test build
* Merge refs/heads/master into receive-with-root
* Merge refs/heads/master into receive-with-root
2020-05-09 01:22:59 +00:00
Nishant Das
5b134239e3 Make IP Static for Public IPs (#5788)
* don't use endpoint predictor
* Merge branch 'master' into staticIP
* Merge refs/heads/master into staticIP
2020-05-09 00:58:11 +00:00
terence tsao
c590029e48 Temporarily remove new state mgmt from e2e (#5786)
* Temporarily remove new state mgmt from e2e
2020-05-09 00:28:06 +00:00
Preston Van Loon
b00bbfeec7 Release enableCustomBlockHTR to all (#5742)
* Release deprecatedEnableCustomBlockHTR to all

* Release deprecatedEnableCustomBlockHTR to all

* Fix builds

* Treat nil body as empty

* Use custom HTR in signing root

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-08 13:50:39 -07:00
terence tsao
7a27d89e68 Reduce memory usage for new-state-mgmt at init sync (#5778)
* Add `GetWithoutCopy`
* Add `StateByRootInitialSync`
* Use `StateByRootInitialSync` for initial syncing using new-state-mgmt
* Merge branch 'master' into new-state-init-sync-mem
* Skip if split slot is 0
* Merge branch 'new-state-init-sync-mem' of github.com:prysmaticlabs/prysm into new-state-init-sync-mem
* Make sure we invalidate cache
* Add test part 1
* Add tests part 2
* Update beacon-chain/cache/hot_state_cache.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
* Comment
* Merge branch 'new-state-init-sync-mem' of github.com:prysmaticlabs/prysm into new-state-init-sync-mem
* Merge branch 'master' of github.com:prysmaticlabs/prysm into new-state-init-sync-mem
* Dont need to run fork choice and save head during intial sync
* Invalidate cache at onBlockInitialSyncStateTransition
* Merge branch 'new-state-init-sync-mem' of github.com:prysmaticlabs/prysm into new-state-init-sync-mem
* Revert saveHeadNoDB changes
* Add back DeleteHotStateInCache
* Removed extra deletion
* Merge branch 'master' of github.com:prysmaticlabs/prysm into new-state-init-sync-mem
* Proper set splitslot for tests
* Merge branch 'master' into new-state-init-sync-mem
* Merge branch 'master' of github.com:prysmaticlabs/prysm into new-state-init-sync-mem
* Merge branch 'new-state-init-sync-mem' of github.com:prysmaticlabs/prysm into new-state-init-sync-mem
2020-05-08 19:02:48 +00:00
Ivan Martinez
ad8716a58e Refactor validator protection for attesters (#5695)
* Finish refactor for attestation protection

* Finish refactor for proposal history

* Revert "Finish refactor for proposal history"

This reverts commit 2f13720063.

* Fix tests

* Implement UpdateProtections

* Implement refactor

* Fixes

* fix

* Undo proposer changes

* Fix test

* Final look through

* Add tests for protections function

* Add lock

* Add flag in front of attester protection code

* Add proper rwlocks and fix flags

* fix

* fix build

* Fix atestation tests

* Fix deprecated flags

* Add protect attester to standard attesting test

* Remove comment

* gofmt

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-08 10:16:53 -05:00
Ivan Martinez
03c7a6c0e2 Validator status improvements and fix (#5775)
* Validator status optimization and bug fix

* Remove uneeded state check

* Change val function to read only

* Change to read only

* Undo error change

* Undo deposit change
2020-05-08 10:04:01 -04:00
Nishant Das
6fed51d380 Don't Copy The Whole Validator Map each time (#5780)
* fix copying issues

* revert some changes
2020-05-08 18:14:18 +08:00
Shay Zluf
c1a8b41347 Is slashable block rpc endpoint implementation (#5765)
* is slashable block rpc endpoint implementation

* add span

* raul feedback

* fix error

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-08 08:51:56 +03:00
Preston Van Loon
63e2e8acd7 Use alpine base image including libstdc++ (#5777)
* fix #5768 with an alpine base including libstdc++
* Merge branch 'master' into fix-docker-alpine
2020-05-07 22:01:48 +00:00
Ivan Martinez
9a1157465e Change non-mutating validator accesses to ReadOnly (#5776)
* Change instances of ValidatorAtIndex to ReadOnly where possible

* Use ReadOnly for VerifyExit and Slashings

* Move length check to before lock

* Improve readonly tests

* undo process attester changes

* Fix test
2020-05-07 14:15:51 -04:00
Ivan Martinez
574bf3deac Clean usage of cli.context in validator/node (#5758)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-07 10:14:08 -05:00
Victor Farazdagi
40c29744e3 Init-sync discard debug logs in tests (#5774)
* refactors block fetching function to fetcher
* more comments fixed
* Merge branch 'master' into init-sync-optimizations
* moves helper code into separate file
* move test setup code to testmain
* removes bench
2020-05-07 13:13:45 +00:00
Victor Farazdagi
f0c0ed80cb BLS code cleanup (#5773)
* removes unused code
* add specs comments
* removes unused VerifyAggregate
* fixes benchmark test
* Merge branch 'master' into blc-cleanup
* updates benchmark test
* Merge branch 'master' into blc-cleanup
2020-05-07 12:05:53 +00:00
Nishant Das
f6090d0a38 Update Bootnode to v0.11.2 (#5767)
* update to v0.11.2

* fix test

* docker

* Update BUILD.bazel

* Update BUILD.bazel

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-07 18:50:03 +08:00
Preston Van Loon
87c3599c33 Update BLS and use constant for SetETHmode (#5772)
* Update BLS and use constant for SetETHmode
2020-05-07 07:49:24 +00:00
Ivan Martinez
4d9da25d02 Add confirm password to validator accounts create (#5762)
* Add confirm password to accounts create

* Fix build

* Update shared/cmd/helpers.go

* Update shared/cmd/helpers.go

Co-authored-by: Shay Zluf <thezluf@gmail.com>

* Change to return empty ""

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Shay Zluf <thezluf@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-06 19:50:19 -04:00
Ivan Martinez
b2ee466f79 Add--disable-historical-detection flag to slasher (#5759)
* Add--disable-historical-detection flag
* Merge branch 'master' into slasher-add-hist-flag
* Fix for comments
* Add flag to usage help
* Merge branch 'master' into slasher-add-hist-flag
* Update slasher/usage.go
* Fix imports
* Merge branch 'master' into slasher-add-hist-flag
* Merge branch 'master' into slasher-add-hist-flag
* Merge branch 'master' into slasher-add-hist-flag
2020-05-06 22:17:36 +00:00
Victor Farazdagi
c4eb8c7a16 Refactors block fetching function to fetcher (#5766)
* refactors block fetching function to fetcher
* more comments fixed
* Merge refs/heads/master into init-sync-upd
* Merge refs/heads/master into init-sync-upd
* Merge refs/heads/master into init-sync-upd
2020-05-06 21:29:50 +00:00
terence tsao
38f2ec6d7d Update gateway max resv size to 16MB (#5756)
* Update max resv size to 16MB
* Merge refs/heads/master into 5752
* Use GrpcMaxCallRecvMsgSizeFlag for beacon node
* Merge branch '5752' of github.com:prysmaticlabs/prysm into 5752
* Typo
* Fix server
* Merge refs/heads/master into 5752
* Merge refs/heads/master into 5752
* Merge refs/heads/master into 5752
* Merge refs/heads/master into 5752
* Update beacon-chain/gateway/server/main.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/master into 5752
2020-05-06 21:01:23 +00:00
terence tsao
ef4dead3fe Fix cold state generation (#5770) 2020-05-06 13:25:37 -07:00
Raul Jordan
d5b1f9f1f9 Implement Stream Duties Functionality (#5685)
* add test for stream duties

* rem gomock

* context cancelation test

* use interface for epoch ticker in duties

* fix build

* compute duties on first call

* pass tests for streams

* gaz

* ss stream duties

* ensure only stream across epoch-wide reorgs

* regen mock for validator

* Update beacon-chain/blockchain/head.go

* Update beacon-chain/rpc/validator/server.go

* update workspac

* pregenesis check

* tests pass

* build fix

* Revert "tests pass"

This reverts commit 9029d63ef3.

* ethereumapis

* gaz

* Update beacon-chain/rpc/validator/assignments.go

* test for reorg

* fix up required tests

* fmt

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-05-06 14:15:31 -05:00
Ivan Martinez
67a2698463 Remove flag requirement from accounts keys (#5757)
* Remove flag requirements for keystore-path and password

* Add test for HandleEmptyFlags

* Add comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-06 01:55:25 -04:00
Nishant Das
ae66f63ec2 Memory Improvements (#5760)
* fix methods
* fix path for only initial sync
* update comment
* Merge refs/heads/master into memImprovements
2020-05-06 05:04:02 +00:00
Nishant Das
bf2a37b27f Remove Old Sync Routine (#5761)
* remove old sync
2020-05-06 04:36:34 +00:00
terence tsao
aea7a8d291 Future slot check for state end point (#5755) 2020-05-05 18:42:11 -07:00
Raul Jordan
840bfc5f6b Add Chain Config File for Validator (#5751)
* builds for config file
2020-05-05 22:19:27 +00:00
terence tsao
f29ea77a14 Update to v0.11.2 badge (#5750)
* Update to v0.11.2 badge
* Merge branch 'master' into terencechain-patch-1
2020-05-05 20:36:53 +00:00
Ivan Martinez
2b7865cb3b Move "Enter a password" cmd instructions to shared/cmd (#5724)
* Move EnterPassword CMD to cmd package

* Add comment

* Add tracking issue and remove log.Fatal

* Add better error handling

* gaz

* Comment fix

* Update shared/cmd/helpers.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-05-05 16:13:18 -04:00
Nishant Das
3a677ef342 Add Persistent Subnets (#5734)
* add params

* add changes

* bug fixes

* fix method

* get new assignments

* add test and comments

* change to slice of uint64

* add test

* lint

* Update beacon-chain/rpc/validator/assignments_test.go

* Update beacon-chain/cache/committee_ids.go

* Update beacon-chain/rpc/validator/assignments.go

* add comment

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-05 14:26:20 -05:00
Shay Zluf
1a27c21d9c Load chain config from file (#5694)
* load chain config from file

* revert flag name change

* add dependencies to image

* Update shared/cmd/flags.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

* Update beacon-chain/main.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

* added test to load config file

* Merge branch 'yaml_chain_config' of github.com:prysmaticlabs/prysm into yaml_chain_config

# Conflicts:
#	beacon-chain/main.go

* replace hex with yaml format of fixed byte array

* fix test and check if comment

* move to node package

* gaz

* added contract address case

* fix key name issue

* add tests

* gaz

* add 1 byte handling

* change to fatal

* add config printout

* revert main changes

* revert line removal

* fix one byte handling

* fix test and one byte handling

* remove log

* Apply suggestions from code review

* change to debug

* Update beacon-chain/node/node.go

* move helper methods

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-05-05 20:58:07 +03:00
Victor Farazdagi
45f2447e6a More explicit error message on wrong process when running prysm.sh (#5747)
* exit when incorrect command passed
* more explicit error message
* grammar
2020-05-05 13:23:09 +00:00
Nishant Das
2eac24cb79 Retry Initialization of ETH1 Connection (#5730)
* retry failures
* Merge branch 'master' into retryInitialization
* fix test
* Merge branch 'retryInitialization' of https://github.com/prysmaticlabs/geth-sharding into retryInitialization
* Merge refs/heads/master into retryInitialization
* Merge refs/heads/master into retryInitialization
* Merge refs/heads/master into retryInitialization
* Merge refs/heads/master into retryInitialization
* Merge refs/heads/master into retryInitialization
* Merge refs/heads/master into retryInitialization
* fix method
* Merge branch 'retryInitialization' of https://github.com/prysmaticlabs/geth-sharding into retryInitialization
* Merge refs/heads/master into retryInitialization
* Merge refs/heads/master into retryInitialization
* Merge refs/heads/master into retryInitialization
* Merge refs/heads/master into retryInitialization
2020-05-05 09:06:43 +00:00
Victor Farazdagi
bbde2a6820 Assert no side effects occur on test (#5714)
* add reset funcs
* init -> TestMain for beacon config params
* fixes build file
* core/epoch spectests updated
* core/blocks spectests updated
* fix the rest of spectests
* SetupTestConfigCleanup
* revert copy() api
* rever all cases of copy()
* fixes spectests
* updates fork_test
* config_test update
* fixes spectest/config_test
* Merge refs/heads/master into assert-no-side-effects-occur-in-tests
* Merge branch 'master' into assert-no-side-effects-occur-in-tests
* Merge refs/heads/master into assert-no-side-effects-occur-in-tests
* Merge refs/heads/master into assert-no-side-effects-occur-in-tests
* Merge refs/heads/master into assert-no-side-effects-occur-in-tests
* Merge refs/heads/master into assert-no-side-effects-occur-in-tests
* Merge refs/heads/master into assert-no-side-effects-occur-in-tests
* Merge refs/heads/master into assert-no-side-effects-occur-in-tests
* Merge refs/heads/master into assert-no-side-effects-occur-in-tests
* Merge branch 'master' into assert-no-side-effects-occur-in-tests
* Merge refs/heads/master into assert-no-side-effects-occur-in-tests
* Merge refs/heads/master into assert-no-side-effects-occur-in-tests
2020-05-05 08:39:38 +00:00
Nishant Das
0b8ee54076 Add More Flags To Dev Mode (#5744)
* add flags
* Merge refs/heads/master into addDevFlags
* Merge refs/heads/master into addDevFlags
2020-05-05 08:00:56 +00:00
Preston Van Loon
eb2abbdd8b libfuzz based tests (#5095)
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* use opt for fuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge branch 'go-1.14-libfuzz' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* use ubuntu as the default build image
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge branch 'master' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* Move fuzz tests
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge branch 'go-1.14-libfuzz' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* making progress on p2p fuzz
* Merge branch 'go-1.14-libfuzz' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* use a single stream repeatedly
* use a single stream repeatedly
* use many streams
* fixes
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge branch 'master' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* Merge branch 'master' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* Add basic test for ssz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge branch 'master' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge branch 'master' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Use a custom go_library to add libfuzzer
* delete old corpus, fix rpc_status_fuzz
* move fuzz.bzl
* minor fixes
* Merge branch 'go-1.14-libfuzz' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* add c-shared library (doesnt work)
* Merge branch 'go-1.14-libfuzz' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* fix build
* fixes
* Merge refs/heads/master into go-1.14-libfuzz
* lint
* more lint
* Merge branch 'go-1.14-libfuzz' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* more lint
* allow environment variable
* Merge refs/heads/master into go-1.14-libfuzz
* Add a fuzzit image with the beacon_states
* Merge branch 'go-1.14-libfuzz' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* lint
* readme
* lint
* fix lint again i think
* gaz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* breakup deps
* Merge branch 'go-1.14-libfuzz' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* don't panic on failure
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* panic if no beacon states to read
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* update rules_foreign_cc
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* remove c_shared target, manually tag tests
* Add readme to fuzz package
* delete a few comments
* delete a few comments
* lint
* gaz
* gaz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
2020-05-05 07:22:26 +00:00
Preston Van Loon
eb27f35982 Use the custom HTR in a few more places (#5743) 2020-05-05 14:42:53 +08:00
Shay Zluf
9ea3f58a5e Verify sig slasher (#5661)
* first cherry pick

* second

* new endpoint

* fork digest by epoch

* fork and sig verify

* fix test

* fix comment

* gaz

* remove unused param

* remove space

* moved to shared attestation_utils.go

* raul comment

* gaz

* goimports

* fix import cycle

* fix test

* shorter helper method

* add another helper function

* fix comment

* param fix

* gaz

* fix error

* Update beacon-chain/core/blocks/block_operations.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

* update pseudocode

* remove teardown

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-05-05 08:15:32 +03:00
Preston Van Loon
9be8a456eb Use stateutil.BlockRoot everywhere (#5739)
* Sweeping replace of ssz.HashTreeRoot(block) to stateutil.BlockRoot, minor tweaks to critical path of proposeBlock
* imports
* Merge branch 'master' into better-block-htr
* Add --enable-custom-block-htr to e2e flags
* Merge branch 'better-block-htr' of github.com:prysmaticlabs/prysm into better-block-htr
2020-05-05 04:30:24 +00:00
terence tsao
4958c0372c Add sent block time metric (#5738) 2020-05-04 19:00:00 -07:00
Shay Zluf
1fa495afc1 Add params and update spec param names (#5732)
* add params and update names
* Merge branch 'master' into update_params
* Merge refs/heads/master into update_params
2020-05-05 00:14:43 +00:00
terence tsao
e8da03a6b2 Use AttestationPreState (#5737) 2020-05-04 18:48:14 -05:00
Ivan Martinez
d3cb4b3059 Log out public key with accounts create (#5723)
* Log public key with deposit data
* Merge refs/heads/master into log-pub-deposit
* Merge refs/heads/master into log-pub-deposit
* Merge refs/heads/master into log-pub-deposit
* Merge refs/heads/master into log-pub-deposit
* Merge refs/heads/master into log-pub-deposit
* Update validator/accounts/account.go

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2020-05-04 15:00:35 +00:00
Victor Farazdagi
140aff4398 fixes runtime panic in state/getters (#5728)
* fixes runtime panic in state/getter
* Merge branch 'master' into check-non-nil-in-state-getter
* return empty map
* Merge branch 'check-non-nil-in-state-getter' of github.com:prysmaticlabs/prysm into check-non-nil-in-state-getter
2020-05-04 10:15:33 +00:00
Ivan Martinez
c614412885 Remove silent errors from slasher (#5722)
* Remove silent errors from slasher
* Merge refs/heads/master into slasher-silent-errors
* Merge refs/heads/master into slasher-silent-errors
* Merge refs/heads/master into slasher-silent-errors
2020-05-04 08:17:18 +00:00
Nishant Das
1432332923 Return Genesis Block (#5709)
* add check
* lint
* lint
* Merge branch 'master' into returnGenesis
* Merge refs/heads/master into returnGenesis
* Merge refs/heads/master into returnGenesis
* Merge refs/heads/master into returnGenesis
* Merge refs/heads/master into returnGenesis
* Merge refs/heads/master into returnGenesis
* terence's review
* Merge branch 'returnGenesis' of https://github.com/prysmaticlabs/geth-sharding into returnGenesis
* Merge refs/heads/master into returnGenesis
* Merge refs/heads/master into returnGenesis
* fix failing test
2020-05-04 07:54:31 +00:00
Victor Farazdagi
b5c4dc2a75 Refactor db teardown to testing.TB.Cleanup (#5725)
* init-sync updates
* slasher/db/kv tests
* beacon-chain/rpc/beacon tests
* update kv_test
* beacon-chain/rpc-validator tests updated
* slasher/db/kv - remove teardown method
* beacon-chain/sync tests updated
* beacon-chain/db/kv tests updated
* beacon-chain/blockchain tests updated
* beacon-chain/state/stategen tests updated
* beacon-chain/powchain updates
* updates rest of slasher tests
* validator/db tests
* rest of the tests
* minor comments update
* gazelle
* Merge refs/heads/master into teardowndb-to-cleanup
2020-05-04 01:14:34 +00:00
terence tsao
17df25f3d7 Add more domains to domain cache (#5726) 2020-05-03 17:34:25 -07:00
Victor Farazdagi
af5b9222da Re-enable skipped block list paging unit test (#5691)
* unskip on a single shard
* Merge branch 'master' into unskip-block-list-test
* fixes incorrect test state modification
* Merge branch 'master' into unskip-block-list-test
* minor formatting
* Merge branch 'unskip-block-list-test' of github.com:prysmaticlabs/prysm into unskip-block-list-test
* set sharding back to 4
* Merge refs/heads/master into unskip-block-list-test
* Merge refs/heads/master into unskip-block-list-test
* Merge refs/heads/master into unskip-block-list-test
2020-05-02 04:03:50 +00:00
terence tsao
42961277e1 Add histogram metrics for block arrival time (#5708)
* Add metrics to track block propogation delay

* Grammar

* Grammar

* Metrics for block sent

* Refactor into own function

* Removed sent

* Lint

* Use histogram. Thanks Preston for the suggestion!

* s/total/histogram

* Update name to `block_arrival_latency_milliseconds`

* Use milliseconds

* Use unix nano

* Use SlotToTime

* Preston's feedback

* Handle err
2020-05-01 19:35:15 -07:00
terence tsao
973fa9f326 Regen historical state: don't skip saving historical state (#5712)
* Save every state at archived point
* Merge branch 'master' into save-every-state
2020-05-02 00:36:54 +00:00
Preston Van Loon
37f2a04e51 SlotToTime helper (#5710)
* SlotToTime helper
* gofmt
* gaz
* overflow
* Merge refs/heads/master into better-time
* Fix tests, move overflow
* Merge branch 'better-time' of github.com:prysmaticlabs/prysm into better-time
* smaller diff
2020-05-01 23:19:43 +00:00
Ivan Martinez
125d022518 Rename Precompute Balance vars to be more descriptive (#5706)
* Rename p *Balance and some Balance vars
* Rename more
* Fix comments
* Fix typo
* Forgot a rename
2020-05-01 19:43:04 +00:00
Victor Farazdagi
b42cc724c5 Enforce validator's keystore dir (#5707)
* enforces keystore dir
* Merge refs/heads/master into recheck-before-raw-tx-data
* do not ignore error
* Merge branch 'recheck-before-raw-tx-data' of github.com:prysmaticlabs/prysm into recheck-before-raw-tx-data
2020-05-01 15:12:26 +00:00
Ivan Martinez
436d545f80 Move recreated constants out of smaller scopes (#5699) 2020-05-01 06:45:24 -07:00
Nishant Das
2c3f974361 Custom Method For Block Signing Root (#5704)
* custom block signing root

* add comment and test

* Update beacon-chain/core/helpers/signing_root.go

* Update beacon-chain/core/helpers/signing_root.go

* fix changes
2020-05-01 03:37:37 -04:00
Nishant Das
7bea237397 Revert Validator Reference Copy (#5701)
* revert all changes
* revert test
* unreachable code
* Merge refs/heads/master into revertValidatorRefCopy
* Merge refs/heads/master into revertValidatorRefCopy
2020-05-01 02:48:46 +00:00
wgknowles
c877efd90e replaced 265 with 256 ...derp (#5702)
* replaced 265 with 256 ...derp
2020-05-01 02:36:52 +00:00
Raul Jordan
2e33595187 Implement GetBeaconState Endpoint (#5668)
* implement get beacon state
* gaz
* Merge branch 'master' into implement-debug-state
* passing tests
* enable with featureconfig
* struct oder
* Update beacon-chain/rpc/beacon/state.go
* Merge refs/heads/master into implement-debug-state
* lint resolve
* Merge branch 'implement-debug-state' of github.com:prysmaticlabs/prysm into implement-debug-state
* tested at runtime
* fix build
* Merge branch 'master' into implement-debug-state
* Merge refs/heads/master into implement-debug-state
* Merge refs/heads/master into implement-debug-state
* Merge refs/heads/master into implement-debug-state
* Merge refs/heads/master into implement-debug-state
* Merge refs/heads/master into implement-debug-state
* build and fmt
* conf
* Merge refs/heads/master into implement-debug-state
* Merge refs/heads/master into implement-debug-state
* Merge refs/heads/master into implement-debug-state
* Merge refs/heads/master into implement-debug-state
* Merge refs/heads/master into implement-debug-state
2020-05-01 01:47:10 +00:00
Shawki Sukkar
e1ac1d3d80 Log out important directories for validator client (#5653)
* .
* print in beacon-chain, validator and slasher
* add source and build
* fix
* fix
* fix
* fix
* run gazelle
* change log
* change log file
* change log file
* change log message
* change log message
* Merge branch 'master' into Display-key-paths-at-startup
* change the logging code place
* change datadir to out of scoop
* Merge branch 'master' into Display-key-paths-at-startup
* Merge branch 'master' into Display-key-paths-at-startup
2020-04-30 23:17:06 +00:00
terence tsao
fdad7e67b0 Update pubsub seen cache to be per epoch instead of per slot (#5698)
* hasSeenAggregatorIndexSlot -> hasSeenAggregatorIndexEpoch
* Fix test
* Update subscriber
* setSeenCommitteeIndicesSlot -> setSeenCommitteeIndicesEpoch
* Fix test
* Revert "setSeenCommitteeIndicesSlot -> setSeenCommitteeIndicesEpoch"

This reverts commit bd638ae556.
* Fixed unaggregated att seen cache to use per slot
2020-04-30 22:16:14 +00:00
Victor Farazdagi
cc07494e67 shared/keystore: cleanup (#5689)
* fixes inconsistent naming

* removes unused methods

* proper temp keystore cleanup in tests

* more naming fixes

* one more fix of keystore teardown in tests

* more robust teardown
2020-04-30 10:00:37 -05:00
Nishant Das
5f53f9fde1 Clean Up Of Dynamic Subscriptions (#5690)
* fix subscriptions

* add reg test
2020-04-30 09:14:59 -05:00
Nishant Das
3a295fb3ec Fix Voluntary Exits Panic (#5688)
* add fix and regression test
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Shay Zluf <thezluf@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Shay Zluf <thezluf@gmail.com>
* Update exit_test.go
2020-04-30 07:57:49 +00:00
Ivan Martinez
2f88174d0d Unify usage of context in slasher client (#5684)
* Unify usage of context in slasher client

* Fix build

* Add error handlers so all processes stop

* Remove unneeded checks
2020-04-29 19:27:01 -04:00
Raul Jordan
546196a6fa Other Package Godocs for Prysm (#5681)
* e2e docs
* slasher docs
* Merge branch 'other-package-godocs' of github.com:prysmaticlabs/prysm into other-package-godocs
* all validator package comments
* Merge branch 'master' into other-package-godocs
* completed all other packages
* Merge branch 'master' into other-package-godocs
* Merge refs/heads/master into other-package-godocs
2020-04-29 21:32:39 +00:00
Nishant Das
4d30519dda fix bugs (#5683) 2020-04-29 14:28:22 -05:00
Victor Farazdagi
2c1e3aa4ee Validator key files: allow symlinks (#5679)
* traverse symlinks
* Merge branch 'master' into account-keys-symlinks
* unit test
* Merge branch 'account-keys-symlinks' of github.com:prysmaticlabs/prysm into account-keys-symlinks
* Merge refs/heads/master into account-keys-symlinks
* removes unncecessary line
* Merge branch 'account-keys-symlinks' of github.com:prysmaticlabs/prysm into account-keys-symlinks
* fixes tests
* better teardown of resources
2020-04-29 18:31:22 +00:00
Raul Jordan
3416962fc2 All Beacon Node Package-Level Godocs (#5677)
* package level godocs fixed

* all beacon node godocs

* comment and gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-04-29 12:40:33 -05:00
Raw Pong Ghmoa
3fc6afa02e beacon/log: augment logs with block roots (#5675)
* beacon/log: augment synced log with block root and finalized root
* beacon/log: augment initial sync log with block root
* beacon/sync: degrade pending block log to DEBUG level
* beacon/log: restore info level for state transition
* beacon/log: improve info logging
* Merge branch 'master' into q9-log-block
* Merge branch 'master' into q9-log-block
2020-04-29 16:57:04 +00:00
terence tsao
05fcb1073a Remove duplicated sig verification (#5676)
* Remove duplicated sig verification
2020-04-29 15:57:58 +00:00
Nishant Das
81a7bc7e05 Add back State To ComputeProposerIndex (#5674)
* add back state
* fix tests
* change method signatures
* lint
* Merge branch 'master' into addBackState
* Merge branch 'master' into addBackState
* terence's review
* preston's review
* add back comment
2020-04-29 15:22:52 +00:00
Ivan Martinez
84e51a5236 Add prometheus metrics test into E2E (#5673)
* Progress on metrics tests
* Progress on metrics test
* Get metrics E2E working
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into e2e-metrics
* Complete most of metrics tests
* Change E2E polling to the middle of a slot, instead of at the start of the middle
* Add metrics to all E2E
* Remove extra types
* Update endtoend/evaluators/metrics.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge branch 'master' into e2e-metrics
* Add more comments, address feedback
* Merge branch 'e2e-metrics' of https://github.com/prysmaticlabs/prysm into e2e-metrics
* Fix build
* Remove unneeded comment
* Set E2E_EPOCHS back
* Improve sync testing reliability
* Remove metrics check from slashing
* Improve time allotted to sync
* Remove possibly flaky sync test change
2020-04-29 13:42:12 +00:00
Raul Jordan
3b2c514c06 Empty Committee Check in Submit Attestation (#5672)
* empty committee check
* Merge refs/heads/master into validator-id-errs
2020-04-29 02:44:51 +00:00
Raul Jordan
94155191b7 Ensure Single Context Used for Beacon Node Cancelation on SIGTERM (#5671)
* ensure single point of entry for ctx and cancelation of expensive func
2020-04-29 02:28:43 +00:00
terence tsao
5636cd3ed8 More efficient ancestor head retrieval for GetAttestationData (#5669)
* Use `BlockRootAtSlot` to look up historical head root

* Update test

* Typo
2020-04-28 19:44:06 -05:00
terence tsao
44611e0fb2 A tool to update genesis time of an existing state (#5664)
* Add a tool to update genesis time
* Minor touchups
* Merge branch 'master' into update-genesis-time
* Added a readme
* Merge refs/heads/master into update-genesis-time
* Merge branch 'update-genesis-time' of github.com:prysmaticlabs/prysm into update-genesis-time
* Merge refs/heads/master into update-genesis-time
* Merge refs/heads/master into update-genesis-time
2020-04-28 20:55:49 +00:00
Raul Jordan
5ed72d4ef9 Add Debug RPC Service to Prysm (#5666)
* adding in proto debug service

* builds

* debug proto

* gaz ignore

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-04-28 15:43:50 -05:00
Raul Jordan
d0e30ceab2 Remove Old Generated Go Protos (#5667)
* remove old proto generated files
* Merge refs/heads/master into remove-old-protos
2020-04-28 19:47:33 +00:00
Preston Van Loon
9e79809586 Update protobuf, skylib, rules_go, grpc-gateway dependencies (#5657)
* Update dependencies
* Merge branch 'master' into update-deps
* Merge refs/heads/master into update-deps
* Update WORKSPACE
2020-04-28 19:27:56 +00:00
Jim McDonald
9d173dcad2 Update remote signer for 0.11 (#5602)
* Update remote signer for 0.11

* use signing function for aggregate and proof signature

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-04-28 09:27:04 -05:00
Preston Van Loon
e30349d410 Add static analysis to enforce usage of InitWithReset (#5654)
* Refactor attestation packing slightly to reduce the skip slot / HTR of process slots
* Merge branch 'master' into refactor-attestation-packing
* gofmt
* Merge branch 'refactor-attestation-packing' of github.com:prysmaticlabs/prysm into refactor-attestation-packing
* Merge branch 'master' of github.com:prysmaticlabs/prysm into refactor-attestation-packing
* Add static analysis to enforce usage of InitWithReset
* Add comment / lint
* fix a few usages
* more fixes for featureconfig.Init
* Fix analyzer
* Merge branch 'sa-fc-init' of github.com:prysmaticlabs/prysm into sa-fc-init
* Merge refs/heads/master into sa-fc-init
* Merge refs/heads/master into sa-fc-init
2020-04-28 01:13:33 +00:00
Ivan Martinez
6700383863 Terminate bootnode after E2E test completion (#5656)
* Terminate bootnode after test completion
2020-04-28 00:49:26 +00:00
Victor Farazdagi
f84e5ad6ef Fix logged errs (#5645)
* fuzz tests, validators stream, and attestation fixed
* hastst
* signinig root fuzz test
* Merge refs/heads/master into fix-logged-errs
* Merge branch 'master' into fix-logged-errs
* Merge branch 'fix-logged-errs' of github.com:prysmaticlabs/prysm into fix-logged-errs
* add head fetchers to inforstream tests
2020-04-28 00:22:37 +00:00
Victor Farazdagi
44b93d9df9 Fix tests depending on configuration flag (#5650)
* fixes segfault
* Merge branch 'master' into fix-nil-segfaults
* upd state
* Merge branch 'master' into fix-nil-segfaults
* updates shard=1
* Merge refs/heads/master into fix-nil-segfaults
* Merge refs/heads/master into fix-nil-segfaults
* Merge refs/heads/master into fix-nil-segfaults
* fixes tests
* Merge branch 'fix-nil-segfaults' of github.com:prysmaticlabs/prysm into fix-nil-segfaults
* reset config to nil
* add flag assertion to other beacon-chain/rpc files
* Merge refs/heads/master into fix-nil-segfaults
* fix the rest featureconfig.init calls
* Merge branch 'fix-nil-segfaults' of github.com:prysmaticlabs/prysm into fix-nil-segfaults
* remove config setting from init
* gazelle
* reset shard back to 4
2020-04-27 19:44:35 +00:00
Ivan Martinez
8327c9d371 Add test for GetValidatorPerformance (#5652)
* Add test for GetValidatorPerformance

* Fix unneeded changes

* Gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-04-27 13:52:12 -05:00
Nishant Das
e9191e7d6c Create Pcli (#5644)
* add new utility
* Merge branch 'master' into pCli
* Update tools/pcli/main.go

Co-Authored-By: Victor Farazdagi <simple.square@gmail.com>
* Update tools/pcli/main.go

Co-Authored-By: Victor Farazdagi <simple.square@gmail.com>
* Merge refs/heads/master into pCli
* readme and docker fix
* readme
* Merge refs/heads/master into pCli
* Merge refs/heads/master into pCli
* Merge refs/heads/master into pCli
* pending feedback
* builds
* Merge refs/heads/master into pCli
* Merge refs/heads/master into pCli
2020-04-27 16:47:28 +00:00
Raw Pong Ghmoa
f19aa93de2 validator/accounts: don't ask for keystore path if specified (#5641)
* validator/accounts: don't ask for keystore path of specified
* Merge branch 'master' into master
* Merge branch 'master' into master
* validator/accounts restore new validator account creation
* Merge branch 'master' into master
2020-04-27 16:26:51 +00:00
Ivan Martinez
f70635a992 Add post-genesis deposit testing to long-running E2E (#5449)
* WIP add deposits
* Modify validator component
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into e2e-add-depsoits
* Fix e2e
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into e2e-add-depsoits
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into e2e-add-depsoits
* Start running with extra deposits
* Begin adding evluator for e2e deposit
* Get deposit E2E working
* Add more rigorous testing for deposits
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into e2e-add-depsoits
* Improve policy for deposits
* Fix build
* Remove sync testing for long running e2e
* Undo shard change
* Undo unneeded changes
* Adjust for comments
* Merge branch 'master' into e2e-add-depsoits
* Fix bug where long running E2E would always run
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into e2e-add-depsoits
* Merge branch 'e2e-add-depsoits' of https://github.com/0xKiwi/Prysm into e2e-add-depsoits
* Merge branch 'master' into e2e-add-depsoits
2020-04-27 15:59:42 +00:00
Nishant Das
b6353da793 Add Block Batch Limit Flag (#5646)
* add flag

* add flag

* gaz

* fix lint

* fix build issues

* revert initial sync changes

* fix tests

* Update beacon-chain/sync/service.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-04-27 09:35:40 -05:00
Shay Zluf
4da7701249 Add slasher config file flag (#5639)
* Add slasher config file

* shorter line

* fix package dependencies

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-04-27 17:14:26 +03:00
terence tsao
e3e9863d92 Add back archival endpoint GetValidatorBalances with fallback (#5620) 2020-04-27 06:44:27 -07:00
Nisen
7d7480527e Missing defer before unlock (#5643)
* missing defer before unlock
2020-04-27 11:20:09 +00:00
Ivan Martinez
eb5513f8c2 Add GenesisValidatorsRoot to GetGenesis (#5619)
* Add GenesisValidatorsRoot to GetGenesis

* Add genesis state

* Add to test

* Change to get val root from genesis fetcher

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-04-27 09:31:50 +03:00
Nishant Das
07e8609b69 Exit Early from Responding (#5637)
* exit early
2020-04-27 03:18:02 +00:00
wgknowles
08a9fe98fc added prysm.bat for native windows support (#5624) (#5626)
* added first cut of prysm.bat (no SHA/gpg file checking yet)

* updated git commit user.email

* Revert "updated git commit user.email"

This reverts commit a067dc8181.

* Added SHA265 Check, autorestart, ANSI Colors

* Check if valid process, Prefixed PRYSM_autorestart

* Correct echo for downloading beacon-chain

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* fixed ANSI escapes

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-04-26 16:04:02 -07:00
Victor Farazdagi
3782e70e6f checks prysm.sh args size (#5634)
* checks boundaries
* Merge branch 'master' into fix-prysm-sh-unbounded-bug
2020-04-26 21:24:32 +00:00
Preston Van Loon
258d0412c1 GetBlock: Refactor attestation packing (#5632)
* Refactor attestation packing slightly to reduce the skip slot / HTR of process slots
* Merge branch 'master' into refactor-attestation-packing
* gofmt
* Merge branch 'refactor-attestation-packing' of github.com:prysmaticlabs/prysm into refactor-attestation-packing
2020-04-26 21:10:24 +00:00
Victor Farazdagi
d0f3bea8d2 prysm.sh download only option (#5628)
* format srcipt
* download only option
2020-04-26 18:25:48 +00:00
Victor Farazdagi
0806807d61 fixes bug with binary verification (#5627)
* fixes bug with binary verification
* Merge branch 'master' into fix-prysm-sh-verification-bug
* better err message
* Merge branch 'fix-prysm-sh-verification-bug' of github.com:prysmaticlabs/prysm into fix-prysm-sh-verification-bug
2020-04-26 14:09:09 +00:00
Nishant Das
aca9691a0d Stagger Block By Range Responses (#5618)
* refactor method

* fix method

* comments

* Update beacon-chain/sync/rpc_beacon_blocks_by_range.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Update beacon-chain/sync/rpc_beacon_blocks_by_range.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* terence's review

* Apply suggestions from code review

Co-Authored-By: Victor Farazdagi <simple.square@gmail.com>

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-04-26 13:49:47 +08:00
Preston Van Loon
faa5b309a2 Update prysm.sh to fail fast on armv7l (#5621)
* Update prysm.sh
* Merge branch 'master' into prestonvanloon-patch-1
2020-04-25 20:44:17 +00:00
Preston Van Loon
8a9c9607a5 Update EthereumAPIs to prysmaticlabs/ethereumapis#137 (#5622)
* Update EthereumAPIs to https://github.com/prysmaticlabs/ethereumapis/pull/137 and implement validator index return field
2020-04-25 20:23:36 +00:00
terence tsao
5fbf38c98f Increase regen historical state cache size (#5613)
* Increase cache size and garbage collect

* Update beacon-chain/db/kv/regen_historical_states.go

* Use const
2020-04-25 07:59:29 -07:00
terence tsao
07f6894db1 Use signature for SignedAggregateAttestationAndProof (#5605)
* Use right signature

* Minor refactor and a regression test

* Use proper DomainRequest for mock

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-04-24 13:12:47 -05:00
terence tsao
3ac15e36a1 Use correct domain to verify aggregate selection (#5604)
* Use domain selection proof
* Merge refs/heads/master into selection-domain
* Merge branch 'master' of github.com:prysmaticlabs/prysm into selection-domain
* Fixed all the tests
* Merge branch 'selection-domain' of github.com:prysmaticlabs/prysm into selection-domain
2020-04-24 17:26:10 +00:00
Preston Van Loon
ffdf248738 Added new PR template (#5598)
* Copied PR template from https://github.com/bazelbuild/rules_go/pull/2425
* Merge branch 'master' into new-pr-template
* Merge refs/heads/master into new-pr-template
* Merge refs/heads/master into new-pr-template
* Merge refs/heads/master into new-pr-template
* Merge refs/heads/master into new-pr-template
2020-04-24 07:29:51 +00:00
Ivan Martinez
cf98de86f4 Fix percentages for validator metric logging (#5609)
* Actually fix percentages
* Merge branch 'master' into fix-percentages
* Merge branch 'master' into fix-percentages
2020-04-24 05:10:09 +00:00
Nishant Das
f9eb54661a Less Noisy P2P Logs (#5607)
* fix logging issues
* change again
* fix errors
* Merge refs/heads/master into lessNoisy
* gaz
* Merge branch 'lessNoisy' of https://github.com/prysmaticlabs/geth-sharding into lessNoisy
2020-04-24 04:10:53 +00:00
Ivan Martinez
921263848a Fix activationEpoch=FarFutureEpoch in validator client logging (#5608)
* Handle pending case where activation epoch is FarFuture

* Fix logs for validators pending activation

* Reword pending activation log
2020-04-23 21:57:36 -04:00
Victor Farazdagi
a33bd94ffb feature default validator keystore path (#5592)
* feature default validator path
* Merge refs/heads/master into fix-validation-path-issue
* Merge refs/heads/master into fix-validation-path-issue
* Merge refs/heads/master into fix-validation-path-issue
* Merge refs/heads/master into fix-validation-path-issue
* Merge refs/heads/master into fix-validation-path-issue
* Merge refs/heads/master into fix-validation-path-issue
2020-04-23 20:02:15 +00:00
Nishant Das
2ebd684924 Add Goodbye Message When Disconnecting With Peers (#5589)
* add goodbye message
* Merge branch 'master' into addGoodbye
* fix interface
* Merge refs/heads/master into addGoodbye
* use wrapper func
* Merge branch 'addGoodbye' of https://github.com/prysmaticlabs/geth-sharding into addGoodbye
* Merge refs/heads/master into addGoodbye
* Merge refs/heads/master into addGoodbye
* Merge refs/heads/master into addGoodbye
* Merge refs/heads/master into addGoodbye
2020-04-23 19:46:24 +00:00
Shay Zluf
37b68ba983 Slasher min span remove lookback (#5591)
* batch db write and read
* fix nil handling
* Merge branch 'master' into batch_min_max_span
* remove commented code
* Merge branch 'master' into batch_min_max_span
* raul feedback
* Merge branch 'batch_min_max_span' of github.com:prysmaticlabs/prysm into batch_min_max_span
* Merge branch 'master' into batch_min_max_span
2020-04-23 19:27:06 +00:00
terence tsao
a78035d6ed Fix getAttPreState memory usage for new-state-mgmt (#5584)
* Starting

* Revert "update commit hash (#5554)"

This reverts commit c69f561fb9.

* Various memory fixes

* Revert "Revert "update commit hash (#5554)""

This reverts commit c9a174bfa7.

* Better error message

* Provide getters

* Using it in DomainData

* Fixed test

* Tests

* Comment

* Update cache for regular mode

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-04-23 14:03:51 -05:00
Preston Van Loon
a3eeced194 Add a 2 second timeout to fetching eth1data (#5583)
* Add a 2 second timeout to fetching eth1data
* goimports
* fix
* use package const
* Merge refs/heads/master into graceful-failure-eth1data
* fix test
* Merge refs/heads/master into graceful-failure-eth1data
* Merge refs/heads/master into graceful-failure-eth1data
* Merge refs/heads/master into graceful-failure-eth1data
* Merge refs/heads/master into graceful-failure-eth1data
2020-04-23 15:54:51 +00:00
Victor Farazdagi
d1b0348cdb fixes param concatenation (#5594)
* fixes param concatenation
* Merge branch 'master' into prysm-sh-weird-output
2020-04-23 15:24:45 +00:00
Preston Van Loon
a6a2ad4409 Update prysm.sh to include slasher and sig verify (#5543)
* Add gpg detached signature checks

* Add slasher

* Pull key

* move recv after log

* use shasum, download pgp keys

* only download key if not present

* revert bazelversion change

* Actually fail and allow bypass

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-04-23 09:57:21 -05:00
Shawki Sukkar
49ca0751b6 GetValidatorPerformance only return its status info if the validator is active (#5568)
* Task #1
* First commit
* Fix
* Clean
* Clean
* Clean
* Merge branch 'master' into only-active-validators-log
* Fix
* Clean
* Fix
* Fix
* Fix
* Fix
* Merge branch 'master' into only-active-validators-log
2020-04-23 06:11:13 +00:00
terence tsao
2f2124973b Add HeadGenesisValidatorRoot for DomainData (#5582)
* Provide getters

* Using it in DomainData

* Fixed test

* Tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-04-22 22:18:29 -05:00
Ivan Martinez
8b058b51d4 Solve -1 ActivationEpoch in validator logging (#5569)
* Fix wrong activation epoch

* Fix bug with activationEpoch

* Add back default of FarFutureEpoch

* Fix tests

* Fix test

* Fix test again

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-04-22 17:46:57 -05:00
Shay Zluf
9c012cc0c8 Deprecate span map cache flag (#5551)
* deprecate span map cache flag
* fix tests
* nishant feedback
* fix startup
* gaz
* nishant feedback
* gaz
* Merge branch 'master' into deprecate_span_cache
* fix img
* Update slasher/usage_test.go
* Merge refs/heads/master into deprecate_span_cache
* Merge refs/heads/master into deprecate_span_cache
* Merge refs/heads/master into deprecate_span_cache
* Merge refs/heads/master into deprecate_span_cache
* Merge refs/heads/master into deprecate_span_cache
* Merge refs/heads/master into deprecate_span_cache
* Merge refs/heads/master into deprecate_span_cache
* Merge refs/heads/master into deprecate_span_cache
* Merge refs/heads/master into deprecate_span_cache
* Merge refs/heads/master into deprecate_span_cache
* Merge refs/heads/master into deprecate_span_cache
* fix up tests
2020-04-22 15:53:09 +00:00
Victor Farazdagi
9e6b6bc009 fixes issue with ipc path checking (#5577)
* fixes issue with ipc path checking
2020-04-22 11:42:12 +00:00
Jim McDonald
db6dbdcde4 More concise duty logging (#5397)
* More concise duty logging
* Merge branch 'master' into concise-duties
* Remove comment
* Merge branch 'concise-duties' of github.com:mcdee/prysm into concise-duties
* Merge branch 'master' into concise-duties
* Merge branch 'master' into concise-duties
* Merge branch 'master' into concise-duties
* Merge branch 'master' into concise-duties
* Use helpers to calculate slot offset; clarify comment
* Merge branch 'master' into concise-duties
* Merge branch 'master' into concise-duties
* Merge branch 'master' into concise-duties
* Merge branch 'master' into concise-duties
* Merge branch 'master' into concise-duties
2020-04-22 07:50:35 +00:00
Nishant Das
482a054ee1 Handle Nil State Object (#5575)
* handle panic
* Merge branch 'master' into handlePanic
* Merge refs/heads/master into handlePanic
* Merge refs/heads/master into handlePanic
2020-04-22 04:44:33 +00:00
Ivan Martinez
1480d2bfa8 Add hotfix for E2E (#5573)
* Hotfix for e2e
* Add TODO
* Merge branch 'master' into hotfix-e2e
* Merge branch 'master' into hotfix-e2e
2020-04-22 04:23:50 +00:00
Raul Jordan
793c3d27f2 No More EthereumAPIs Patch (#5570)
* no more patch
* Merge refs/heads/master into no-more-patch
* fix build
* Merge refs/heads/master into no-more-patch
* Merge refs/heads/master into no-more-patch
2020-04-22 03:56:06 +00:00
Nishant Das
f880fb4ee4 Shift To Trace Logs (#5561)
* shift to trace
* Merge branch 'master' into shiftTrace
* Merge refs/heads/master into shiftTrace
* Merge refs/heads/master into shiftTrace
* Merge refs/heads/master into shiftTrace
* Merge refs/heads/master into shiftTrace
* Merge refs/heads/master into shiftTrace
* Merge refs/heads/master into shiftTrace
* fix test
* Merge branch 'shiftTrace' of https://github.com/prysmaticlabs/geth-sharding into shiftTrace
2020-04-22 01:21:12 +00:00
terence tsao
7b38e97e83 Invert --disable-new-state-mgmt to --enable-new-state-mgmt (#5567)
* Invert the flag
* Merge branch 'master' into invert-disable-new-state-mgmt
* Merge refs/heads/master into invert-disable-new-state-mgmt
* Add new-state-mgmt to e2e flag
* Merge branch 'invert-disable-new-state-mgmt' of github.com:prysmaticlabs/prysm into invert-disable-new-state-mgmt
* Merge refs/heads/master into invert-disable-new-state-mgmt
2020-04-21 22:30:22 +00:00
Victor Farazdagi
e753cbb04b /healthz endpoint accepts JSON now (#5558)
* /healthz endpoint accepts JSON now
* Merge refs/heads/master into json-healthz
* Merge refs/heads/master into json-healthz
* Merge refs/heads/master into json-healthz
* Merge refs/heads/master into json-healthz
* Merge refs/heads/master into json-healthz
* Merge refs/heads/master into json-healthz
2020-04-21 20:58:53 +00:00
Ivan Martinez
c2daf8d82e Change Validator Status for visible deposit back to DEPOSITED (#5563)
* Change valstatus for visible deposit back to DEPOSITED
* Rename logs
* Merge branch 'master' into val-status
* Fix tests
* Merge branch 'val-status' of https://github.com/prysmaticlabs/prysm into val-status
* Fix tests
* Change test
* Merge branch 'master' into val-status
* Text changes
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into val-status
* Merge branch 'val-status' of https://github.com/prysmaticlabs/prysm into val-status
* Update validator/client/validator.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update validator/client/validator.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge branch 'val-status' of https://github.com/prysmaticlabs/prysm into val-status
* Merge branch 'master' into val-status
2020-04-21 19:11:27 +00:00
terence tsao
480ddb7c3d Continue if archival state is not in DB (#5565)
* Continue if archival state is not availble to generate
* Fixed test
* Merge branch 'master' into save-blocks-splitslot
* Merge refs/heads/master into save-blocks-splitslot
2020-04-21 18:18:55 +00:00
Victor Farazdagi
0f08bd288c fix tests in //beacon-chain/rpc/beacon (#5559)
* assert new state is not disabled
* Merge refs/heads/master into fix-list-indexed-atts-test
* add helper method
* Merge branch 'fix-list-indexed-atts-test' of github.com:prysmaticlabs/prysm into fix-list-indexed-atts-test
* xx
* xx
* Merge refs/heads/master into fix-list-indexed-atts-test
2020-04-21 17:44:18 +00:00
Shay Zluf
8cba109350 Slasher batch db read and a write (#5534)
* batch db read

* fix test

* debug

* write function and test

* test rename

* add to interface

* change order

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-04-21 11:50:04 -05:00
terence tsao
199c50be47 Migration should not fail process block routine (#5557)
* Continue rather than fail
* Merge branch 'master' into migrate-should-not-fail-block
* Merge refs/heads/master into migrate-should-not-fail-block
2020-04-21 14:44:43 +00:00
Nishant Das
c69f561fb9 update commit hash (#5554) 2020-04-21 09:07:06 -05:00
Ivan Martinez
19b879ec17 Change ListIndexedAttestations to convert attestations from target root (#5548) 2020-04-20 22:48:31 -04:00
terence tsao
54faf8564f Add back archival endpoint GetValidatorActiveSetChanges with fallback (#5519)
* Modify `GetValidatorActiveSetChanges` to use new state service  (#5408)

* Fix import and not use
* Update workspace for new spec test
* Fix workspace
* Merge refs/heads/master into v0.10.1
* Update workspace with new ethapi commit
* Unblock a few tests
* Merge refs/heads/master into v0.10.1
* fixed block op test
* gaz
* Merge refs/heads/master into v0.10.1
* Skip gen state test (test setup issue
* Updated hysteresis config
* Updated epoch processing for new hyteresis
* Updated tests
* regen proto beacon
* update state util for state root
* update state types
* update getter and setters
* update compute domain and get domain and tests
* update validators
* Add forkdata proto
* Updated compute domain api, moved it to helper pkg
* Merge refs/heads/master into v0.10.1
* Fixed all core tests
* Fixed all the sync tests
* Fixed all the rpc tests
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Fixed conflict
* Fixed conflict
* Conflict fix
* visibility
* Fixed validator tests
* Fixing test util
* Fixed rest of non spec tests
* Fixed a bug proposer index wasn't included
* gaz
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Updated eth1 data voting period to epoch based
* Fixed failed tests
* fix bug
* fix error
* Fixed more misc tests
* Add new SignedAggregateAndProof to pass spec test
* Update minimalConfig.PersistentCommitteePeriod
* allow to rebuild trie
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Skip e2e tests
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Align aggregator action with v0.11 (#5146)
* Remove Head Root from Beacon Block by Range Request (#5165)

* make proto changes
* remove head root
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.11
* add back herumi's library
* Update ethapi in workspace, started fixing test. Hand off to Nishant
* fix build
* All tests passing
* Align finalized slot check with v0.11 (#5166)
* Merge branch 'master' into v0.11
* Add DoS resistance for v0.11 (#5158)
* Add Fork Digest Helper (#5173)
* Extend DoS prevention to rest of operation objects (#5174)

* Update mapping

* Add caches

* Update seen block in validation pipeline

* Update seen att in validation pipeline

* Update seen att in validation pipeline

* Fixed rest of tests

* Gazelle

* Better writes

* Lint

* Preston's feedback

* Switched to LRU cache and fixed tests

* Gazelle

* Fix test

* Update proposer slashing

* Update proposer slashing

* Fixed a block test

* Update exit

* Update atteser slashing

* Raul's feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Add remote keymanager (#5133)

* Add remote keymanager

* Add generic signRoot() helper

* Add tests for remote keymanager

* NewRemote -> NewRemoteWallet

* signRoot -> signOject, to increase reuse

* Fix end-to-end compile error

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Add Snappy Framing to the Encoder (#5172)

* change to framing

* more fixes

* fix everything

* add stricter limits

* preston feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: rauljordan <raul@prysmaticlabs.com>
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Move Subnet Functionality to its Own File (#5179)

* move subnets to their own file

* fix build fail

* build

* Update beacon-chain/p2p/discovery_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Sync with master
* Verify proposer signature in sync (#5206)
* Fix Signed Attestation In Sync (#5207)
* Add Eth2 Fork ENR Functionality (#5181)

* add fork entry enr

* add in fork

* add the required fork entry to node

* add and retrieve fork entry

* await state initialized

* utilize new structure

* more progress, utilizing a config map instead

* send the genesis validators root via the event feed

* struct method for discovery

* fix broken builds

* fixed up more tsts using state feed initializer

* fix up most tests

* only one more failing test

* almost done with tests

* p2p tests all pass

* config fix

* fix blockchain test

* gaz

* add in todo

* lint

* add compare func

* ensure fork ENR versions match between peers

* add in test for discovery

* test name

* tests complete

* tests done

* done

* comments

* fix all flakes

* addressed comments

* build using ssz gen

* marshal record

* use custom ssz

* deduplicate import

* fix build

* add enr proto

* p2p tests done

Co-authored-by: nisdas <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Verify aggregator signature in sync (#5208)
* Add Fork Digest For Gossip Topics (#5191)

* update for the day

* fix remaining failing test

* fix one more test

* change message

* Apply suggestions from code review

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* terence's review

* implement fork digest'

* align digest to interface'

* passed all tests

* spawn in goroutine

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Fix Incorrect Attester Slashing Method (#5229)
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Remove keystore keymanager from validator (#5236)

* Remove keystore keymanager from validator

* Update dependency

* Update validator/flags/flags.go

* Update validator/flags/flags.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
* fix broadcaster
* update metrics with fork digest for p2p (#5251)

* update metrics with fork digest for p2p

* update p2p metrics

* update metrics using att values

* wrapped up

* fix bug

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Fix incorrect domain type comments (#5250)

* Fix incorrect domain type comments
* resolve conflicts
* fix broken broadcast test
* fix tests
* include protocol suffix
* fix confs
* lint
* fix test
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* resolve broken slasher test'
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Merge branch 'master' into v0.11
* fix config override
* Remove deprecated parameters (#5249)
* Avoid div by zero in extreme balance case (#5273)

* Return effective balance increment instead of 1

* Update to new spec tests v0.11.1

* Revert "Regen historical states for `new-state-mgmt` compatibility (#5261)"

This reverts commit df9a534826.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Revert "Remove deprecated parameters (#5249)" (#5276)

This reverts commit 7d17c9ac34.
* Verify block proposer index before gossip  (#5274)

* Update pipeline

* Update tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Add in Proposer Index to Custom HTR (#5269)

* fix test

* Update beacon-chain/state/stateutil/blocks_test.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Resolve Flakey P2P Tests (#5285)

* double time for flakey test

* fix test flakeyness in p2p:

* flakey

* time tolerance

* greater tolerance
* Merge branch 'master' into v0.11
* release resources correctly (#5287)
* Merge refs/heads/master into v0.11
* Enable NOISE Handshake by Default v0.11 (#5272)

* noise handshakes by default

* fix build

* noisy noise everywhere

* deprecated noisy noise flag with more noise

* add secio as fallback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
* Merge refs/heads/master into v0.11
* new ports
* fix broken build
* Make `new-state-mgmt` canonical  (#5289)

* Invert the flags
* Update checking messages
* Fixed all db tests
* Fixed rest of the block chain tests
* Fix chain race tests
* Fixed rpc tests
* Disable soudns better...
* Merge branch 'v0.11' into invert-new-state-mgmt
* Merge refs/heads/v0.11 into invert-new-state-mgmt
* Fix export
* Merge branch 'invert-new-state-mgmt' of github.com:prysmaticlabs/prysm into invert-new-state-mgmt
* Fix conflict tests
* Gazelle
* Merge refs/heads/v0.11 into invert-new-state-mgmt
* Merge refs/heads/v0.11 into invert-new-state-mgmt
* Merge branch 'master' into v0.11
* resolve flakeyness
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Detect Proposer Slashing Implementation (#5139)

* detect blocks

* detect blocks

* use stub

* use stub

* use stub

* todo

* fix test

* add tests and utils

* fix imports

* fix imports

* fix comment

* todo

* proposerIndex

* fix broken test

* formatting and simplified if

* Update slasher/detection/service.go

* Update slasher/detection/testing/utils.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* fixed up final comments

* better naming

* Update slasher/detection/service.go

* Update slasher/detection/service.go

* Update slasher/detection/service.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

* no more named args

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.11
* Add Metadata And Ping RPC methods (#5271)

* add new proto files

* add flag and helper

* add initializer

* imports

* add ping method

* add receive/send ping request

* add ping test

* refactor rpc methods and add ping test

* finish adding all tests

* fix up tests

* Apply suggestions from code review

* lint

* imports

* lint

* Update beacon-chain/p2p/service.go

* Update shared/cmd/flags.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.11
* Updates for remote keymanager (#5260)
* Merge branch 'spec-v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Merge remote-tracking branch 'origin' into v0.11
* Update to slash by slot instead of epoch (#5297)

* change to slash by slot instead of epoch

* gaz

* fix test

* fix test

* fix infinite loop on error parse
* Sync with master
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Update proposer protection to v0.11 (#5292)

* Complete most of changes

* Fix other tests

* Test progress

* Tests

* Finish tests

* update pbs

* Fix mocked tests

* Gazelle

* pt 2

* Fix

* Fixes

* Fix tests wit hwrong copying
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Implement `SubscribeCommitteeSubnet` method (#5299)

* Add client implementation

* Update workspace

* Update server

* Update service

* Gaz

* Mocks

* Fixed validator tests

* Add round tirp tests

* Fixed subnet test

* Comment

* Update committee cache

* Comment

* Update RPC

* Fixed test

* Nishant's comment

* Gaz

* Refresh ENR is for epoch

* Needs to be append
* Merge refs/heads/master into v0.11
* resolve confs
* Validator subscribe subnet to next epoch (#5312)

* Alert to subscribe to next epoch

* Fixed tests

* Comments

* Fixed tests

* Update validator/client/validator.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Revert "Revert "Remove deprecated parameters (#5249)" (#5276)" (#5277)

This reverts commit 47e5a2cf96.
* Aggregate on demand for v0.11 (#5302)

* Add client implementation

* Update workspace

* Update server

* Update service

* Gaz

* Mocks

* Fixed validator tests

* Add round tirp tests

* Fixed subnet test

* Wait 1/3 on validator side

* Lint

* Comment

* Update committee cache

* Comment

* Update RPC

* Fixed test

* Nishant's comment

* Gaz

* Refresh ENR is for epoch

* Needs to be append

* Fixed duplication

* Tests

* Skip e2e

* Update beacon-chain/rpc/validator/aggregator.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

* Apply suggestions from code review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: shayzluf <thezluf@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Refactor Dynamic Subscriptions (#5318)

* clean up

* comment

* metrics

* fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge refs/heads/master into v0.11
* Fix listindexed attestations and detect historic attestations (#5321)

* fix list indexed attestations

* fix tests

* goimports

* names
* Add check for slot == 0 (#5322)
* Change attester protection to return default if DB is empty (#5323)

* Change how default values are set

* Remove unused imports

* Remove wasteful db call

* Fix db tests

* Fix db test
* Merge refs/heads/master into v0.11
* fix it (#5326)
* V0.11 run time fixes to use interop config (#5324)

* Started testing
* Bunch of fixes
* use-interop
* Sync with v0.11
* Conflict
* Uncomment wait for activation
* Move pending block queue from subscriber to validator pipeline
* Merge branch 'v0.11' into use-interop-config
* passing tests
* Merge refs/heads/v0.11 into use-interop-config
* Merge refs/heads/v0.11 into use-interop-config
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Nil Checks in Process Attestation v0.11 (#5331)

* Started testing

* Bunch of fixes

* use-interop

* Sync with v0.11

* Uncomment wait for activation

* Move pending block queue from subscriber to validator pipeline

* passing tests

* nil checks to prevent panics

* lint

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Validator batch subscribe subnets (#5332)

* Update both beacon node and validator

* Comments

* Tests

* Lint

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Validator smarter subscribe (#5334)
* Fix incorrect proposer index calculation (#5336)

* Use correct parent state

* Fixed test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* enhance error
* enhance error
* Update P2P Service to Handle Local Metadata (#5319)

* add metadata to ENR

* add new methods

* glue everything

* fix all tests and refs

* add tests

* add more tests

* Apply suggestions from code review

* fix method

* raul's review

* gaz

* fix test setup

* fix all tests

* better naming

* fix broken test

* validate nil

Co-authored-by: rauljordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Revert "Revert "Revert "Remove deprecated parameters (#5249)" (#5276)" (#5277)" (#5343)

This reverts commit e5aef1686e.
* Wait for Genesis Event to Start P2P (#5303)

* use event feed for state initialized events

* add in handler for tests

* wait till genesis for p2p

* Apply suggestions from code review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge refs/heads/master into v0.11
* Avoid duplicated aggregation request (#5346)

* Avoid duplicated aggregation request

* Test and lock

* Gaz
* Fix Validate For Metadata (#5348)

* return true

* shay's review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Multiple Proposer Slots Allowed Per Epoch for Validators (#5344)

* allow multiple proposer slots

* multi propose

* proposer indices to slots map

* remove deprecated comm assign

* Apply suggestions from code review

* resolve broken tests, add logic in validator client

* fix val tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Networking Fixes (#5349)

* close stream later

* add ping method

* add method

* lint
* More efficient aggregation on demand (#5354)
* Return Nil Error if Pre-Genesis in P2P Service Healthz Check (#5355)

* pregenesis healthz check:

* optimal

* right order

* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* no comment

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Release DiscoveryV5 for Testnet Restart (#5357)

* release discv5

* fix build
* Fix Overflow in Status Check (#5361)

* fix overflow

* Apply suggestions from code review
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* fix after merge
* Merge refs/heads/master into v0.11
* Make Mainnet Config Default, No More Demo Config  (#5367)

* bye bye demo config

* gaz

* fix usage

* fix dep

* gaz

* Update default balance for sendDeposits

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Use FastSSZ Marshal/Unmarshal for DB Encodings in v0.11.1 (#5351)

* try

* use marshaler structure for db instead of proto

* white list types

* attempt

* revert

* testutil.NewBeaconState()

* Fully populate fields for round trip ssz marshal

* fix //beacon-chain/db/kv:go_default_test

* more passing tests

* another test target passed

* fixed stategen

* blockchain tests green

* passing sync

* more targets fixed

* more test fixes in rpc/validator

* most rpc val

* validators test fixes

* skip round robin old

* aggregate test

* whitelist done

* Update beacon-chain/rpc/validator/attester_test.go

* edit baz

* Fixed tests

* Fixed getblock test

* Add back init

* reduce test size

* fix broken build

* tests pass

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Reconnect slasher streams on beacon node shutdown (#5376)

* restart streams on beacon node shutdown

* fix comment

* remove export

* ivan feedback

* ivan feedback

* case insensitive

* Update slasher/beaconclient/receivers.go

* raul feedback

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' into v0.11
* Merge refs/heads/master into v0.11
* Amend Faucet to Offer 32.5 ETH for v0.11 (#5378)

* deposit amount in faucet

* fix eth amount

* gas cost
* unskip exec transition test
* Revert "Enable NOISE Handshake by Default v0.11 (#5272)" (#5381)

This reverts commit a8d32d504a.
* Merge refs/heads/master into v0.11
* use string for deposit flag
* Update Bootnode to v0.11 (#5387)

* fix bootnode

* add changes

* gaz

* fix docker
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* build fix
* fix flaky test
* Merge refs/heads/master into v0.11
* Unskip E2E for V0.11 (#5386)

* Begin work on fixing e2e for v0.11

* Start bootnode work

* Begin implementing bootnode into e2e

* Fix E2E for v0.11

* Remove extra

* gaz

* Remove unused key gen code

* Remove trailing multiaddr code

* add skip for slashing

* Fix slashing e2e

* Fix docker image build
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.11
* Merge refs/heads/master into v0.11
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* Update beacon-chain/p2p/broadcaster_test.go
* Merge refs/heads/master into v0.11
* Update GetValidatorActiveSetChanges
* Fixed tests
* Pass E2E Tests for v0.11 and Enable Attestation Subnets By Default (#5407)
* Merge refs/heads/v0.11 into fix-active-set-changes
* Update README.md

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Apply suggestions from code review

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Update beacon-chain/p2p/config.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Update shared/keystore/deposit_input.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Update tools/faucet/server.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Update proto/beacon/p2p/v1/BUILD.bazel

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Update shared/bls/spectest/aggregate_verify_test.go
* Merge refs/heads/v0.11 into fix-active-set-changes
* Addressed feedback. All test passing
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Merge refs/heads/v0.11 into fix-active-set-changes
* Update beacon-chain/core/blocks/block_operations_fuzz_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Update beacon-chain/core/blocks/block_operations_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Update shared/testutil/helpers.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Update beacon-chain/core/helpers/signing_root.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Resolve Misc v0.11 Items (Raul) (#5414)

* address all comments

* set faucet

* nishant feedback

* Update beacon-chain/p2p/service.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Revert keymanager changes (#5416)

* Revert "Updates for remote keymanager (#5260)"

This reverts commit bbcd895db5.

* Revert "Remove keystore keymanager from validator (#5236)"

This reverts commit 46008770c1.

* Revert "Update eth2 wallet keymanager (#4984)"

This reverts commit 7f7ef43f21.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Update BLS and limit visibility (#5415)

* remove duplicated BLS, add golang.org/x/mod

* Update BLS and restrict visibility

* fix build
* Merge refs/heads/v0.11 into fix-active-set-changes
* Fix eth1data test and fix order of ops (#5413)
* Merge refs/heads/v0.11 into fix-active-set-changes
* use multiaddr builder (#5419)
* Merge refs/heads/v0.11 into fix-active-set-changes
* Unskip benchutil and minor v0.11 fixes (#5417)

* Unskip benchutil tests

* Remove protos and gaz

* Fixes
* Merge refs/heads/v0.11 into fix-active-set-changes
* Networking Fixes (#5421)

* check

* fix test

* fix size

* fix test

* more fixes

* fix test again
* Merge refs/heads/v0.11 into fix-active-set-changes
* Update ethereum APIs with latest master
* Merge refs/heads/v0.11 into fix-active-set-changes
* Error handling for v0.11 tests (#5428)

* Proper err handling for tests

* Lint

* Fixed rest of the tests

* Gaz

* Fixed old master tests
* Sync with master
* Rm old aggregate_test.go
* Sync with v0.11
* Sync with master
* Merge refs/heads/master into fix-active-set-changes
* Merge refs/heads/master into fix-active-set-changes
* Merge refs/heads/master into fix-active-set-changes
* Merge refs/heads/master into fix-active-set-changes
* Merge refs/heads/master into fix-active-set-changes
* Merge branch 'master' into fix-active-set-changes
* Gaz
* Merge refs/heads/master into fix-active-set-changes
* Fixed test
* Merge branch 'fix-active-set-changes' of github.com:prysmaticlabs/prysm into fix-active-set-changes
* Merge refs/heads/master into fix-active-set-changes

* Active set chagne with fallback

* Add back the old tests

* Fixed test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-04-20 19:59:34 -05:00
Ivan Martinez
c8fa31ac97 WaitForActivation optimizations (#5396)
* WaitForActrivation improvements
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into waitforactivation
* Changes to handle new statuses
* Merge branch 'master' into waitforactivation
* Fix tests
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into waitforactivation
* Merge branch 'waitforactivation' of https://github.com/0xKiwi/Prysm into waitforactivation
* Remove unkown status being set
* Fix test
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into waitforactivation
* Merge branch 'master' into waitforactivation
* Modify logging slightly
* Merge branch 'waitforactivation' of https://github.com/0xKiwi/Prysm into waitforactivation
* Improve deposit ux
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into waitforactivation
* Add test for logs
* Rename
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into waitforactivation
* Merge branch 'master' into waitforactivation
* Merge branch 'master' into waitforactivation
* Merge branch 'master' into waitforactivation
* Merge branch 'master' into waitforactivation
* Merge branch 'master' into waitforactivation
* Merge branch 'master' into waitforactivation
* Merge branch 'master' into waitforactivation
2020-04-20 23:52:39 +00:00
Ivan Martinez
25102e0978 Use WaitForSynced in validator client for startup (#5465)
* Add WaitForSynced to beacon-chain
* Fix build issues
* Fix comment
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into simplify-validator
* Fix tests
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into simplify-validator
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into simplify-validator
* Change validator client to use WaitForSynced
* Change prysm script to support windows
* Update prysm.sh
* Merge branch '0xKiwi-patch-1' of https://github.com/prysmaticlabs/prysm into simplify-val-client
* Merge branch 'master' into simplify-val-client
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into simplify-val-client
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into simplify-val-client
* Add to mock
* Merge branch 'simplify-val-client' of https://github.com/0xKiwi/Prysm into simplify-val-client
* Fix mocks
* Add feature flag for WaitForSynced
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into simplify-val-client
* Fix flag
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into simplify-val-client
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into simplify-val-client
* Remove comment
* Merge branch 'master' into simplify-val-client
* Merge branch 'master' into simplify-val-client
* Merge branch 'master' into simplify-val-client
* Merge branch 'master' into simplify-val-client
2020-04-20 23:16:53 +00:00
Ivan Martinez
60bf3edf0a Optimize GetDuties to not call Eth1 functions (#5516)
* Remove unneeded validator status calls for GetDuties
* Cleanup validator runner function
* Improvements
* Undo validator client changes
* Merge branch 'master' into optimize-getduties
* Revert "Undo validator client changes"

This reverts commit 4385181eaf.
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into optimize-getduties
* Remove unneeded tests
* Merge branch 'optimize-getduties' of https://github.com/0xKiwi/Prysm into optimize-getduties
* Revert "Cleanup validator runner function"

This reverts commit f0c8c5e8bc.
* Add back test
* Optimize getduties
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into optimize-getduties
* Merge branch 'master' into optimize-getduties
2020-04-20 22:24:50 +00:00
terence tsao
982d93547a Log warn instead of return error (#5547)
* Fix 5395
2020-04-20 21:32:56 +00:00
Nishant Das
7b89b17ed2 Update to 1.14.1 (#5535)
* update to 1.14.1
* Merge branch 'master' into updatego1.14.1
* Merge refs/heads/master into updatego1.14.1
* Merge refs/heads/master into updatego1.14.1
* Merge refs/heads/master into updatego1.14.1
* Merge refs/heads/master into updatego1.14.1
2020-04-20 20:49:03 +00:00
terence tsao
2a0711ab01 Regen and save historical states as default (#5544)
* Better error
* Use copy
* Conflict
* Fixed a bug on using pre state. Added a LRU cache for saving pre state.
* Removed interaction menu. Ran Gazelle
* Fixed delete state to continue than exit
* Gazelle
* Merge branch 'master' of github.com:prysmaticlabs/prysm into regen-default
* Merge refs/heads/master into regen-default
* Added a warning message
* Merge branch 'regen-default' of github.com:prysmaticlabs/prysm into regen-default
* Merge refs/heads/master into regen-default
2020-04-20 20:19:53 +00:00
Raul Jordan
9255a2b3c1 Add Clock-Based Slot to Metrics (#5545)
* clock based slot
* Merge refs/heads/master into highest-possible-slot
2020-04-20 19:49:25 +00:00
Raul Jordan
61c96f50a0 Add Metrics to Bootnode (#5460)
* bootnode metrics
* Merge refs/heads/master into bootnode-metrics
* Merge refs/heads/master into bootnode-metrics
* Merge refs/heads/master into bootnode-metrics
* Merge refs/heads/master into bootnode-metrics
* Merge refs/heads/master into bootnode-metrics
* Merge refs/heads/master into bootnode-metrics
* Merge refs/heads/master into bootnode-metrics
* docker bootnode deps
* Merge branch 'master' into bootnode-metrics
* fix
* Merge refs/heads/master into bootnode-metrics
* Merge refs/heads/master into bootnode-metrics
* Merge refs/heads/master into bootnode-metrics
* Merge refs/heads/master into bootnode-metrics
* Merge refs/heads/master into bootnode-metrics
* Merge refs/heads/master into bootnode-metrics
* Merge refs/heads/master into bootnode-metrics
* Merge refs/heads/master into bootnode-metrics
* Merge refs/heads/master into bootnode-metrics
* Merge refs/heads/master into bootnode-metrics
* Merge refs/heads/master into bootnode-metrics
* Merge refs/heads/master into bootnode-metrics
* Merge refs/heads/master into bootnode-metrics
* Merge refs/heads/master into bootnode-metrics
* Merge refs/heads/master into bootnode-metrics
* Merge refs/heads/master into bootnode-metrics
* Merge refs/heads/master into bootnode-metrics
* Merge refs/heads/master into bootnode-metrics
* Merge refs/heads/master into bootnode-metrics
* Merge refs/heads/master into bootnode-metrics
* Merge refs/heads/master into bootnode-metrics
* Merge refs/heads/master into bootnode-metrics
* Merge refs/heads/master into bootnode-metrics
* Merge refs/heads/master into bootnode-metrics
* Merge refs/heads/master into bootnode-metrics
* resolve build
2020-04-20 19:02:09 +00:00
Victor Farazdagi
b663086a88 add --dev flag (#5539)
* add --dev flag
* Merge refs/heads/master into dev-flag
* Update shared/featureconfig/config.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update shared/featureconfig/flags.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/master into dev-flag
2020-04-20 17:27:24 +00:00
Raul Jordan
37cba662f1 Run Gofmt on Repo (#5541)
* toggle flags
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* run gofmt
* Merge branch 'master' into lint-repo
2020-04-20 16:35:04 +00:00
Shay Zluf
fae307e5a2 Fix slasher historic attestation detection (#5532)
* fix slasher historic attestation detection

* revert log changes
2020-04-20 11:11:53 -05:00
Nishant Das
639e3072fe Cancel Initial Sync Properly (#5529)
* fix cancelling
* Merge branch 'master' into cancelServices
* Merge refs/heads/master into cancelServices
* Merge refs/heads/master into cancelServices
2020-04-20 04:04:45 +00:00
terence tsao
984644257e Add back archival endpoint ListBeaconCommittees with fallback (#5518)
* Add fallback
* Add back the old tests
* Merge refs/heads/master into list-beacon-committee-fallback
* Merge refs/heads/master into list-beacon-committee-fallback
* Merge refs/heads/master into list-beacon-committee-fallback
* Merge refs/heads/master into list-beacon-committee-fallback
* Merge refs/heads/master into list-beacon-committee-fallback
* Merge refs/heads/master into list-beacon-committee-fallback
2020-04-20 03:41:32 +00:00
Preston Van Loon
9ae492e2cd Validator: fix percentages to be per 100 (#5527)
* fix percentages to be per 100
* Merge branch 'master' into fix-percentages
* Merge refs/heads/master into fix-percentages
2020-04-20 03:10:53 +00:00
Preston Van Loon
1c6d1fabf3 Update prysm.sh to use prysmaticlabs.com/releases (#5528)
* Update prysm.sh to use prysmaticlabs.com/releases to avoid github rate limits
2020-04-20 02:56:59 +00:00
Nishant Das
b61d320613 Make Connections Less Aggressive (#5488)
* make less aggressive
* Merge branch 'master' into lessAggressive
* Merge branch 'master' into lessAggressive
* Merge refs/heads/master into lessAggressive
* Merge refs/heads/master into lessAggressive
* Merge refs/heads/master into lessAggressive
2020-04-20 02:11:18 +00:00
terence tsao
62da94bd0b Add back archival endpoint ValidatorParticipation with fallback (#5520)
* Modify `GetValidatorParticipation` to use new state service (#5409)

* Skip 2 more minimal tests

* Update readme

* gaz

* Fix import and not use

* Update workspace for new spec test

* Fix workspace

* Update workspace with new ethapi commit

* Unblock a few tests

* fixed block op test

* gaz

* Skip gen state test (test setup issue

* Updated hysteresis config

* Updated epoch processing for new hyteresis

* Updated tests

* regen proto beacon

* update state util for state root

* update state types

* update getter and setters

* update compute domain and get domain and tests

* update validators

* Add forkdata proto

* Updated compute domain api, moved it to helper pkg

* Fixed all core tests

* Fixed all the sync tests

* Fixed all the rpc tests

* Conflict fix

* visibility

* Fixed validator tests

* Fixing test util

* Fixed rest of non spec tests

* Fixed a bug proposer index wasn't included

* gaz

* Updated eth1 data voting period to epoch based

* Fixed failed tests

* fix bug

* fix error

* Fixed more misc tests

* Add new SignedAggregateAndProof to pass spec test

* Update minimalConfig.PersistentCommitteePeriod

* allow to rebuild trie

* Skip e2e tests

* Align aggregator action with v0.11 (#5146)

* Remove Head Root from Beacon Block by Range Request (#5165)

* make proto changes
* remove head root

* add back herumi's library

* Update ethapi in workspace, started fixing test. Hand off to Nishant

* fix build

* Align finalized slot check with v0.11 (#5166)

* Add DoS resistance for v0.11 (#5158)

* Add Fork Digest Helper (#5173)

* Extend DoS prevention to rest of operation objects (#5174)

* Update mapping

* Add caches

* Update seen block in validation pipeline

* Update seen att in validation pipeline

* Update seen att in validation pipeline

* Fixed rest of tests

* Gazelle

* Better writes

* Lint

* Preston's feedback

* Switched to LRU cache and fixed tests

* Gazelle

* Fix test

* Update proposer slashing

* Update proposer slashing

* Fixed a block test

* Update exit

* Update atteser slashing

* Raul's feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Add remote keymanager (#5133)

* Add remote keymanager

* Add generic signRoot() helper

* Add tests for remote keymanager

* NewRemote -> NewRemoteWallet

* signRoot -> signOject, to increase reuse

* Fix end-to-end compile error

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Add Snappy Framing to the Encoder (#5172)

* change to framing

* more fixes

* fix everything

* add stricter limits

* preston feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: rauljordan <raul@prysmaticlabs.com>

* Move Subnet Functionality to its Own File (#5179)

* move subnets to their own file

* fix build fail

* build

* Update beacon-chain/p2p/discovery_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Verify proposer signature in sync (#5206)

* Fix Signed Attestation In Sync (#5207)

* Add Eth2 Fork ENR Functionality (#5181)

* add fork entry enr

* add in fork

* add the required fork entry to node

* add and retrieve fork entry

* await state initialized

* utilize new structure

* more progress, utilizing a config map instead

* send the genesis validators root via the event feed

* struct method for discovery

* fix broken builds

* fixed up more tsts using state feed initializer

* fix up most tests

* only one more failing test

* almost done with tests

* p2p tests all pass

* config fix

* fix blockchain test

* gaz

* add in todo

* lint

* add compare func

* ensure fork ENR versions match between peers

* add in test for discovery

* test name

* tests complete

* tests done

* done

* comments

* fix all flakes

* addressed comments

* build using ssz gen

* marshal record

* use custom ssz

* deduplicate import

* fix build

* add enr proto

* p2p tests done

Co-authored-by: nisdas <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Verify aggregator signature in sync (#5208)

* Add Fork Digest For Gossip Topics (#5191)

* update for the day

* fix remaining failing test

* fix one more test

* change message

* Apply suggestions from code review

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* terence's review

* implement fork digest'

* align digest to interface'

* passed all tests

* spawn in goroutine

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Fix Incorrect Attester Slashing Method (#5229)

* Remove keystore keymanager from validator (#5236)

* Remove keystore keymanager from validator

* Update dependency

* Update validator/flags/flags.go

* Update validator/flags/flags.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>

* fix broadcaster

* update metrics with fork digest for p2p (#5251)

* update metrics with fork digest for p2p

* update p2p metrics

* update metrics using att values

* wrapped up

* fix bug

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Fix incorrect domain type comments (#5250)

* Fix incorrect domain type comments

* fix broken broadcast test

* fix tests

* include protocol suffix

* lint

* fix test

* resolve broken slasher test'

* fix config override

* Remove deprecated parameters (#5249)

* Avoid div by zero in extreme balance case (#5273)

* Return effective balance increment instead of 1

* Update to new spec tests v0.11.1

* Revert "Regen historical states for `new-state-mgmt` compatibility (#5261)"

This reverts commit df9a534826.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Revert "Remove deprecated parameters (#5249)" (#5276)

This reverts commit 7d17c9ac34.

* Verify block proposer index before gossip  (#5274)

* Update pipeline

* Update tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Add in Proposer Index to Custom HTR (#5269)

* fix test

* Update beacon-chain/state/stateutil/blocks_test.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Resolve Flakey P2P Tests (#5285)

* double time for flakey test

* fix test flakeyness in p2p:

* flakey

* time tolerance

* greater tolerance

* release resources correctly (#5287)

* Enable NOISE Handshake by Default v0.11 (#5272)

* noise handshakes by default

* fix build

* noisy noise everywhere

* deprecated noisy noise flag with more noise

* add secio as fallback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: nisdas <nishdas93@gmail.com>

* new ports

* fix broken build

* Make `new-state-mgmt` canonical  (#5289)

* Invert the flags
* Update checking messages
* Fixed all db tests
* Fixed rest of the block chain tests
* Fix chain race tests
* Fixed rpc tests
* Disable soudns better...
* Merge branch 'v0.11' into invert-new-state-mgmt
* Merge refs/heads/v0.11 into invert-new-state-mgmt
* Fix export
* Merge branch 'invert-new-state-mgmt' of github.com:prysmaticlabs/prysm into invert-new-state-mgmt
* Fix conflict tests
* Gazelle
* Merge refs/heads/v0.11 into invert-new-state-mgmt
* Merge refs/heads/v0.11 into invert-new-state-mgmt

* resolve flakeyness

* Detect Proposer Slashing Implementation (#5139)

* detect blocks

* detect blocks

* use stub

* use stub

* use stub

* todo

* fix test

* add tests and utils

* fix imports

* fix imports

* fix comment

* todo

* proposerIndex

* fix broken test

* formatting and simplified if

* Update slasher/detection/service.go

* Update slasher/detection/testing/utils.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* fixed up final comments

* better naming

* Update slasher/detection/service.go

* Update slasher/detection/service.go

* Update slasher/detection/service.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

* no more named args

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>

* Add Metadata And Ping RPC methods (#5271)

* add new proto files

* add flag and helper

* add initializer

* imports

* add ping method

* add receive/send ping request

* add ping test

* refactor rpc methods and add ping test

* finish adding all tests

* fix up tests

* Apply suggestions from code review

* lint

* imports

* lint

* Update beacon-chain/p2p/service.go

* Update shared/cmd/flags.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Updates for remote keymanager (#5260)

* Update to slash by slot instead of epoch (#5297)

* change to slash by slot instead of epoch

* gaz

* fix test

* fix test

* fix infinite loop on error parse

* Update proposer protection to v0.11 (#5292)

* Complete most of changes

* Fix other tests

* Test progress

* Tests

* Finish tests

* update pbs

* Fix mocked tests

* Gazelle

* pt 2

* Fix

* Fixes

* Fix tests wit hwrong copying

* Implement `SubscribeCommitteeSubnet` method (#5299)

* Add client implementation

* Update workspace

* Update server

* Update service

* Gaz

* Mocks

* Fixed validator tests

* Add round tirp tests

* Fixed subnet test

* Comment

* Update committee cache

* Comment

* Update RPC

* Fixed test

* Nishant's comment

* Gaz

* Refresh ENR is for epoch

* Needs to be append

* Validator subscribe subnet to next epoch (#5312)

* Alert to subscribe to next epoch

* Fixed tests

* Comments

* Fixed tests

* Update validator/client/validator.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Revert "Revert "Remove deprecated parameters (#5249)" (#5276)" (#5277)

This reverts commit 47e5a2cf96.

* Aggregate on demand for v0.11 (#5302)

* Add client implementation

* Update workspace

* Update server

* Update service

* Gaz

* Mocks

* Fixed validator tests

* Add round tirp tests

* Fixed subnet test

* Wait 1/3 on validator side

* Lint

* Comment

* Update committee cache

* Comment

* Update RPC

* Fixed test

* Nishant's comment

* Gaz

* Refresh ENR is for epoch

* Needs to be append

* Fixed duplication

* Tests

* Skip e2e

* Update beacon-chain/rpc/validator/aggregator.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

* Apply suggestions from code review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: shayzluf <thezluf@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Refactor Dynamic Subscriptions (#5318)

* clean up

* comment

* metrics

* fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Fix listindexed attestations and detect historic attestations (#5321)

* fix list indexed attestations

* fix tests

* goimports

* names

* Add check for slot == 0 (#5322)

* Change attester protection to return default if DB is empty (#5323)

* Change how default values are set

* Remove unused imports

* Remove wasteful db call

* Fix db tests

* Fix db test

* fix it (#5326)

* V0.11 run time fixes to use interop config (#5324)

* Started testing
* Bunch of fixes
* use-interop
* Sync with v0.11
* Conflict
* Uncomment wait for activation
* Move pending block queue from subscriber to validator pipeline
* Merge branch 'v0.11' into use-interop-config
* passing tests
* Merge refs/heads/v0.11 into use-interop-config
* Merge refs/heads/v0.11 into use-interop-config

* Nil Checks in Process Attestation v0.11 (#5331)

* Started testing

* Bunch of fixes

* use-interop

* Sync with v0.11

* Uncomment wait for activation

* Move pending block queue from subscriber to validator pipeline

* passing tests

* nil checks to prevent panics

* lint

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Validator batch subscribe subnets (#5332)

* Update both beacon node and validator

* Comments

* Tests

* Lint

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Validator smarter subscribe (#5334)

* Fix incorrect proposer index calculation (#5336)

* Use correct parent state

* Fixed test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* enhance error

* enhance error

* Update P2P Service to Handle Local Metadata (#5319)

* add metadata to ENR

* add new methods

* glue everything

* fix all tests and refs

* add tests

* add more tests

* Apply suggestions from code review

* fix method

* raul's review

* gaz

* fix test setup

* fix all tests

* better naming

* fix broken test

* validate nil

Co-authored-by: rauljordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Revert "Revert "Revert "Remove deprecated parameters (#5249)" (#5276)" (#5277)" (#5343)

This reverts commit e5aef1686e.

* Wait for Genesis Event to Start P2P (#5303)

* use event feed for state initialized events

* add in handler for tests

* wait till genesis for p2p

* Apply suggestions from code review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Avoid duplicated aggregation request (#5346)

* Avoid duplicated aggregation request

* Test and lock

* Gaz

* Fix Validate For Metadata (#5348)

* return true

* shay's review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Multiple Proposer Slots Allowed Per Epoch for Validators (#5344)

* allow multiple proposer slots

* multi propose

* proposer indices to slots map

* remove deprecated comm assign

* Apply suggestions from code review

* resolve broken tests, add logic in validator client

* fix val tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Networking Fixes (#5349)

* close stream later

* add ping method

* add method

* lint

* More efficient aggregation on demand (#5354)

* Return Nil Error if Pre-Genesis in P2P Service Healthz Check (#5355)

* pregenesis healthz check:

* optimal

* right order

* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* no comment

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Release DiscoveryV5 for Testnet Restart (#5357)

* release discv5

* fix build

* Fix Overflow in Status Check (#5361)

* fix overflow

* Apply suggestions from code review

* fix after merge

* Make Mainnet Config Default, No More Demo Config  (#5367)

* bye bye demo config

* gaz

* fix usage

* fix dep

* gaz

* Update default balance for sendDeposits

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Use FastSSZ Marshal/Unmarshal for DB Encodings in v0.11.1 (#5351)

* try

* use marshaler structure for db instead of proto

* white list types

* attempt

* revert

* testutil.NewBeaconState()

* Fully populate fields for round trip ssz marshal

* fix //beacon-chain/db/kv:go_default_test

* more passing tests

* another test target passed

* fixed stategen

* blockchain tests green

* passing sync

* more targets fixed

* more test fixes in rpc/validator

* most rpc val

* validators test fixes

* skip round robin old

* aggregate test

* whitelist done

* Update beacon-chain/rpc/validator/attester_test.go

* edit baz

* Fixed tests

* Fixed getblock test

* Add back init

* reduce test size

* fix broken build

* tests pass

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Reconnect slasher streams on beacon node shutdown (#5376)

* restart streams on beacon node shutdown

* fix comment

* remove export

* ivan feedback

* ivan feedback

* case insensitive

* Update slasher/beaconclient/receivers.go

* raul feedback

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Amend Faucet to Offer 32.5 ETH for v0.11 (#5378)

* deposit amount in faucet

* fix eth amount

* gas cost

* unskip exec transition test

* Revert "Enable NOISE Handshake by Default v0.11 (#5272)" (#5381)

This reverts commit a8d32d504a.

* use string for deposit flag

* Update Bootnode to v0.11 (#5387)

* fix bootnode

* add changes

* gaz

* fix docker

* build fix

* fix flaky test

* Unskip E2E for V0.11 (#5386)

* Begin work on fixing e2e for v0.11

* Start bootnode work

* Begin implementing bootnode into e2e

* Fix E2E for v0.11

* Remove extra

* gaz

* Remove unused key gen code

* Remove trailing multiaddr code

* add skip for slashing

* Fix slashing e2e

* Fix docker image build

* Update beacon-chain/p2p/broadcaster_test.go

* Update GetValidatorParticipation

* Update tests

* Gaz

* Pass E2E Tests for v0.11 and Enable Attestation Subnets By Default (#5407)

* Update README.md

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Apply suggestions from code review

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/p2p/config.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update shared/keystore/deposit_input.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update tools/faucet/server.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update proto/beacon/p2p/v1/BUILD.bazel

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update shared/bls/spectest/aggregate_verify_test.go

* Addressed feedback. All test passing

* Update beacon-chain/core/blocks/block_operations_fuzz_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

* Update beacon-chain/core/blocks/block_operations_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

* Update shared/testutil/helpers.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

* Update beacon-chain/core/helpers/signing_root.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

* Resolve Misc v0.11 Items (Raul) (#5414)

* address all comments

* set faucet

* nishant feedback

* Update beacon-chain/p2p/service.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Revert keymanager changes (#5416)

* Revert "Updates for remote keymanager (#5260)"

This reverts commit bbcd895db5.

* Revert "Remove keystore keymanager from validator (#5236)"

This reverts commit 46008770c1.

* Revert "Update eth2 wallet keymanager (#4984)"

This reverts commit 7f7ef43f21.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Update BLS and limit visibility (#5415)

* remove duplicated BLS, add golang.org/x/mod

* Update BLS and restrict visibility

* fix build

* Fix eth1data test and fix order of ops (#5413)

* use multiaddr builder (#5419)

* Unskip benchutil and minor v0.11 fixes (#5417)

* Unskip benchutil tests

* Remove protos and gaz

* Fixes

* Networking Fixes (#5421)

* check

* fix test

* fix size

* fix test

* more fixes

* fix test again

* Update ethereum APIs with latest master

* Error handling for v0.11 tests (#5428)

* Proper err handling for tests

* Lint

* Fixed rest of the tests

* Gaz

* Fixed old master tests

* Rm old aggregate_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
Co-authored-by: Jim McDonald <Jim@mcdee.net>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: shayzluf <thezluf@gmail.com>
* Validator participation with fallback
* Add back the old tests
* Fixed test
* Merge branch 'master' into validator-participation-fallback
* Merge refs/heads/master into validator-participation-fallback
2020-04-20 01:39:15 +00:00
terence tsao
7d8516cd71 Add back archival endpoint ListValidatorAssignments with fallback (#5517)
* Modify `ListValidatorAssignments` to use new state service (#5365)

* Fixed rest of non spec tests
* Fixed a bug proposer index wasn't included
* gaz
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Updated eth1 data voting period to epoch based
* Fixed failed tests
* fix bug
* fix error
* Fixed more misc tests
* Add new SignedAggregateAndProof to pass spec test
* Update minimalConfig.PersistentCommitteePeriod
* allow to rebuild trie
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Skip e2e tests
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Align aggregator action with v0.11 (#5146)
* Remove Head Root from Beacon Block by Range Request (#5165)

* make proto changes
* remove head root
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.11
* add back herumi's library
* Update ethapi in workspace, started fixing test. Hand off to Nishant
* fix build
* All tests passing
* Align finalized slot check with v0.11 (#5166)
* Merge branch 'master' into v0.11
* Add DoS resistance for v0.11 (#5158)
* Add Fork Digest Helper (#5173)
* Extend DoS prevention to rest of operation objects (#5174)

* Update mapping

* Add caches

* Update seen block in validation pipeline

* Update seen att in validation pipeline

* Update seen att in validation pipeline

* Fixed rest of tests

* Gazelle

* Better writes

* Lint

* Preston's feedback

* Switched to LRU cache and fixed tests

* Gazelle

* Fix test

* Update proposer slashing

* Update proposer slashing

* Fixed a block test

* Update exit

* Update atteser slashing

* Raul's feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Add remote keymanager (#5133)

* Add remote keymanager

* Add generic signRoot() helper

* Add tests for remote keymanager

* NewRemote -> NewRemoteWallet

* signRoot -> signOject, to increase reuse

* Fix end-to-end compile error

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Add Snappy Framing to the Encoder (#5172)

* change to framing

* more fixes

* fix everything

* add stricter limits

* preston feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: rauljordan <raul@prysmaticlabs.com>
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Move Subnet Functionality to its Own File (#5179)

* move subnets to their own file

* fix build fail

* build

* Update beacon-chain/p2p/discovery_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Sync with master
* Verify proposer signature in sync (#5206)
* Fix Signed Attestation In Sync (#5207)
* Add Eth2 Fork ENR Functionality (#5181)

* add fork entry enr

* add in fork

* add the required fork entry to node

* add and retrieve fork entry

* await state initialized

* utilize new structure

* more progress, utilizing a config map instead

* send the genesis validators root via the event feed

* struct method for discovery

* fix broken builds

* fixed up more tsts using state feed initializer

* fix up most tests

* only one more failing test

* almost done with tests

* p2p tests all pass

* config fix

* fix blockchain test

* gaz

* add in todo

* lint

* add compare func

* ensure fork ENR versions match between peers

* add in test for discovery

* test name

* tests complete

* tests done

* done

* comments

* fix all flakes

* addressed comments

* build using ssz gen

* marshal record

* use custom ssz

* deduplicate import

* fix build

* add enr proto

* p2p tests done

Co-authored-by: nisdas <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Verify aggregator signature in sync (#5208)
* Add Fork Digest For Gossip Topics (#5191)

* update for the day

* fix remaining failing test

* fix one more test

* change message

* Apply suggestions from code review

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* terence's review

* implement fork digest'

* align digest to interface'

* passed all tests

* spawn in goroutine

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Fix Incorrect Attester Slashing Method (#5229)
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Remove keystore keymanager from validator (#5236)

* Remove keystore keymanager from validator

* Update dependency

* Update validator/flags/flags.go

* Update validator/flags/flags.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
* fix broadcaster
* update metrics with fork digest for p2p (#5251)

* update metrics with fork digest for p2p

* update p2p metrics

* update metrics using att values

* wrapped up

* fix bug

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Fix incorrect domain type comments (#5250)

* Fix incorrect domain type comments
* resolve conflicts
* fix broken broadcast test
* fix tests
* include protocol suffix
* fix confs
* lint
* fix test
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* resolve broken slasher test'
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Merge branch 'master' into v0.11
* fix config override
* Remove deprecated parameters (#5249)
* Avoid div by zero in extreme balance case (#5273)

* Return effective balance increment instead of 1

* Update to new spec tests v0.11.1

* Revert "Regen historical states for `new-state-mgmt` compatibility (#5261)"

This reverts commit df9a534826.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Revert "Remove deprecated parameters (#5249)" (#5276)

This reverts commit 7d17c9ac34.
* Verify block proposer index before gossip  (#5274)

* Update pipeline

* Update tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Add in Proposer Index to Custom HTR (#5269)

* fix test

* Update beacon-chain/state/stateutil/blocks_test.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Resolve Flakey P2P Tests (#5285)

* double time for flakey test

* fix test flakeyness in p2p:

* flakey

* time tolerance

* greater tolerance
* Merge branch 'master' into v0.11
* release resources correctly (#5287)
* Merge refs/heads/master into v0.11
* Enable NOISE Handshake by Default v0.11 (#5272)

* noise handshakes by default

* fix build

* noisy noise everywhere

* deprecated noisy noise flag with more noise

* add secio as fallback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
* Merge refs/heads/master into v0.11
* new ports
* fix broken build
* Make `new-state-mgmt` canonical  (#5289)

* Invert the flags
* Update checking messages
* Fixed all db tests
* Fixed rest of the block chain tests
* Fix chain race tests
* Fixed rpc tests
* Disable soudns better...
* Merge branch 'v0.11' into invert-new-state-mgmt
* Merge refs/heads/v0.11 into invert-new-state-mgmt
* Fix export
* Merge branch 'invert-new-state-mgmt' of github.com:prysmaticlabs/prysm into invert-new-state-mgmt
* Fix conflict tests
* Gazelle
* Merge refs/heads/v0.11 into invert-new-state-mgmt
* Merge refs/heads/v0.11 into invert-new-state-mgmt
* Merge branch 'master' into v0.11
* resolve flakeyness
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Detect Proposer Slashing Implementation (#5139)

* detect blocks

* detect blocks

* use stub

* use stub

* use stub

* todo

* fix test

* add tests and utils

* fix imports

* fix imports

* fix comment

* todo

* proposerIndex

* fix broken test

* formatting and simplified if

* Update slasher/detection/service.go

* Update slasher/detection/testing/utils.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* fixed up final comments

* better naming

* Update slasher/detection/service.go

* Update slasher/detection/service.go

* Update slasher/detection/service.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

* no more named args

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.11
* Add Metadata And Ping RPC methods (#5271)

* add new proto files

* add flag and helper

* add initializer

* imports

* add ping method

* add receive/send ping request

* add ping test

* refactor rpc methods and add ping test

* finish adding all tests

* fix up tests

* Apply suggestions from code review

* lint

* imports

* lint

* Update beacon-chain/p2p/service.go

* Update shared/cmd/flags.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.11
* Updates for remote keymanager (#5260)
* Merge branch 'spec-v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Merge remote-tracking branch 'origin' into v0.11
* Update to slash by slot instead of epoch (#5297)

* change to slash by slot instead of epoch

* gaz

* fix test

* fix test

* fix infinite loop on error parse
* Sync with master
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Update proposer protection to v0.11 (#5292)

* Complete most of changes

* Fix other tests

* Test progress

* Tests

* Finish tests

* update pbs

* Fix mocked tests

* Gazelle

* pt 2

* Fix

* Fixes

* Fix tests wit hwrong copying
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Implement `SubscribeCommitteeSubnet` method (#5299)

* Add client implementation

* Update workspace

* Update server

* Update service

* Gaz

* Mocks

* Fixed validator tests

* Add round tirp tests

* Fixed subnet test

* Comment

* Update committee cache

* Comment

* Update RPC

* Fixed test

* Nishant's comment

* Gaz

* Refresh ENR is for epoch

* Needs to be append
* Merge refs/heads/master into v0.11
* resolve confs
* Validator subscribe subnet to next epoch (#5312)

* Alert to subscribe to next epoch

* Fixed tests

* Comments

* Fixed tests

* Update validator/client/validator.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Revert "Revert "Remove deprecated parameters (#5249)" (#5276)" (#5277)

This reverts commit 47e5a2cf96.
* Aggregate on demand for v0.11 (#5302)

* Add client implementation

* Update workspace

* Update server

* Update service

* Gaz

* Mocks

* Fixed validator tests

* Add round tirp tests

* Fixed subnet test

* Wait 1/3 on validator side

* Lint

* Comment

* Update committee cache

* Comment

* Update RPC

* Fixed test

* Nishant's comment

* Gaz

* Refresh ENR is for epoch

* Needs to be append

* Fixed duplication

* Tests

* Skip e2e

* Update beacon-chain/rpc/validator/aggregator.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

* Apply suggestions from code review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: shayzluf <thezluf@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Refactor Dynamic Subscriptions (#5318)

* clean up

* comment

* metrics

* fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge refs/heads/master into v0.11
* Fix listindexed attestations and detect historic attestations (#5321)

* fix list indexed attestations

* fix tests

* goimports

* names
* Add check for slot == 0 (#5322)
* Change attester protection to return default if DB is empty (#5323)

* Change how default values are set

* Remove unused imports

* Remove wasteful db call

* Fix db tests

* Fix db test
* Merge refs/heads/master into v0.11
* fix it (#5326)
* V0.11 run time fixes to use interop config (#5324)

* Started testing
* Bunch of fixes
* use-interop
* Sync with v0.11
* Conflict
* Uncomment wait for activation
* Move pending block queue from subscriber to validator pipeline
* Merge branch 'v0.11' into use-interop-config
* passing tests
* Merge refs/heads/v0.11 into use-interop-config
* Merge refs/heads/v0.11 into use-interop-config
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Nil Checks in Process Attestation v0.11 (#5331)

* Started testing

* Bunch of fixes

* use-interop

* Sync with v0.11

* Uncomment wait for activation

* Move pending block queue from subscriber to validator pipeline

* passing tests

* nil checks to prevent panics

* lint

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Validator batch subscribe subnets (#5332)

* Update both beacon node and validator

* Comments

* Tests

* Lint

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Validator smarter subscribe (#5334)
* Fix incorrect proposer index calculation (#5336)

* Use correct parent state

* Fixed test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* enhance error
* enhance error
* Update P2P Service to Handle Local Metadata (#5319)

* add metadata to ENR

* add new methods

* glue everything

* fix all tests and refs

* add tests

* add more tests

* Apply suggestions from code review

* fix method

* raul's review

* gaz

* fix test setup

* fix all tests

* better naming

* fix broken test

* validate nil

Co-authored-by: rauljordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Revert "Revert "Revert "Remove deprecated parameters (#5249)" (#5276)" (#5277)" (#5343)

This reverts commit e5aef1686e.
* Wait for Genesis Event to Start P2P (#5303)

* use event feed for state initialized events

* add in handler for tests

* wait till genesis for p2p

* Apply suggestions from code review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge refs/heads/master into v0.11
* Avoid duplicated aggregation request (#5346)

* Avoid duplicated aggregation request

* Test and lock

* Gaz
* Fix Validate For Metadata (#5348)

* return true

* shay's review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Multiple Proposer Slots Allowed Per Epoch for Validators (#5344)

* allow multiple proposer slots

* multi propose

* proposer indices to slots map

* remove deprecated comm assign

* Apply suggestions from code review

* resolve broken tests, add logic in validator client

* fix val tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Networking Fixes (#5349)

* close stream later

* add ping method

* add method

* lint
* More efficient aggregation on demand (#5354)
* Return Nil Error if Pre-Genesis in P2P Service Healthz Check (#5355)

* pregenesis healthz check:

* optimal

* right order

* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* no comment

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* STarting
* Release DiscoveryV5 for Testnet Restart (#5357)

* release discv5

* fix build
* Fix Overflow in Status Check (#5361)

* fix overflow

* Apply suggestions from code review
* Update assignments
* Genesis short circuit
* Fixed tests
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* fix after merge
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Merge refs/heads/master into v0.11
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Make Mainnet Config Default, No More Demo Config  (#5367)

* bye bye demo config

* gaz

* fix usage

* fix dep

* gaz

* Update default balance for sendDeposits

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Use FastSSZ Marshal/Unmarshal for DB Encodings in v0.11.1 (#5351)

* try

* use marshaler structure for db instead of proto

* white list types

* attempt

* revert

* testutil.NewBeaconState()

* Fully populate fields for round trip ssz marshal

* fix //beacon-chain/db/kv:go_default_test

* more passing tests

* another test target passed

* fixed stategen

* blockchain tests green

* passing sync

* more targets fixed

* more test fixes in rpc/validator

* most rpc val

* validators test fixes

* skip round robin old

* aggregate test

* whitelist done

* Update beacon-chain/rpc/validator/attester_test.go

* edit baz

* Fixed tests

* Fixed getblock test

* Add back init

* reduce test size

* fix broken build

* tests pass

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Reconnect slasher streams on beacon node shutdown (#5376)

* restart streams on beacon node shutdown

* fix comment

* remove export

* ivan feedback

* ivan feedback

* case insensitive

* Update slasher/beaconclient/receivers.go

* raul feedback

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Merge branch 'master' into v0.11
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Merge refs/heads/master into v0.11
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Amend Faucet to Offer 32.5 ETH for v0.11 (#5378)

* deposit amount in faucet

* fix eth amount

* gas cost
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* unskip exec transition test
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Revert "Enable NOISE Handshake by Default v0.11 (#5272)" (#5381)

This reverts commit a8d32d504a.
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Merge refs/heads/master into v0.11
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* use string for deposit flag
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update Bootnode to v0.11 (#5387)

* fix bootnode

* add changes

* gaz

* fix docker
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Fixed conflict
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* build fix
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* fix flaky test
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Merge refs/heads/master into v0.11
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into fix-list-validator-assignments
* Fixed more conflicts
* Merge branch 'fix-list-validator-assignments' of github.com:prysmaticlabs/prysm into fix-list-validator-assignments
* Unskip E2E for V0.11 (#5386)

* Begin work on fixing e2e for v0.11

* Start bootnode work

* Begin implementing bootnode into e2e

* Fix E2E for v0.11

* Remove extra

* gaz

* Remove unused key gen code

* Remove trailing multiaddr code

* add skip for slashing

* Fix slashing e2e

* Fix docker image build
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.11
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Merge refs/heads/master into v0.11
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update beacon-chain/p2p/broadcaster_test.go
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into fix-list-validator-assignments
* Fixed test
* Merge branch 'fix-list-validator-assignments' of github.com:prysmaticlabs/prysm into fix-list-validator-assignments
* Merge refs/heads/master into v0.11
* Merge branch 'v0.11' into fix-list-validator-assignments
* Pass E2E Tests for v0.11 and Enable Attestation Subnets By Default (#5407)
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update README.md

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Apply suggestions from code review

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update beacon-chain/p2p/config.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update shared/keystore/deposit_input.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update tools/faucet/server.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update proto/beacon/p2p/v1/BUILD.bazel

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update shared/bls/spectest/aggregate_verify_test.go
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Addressed feedback. All test passing
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update beacon-chain/core/blocks/block_operations_fuzz_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update beacon-chain/core/blocks/block_operations_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update shared/testutil/helpers.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update beacon-chain/core/helpers/signing_root.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Resolve Misc v0.11 Items (Raul) (#5414)

* address all comments

* set faucet

* nishant feedback

* Update beacon-chain/p2p/service.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Revert keymanager changes (#5416)

* Revert "Updates for remote keymanager (#5260)"

This reverts commit bbcd895db5.

* Revert "Remove keystore keymanager from validator (#5236)"

This reverts commit 46008770c1.

* Revert "Update eth2 wallet keymanager (#4984)"

This reverts commit 7f7ef43f21.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update BLS and limit visibility (#5415)

* remove duplicated BLS, add golang.org/x/mod

* Update BLS and restrict visibility

* fix build
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Fix eth1data test and fix order of ops (#5413)
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* use multiaddr builder (#5419)
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Unskip benchutil and minor v0.11 fixes (#5417)

* Unskip benchutil tests

* Remove protos and gaz

* Fixes
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Networking Fixes (#5421)

* check

* fix test

* fix size

* fix test

* more fixes

* fix test again
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update ethereum APIs with latest master
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Error handling for v0.11 tests (#5428)

* Proper err handling for tests

* Lint

* Fixed rest of the tests

* Gaz

* Fixed old master tests
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Sync with master
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Rm old aggregate_test.go
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Merge branch 'master' into fix-list-validator-assignments
* Fixed conflicts
* Merge refs/heads/master into fix-list-validator-assignments
* Merge refs/heads/master into fix-list-validator-assignments
* Merge refs/heads/master into fix-list-validator-assignments
* Merge refs/heads/master into fix-list-validator-assignments
* Merge refs/heads/master into fix-list-validator-assignments
* Merge refs/heads/master into fix-list-validator-assignments
* Merge refs/heads/master into fix-list-validator-assignments
* Merge refs/heads/master into fix-list-validator-assignments
* Merge refs/heads/master into fix-list-validator-assignments
* Merge refs/heads/master into fix-list-validator-assignments
* List validator assignments with fallback
* Invert the flag
* Merge refs/heads/master into list-validator-assignment-fallback
2020-04-20 01:19:11 +00:00
Shay Zluf
95f1676d5f Fix broadcast slashing flag (#5515) 2020-04-19 16:41:59 -04:00
Ivan Martinez
42e6532d5b Fix prysm.sh for arm64 (#5511) 2020-04-19 11:58:01 -04:00
Shay Zluf
acd7dd137f More efficient list indexed attestation endpoint (#5441)
* use state gen to retrieve state fo computing committees

* gaz + goimports

* remove duplicate

* main net config

* committee instead of committee

* remove

* handle feature config

* Update beacon-chain/rpc/beacon/attestations.go

Co-Authored-By: Victor Farazdagi <simple.square@gmail.com>

* remove duplication

* group attestations by block root

* fix errors

* fix test

* ivan feedback

* add test

* return error if new state management is turned off

* fix featureconfig handling

* gaz

* Update beacon-chain/rpc/beacon/attestations.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-04-19 18:13:33 +03:00
Nishant Das
626b3e0c66 Add Reference Copying (#5245)
* add state changes
* add new changes
* add flag
* lint
* add flag
* change to correct bool
* fixing and consolidating trie
* lint
* Apply suggestions from code review
* Merge refs/heads/master into addBetterCopying
* Merge branch 'master' into addBetterCopying
* refCopy -> stateRefCopy
* Merge refs/heads/master into addBetterCopying
* tests whether unexpected mutation of validators within state is avoided
* Merge branch 'addBetterCopying' of github.com:prysmaticlabs/prysm into addBetterCopying
* remove unnecessary fields
* gazelle
* updates test
* avoid unexpected mutation in block roots on refcopy
* avoid unexpected mutation in state roots on refcopy
* Merge refs/heads/master into addBetterCopying
* Merge branch 'master' into addBetterCopying
* fix test
* randao tests
* simplify tests
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* test cur/prev attestations mutation
* Merge branch 'addBetterCopying' of github.com:prysmaticlabs/prysm into addBetterCopying
* gazelle
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* fixes tests
* minor naming update
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
2020-04-19 13:57:43 +00:00
Ivan Martinez
dee3f02e2c Change prometheus port for different parts of Prysm (#5504)
* Change prometheus flag from default 8080 and sort flags
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into change-prometheus-port
* Fix build
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into change-prometheus-port
* Merge branch 'master' into change-prometheus-port
2020-04-19 08:36:19 +00:00
Ivan Martinez
f2d88ea920 Fix prysm.sh for arm64 (#5507)
* Fix prysm script for arm
2020-04-19 08:15:52 +00:00
Preston Van Loon
33774721bb p2p/sync: Wrong genesis digest is used to register pubsub topic subscriptions (race condition) (#5500)
* Return error when genesis time or genesis root is not set
* Merge refs/heads/master into fail-on-unset-fork-digest
* move to own helpers
* make it non public
* Merge branch 'fail-on-unset-fork-digest' of https://github.com/prysmaticlabs/geth-sharding into fail-on-unset-fork-digest
* lint
* fix
* return error
* fix tests and error
* Merge refs/heads/master into fail-on-unset-fork-digest
* first round of test fixes
* second round of fixes
* Merge branch 'fail-on-unset-fork-digest' of https://github.com/prysmaticlabs/geth-sharding into fail-on-unset-fork-digest
* lint
* Merge refs/heads/master into fail-on-unset-fork-digest
* gaz
* Merge branch 'fail-on-unset-fork-digest' of https://github.com/prysmaticlabs/geth-sharding into fail-on-unset-fork-digest
2020-04-19 02:32:53 +00:00
Preston Van Loon
0ed0cb58f8 Configurable pubsub router (#5499)
* A p2p flag for configuring pubsub router
* Add randomsub
* lint
* Merge refs/heads/master into configurable-pubsub-router
* Default to gossip
* Merge branch 'configurable-pubsub-router' of github.com:prysmaticlabs/prysm into configurable-pubsub-router
* Merge refs/heads/master into configurable-pubsub-router
* Add flag to usage
* Merge branch 'configurable-pubsub-router' of github.com:prysmaticlabs/prysm into configurable-pubsub-router
* Fix build
* Merge refs/heads/master into configurable-pubsub-router
2020-04-19 01:53:32 +00:00
Raul Jordan
62fa6ed2e9 Better Bug Report and Issue Templates for Prysm (#5505)
* issue templates
* ask for flags
* Merge branch 'master' into issue-templ
* rules node
* Merge branch 'issue-templ' of github.com:prysmaticlabs/prysm into issue-templ
2020-04-19 01:32:02 +00:00
terence tsao
c3217abb83 Save init sync cached blocks before regenerate state (#5501)
* Revert "Add WaitForSynced to beacon node for validator startup (#5366)"

This reverts commit 1224e75ba1.
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Revert "Add Separate Network Config (#5454)"

This reverts commit 28733f2c9e.
* Save blocks to db before generating state
* Revert "Revert "Add Separate Network Config (#5454)""

This reverts commit d43c2b7dbd.
* Revert "Revert "Add WaitForSynced to beacon node for validator startup (#5366)""

This reverts commit 927d8d9e06.
2020-04-18 23:29:58 +00:00
Raul Jordan
fb26177a2a Disable Attester/Proposer Slashing Protection (#5497)
* toggle flags
* Merge branch 'master' into toggle-attester
* Merge refs/heads/master into toggle-attester
2020-04-18 21:47:41 +00:00
terence tsao
0078f42554 Disable some block checks without state mgmt (#5498)
* Revert "Add WaitForSynced to beacon node for validator startup (#5366)"

This reverts commit 1224e75ba1.
* Disable various checks in block validator sync without new state mgmt
* Revert "Revert "Add WaitForSynced to beacon node for validator startup (#5366)""

This reverts commit 927d8d9e06.
* Invert
2020-04-18 20:08:55 +00:00
Ivan Martinez
1224e75ba1 Add WaitForSynced to beacon node for validator startup (#5366)
* Add WaitForSynced to beacon-chain
* Fix build issues
* Fix comment
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into simplify-validator
* Fix tests
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into simplify-validator
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into simplify-validator
* Merge branch 'master' into simplify-validator
* Merge branch 'master' into simplify-validator
* Merge branch 'master' into simplify-validator
2020-04-18 10:45:20 +00:00
Nishant Das
b0af12d5dc Add Geth Logging (#5474)
* add new logging
* Update beacon-chain/main.go
* Merge refs/heads/master into gethLogging
* Merge refs/heads/master into gethLogging
* Merge refs/heads/master into gethLogging
* Merge refs/heads/master into gethLogging
* fix image
* Merge refs/heads/master into gethLogging
2020-04-18 06:23:45 +00:00
Nishant Das
c0ca567175 Less Scary Connection Logs (#5480)
* change logs
* Merge refs/heads/master into lessScaryLogs
* Merge refs/heads/master into lessScaryLogs
* Merge refs/heads/master into lessScaryLogs
2020-04-18 05:58:21 +00:00
Nishant Das
9cec33fe24 Fix Bootnode Again (#5473)
* bootnodeFix
* fix again
* remove iputils
* revert changes
* only add fallback
* fix test
* take it outside of func
* fix docker
* Merge branch 'master' into fixBootnodeAgain
* Merge refs/heads/master into fixBootnodeAgain
2020-04-18 05:06:06 +00:00
terence tsao
62213ca602 Retrieve last ancestor state from DB. (Radical improvement from 5466) (#5475)
* Add `lastAncestorState` method
* Add new tests and fix existing test
* Check context deadline exceeds
* Merge refs/heads/master into last-ancestor-state
* Merge refs/heads/master into last-ancestor-state
2020-04-18 01:04:25 +00:00
terence tsao
ce6ce5a397 Add HasBlock to verify parent check (#5478)
* Add HasBlock to verify parent check
* Comment
* Fixed existing tests
* Merge refs/heads/master into fix-verify-blk-pre-state
2020-04-18 00:32:45 +00:00
Ivan Martinez
7a65e0ee3f Change prysm script to support windows (#5479)
* Change prysm script to support windows
* Update prysm.sh
2020-04-17 19:11:51 +00:00
Preston Van Loon
475d428fce Add default bootnode for discv5 (#5471)
* Add default bootnode for discv5
2020-04-17 06:19:22 +00:00
Nishant Das
2f15f16650 Fix Bootnode (#5469)
* disable kademlia
* Merge branch 'master' into fixBootnode2
* fix
* Merge branch 'fixBootnode2' of https://github.com/prysmaticlabs/geth-sharding into fixBootnode2
* fix
* fix test
2020-04-17 05:40:14 +00:00
Preston Van Loon
46b19072f4 Fix overflow in currentSlot (#5467)
* Fix overflow
* Merge branch 'master' into overflow-current-slot
2020-04-17 03:18:07 +00:00
terence tsao
50d67f52bb Replay and generate state using proper parent root (#5466)
* Start replay using parent root
* Merge branch 'master' into fix-replay-bug
* Fixed TestStateByRoot_HotStateDB
* Merge branch 'fix-replay-bug' of github.com:prysmaticlabs/prysm into fix-replay-bug
* Fixed TestOnAtts
* Update beacon-chain/state/stategen/getter_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update beacon-chain/state/stategen/getter_test.go
* Fixed TestLoadHoteStateByRoot_FromDBCanProcess
* Merge branch 'fix-replay-bug' of github.com:prysmaticlabs/prysm into fix-replay-bug
2020-04-17 02:06:23 +00:00
terence tsao
62c25eaedb Revert archival deprecations (#5464)
* Revert "Fix `ListValidatorBalances` for v0.11 (#5458)"

This reverts commit 3763a8ce5d.
* Revert "Deprecate archival service (#5445)"

This reverts commit 4fbcedf541.
* Revert "Delete archival DB methods (#5459)"

This reverts commit 8a3ea1e936.
* Revert "Modify `ListBeaconCommittees ` to use new state service  (#5411)"

This reverts commit 497fa6ed50.
* Revert "Modify `GetValidatorParticipation` to use new state service (#5409)"

This reverts commit 046a00aa87.
* Revert "Modify `GetValidatorActiveSetChanges` to use new state service  (#5408)"

This reverts commit 5eb6485e14.
* Revert "Modify `ListValidatorAssignments` to use new state service (#5365)"

This reverts commit 5a1a768135.
2020-04-16 23:26:10 +00:00
Nishant Das
28733f2c9e Add Separate Network Config (#5454)
* clean up config
* add discv5 config
* add gaz
* Merge branch 'master' into configCleanup
* Merge refs/heads/master into configCleanup
* Merge refs/heads/master into configCleanup
* Merge refs/heads/master into configCleanup
* Merge refs/heads/master into configCleanup
* Merge refs/heads/master into configCleanup
* Merge refs/heads/master into configCleanup
2020-04-16 22:21:44 +00:00
Jim McDonald
0803f0f0ff Update wallet keymanager for v0.11 (#5453)
* Update wallet keystore for spec 0.11
* Add herumi
* Use local herumi
* Merge branch 'master' into wallet-update
* Updates after feedback
* Merge branch 'wallet-update' of github.com:mcdee/prysm into wallet-update
* Merge branch 'master' into wallet-update
* Update WORKSPACE
2020-04-16 21:56:13 +00:00
Raul Jordan
52b46d4570 Debug log pubkey chainstart already added (#5461)
* debug logging
* Merge refs/heads/master into pubkey-log
2020-04-16 20:46:14 +00:00
terence tsao
8a3ea1e936 Delete archival DB methods (#5459) 2020-04-16 13:21:25 -07:00
Eduardo Antuña Díez
17c1fc3250 Added dappnode-release-trigger.yml workflow (#5455)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-04-16 10:10:18 -07:00
terence tsao
3763a8ce5d Fix ListValidatorBalances for v0.11 (#5458)
* Fix ListValidatorBalances
* Fixed all the tests
* Merge refs/heads/master into fix-list-balances
* Use requestedEpoch as default
* Merge branch 'fix-list-balances' of github.com:prysmaticlabs/prysm into fix-list-balances
* Fixed a test
2020-04-16 16:37:27 +00:00
Victor Farazdagi
075d29f5b4 nil attestation tests added (#5457)
* nil attestation tests added
2020-04-16 15:39:53 +00:00
Ivan Martinez
c70103bdb3 Add healthz check to E2E (#5426)
* Add healthz check to E2E
* Log fixes
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into e2e-healthz
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into e2e-healthz
* Merge branch 'master' into e2e-healthz
* Merge branch 'master' into e2e-healthz
* Merge branch 'master' into e2e-healthz
* Merge branch 'master' into e2e-healthz
2020-04-15 21:20:58 +00:00
Victor Farazdagi
0ecd2a6dc2 updates UpdateRandaoMixesAtIndex signature (#5447)
* updates signature
* Merge branch 'master' into fix-update-randao-mixes-signature
2020-04-15 20:35:19 +00:00
terence tsao
4fbcedf541 Deprecate archival service (#5445)
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Remove keystore keymanager from validator (#5236)

* Remove keystore keymanager from validator

* Update dependency

* Update validator/flags/flags.go

* Update validator/flags/flags.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
* fix broadcaster
* update metrics with fork digest for p2p (#5251)

* update metrics with fork digest for p2p

* update p2p metrics

* update metrics using att values

* wrapped up

* fix bug

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Fix incorrect domain type comments (#5250)

* Fix incorrect domain type comments
* resolve conflicts
* fix broken broadcast test
* fix tests
* include protocol suffix
* fix confs
* lint
* fix test
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* resolve broken slasher test'
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Merge branch 'master' into v0.11
* fix config override
* Remove deprecated parameters (#5249)
* Avoid div by zero in extreme balance case (#5273)

* Return effective balance increment instead of 1

* Update to new spec tests v0.11.1

* Revert "Regen historical states for `new-state-mgmt` compatibility (#5261)"

This reverts commit df9a534826.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Revert "Remove deprecated parameters (#5249)" (#5276)

This reverts commit 7d17c9ac34.
* Verify block proposer index before gossip  (#5274)

* Update pipeline

* Update tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Add in Proposer Index to Custom HTR (#5269)

* fix test

* Update beacon-chain/state/stateutil/blocks_test.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Resolve Flakey P2P Tests (#5285)

* double time for flakey test

* fix test flakeyness in p2p:

* flakey

* time tolerance

* greater tolerance
* Merge branch 'master' into v0.11
* release resources correctly (#5287)
* Merge refs/heads/master into v0.11
* Enable NOISE Handshake by Default v0.11 (#5272)

* noise handshakes by default

* fix build

* noisy noise everywhere

* deprecated noisy noise flag with more noise

* add secio as fallback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
* Merge refs/heads/master into v0.11
* new ports
* fix broken build
* Make `new-state-mgmt` canonical  (#5289)

* Invert the flags
* Update checking messages
* Fixed all db tests
* Fixed rest of the block chain tests
* Fix chain race tests
* Fixed rpc tests
* Disable soudns better...
* Merge branch 'v0.11' into invert-new-state-mgmt
* Merge refs/heads/v0.11 into invert-new-state-mgmt
* Fix export
* Merge branch 'invert-new-state-mgmt' of github.com:prysmaticlabs/prysm into invert-new-state-mgmt
* Fix conflict tests
* Gazelle
* Merge refs/heads/v0.11 into invert-new-state-mgmt
* Merge refs/heads/v0.11 into invert-new-state-mgmt
* Merge branch 'master' into v0.11
* resolve flakeyness
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Detect Proposer Slashing Implementation (#5139)

* detect blocks

* detect blocks

* use stub

* use stub

* use stub

* todo

* fix test

* add tests and utils

* fix imports

* fix imports

* fix comment

* todo

* proposerIndex

* fix broken test

* formatting and simplified if

* Update slasher/detection/service.go

* Update slasher/detection/testing/utils.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* fixed up final comments

* better naming

* Update slasher/detection/service.go

* Update slasher/detection/service.go

* Update slasher/detection/service.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

* no more named args

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.11
* Add Metadata And Ping RPC methods (#5271)

* add new proto files

* add flag and helper

* add initializer

* imports

* add ping method

* add receive/send ping request

* add ping test

* refactor rpc methods and add ping test

* finish adding all tests

* fix up tests

* Apply suggestions from code review

* lint

* imports

* lint

* Update beacon-chain/p2p/service.go

* Update shared/cmd/flags.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.11
* Updates for remote keymanager (#5260)
* Merge branch 'spec-v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Merge remote-tracking branch 'origin' into v0.11
* Update to slash by slot instead of epoch (#5297)

* change to slash by slot instead of epoch

* gaz

* fix test

* fix test

* fix infinite loop on error parse
* Sync with master
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Update proposer protection to v0.11 (#5292)

* Complete most of changes

* Fix other tests

* Test progress

* Tests

* Finish tests

* update pbs

* Fix mocked tests

* Gazelle

* pt 2

* Fix

* Fixes

* Fix tests wit hwrong copying
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Implement `SubscribeCommitteeSubnet` method (#5299)

* Add client implementation

* Update workspace

* Update server

* Update service

* Gaz

* Mocks

* Fixed validator tests

* Add round tirp tests

* Fixed subnet test

* Comment

* Update committee cache

* Comment

* Update RPC

* Fixed test

* Nishant's comment

* Gaz

* Refresh ENR is for epoch

* Needs to be append
* Merge refs/heads/master into v0.11
* resolve confs
* Validator subscribe subnet to next epoch (#5312)

* Alert to subscribe to next epoch

* Fixed tests

* Comments

* Fixed tests

* Update validator/client/validator.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Revert "Revert "Remove deprecated parameters (#5249)" (#5276)" (#5277)

This reverts commit 47e5a2cf96.
* Aggregate on demand for v0.11 (#5302)

* Add client implementation

* Update workspace

* Update server

* Update service

* Gaz

* Mocks

* Fixed validator tests

* Add round tirp tests

* Fixed subnet test

* Wait 1/3 on validator side

* Lint

* Comment

* Update committee cache

* Comment

* Update RPC

* Fixed test

* Nishant's comment

* Gaz

* Refresh ENR is for epoch

* Needs to be append

* Fixed duplication

* Tests

* Skip e2e

* Update beacon-chain/rpc/validator/aggregator.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

* Apply suggestions from code review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: shayzluf <thezluf@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Refactor Dynamic Subscriptions (#5318)

* clean up

* comment

* metrics

* fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge refs/heads/master into v0.11
* Fix listindexed attestations and detect historic attestations (#5321)

* fix list indexed attestations

* fix tests

* goimports

* names
* Add check for slot == 0 (#5322)
* Change attester protection to return default if DB is empty (#5323)

* Change how default values are set

* Remove unused imports

* Remove wasteful db call

* Fix db tests

* Fix db test
* Merge refs/heads/master into v0.11
* fix it (#5326)
* V0.11 run time fixes to use interop config (#5324)

* Started testing
* Bunch of fixes
* use-interop
* Sync with v0.11
* Conflict
* Uncomment wait for activation
* Move pending block queue from subscriber to validator pipeline
* Merge branch 'v0.11' into use-interop-config
* passing tests
* Merge refs/heads/v0.11 into use-interop-config
* Merge refs/heads/v0.11 into use-interop-config
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Nil Checks in Process Attestation v0.11 (#5331)

* Started testing

* Bunch of fixes

* use-interop

* Sync with v0.11

* Uncomment wait for activation

* Move pending block queue from subscriber to validator pipeline

* passing tests

* nil checks to prevent panics

* lint

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Validator batch subscribe subnets (#5332)

* Update both beacon node and validator

* Comments

* Tests

* Lint

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Validator smarter subscribe (#5334)
* Fix incorrect proposer index calculation (#5336)

* Use correct parent state

* Fixed test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* enhance error
* enhance error
* Update P2P Service to Handle Local Metadata (#5319)

* add metadata to ENR

* add new methods

* glue everything

* fix all tests and refs

* add tests

* add more tests

* Apply suggestions from code review

* fix method

* raul's review

* gaz

* fix test setup

* fix all tests

* better naming

* fix broken test

* validate nil

Co-authored-by: rauljordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Revert "Revert "Revert "Remove deprecated parameters (#5249)" (#5276)" (#5277)" (#5343)

This reverts commit e5aef1686e.
* Wait for Genesis Event to Start P2P (#5303)

* use event feed for state initialized events

* add in handler for tests

* wait till genesis for p2p

* Apply suggestions from code review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge refs/heads/master into v0.11
* Avoid duplicated aggregation request (#5346)

* Avoid duplicated aggregation request

* Test and lock

* Gaz
* Fix Validate For Metadata (#5348)

* return true

* shay's review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Multiple Proposer Slots Allowed Per Epoch for Validators (#5344)

* allow multiple proposer slots

* multi propose

* proposer indices to slots map

* remove deprecated comm assign

* Apply suggestions from code review

* resolve broken tests, add logic in validator client

* fix val tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Networking Fixes (#5349)

* close stream later

* add ping method

* add method

* lint
* More efficient aggregation on demand (#5354)
* Return Nil Error if Pre-Genesis in P2P Service Healthz Check (#5355)

* pregenesis healthz check:

* optimal

* right order

* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* no comment

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* STarting
* Release DiscoveryV5 for Testnet Restart (#5357)

* release discv5

* fix build
* Fix Overflow in Status Check (#5361)

* fix overflow

* Apply suggestions from code review
* Update assignments
* Genesis short circuit
* Fixed tests
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* fix after merge
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Merge refs/heads/master into v0.11
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Make Mainnet Config Default, No More Demo Config  (#5367)

* bye bye demo config

* gaz

* fix usage

* fix dep

* gaz

* Update default balance for sendDeposits

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Use FastSSZ Marshal/Unmarshal for DB Encodings in v0.11.1 (#5351)

* try

* use marshaler structure for db instead of proto

* white list types

* attempt

* revert

* testutil.NewBeaconState()

* Fully populate fields for round trip ssz marshal

* fix //beacon-chain/db/kv:go_default_test

* more passing tests

* another test target passed

* fixed stategen

* blockchain tests green

* passing sync

* more targets fixed

* more test fixes in rpc/validator

* most rpc val

* validators test fixes

* skip round robin old

* aggregate test

* whitelist done

* Update beacon-chain/rpc/validator/attester_test.go

* edit baz

* Fixed tests

* Fixed getblock test

* Add back init

* reduce test size

* fix broken build

* tests pass

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Reconnect slasher streams on beacon node shutdown (#5376)

* restart streams on beacon node shutdown

* fix comment

* remove export

* ivan feedback

* ivan feedback

* case insensitive

* Update slasher/beaconclient/receivers.go

* raul feedback

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Merge branch 'master' into v0.11
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Merge refs/heads/master into v0.11
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Amend Faucet to Offer 32.5 ETH for v0.11 (#5378)

* deposit amount in faucet

* fix eth amount

* gas cost
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* unskip exec transition test
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Revert "Enable NOISE Handshake by Default v0.11 (#5272)" (#5381)

This reverts commit a8d32d504a.
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Merge refs/heads/master into v0.11
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* use string for deposit flag
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update Bootnode to v0.11 (#5387)

* fix bootnode

* add changes

* gaz

* fix docker
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Fixed conflict
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* build fix
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* fix flaky test
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Merge refs/heads/master into v0.11
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into fix-list-validator-assignments
* Fixed more conflicts
* Merge branch 'fix-list-validator-assignments' of github.com:prysmaticlabs/prysm into fix-list-validator-assignments
* Unskip E2E for V0.11 (#5386)

* Begin work on fixing e2e for v0.11

* Start bootnode work

* Begin implementing bootnode into e2e

* Fix E2E for v0.11

* Remove extra

* gaz

* Remove unused key gen code

* Remove trailing multiaddr code

* add skip for slashing

* Fix slashing e2e

* Fix docker image build
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.11
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Merge refs/heads/master into v0.11
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update beacon-chain/p2p/broadcaster_test.go
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into fix-list-validator-assignments
* Fixed test
* Merge branch 'fix-list-validator-assignments' of github.com:prysmaticlabs/prysm into fix-list-validator-assignments
* Merge refs/heads/master into v0.11
* Update GetValidatorActiveSetChanges
* Fixed tests
* Merge branch 'v0.11' into fix-list-validator-assignments
* Pass E2E Tests for v0.11 and Enable Attestation Subnets By Default (#5407)
* Merge refs/heads/v0.11 into fix-active-set-changes
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update README.md

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Apply suggestions from code review

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update beacon-chain/p2p/config.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update shared/keystore/deposit_input.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update tools/faucet/server.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update proto/beacon/p2p/v1/BUILD.bazel

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update shared/bls/spectest/aggregate_verify_test.go
* Merge refs/heads/v0.11 into fix-active-set-changes
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Addressed feedback. All test passing
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Merge refs/heads/v0.11 into fix-active-set-changes
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update beacon-chain/core/blocks/block_operations_fuzz_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update beacon-chain/core/blocks/block_operations_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update shared/testutil/helpers.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update beacon-chain/core/helpers/signing_root.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Resolve Misc v0.11 Items (Raul) (#5414)

* address all comments

* set faucet

* nishant feedback

* Update beacon-chain/p2p/service.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Revert keymanager changes (#5416)

* Revert "Updates for remote keymanager (#5260)"

This reverts commit bbcd895db5.

* Revert "Remove keystore keymanager from validator (#5236)"

This reverts commit 46008770c1.

* Revert "Update eth2 wallet keymanager (#4984)"

This reverts commit 7f7ef43f21.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update BLS and limit visibility (#5415)

* remove duplicated BLS, add golang.org/x/mod

* Update BLS and restrict visibility

* fix build
* Merge refs/heads/v0.11 into fix-active-set-changes
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Fix eth1data test and fix order of ops (#5413)
* Merge refs/heads/v0.11 into fix-active-set-changes
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* use multiaddr builder (#5419)
* Merge refs/heads/v0.11 into fix-active-set-changes
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Unskip benchutil and minor v0.11 fixes (#5417)

* Unskip benchutil tests

* Remove protos and gaz

* Fixes
* Merge refs/heads/v0.11 into fix-active-set-changes
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Networking Fixes (#5421)

* check

* fix test

* fix size

* fix test

* more fixes

* fix test again
* Merge refs/heads/v0.11 into fix-active-set-changes
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update ethereum APIs with latest master
* Merge refs/heads/v0.11 into fix-active-set-changes
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Error handling for v0.11 tests (#5428)

* Proper err handling for tests

* Lint

* Fixed rest of the tests

* Gaz

* Fixed old master tests
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Sync with master
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Rm old aggregate_test.go
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Merge branch 'master' into fix-list-validator-assignments
* Fixed conflicts
* Sync with v0.11
* Sync with master
* Merge refs/heads/master into fix-active-set-changes
* Merge refs/heads/master into fix-active-set-changes
* Merge refs/heads/master into fix-active-set-changes
* Merge refs/heads/master into fix-active-set-changes
* Merge refs/heads/master into fix-active-set-changes
* Merge branch 'master' into fix-active-set-changes
* Gaz
* Merge branch 'fix-list-validator-assignments' into deprecate-archival-service
* Rm archival service
* Rm flags
* Fixed flags
* Merge refs/heads/master into deprecate-archival-service
* Gaz
* Merge branch 'deprecate-archival-service' of github.com:prysmaticlabs/prysm into deprecate-archival-service
* Merge refs/heads/master into deprecate-archival-service
* Gaz
* Merge branch 'deprecate-archival-service' of github.com:prysmaticlabs/prysm into deprecate-archival-service
* Merge refs/heads/master into deprecate-archival-service
* Merge refs/heads/master into deprecate-archival-service
2020-04-15 19:41:59 +00:00
terence tsao
5a1a768135 Modify ListValidatorAssignments to use new state service (#5365)
* Fixed rest of non spec tests
* Fixed a bug proposer index wasn't included
* gaz
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Updated eth1 data voting period to epoch based
* Fixed failed tests
* fix bug
* fix error
* Fixed more misc tests
* Add new SignedAggregateAndProof to pass spec test
* Update minimalConfig.PersistentCommitteePeriod
* allow to rebuild trie
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Skip e2e tests
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Align aggregator action with v0.11 (#5146)
* Remove Head Root from Beacon Block by Range Request (#5165)

* make proto changes
* remove head root
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.11
* add back herumi's library
* Update ethapi in workspace, started fixing test. Hand off to Nishant
* fix build
* All tests passing
* Align finalized slot check with v0.11 (#5166)
* Merge branch 'master' into v0.11
* Add DoS resistance for v0.11 (#5158)
* Add Fork Digest Helper (#5173)
* Extend DoS prevention to rest of operation objects (#5174)

* Update mapping

* Add caches

* Update seen block in validation pipeline

* Update seen att in validation pipeline

* Update seen att in validation pipeline

* Fixed rest of tests

* Gazelle

* Better writes

* Lint

* Preston's feedback

* Switched to LRU cache and fixed tests

* Gazelle

* Fix test

* Update proposer slashing

* Update proposer slashing

* Fixed a block test

* Update exit

* Update atteser slashing

* Raul's feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Add remote keymanager (#5133)

* Add remote keymanager

* Add generic signRoot() helper

* Add tests for remote keymanager

* NewRemote -> NewRemoteWallet

* signRoot -> signOject, to increase reuse

* Fix end-to-end compile error

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Add Snappy Framing to the Encoder (#5172)

* change to framing

* more fixes

* fix everything

* add stricter limits

* preston feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: rauljordan <raul@prysmaticlabs.com>
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Move Subnet Functionality to its Own File (#5179)

* move subnets to their own file

* fix build fail

* build

* Update beacon-chain/p2p/discovery_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Sync with master
* Verify proposer signature in sync (#5206)
* Fix Signed Attestation In Sync (#5207)
* Add Eth2 Fork ENR Functionality (#5181)

* add fork entry enr

* add in fork

* add the required fork entry to node

* add and retrieve fork entry

* await state initialized

* utilize new structure

* more progress, utilizing a config map instead

* send the genesis validators root via the event feed

* struct method for discovery

* fix broken builds

* fixed up more tsts using state feed initializer

* fix up most tests

* only one more failing test

* almost done with tests

* p2p tests all pass

* config fix

* fix blockchain test

* gaz

* add in todo

* lint

* add compare func

* ensure fork ENR versions match between peers

* add in test for discovery

* test name

* tests complete

* tests done

* done

* comments

* fix all flakes

* addressed comments

* build using ssz gen

* marshal record

* use custom ssz

* deduplicate import

* fix build

* add enr proto

* p2p tests done

Co-authored-by: nisdas <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Verify aggregator signature in sync (#5208)
* Add Fork Digest For Gossip Topics (#5191)

* update for the day

* fix remaining failing test

* fix one more test

* change message

* Apply suggestions from code review

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* terence's review

* implement fork digest'

* align digest to interface'

* passed all tests

* spawn in goroutine

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Fix Incorrect Attester Slashing Method (#5229)
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Remove keystore keymanager from validator (#5236)

* Remove keystore keymanager from validator

* Update dependency

* Update validator/flags/flags.go

* Update validator/flags/flags.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
* fix broadcaster
* update metrics with fork digest for p2p (#5251)

* update metrics with fork digest for p2p

* update p2p metrics

* update metrics using att values

* wrapped up

* fix bug

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Fix incorrect domain type comments (#5250)

* Fix incorrect domain type comments
* resolve conflicts
* fix broken broadcast test
* fix tests
* include protocol suffix
* fix confs
* lint
* fix test
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* resolve broken slasher test'
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Merge branch 'master' into v0.11
* fix config override
* Remove deprecated parameters (#5249)
* Avoid div by zero in extreme balance case (#5273)

* Return effective balance increment instead of 1

* Update to new spec tests v0.11.1

* Revert "Regen historical states for `new-state-mgmt` compatibility (#5261)"

This reverts commit df9a534826.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Revert "Remove deprecated parameters (#5249)" (#5276)

This reverts commit 7d17c9ac34.
* Verify block proposer index before gossip  (#5274)

* Update pipeline

* Update tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Add in Proposer Index to Custom HTR (#5269)

* fix test

* Update beacon-chain/state/stateutil/blocks_test.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Resolve Flakey P2P Tests (#5285)

* double time for flakey test

* fix test flakeyness in p2p:

* flakey

* time tolerance

* greater tolerance
* Merge branch 'master' into v0.11
* release resources correctly (#5287)
* Merge refs/heads/master into v0.11
* Enable NOISE Handshake by Default v0.11 (#5272)

* noise handshakes by default

* fix build

* noisy noise everywhere

* deprecated noisy noise flag with more noise

* add secio as fallback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
* Merge refs/heads/master into v0.11
* new ports
* fix broken build
* Make `new-state-mgmt` canonical  (#5289)

* Invert the flags
* Update checking messages
* Fixed all db tests
* Fixed rest of the block chain tests
* Fix chain race tests
* Fixed rpc tests
* Disable soudns better...
* Merge branch 'v0.11' into invert-new-state-mgmt
* Merge refs/heads/v0.11 into invert-new-state-mgmt
* Fix export
* Merge branch 'invert-new-state-mgmt' of github.com:prysmaticlabs/prysm into invert-new-state-mgmt
* Fix conflict tests
* Gazelle
* Merge refs/heads/v0.11 into invert-new-state-mgmt
* Merge refs/heads/v0.11 into invert-new-state-mgmt
* Merge branch 'master' into v0.11
* resolve flakeyness
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Detect Proposer Slashing Implementation (#5139)

* detect blocks

* detect blocks

* use stub

* use stub

* use stub

* todo

* fix test

* add tests and utils

* fix imports

* fix imports

* fix comment

* todo

* proposerIndex

* fix broken test

* formatting and simplified if

* Update slasher/detection/service.go

* Update slasher/detection/testing/utils.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* fixed up final comments

* better naming

* Update slasher/detection/service.go

* Update slasher/detection/service.go

* Update slasher/detection/service.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

* no more named args

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.11
* Add Metadata And Ping RPC methods (#5271)

* add new proto files

* add flag and helper

* add initializer

* imports

* add ping method

* add receive/send ping request

* add ping test

* refactor rpc methods and add ping test

* finish adding all tests

* fix up tests

* Apply suggestions from code review

* lint

* imports

* lint

* Update beacon-chain/p2p/service.go

* Update shared/cmd/flags.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.11
* Updates for remote keymanager (#5260)
* Merge branch 'spec-v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Merge remote-tracking branch 'origin' into v0.11
* Update to slash by slot instead of epoch (#5297)

* change to slash by slot instead of epoch

* gaz

* fix test

* fix test

* fix infinite loop on error parse
* Sync with master
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Update proposer protection to v0.11 (#5292)

* Complete most of changes

* Fix other tests

* Test progress

* Tests

* Finish tests

* update pbs

* Fix mocked tests

* Gazelle

* pt 2

* Fix

* Fixes

* Fix tests wit hwrong copying
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Implement `SubscribeCommitteeSubnet` method (#5299)

* Add client implementation

* Update workspace

* Update server

* Update service

* Gaz

* Mocks

* Fixed validator tests

* Add round tirp tests

* Fixed subnet test

* Comment

* Update committee cache

* Comment

* Update RPC

* Fixed test

* Nishant's comment

* Gaz

* Refresh ENR is for epoch

* Needs to be append
* Merge refs/heads/master into v0.11
* resolve confs
* Validator subscribe subnet to next epoch (#5312)

* Alert to subscribe to next epoch

* Fixed tests

* Comments

* Fixed tests

* Update validator/client/validator.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Revert "Revert "Remove deprecated parameters (#5249)" (#5276)" (#5277)

This reverts commit 47e5a2cf96.
* Aggregate on demand for v0.11 (#5302)

* Add client implementation

* Update workspace

* Update server

* Update service

* Gaz

* Mocks

* Fixed validator tests

* Add round tirp tests

* Fixed subnet test

* Wait 1/3 on validator side

* Lint

* Comment

* Update committee cache

* Comment

* Update RPC

* Fixed test

* Nishant's comment

* Gaz

* Refresh ENR is for epoch

* Needs to be append

* Fixed duplication

* Tests

* Skip e2e

* Update beacon-chain/rpc/validator/aggregator.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

* Apply suggestions from code review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: shayzluf <thezluf@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Refactor Dynamic Subscriptions (#5318)

* clean up

* comment

* metrics

* fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge refs/heads/master into v0.11
* Fix listindexed attestations and detect historic attestations (#5321)

* fix list indexed attestations

* fix tests

* goimports

* names
* Add check for slot == 0 (#5322)
* Change attester protection to return default if DB is empty (#5323)

* Change how default values are set

* Remove unused imports

* Remove wasteful db call

* Fix db tests

* Fix db test
* Merge refs/heads/master into v0.11
* fix it (#5326)
* V0.11 run time fixes to use interop config (#5324)

* Started testing
* Bunch of fixes
* use-interop
* Sync with v0.11
* Conflict
* Uncomment wait for activation
* Move pending block queue from subscriber to validator pipeline
* Merge branch 'v0.11' into use-interop-config
* passing tests
* Merge refs/heads/v0.11 into use-interop-config
* Merge refs/heads/v0.11 into use-interop-config
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Nil Checks in Process Attestation v0.11 (#5331)

* Started testing

* Bunch of fixes

* use-interop

* Sync with v0.11

* Uncomment wait for activation

* Move pending block queue from subscriber to validator pipeline

* passing tests

* nil checks to prevent panics

* lint

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Validator batch subscribe subnets (#5332)

* Update both beacon node and validator

* Comments

* Tests

* Lint

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Validator smarter subscribe (#5334)
* Fix incorrect proposer index calculation (#5336)

* Use correct parent state

* Fixed test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* enhance error
* enhance error
* Update P2P Service to Handle Local Metadata (#5319)

* add metadata to ENR

* add new methods

* glue everything

* fix all tests and refs

* add tests

* add more tests

* Apply suggestions from code review

* fix method

* raul's review

* gaz

* fix test setup

* fix all tests

* better naming

* fix broken test

* validate nil

Co-authored-by: rauljordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Revert "Revert "Revert "Remove deprecated parameters (#5249)" (#5276)" (#5277)" (#5343)

This reverts commit e5aef1686e.
* Wait for Genesis Event to Start P2P (#5303)

* use event feed for state initialized events

* add in handler for tests

* wait till genesis for p2p

* Apply suggestions from code review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge refs/heads/master into v0.11
* Avoid duplicated aggregation request (#5346)

* Avoid duplicated aggregation request

* Test and lock

* Gaz
* Fix Validate For Metadata (#5348)

* return true

* shay's review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Multiple Proposer Slots Allowed Per Epoch for Validators (#5344)

* allow multiple proposer slots

* multi propose

* proposer indices to slots map

* remove deprecated comm assign

* Apply suggestions from code review

* resolve broken tests, add logic in validator client

* fix val tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Networking Fixes (#5349)

* close stream later

* add ping method

* add method

* lint
* More efficient aggregation on demand (#5354)
* Return Nil Error if Pre-Genesis in P2P Service Healthz Check (#5355)

* pregenesis healthz check:

* optimal

* right order

* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* no comment

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* STarting
* Release DiscoveryV5 for Testnet Restart (#5357)

* release discv5

* fix build
* Fix Overflow in Status Check (#5361)

* fix overflow

* Apply suggestions from code review
* Update assignments
* Genesis short circuit
* Fixed tests
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* fix after merge
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Merge refs/heads/master into v0.11
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Make Mainnet Config Default, No More Demo Config  (#5367)

* bye bye demo config

* gaz

* fix usage

* fix dep

* gaz

* Update default balance for sendDeposits

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Use FastSSZ Marshal/Unmarshal for DB Encodings in v0.11.1 (#5351)

* try

* use marshaler structure for db instead of proto

* white list types

* attempt

* revert

* testutil.NewBeaconState()

* Fully populate fields for round trip ssz marshal

* fix //beacon-chain/db/kv:go_default_test

* more passing tests

* another test target passed

* fixed stategen

* blockchain tests green

* passing sync

* more targets fixed

* more test fixes in rpc/validator

* most rpc val

* validators test fixes

* skip round robin old

* aggregate test

* whitelist done

* Update beacon-chain/rpc/validator/attester_test.go

* edit baz

* Fixed tests

* Fixed getblock test

* Add back init

* reduce test size

* fix broken build

* tests pass

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Reconnect slasher streams on beacon node shutdown (#5376)

* restart streams on beacon node shutdown

* fix comment

* remove export

* ivan feedback

* ivan feedback

* case insensitive

* Update slasher/beaconclient/receivers.go

* raul feedback

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Merge branch 'master' into v0.11
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Merge refs/heads/master into v0.11
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Amend Faucet to Offer 32.5 ETH for v0.11 (#5378)

* deposit amount in faucet

* fix eth amount

* gas cost
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* unskip exec transition test
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Revert "Enable NOISE Handshake by Default v0.11 (#5272)" (#5381)

This reverts commit a8d32d504a.
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Merge refs/heads/master into v0.11
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* use string for deposit flag
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update Bootnode to v0.11 (#5387)

* fix bootnode

* add changes

* gaz

* fix docker
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Fixed conflict
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* build fix
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* fix flaky test
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Merge refs/heads/master into v0.11
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into fix-list-validator-assignments
* Fixed more conflicts
* Merge branch 'fix-list-validator-assignments' of github.com:prysmaticlabs/prysm into fix-list-validator-assignments
* Unskip E2E for V0.11 (#5386)

* Begin work on fixing e2e for v0.11

* Start bootnode work

* Begin implementing bootnode into e2e

* Fix E2E for v0.11

* Remove extra

* gaz

* Remove unused key gen code

* Remove trailing multiaddr code

* add skip for slashing

* Fix slashing e2e

* Fix docker image build
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.11
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Merge refs/heads/master into v0.11
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update beacon-chain/p2p/broadcaster_test.go
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into fix-list-validator-assignments
* Fixed test
* Merge branch 'fix-list-validator-assignments' of github.com:prysmaticlabs/prysm into fix-list-validator-assignments
* Merge refs/heads/master into v0.11
* Merge branch 'v0.11' into fix-list-validator-assignments
* Pass E2E Tests for v0.11 and Enable Attestation Subnets By Default (#5407)
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update README.md

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Apply suggestions from code review

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update beacon-chain/p2p/config.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update shared/keystore/deposit_input.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update tools/faucet/server.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update proto/beacon/p2p/v1/BUILD.bazel

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update shared/bls/spectest/aggregate_verify_test.go
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Addressed feedback. All test passing
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update beacon-chain/core/blocks/block_operations_fuzz_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update beacon-chain/core/blocks/block_operations_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update shared/testutil/helpers.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update beacon-chain/core/helpers/signing_root.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Resolve Misc v0.11 Items (Raul) (#5414)

* address all comments

* set faucet

* nishant feedback

* Update beacon-chain/p2p/service.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Revert keymanager changes (#5416)

* Revert "Updates for remote keymanager (#5260)"

This reverts commit bbcd895db5.

* Revert "Remove keystore keymanager from validator (#5236)"

This reverts commit 46008770c1.

* Revert "Update eth2 wallet keymanager (#4984)"

This reverts commit 7f7ef43f21.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update BLS and limit visibility (#5415)

* remove duplicated BLS, add golang.org/x/mod

* Update BLS and restrict visibility

* fix build
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Fix eth1data test and fix order of ops (#5413)
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* use multiaddr builder (#5419)
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Unskip benchutil and minor v0.11 fixes (#5417)

* Unskip benchutil tests

* Remove protos and gaz

* Fixes
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Networking Fixes (#5421)

* check

* fix test

* fix size

* fix test

* more fixes

* fix test again
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Update ethereum APIs with latest master
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Error handling for v0.11 tests (#5428)

* Proper err handling for tests

* Lint

* Fixed rest of the tests

* Gaz

* Fixed old master tests
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Sync with master
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Rm old aggregate_test.go
* Merge refs/heads/v0.11 into fix-list-validator-assignments
* Merge branch 'master' into fix-list-validator-assignments
* Fixed conflicts
* Merge refs/heads/master into fix-list-validator-assignments
* Merge refs/heads/master into fix-list-validator-assignments
* Merge refs/heads/master into fix-list-validator-assignments
* Merge refs/heads/master into fix-list-validator-assignments
* Merge refs/heads/master into fix-list-validator-assignments
* Merge refs/heads/master into fix-list-validator-assignments
* Merge refs/heads/master into fix-list-validator-assignments
* Merge refs/heads/master into fix-list-validator-assignments
* Merge refs/heads/master into fix-list-validator-assignments
* Merge refs/heads/master into fix-list-validator-assignments
2020-04-15 19:18:25 +00:00
Raul Jordan
21f51993c5 rem old docs (#5446) 2020-04-15 13:53:02 -05:00
terence tsao
5eb6485e14 Modify GetValidatorActiveSetChanges to use new state service (#5408)
* Fix import and not use
* Update workspace for new spec test
* Fix workspace
* Merge refs/heads/master into v0.10.1
* Update workspace with new ethapi commit
* Unblock a few tests
* Merge refs/heads/master into v0.10.1
* fixed block op test
* gaz
* Merge refs/heads/master into v0.10.1
* Skip gen state test (test setup issue
* Updated hysteresis config
* Updated epoch processing for new hyteresis
* Updated tests
* regen proto beacon
* update state util for state root
* update state types
* update getter and setters
* update compute domain and get domain and tests
* update validators
* Add forkdata proto
* Updated compute domain api, moved it to helper pkg
* Merge refs/heads/master into v0.10.1
* Fixed all core tests
* Fixed all the sync tests
* Fixed all the rpc tests
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Fixed conflict
* Fixed conflict
* Conflict fix
* visibility
* Fixed validator tests
* Fixing test util
* Fixed rest of non spec tests
* Fixed a bug proposer index wasn't included
* gaz
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Updated eth1 data voting period to epoch based
* Fixed failed tests
* fix bug
* fix error
* Fixed more misc tests
* Add new SignedAggregateAndProof to pass spec test
* Update minimalConfig.PersistentCommitteePeriod
* allow to rebuild trie
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Skip e2e tests
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Align aggregator action with v0.11 (#5146)
* Remove Head Root from Beacon Block by Range Request (#5165)

* make proto changes
* remove head root
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.11
* add back herumi's library
* Update ethapi in workspace, started fixing test. Hand off to Nishant
* fix build
* All tests passing
* Align finalized slot check with v0.11 (#5166)
* Merge branch 'master' into v0.11
* Add DoS resistance for v0.11 (#5158)
* Add Fork Digest Helper (#5173)
* Extend DoS prevention to rest of operation objects (#5174)

* Update mapping

* Add caches

* Update seen block in validation pipeline

* Update seen att in validation pipeline

* Update seen att in validation pipeline

* Fixed rest of tests

* Gazelle

* Better writes

* Lint

* Preston's feedback

* Switched to LRU cache and fixed tests

* Gazelle

* Fix test

* Update proposer slashing

* Update proposer slashing

* Fixed a block test

* Update exit

* Update atteser slashing

* Raul's feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Add remote keymanager (#5133)

* Add remote keymanager

* Add generic signRoot() helper

* Add tests for remote keymanager

* NewRemote -> NewRemoteWallet

* signRoot -> signOject, to increase reuse

* Fix end-to-end compile error

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Add Snappy Framing to the Encoder (#5172)

* change to framing

* more fixes

* fix everything

* add stricter limits

* preston feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: rauljordan <raul@prysmaticlabs.com>
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Move Subnet Functionality to its Own File (#5179)

* move subnets to their own file

* fix build fail

* build

* Update beacon-chain/p2p/discovery_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Sync with master
* Verify proposer signature in sync (#5206)
* Fix Signed Attestation In Sync (#5207)
* Add Eth2 Fork ENR Functionality (#5181)

* add fork entry enr

* add in fork

* add the required fork entry to node

* add and retrieve fork entry

* await state initialized

* utilize new structure

* more progress, utilizing a config map instead

* send the genesis validators root via the event feed

* struct method for discovery

* fix broken builds

* fixed up more tsts using state feed initializer

* fix up most tests

* only one more failing test

* almost done with tests

* p2p tests all pass

* config fix

* fix blockchain test

* gaz

* add in todo

* lint

* add compare func

* ensure fork ENR versions match between peers

* add in test for discovery

* test name

* tests complete

* tests done

* done

* comments

* fix all flakes

* addressed comments

* build using ssz gen

* marshal record

* use custom ssz

* deduplicate import

* fix build

* add enr proto

* p2p tests done

Co-authored-by: nisdas <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Verify aggregator signature in sync (#5208)
* Add Fork Digest For Gossip Topics (#5191)

* update for the day

* fix remaining failing test

* fix one more test

* change message

* Apply suggestions from code review

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* terence's review

* implement fork digest'

* align digest to interface'

* passed all tests

* spawn in goroutine

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Fix Incorrect Attester Slashing Method (#5229)
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Remove keystore keymanager from validator (#5236)

* Remove keystore keymanager from validator

* Update dependency

* Update validator/flags/flags.go

* Update validator/flags/flags.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
* fix broadcaster
* update metrics with fork digest for p2p (#5251)

* update metrics with fork digest for p2p

* update p2p metrics

* update metrics using att values

* wrapped up

* fix bug

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Fix incorrect domain type comments (#5250)

* Fix incorrect domain type comments
* resolve conflicts
* fix broken broadcast test
* fix tests
* include protocol suffix
* fix confs
* lint
* fix test
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* resolve broken slasher test'
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Merge branch 'master' into v0.11
* fix config override
* Remove deprecated parameters (#5249)
* Avoid div by zero in extreme balance case (#5273)

* Return effective balance increment instead of 1

* Update to new spec tests v0.11.1

* Revert "Regen historical states for `new-state-mgmt` compatibility (#5261)"

This reverts commit df9a534826.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Revert "Remove deprecated parameters (#5249)" (#5276)

This reverts commit 7d17c9ac34.
* Verify block proposer index before gossip  (#5274)

* Update pipeline

* Update tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Add in Proposer Index to Custom HTR (#5269)

* fix test

* Update beacon-chain/state/stateutil/blocks_test.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Resolve Flakey P2P Tests (#5285)

* double time for flakey test

* fix test flakeyness in p2p:

* flakey

* time tolerance

* greater tolerance
* Merge branch 'master' into v0.11
* release resources correctly (#5287)
* Merge refs/heads/master into v0.11
* Enable NOISE Handshake by Default v0.11 (#5272)

* noise handshakes by default

* fix build

* noisy noise everywhere

* deprecated noisy noise flag with more noise

* add secio as fallback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
* Merge refs/heads/master into v0.11
* new ports
* fix broken build
* Make `new-state-mgmt` canonical  (#5289)

* Invert the flags
* Update checking messages
* Fixed all db tests
* Fixed rest of the block chain tests
* Fix chain race tests
* Fixed rpc tests
* Disable soudns better...
* Merge branch 'v0.11' into invert-new-state-mgmt
* Merge refs/heads/v0.11 into invert-new-state-mgmt
* Fix export
* Merge branch 'invert-new-state-mgmt' of github.com:prysmaticlabs/prysm into invert-new-state-mgmt
* Fix conflict tests
* Gazelle
* Merge refs/heads/v0.11 into invert-new-state-mgmt
* Merge refs/heads/v0.11 into invert-new-state-mgmt
* Merge branch 'master' into v0.11
* resolve flakeyness
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Detect Proposer Slashing Implementation (#5139)

* detect blocks

* detect blocks

* use stub

* use stub

* use stub

* todo

* fix test

* add tests and utils

* fix imports

* fix imports

* fix comment

* todo

* proposerIndex

* fix broken test

* formatting and simplified if

* Update slasher/detection/service.go

* Update slasher/detection/testing/utils.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* fixed up final comments

* better naming

* Update slasher/detection/service.go

* Update slasher/detection/service.go

* Update slasher/detection/service.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

* no more named args

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.11
* Add Metadata And Ping RPC methods (#5271)

* add new proto files

* add flag and helper

* add initializer

* imports

* add ping method

* add receive/send ping request

* add ping test

* refactor rpc methods and add ping test

* finish adding all tests

* fix up tests

* Apply suggestions from code review

* lint

* imports

* lint

* Update beacon-chain/p2p/service.go

* Update shared/cmd/flags.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.11
* Updates for remote keymanager (#5260)
* Merge branch 'spec-v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Merge remote-tracking branch 'origin' into v0.11
* Update to slash by slot instead of epoch (#5297)

* change to slash by slot instead of epoch

* gaz

* fix test

* fix test

* fix infinite loop on error parse
* Sync with master
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Update proposer protection to v0.11 (#5292)

* Complete most of changes

* Fix other tests

* Test progress

* Tests

* Finish tests

* update pbs

* Fix mocked tests

* Gazelle

* pt 2

* Fix

* Fixes

* Fix tests wit hwrong copying
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Implement `SubscribeCommitteeSubnet` method (#5299)

* Add client implementation

* Update workspace

* Update server

* Update service

* Gaz

* Mocks

* Fixed validator tests

* Add round tirp tests

* Fixed subnet test

* Comment

* Update committee cache

* Comment

* Update RPC

* Fixed test

* Nishant's comment

* Gaz

* Refresh ENR is for epoch

* Needs to be append
* Merge refs/heads/master into v0.11
* resolve confs
* Validator subscribe subnet to next epoch (#5312)

* Alert to subscribe to next epoch

* Fixed tests

* Comments

* Fixed tests

* Update validator/client/validator.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Revert "Revert "Remove deprecated parameters (#5249)" (#5276)" (#5277)

This reverts commit 47e5a2cf96.
* Aggregate on demand for v0.11 (#5302)

* Add client implementation

* Update workspace

* Update server

* Update service

* Gaz

* Mocks

* Fixed validator tests

* Add round tirp tests

* Fixed subnet test

* Wait 1/3 on validator side

* Lint

* Comment

* Update committee cache

* Comment

* Update RPC

* Fixed test

* Nishant's comment

* Gaz

* Refresh ENR is for epoch

* Needs to be append

* Fixed duplication

* Tests

* Skip e2e

* Update beacon-chain/rpc/validator/aggregator.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

* Apply suggestions from code review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: shayzluf <thezluf@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Refactor Dynamic Subscriptions (#5318)

* clean up

* comment

* metrics

* fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge refs/heads/master into v0.11
* Fix listindexed attestations and detect historic attestations (#5321)

* fix list indexed attestations

* fix tests

* goimports

* names
* Add check for slot == 0 (#5322)
* Change attester protection to return default if DB is empty (#5323)

* Change how default values are set

* Remove unused imports

* Remove wasteful db call

* Fix db tests

* Fix db test
* Merge refs/heads/master into v0.11
* fix it (#5326)
* V0.11 run time fixes to use interop config (#5324)

* Started testing
* Bunch of fixes
* use-interop
* Sync with v0.11
* Conflict
* Uncomment wait for activation
* Move pending block queue from subscriber to validator pipeline
* Merge branch 'v0.11' into use-interop-config
* passing tests
* Merge refs/heads/v0.11 into use-interop-config
* Merge refs/heads/v0.11 into use-interop-config
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Nil Checks in Process Attestation v0.11 (#5331)

* Started testing

* Bunch of fixes

* use-interop

* Sync with v0.11

* Uncomment wait for activation

* Move pending block queue from subscriber to validator pipeline

* passing tests

* nil checks to prevent panics

* lint

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Validator batch subscribe subnets (#5332)

* Update both beacon node and validator

* Comments

* Tests

* Lint

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Validator smarter subscribe (#5334)
* Fix incorrect proposer index calculation (#5336)

* Use correct parent state

* Fixed test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* enhance error
* enhance error
* Update P2P Service to Handle Local Metadata (#5319)

* add metadata to ENR

* add new methods

* glue everything

* fix all tests and refs

* add tests

* add more tests

* Apply suggestions from code review

* fix method

* raul's review

* gaz

* fix test setup

* fix all tests

* better naming

* fix broken test

* validate nil

Co-authored-by: rauljordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Revert "Revert "Revert "Remove deprecated parameters (#5249)" (#5276)" (#5277)" (#5343)

This reverts commit e5aef1686e.
* Wait for Genesis Event to Start P2P (#5303)

* use event feed for state initialized events

* add in handler for tests

* wait till genesis for p2p

* Apply suggestions from code review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge refs/heads/master into v0.11
* Avoid duplicated aggregation request (#5346)

* Avoid duplicated aggregation request

* Test and lock

* Gaz
* Fix Validate For Metadata (#5348)

* return true

* shay's review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Multiple Proposer Slots Allowed Per Epoch for Validators (#5344)

* allow multiple proposer slots

* multi propose

* proposer indices to slots map

* remove deprecated comm assign

* Apply suggestions from code review

* resolve broken tests, add logic in validator client

* fix val tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Networking Fixes (#5349)

* close stream later

* add ping method

* add method

* lint
* More efficient aggregation on demand (#5354)
* Return Nil Error if Pre-Genesis in P2P Service Healthz Check (#5355)

* pregenesis healthz check:

* optimal

* right order

* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* no comment

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Release DiscoveryV5 for Testnet Restart (#5357)

* release discv5

* fix build
* Fix Overflow in Status Check (#5361)

* fix overflow

* Apply suggestions from code review
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* fix after merge
* Merge refs/heads/master into v0.11
* Make Mainnet Config Default, No More Demo Config  (#5367)

* bye bye demo config

* gaz

* fix usage

* fix dep

* gaz

* Update default balance for sendDeposits

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Use FastSSZ Marshal/Unmarshal for DB Encodings in v0.11.1 (#5351)

* try

* use marshaler structure for db instead of proto

* white list types

* attempt

* revert

* testutil.NewBeaconState()

* Fully populate fields for round trip ssz marshal

* fix //beacon-chain/db/kv:go_default_test

* more passing tests

* another test target passed

* fixed stategen

* blockchain tests green

* passing sync

* more targets fixed

* more test fixes in rpc/validator

* most rpc val

* validators test fixes

* skip round robin old

* aggregate test

* whitelist done

* Update beacon-chain/rpc/validator/attester_test.go

* edit baz

* Fixed tests

* Fixed getblock test

* Add back init

* reduce test size

* fix broken build

* tests pass

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Reconnect slasher streams on beacon node shutdown (#5376)

* restart streams on beacon node shutdown

* fix comment

* remove export

* ivan feedback

* ivan feedback

* case insensitive

* Update slasher/beaconclient/receivers.go

* raul feedback

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' into v0.11
* Merge refs/heads/master into v0.11
* Amend Faucet to Offer 32.5 ETH for v0.11 (#5378)

* deposit amount in faucet

* fix eth amount

* gas cost
* unskip exec transition test
* Revert "Enable NOISE Handshake by Default v0.11 (#5272)" (#5381)

This reverts commit a8d32d504a.
* Merge refs/heads/master into v0.11
* use string for deposit flag
* Update Bootnode to v0.11 (#5387)

* fix bootnode

* add changes

* gaz

* fix docker
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* build fix
* fix flaky test
* Merge refs/heads/master into v0.11
* Unskip E2E for V0.11 (#5386)

* Begin work on fixing e2e for v0.11

* Start bootnode work

* Begin implementing bootnode into e2e

* Fix E2E for v0.11

* Remove extra

* gaz

* Remove unused key gen code

* Remove trailing multiaddr code

* add skip for slashing

* Fix slashing e2e

* Fix docker image build
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.11
* Merge refs/heads/master into v0.11
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* Update beacon-chain/p2p/broadcaster_test.go
* Merge refs/heads/master into v0.11
* Update GetValidatorActiveSetChanges
* Fixed tests
* Pass E2E Tests for v0.11 and Enable Attestation Subnets By Default (#5407)
* Merge refs/heads/v0.11 into fix-active-set-changes
* Update README.md

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Apply suggestions from code review

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Update beacon-chain/p2p/config.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Update shared/keystore/deposit_input.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Update tools/faucet/server.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Update proto/beacon/p2p/v1/BUILD.bazel

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Update shared/bls/spectest/aggregate_verify_test.go
* Merge refs/heads/v0.11 into fix-active-set-changes
* Addressed feedback. All test passing
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Merge refs/heads/v0.11 into fix-active-set-changes
* Update beacon-chain/core/blocks/block_operations_fuzz_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Update beacon-chain/core/blocks/block_operations_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Update shared/testutil/helpers.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Update beacon-chain/core/helpers/signing_root.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Resolve Misc v0.11 Items (Raul) (#5414)

* address all comments

* set faucet

* nishant feedback

* Update beacon-chain/p2p/service.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Revert keymanager changes (#5416)

* Revert "Updates for remote keymanager (#5260)"

This reverts commit bbcd895db5.

* Revert "Remove keystore keymanager from validator (#5236)"

This reverts commit 46008770c1.

* Revert "Update eth2 wallet keymanager (#4984)"

This reverts commit 7f7ef43f21.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge refs/heads/v0.11 into fix-active-set-changes
* Update BLS and limit visibility (#5415)

* remove duplicated BLS, add golang.org/x/mod

* Update BLS and restrict visibility

* fix build
* Merge refs/heads/v0.11 into fix-active-set-changes
* Fix eth1data test and fix order of ops (#5413)
* Merge refs/heads/v0.11 into fix-active-set-changes
* use multiaddr builder (#5419)
* Merge refs/heads/v0.11 into fix-active-set-changes
* Unskip benchutil and minor v0.11 fixes (#5417)

* Unskip benchutil tests

* Remove protos and gaz

* Fixes
* Merge refs/heads/v0.11 into fix-active-set-changes
* Networking Fixes (#5421)

* check

* fix test

* fix size

* fix test

* more fixes

* fix test again
* Merge refs/heads/v0.11 into fix-active-set-changes
* Update ethereum APIs with latest master
* Merge refs/heads/v0.11 into fix-active-set-changes
* Error handling for v0.11 tests (#5428)

* Proper err handling for tests

* Lint

* Fixed rest of the tests

* Gaz

* Fixed old master tests
* Sync with master
* Rm old aggregate_test.go
* Sync with v0.11
* Sync with master
* Merge refs/heads/master into fix-active-set-changes
* Merge refs/heads/master into fix-active-set-changes
* Merge refs/heads/master into fix-active-set-changes
* Merge refs/heads/master into fix-active-set-changes
* Merge refs/heads/master into fix-active-set-changes
* Merge branch 'master' into fix-active-set-changes
* Gaz
* Merge refs/heads/master into fix-active-set-changes
* Fixed test
* Merge branch 'fix-active-set-changes' of github.com:prysmaticlabs/prysm into fix-active-set-changes
* Merge refs/heads/master into fix-active-set-changes
2020-04-15 17:27:35 +00:00
terence tsao
ea84f44307 Use debug for Skipping deposit... log (#5444)
* Use debug
* Merge branch 'master' into debug-deposit-logs
* Merge refs/heads/master into debug-deposit-logs
2020-04-15 16:53:36 +00:00
Nishant Das
144f37e994 Fix Panics in P2P Service (#5442)
* fix panics
* Merge branch 'master' into smallFixes
* Merge refs/heads/master into smallFixes
* Merge refs/heads/master into smallFixes
2020-04-15 16:20:44 +00:00
terence tsao
046a00aa87 Modify GetValidatorParticipation to use new state service (#5409)
* Skip 2 more minimal tests

* Update readme

* gaz

* Fix import and not use

* Update workspace for new spec test

* Fix workspace

* Update workspace with new ethapi commit

* Unblock a few tests

* fixed block op test

* gaz

* Skip gen state test (test setup issue

* Updated hysteresis config

* Updated epoch processing for new hyteresis

* Updated tests

* regen proto beacon

* update state util for state root

* update state types

* update getter and setters

* update compute domain and get domain and tests

* update validators

* Add forkdata proto

* Updated compute domain api, moved it to helper pkg

* Fixed all core tests

* Fixed all the sync tests

* Fixed all the rpc tests

* Conflict fix

* visibility

* Fixed validator tests

* Fixing test util

* Fixed rest of non spec tests

* Fixed a bug proposer index wasn't included

* gaz

* Updated eth1 data voting period to epoch based

* Fixed failed tests

* fix bug

* fix error

* Fixed more misc tests

* Add new SignedAggregateAndProof to pass spec test

* Update minimalConfig.PersistentCommitteePeriod

* allow to rebuild trie

* Skip e2e tests

* Align aggregator action with v0.11 (#5146)

* Remove Head Root from Beacon Block by Range Request (#5165)

* make proto changes
* remove head root

* add back herumi's library

* Update ethapi in workspace, started fixing test. Hand off to Nishant

* fix build

* Align finalized slot check with v0.11 (#5166)

* Add DoS resistance for v0.11 (#5158)

* Add Fork Digest Helper (#5173)

* Extend DoS prevention to rest of operation objects (#5174)

* Update mapping

* Add caches

* Update seen block in validation pipeline

* Update seen att in validation pipeline

* Update seen att in validation pipeline

* Fixed rest of tests

* Gazelle

* Better writes

* Lint

* Preston's feedback

* Switched to LRU cache and fixed tests

* Gazelle

* Fix test

* Update proposer slashing

* Update proposer slashing

* Fixed a block test

* Update exit

* Update atteser slashing

* Raul's feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Add remote keymanager (#5133)

* Add remote keymanager

* Add generic signRoot() helper

* Add tests for remote keymanager

* NewRemote -> NewRemoteWallet

* signRoot -> signOject, to increase reuse

* Fix end-to-end compile error

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Add Snappy Framing to the Encoder (#5172)

* change to framing

* more fixes

* fix everything

* add stricter limits

* preston feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: rauljordan <raul@prysmaticlabs.com>

* Move Subnet Functionality to its Own File (#5179)

* move subnets to their own file

* fix build fail

* build

* Update beacon-chain/p2p/discovery_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Verify proposer signature in sync (#5206)

* Fix Signed Attestation In Sync (#5207)

* Add Eth2 Fork ENR Functionality (#5181)

* add fork entry enr

* add in fork

* add the required fork entry to node

* add and retrieve fork entry

* await state initialized

* utilize new structure

* more progress, utilizing a config map instead

* send the genesis validators root via the event feed

* struct method for discovery

* fix broken builds

* fixed up more tsts using state feed initializer

* fix up most tests

* only one more failing test

* almost done with tests

* p2p tests all pass

* config fix

* fix blockchain test

* gaz

* add in todo

* lint

* add compare func

* ensure fork ENR versions match between peers

* add in test for discovery

* test name

* tests complete

* tests done

* done

* comments

* fix all flakes

* addressed comments

* build using ssz gen

* marshal record

* use custom ssz

* deduplicate import

* fix build

* add enr proto

* p2p tests done

Co-authored-by: nisdas <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Verify aggregator signature in sync (#5208)

* Add Fork Digest For Gossip Topics (#5191)

* update for the day

* fix remaining failing test

* fix one more test

* change message

* Apply suggestions from code review

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* terence's review

* implement fork digest'

* align digest to interface'

* passed all tests

* spawn in goroutine

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Fix Incorrect Attester Slashing Method (#5229)

* Remove keystore keymanager from validator (#5236)

* Remove keystore keymanager from validator

* Update dependency

* Update validator/flags/flags.go

* Update validator/flags/flags.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>

* fix broadcaster

* update metrics with fork digest for p2p (#5251)

* update metrics with fork digest for p2p

* update p2p metrics

* update metrics using att values

* wrapped up

* fix bug

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Fix incorrect domain type comments (#5250)

* Fix incorrect domain type comments

* fix broken broadcast test

* fix tests

* include protocol suffix

* lint

* fix test

* resolve broken slasher test'

* fix config override

* Remove deprecated parameters (#5249)

* Avoid div by zero in extreme balance case (#5273)

* Return effective balance increment instead of 1

* Update to new spec tests v0.11.1

* Revert "Regen historical states for `new-state-mgmt` compatibility (#5261)"

This reverts commit df9a534826.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Revert "Remove deprecated parameters (#5249)" (#5276)

This reverts commit 7d17c9ac34.

* Verify block proposer index before gossip  (#5274)

* Update pipeline

* Update tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Add in Proposer Index to Custom HTR (#5269)

* fix test

* Update beacon-chain/state/stateutil/blocks_test.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Resolve Flakey P2P Tests (#5285)

* double time for flakey test

* fix test flakeyness in p2p:

* flakey

* time tolerance

* greater tolerance

* release resources correctly (#5287)

* Enable NOISE Handshake by Default v0.11 (#5272)

* noise handshakes by default

* fix build

* noisy noise everywhere

* deprecated noisy noise flag with more noise

* add secio as fallback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: nisdas <nishdas93@gmail.com>

* new ports

* fix broken build

* Make `new-state-mgmt` canonical  (#5289)

* Invert the flags
* Update checking messages
* Fixed all db tests
* Fixed rest of the block chain tests
* Fix chain race tests
* Fixed rpc tests
* Disable soudns better...
* Merge branch 'v0.11' into invert-new-state-mgmt
* Merge refs/heads/v0.11 into invert-new-state-mgmt
* Fix export
* Merge branch 'invert-new-state-mgmt' of github.com:prysmaticlabs/prysm into invert-new-state-mgmt
* Fix conflict tests
* Gazelle
* Merge refs/heads/v0.11 into invert-new-state-mgmt
* Merge refs/heads/v0.11 into invert-new-state-mgmt

* resolve flakeyness

* Detect Proposer Slashing Implementation (#5139)

* detect blocks

* detect blocks

* use stub

* use stub

* use stub

* todo

* fix test

* add tests and utils

* fix imports

* fix imports

* fix comment

* todo

* proposerIndex

* fix broken test

* formatting and simplified if

* Update slasher/detection/service.go

* Update slasher/detection/testing/utils.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* fixed up final comments

* better naming

* Update slasher/detection/service.go

* Update slasher/detection/service.go

* Update slasher/detection/service.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

* no more named args

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>

* Add Metadata And Ping RPC methods (#5271)

* add new proto files

* add flag and helper

* add initializer

* imports

* add ping method

* add receive/send ping request

* add ping test

* refactor rpc methods and add ping test

* finish adding all tests

* fix up tests

* Apply suggestions from code review

* lint

* imports

* lint

* Update beacon-chain/p2p/service.go

* Update shared/cmd/flags.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Updates for remote keymanager (#5260)

* Update to slash by slot instead of epoch (#5297)

* change to slash by slot instead of epoch

* gaz

* fix test

* fix test

* fix infinite loop on error parse

* Update proposer protection to v0.11 (#5292)

* Complete most of changes

* Fix other tests

* Test progress

* Tests

* Finish tests

* update pbs

* Fix mocked tests

* Gazelle

* pt 2

* Fix

* Fixes

* Fix tests wit hwrong copying

* Implement `SubscribeCommitteeSubnet` method (#5299)

* Add client implementation

* Update workspace

* Update server

* Update service

* Gaz

* Mocks

* Fixed validator tests

* Add round tirp tests

* Fixed subnet test

* Comment

* Update committee cache

* Comment

* Update RPC

* Fixed test

* Nishant's comment

* Gaz

* Refresh ENR is for epoch

* Needs to be append

* Validator subscribe subnet to next epoch (#5312)

* Alert to subscribe to next epoch

* Fixed tests

* Comments

* Fixed tests

* Update validator/client/validator.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Revert "Revert "Remove deprecated parameters (#5249)" (#5276)" (#5277)

This reverts commit 47e5a2cf96.

* Aggregate on demand for v0.11 (#5302)

* Add client implementation

* Update workspace

* Update server

* Update service

* Gaz

* Mocks

* Fixed validator tests

* Add round tirp tests

* Fixed subnet test

* Wait 1/3 on validator side

* Lint

* Comment

* Update committee cache

* Comment

* Update RPC

* Fixed test

* Nishant's comment

* Gaz

* Refresh ENR is for epoch

* Needs to be append

* Fixed duplication

* Tests

* Skip e2e

* Update beacon-chain/rpc/validator/aggregator.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

* Apply suggestions from code review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: shayzluf <thezluf@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Refactor Dynamic Subscriptions (#5318)

* clean up

* comment

* metrics

* fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Fix listindexed attestations and detect historic attestations (#5321)

* fix list indexed attestations

* fix tests

* goimports

* names

* Add check for slot == 0 (#5322)

* Change attester protection to return default if DB is empty (#5323)

* Change how default values are set

* Remove unused imports

* Remove wasteful db call

* Fix db tests

* Fix db test

* fix it (#5326)

* V0.11 run time fixes to use interop config (#5324)

* Started testing
* Bunch of fixes
* use-interop
* Sync with v0.11
* Conflict
* Uncomment wait for activation
* Move pending block queue from subscriber to validator pipeline
* Merge branch 'v0.11' into use-interop-config
* passing tests
* Merge refs/heads/v0.11 into use-interop-config
* Merge refs/heads/v0.11 into use-interop-config

* Nil Checks in Process Attestation v0.11 (#5331)

* Started testing

* Bunch of fixes

* use-interop

* Sync with v0.11

* Uncomment wait for activation

* Move pending block queue from subscriber to validator pipeline

* passing tests

* nil checks to prevent panics

* lint

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Validator batch subscribe subnets (#5332)

* Update both beacon node and validator

* Comments

* Tests

* Lint

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Validator smarter subscribe (#5334)

* Fix incorrect proposer index calculation (#5336)

* Use correct parent state

* Fixed test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* enhance error

* enhance error

* Update P2P Service to Handle Local Metadata (#5319)

* add metadata to ENR

* add new methods

* glue everything

* fix all tests and refs

* add tests

* add more tests

* Apply suggestions from code review

* fix method

* raul's review

* gaz

* fix test setup

* fix all tests

* better naming

* fix broken test

* validate nil

Co-authored-by: rauljordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Revert "Revert "Revert "Remove deprecated parameters (#5249)" (#5276)" (#5277)" (#5343)

This reverts commit e5aef1686e.

* Wait for Genesis Event to Start P2P (#5303)

* use event feed for state initialized events

* add in handler for tests

* wait till genesis for p2p

* Apply suggestions from code review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Avoid duplicated aggregation request (#5346)

* Avoid duplicated aggregation request

* Test and lock

* Gaz

* Fix Validate For Metadata (#5348)

* return true

* shay's review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Multiple Proposer Slots Allowed Per Epoch for Validators (#5344)

* allow multiple proposer slots

* multi propose

* proposer indices to slots map

* remove deprecated comm assign

* Apply suggestions from code review

* resolve broken tests, add logic in validator client

* fix val tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Networking Fixes (#5349)

* close stream later

* add ping method

* add method

* lint

* More efficient aggregation on demand (#5354)

* Return Nil Error if Pre-Genesis in P2P Service Healthz Check (#5355)

* pregenesis healthz check:

* optimal

* right order

* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* no comment

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Release DiscoveryV5 for Testnet Restart (#5357)

* release discv5

* fix build

* Fix Overflow in Status Check (#5361)

* fix overflow

* Apply suggestions from code review

* fix after merge

* Make Mainnet Config Default, No More Demo Config  (#5367)

* bye bye demo config

* gaz

* fix usage

* fix dep

* gaz

* Update default balance for sendDeposits

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Use FastSSZ Marshal/Unmarshal for DB Encodings in v0.11.1 (#5351)

* try

* use marshaler structure for db instead of proto

* white list types

* attempt

* revert

* testutil.NewBeaconState()

* Fully populate fields for round trip ssz marshal

* fix //beacon-chain/db/kv:go_default_test

* more passing tests

* another test target passed

* fixed stategen

* blockchain tests green

* passing sync

* more targets fixed

* more test fixes in rpc/validator

* most rpc val

* validators test fixes

* skip round robin old

* aggregate test

* whitelist done

* Update beacon-chain/rpc/validator/attester_test.go

* edit baz

* Fixed tests

* Fixed getblock test

* Add back init

* reduce test size

* fix broken build

* tests pass

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Reconnect slasher streams on beacon node shutdown (#5376)

* restart streams on beacon node shutdown

* fix comment

* remove export

* ivan feedback

* ivan feedback

* case insensitive

* Update slasher/beaconclient/receivers.go

* raul feedback

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Amend Faucet to Offer 32.5 ETH for v0.11 (#5378)

* deposit amount in faucet

* fix eth amount

* gas cost

* unskip exec transition test

* Revert "Enable NOISE Handshake by Default v0.11 (#5272)" (#5381)

This reverts commit a8d32d504a.

* use string for deposit flag

* Update Bootnode to v0.11 (#5387)

* fix bootnode

* add changes

* gaz

* fix docker

* build fix

* fix flaky test

* Unskip E2E for V0.11 (#5386)

* Begin work on fixing e2e for v0.11

* Start bootnode work

* Begin implementing bootnode into e2e

* Fix E2E for v0.11

* Remove extra

* gaz

* Remove unused key gen code

* Remove trailing multiaddr code

* add skip for slashing

* Fix slashing e2e

* Fix docker image build

* Update beacon-chain/p2p/broadcaster_test.go

* Update GetValidatorParticipation

* Update tests

* Gaz

* Pass E2E Tests for v0.11 and Enable Attestation Subnets By Default (#5407)

* Update README.md

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Apply suggestions from code review

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/p2p/config.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update shared/keystore/deposit_input.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update tools/faucet/server.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update proto/beacon/p2p/v1/BUILD.bazel

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update shared/bls/spectest/aggregate_verify_test.go

* Addressed feedback. All test passing

* Update beacon-chain/core/blocks/block_operations_fuzz_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

* Update beacon-chain/core/blocks/block_operations_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

* Update shared/testutil/helpers.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

* Update beacon-chain/core/helpers/signing_root.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

* Resolve Misc v0.11 Items (Raul) (#5414)

* address all comments

* set faucet

* nishant feedback

* Update beacon-chain/p2p/service.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Revert keymanager changes (#5416)

* Revert "Updates for remote keymanager (#5260)"

This reverts commit bbcd895db5.

* Revert "Remove keystore keymanager from validator (#5236)"

This reverts commit 46008770c1.

* Revert "Update eth2 wallet keymanager (#4984)"

This reverts commit 7f7ef43f21.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Update BLS and limit visibility (#5415)

* remove duplicated BLS, add golang.org/x/mod

* Update BLS and restrict visibility

* fix build

* Fix eth1data test and fix order of ops (#5413)

* use multiaddr builder (#5419)

* Unskip benchutil and minor v0.11 fixes (#5417)

* Unskip benchutil tests

* Remove protos and gaz

* Fixes

* Networking Fixes (#5421)

* check

* fix test

* fix size

* fix test

* more fixes

* fix test again

* Update ethereum APIs with latest master

* Error handling for v0.11 tests (#5428)

* Proper err handling for tests

* Lint

* Fixed rest of the tests

* Gaz

* Fixed old master tests

* Rm old aggregate_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
Co-authored-by: Jim McDonald <Jim@mcdee.net>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: shayzluf <thezluf@gmail.com>
2020-04-15 11:00:39 -05:00
terence tsao
497fa6ed50 Modify ListBeaconCommittees to use new state service (#5411) 2020-04-15 08:15:23 -07:00
Jim McDonald
899670e1e9 Fix bad log messages for duplicate deposits (#5440)
* Fix bad log messages
2020-04-15 14:08:45 +00:00
terence tsao
ef21456c8a nil check for SignedAggregateAndProof (#5420) 2020-04-14 20:38:24 -07:00
Nishant Das
92131695b1 Fix Network Encoder (#5435)
* add new block encoding for snappy
* Merge branch 'master' into fixEncoder
* flip flag
* Merge branch 'fixEncoder' of https://github.com/prysmaticlabs/geth-sharding into fixEncoder
* not an underscore
2020-04-15 03:03:44 +00:00
terence tsao
24fc67a6dd Verify historical states exist in node pkg (#5432) 2020-04-14 17:38:52 -07:00
Preston Van Loon
dc5ed7a80e Update deposit contract default value for topaz (#5431)
* Update for topaz
* Update block number for deposit contract
2020-04-14 23:00:36 +00:00
terence tsao
cb045dd0e3 Align code base to v0.11 (#5127)
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* add in new patch and workspace
* update cloners
* Handle rewards overflow (#5122)

* Refactoring of initial sync (#5096)

* implements blocks queue

* refactors updateCounter method

* fixes deadlock on stop w/o start

* refactors updateSchedulerState

* more tests on schduler

* parseFetchResponse tests

* wraps up tests for blocks queue

* eod commit

* fixes data race in round robin

* revamps fetcher

* fixes race conditions + livelocks + deadlocks

* less verbose output

* fixes data race, by isolating critical sections

* minor refactoring: resolves blocking calls

* implements init-sync queue

* udpate fetch/send buffers in blocks fetcher

* blockState enum-like type alias

* refactors common code into releaseTicket()

* better gc

* linter

* minor fix to round robin

* moves original round robin into its own package

* adds enableInitSyncQueue flag

* fixes issue with init-sync service selection

* Update beacon-chain/sync/initial-sync/round_robin.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* initsyncv1 -> initsyncold

* adds span

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Handle rewards overflow

* Revert "Refactoring of initial sync (#5096)"

This reverts commit 3ec2a0f9e0.

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* updated block operations
* updated validator client
* Merge refs/heads/master into v0.10.1
* updated block operations test
* skip benchmark test
* updated transition test
* updated db kv tests
* updated ops tests
* updated ops tests
* updated slashing tests
* updated rpc tests
* updated state utils
* updated test utils and miscs
* Temp skips minimal spec tests
* Fixed proposer slashing test
* Gaz
* Skip 2 more minimal tests
* Skip 2 more minimal tests
* Update readme
* gaz
* Conflict
* Fix import and not use
* Update workspace for new spec test
* Fix workspace
* Merge refs/heads/master into v0.10.1
* Update workspace with new ethapi commit
* Unblock a few tests
* Merge refs/heads/master into v0.10.1
* fixed block op test
* gaz
* Merge refs/heads/master into v0.10.1
* Skip gen state test (test setup issue
* Updated hysteresis config
* Updated epoch processing for new hyteresis
* Updated tests
* regen proto beacon
* update state util for state root
* update state types
* update getter and setters
* update compute domain and get domain and tests
* update validators
* Add forkdata proto
* Updated compute domain api, moved it to helper pkg
* Merge refs/heads/master into v0.10.1
* Fixed all core tests
* Fixed all the sync tests
* Fixed all the rpc tests
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Fixed conflict
* Fixed conflict
* Conflict fix
* visibility
* Fixed validator tests
* Fixing test util
* Fixed rest of non spec tests
* Fixed a bug proposer index wasn't included
* gaz
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Updated eth1 data voting period to epoch based
* Fixed failed tests
* fix bug
* fix error
* Fixed more misc tests
* Add new SignedAggregateAndProof to pass spec test
* Update minimalConfig.PersistentCommitteePeriod
* allow to rebuild trie
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Skip e2e tests
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Align aggregator action with v0.11 (#5146)
* Remove Head Root from Beacon Block by Range Request (#5165)

* make proto changes
* remove head root
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.11
* add back herumi's library
* Update ethapi in workspace, started fixing test. Hand off to Nishant
* fix build
* All tests passing
* Align finalized slot check with v0.11 (#5166)
* Merge branch 'master' into v0.11
* Add DoS resistance for v0.11 (#5158)
* Add Fork Digest Helper (#5173)
* Extend DoS prevention to rest of operation objects (#5174)

* Update mapping

* Add caches

* Update seen block in validation pipeline

* Update seen att in validation pipeline

* Update seen att in validation pipeline

* Fixed rest of tests

* Gazelle

* Better writes

* Lint

* Preston's feedback

* Switched to LRU cache and fixed tests

* Gazelle

* Fix test

* Update proposer slashing

* Update proposer slashing

* Fixed a block test

* Update exit

* Update atteser slashing

* Raul's feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Add remote keymanager (#5133)

* Add remote keymanager

* Add generic signRoot() helper

* Add tests for remote keymanager

* NewRemote -> NewRemoteWallet

* signRoot -> signOject, to increase reuse

* Fix end-to-end compile error

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Add Snappy Framing to the Encoder (#5172)

* change to framing

* more fixes

* fix everything

* add stricter limits

* preston feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: rauljordan <raul@prysmaticlabs.com>
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Move Subnet Functionality to its Own File (#5179)

* move subnets to their own file

* fix build fail

* build

* Update beacon-chain/p2p/discovery_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Sync with master
* Verify proposer signature in sync (#5206)
* Fix Signed Attestation In Sync (#5207)
* Add Eth2 Fork ENR Functionality (#5181)

* add fork entry enr

* add in fork

* add the required fork entry to node

* add and retrieve fork entry

* await state initialized

* utilize new structure

* more progress, utilizing a config map instead

* send the genesis validators root via the event feed

* struct method for discovery

* fix broken builds

* fixed up more tsts using state feed initializer

* fix up most tests

* only one more failing test

* almost done with tests

* p2p tests all pass

* config fix

* fix blockchain test

* gaz

* add in todo

* lint

* add compare func

* ensure fork ENR versions match between peers

* add in test for discovery

* test name

* tests complete

* tests done

* done

* comments

* fix all flakes

* addressed comments

* build using ssz gen

* marshal record

* use custom ssz

* deduplicate import

* fix build

* add enr proto

* p2p tests done

Co-authored-by: nisdas <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Verify aggregator signature in sync (#5208)
* Add Fork Digest For Gossip Topics (#5191)

* update for the day

* fix remaining failing test

* fix one more test

* change message

* Apply suggestions from code review

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* terence's review

* implement fork digest'

* align digest to interface'

* passed all tests

* spawn in goroutine

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Fix Incorrect Attester Slashing Method (#5229)
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Remove keystore keymanager from validator (#5236)

* Remove keystore keymanager from validator

* Update dependency

* Update validator/flags/flags.go

* Update validator/flags/flags.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
* fix broadcaster
* update metrics with fork digest for p2p (#5251)

* update metrics with fork digest for p2p

* update p2p metrics

* update metrics using att values

* wrapped up

* fix bug

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Fix incorrect domain type comments (#5250)

* Fix incorrect domain type comments
* resolve conflicts
* fix broken broadcast test
* fix tests
* include protocol suffix
* fix confs
* lint
* fix test
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* resolve broken slasher test'
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Merge branch 'master' into v0.11
* fix config override
* Remove deprecated parameters (#5249)
* Avoid div by zero in extreme balance case (#5273)

* Return effective balance increment instead of 1

* Update to new spec tests v0.11.1

* Revert "Regen historical states for `new-state-mgmt` compatibility (#5261)"

This reverts commit df9a534826.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Revert "Remove deprecated parameters (#5249)" (#5276)

This reverts commit 7d17c9ac34.
* Verify block proposer index before gossip  (#5274)

* Update pipeline

* Update tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Add in Proposer Index to Custom HTR (#5269)

* fix test

* Update beacon-chain/state/stateutil/blocks_test.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Resolve Flakey P2P Tests (#5285)

* double time for flakey test

* fix test flakeyness in p2p:

* flakey

* time tolerance

* greater tolerance
* Merge branch 'master' into v0.11
* release resources correctly (#5287)
* Merge refs/heads/master into v0.11
* Enable NOISE Handshake by Default v0.11 (#5272)

* noise handshakes by default

* fix build

* noisy noise everywhere

* deprecated noisy noise flag with more noise

* add secio as fallback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
* Merge refs/heads/master into v0.11
* new ports
* fix broken build
* Make `new-state-mgmt` canonical  (#5289)

* Invert the flags
* Update checking messages
* Fixed all db tests
* Fixed rest of the block chain tests
* Fix chain race tests
* Fixed rpc tests
* Disable soudns better...
* Merge branch 'v0.11' into invert-new-state-mgmt
* Merge refs/heads/v0.11 into invert-new-state-mgmt
* Fix export
* Merge branch 'invert-new-state-mgmt' of github.com:prysmaticlabs/prysm into invert-new-state-mgmt
* Fix conflict tests
* Gazelle
* Merge refs/heads/v0.11 into invert-new-state-mgmt
* Merge refs/heads/v0.11 into invert-new-state-mgmt
* Merge branch 'master' into v0.11
* resolve flakeyness
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Detect Proposer Slashing Implementation (#5139)

* detect blocks

* detect blocks

* use stub

* use stub

* use stub

* todo

* fix test

* add tests and utils

* fix imports

* fix imports

* fix comment

* todo

* proposerIndex

* fix broken test

* formatting and simplified if

* Update slasher/detection/service.go

* Update slasher/detection/testing/utils.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* fixed up final comments

* better naming

* Update slasher/detection/service.go

* Update slasher/detection/service.go

* Update slasher/detection/service.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

* no more named args

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.11
* Add Metadata And Ping RPC methods (#5271)

* add new proto files

* add flag and helper

* add initializer

* imports

* add ping method

* add receive/send ping request

* add ping test

* refactor rpc methods and add ping test

* finish adding all tests

* fix up tests

* Apply suggestions from code review

* lint

* imports

* lint

* Update beacon-chain/p2p/service.go

* Update shared/cmd/flags.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.11
* Updates for remote keymanager (#5260)
* Merge branch 'spec-v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Merge remote-tracking branch 'origin' into v0.11
* Update to slash by slot instead of epoch (#5297)

* change to slash by slot instead of epoch

* gaz

* fix test

* fix test

* fix infinite loop on error parse
* Sync with master
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Update proposer protection to v0.11 (#5292)

* Complete most of changes

* Fix other tests

* Test progress

* Tests

* Finish tests

* update pbs

* Fix mocked tests

* Gazelle

* pt 2

* Fix

* Fixes

* Fix tests wit hwrong copying
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Implement `SubscribeCommitteeSubnet` method (#5299)

* Add client implementation

* Update workspace

* Update server

* Update service

* Gaz

* Mocks

* Fixed validator tests

* Add round tirp tests

* Fixed subnet test

* Comment

* Update committee cache

* Comment

* Update RPC

* Fixed test

* Nishant's comment

* Gaz

* Refresh ENR is for epoch

* Needs to be append
* Merge refs/heads/master into v0.11
* resolve confs
* Validator subscribe subnet to next epoch (#5312)

* Alert to subscribe to next epoch

* Fixed tests

* Comments

* Fixed tests

* Update validator/client/validator.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Revert "Revert "Remove deprecated parameters (#5249)" (#5276)" (#5277)

This reverts commit 47e5a2cf96.
* Aggregate on demand for v0.11 (#5302)

* Add client implementation

* Update workspace

* Update server

* Update service

* Gaz

* Mocks

* Fixed validator tests

* Add round tirp tests

* Fixed subnet test

* Wait 1/3 on validator side

* Lint

* Comment

* Update committee cache

* Comment

* Update RPC

* Fixed test

* Nishant's comment

* Gaz

* Refresh ENR is for epoch

* Needs to be append

* Fixed duplication

* Tests

* Skip e2e

* Update beacon-chain/rpc/validator/aggregator.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

* Apply suggestions from code review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: shayzluf <thezluf@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Refactor Dynamic Subscriptions (#5318)

* clean up

* comment

* metrics

* fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge refs/heads/master into v0.11
* Fix listindexed attestations and detect historic attestations (#5321)

* fix list indexed attestations

* fix tests

* goimports

* names
* Add check for slot == 0 (#5322)
* Change attester protection to return default if DB is empty (#5323)

* Change how default values are set

* Remove unused imports

* Remove wasteful db call

* Fix db tests

* Fix db test
* Merge refs/heads/master into v0.11
* fix it (#5326)
* V0.11 run time fixes to use interop config (#5324)

* Started testing
* Bunch of fixes
* use-interop
* Sync with v0.11
* Conflict
* Uncomment wait for activation
* Move pending block queue from subscriber to validator pipeline
* Merge branch 'v0.11' into use-interop-config
* passing tests
* Merge refs/heads/v0.11 into use-interop-config
* Merge refs/heads/v0.11 into use-interop-config
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Nil Checks in Process Attestation v0.11 (#5331)

* Started testing

* Bunch of fixes

* use-interop

* Sync with v0.11

* Uncomment wait for activation

* Move pending block queue from subscriber to validator pipeline

* passing tests

* nil checks to prevent panics

* lint

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Validator batch subscribe subnets (#5332)

* Update both beacon node and validator

* Comments

* Tests

* Lint

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Validator smarter subscribe (#5334)
* Fix incorrect proposer index calculation (#5336)

* Use correct parent state

* Fixed test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* enhance error
* enhance error
* Update P2P Service to Handle Local Metadata (#5319)

* add metadata to ENR

* add new methods

* glue everything

* fix all tests and refs

* add tests

* add more tests

* Apply suggestions from code review

* fix method

* raul's review

* gaz

* fix test setup

* fix all tests

* better naming

* fix broken test

* validate nil

Co-authored-by: rauljordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Revert "Revert "Revert "Remove deprecated parameters (#5249)" (#5276)" (#5277)" (#5343)

This reverts commit e5aef1686e.
* Wait for Genesis Event to Start P2P (#5303)

* use event feed for state initialized events

* add in handler for tests

* wait till genesis for p2p

* Apply suggestions from code review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge refs/heads/master into v0.11
* Avoid duplicated aggregation request (#5346)

* Avoid duplicated aggregation request

* Test and lock

* Gaz
* Fix Validate For Metadata (#5348)

* return true

* shay's review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Multiple Proposer Slots Allowed Per Epoch for Validators (#5344)

* allow multiple proposer slots

* multi propose

* proposer indices to slots map

* remove deprecated comm assign

* Apply suggestions from code review

* resolve broken tests, add logic in validator client

* fix val tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Networking Fixes (#5349)

* close stream later

* add ping method

* add method

* lint
* More efficient aggregation on demand (#5354)
* Return Nil Error if Pre-Genesis in P2P Service Healthz Check (#5355)

* pregenesis healthz check:

* optimal

* right order

* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* no comment

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Release DiscoveryV5 for Testnet Restart (#5357)

* release discv5

* fix build
* Fix Overflow in Status Check (#5361)

* fix overflow

* Apply suggestions from code review
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* fix after merge
* Merge refs/heads/master into v0.11
* Make Mainnet Config Default, No More Demo Config  (#5367)

* bye bye demo config

* gaz

* fix usage

* fix dep

* gaz

* Update default balance for sendDeposits

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Use FastSSZ Marshal/Unmarshal for DB Encodings in v0.11.1 (#5351)

* try

* use marshaler structure for db instead of proto

* white list types

* attempt

* revert

* testutil.NewBeaconState()

* Fully populate fields for round trip ssz marshal

* fix //beacon-chain/db/kv:go_default_test

* more passing tests

* another test target passed

* fixed stategen

* blockchain tests green

* passing sync

* more targets fixed

* more test fixes in rpc/validator

* most rpc val

* validators test fixes

* skip round robin old

* aggregate test

* whitelist done

* Update beacon-chain/rpc/validator/attester_test.go

* edit baz

* Fixed tests

* Fixed getblock test

* Add back init

* reduce test size

* fix broken build

* tests pass

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Reconnect slasher streams on beacon node shutdown (#5376)

* restart streams on beacon node shutdown

* fix comment

* remove export

* ivan feedback

* ivan feedback

* case insensitive

* Update slasher/beaconclient/receivers.go

* raul feedback

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' into v0.11
* Merge refs/heads/master into v0.11
* Amend Faucet to Offer 32.5 ETH for v0.11 (#5378)

* deposit amount in faucet

* fix eth amount

* gas cost
* unskip exec transition test
* Revert "Enable NOISE Handshake by Default v0.11 (#5272)" (#5381)

This reverts commit a8d32d504a.
* Merge refs/heads/master into v0.11
* use string for deposit flag
* Update Bootnode to v0.11 (#5387)

* fix bootnode

* add changes

* gaz

* fix docker
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* build fix
* fix flaky test
* Merge refs/heads/master into v0.11
* Unskip E2E for V0.11 (#5386)

* Begin work on fixing e2e for v0.11

* Start bootnode work

* Begin implementing bootnode into e2e

* Fix E2E for v0.11

* Remove extra

* gaz

* Remove unused key gen code

* Remove trailing multiaddr code

* add skip for slashing

* Fix slashing e2e

* Fix docker image build
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.11
* Merge refs/heads/master into v0.11
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* Update beacon-chain/p2p/broadcaster_test.go
* Merge refs/heads/master into v0.11
* Pass E2E Tests for v0.11 and Enable Attestation Subnets By Default (#5407)
* Update README.md

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Apply suggestions from code review

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update beacon-chain/p2p/config.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update shared/keystore/deposit_input.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update tools/faucet/server.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update proto/beacon/p2p/v1/BUILD.bazel

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update shared/bls/spectest/aggregate_verify_test.go
* Addressed feedback. All test passing
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Update beacon-chain/core/blocks/block_operations_fuzz_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/core/blocks/block_operations_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update shared/testutil/helpers.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/core/helpers/signing_root.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Resolve Misc v0.11 Items (Raul) (#5414)

* address all comments

* set faucet

* nishant feedback

* Update beacon-chain/p2p/service.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Revert keymanager changes (#5416)

* Revert "Updates for remote keymanager (#5260)"

This reverts commit bbcd895db5.

* Revert "Remove keystore keymanager from validator (#5236)"

This reverts commit 46008770c1.

* Revert "Update eth2 wallet keymanager (#4984)"

This reverts commit 7f7ef43f21.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Update BLS and limit visibility (#5415)

* remove duplicated BLS, add golang.org/x/mod

* Update BLS and restrict visibility

* fix build
* Fix eth1data test and fix order of ops (#5413)
* use multiaddr builder (#5419)
* Unskip benchutil and minor v0.11 fixes (#5417)

* Unskip benchutil tests

* Remove protos and gaz

* Fixes
* Networking Fixes (#5421)

* check

* fix test

* fix size

* fix test

* more fixes

* fix test again
* Update ethereum APIs with latest master
* Error handling for v0.11 tests (#5428)

* Proper err handling for tests

* Lint

* Fixed rest of the tests

* Gaz

* Fixed old master tests
* Sync with master
* Rm old aggregate_test.go
2020-04-14 20:27:03 +00:00
Victor Farazdagi
748d513c62 proper error checking and type assertions (#5424)
* proper error checking and type assertions
2020-04-14 16:41:09 +00:00
terence tsao
3b9a03d517 State service clean up - better cold state getter (#5393)
* Update load cold states by root and slot

* Update cold tests

* Add a way to recover state summary

* Short circuit slot 0 to return genesis state

* Short circuit if there's no block in DB

* Fixed test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-04-13 11:32:02 -05:00
Preston Van Loon
d5ddd012bc Enforce error handling and checking type assertions (#5403)
* Enforce error handling and checking type assertions
* Reference issue #5404 in the TODO message
* doc description
* Merge branch 'master' into errcheck
* fix tests and address @nisdas feedbacK
* gaz
* fix docker image
2020-04-13 04:11:09 +00:00
terence tsao
a85bf9305d Fixed a bug in ActivatedValidatorIndices (#5400) 2020-04-12 16:20:50 -07:00
Preston Van Loon
c76034147e Set ssz cache enabled by default (#5398)
* Set ssz cache enabled by default
* gofmt
2020-04-12 21:33:26 +00:00
Ivan Martinez
d8c57541e4 Validator client cleanup (#5389)
* Validator client status optimizations
* Reorder WaitForSync
* Change log to switch case
* Show activation epoch
* Change sync timing to half slot, inaccurate previously
* Merge branch 'master' into validator-imporve
2020-04-12 02:05:23 +00:00
Jim McDonald
28a9adc49b Add missing block fields to log (#5394) 2020-04-11 16:04:05 -07:00
terence tsao
e59721f264 State service clean up - better migration (#5391) 2020-04-11 13:54:19 -07:00
Ivan Martinez
40f7b258eb Add log-file flag to bootnode (#5392)
* Add log-file flag to bootnode
* Merge branch 'master' into bootnode-add-logfile
2020-04-11 20:12:48 +00:00
Preston Van Loon
8c9ae8e3bf Use state validator index map (#5383)
* remove validator index tracking from DB

* Builds production code path

* tests build

* fix tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-04-11 14:47:22 -05:00
Nishant Das
29f43190fc Revert "Validator client improvements" (#5388)
* Revert "Validator client improvements (#5300)"

This reverts commit 06db5f6471.
2020-04-11 03:31:01 +00:00
Preston Van Loon
b69c76c879 Remove next slot requirement for processing attestations (#5384)
* Remove next slot requirement for processing attestations
2020-04-11 00:12:47 +00:00
Preston Van Loon
ca0912e5d8 Add probot no-response configuration (#5382)
* Add probot no-response configuration
* use existing label Need-Info
2020-04-10 21:32:34 +00:00
Preston Van Loon
2e570d7427 Update CODEOWNERS (#5379)
* Update CODEOWNERS
2020-04-10 19:30:13 +00:00
Preston Van Loon
973a0a89b5 Update state root cache to fuzz in parallel (#5364)
* Update state root cache to fuzz in parallel

* confirmed arrays passing

Co-authored-by: rauljordan <raul@prysmaticlabs.com>
2020-04-10 13:12:59 -05:00
terence tsao
1d8b207d7c Fix incorrect proposer slashing domain calculation (#5369)
* StartSlot != SlotToEpoch
* Merge refs/heads/master into fix-proposer-slashing
2020-04-10 00:19:13 +00:00
Preston Van Loon
7a5010ecea Add roughtime static code analysis and fix all violations (#5370)
* Add roughtime static code analysis and fix all violations
* Merge branch 'master' into enforce-roughtime
2020-04-09 23:35:42 +00:00
Raul Jordan
7b8bad35e7 update readme (#5368) 2020-04-09 17:22:44 -05:00
Preston Van Loon
fa435d39bb Update bazel to 3.0.0 (#5363)
* Update bazel to 3.0.0
* Merge branch 'master' into update-bazelversion
2020-04-09 21:00:34 +00:00
Ivan Martinez
159ef3d702 Add cache to Eth1 RPC functions (#5347)
* ADd cache to other function and add comment

* Add cache for eth1data

* Fix test

* Reset cache before test

* Move cache to beacon node end

* Add metrics

* Disable blocktimebyheight cache

* Fix time
2020-04-09 14:29:09 -05:00
Ivan Martinez
06db5f6471 Validator client improvements (#5300)
* Look into what makes the validator client slow
* Comment
* Refactor status function to be cleaner
* Use timings from params
* Improve coverage
* Remove unneeded lock in chainstart
* Reorganize validator logging
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into optimize-validator
* cleanup
* Fix bug
* Undo depositbypukey changes
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into optimize-validator
* Fix deposit cache
* Fix multiple status test
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into optimize-validator
* Remove logs
* Remove unused import
* Add comment
* comment
* Remove comments
* Merge branch 'master' into optimize-validator
* Undo time change for activation
* Merge branch 'optimize-validator' of https://github.com/0xKiwi/Prysm into optimize-validator
* Fix check for sync
* Change value back to half slot
* comment
* Rephrase logs
* Merge branch 'master' into optimize-validator
2020-04-08 14:41:00 +00:00
terence tsao
a7bee79988 Use the config value (#5342) 2020-04-07 20:11:19 -05:00
Preston Van Loon
0fff07a93b Create CODEOWNERS (#5330)
* Create CODEOWNERS
* Move codeowners file to proper location
2020-04-07 03:22:43 +00:00
Preston Van Loon
70e64be8d6 Remove old cross compile starlark rules (#5329)
* Add buildbuddy BES (#5325)

* Add buildbuddy BES
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* remove old cross compile rules
2020-04-07 03:01:20 +00:00
Preston Van Loon
33ffa34ea7 Use less goroutines in validator runner (#5328)
* Add buildbuddy BES (#5325)

* Add buildbuddy BES
* Use less goroutines when running validator
* per-role based goroutines
* Merge branch 'master' into validator-issue-4702
2020-04-07 01:34:01 +00:00
Preston Van Loon
bcebf63cab Add buildbuddy BES (#5325)
* Add buildbuddy BES
2020-04-07 00:54:21 +00:00
terence tsao
d6f7d67ee9 Interop batch save validator indices (#5320)
* Batch save indices

* Update beacon-chain/interop-cold-start/service.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

* Update beacon-chain/interop-cold-start/service.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

* Update service.go

Co-authored-by: shayzluf <thezluf@gmail.com>
2020-04-06 13:40:42 -05:00
Chris Hobcroft
b7afc90266 Fixed typo in stdout log (#5317)
"Round robin**g** sync request failed"

changed to

"Round robin sync request failed"
2020-04-06 20:34:24 +08:00
Ivan Martinez
4b64a75c77 Remove unused validator protos (#5304)
* Remove unneeded protos

* Remove unused api point

* Gazelle

* Fix visibility

* Rename

* Change type

* Use iota for validator role
2020-04-06 11:24:24 +08:00
Ivan Martinez
fcf131412f Fix cluster in bazel and remove unused file (#5316)
* Fix cluster vazel

* remove unneeded file
2020-04-06 10:23:05 +08:00
Mattia
279dd5ac8d fix broken links in readme (#5313)
* fix broken links in readme

Activating a validator is referencing a broken link. This commit points to the new location of the documentation.
* Merge branch 'master' into master
2020-04-05 20:19:36 +00:00
Jim McDonald
c7a4fcd098 Reduce noise in validator logs (#5307)
* Reduce number of info-level messages on start of validator

* Test service, not log entry

* Gazelle

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
2020-04-05 15:36:18 -04:00
Ivan Martinez
07753189fd Remove unused slashing protos (#5308) 2020-04-05 12:50:52 -04:00
terence tsao
e162d27634 Save init synced cached blocks to db (#5309) 2020-04-04 16:25:04 -07:00
Victor Farazdagi
f440c815f9 Init sync update highest slot (#5298)
* updates highest slot before wrapping up
* more verbose error message
* error w/o stack
* revert back
2020-04-04 17:11:38 +03:00
Preston Van Loon
9aac572c21 Update @terencechain public key. (#5301)
* Update @terencechain public key.
2020-04-03 22:01:44 +00:00
Preston Van Loon
7bdd1355b8 Add maligned struct static check (#5296)
* Add maligned static check
* Add file, oops
* lint
2020-04-03 05:09:15 +00:00
Preston Van Loon
477b014bd1 Set a max limit for decoding ssz objects from p2p (#5295)
* Set a max limit for decoding ssz objects from p2p
2020-04-02 23:51:54 +00:00
terence tsao
ec7f7aebdc Clear init sync blocks on the correct line (#5294)
* Add disable-init-sync-batch-save-blocks
* Fix test
* Remove flag
* Merge branch 'master' into disable-init-sync-batch-save
* Quick fix
* Quick fix
* Merge branch 'master' of github.com:prysmaticlabs/prysm into disable-init-sync-batch-save
* Merge branch 'disable-init-sync-batch-save' of github.com:prysmaticlabs/prysm into disable-init-sync-batch-save
* Clear init sync blocks at the right place
2020-04-02 22:27:51 +00:00
terence tsao
3544ed2818 Invert init-sync-batch-save-blocks flag for v0.11 (#5293) 2020-04-02 14:46:14 -07:00
Victor Farazdagi
b43e43b4a9 Init sync release queue (#5286)
* fix naming slot -> epoch

* better handling of long periods w/o finality

* bazel

* fixes issue with pointer goint to far ahead

* adds func comment

* hides original sync behind --disable-init-sync-queue

* adds func comment

* deprecated

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-04-02 11:06:32 -05:00
Victor Farazdagi
c26a492225 Init sync optimizations (#5284)
* fix naming slot -> epoch
* better handling of long periods w/o finality
* fixes issue with pointer going too far ahead
2020-04-02 06:54:05 +03:00
shayzluf
0df12261a1 slasher retrieve and cache validator public key (#5220)
* cache and retrieval of validator public keys

* fix comments

* fix comment

* fix variables

* gaz

* ivan feedback fixes

* goimports

* fix test

* comments on in line slice update

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-04-02 06:08:23 +03:00
Raul Jordan
f385a1dea6 Release Skip Slot Cache to All (#5280)
* no more skip slot cache

* imports

* deprecated

* fix flakeyness

* disable in e2e

* build

* fix viz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-04-01 19:09:54 -07:00
Preston Van Loon
8376fb36ca Enable slashing protection in validator by default (#5278)
* Invert slashing protection validator flags for issue #5267
* remove from e2e flags
* Make error level
* Merge refs/heads/master into flip-propose
2020-04-01 23:17:32 +00:00
Jim McDonald
02b238bda2 Add latch for proposer warnings (#5258)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
2020-04-01 17:28:02 -05:00
Ivan Martinez
3dd5576e33 Improvement, flake fixes (#5263) 2020-04-01 10:23:23 -05:00
terence tsao
df9a534826 Regen historical states for new-state-mgmt compatibility (#5261) 2020-03-31 16:54:24 -07:00
Nishant Das
7e50c36725 Add Hasher To State Data Types (#5244)
* add hasher
* re-order stuff
* Merge refs/heads/master into addHasher
* Merge refs/heads/master into addHasher
* Merge refs/heads/master into addHasher
* Merge refs/heads/master into addHasher
* Merge refs/heads/master into addHasher
* Merge refs/heads/master into addHasher
* Merge refs/heads/master into addHasher
* Merge refs/heads/master into addHasher
* Merge refs/heads/master into addHasher
* Merge refs/heads/master into addHasher
2020-03-31 18:57:19 +00:00
terence tsao
e22365c4a8 Uncomment out cold state tests (#5252)
* Fixed most of the tests

* All tests passing

* All tests passing

* Fix merge conflict

* Fixed error test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-31 11:23:39 -05:00
Nishant Das
c8f8e3f1e0 Unmarshal Block instead of State (#5246)
* unmarshal block instead of state
* add fallback
* Merge refs/heads/master into dontUnmarshal
* Merge refs/heads/master into dontUnmarshal
* Merge refs/heads/master into dontUnmarshal
* Merge refs/heads/master into dontUnmarshal
* Merge refs/heads/master into dontUnmarshal
* Merge refs/heads/master into dontUnmarshal
* Merge refs/heads/master into dontUnmarshal
* Merge refs/heads/master into dontUnmarshal
2020-03-31 15:25:58 +00:00
Ivan Martinez
3e81afd7ab Skip anti-flake E2E tests (#5257)
* Skip anti-flake

* Log out the shard index to see it per shard

* Attempt fixes

* Remove unneeded log

* Change eth1 ports

* Remove skips

* Remove log

* Attempt local build

* Fix formatting

* Formatting

* Skip anti flake tests
2020-03-31 23:15:33 +08:00
Ivan Martinez
404a0f6bda Attempt E2E flaking fix (#5256)
* Fix test sharding

* Attempt fix
2020-03-31 12:39:56 +08:00
Preston Van Loon
00ef08b3dc Debug: add cgo symbolizer (#5255)
* Add cgo_symbolizer config

* Add comment

* use import block
2020-03-30 20:20:27 -07:00
Preston Van Loon
6edb3018f9 Add configurations for BLS builds (#5254)
* Add configurations for BLS builds
* Merge refs/heads/master into bls-configurations
2020-03-31 01:58:27 +00:00
Preston Van Loon
17516b625e Use math.Sqrt for IntegerSquareRoot (#5253)
* use std
* Merge refs/heads/master into faster-sqrt
2020-03-31 01:27:37 +00:00
terence tsao
7f7866ff2a Micro optimizations on new-state-mgmt service for initial syncing (#5241)
* Starting a quick PoC

* Rate limit to one epoch worth of blocks in memory

* Proof of concept working

* Quick comment out

* Save previous finalized checkpoint

* Test

* Minor fixes

* More run time fixes

* Remove panic

* Feature flag

* Removed unused methods

* Fixed tests

* E2e test

* comment

* Compatible with current initial sync

* Starting

* New cache

* Cache getters and setters

* It should be part of state gen

* Need to use cache for DB

* Don't have to use finalized state

* Rm unused file

* some changes to memory mgmt when using mempool

* More run time fixes

* Can sync to head

* Feedback

* Revert "some changes to memory mgmt when using mempool"

This reverts commit f5b3e7ff47.

* Fixed sync tests

* Fixed existing tests

* Test for state summary getter

* Gaz

* Fix kafka passthrough

* Fixed inputs

* Gaz

* Fixed build

* Fixed visibility

* Trying without the ignore

* Didn't work..

* Fix kafka

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-03-30 17:10:45 -05:00
terence tsao
c5f186d56f Batch save blocks for initial sync. 80% faster BPS (#5215)
* Starting a quick PoC
* Rate limit to one epoch worth of blocks in memory
* Proof of concept working
* Quick comment out
* Save previous finalized checkpoint
* Merge branch 'master' of github.com:prysmaticlabs/prysm into batch-save
* Test
* Merge branch 'prev-finalized-getter' into batch-save
* Minor fixes
* Use a map
* More run time fixes
* Remove panic
* Feature flag
* Removed unused methods
* Fixed tests
* E2e test
* Merge branch 'master' into batch-save
* comment
* Merge branch 'master' into batch-save
* Compatible with current initial sync
* Merge branch 'batch-save' of github.com:prysmaticlabs/prysm into batch-save
* Merge refs/heads/master into batch-save
* Merge refs/heads/master into batch-save
* Merge refs/heads/master into batch-save
* Merge branch 'master' of github.com:prysmaticlabs/prysm into batch-save
* Feedback
* Merge branch 'batch-save' of github.com:prysmaticlabs/prysm into batch-save
* Merge refs/heads/master into batch-save
2020-03-30 18:04:10 +00:00
Ivan Martinez
0982ff124e Fix E2E test sharding (#5248) 2020-03-30 12:10:00 -04:00
Nishant Das
63df1d0b8d Add Merkleize With Customized Hasher (#5234)
* add buffer for merkleizer
* add comment
* Merge refs/heads/master into merkleize
* Merge refs/heads/master into merkleize
* Merge refs/heads/master into merkleize
* Merge refs/heads/master into merkleize
* Merge refs/heads/master into merkleize
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into merkleize
* Merge branch 'merkleize' of https://github.com/prysmaticlabs/geth-sharding into merkleize
* lint
* Merge refs/heads/master into merkleize
2020-03-29 06:13:24 +00:00
Ivan Martinez
cb9ac6282f Separate anti flakes to prevent E2E issues (#5238)
* Separate anti flakes

* Gaz
2020-03-29 13:54:13 +08:00
terence tsao
c67b01e5d3 Check new state mgmt service is compatible with DB (#5231) 2020-03-28 18:07:51 -07:00
terence tsao
b40e6db1e5 Fix save blocks return nil (#5237)
* Fixed save blocks return nil
* Merge refs/heads/master into fix-batch-save-blocks
* Merge refs/heads/master into fix-batch-save-blocks
2020-03-28 19:05:56 +00:00
Preston Van Loon
f89d753275 Add configurable e2e epochs (#5235)
* Add configurable e2e epochs
* Merge refs/heads/master into configurable-test-epochs
* Merge refs/heads/master into configurable-test-epochs
2020-03-28 18:47:31 +00:00
Preston Van Loon
a24546152b HashProto: Use fastssz when available (#5218)
* Use fastssz when available
* fix tests
* fix most tests
* Merge branch 'master' into faster-hash-proto
* Merge refs/heads/master into faster-hash-proto
* Merge refs/heads/master into faster-hash-proto
* Merge refs/heads/master into faster-hash-proto
* fix last test
* Merge branch 'faster-hash-proto' of github.com:prysmaticlabs/prysm into faster-hash-proto-2
* lint
* fix last test
* fix again
* Update beacon-chain/cache/checkpoint_state_test.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>
* Merge refs/heads/master into faster-hash-proto
2020-03-28 18:32:11 +00:00
Preston Van Loon
6bc70e228f Prevent panic for different size bitlists (#5233)
* Fix #5232
* Merge branch 'master' into bugfix-5232
2020-03-28 06:25:49 +00:00
terence tsao
f2a3fadda7 Productionization new state service part 1 (#5230)
* Fixed last play methods

* Fixed a regression. Genesis case for state gen

* Comment

* Starting

* Update proto

* Remove boundary root usages

* Update migrate

* Clean up

* Remove unused db methods

* Kafta

* Kafta

* Update tests

* Comments

* Fix state summary tests

* Missed one pass through for kafta
2020-03-27 13:28:38 -07:00
terence tsao
6a4b17f237 Prune garbage state is not for new state mgmt (#5225)
* Prune garbage state is not for new state mgmt
* Merge branch 'master' into state-mgmt-pruning
* Merge branch 'master' into state-mgmt-pruning
* Merge branch 'master' into state-mgmt-pruning
2020-03-27 14:30:24 +00:00
Victor Farazdagi
7ebb3c1784 init-sync revamp (#5148)
* fix issue with rate limiting
* force fetcher to wait for minimum peers
* adds backoff interval
* cap the max blocks requested from a peer
* queue rewritten
* adds docs to fsm
* fix visibility
* updates fsm
* fsm tests added
* optimizes queue resource allocations
* removes debug log
* replace auto-fixed comment
* fixes typo
* better handling of evil peers
* fixes test
* minor fixes to fsm
* better interface for findEpochState func
2020-03-27 09:54:57 +03:00
Nishant Das
33f6c22607 Revert "Add Fast Copy of Trie" (#5228)
* Revert "new fixes (#5221)"

This reverts commit 4118fa5242.
2020-03-27 01:06:30 +00:00
terence tsao
1a0a399bed Handle genesis case for blocks/states at slot index (#5224)
* Handle highest slot = 0
* TestStore_GenesisState_CanGetHighestBelow
* TestStore_GenesisBlock_CanGetHighestAt
* Merge refs/heads/master into handle-genesis
2020-03-27 00:09:14 +00:00
Preston Van Loon
c4c9a8465a Faster hashing for attestation pool (#5217)
* use faster hash proto
* Merge branch 'master' into faster-att-pool
* gaz
* Merge branch 'faster-att-pool' of github.com:prysmaticlabs/prysm into faster-att-pool
* nil checks and failing tests
* Merge refs/heads/master into faster-att-pool
* Merge refs/heads/master into faster-att-pool
* Merge refs/heads/master into faster-att-pool
* Merge refs/heads/master into faster-att-pool
* Merge refs/heads/master into faster-att-pool
* Fix
* Merge branch 'faster-att-pool' of github.com:prysmaticlabs/prysm into faster-att-pool
* Fix tests
2020-03-26 23:55:25 +00:00
terence tsao
5e2faf1a9d Short circuit genesis condition for new state mgmt (#5223)
* Fixed a regression. Genesis case for state gen

* Comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-26 14:13:45 -05:00
shayzluf
93e68db5e6 is slashable attestation endpoint implementation (#5209)
* is slashable attestation endpoint implementation
* fix todo
* comment
* Merge refs/heads/master into is_slashable_attestation
* Merge refs/heads/master into is_slashable_attestation
* Merge refs/heads/master into is_slashable_attestation
* Update slasher/rpc/server.go
* Update slasher/rpc/server.go
* Update slasher/rpc/service.go
2020-03-26 18:31:20 +00:00
Nishant Das
cdac3d61ea Custom Block HTR (#5219)
* add custom htr

* fix root

* fix everything

* Apply suggestions from code review

* Update beacon-chain/state/stateutil/blocks.go

* Update beacon-chain/blockchain/receive_block.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Update beacon-chain/blockchain/process_block.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* terence's review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-26 13:10:22 -05:00
Nishant Das
4118fa5242 new fixes (#5221)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-26 11:21:02 -05:00
terence tsao
2df76798bc Add HighestSlotStatesBelow DB getter (#5213)
* Add HighestSlotStatesBelow
* Tests for HighestSlotStatesBelow
* Typos
* Comment
* Merge refs/heads/master into states-slots-saved-at
* Quick fix
* Merge branch 'states-slots-saved-at' of github.com:prysmaticlabs/prysm into states-slots-saved-at
* Prevent underflow foreal, thanks nishant!
2020-03-26 15:37:40 +00:00
Preston Van Loon
3792bf67b6 Add alpine based docker images for validator and beacon chain (#5214)
* Add alpine based images for validator and beacon chain

* Use an alpine image with glibc

* manual tags on transitional targets

* poke buildkite

* poke buildkite
2020-03-25 19:36:28 -05:00
Nishant Das
e077d3ddc9 Fix Incorrect Logging for IPV6 Addresses (#5204)
* fix ipv6 issues
* Merge branch 'master' into fixIPV6
* imports
* Merge branch 'fixIPV6' of https://github.com/prysmaticlabs/geth-sharding into fixIPV6
* Merge branch 'master' into fixIPV6
2020-03-25 17:19:11 +00:00
Preston Van Loon
2ad5cec56c Add gRPC headers flag support for validator client side (#5203)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-25 11:29:04 -05:00
terence tsao
6b1e60c404 Remove extra udp port log (#5205)
* Remove extra udp port log
* Merge branch 'master' into rm-log
2020-03-25 15:28:29 +00:00
terence tsao
48e984f526 Add HighestSlotBlocksBelow getter for db (#5195)
* Add HighestSlotBlockAt

* Start testing

* Apply fixes

* Typo

* Test

* Rename for clarity

* Use length

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-25 08:41:48 -05:00
Preston Van Loon
fbee94a0e9 Remove deprecated aggregator (#5200) 2020-03-25 06:14:21 -07:00
Preston Van Loon
9740245ca5 Add enable-state-field-trie for e2e (#5198)
* Add enable-state-field-trie for e2e
* Merge refs/heads/master into e2e-enable-state-field-trie
* Merge refs/heads/master into e2e-enable-state-field-trie
* fix all this
* Update shared/sliceutil/slice.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>
* terence's review
* comment
* Merge branch 'e2e-enable-state-field-trie' of https://github.com/prysmaticlabs/geth-sharding into e2e-enable-state-field-trie
2020-03-25 06:54:56 +00:00
Preston Van Loon
48d4a8655a Add ipv6 multiaddr support (#5199)
* Add ipv6 multiaddr support
* allow ipv6 for discv5
2020-03-25 04:03:51 +00:00
terence tsao
e15d92df06 Apply fixes to block slots methods in DB (#5194)
* Apply fixes
* Typo
* Merge refs/heads/master into fix-slots-saved-for-blocks
2020-03-25 03:05:20 +00:00
Preston Van Loon
729bd83734 Add span to HTR and skip slot cache (#5197)
* Add a bit more span data
* missing import
* Merge branch 'master' into more-spans
2020-03-25 01:15:00 +00:00
terence tsao
c63fb2cd44 Add HighestSlotState Getter for db (#5192) 2020-03-24 14:51:24 -07:00
terence tsao
78a865eb0b Replace boltdb imports with bbolt import (#5193)
* Replaced. Debugging missing strict dependencies...
* Merge branch 'master' into bbolt-import
* Update import path
* Merge branch 'bbolt-import' of github.com:prysmaticlabs/prysm into bbolt-import
* use forked prombbolt
* Merge branch 'bbolt-import' of github.com:prysmaticlabs/prysm into bbolt-import
* fix
* remove old boltdb reference
* Use correct bolt for pk manager
* Merge branch 'bbolt-import' of github.com:prysmaticlabs/prysm into bbolt-import
* fix for docker build
* gaz, oops
2020-03-24 20:00:54 +00:00
shayzluf
6e516dabf9 Setup Slasher RPC server (#5190)
* slasher rpc server

* fix comment

* fix comment

* remove server implementation from pr

* Apply suggestions from code review

* Gazelle

* Update slasher/rpc/service.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

* Update slasher/detection/detect.go

* Update slasher/detection/detect.go

* Update slasher/detection/detect.go

* Update slasher/detection/detect.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-03-24 14:30:21 -04:00
Nishant Das
454e02ac4f Add Improvements to State Benchmarks (#5187)
* add improvements

* clean up
2020-03-24 09:16:07 -05:00
Preston Van Loon
35d74981a0 Correctly return attestation data for late requests (#5183)
* Add functionality to support attestation requests that are older than the head state

* lint

* gaz

* Handle nil state case

* handle underflow of first epoch

* Remove redundant and possibly wrong genesisTime struct field

* fix remaining tests

* gofmt

* remove debug comment

* use stategen.StateByRoot interchangably with beaconDB.State

* gofmt

* goimports

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-23 21:30:28 -07:00
Ivan Martinez
d8bcd891c4 Change E2E config ports to be non-commonly used (#5184)
* Change config ports to be non-commonly used
2020-03-24 03:20:38 +00:00
terence tsao
2e0158d7c5 Add HighestSlotBlock Getter for db (#5182)
* Starting

* Update block getters in db

* New test

* New test for save blocks as well

* Delete blocks can clear bits tests

* Fmt
2020-03-23 18:42:41 -05:00
Ivan Martinez
bdb80f4639 Change ListAttestations to get attestations from blocks (#5145)
* Start fixing api to get from blocks

* Fix listatts tests

* Fix slasher

* Improve blocks

* Change att grouping

* Use faster att concat

* Try to fix e2e

* Change back time

* tiny e2e fix

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-03-23 16:22:37 -04:00
Preston Van Loon
0043fb0441 Shard //beacon-chain/core/state:go_default_test (#5180)
* Add shards to core state tests, some of the fuzzing tests can be very slow
* Merge refs/heads/master into shard-core-state-tests
2020-03-23 19:08:39 +00:00
Preston Van Loon
f520472fc6 Buildkite changes (#5178)
* Do not override jobs, dont print colors
* Merge branch 'master' of github.com:prysmaticlabs/prysm into buildkite-changes
* use composite flag for minimal downloads
* Add repository cache
* use hardlinks
* repository cache common
* query and build repo cache
2020-03-23 19:00:37 +00:00
Preston Van Loon
5241582ece Add CORS preflight support (#5177)
* Add CORS preflight support

* lint

* clarify description
2020-03-23 13:17:17 -05:00
Nishant Das
b0128ad894 Add Attestation Subnet Bitfield (#4989)
* bump bitfield dep

* add new methods

* get it working

* add nil check

* add check

* one more check

* add flag

* everything works local run

* add debug log

* more changes

* ensuring p2p interface works enough for tests to pass

* all tests pass

* include proper naming and comments to fix lint

* Apply suggestions from code review

* discover by peers

* cannot figure out why 0 peers

* remove keys

* fix test

* fix it

* fix again

* remove log

* change back

* gaz

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-03-23 09:41:47 -05:00
Preston Van Loon
5d1c3da85c BLS: some minor improvements (#5161)
* some improvements
* gofmt
* Merge refs/heads/master into bls-improvements
* Merge refs/heads/master into bls-improvements
* Merge refs/heads/master into bls-improvements
2020-03-22 23:40:39 +00:00
terence tsao
301c2a1448 New byteutils for state gen (#5163)
* New byteutils for state gen
* Added empty slice and nil checks
* Merge branch 'master' into bit-utils
* SetBit to extend bit
* Merge branch 'bit-utils' of github.com:prysmaticlabs/prysm into bit-utils
* Comment
* Add HighestBitIndexBelow
* Test for HighestBitIndexBelow
* another test and better test fail output
* Feedback
* Merge branch 'bit-utils' of github.com:prysmaticlabs/prysm into bit-utils
* Feedback
* Preston's feedback, thanks!
* Use var
* Use var
* Merge refs/heads/master into bit-utils
2020-03-22 23:19:38 +00:00
Ivan Martinez
bc3d673ea4 Parallelize E2E Testing (#5168)
* Begin cleanup for E2E

* Parellize testing

* Add comments

* Add comment
2020-03-22 19:04:23 -04:00
Preston Van Loon
3d092d3eed Update go-bitfield (#5162)
* Update go-bitfield from https://github.com/prysmaticlabs/go-bitfield/pull/28
2020-03-22 04:51:06 +00:00
Preston Van Loon
4df5c042d9 Use faster bitfield BitIndices to build attesting indices (#5160)
* Refactor AttestingIndices to not return any error. Add tests. Add shortcut for fully attested attestation
* attestationutil.ConvertToIndexed never returned error either
* Working with benchmark:
* fix test
* Merge branch 'attestationutil-improvements-0' into attestationutil-improvements-1
* out of bounds check
* Update after merge of https://github.com/prysmaticlabs/go-bitfield/pull/26
* remove shortcut
* Merge refs/heads/attestationutil-improvements-0 into attestationutil-improvements-1
* Merge branch 'attestationutil-improvements-0' into attestationutil-improvements-1
* Merge branch 'attestationutil-improvements-1' of github.com:prysmaticlabs/prysm into attestationutil-improvements-1
* revert test...
* Merge refs/heads/attestationutil-improvements-0 into attestationutil-improvements-1
* Merge branch 'master' of github.com:prysmaticlabs/prysm into attestationutil-improvements-1
* Merge branch 'attestationutil-improvements-1' of github.com:prysmaticlabs/prysm into attestationutil-improvements-1
* Update go-bitfield after https://github.com/prysmaticlabs/go-bitfield/pull/27
2020-03-22 01:42:51 +00:00
Preston Van Loon
d06b0e8a86 Refactor attestationutil.AttestingIndices (#5159)
* Refactor AttestingIndices to not return any error. Add tests. Add shortcut for fully attested attestation
* attestationutil.ConvertToIndexed never returned error either
* fix test
* remove shortcut
* revert test...
2020-03-22 00:23:37 +00:00
Jim McDonald
4f8d9c59dd Replace default value for datadir (#5147) 2020-03-21 23:30:51 +08:00
Ivan Martinez
021d777b5e Add Anti-Flake test for E2E (#5149)
* Add antiflake test

* Respond to comments

* Comment

* Change issue num
2020-03-21 14:42:51 +08:00
terence tsao
dc3fb018fe Fix new state mgmt sync stuck in a loop (#5142) 2020-03-19 18:46:35 -07:00
Preston Van Loon
2ab4b86f9b Allow setting flags via yaml config file. (#4878) 2020-03-19 14:46:44 -07:00
Ivan Martinez
b30a089548 Add fetching validators by indices and public keys (#5141)
* update ethereumapis with patch
* Add indices and pubkeys to ListValidators request
* Add sorting
* Merge branch 'master' into validators-by-keys-indices
* Rename to index
* Merge branch 'validators-by-keys-indices' of https://github.com/prysmaticlabs/prysm into validators-by-keys-indices
* Add comment
2020-03-19 20:30:40 +00:00
Ivan Martinez
271938202e Improve validator logs (#5140)
* Imporve validator logging

* Update validator/client/validator_log.go
2020-03-19 13:34:50 -04:00
shayzluf
6fe814c5aa double proposal detector (#5120)
* proposal detector

* comment fixes

* comment fixes

* raul feedback

* fix todo

* gaz

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-03-19 17:29:35 +05:30
Preston Van Loon
a9f4d1d02d Attestation: Add a check for overflow (#5136)
* Add a check for overflow
* gofmt beacon-chain/cache/committee_test.go
2020-03-19 04:41:05 +00:00
Preston Van Loon
7c110e54f0 Add ssz marshal and unmarshal for most data structures (#5121)
* Add ssz marshal and unmarshal for most data structures
* Merge refs/heads/master into ssz-stuff
* Merge refs/heads/master into ssz-stuff
* Merge refs/heads/master into ssz-stuff
* Merge refs/heads/master into ssz-stuff
* Merge refs/heads/master into ssz-stuff
* Merge refs/heads/master into ssz-stuff
* Update ferran SSZ
* Update ferran's SSZ
* Merge refs/heads/master into ssz-stuff
* fix tests
* Merge branch 'ssz-stuff' of github.com:prysmaticlabs/prysm into ssz-stuff
* gaz
2020-03-19 02:39:23 +00:00
Raul Jordan
3043d4722f Attestation Dynamic Committee Subnets (#5123)
* initiate cache
* imports fix
* add in feature config flag
* utilize a dynamic set of subnets
* Merge branch 'master' into att-subnets
* add in feature config flag
* Merge branch 'att-subnets' of github.com:prysmaticlabs/prysm into att-subnets
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into att-subnets
* shift
* more changes
* gaz
* Update beacon-chain/rpc/validator/assignments.go
* Update beacon-chain/rpc/validator/assignments.go
* add flag
* Merge branch 'att-subnets' of https://github.com/prysmaticlabs/geth-sharding into att-subnets
* Merge branch 'master' into att-subnets
* Merge refs/heads/master into att-subnets
* no double flag
* Merge branch 'att-subnets' of github.com:prysmaticlabs/prysm into att-subnets
* amend committee ids to better name
* gaz
2020-03-18 23:13:37 +00:00
Ivan Martinez
c96c8b4aa3 Minor slasher fixes (#5129)
* Minor fixes

* Change log
2020-03-18 14:49:20 -05:00
Nishant Das
9f46000dba change to batch size (#5128)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-18 17:57:20 +08:00
Nishant Das
5450b3155e Integrate Field Tries into Current State (#5082)
* add new methods
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into improvedHTRArrays
* new field trie
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into improvedHTRArrays
* perform better copying
* fix bug
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into improvedHTRArrays
* add support for variable length arrays
* get it running
* save all new progress
* more fixes
* more fixes
* more cleanup
* some more clean up
* new memory pool
* remove lock
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into improvedHTRArrays
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into improvedHTRArrays
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into improvedHTRArrays
* use wrapper
* remove redundant methods
* cleanup
* cleanup
* remove unused method
* change field
* Update beacon-chain/state/types.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>
* Update beacon-chain/state/types.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>
* Apply suggestions from code review

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>
2020-03-18 04:52:08 +00:00
Nishant Das
1bb12c3568 Add Field Trie to State (#5118)
* add new helpers

* make zerohash public

* remove unused method

* add more tests

* cleanup

* add in new tests

* fix all tests

* Apply suggestions from code review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-03-18 10:09:31 +08:00
Ivan Martinez
1be8b3aa5e Slasher lag fix (#5124)
* Slasher fixes

* fix
2020-03-17 16:53:08 -05:00
Nishant Das
431762164e Add New State Utils (#5117)
* add new helpers

* make zerohash public

* remove unused method

* add more tests

* cleanup

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-17 14:25:17 -05:00
Victor Farazdagi
3ec2a0f9e0 Refactoring of initial sync (#5096)
* implements blocks queue

* refactors updateCounter method

* fixes deadlock on stop w/o start

* refactors updateSchedulerState

* more tests on schduler

* parseFetchResponse tests

* wraps up tests for blocks queue

* eod commit

* fixes data race in round robin

* revamps fetcher

* fixes race conditions + livelocks + deadlocks

* less verbose output

* fixes data race, by isolating critical sections

* minor refactoring: resolves blocking calls

* implements init-sync queue

* udpate fetch/send buffers in blocks fetcher

* blockState enum-like type alias

* refactors common code into releaseTicket()

* better gc

* linter

* minor fix to round robin

* moves original round robin into its own package

* adds enableInitSyncQueue flag

* fixes issue with init-sync service selection

* Update beacon-chain/sync/initial-sync/round_robin.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* initsyncv1 -> initsyncold

* adds span

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-03-17 12:27:18 -05:00
Victor Farazdagi
e96b45b29c asserts non-nil state (#5115) 2020-03-17 07:58:16 -07:00
terence tsao
e529f5b1d6 Part 1 of integrating new state mgmt to run time (#5108) 2020-03-16 12:07:07 -07:00
Victor Farazdagi
f18bada8c9 Init sync blocks queue (#5064)
* fixes data race, by isolating critical sections

* minor refactoring: resolves blocking calls

* implements init-sync queue

* udpate fetch/send buffers in blocks fetcher

* blockState enum-like type alias

* refactors common code into releaseTicket()

* better gc

* linter

* Update beacon-chain/sync/initial-sync/blocks_queue.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

* Update beacon-chain/sync/initial-sync/blocks_queue_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: shayzluf <thezluf@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-03-16 18:21:36 +03:00
terence tsao
5657535c52 Fixed saveNewValidators error log (#5109) 2020-03-15 16:21:56 -07:00
Preston Van Loon
9da9fbdfba Fix reward and penality zero epoch bug (#5107)
* Fix reward and penality bug https://github.com/prysmaticlabs/prysm/issues/5105
* Merge branch 'master' into fuzz-fix-attestationDelta
2020-03-15 19:14:52 +00:00
Ivan Martinez
de2ec8e575 Update README for Slasher (#5106)
* Add readme
2020-03-15 18:46:21 +00:00
terence tsao
3660732f44 Resume new state mgmt (#5102) 2020-03-15 09:47:49 -07:00
Jim McDonald
8e6c16d416 Tweak validator logging (#5103)
* Tidy up logging
2020-03-15 15:46:22 +00:00
Ivan Martinez
8143cc36bc Add Slasher to E2E (#5061)
* Start adding "inject slashing into pool"

* Attempt at slashing

* Remove unneded

* Fix

* Begin adding slasher client to e2e

* Start slasher in e2e

* Get slashing detection working

* Get slashing evaluators working

* Progress on e2e

* Cleanup e2e

* Fix slasher e2e!

* lint

* Comment

* Fixes

* Improve accuracy of balance check

* REmove extra

* Remove extra

* Make more accurate
2020-03-15 01:09:23 -04:00
terence tsao
eeffa4fb30 New state getter (#5101)
* getter.go
* getter_test.go
* fixed a cold bug
* fmt gaz
* All tests pass
* Merge branch 'master' into new-state-getter
* Merge refs/heads/master into new-state-getter
2020-03-14 18:39:23 +00:00
Victor Farazdagi
1137403e4b Init sync pre queue (#5098)
* fixes data race, by isolating critical sections

* minor refactoring: resolves blocking calls

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-14 13:21:07 -05:00
terence tsao
f17818b1c0 New state setter (#5100)
* setter.go
* tests
* fmt
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-state-setter
* Merge refs/heads/master into new-state-setter
2020-03-14 16:31:21 +00:00
Nishant Das
691f0bba70 Minor Improvements (#5099)
* add fixes
2020-03-14 16:12:22 +00:00
terence tsao
b024191887 Get cold intermediate state with slot (#5097)
* loadColdIntermediateStateWithSlot

* Starting test

* Tests
2020-03-14 10:34:37 -05:00
Raul Jordan
1f87cb11fc Use Current Time Slot for Fetching Committees in RPC (#5094)
* use genesis time fetcher
* Merge branch 'master' into use-time-fetcher
* fix breaking
* Merge branch 'use-time-fetcher' of github.com:prysmaticlabs/prysm into use-time-fetcher
* list beacon committees tests fixed
* Merge branch 'master' into use-time-fetcher
* Merge branch 'master' into use-time-fetcher
* Merge refs/heads/master into use-time-fetcher
* Update beacon-chain/rpc/beacon/committees_test.go
2020-03-14 03:32:51 +00:00
Preston Van Loon
a0b142a26c Update to go 1.14 (#4947)
* Update to go 1.14
* Update with fix from https://github.com/bazelbuild/rules_go/pull/2388
* Merge branch 'master' into go-1.14
* Merge refs/heads/master into go-1.14
* Merge branch 'master' of github.com:prysmaticlabs/prysm into go-1.14
* Update gRPC
* Merge branch 'go-1.14' of github.com:prysmaticlabs/prysm into go-1.14
* Update golang.org/x/crypto
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Merge refs/heads/master into go-1.14
* Committing gc_goopts for issue repro
* Fix race and msan builds
* Merge branch 'master' of github.com:prysmaticlabs/prysm into go-1.14
* Merge refs/heads/master into go-1.14
* switch to LRU
* Merge branch 'go-1.14' of github.com:prysmaticlabs/prysm into go-1.14
* Fixed, but dont feel good about this
* Switch append ordering
2020-03-14 00:12:52 +00:00
shayzluf
035eaffd9d handle slashing from p2p (#5047)
* handle slashing from p2p

* gaz

* remove unneeded check

* add tests

* gaz  goimports

* text update

* Apply suggestions from code review

* add proto.equal

* fix test

* add context to call

* fix state bug found by terence

* fix tests add error type handling

* nil checks

* nil head state check

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-03-13 16:47:27 -05:00
Ivan Martinez
c41244ad34 Make spanner tests more thorough, fixes (#5093)
* Fix tests for spanner

* Start change to indexed atts

* Improve tests for spanner

* Fix tests

* Remove extra
2020-03-13 14:04:22 -04:00
terence tsao
c20d9ccbb3 Better attestation pool with map instead of expiration cache (#5087)
* update aggregated

* update block att

* update forkchoice att

* update unaggregated att

* gazelle

* Use copy

* Locks

* Genesis time

* Fixed tests

* Comments

* Fixed tests
2020-03-13 12:35:28 -05:00
Ivan Martinez
3380d14475 Include ejected indices in ActiveSetChanges endpoint (#5066)
* Fix ActiveSetChanges

* Include ejected indices in ActiveSetChanges RPC

* Fix test fails

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-03-13 12:23:19 -04:00
shayzluf
4f031d1988 fix slasher rpc disconnect on error (#5092) 2020-03-13 10:59:14 -05:00
Jim McDonald
02afb53ea4 Remove suprious error messages in wallet keymanager (#5090)
* Handle multiple passphrases

* Add tests
2020-03-13 05:26:10 -07:00
terence tsao
0974c02a00 Load cold state by root (#5086) 2020-03-12 15:27:55 -07:00
Raul Jordan
c6acf0a28c Use Target Epoch to Determine Indexed Attestations for Slasher (#5085)
* no more head fetchre

* no need for head fetcher

* nil checks
2020-03-12 17:02:12 -05:00
terence tsao
ed7ad4525e Method to retrieve block slot via block root (#5084)
* blockRootSlot

* Tests

* Gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-12 16:04:24 -05:00
terence tsao
7fcc07fb45 Save hot state (#5083)
* loadEpochBoundaryRoot
* Tests
* Span
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into save-hot-state
* Starting test
* Tests
* Merge refs/heads/master into save-hot-state
* Merge branch 'master' into save-hot-state
* Use copy
* Merge branch 'save-hot-state' of https://github.com/prysmaticlabs/prysm into save-hot-state
* Merge refs/heads/master into save-hot-state
2020-03-12 20:48:07 +00:00
shayzluf
f937713fe9 Broadcast slashing (#5073)
* add flag
* broadcast slashings
* Merge branch 'master' of github.com:prysmaticlabs/prysm into broadcast_slashing

# Conflicts:
#	beacon-chain/rpc/beacon/slashings_test.go
* fix tests
* goimports
* goimports
* Merge branch 'master' into broadcast_slashing
* Merge branch 'master' into broadcast_slashing
* Merge branch 'master' into broadcast_slashing
2020-03-12 20:29:23 +00:00
terence tsao
359e0abe1d Load epoch boundary root (#5079)
* loadEpochBoundaryRoot

* Tests

* Span

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-12 15:00:37 -05:00
tzapu
0704ba685a Return statuses on duties (#5069)
* try to return somethign for everything
* default to unknown
* debug
* moar debug
* move else to outer check
* working
* reorder imports
* cleanup
* fix TestGetDuties_NextEpoch_CantFindValidatorIdx
* Merge branch 'master' into return-statuses-on-duties
* Update validator/client/validator.go
* Merge branch 'master' into return-statuses-on-duties
* Merge branch 'master' into return-statuses-on-duties
2020-03-12 19:07:37 +00:00
shayzluf
0f95b797af Save slashings to slasher DB (#5081)
* fix tests add error type handling

* Update slasher/detection/detect_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

* goimports

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
2020-03-12 22:08:58 +05:30
terence tsao
43722e45f4 Save cold state (#5077) 2020-03-12 05:58:06 -07:00
terence tsao
ff4ed413a3 State migration from hot to cold (archived) (#5076)
* Starting

* Test

* Tests

* comments

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-11 21:27:16 -05:00
Raul Jordan
f1a42eb589 Verify Slashing Signatures Before Putting Into Blocks (#5071)
* rem slasher proto
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* verify slashing
* added in test for pending att slashing
* tests starting to apss
* sig failed verify regression test
* tests passing for ops pool
* Update beacon-chain/operations/slashings/service.go
* Merge refs/heads/master into verify-slash-sig
* verify on insert
* tests starting to pass
* all code paths fixed
* imports
* fix build
* fix rpc errors
* Merge refs/heads/master into verify-slash-sig
2020-03-12 01:16:55 +00:00
terence tsao
a90ffaba49 Archived point retrieval and recovery (#5075) 2020-03-11 17:38:30 -07:00
Raul Jordan
663d919b6f Include Bazel Genrule for Fast SSZ (#5070)
* rem slasher proto
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* included new ssz bzl rule
* Merge branch 'master' into add-in-starlark-rule
* Update tools/ssz.bzl

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
2020-03-11 19:50:22 +00:00
Victor Farazdagi
7b30845c01 fixes races in blocks fetcher (#5068) 2020-03-11 14:21:41 +03:00
Victor Farazdagi
46eb228379 fixes data race in state.Slot (#5067)
* fixes data race in state/getters
2020-03-11 09:11:07 +00:00
Raul Jordan
8d3fc1ad3e Add in Slasher Metrics (#5060)
* added in slasher metrics
* Merge branch 'master' into slasher-metrics
* add in prom bolt metrics for slasher
* Merge branch 'slasher-metrics' of github.com:prysmaticlabs/prysm into slasher-metrics
* imports
* include all metrics
* no dup bolt collector
* Update slasher/detection/attestations/spanner.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* naming best practices for prom, thx Terence
* Merge branch 'slasher-metrics' of github.com:prysmaticlabs/prysm into slasher-metrics
2020-03-10 19:41:55 +00:00
Nishant Das
93195b762b Improve HTR of State (#5058)
* add cache
* Update beacon-chain/state/stateutil/blocks.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>
* Update beacon-chain/state/stateutil/blocks.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>
* Update beacon-chain/state/stateutil/hash_function.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>
* Merge branch 'master' into improveHTR
* add back string casting
* fix imports
2020-03-10 16:26:54 +00:00
Jim McDonald
f0abf0d7d5 Reduce frequency of 'eth1 client not syncing' messages (#5057) 2020-03-10 09:51:41 -05:00
Nishant Das
9d27449212 Discovery Fixes (#5050)
* connect to dv5 bootnodes

* fix test

* change polling period

* ignore

* Update beacon-chain/p2p/service.go

* Update beacon-chain/p2p/service_test.go

* fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-03-09 19:53:37 -07:00
Preston Van Loon
edb6590764 Build herumi's BLS from source (#5055)
* Build herumi from source. Working so far on linux_amd64 for compile, but tests fail to initialize the curve appropriately

* Add copts to go_default_library

* llvm toolchain, still WIP

* Fixes, make llvm a config flag

* fix gazelle resolution

* comment

* comment

* update herumi to the v0.9.4 version

* Apply @nisdas patch from https://github.com/herumi/bls-eth-go-binary/pull/5
2020-03-09 21:22:41 -05:00
Raul Jordan
e77cf724b8 Better Nil Check in Slasher (#5053)
* rem slasher proto
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* some nil checks in slasher
2020-03-09 21:21:39 +00:00
Ivan Martinez
b633dfe880 Change detection and updating in Slasher to per attestation (#5043)
* Change span updates to update multiple validators at once

* Change detection to perform on multiple validators at once

* Fix minspan issue

* Fix indices

* Fix test

* Remove logs

* Remove more logs

* Update slasher/detection/attestations/spanner_test.go

* Update slasher/detection/attestations/spanner_test.go

* Update slasher/detection/attestations/spanner_test.go

* Update slasher/detection/detect.go

* nil check

* fix ununsed import

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-03-09 13:14:19 -05:00
Ivan Martinez
8334aac111 Batch saving of attestations from stream for slasher (#5041)
* Batch saving of attestations from stream for slasher

* Progress on test

* Fixes

* Fix test

* Rename

* Modify logs and timing

* Change

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-03-09 12:49:40 -05:00
Preston Van Loon
4c1e2ba196 Add prysm.sh script (#5042)
* Add prysm.sh script

* Add dist to gitignore

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-09 12:19:53 -05:00
terence tsao
25c13663d2 Add hot state by slot retrival (#5052)
* Update replay conditions

* loadHotStateBySlot

* Tests and gaz

* Tests
2020-03-09 11:22:45 -05:00
Jim McDonald
0c3af32274 Use BeaconBlockHeader in place of BeaconBlock (#5049) 2020-03-09 21:08:30 +08:00
shayzluf
01cb01a8f2 On eviction test fix (#5046) 2020-03-09 01:35:39 -04:00
Raul Jordan
0c9e99e04a Aggregate Attestations Before Streaming to Slasher (#5029)
* rem slasher proto
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* aggregate before streaming
* Merge branch 'agg-idx-atts' of github.com:prysmaticlabs/prysm into agg-idx-atts
* collect atts and increase buffer size
* fix test for func
* Merge refs/heads/master into agg-idx-atts
* Update beacon-chain/rpc/beacon/attestations.go
* Merge refs/heads/master into agg-idx-atts
* naming
* Merge branch 'agg-idx-atts' of github.com:prysmaticlabs/prysm into agg-idx-atts
* Merge refs/heads/master into agg-idx-atts
* comment terence feedback
* Merge refs/heads/master into agg-idx-atts
* Merge refs/heads/master into agg-idx-atts
* Merge refs/heads/master into agg-idx-atts
* Merge refs/heads/master into agg-idx-atts
* Merge refs/heads/master into agg-idx-atts
* Merge refs/heads/master into agg-idx-atts
* Merge refs/heads/master into agg-idx-atts
* Merge refs/heads/master into agg-idx-atts
* Merge refs/heads/master into agg-idx-atts
* Merge refs/heads/master into agg-idx-atts
* Fix tests
2020-03-08 21:39:54 +00:00
Ivan Martinez
d4cd51f23e Change slasher cache to LRU cache (#5037)
* Change cache to LRU cache

* fixes

* REduce db usage

* Fix function name

* Merge issues

* Save on eviction

* Fixes

* Fix

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-03-08 17:11:59 -04:00
terence tsao
962fe8552d Compute state up to slot (#5035) 2020-03-08 21:41:24 +01:00
Raul Jordan
eddaea869b Prepare Slasher for Production (#5020)
* rem slasher proto
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* add a bit more better logging
* Empty db fix
* Improve logs
* Fix small issues in spanner, improvements
* Change costs back to 1 for now
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into cleanup-slasher
* Change the cache back to 0
* Cleanup
* Merge branch 'master' into cleanup-slasher
* lint
* added in better spans
* log
* rem spanner in super intensive operation
* Merge branch 'master' into cleanup-slasher
* add todo
* Merge branch 'cleanup-slasher' of github.com:prysmaticlabs/prysm into cleanup-slasher
* Merge branch 'master' into cleanup-slasher
* Apply suggestions from code review
* no logrus
* Merge branch 'master' into cleanup-slasher
* Merge branch 'cleanup-slasher' of https://github.com/prysmaticlabs/Prysm into cleanup-slasher
* Remove spammy logs
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into cleanup-slasher
* gaz
* Rename func
* Add back needed code
* Add todo
* Add span to cache func
2020-03-08 17:56:43 +00:00
Nishant Das
300d072456 Add Config Change for Validator (#5038)
* add config for validator
* gaz
* Merge refs/heads/master into configureValidator
* Merge refs/heads/master into configureValidator
2020-03-08 06:45:36 +00:00
Nishant Das
ac1c92e241 Add Prometheus Service for Slasher (#5039)
* add prometheus service
* Update slasher/node/node.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge refs/heads/master into addPromServiceSlasher
2020-03-08 06:35:37 +00:00
terence tsao
2452c7403b Load hot state by root (#5034)
* Add loadHotStateByRoot

* Touchup loadHotStateByRoot

* Tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-08 14:24:57 +08:00
Preston Van Loon
b97e22107c Update rbe_autoconf (#5036)
* Update rbe_autoconf
* Update timestamps
2020-03-07 21:18:16 +00:00
Preston Van Loon
98faf95943 Define -c opt for release builds (#5033)
* define -c opt for release builds
* Merge branch 'master' into c-opt
2020-03-07 05:50:26 +00:00
Preston Van Loon
af28862e94 Add sha256 to external dependency librdkafka (#5032)
* Add sha256 to external dependency librdkafka
2020-03-07 05:31:07 +00:00
Jim McDonald
b133eb6c4a Warn rather than fail on incorrect keystore password (#5024)
* Warn on failure to decrypt a keystore validator

* Update test

* Update tools

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-03-06 23:05:48 -06:00
Nishant Das
345ec1bf8c Fix Custom Delay Flag (#5026)
* fix flag
* Merge refs/heads/master into fixFlag
* Merge refs/heads/master into fixFlag
* Merge refs/heads/master into fixFlag
* Merge refs/heads/master into fixFlag
* Merge refs/heads/master into fixFlag
* fix config
* Merge branch 'fixFlag' of https://github.com/prysmaticlabs/geth-sharding into fixFlag
2020-03-07 03:52:40 +00:00
Nishant Das
d1fea430d6 change limit (#5027)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-06 17:26:08 -06:00
terence tsao
054b15bc45 Add SlotsPerArchivedPoint flag and a check (#5023)
* Flag

* Service

* Tests

* Tests and comments

* Lint

* Add to usages

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-06 17:06:01 -06:00
Preston Van Loon
6a2955d43c Update bazel.sh (#5028)
* Add google auth creds as environment variable for CI. Add a comment why this script is helpful
* Add google auth creds as environment variable for CI. Add a comment why this script is helpful
2020-03-06 17:43:06 +00:00
Jim McDonald
0ecd83afbb Avoid crash due to invalid index (#5025)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-03-06 09:38:43 -06:00
Nishant Das
069f2c5fb6 Asynchronous Dials To Peers (#5021)
* make dial non-blocking

* add sleep

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-06 22:57:47 +08:00
Raul Jordan
acb15a1f04 Report Validator Status in Duties (#5017)
* fix up status reporting
* Merge refs/heads/master into properly-report-status
* Merge refs/heads/master into properly-report-status
* Merge refs/heads/master into properly-report-status
2020-03-06 06:18:14 +00:00
Preston Van Loon
e2af70f692 Run buildifer, remove duplicated WORKSPACE entries (#5018)
* Buildifier, add release config
* Merge branch 'master' into bazel-stuff
* Merge refs/heads/master into bazel-stuff
* Merge refs/heads/master into bazel-stuff
* revert gnostic
* Set kafka for CI tests only
* add bazel.sh script
* set home
2020-03-06 04:42:27 +00:00
Raul Jordan
15b5ec89b2 Cross-compile OSX: Remove dropbox link, add sha256 check (#5019)
* Remove dropbox link, add sha256 check

* Use docker's S3 link instead of dropbox

* Update image sha, workspace

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-05 16:32:52 -06:00
Victor Farazdagi
b4aaa610a1 Fixes race condition at genesis (#5016)
* fixes race condition at genesis
2020-03-06 00:20:38 +03:00
Raul Jordan
6158a648cd Plugging in spanner db (#5009)
* rem slasher proto

* with cache

* delete old code

* moving to bytes.go fix traces

* moving to bytes.go fix traces

* raul feedback

* raul feedback

* begin

* add eviction test

* ivan feedback

* ivan feedback

* test is running

* some comment improvements

* test included for bytes and bool

* import

* cleanup

* tests pass

* fill in all fields in test

* gaz

* fix integration

* gaz + goimports

* fix service.go

* remove sleep

* cleanup

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-05 12:11:54 -06:00
prylabs-bulldozer[bot]
e2a6f5a6ea First cut at multi-arch cross compiling toolchain (#4945)
*  PRYSM-2849 first cut at multi-arch cross compiling toolchain.  currently supports arm64 and amd64 via docker cross compiler image
* picky linter
* some readme cleanup
* remove arm 8.2 revision for arm64 builds (cortex a72 is ARMv8.0-A)
remove arm32 toolchain from multiarch dockerfile
* remove extranous WORKSPACE entries
* add docker remote execution configs for amd64 and arm64
* add osx bazelrc configs
* working osx toolchain
* update readme
* cleanup for amd, arm and osx cross before beginning windows
* initial stab at mingw windows cross
* add docker target for windows_amd64 and update readme for cross-compiling
* little more cleanup for readability
* Check in generated RBE. Still tweaking config but linux amd64 -> linux amd64 on RBE works OK. Cross compile does not work properly in RBE yet.
* fix
* update image
* Making some progress
* delete artifacts
* Working build
* Add remote config
* remove some things I added to README
* Tidy
* Update readme
* remove 2 commented lines
* buildifer
* Merge pull request #1 from prysmaticlabs/cross-compile-with-suburbandad

Cross compile with suburbandad
* Merge branch 'master' into clang-cross-compile
* buildifier on generated stuff
* Merge branch 'master' into clang-cross-compile
* Merge branch 'master' into clang-cross-compile
* Merge branch 'master' into clang-cross-compile
2020-03-05 16:59:56 +00:00
Raul Jordan
aebc883a0d Methods to retrieves last saved state and block for stategen pkg (#5005)
* Added last saved block and state

* Genesis tests

* Gaz

* Added state tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-05 10:22:20 -06:00
Raul Jordan
f3dc113dba add multi-options (#5012) 2020-03-05 08:49:26 -06:00
prylabs-bulldozer[bot]
5961aaf588 Windows friendly stdin reads for passwords (#5010)
* cast os.stdin filehandle since on windows syscall.Stdin is not int
* import ordering
* Merge branch 'master' into prysm-5008-windows-stdin-not-int
2020-03-05 07:31:11 +00:00
prylabs-bulldozer[bot]
e635e5b205 Feature flag to gate prune state upon start up (#5011)
* Added feature flag to gate prune state upon start up
2020-03-05 06:24:59 +00:00
Raul Jordan
66991f0efe Spanner db (#4997)
* rem slasher proto

* with cache

* delete old code

* moving to bytes.go fix traces

* moving to bytes.go fix traces

* raul feedback

* raul feedback

* add eviction test

* ivan feedback

* ivan feedback

* some comment improvements

* test included for bytes and bool

* import

* cleanup

* tests pass

* fill in all fields in test

* gaz

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-03-04 23:35:14 -06:00
prylabs-bulldozer[bot]
e339b07ac7 Remove unused DB functions and proto from Slasher (#4996)
* Remove unused DB functions
* goimports
* Fix bug and improve tests
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into slasher-remove-old-db
2020-03-04 22:29:36 +00:00
prylabs-bulldozer[bot]
139f51efaa LRU cache for state gen (#5004)
* Add hot state cache
* Gaz
* Test
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into state-gen-lru-cache
* Merge refs/heads/master into state-gen-lru-cache
2020-03-04 22:09:21 +00:00
prylabs-bulldozer[bot]
a43a40c1c9 Create .bazelversion (#5003)
* Create .bazelversion
2020-03-04 21:49:22 +00:00
Raul Jordan
0bdd0dba67 Add warning if shell expansion characters make it in to the path (#5001) 2020-03-04 12:34:23 -06:00
Victor Farazdagi
239efe7410 init sync: adds blocks fetcher service (#4978)
* init sync: adds blocks fetcher service

* init-sync: rework ctx handling

* init-sync: fix long lines

* removes redundant method

* adds buffer to requests channel

* adds jaeger spans

* fixes overly long comment line
2020-03-04 20:19:09 +03:00
terence tsao
e5da756c47 Add error definitions for state gen pkg (#5000)
* Add error file
* Fmt
* Merge branch 'master' into error-file
2020-03-04 16:50:08 +00:00
terence tsao
a612557fe7 Add log file (#4999) 2020-03-04 10:30:52 -06:00
Raul Jordan
26582cbf2e Stub Slasher RPC Methods (#4995)
* rem slasher proto
* Remove unneeded protos
* Rework api proto
* Add back proto
* regen slashing proto
* Merge branch 'rem-rpc' of github.com:prysmaticlabs/prysm into rem-rpc
* Fix comments
* Merge branch 'rem-rpc' of https://github.com/prysmaticlabs/Prysm into rem-rpc
2020-03-03 22:09:35 +00:00
Raul Jordan
d68636bc7f Remove Deprecated Slasher Code (#4994)
* remove old code
* Clear out service from bazel
2020-03-03 19:40:09 +00:00
terence tsao
699e7efc61 Add epoch boundary root map (#4993)
* Add to struct

* Add implementations

* Tests
2020-03-03 13:07:34 -06:00
Ivan Martinez
ba6b8c9321 Update outdated spec function names and comments (#4992)
* Update outdated spec function names and comments
* VerifyMerkleBranch
* Remove error handle
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into slash-spec-refresh
* Merge branch 'master' into slash-spec-refresh
2020-03-03 18:29:41 +00:00
Ivan Martinez
cc5fc0af1a Plug-in double voting detection into detection service (#4960)
* Add double vote detection to spanner
* Add documentation
* Update slasher/detection/attestations/spanner.go
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into slasher-spanner-double
* Merge branch 'slasher-spanner-double' of https://github.com/0xKiwi/Prysm into slasher-spanner-double
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into slasher-spanner-double
* Gazelle
* Add double vote detection func
* Implement double voting detection
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into slasher-implement-double
* Merge branch 'master' into slasher-implement-double
* Merge branch 'slasher-implement-double' of https://github.com/0xKiwi/Prysm into slasher-implement-double
* Fix typo
* Remove filter, replace with slot + committee index
* Change bloom filter to 2 sig bytes
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into slasher-change-filter
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into slasher-implement-double
* Merge branch 'slasher-change-filter' of https://github.com/0xKiwi/Prysm into slasher-implement-double
* Change detection to use prefix
* Fix runtime
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into slasher-implement-double
* Fix bug and comments
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into slasher-implement-double
* Fix flaky test
* Merge branch 'master' into slasher-implement-double
* Improve logs
* Merge branch 'slasher-implement-double' of https://github.com/0xKiwi/Prysm into slasher-implement-double
* Add ok check
* Fix test
* Merge branch 'master' into slasher-implement-double
2020-03-03 18:08:21 +00:00
Nishant Das
0093218e41 Add Noise Support To Prysm (#4991)
* add dep
* add feature config
* Merge branch 'master' into addNoiseSupport
* gaz and victor's review
* Merge branch 'addNoiseSupport' of https://github.com/prysmaticlabs/geth-sharding into addNoiseSupport
2020-03-03 17:45:51 +00:00
tzapu
c09ae21ab0 show full public key in metrics (#4988)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-03-03 10:36:49 -06:00
Jim McDonald
4c43616647 Add ProtecingKeymanager interface and calls (#4982)
* Add ProtectedKeymanager interface and calls

* Rename interface

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-03-03 10:21:58 -06:00
shayzluf
59575bcac9 fuzz core/blocks package (#4907)
* fuzz core/blocks package

* gaz goimports

* fix test

* terence feedback

* terence feedback

* add error to domain. halfway through

* adding error to domain

* goimports

* added error handling to test

* error instead of continue

* terence and nishant feedback

* domain error handling

* domain error handling

* handle nil validator in ReadOnlyValidator creation

* goinmports

* [4]byte domain type

* [4]byte domain type

* [4]byte domain type fix tests

* fix tests

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-03-03 19:02:14 +05:30
Ivan Martinez
703ce63c12 Change span representation to a struct in Slasher (#4981)
* Remove filter, replace with slot + committee index

* Change bloom filter to 2 sig bytes

* Fix comment

* Fix line
2020-03-03 15:28:23 +05:30
Ivan Martinez
69845cad77 Fix stalling bug in slashing pool (#4985)
* Fix bug in slashing pool

* Remove unneeded logs

* remove line

* Remove val

* Fix tests

* Add regression test

* Add nother regresstion test
2020-03-03 00:52:35 -05:00
terence tsao
a07e604eea Better logs for forking (#4966)
* Move `updateHead` to ReceiveAttestationNoPubsub and better forking mesasges

* Typo

* Import

* f

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-02 09:51:26 -08:00
Jim McDonald
044d72064f Pre-allocate slices when reporting validator performance (#4979) 2020-03-02 23:00:23 +08:00
Jim McDonald
5cb51263b0 Fix crash when reporting validator metrics (#4968)
* Fix crash when reporting validator metrics
* Merge branch 'master' into validator-reporting-crash
* Merge branch 'master' into validator-reporting-crash
2020-03-02 08:51:07 +00:00
terence tsao
d9d4a9954e Archived point index DB methods (#4977) 2020-03-02 08:55:38 +01:00
Nishant Das
3989b65667 Add Flag for Checking HeadState (#4974)
* gate feature
* imports
* add flag
* Merge branch 'master' into gateFeature
2020-03-02 06:06:20 +00:00
terence tsao
9fe2cdd5ca State summary DB methods (#4971)
* Define proto
* Regen
* Delete slasher.pb.go
* Gaz
* Merge branch 'state-summary-proto' of https://github.com/prysmaticlabs/prysm into state-summary-proto
* Revert "Delete slasher.pb.go"

This reverts commit 19bfa21cd3.
* Add state_summary.go
* Test
* Gaz
* Interaces
* pass through
* Merge refs/heads/master into state-summary-db
* Merge refs/heads/master into state-summary-db
2020-03-02 04:27:55 +00:00
terence tsao
cb163d8910 Define state summary proto (#4967)
* Define proto

* Regen

* Delete slasher.pb.go

* Gaz

* Revert "Delete slasher.pb.go"

This reverts commit 19bfa21cd3.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-01 23:10:52 -05:00
Nishant Das
cd6e06f01e add helpers (#4972) 2020-03-01 16:22:49 +01:00
Preston Van Loon
af5cc31565 Use correct image name for validator debug image (#4963)
* Use correct image name for validator debug image
* Merge refs/heads/master into fix-validator-debug-image
2020-02-28 01:22:32 +00:00
terence tsao
5a5cdc1b02 Removed (#4962) 2020-02-27 17:06:25 -08:00
Ivan Martinez
31b1e6a7a8 Add double vote detection to spanner (#4954)
* Add double vote detection to spanner

* Add documentation

* Update slasher/detection/attestations/spanner.go

* Gazelle

* Fix filter output
2020-02-27 17:21:05 -05:00
Preston Van Loon
05a5bad476 Migrate SubmitAggregateAndProof (#4951)
* Remove unused services, mark everything as deprecated, regen pb.go
* remove some code from cluster pk manager, gazelle
* goimports
* remove mocks
* Update WORKSPACE, deprecate old method, stub new method
* Move implementation to ethereumapis definition
* gofmt
* Add TODO for #4952
* Merge branch 'master' into migrate-submitaggregateandproof
* Update validator client to use new submit aggregate and proof method
* Merge branch 'migrate-submitaggregateandproof' of github.com:prysmaticlabs/prysm into migrate-submitaggregateandproof
* gaz
* rename
* rename
* Merge refs/heads/master into migrate-submitaggregateandproof
* Merge refs/heads/master into migrate-submitaggregateandproof
* Merge refs/heads/master into migrate-submitaggregateandproof
* Merge refs/heads/master into migrate-submitaggregateandproof
* Merge refs/heads/master into migrate-submitaggregateandproof
* fix tests
* Merge branch 'migrate-submitaggregateandproof' of github.com:prysmaticlabs/prysm into migrate-submitaggregateandproof
2020-02-27 20:23:35 +00:00
terence tsao
2fef9d3e5e Move blockchain service metrics package (#4959)
* Moved metrics package
2020-02-27 19:38:22 +00:00
Raul Jordan
14b3181e67 Plug-In Attester Slashing Detection Into Slasher Runtime (#4937)
* more spanner additions

* implement iface

* begin implement

* wrapped up spanner functions

* rem interface

* added in necessary comments

* comments on enums

* begin adding tests

* plug in surround vote detection

* saved indexed db implementation

* finally plugin slashing for historical data

* Small fixes

* add in all gazelle

* save incoming new functions

* resolve todo

* fix broken test channel item

* tests passing when fixing certain arguments and setups

* Add comment and change unimplemented

* find surround

* added in gazelle

* gazz

* feedback from shay

* fixed up naming

* Update

* Add tests for detectSurroundVotes

* Remove logs

* Fix slasher test

* formatting

* Remove unneeded condition

* Test indices better

* fixing broken build

* pass tests

* skip tests

* imports

* Update slasher/detection/attestations/attestations_test.go

* Update slasher/beaconclient/historical_data_retrieval_test.go

* Address comments

* Rename function

* Add comment for future optimization

* Fix comment

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
2020-02-27 12:22:39 -05:00
Ivan Martinez
e7b94123ce Add interface for spanner and MockSpanner (#4956)
* Add interface for spanner

* Add MockSpanner for future testing

* Add comment

* gazelle
2020-02-26 22:48:02 -06:00
Ivan Martinez
76aad0f444 Add simple bloom filter implementation for double vote detection (#4948)
* Add simple bloom filter implementation for detecting similarity of 1 key

* Change hash to keccak

* Fix receiver name

* Fix bug

* Fix comments and organize test

* Add comment detailing hash functions

* Add bloom to test names
2020-02-26 19:37:36 -05:00
terence tsao
2c1c41d1d6 Move stategen package under /state (#4950)
* Move state gen to state
* Merge branch 'master' into move-state-gen
* Merge refs/heads/master into move-state-gen
2020-02-26 22:09:14 +00:00
Preston Van Loon
921a44d9fd Clean up unused / deprecated protobuf definitions (#4949)
* Remove unused services, mark everything as deprecated, regen pb.go
* remove some code from cluster pk manager, gazelle
* goimports
* remove mocks
* Merge branch 'master' into remove-deprecated-proto-defs
2020-02-26 21:15:36 +00:00
Raul Jordan
22bbed0059 Stream Indexed Attestations RPC Implementation (#4941)
* stream indexed attestations impl
* mock regen
* test for stream indexed
* atts test
* no bls
* gaz
* Merge refs/heads/master into implement-stream-indexed
* use feed for atts instead
* remove unused imports
* Merge refs/heads/master into implement-stream-indexed
* fix tests in beacon
* properly use pointers
* imports
* import
2020-02-26 20:14:22 +00:00
terence tsao
b1231f3ddf Replay blocks and generate state without sig verification (#4943)
* New file

* Add transition_stategen.go

* Update ProcessBlockForStateRoot

* Feature flags

* Fixed tests

* Gaz

* Make them private

* E2e flags

* Gazelle

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-02-26 10:40:33 -06:00
Ivan Martinez
c2b30cf801 Prepare spanner for double vote detection and fix a few bugs (#4940)
* Rename vars for clarity

* Change spanner to take target epoch as key

* Fix tests, add multiple val test

* Fixes

* Change the spanner to take in att on detect

* Add back proto diagram tests

* Remove unneeded comments
2020-02-25 21:35:34 -06:00
Preston Van Loon
b647ca5dd2 Release --initial-sync-cache-state (#4938)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-02-24 19:20:00 -06:00
Preston Van Loon
c0f1a1d674 Validator: cache domain data calls (#4914)
* Use a domain data cache to reduce the number of calls per epoch

* fix fakevalidator

* Refactor to use a feature flag, use proto.clone, move interceptor to its own file

* gofmt

* fix comment

* tune cache slightly

* log if error on domain data

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-02-24 13:02:45 -08:00
Preston Van Loon
855f5d2986 Add a configurable flag for gRPC retries (#4926)
* Add a configurable flag for gRPC retries
* Merge refs/heads/master into configurable-retry
* Merge refs/heads/master into configurable-retry
* Merge refs/heads/master into configurable-retry
* Merge refs/heads/master into configurable-retry
* Merge refs/heads/master into configurable-retry
* add in flag to usage
2020-02-24 18:00:22 +00:00
Jim McDonald
5f0ed8388e Use --deposit-contract with default value (#4925)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-02-24 11:24:05 -06:00
Jim McDonald
a951c4f6ab Add Ethereum 1 block->timestamp cache (#4924)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-02-24 10:53:35 -06:00
terence tsao
0470d37072 Consider missing validator count for performance metric (#4928)
* Consider missing validator count
* Use validator count reported
* Merge branch 'master' into missing-validators
* Merge refs/heads/master into missing-validators
2020-02-24 16:28:22 +00:00
terence tsao
15b649d760 Fix aggregated attestation pool grows large in size (#4932)
* Add metrics

* Use it

* Use it

* Fixed exp time and tests

* Update on save too

* Expose getters

* One epoch purge time

* Fixed a timing issue

* Clean up

* Gazelle

* Interface

* Prune every epoch

* Aggregate twice per slot

* Revert attsToBeAggregated

* Delete expired atts

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-02-24 10:18:34 -06:00
terence tsao
2e56a59473 attestations in pool count metrics (#4930)
* Add metrics
* Use it
* Use it
* Expose getters
2020-02-23 22:30:52 +00:00
Raul Jordan
6fe86a3b30 Define an Efficient Spanner Struct Implementation for Slasher (#4920)
* more spanner additions

* implement iface

* begin implement

* wrapped up spanner functions

* rem interface

* added in necessary comments

* comments on enums

* begin adding tests

* test for detection

* add all detection tests

* moar tests

* tests for deleting pass

* dd test for update spans

* tests for updating

* include tracing utils

* gaz

* add mutexes

* ivan feedback
2020-02-22 08:57:24 -06:00
Nishant Das
83945ca54b Shift Stateutils to State Package (#4921)
* shift over
* new changes
* imports
* Merge branch 'master' into shiftUtils
2020-02-21 16:52:21 +00:00
Nishant Das
47bb927029 Fix Fork Copying (#4922)
* add fix and reg test

* goimports
2020-02-21 08:49:42 -05:00
garyschulte
597b21c40a fix missing metrics label on attetation fail (#4917)
Co-authored-by: garyschulteog <30323939+garyschulteog@users.noreply.github.com>
2020-02-20 21:28:55 -06:00
Raul Jordan
39aa791dcc Add Slashings Into Blocks from Pool (#4902)
* tests pass

* fix broken test

* addressed feedback

* Update beacon-chain/rpc/validator/proposer_test.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

* Update beacon-chain/rpc/validator/proposer_test.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: shayzluf <thezluf@gmail.com>
2020-02-20 15:10:51 -06:00
Ivan Martinez
90ed37a655 Cleanup detection code (#4915) 2020-02-20 08:56:37 -06:00
Raul Jordan
d143187b7e Request All Indexed Attestations Since Genesis in Slasher on Startup (#4894)
* include fixes

* rev

* logrus

* tests for query sync status and chain head

* begin tests for indexed atts

* test passing for requesting historical atts

* Update slasher/beaconclient/chain_data_test.go

* Update slasher/beaconclient/historical_data_retrieval.go

* lint

* fixed up wanted vs receied

* fix mock

* gazelle

* fix broken build

* tests pass

* dep

* gaz

* add dep

* tests pass

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-02-19 16:26:14 -06:00
Preston Van Loon
3735e6b8af Add a clarifying comment from #4909 (#4911)
* Add a clarifying comment from #4909
* Merge refs/heads/master into clarify
2020-02-19 21:10:56 +00:00
Jim McDonald
deb76f1c15 Fix double period in span name (#4910) 2020-02-19 14:52:44 -06:00
Jim McDonald
6baffd4ccb Infostream (#4760)
* Add validators stream

* Ignore unknown keys rather than error on them

* Reduce accesses to common structures

* Ensure correct information returned for deposited validators

* Short-term cache for remote deposit data

* Name epoch duration for clarity

* Break out duplicated logic in to a single function

* Add capacities for slices and maps where appropriate

* Break out functions; add tests

* Allow stream errors not related to context

Co-authored-by: Nishant Das <nish1993@hotmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-02-19 11:09:23 -06:00
terence tsao
731cc0bd44 Handing pending atts if they dont have the state (#4904) 2020-02-19 06:46:30 -08:00
Nishant Das
641ad51dd4 check db for justified state (#4905) 2020-02-19 22:18:44 +08:00
Nishant Das
8e55c81bd5 Delete States More Efficiently (#4909)
* only sync at end of method

* chunk roots

* very fast iteration

* delete correctly
2020-02-19 19:36:37 +08:00
Preston Van Loon
f737267e54 gRPC retry requests (#4908)
* gRPC retry requests

* with 5 retries default
2020-02-19 18:42:17 +08:00
Nishant Das
44856f9500 Add Unsafe Sync (#4906)
* add unsafe flag

* imports

* Use finalized epoch

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-02-19 00:25:07 -08:00
Nishant Das
4389e9d3c9 Add mempool feature flag (#4824) (#4903)
* Add mempool feature flag

* gate put too

* fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-02-19 14:29:49 +08:00
Nishant Das
655f57e3f2 Bound Initial Sync Cache Size (#4844)
* bound initial sync

* fix lint

* Revert "Better block attestation inclusion (#4838)"

This reverts commit 090d9627fe.

* add memory pool

* more fixes

* revert changes

* add hack

* revert hack

* push halving

* bring back hack

* increase cache size

* more fixes

* more changes

* new fixes

* add test

* add reverse test

* more tests and clean up

* add helper

* more cleanup and tests

* fix test

* remove code

* set gc percent flag

* lint

* lint

* Fix comment formatting

* Fix some formatting

* inverse if statement

* remove debug log

* Apply suggestions from code review

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

* Update beacon-chain/state/getters.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

* Update beacon-chain/db/kv/state.go

* integrate state generator

* gaz

* fixes

* terence's review

* reduce bound further

* fix test

* separate into new files

* gaz

* mod build file

* add test

* revert changes

* fix test

* Update beacon-chain/core/helpers/slot_epoch.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* handle edge case

* add back test

* fix test again

* handle edge case

* Update beacon-chain/blockchain/init_sync_process_block.go

* Update beacon-chain/blockchain/init_sync_process_block.go

* Update beacon-chain/stategen/service_test.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* Update beacon-chain/blockchain/init_sync_process_block.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* Update beacon-chain/stategen/service.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* Update beacon-chain/stategen/service.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* raul's review

* raul's review

* fix refs

* terence's review

* one more fix

* Update beacon-chain/blockchain/init_sync_process_block.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-02-18 15:10:54 -06:00
terence tsao
c0d4cabdb7 Check seeds (#4901) 2020-02-18 12:05:36 -06:00
terence tsao
0e37b4926a Export LoadBlocks and ReplayBlocks (#4898)
* Define StateGenerator

* Gaz

* Delete interface.go

* Update BUILD.bazel

Co-authored-by: Nishant Das <nish1993@hotmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-02-18 13:54:56 +08:00
shayzluf
25308ef9fa fuzzing core/state package without skip slot cache (#4883)
* fuzzing core/state package

* named error msg

* err comment

* terence feedback

* preston feedback

* preston feedback

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-02-18 10:38:15 +05:30
Preston Van Loon
40afef8b9e Only set gc percent if the flag is set (#4899)
* only set gc percent if the flag is set
2020-02-18 01:37:35 +00:00
Nishant Das
c7d0ced5d1 Utilise a Flag to Toggle With the GC (#4897)
* set flag
* Merge refs/heads/master into setGC
2020-02-18 00:37:37 +00:00
Nishant Das
3d12322103 Use Memory Pool for Randao Mixes (#4896)
* add mem pool

* use mem pool

* Update shared/memorypool/memorypool.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Update shared/memorypool/memorypool.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-02-18 08:25:39 +08:00
Raul Jordan
b4881e3cd5 Save Attestations In Initial Sync if Archive Enabled (#4895)
* receive block enable archive
* add to initial sync func
* Merge branch 'master' into archive-save-atts
2020-02-17 22:42:21 +00:00
Preston Van Loon
d1eaa8e09e Define debug images for Prysm beacon chain and validator binaries (#4893)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-02-17 14:13:34 -08:00
Raul Jordan
5db8c5ad0c Implement ListIndexedAttestations Endpoint in Prysm (#4892)
* update patch and workspace

* stub methods

* implementation of indexed attestations list

* include latest ethereumapis

* update request type

* compute committee pure function

* use compute committee helper

* add test into list indexed attestations

* regenerate mock

* imports and out of range check

* test passing for archived epoch

* add comment

* comment

* better comment on func

* throw in continue instead
2020-02-17 15:57:13 -06:00
terence tsao
d7db8b1f5d Replay block same slots different root edge case (#4889) 2020-02-17 12:33:00 -07:00
terence tsao
6b8ec26c56 Handle nil head block in cache (#4888)
* Nil check

* Fixed tests

* Covered rest of the codebase

* Race tests

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-02-17 12:21:42 -06:00
terence tsao
9b2aa66667 Fix reply block edge cases and tests (#4881)
* Covered 2 edge cases and tests

* Consider process epoch
2020-02-17 13:10:23 +08:00
terence tsao
b9c140c17d Hot/cold state management: Replay blocks and gen state (#4877)
* Starting stategen

* Replay implementations

* Replay tests

* Gazelle

* Fixed tests

* Dont have to verify sig
2020-02-17 07:28:20 +08:00
Aranha
8885d715f2 fixed panic: runtime error: integer divide by zero #4777 (#4823) 2020-02-16 15:13:03 -07:00
Preston Van Loon
0a2763b380 Check attestation bitlist length in aggregation to prevent panic (#4876)
* Check attestation bitlist length in aggregation to prevent panic

* Add case for overlap too

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-02-16 10:18:48 -07:00
Raul Jordan
3fcb4e8a12 Update README for Patching Ethereum APIs (#4871)
* edit readme
* Merge branch 'master' into readme-patch
* Merge branch 'master' into readme-patch
2020-02-16 17:07:06 +00:00
Preston Van Loon
db68c8a57b Enable attestation cache flag by default, deprecate feature flag (#4873)
* Enable attester flag by default
2020-02-16 01:07:14 +00:00
SuburbanDad
7899dc115e prevent additional array OOB errors for validator balances (#4872)
Co-authored-by: garyschulte <garyschulte@gmail.com>
2020-02-15 15:47:45 -07:00
terence tsao
456ac5f9a3 Better head object coupling for chain service (#4869)
* Done
* Fixed lock
* Fixed all the tests
* Comments
* Fixed more tests
* Merge branch 'master' into better-head-obj
* Fixed more tests
* Merge branch 'better-head-obj' of git+ssh://github.com/prysmaticlabs/prysm into better-head-obj
* Prestons feedback & fixed test
* Nishant's feedback
* Participation edge case
* Gaz
* Merge branch 'master' into better-head-obj
* Merge branch 'master' of git+ssh://github.com/prysmaticlabs/prysm into better-head-obj
* Raul's feedback
* Merge branch 'better-head-obj' of git+ssh://github.com/prysmaticlabs/prysm into better-head-obj
2020-02-15 18:57:49 +00:00
terence tsao
92a91476ef Better log (#4870) 2020-02-15 04:09:26 -08:00
Raul Jordan
868c8f5dd4 Detection Service Creation (#4867)
* visibility added

* register in node

* fixed up imports

* include detection listeners for feed

* subscribe to blocks and todos

* tests passing

* todos

* pkg comment
2020-02-14 13:03:25 -06:00
Raul Jordan
38fed735b2 Send Slashing Objects to Beacon Node via RPC (#4866)
* submit slashing objects

* tests complete
2020-02-14 11:11:14 -07:00
terence tsao
4a446329b2 Prevent balance goes out of bound (#4865)
* Prevent balance goes out of bound
* Prevent balance goes out of bound
* Merge branch 'master' into fix-balance
2020-02-14 16:59:20 +00:00
Ivan Martinez
6b40fa01ec Add detection package for slashing detection functions (#4861)
* Move detection to its own package

* Fix renames

* More fixes

* Revert "Fix renames"

This reverts commit 3200f89a1b.

* Fix

* Fix renames again

* Fix another rename

* Fix comment

* unused

* add comment

* gazelle

* Add spans

* Unexport helper functions

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-02-14 10:46:55 -06:00
Nishant Das
214121b0ab Dont Initialize Zeroed Out State (#4863)
* dont initialize map with empty registry
* check zeroed pointer
* check zeroed pointer
* imports
* gaz
* better check
* gaz
* fix it finally
* finally fix it
* gaz
2020-02-14 12:19:50 +00:00
Raul Jordan
b99779fe94 Implementing Slasher Node Runtime (#4856)
* include slasher node
* slasher node runtime added
* added in register for beacon client
* streaming blocks fixed up
* all subs working
* gazelle
* handle errors
* Merge branch 'master' into slasher-node
* Update slasher/node/BUILD.bazel
* x up slasher test
* Merge refs/heads/master into slasher-node
* Merge refs/heads/master into slasher-node
* add in force clear into usage
* Merge refs/heads/master into slasher-node
* usage
* Merge refs/heads/master into slasher-node
* Fix streamblocks test
* Merge refs/heads/master into slasher-node
* Fix docker image compile
* Merge branch 'slasher-node' of https://github.com/prysmaticlabs/Prysm into slasher-node
2020-02-14 07:09:54 +00:00
Nishant Das
b263efefeb Copy Checkpoint Root Properly (#4862)
* change to custom hashing
* Merge branch 'master' into minorOpt
* goimports
* Merge branch 'minorOpt' of https://github.com/prysmaticlabs/geth-sharding into minorOpt
* gaz
* pad to 32 bytes
* one more case
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into minorOpt
* one more case
* more cases
* some more cases
* do it better
2020-02-14 06:35:16 +00:00
Nishant Das
ecfd7bdfa1 Change to Custom Hashing for BlockHeaders (#4860)
* change to custom hashing
* Merge branch 'master' into minorOpt
* goimports
* Merge branch 'minorOpt' of https://github.com/prysmaticlabs/geth-sharding into minorOpt
* gaz
* pad to 32 bytes
2020-02-14 05:19:20 +00:00
Raul Jordan
549b0f66fa Include Slashing Submission Endpoints + Slashing Pool in Beacon Node (#4858)
* add to workspace

* impl

* include tests for func

* fix broken build

* test passing, found 2 bugs

* add errors package

* added in mockgen

* we check for insertion into the pool based on attester slashings

* test passing

* proper test

* Update beacon-chain/rpc/beacon/slashings.go

* Update beacon-chain/rpc/beacon/slashings_test.go
2020-02-13 22:20:45 -06:00
garyschulteog
27ec40f269 Remove remaining instances of proto.clone() (#4806)
* prysm-4757 remove proto.Clone() in favor of existing getters.Copy* methods
* prysm-4757 added a bunch of copy methods, and broke some tests
* squash commits
 fix tests and getter implementations
 remove usage of CopySignedBeaconBlock from ReceiveBlockNoVerify
* correctly copy Deposit proof and remove proto.clone() again
* Merge branch 'master' into prysm-4757-no-proto-clone
* Merge branch 'master' into prysm-4757-no-proto-clone
* Fix for comments, inline possible function calls
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into prysm-4757-no-proto-clone
* Merge branch 'master' into prysm-4757-no-proto-clone
* updated with feedback from review
* Merge branch 'master' into prysm-4757-no-proto-clone
* Merge branch 'master' into prysm-4757-no-proto-clone
* Merge branch 'master' into prysm-4757-no-proto-clone
2020-02-14 01:03:51 +00:00
terence tsao
bb60b2f523 Add balances to voting summary log (#4857) 2020-02-13 14:52:35 -08:00
Preston Van Loon
4072eb711f Beacon State: More consistent nil return for state (#4854)
* More consistent nil return for state
* Merge refs/heads/master into nil-state
* Add a check for encode(nil)
* Merge branch 'nil-state' of github.com:prysmaticlabs/prysm into nil-state
* fix test, thanks @rauljordan
* fix tests
* gofmt
2020-02-13 20:34:50 +00:00
Ivan Martinez
2473680759 Add spans to Slasher DB functions (#4855)
* Add interface and move slashing types to /types package

* Add spans for all DB functions

* Fix packages

* Fix func call
2020-02-13 13:51:30 -06:00
Ivan Martinez
c44a30672e Change slasher DB structure to mirror beacon-chains (#4848)
* Add interface and move slashing types to /types package

* WIP restructure to match beacon chain DB

* Fix build

* Fix comment

* Fix comments

* fix comments for sure

* Use wrapper function for evict

* Remove unused

* Update slasher/db/kv/kv.go

* Update slasher/db/testing/BUILD.bazel

* Update slasher/db/types/BUILD.bazel

* Update slasher/db/types/types.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-02-13 10:19:46 -06:00
Nishant Das
db21f98053 Change Positionining of Warning Log (#4850)
* change logging

* change positioning of log

* fix status test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-02-13 00:39:15 -08:00
Nishant Das
b7adf55336 Revert "Check HeadState First (#4830)" (#4851)
This reverts commit 601f93a0a1.
2020-02-12 23:33:45 -08:00
Preston Van Loon
f06dfd6108 Secure lock when accessing the map only (#4849)
* Secure lock when accessing the map

* wrong lock

* Remove some deadlocks
2020-02-12 20:33:14 -06:00
Preston Van Loon
bb4c8ba83e Create backups without freelist. (#4847)
* Create backups without freelist. This is a bit slower, but more accurate
* Merge refs/heads/master into better-backups
2020-02-12 22:01:07 +00:00
terence tsao
16fef1c658 Better attesting summary reporting (#4845) 2020-02-12 13:38:19 -08:00
Preston Van Loon
090d9627fe Better block attestation inclusion (#4838)
* Fill blocks with unaggregated atts when possible, don't delete from pool until the block is submitted

* Add back invalid atts removal

* Don't delete attestations unless they can be aggregated

* comment and test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-02-12 11:54:07 -08:00
Nishant Das
c7698cda1c Add Pending Attestation Lock fix (#4840)
* add pending lock fix

* use easier locking
2020-02-12 08:58:40 -08:00
Nishant Das
a11f1804a2 lock fix (#4839) 2020-02-11 23:33:15 -08:00
Preston Van Loon
0882908d2c Improve attestation cache check from O(n) to O(1) access time (#4837)
* use O(1) method to access the blocks
* Merge branch 'master' into better-has-aggregated-attestation
2020-02-11 21:57:42 +00:00
Raul Jordan
a7325315a8 Include Beacon Client Package in Slasher (#4835)
* begin beacon client

* adding in the proper receivers

* include all parts of the beacon client

* all comments included

* visibility and package comment
2020-02-11 15:35:31 -06:00
Raul Jordan
c3785e03ba Add Lock to Processing Pending Attestations (#4833)
* add lock to pending atts
* Merge branch 'master' into add-att-lock
* changed to rlock
* broken build
* Merge refs/heads/master into add-att-lock
* refactor to prevent deadlock
2020-02-11 20:01:33 +00:00
Raul Jordan
297247d915 Add Paginated Attestation Pool to Prysm (#4827)
* added pagination to atts

* tests pass for atts

* add mock

* fix

* add to val

* fix

* add in proper mock

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-02-11 13:18:30 -06:00
Nishant Das
601f93a0a1 Check HeadState First (#4830)
* easy optimization
* Merge refs/heads/master into easyOptimization
* cache miss
2020-02-11 16:22:51 +00:00
Nishant Das
8c90e38770 Fix Goodbye RPC handler (#4831)
* fix goodbye messages

* fix test

* fix test
2020-02-11 09:08:01 -06:00
Preston Van Loon
661e48f549 Revert state copy PR #4811 (#4825)
* Revert "Add mempool feature flag (#4824)"

This reverts commit b3f2a330dc.
* Revert "Optimize Copying of Fields (#4811)"

This reverts commit 4f654d30ac.
2020-02-11 01:47:31 +00:00
Preston Van Loon
b3f2a330dc Add mempool feature flag (#4824)
* Add mempool feature flag

* gate put too

* fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-02-10 16:20:48 -06:00
Preston Van Loon
5c14cd64c5 nil check (#4822)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-02-10 14:01:37 -08:00
terence tsao
56fcca69d7 Optimize Hasblock (#4821)
* Expose node

* Comments

* Extra line

* Add has block

* Test

* Usages

* Fixed tests
2020-02-10 15:48:28 -06:00
shayzluf
02b6d7706f Slasher committees cache (#4812)
* add committees cache
* committees cache usage
* fix test
* fix log
* goimports
* Merge branch 'master' of github.com:prysmaticlabs/prysm into slasher_committees_cache

# Conflicts:
#	slasher/service/data_update.go
* fix imports
* fix comment
* fix comment
* Merge refs/heads/master into slasher_committees_cache
* Merge refs/heads/master into slasher_committees_cache
* Update slasher/cache/BUILD.bazel

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge refs/heads/master into slasher_committees_cache
* Merge refs/heads/master into slasher_committees_cache
* Merge refs/heads/master into slasher_committees_cache
* added in the service context
* baz
* Merge refs/heads/master into slasher_committees_cache
* Merge refs/heads/master into slasher_committees_cache
2020-02-10 20:09:14 +00:00
Preston Van Loon
0ed8246e28 Release flag to aggregate attestations in fork choice. (#4820)
* Release flag to aggregate attestations in fork choice.
* Merge refs/heads/master into aggregate-atts-in-fc
* fix test
* Merge branch 'aggregate-atts-in-fc' of github.com:prysmaticlabs/prysm into aggregate-atts-in-fc
* Merge refs/heads/master into aggregate-atts-in-fc
2020-02-10 19:57:30 +00:00
Jim McDonald
dfe52e1752 Add command to display private keys from keystore (#4793)
* Add command to display private keys from keystore

* Update output format

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-02-10 13:47:54 -06:00
terence tsao
52524d5acc Expose fork choice node (#4819)
* Expose node

* Comments

* Extra line

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-02-10 13:09:12 -06:00
Preston Van Loon
4598344918 Clear initial sync state caches after round robin sync (#4817)
* Clear initial sync state caches after round robin sync

* fix test mock

* lint

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-02-10 10:39:18 -08:00
terence tsao
1a5c5153be Increase size to 10 (#4818) 2020-02-10 10:14:31 -08:00
terence tsao
6c00f5fff7 Update attester wait time (#4791)
* Update attester submit strategy

* Tests

* Gaz

* Fixed rest of the tests

* Updated design to use feed

* Use roughtime for Now

* gaz

* Gaz

* Send block processed after fork choice

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-02-10 10:59:55 -06:00
Nishant Das
4f654d30ac Optimize Copying of Fields (#4811)
* add new changes

* memory pool

* add test

* final optimization

* preston's review
2020-02-10 23:05:58 +08:00
Ivan Martinez
18fbdd53b9 Slasher proto and function renames (#4797)
* Rename elements for clarity
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into slasher-renames
* Fix test
* Rename more functions
* Cleanup
* Fix logs
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into slasher-renames
* Reorganize and clean up logs
* Address comments
* Add comments
2020-02-10 05:57:20 +00:00
terence tsao
5be4fee810 Removed spans for fork choice helpers (#4808)
* Removed spans
2020-02-09 23:32:08 +00:00
terence tsao
8a02003d4b Feature flag to disable head update on attestation basis (#4802) 2020-02-09 11:44:17 -08:00
terence tsao
bdcd06a708 Handle head state for init sync cache state (#4800)
* Don't save nil head state
* Update head
* Don't update head on new att
* Handle initial sync state in DB can be nil
* Don't update head if it's nil
* Merge branch 'master' of git+ssh://github.com/prysmaticlabs/prysm into handle-init-sync-cache-state
* Merge refs/heads/master into handle-init-sync-cache-state
* Update beacon-chain/blockchain/head.go
2020-02-09 16:12:06 +00:00
Preston Van Loon
7e0d0502aa Prevent panic on wrong interface conversion (#4803)
* Prevent panic on wrong interface conversion
* remove import
2020-02-09 08:41:50 +00:00
terence tsao
f14ff34797 Delete block attestations from the pool (#4798) 2020-02-08 21:30:45 -08:00
terence tsao
16a0c9f463 Don't save nil head state (#4799)
* Don't save nil head state
* Update head
2020-02-09 05:08:21 +00:00
Preston Van Loon
70cb06d50f Report unhealthy if we think we are out of sync (#4796)
* Report unhealthy if we think we are out of sync
* gofmt
* Merge refs/heads/master into out-of-sync-unhealthy
2020-02-08 18:50:14 +00:00
Jim McDonald
0725e2dba7 Downgrade log entry (#4795) 2020-02-08 10:05:13 -08:00
terence tsao
031b51e294 Update head on per attestation and minor refactor clean ups (#4786)
* head.go
* Tests
* Tests
* Merge branch 'master' into call-head
* Merge refs/heads/master into call-head
* Merge refs/heads/master into call-head
* Merge refs/heads/master into call-head
* Merge branch 'master' into call-head
* Merge branch 'master' of git+ssh://github.com/prysmaticlabs/prysm into call-head
* Conflict
2020-02-08 02:05:43 +00:00
Preston Van Loon
015c8c4cd2 Use helper to aggregate attestations in pool (#4794)
* Aggregate attestations in pool
* test
* clarify test
* fix test
2020-02-08 01:03:50 +00:00
Nishant Das
efd27c7b2b Fix Dynamic Topic Subscriptions (#4767)
* check for sync status

* check for chainstart

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-02-08 06:49:17 +08:00
Nishant Das
f16a71f178 Revert "Update Slices More Efficiently" (#4790)
* Revert "Update Slices More Efficiently (#4789)"

This reverts commit 669e1ea787.
2020-02-07 16:58:36 +00:00
Nishant Das
669e1ea787 Update Slices More Efficiently (#4789)
* better cached states

* lint

* jim's review
2020-02-07 09:06:39 -06:00
Jim McDonald
7ba2c897ad Add location option for wallet keymanager (#4788)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-02-06 23:03:33 -06:00
Preston Van Loon
34178aff2a Strict verify attestations in pubsub (#4782)
* Verify attestations before putting them into the pool
* use existing method
* Validate aggregated ones too
* Revert "Validate aggregated ones too"

This reverts commit a55646d131.
* Merge branch 'master' of github.com:prysmaticlabs/prysm into verify-all-atts
* Add feature flag
* The remaining shared reference fields with conditional copy on write
* Merge branch 'master' into better-copy-2
* Merge branch 'better-copy-2' of github.com:prysmaticlabs/prysm into verify-all-atts
* gaz
* fix build, put into validate
* lint
* Merge branch 'master' of github.com:prysmaticlabs/prysm into verify-all-atts
* why does goland do this to me
* revert unrelated change
* fix tests
* Update shared/featureconfig/config.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>
* Merge refs/heads/master into verify-all-atts
* Update beacon-chain/blockchain/testing/mock.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>
* gofmt
2020-02-07 03:21:55 +00:00
shayzluf
4df74a3b9d Slashing operations pool (#4726)
* first iteration

* with initial tests

* comment fix

* fix lint

* Start fixing other tests

* Cleanup

* Finish att slashing tests

* Finish up tests for proposer

* Fix docs

* Fix tests

* Fix pending att slashings to not include duplicates

* Fix max list

* Add test to make sure no duplicate slashings

* Address comments, improve insertion

* Fix error

* Update beacon-chain/operations/slashings/service.go

* Update beacon-chain/operations/slashings/service.go

* Update beacon-chain/operations/slashings/service.go

* Update beacon-chain/operations/slashings/service.go

* include a helper function for deduplication, fix some comments

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-02-07 10:32:51 +08:00
Preston Van Loon
f6dfaef537 BeaconState: remaining shared reference fields with conditional copy on write (#4785)
* The remaining shared reference fields with conditional copy on write
* Merge branch 'master' into better-copy-2
2020-02-06 21:55:29 +00:00
Raul Jordan
a9d144ad1f Stream Blocks Functionality for RPC (#4771)
* stream blocks functionality included
* necessary tests for stream blocks and notifier
* gazelle and tests passing
* gazelle and tests passing
* Merge branch 'master' into stream-block
* Update beacon-chain/core/feed/block/events.go
* Merge refs/heads/master into stream-block
* Merge refs/heads/master into stream-block
* Merge refs/heads/master into stream-block
* Merge refs/heads/master into stream-block
* naming
* build
* Merge refs/heads/master into stream-block
* Merge refs/heads/master into stream-block
* Merge refs/heads/master into stream-block
* fix up tests
* Merge branch 'stream-block' of github.com:prysmaticlabs/prysm into stream-block
* Merge refs/heads/master into stream-block
* shay comment
* Merge refs/heads/master into stream-block
* Merge branch 'stream-block' of github.com:prysmaticlabs/prysm into stream-block
* Merge refs/heads/master into stream-block
2020-02-06 20:14:38 +00:00
terence tsao
9cf30002d4 Rlock for computing head (#4784)
* Add lock
* Space
2020-02-06 19:59:50 +00:00
AgentJ-WR
e2faa391c3 Fix(Genesis): Api genesis block now returns properly (#4736)
* begin state service
* begin on the state trie idea
* created beacon state structure
* add in the full clone getter
* return by value instead
* add all setters
* new state setters are being completed
* arrays roots exposed
*  close to finishing all these headerssss
* functionality complete
* added in proto benchmark test
* test for compatibility
* add test for compat
* comments fixed
* Merge branch 'master' into state-service
* add clone
* add clone
* remove underlying copies
* make it immutable
* integrate it into chainservice
* revert
* wrap up comments for package
* address all comments and godocs
* address all comments
* Merge branch 'master' into state-service
* clone the pending attestation properly
* Merge branch 'state-service' of github.com:prysmaticlabs/prysm into state-service
* properly clone remaining items
* tests pass fixed bug
* begin using it instead of head state
* prevent nil pointer exceptions
* Merge branch 'state-service' into use-state-in-runtime
* begin using new struct in db
* integrated new type into db package
* add proper nil checks
* using new state in archiver
* refactored much of core
* editing all the precompute functions
* done with most core refactor
* fixed up some bugs in the clone comparisons
* Merge branch 'state-service' into use-state-in-runtime
* append current epoch atts
* merged master
* add missing setters
* add new setters
* fix other core methods
* fix up transition
* main service and forkchoice
* fix rpc
* integrated to powchain
* some more changes
* fix build
* improve processing of deposits
* fix error
* prevent panic
* comment
* fix process att
* gaz
* fix up att process
* resolve existing review comments
* Merge branch 'master' into use-state-in-runtime
* resolve another batch of gh comments
* resolve broken cpt state
* revise testutil to use the new state
* begin updating the state transition func to pass in more compartmentalized args
* finish editing transition function to return errors
* block operations pretty much done with refactor
* state transition fully refactored
* got epoch processing completed
* fix build in fork choice
* fixing more of the build
* fix up broken sync package
* it builds nowww it buildssss
* revert registry changes
* Recompute on Read (#4627)

* compute on read

* fix up eth1 data votes

* looking into slashings bug introduced in core/

* able to advance more slots

* add logging

* can now sync with testnet yay

* remove the leaves algorithm and other merkle imports

* expose initialize unsafe funcs

* Update beacon-chain/db/kv/state.go

* lint

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* include master
* More Optimizations for New State (#4641)

* map optimization

* more optimizations

* use a custom hasher

* comment

* block operations optimizations

* Update beacon-chain/state/types.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* fixed up various operations to use the validator index map access

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* archiver tests pass
* fixing cache tests
* cache tests passing
* edited validator tests
* powchain tests passing
* halfway thru sync tests
* more sync test fixes
* add in tests for state/
* working through rpc tests
* assignments tests passed
* almost done with rpc/beacon tests
* resolved painful validator test
* fixed up even more tests
* resolve tests
* fix build
* reduce a randao mixes copy
* fixes under //beacon-chain/blockchain/...
* build //beacon-chain/core/...
* fixes
* Runtime Optimizations (#4648)

* parallelize shuffling

* clean up

* lint

* fix build

* use callback to read from registry

* fix array roots and size map

* new improvements

* reduce hash allocs

* improved shuffling

* terence's review

* use different method

* raul's comment

* new array roots

* remove clone in pre-compute

* Update beacon-chain/state/types.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* raul's review

* lint

* fix build issues

* fix visibility

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Merge branch 'use-state-in-runtime' of https://github.com/prysmaticlabs/geth-sharding into resolve-tests
* fix visibility
* build works for all
* fix blockchain test
* fix a few tests
* fix more tests
* resolve conf
* sync
* update validator in slashing
* Merge branch 'resolve-tests' of github.com:prysmaticlabs/prysm into resolve-tests
* archiver passing
* Merge branch 'resolve-tests' of github.com:prysmaticlabs/prysm into resolve-tests
* fixed rpc/validator
* progress on core tests
* Merge branch 'resolve-tests' of github.com:prysmaticlabs/prysm into resolve-tests
* resolve broken rpc tests
* Merge branch 'resolve-tests' of github.com:prysmaticlabs/prysm into resolve-tests
* blockchain tests passed
* fix up some tests in core
* Merge branch 'master' of github.com:prysmaticlabs/prysm into resolve-tests
* fix message diff
* remove unnecessary save
* Merge branch 'master' of github.com:prysmaticlabs/prysm into resolve-tests
* Save validator after slashing
* Update validators one by one
* another update
* fix everything
* Merge branch 'resolve-tests' of https://github.com/prysmaticlabs/geth-sharding into resolve-tests
* fix more precompute tests
* fix blocks tests
* Merge branch 'resolve-tests' of github.com:prysmaticlabs/prysm into resolve-tests
* more elegant fix
* more helper fixes
* Merge branch 'resolve-tests' of https://github.com/prysmaticlabs/geth-sharding into resolve-tests
* change back ?
* fix test
* fix skip slot
* fix test
* reset caches
* fix testutil
* raceoff fixed
* passing
* Retrieve cached state in the beginning
* lint
* Merge branch 'master' of github.com:prysmaticlabs/prysm into resolve-tests
* Fixed tests part 1
* Fixed rest of the tests
* Merge branch 'master' into optimize-process-att
* Minor changes to avoid copying, small refactor to reduce deplicated code
* Merge branch 'master' into resolve-tests
* Handle att req for slot 0
* New beacon state: Only populate merkle layers as needed, copy merkle layers on copy/clone. (#4689)

* Only populate merkle layers as needed, copy merkle layers on copy/clone.

* use custom copy

* Make maps of correct size

* slightly fast, doesn't wait for lock

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'resolve-tests' of github.com:prysmaticlabs/prysm into resolve-tests
* Merge branch 'master' into resolve-tests
* Target root can't be 0x00
* Merge refs/heads/master into resolve-tests
* Merge branch 'resolve-tests' of github.com:prysmaticlabs/prysm into resolve-tests
* Don't use cache for current slot (may not be the right fix)
* Merge branch 'resolve-tests' of git+ssh://github.com/prysmaticlabs/prysm into resolve-tests
* Merge branch 'resolve-tests' of github.com:prysmaticlabs/prysm into resolve-tests
* fixed up tests
* Remove some copy for init sync. Not sure if it is safe enough for runtime though... testing...
* Align with prev logic for process slots cachedState.Slot() < slot
* Fix Initial Sync Flag (#4692)

* fixes

* fix up some test failures due to lack of nil checks

* fix up some test failures due to lack of nil checks

* fix up imports

* revert some changes

* imports

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Conflict
* Conflict
* resolve confs
* resolving further conflicts
* Better skip slot cache (#4694)

* Return copy of skip slot cache state, disable skip slot cache on sync

* fix
* Fix pruning
* Merge refs/heads/master into resolve-tests
* Merge refs/heads/master into resolve-tests
* copy on write method
* gaz
* fix confs
* Merge refs/heads/resolve-tests into copy-on-write
* fix tests
* Merge branch 'copy-on-write' of github.com:prysmaticlabs/prysm into copy-on-write
* fix up issues with broken tests
* Merge refs/heads/resolve-tests into copy-on-write
* Merge branch 'master' of github.com:prysmaticlabs/prysm into copy-on-write
* remove extra update
* remove debugging lines
* gofmt
* Merge refs/heads/master into copy-on-write
* Merge refs/heads/master into copy-on-write
* Merge refs/heads/master into copy-on-write
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Passing 1 to SetEndSlot
* Better way to get genesis block
* Merge branch 'master' of github.com:prysmaticlabs/prysm into bug/ApiGenesisBlock
* Checking for nil genBlk
* Testing conditional order
* Reverting conditions, and no idea why build fails
* Merge branch 'master' into bug/ApiGenesisBlock
* Saving to genesis block root in tests
* Merge branch 'bug/ApiGenesisBlock' of github.com:AgentJ-WR/prysm into bug/ApiGenesisBlock
* Saving root
* Merge branch 'master' of github.com:prysmaticlabs/prysm into bug/ApiGenesisBlock
* Adding regression test
* Updating regression test
* Merge branch 'master' into bug/ApiGenesisBlock
* Merge branch 'master' into bug/ApiGenesisBlock
* Merge branch 'master' into bug/ApiGenesisBlock
* Merge branch 'master' into bug/ApiGenesisBlock
2020-02-06 19:23:39 +00:00
terence tsao
5b83dffbe4 Use proto array forkchoice as default (#4778)
* Starting

* Removing feature flag

* Minor touchups service.go

* Conflict

* Started fixing test

* Init fork choice store for tests
2020-02-06 13:03:26 -06:00
terence tsao
b8383da468 Add forkchoiceAggregateAttestations to flag list (#4780)
* Added
* Merge branch 'master' into fix-flag
2020-02-06 18:09:51 +00:00
Preston Van Loon
c7fb28d42e Faster BLS publickey.Copy (#4770)
* Use balancesLength and randaoMixesLength to save copy on read
* use a cheaper copy for BLS publickey.Copy()
* Merge branch 'master' into bls-better-copy
* Merge refs/heads/master into bls-better-copy
* Merge refs/heads/master into bls-better-copy
* Merge refs/heads/master into bls-better-copy
* Merge refs/heads/master into bls-better-copy
* Merge branch 'master' of github.com:prysmaticlabs/prysm into bls-better-copy
* quick test
* Merge refs/heads/master into bls-better-copy
2020-02-06 17:35:38 +00:00
Preston Van Loon
3a9c8eb8b1 Log a warning if attempting to save a nil state (#4779)
* Log a warning if attempting to save a nil state
* Log a warning if attempting to save a nil state
2020-02-06 17:22:44 +00:00
Preston Van Loon
a9f1de354b Use balancesLength and randaoMixesLength to save copy on read (#4769)
* Use balancesLength and randaoMixesLength to save copy on read

* Update beacon-chain/state/getters.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: shayzluf <thezluf@gmail.com>
2020-02-06 10:06:27 -06:00
terence tsao
69c3d9dec2 Save attestation to DB gated by archival flag (#4776)
* Gated by flag
* Gaz
* Merge branch 'master' into archival-saves-att
2020-02-06 15:14:52 +00:00
Nishant Das
b99ae2cbe4 Remove All Batch DB Calls (#4775)
* remove all batch calls
2020-02-06 08:23:06 +00:00
Preston Van Loon
bfa103317e Disable forkchoice pre-processing of attestations (#4774)
* Disable forkchoice pre-processing of attestations
* space
* gaz
2020-02-06 07:53:08 +00:00
Nishant Das
dc1432f8d8 Attestation Verification Improvements (#4753)
* add fixes for sig verify
* minor fix
* Merge branch 'master' into fixSigVerify
* fmt
* Merge branch 'fixSigVerify' of https://github.com/prysmaticlabs/geth-sharding into fixSigVerify
* Merge refs/heads/master into fixSigVerify
* Merge refs/heads/master into fixSigVerify
* Merge refs/heads/master into fixSigVerify
* Merge refs/heads/master into fixSigVerify
* use custom att copy
* Merge refs/heads/master into fixSigVerify
* Merge refs/heads/master into fixSigVerify
* Merge refs/heads/master into fixSigVerify
* Merge refs/heads/master into fixSigVerify
* Merge refs/heads/master into fixSigVerify
* Merge refs/heads/master into fixSigVerify
* Merge refs/heads/master into fixSigVerify
* Merge refs/heads/master into fixSigVerify
* Merge refs/heads/master into fixSigVerify
* Merge refs/heads/master into fixSigVerify
* finnaly fixed all this
* Merge branch 'fixSigVerify' of https://github.com/prysmaticlabs/geth-sharding into fixSigVerify
2020-02-06 05:46:25 +00:00
terence tsao
85b379c08c Fix block tree cosmetic bugs (#4768)
* Fixed
* Merge branch 'master' into check-ready
2020-02-06 04:49:12 +00:00
Preston Van Loon
91b8760632 Beacon State: Track field references (#4751)
* track field references
* gofmt, RLock
* Merge refs/heads/master into randao-ref-tracking
* Merge refs/heads/master into randao-ref-tracking
* Merge refs/heads/master into randao-ref-tracking
* Merge refs/heads/master into randao-ref-tracking
* Merge refs/heads/master into randao-ref-tracking
* Merge refs/heads/master into randao-ref-tracking
* Merge refs/heads/master into randao-ref-tracking
* cleanup comments
* Merge branch 'randao-ref-tracking' of github.com:prysmaticlabs/prysm into randao-ref-tracking
* Add a test for finalizer
* Merge refs/heads/master into randao-ref-tracking
* Merge refs/heads/master into randao-ref-tracking
* maybe fix data race
* maybe fix data race
* temp comment out test file to find which one fails race test
* its definitely something with the checkpoint state cache
* its definitely something with the checkpoint state cache
* its definitely something with the checkpoint state cache
* Merge refs/heads/master into randao-ref-tracking
* This should fix it
* Merge branch 'randao-ref-tracking' of github.com:prysmaticlabs/prysm into randao-ref-tracking
* gaz
* Merge refs/heads/master into randao-ref-tracking
* Merge refs/heads/master into randao-ref-tracking
* turn off race detection, i dont understand why is broken
* Merge branch 'randao-ref-tracking' of github.com:prysmaticlabs/prysm into randao-ref-tracking
* feedback
* @nisdas feedback
* Revert "@nisdas feedback"

This reverts commit 6129cf84e6.
* Merge refs/heads/master into randao-ref-tracking
2020-02-06 04:07:23 +00:00
Preston Van Loon
27e7be6279 Add & use FinalizedCheckpointEpoch() to state (#4766)
* Add and use FinalizedCheckpointEpoch() to state
* comment
2020-02-06 03:26:15 +00:00
Ivan Martinez
ebd4541dcd Add test for GetBlock (#4765)
* Add a test for GetBlock
* Fix formatting
* Fix grafitti
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into test-getblock
* Merge branch 'master' into test-getblock
2020-02-05 23:56:38 +00:00
Raul Jordan
32b5b8fa69 Include Latest Ethereum APIs Definitions in Prysm (#4759)
* include latest ethereumapis and implement streams
* add comments and error unimpl
* Merge branch 'master' into blocks-stream
* add strm
* Merge refs/heads/master into blocks-stream
* Merge refs/heads/master into blocks-stream
* Merge refs/heads/master into blocks-stream
* add in mocks
* Merge branch 'blocks-stream' of github.com:prysmaticlabs/prysm into blocks-stream
* use right mock
* gaz
* ptypes
* gaz
* gomock dep
* Merge refs/heads/master into blocks-stream
2020-02-05 23:43:36 +00:00
terence tsao
c6e3d67241 Block tree enhancements (#4764)
* Add votes, correct conversion and green for head

* Starting testing

* Fix for run time
2020-02-05 17:30:55 -06:00
Raul Jordan
cb33deab36 batch to update (#4763) 2020-02-05 13:01:56 -08:00
terence tsao
113ac460c0 Update higherThanFinalized in the loop (#4761)
* Update higherThanFinalized in the loop
* Addded a test
* Fixed test
* Merge branch 'master' into update-finalized
2020-02-05 20:13:38 +00:00
Preston Van Loon
c496170c33 Do not attempt to save a nil state (#4758) 2020-02-05 11:52:14 -08:00
Jim McDonald
945edb6c8f Sync to highest possible head given the peers available (#4570)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-02-05 12:44:32 -06:00
garyschulte
24a5a9c112 add prometheus metrics for validator accounts (#4724)
* add prometheus metrics for validator accounts:
  * gauge for balances
  * counters for attestations and failures
  * counters for aggregations and failures
  * counters for proposals and failures

put validator account metrics behind flag

* run gazelle to reorganize deps

* fix typo

Co-authored-by: garyschulteog <30323939+garyschulteog@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-02-05 11:49:27 -06:00
terence tsao
0180051b5e Update head slot metric after compute head (#4754)
* Update head slot after compute head
* Merge branch 'master' into properly-update-head-metric
* Merge refs/heads/master into properly-update-head-metric
2020-02-05 17:24:05 +00:00
terence tsao
ce79d8e295 Insert initial sync missing blocks to fork choice store (#4750)
* Upon start up, don't insert head to proto array node DAG as index 0
* Merge branch 'master' of git+ssh://github.com/prysmaticlabs/prysm into fix-save-head-root
* Add HasNode
* Add fillInForkChoiceMissingBlocks
* Comments
* Test
* Fixed test
* Merge branch 'master' into proto-array-process-missing-block
* Merge branch 'master' into proto-array-process-missing-block
* Merge branch 'master' into proto-array-process-missing-block
2020-02-05 17:05:51 +00:00
Raul Jordan
9579a5520b Update All Libp2p Dependencies (#4746)
* all libp2p deps added
* Merge branch 'master' into update-libp2p
* add in dep
* kad
* fix p2p test
* p2p
* Merge refs/heads/master into update-libp2p
* Merge refs/heads/master into update-libp2p
* Merge refs/heads/master into update-libp2p
* Merge refs/heads/master into update-libp2p
2020-02-05 16:37:33 +00:00
Preston Van Loon
9958afe79d RPC: Use the proper db access level, use head root from head fetcher (#4752)
* Use the proper db access level, use head root from head fetcher
* Reuse head root
2020-02-05 08:50:07 +00:00
terence tsao
8ad174ffd8 Upon start up, don't insert head to proto array node DAG as index 0 (#4749) 2020-02-05 11:11:51 +08:00
Preston Van Loon
68b6a7c172 Render graphviz graph in page (#4748) 2020-02-04 17:14:43 -08:00
terence tsao
8c5c7352b1 Update blockchain metrics (#4747)
* Make new metrics the canonical one
* Both threads share same metric set
* Comments
* Fix vis
2020-02-04 22:57:36 +00:00
Raul Jordan
b705ab0239 Update dependencies from renovate (#4745)
* add in proto deps

* more deps

* more deps

* some revert

* add machinery

* downgrade k8s api

* gaz

* thrift
2020-02-04 14:13:40 -06:00
Ivan Martinez
923d5fc903 Cleanup slasher codebase (#4698)
* First wave of changes
* More changes
* More renames, changes
* Fix errors
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into cleanup-slasher
* Fix errors, more cleaning
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into cleanup-slasher
* Merge branch 'master' into cleanup-slasher
* fix err
* Merge branch 'cleanup-slasher' of https://github.com/0xKiwi/Prysm into cleanup-slasher
* Fix strings
* More cleanup
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into cleanup-slasher
* Fix interface
* Fix
* Merge branch 'master' into cleanup-slasher
* Merge branch 'master' into cleanup-slasher
* Merge branch 'master' into cleanup-slasher
* Address comments
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into cleanup-slasher
* Merge branch 'cleanup-slasher' of https://github.com/0xKiwi/Prysm into cleanup-slasher
2020-02-04 18:48:51 +00:00
terence tsao
9c1a294bf7 Misc fork choice improvements (#4744)
* Added missing spec implementation
* Use it
* Rename
* Merge branch 'master' into update-justified
2020-02-04 17:35:06 +00:00
Raul Jordan
061960c9e2 Resolve Miscellaneous Bugs in Beacon Node (#4743)
* add in nil check for head block

* fix logic

* unused import
2020-02-04 11:21:02 -06:00
terence tsao
80248cd296 Fix pre state of target block does not exist error (#4740)
* Save state even w/ initial sync cache state flag

* Tested

Co-authored-by: Nishant Das <nish1993@hotmail.com>
2020-02-04 20:19:09 +08:00
Nishant Das
95b6cca399 add lock (#4739) 2020-02-04 16:31:31 +08:00
Raul Jordan
1478882b41 Add Endpoint to Return Current Chain Config Parameters (#4595)
* add patch and endpoint
* formatting
* Merge branch 'master' into chain-config-endpoint
* Merge branch 'master' into chain-config-endpoint
* include beacon config
* config params
* Merge branch 'chain-config-endpoint' of github.com:prysmaticlabs/prysm into chain-config-endpoint
* include tests
* resolve confs
* use patch
* Merge branch 'master' into chain-config-endpoint
* passing tests
* Merge branch 'chain-config-endpoint' of github.com:prysmaticlabs/prysm into chain-config-endpoint
* Merge branch 'master' into chain-config-endpoint
* Merge branch 'master' into chain-config-endpoint
* Merge refs/heads/master into chain-config-endpoint
2020-02-04 05:28:35 +00:00
Preston Van Loon
7c4950832c Increase BLS pubkey cache to 100k from 10k (#4737)
* Increase BLS pubkey cache to 100k from 10k
2020-02-04 05:00:20 +00:00
Preston Van Loon
ce0b55d13e Ensure all fields are dirty on initialization (#4735) 2020-02-03 20:38:13 -06:00
Preston Van Loon
e63119b254 Better state locking (#4733)
* Rearrange lock a bit

* better locking without deadlock

* reorder lock
2020-02-03 17:19:22 -06:00
terence tsao
8492273fa7 Better log for Requesting block for pending attestation... (#4731)
* Better log
* Use debug
* Merge branch 'master' into req-blk-log
* Merge branch 'master' into req-blk-log
2020-02-03 21:11:21 +00:00
Preston Van Loon
5b4025efcd SSZ state cache: Only use cached value when flag is on (#4732)
* Only use cached value when flag is on
2020-02-03 20:49:07 +00:00
terence tsao
c69385e71d Clean up verify attestation and better error log (#4729) 2020-02-03 11:46:26 -08:00
terence tsao
cdfa969ced Insert block to fork choice after saving the block to DB (#4728)
* Let's try this
* Merge branch 'master' of git+ssh://github.com/prysmaticlabs/prysm
* Better place to insert block to fork choice store
* Fmt
* Revert a few changes
* Revert a few changes
* Comments
* Merge branch 'master' of git+ssh://github.com/prysmaticlabs/prysm into insert-blk-forkchoice
* Merge refs/heads/master into insert-blk-forkchoice
2020-02-03 17:41:36 +00:00
shayzluf
a1dc4ddc40 Add get slashing endpoints (#4674)
* update go pbs
* protos
* merge
* pbs
* implement first version
* add slashing status endpoints and test
* Merge branch 'master' of github.com:prysmaticlabs/prysm into get_slashings
* add tests
* gaz and goimports
* gaz
* Merge refs/heads/master into get_slashings
* Merge refs/heads/master into get_slashings
* Merge refs/heads/master into get_slashings
* Merge refs/heads/master into get_slashings
* Merge refs/heads/master into get_slashings
* Merge refs/heads/master into get_slashings
* Merge refs/heads/master into get_slashings
* Merge refs/heads/master into get_slashings
* Merge refs/heads/master into get_slashings
* Merge refs/heads/master into get_slashings
* Merge refs/heads/master into get_slashings
* Merge refs/heads/master into get_slashings
* Merge refs/heads/master into get_slashings
* Merge refs/heads/master into get_slashings
* Merge refs/heads/master into get_slashings
* Merge refs/heads/master into get_slashings
* Merge refs/heads/master into get_slashings
* Merge refs/heads/master into get_slashings
* Merge refs/heads/master into get_slashings
* Merge refs/heads/master into get_slashings
* Update proto/slashing/slashing.proto
* Update proto/slashing/slashing.proto
* Update proto/slashing/slashing.proto
* Merge refs/heads/master into get_slashings
2020-02-03 17:31:54 +00:00
Jim McDonald
648584b356 Add wallet keymanager (#4687)
* Add wallet keymanager

* Read keymanageropts from file if not JSON

Co-authored-by: Nishant Das <nish1993@hotmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-02-03 11:13:58 -06:00
Nishant Das
fb7a75d2c3 Release Proposer Index Cache (#4717)
* release cache

* gaz

* fix all tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-02-03 10:23:04 -06:00
Jim McDonald
00a6361c66 Relegate some p2p messages (#4725)
* Relegate some p2p messages
2020-02-03 11:27:20 +00:00
Preston Van Loon
6213c94a14 Fork choice: only update head if the new block is a higher block slot (#4722)
* only update head IF the new block is a higher block slot
* Merge refs/heads/master into forkchoice-by-highest-blockslot
2020-02-03 01:40:20 +00:00
Preston Van Loon
397b7d807a Pubsub: Ignore block already in database (#4721)
* Ignore block already in database
* Merge refs/heads/master into skip-block-already-in-db
2020-02-03 00:37:50 +00:00
Preston Van Loon
05876d6250 Only update committee cache if it doesn't have that key already (#4719)
* Only update committee cache if it doesn't have that key already
2020-02-03 00:24:54 +00:00
Preston Van Loon
bd334c4192 Minor fixes (#4716)
* copy state in cache, ensure pre state exists before attepting to process attestation in fork choice
* copy state in cache, ensure pre state exists before attepting to process attestation in fork choice
* fix test
2020-02-02 05:23:05 +00:00
Preston Van Loon
4f38333e54 Copy head state to ensure it is never mutated (#4715)
* Copy head state to ensure it is never mutated
* Merge refs/heads/master into copy-head-state
* fix tests
* Merge branch 'copy-head-state' of github.com:prysmaticlabs/prysm into copy-head-state
2020-02-02 03:19:51 +00:00
Nishant Das
d6bd389d5c Custom Copy of Pending Attestations (#4711)
* custom copy
* lint
* Merge refs/heads/master into customCopy
* preston's review
* Merge branch 'customCopy' of https://github.com/prysmaticlabs/geth-sharding into customCopy
* Merge refs/heads/master into customCopy
* Merge refs/heads/master into customCopy
* nil check
* Merge refs/heads/master into customCopy
* Merge branch 'customCopy' of https://github.com/prysmaticlabs/geth-sharding into customCopy
* Merge refs/heads/master into customCopy
* fixed test
* Merge branch 'customCopy' of https://github.com/prysmaticlabs/geth-sharding into customCopy
2020-02-02 02:56:53 +00:00
Preston Van Loon
962be9b4f8 Propagate blocks again after we process it in pending blocks queue (#4714)
* propagate blocks again after we process it in pending blocks queue
* Merge refs/heads/master into reprop-block
2020-02-02 02:22:59 +00:00
terence tsao
f77049ae74 Handle attestations with missing block (#4705)
* Fmt
* Starting
* Cont
* Store aggregate attestation is better
* Conflict
* Done
* Merge branch 'master' of git+ssh://github.com/prysmaticlabs/prysm into process-pending-atts
* Comment
* Better logs
* Better logs
* Fix existing tests
* Update metric names
* Preston's feedback
* Broadcast atts once it's valid
* Gazelle
* Test for validating atts and pruning
* Tests
* Removed debug log
* Conflict
* Feedback
* Merge refs/heads/master into process-pending-atts
* Merge refs/heads/master into process-pending-atts
2020-02-02 01:42:29 +00:00
Preston Van Loon
f432f7851e BeaconState: Use copy on write for validator index map (#4713)
* Use copy on write for validator index map
* Merge refs/heads/master into copy-on-write-2
2020-02-02 01:21:50 +00:00
Preston Van Loon
b7e6012628 Better eth1data equals (#4712)
* Better eth1data equals
* Merge branch 'master' into better-eth1-data-equal
2020-02-02 01:03:21 +00:00
Preston Van Loon
79434fc2d1 Use sync.pool for keccak256 and sha256 (#4710)
* Use sync.pool for keccak

* Add sha256 too

* custom hasher use pool too

* reset custom hasher

* fix?

* Add comment that customHasher should only be used in cases of more than 5 usages
2020-02-01 16:37:37 -08:00
Preston Van Loon
069ec1726b Pending blocks queue: Better locking priority (#4709)
* Better locking priority, use correct lock on validating pending blocks
2020-02-01 22:47:51 +00:00
Preston Van Loon
2a79c572a5 Pruning old states: Use a warning level log instead of fatal (#4707)
* Use a warning level log instead of fatal
2020-02-01 20:51:20 +00:00
Preston Van Loon
c2fbb40909 Beacon state: copy on write for certain large fields (#4699)
* begin state service

* begin on the state trie idea

* created beacon state structure

* add in the full clone getter

* return by value instead

* add all setters

* new state setters are being completed

* arrays roots exposed

*  close to finishing all these headerssss

* functionality complete

* added in proto benchmark test

* test for compatibility

* add test for compat

* comments fixed

* add clone

* add clone

* remove underlying copies

* make it immutable

* integrate it into chainservice

* revert

* wrap up comments for package

* address all comments and godocs

* address all comments

* clone the pending attestation properly

* properly clone remaining items

* tests pass fixed bug

* begin using it instead of head state

* prevent nil pointer exceptions

* begin using new struct in db

* integrated new type into db package

* add proper nil checks

* using new state in archiver

* refactored much of core

* editing all the precompute functions

* done with most core refactor

* fixed up some bugs in the clone comparisons

* append current epoch atts

* add missing setters

* add new setters

* fix other core methods

* fix up transition

* main service and forkchoice

* fix rpc

* integrated to powchain

* some more changes

* fix build

* improve processing of deposits

* fix error

* prevent panic

* comment

* fix process att

* gaz

* fix up att process

* resolve existing review comments

* resolve another batch of gh comments

* resolve broken cpt state

* revise testutil to use the new state

* begin updating the state transition func to pass in more compartmentalized args

* finish editing transition function to return errors

* block operations pretty much done with refactor

* state transition fully refactored

* got epoch processing completed

* fix build in fork choice

* fixing more of the build

* fix up broken sync package

* it builds nowww it buildssss

* revert registry changes

* Recompute on Read (#4627)

* compute on read

* fix up eth1 data votes

* looking into slashings bug introduced in core/

* able to advance more slots

* add logging

* can now sync with testnet yay

* remove the leaves algorithm and other merkle imports

* expose initialize unsafe funcs

* Update beacon-chain/db/kv/state.go

* lint

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* More Optimizations for New State (#4641)

* map optimization

* more optimizations

* use a custom hasher

* comment

* block operations optimizations

* Update beacon-chain/state/types.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* fixed up various operations to use the validator index map access

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* archiver tests pass

* fixing cache tests

* cache tests passing

* edited validator tests

* powchain tests passing

* halfway thru sync tests

* more sync test fixes

* add in tests for state/

* working through rpc tests

* assignments tests passed

* almost done with rpc/beacon tests

* resolved painful validator test

* fixed up even more tests

* resolve tests

* fix build

* reduce a randao mixes copy

* fixes under //beacon-chain/blockchain/...

* build //beacon-chain/core/...

* fixes

* Runtime Optimizations (#4648)

* parallelize shuffling

* clean up

* lint

* fix build

* use callback to read from registry

* fix array roots and size map

* new improvements

* reduce hash allocs

* improved shuffling

* terence's review

* use different method

* raul's comment

* new array roots

* remove clone in pre-compute

* Update beacon-chain/state/types.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* raul's review

* lint

* fix build issues

* fix visibility

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* fix visibility

* build works for all

* fix blockchain test

* fix a few tests

* fix more tests

* update validator in slashing

* archiver passing

* fixed rpc/validator

* progress on core tests

* resolve broken rpc tests

* blockchain tests passed

* fix up some tests in core

* fix message diff

* remove unnecessary save

* Save validator after slashing

* Update validators one by one

* another update

* fix everything

* fix more precompute tests

* fix blocks tests

* more elegant fix

* more helper fixes

* change back ?

* fix test

* fix skip slot

* fix test

* reset caches

* fix testutil

* raceoff fixed

* passing

* Retrieve cached state in the beginning

* lint

* Fixed tests part 1

* Fixed rest of the tests

* Minor changes to avoid copying, small refactor to reduce deplicated code

* Handle att req for slot 0

* New beacon state: Only populate merkle layers as needed, copy merkle layers on copy/clone. (#4689)

* Only populate merkle layers as needed, copy merkle layers on copy/clone.

* use custom copy

* Make maps of correct size

* slightly fast, doesn't wait for lock

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Target root can't be 0x00

* Don't use cache for current slot (may not be the right fix)

* fixed up tests

* Remove some copy for init sync. Not sure if it is safe enough for runtime though... testing...

* Align with prev logic for process slots cachedState.Slot() < slot

* Fix Initial Sync Flag (#4692)

* fixes

* fix up some test failures due to lack of nil checks

* fix up some test failures due to lack of nil checks

* fix up imports

* revert some changes

* imports

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* resolving further conflicts

* Better skip slot cache (#4694)

* Return copy of skip slot cache state, disable skip slot cache on sync

* fix

* Fix pruning

* copy on write method

* gaz

* fix tests

* fix up issues with broken tests

* remove extra update

* remove debugging lines

* gofmt

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nish1993@hotmail.com>
Co-authored-by: shayzluf <thezluf@gmail.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-01-31 23:23:34 -08:00
Preston Van Loon
d32493d43b Ensure exits are not for an already exited validator (#4701)
* ensure exits are not for an already exited validator
2020-02-01 01:18:36 +00:00
terence tsao
0b2b77c5b0 Remove validate_beacon_attestation (#4700) 2020-01-31 15:35:13 -08:00
terence tsao
d8c26590ca Prune dangling states in DB upon start up (#4697)
* Add pruneGarbageState and test
* Comments
* Update beacon-chain/blockchain/service.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/blockchain/service.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/blockchain/service.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Fixed test
* Merge refs/heads/master into prune-start-up
* Fixed test
2020-01-31 23:15:35 +00:00
Raul Jordan
cc741ed8af Ensure New State Type Tests Pass in Prysm (#4646)
* begin state service

* begin on the state trie idea

* created beacon state structure

* add in the full clone getter

* return by value instead

* add all setters

* new state setters are being completed

* arrays roots exposed

*  close to finishing all these headerssss

* functionality complete

* added in proto benchmark test

* test for compatibility

* add test for compat

* comments fixed

* add clone

* add clone

* remove underlying copies

* make it immutable

* integrate it into chainservice

* revert

* wrap up comments for package

* address all comments and godocs

* address all comments

* clone the pending attestation properly

* properly clone remaining items

* tests pass fixed bug

* begin using it instead of head state

* prevent nil pointer exceptions

* begin using new struct in db

* integrated new type into db package

* add proper nil checks

* using new state in archiver

* refactored much of core

* editing all the precompute functions

* done with most core refactor

* fixed up some bugs in the clone comparisons

* append current epoch atts

* add missing setters

* add new setters

* fix other core methods

* fix up transition

* main service and forkchoice

* fix rpc

* integrated to powchain

* some more changes

* fix build

* improve processing of deposits

* fix error

* prevent panic

* comment

* fix process att

* gaz

* fix up att process

* resolve existing review comments

* resolve another batch of gh comments

* resolve broken cpt state

* revise testutil to use the new state

* begin updating the state transition func to pass in more compartmentalized args

* finish editing transition function to return errors

* block operations pretty much done with refactor

* state transition fully refactored

* got epoch processing completed

* fix build in fork choice

* fixing more of the build

* fix up broken sync package

* it builds nowww it buildssss

* revert registry changes

* Recompute on Read (#4627)

* compute on read

* fix up eth1 data votes

* looking into slashings bug introduced in core/

* able to advance more slots

* add logging

* can now sync with testnet yay

* remove the leaves algorithm and other merkle imports

* expose initialize unsafe funcs

* Update beacon-chain/db/kv/state.go

* lint

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* More Optimizations for New State (#4641)

* map optimization

* more optimizations

* use a custom hasher

* comment

* block operations optimizations

* Update beacon-chain/state/types.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* fixed up various operations to use the validator index map access

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* archiver tests pass

* fixing cache tests

* cache tests passing

* edited validator tests

* powchain tests passing

* halfway thru sync tests

* more sync test fixes

* add in tests for state/

* working through rpc tests

* assignments tests passed

* almost done with rpc/beacon tests

* resolved painful validator test

* fixed up even more tests

* resolve tests

* fix build

* reduce a randao mixes copy

* fixes under //beacon-chain/blockchain/...

* build //beacon-chain/core/...

* fixes

* Runtime Optimizations (#4648)

* parallelize shuffling

* clean up

* lint

* fix build

* use callback to read from registry

* fix array roots and size map

* new improvements

* reduce hash allocs

* improved shuffling

* terence's review

* use different method

* raul's comment

* new array roots

* remove clone in pre-compute

* Update beacon-chain/state/types.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* raul's review

* lint

* fix build issues

* fix visibility

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* fix visibility

* build works for all

* fix blockchain test

* fix a few tests

* fix more tests

* update validator in slashing

* archiver passing

* fixed rpc/validator

* progress on core tests

* resolve broken rpc tests

* blockchain tests passed

* fix up some tests in core

* fix message diff

* remove unnecessary save

* Save validator after slashing

* Update validators one by one

* another update

* fix everything

* fix more precompute tests

* fix blocks tests

* more elegant fix

* more helper fixes

* change back ?

* fix test

* fix skip slot

* fix test

* reset caches

* fix testutil

* raceoff fixed

* passing

* Retrieve cached state in the beginning

* lint

* Fixed tests part 1

* Fixed rest of the tests

* Minor changes to avoid copying, small refactor to reduce deplicated code

* Handle att req for slot 0

* New beacon state: Only populate merkle layers as needed, copy merkle layers on copy/clone. (#4689)

* Only populate merkle layers as needed, copy merkle layers on copy/clone.

* use custom copy

* Make maps of correct size

* slightly fast, doesn't wait for lock

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Target root can't be 0x00

* Don't use cache for current slot (may not be the right fix)

* fixed up tests

* Remove some copy for init sync. Not sure if it is safe enough for runtime though... testing...

* Align with prev logic for process slots cachedState.Slot() < slot

* Fix Initial Sync Flag (#4692)

* fixes

* fix up some test failures due to lack of nil checks

* fix up some test failures due to lack of nil checks

* fix up imports

* revert some changes

* imports

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* resolving further conflicts

* Better skip slot cache (#4694)

* Return copy of skip slot cache state, disable skip slot cache on sync

* fix

* Fix pruning

* fix up issues with broken tests

Co-authored-by: Nishant Das <nish1993@hotmail.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: shayzluf <thezluf@gmail.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-01-31 12:57:01 -08:00
Raul Jordan
f97ac5f0d7 Remove Already Exited Validators From Queue (#4695)
* added regression test

* fixed the regression test units

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-01-30 17:57:42 -08:00
Ivan Martinez
85a38e6053 Make E2E more consistent, change log file setup (#4696)
* Make E2E more consistent, change log file setup
* Merge branch 'master' into fix-e2e-sync
2020-01-31 00:16:36 +00:00
terence tsao
7f07ad831e Update seen for attestation pool (#4669)
* Use `contain` instead of `overlap`
* Tests
* Update seen
* Merge branch 'master' into use-contain
* Use OR to track all of the bits we have seen so far
* Added one more test case
* gofmt
* Conflict
* Picked up Preston's changes
* Merge branch 'use-contain' of git+ssh://github.com/prysmaticlabs/prysm into use-contain
2020-01-30 23:30:37 +00:00
Jim McDonald
ad7d9ab1da Validator status updates (#4675)
* Update ValidatorStatus to match Ethereum APIs
* Tidy up status calculation
* Merge branch 'master' into validator-status-updates
* Merge branch 'master' into validator-status-updates
* Update beacon-chain/rpc/beacon/config.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update test names
2020-01-30 19:46:37 +00:00
terence tsao
e452b950d0 Better caching of attestation pre state (#4688) 2020-01-30 11:06:20 -08:00
Ivan Martinez
2e2cec3a61 Make E2E more resilient, check balance and participation every epoch (#4679) 2020-01-29 16:14:10 -08:00
Ivan Martinez
c80ffc640f Fix flag bug (#4690) 2020-01-29 16:16:37 -06:00
Dmitri Tsumak
f6b4637a91 Update Eth1FollowDistance to 16 for minimal config (#4566)
* Update Eth1FollowDistance to 16 for minimal config
* Merge branch 'master' into fix-minimal-config
* Merge branch 'master' into fix-minimal-config
* Merge branch 'master' into fix-minimal-config
* Merge branch 'master' into fix-minimal-config
* Merge branch 'master' into fix-minimal-config
* Fix tests after minimal config changes
2020-01-29 17:56:21 +00:00
Ivan Martinez
3e9bf58d81 Fix validator assignments on slot 0 (#4682)
* Fix validator acting upon first slot
* Change log to debug
* Fix roles at slot 0
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into fix-val
* Add regression test
* Formatting
* Add slot ticker regression test
2020-01-29 04:30:30 +00:00
terence tsao
a22c97739e Remove prune state (#4680)
* Rm prune state
* Merge branch 'master' into rm-prune-state
2020-01-29 03:53:57 +00:00
shayzluf
ade61717a4 Slasher data update from archive (#4563)
* first version

* cli context

* fix service

* starting change to ccache

* ristretto cache

* added test

* test on evict

* remove evict test

* test onevict

* comment for exported flag

* update all span maps on load

* fix setup db

* span cache added to help flags

* start save cache on exit

* save cache to db before close

* comment fix

* fix flags

* setup db new

* data update from archive node

* gaz

* slashing detection on old attestations

* un-export

* rename

* nishant feedback

* workspace cr

* lint fix

* fix calls

* start db

* fix test

* Update slasher/db/db.go

Co-Authored-By: Nishant Das <nishdas93@gmail.com>

* add flag

* fix fail to start beacon client

* mock beacon service

* fix imports

* gaz

* goimports

* add clear db flag

* print finalized epoch

* better msg

* Update slasher/db/attester_slashings.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* raul feedback

* raul feedback

* raul feedback

* raul feedback

* raul feedback

* add detection in runtime

* fix tests

* raul feedbacks

* raul feedback

* raul feedback

* goimports

* Update beacon-chain/blockchain/process_attestation_helpers.go

* Update beacon-chain/blockchain/receive_block.go

* Update beacon-chain/core/blocks/block_operations_test.go

* Update beacon-chain/core/blocks/block_operations.go

* Update beacon-chain/core/epoch/epoch_processing.go

* Update beacon-chain/sync/validate_aggregate_proof_test.go

* Update shared/testutil/block.go

* Update slasher/service/data_update.go

* Update tools/blocktree/main.go

* Update slasher/service/service.go

* Update beacon-chain/core/epoch/precompute/attestation_test.go

* Update beacon-chain/core/helpers/committee_test.go

* Update beacon-chain/core/state/transition_test.go

* Update beacon-chain/rpc/aggregator/server_test.go

* Update beacon-chain/sync/validate_aggregate_proof.go

* Update beacon-chain/rpc/validator/proposer_test.go

* Update beacon-chain/blockchain/forkchoice/process_attestation.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Update slasher/db/indexed_attestations.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Update slasher/service/data_update.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* terence feedback

* terence feedback

* goimports

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nish1993@hotmail.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-01-29 07:14:51 +05:30
Ivan Martinez
9149c2e4f4 Replace no-genesis-delay with custom-genesis-delay (#4678)
* Change NoGenesisDelay to CustomGenesisDelay
* Implement flag
* gazelle
* Fix docs
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into custom-genesis-delay
* Fix
* Gazelle
* Add case to fix bad math
* Merge branch 'master' into custom-genesis-delay
2020-01-28 21:07:43 +00:00
terence tsao
07ba594023 Fix initial sync cache state (#4677) 2020-01-28 12:43:54 -08:00
Ivan Martinez
ad01bfbcde Add sync test to E2E (#4654)
* Complete evaluator for chain consensus

* Add sync e2e test

* Cleanup

* Rename

* Add tad more offset for correct head

* Change offset to middle of slot

* Change head block root to head epoch

* comment

* Fix eth1

* Address comments

* Gazelle

* Change to use file

* Change to use reader

* Use fil
2020-01-28 13:16:00 -06:00
Raul Jordan
439a84fcb9 Clear Run Error in Powchain Service Upon Reconnect (#4671)
* clear the run err on reconnect
* Merge refs/heads/master into clear-err-on-reconnect
* nishant feedback
2020-01-28 04:04:38 +00:00
terence tsao
e2be2a21d0 Part 2 of block chain service refactor - move process attestation (#4672) 2020-01-27 18:04:43 -08:00
terence tsao
eaf7ae3774 Part 1 of block chain service refactor - move process block (#4670) 2020-01-27 13:48:16 -08:00
terence tsao
1fa301c79c Update node count based on insertion (#4653)
* Update node count based on insertion

* Update nodes.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-01-27 15:24:58 -06:00
Preston Van Loon
1c759f6404 Disable more fork choice options with flag on (#4665)
* Disable updating latest votes if disable fork choice
* do not recompute the block tree cache if the fork choice is not being used
2020-01-27 09:11:28 +00:00
Preston Van Loon
4960acb285 Fork choice: Ensure lengths are the same before checking overlap (#4663)
* Ensure lengths are the same before checking overlap
2020-01-27 07:18:02 +00:00
Ivan Martinez
127f05d531 Allow easy plugin of featureflags into E2E (#4659)
* Enable easy plugin of featureflags into E2E

* Gazelle

* Fix text

* Fix whitespace
2020-01-26 21:42:10 -05:00
Preston Van Loon
2f02a2baa3 Actually wire up exits (#4661)
* Actually wire up exits
* Merge branch 'master' into exit-fixes
2020-01-27 01:49:37 +00:00
terence tsao
d4bea51482 Proto array fork choice tree handler (#4658) 2020-01-26 12:25:33 -08:00
Nishant Das
4ea5661f8f Clear Pre-Genesis Objects (#4656)
* remove pre-genesis data
* Merge branch 'master' into clearUnusedObjects
* lint
* Merge branch 'clearUnusedObjects' of https://github.com/prysmaticlabs/geth-sharding into clearUnusedObjects
* fic build
* gaz
* faulty mock
* Update beacon-chain/blockchain/service.go
2020-01-26 17:50:40 +00:00
terence tsao
5eece9a507 Integrate proto array forkchoice to run time (#4649)
* Run time

* Fixed pruning

* Fixed test

* Fixed test

* Process attestations during init sync

* Raul's feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-01-25 14:22:25 -06:00
Nishant Das
417480ffa8 fix bug (#4650) 2020-01-25 09:19:50 -08:00
Jim McDonald
dd5a3fe80d Update docs for keymanager (#4651) 2020-01-25 06:25:58 -08:00
Ivan Martinez
fa2acb3632 Improve E2E to be more consistent with timing, and allow for custom flags (#4620)
* Add committees helper, benchmark, results show 62ms for 8k validators which was previously 4 minutes

* Add regression test with same data

* fix epoch conversion

* lint

* undo and lint

* Begin work on adding mainnet config benchmark

* Try more to get mainnet e2e

* Try to fix delay

* Get past chainstart on e2e

* Try to fix flaky

* Get demo config working

* Remove unneeded changes

* Change how flags are enabled

* Lower shard count

* Temp skip

* Fix e2e

* Fix testing to run until last epoch

* Fix

* Add ending time log and remove att cache flag

* Fix ordering

* Reenable flag

* Change ports from default

* Add no log for if there are no err logs

* Add block evaluator

* Try to improve evaluators

* Progress on attestation evaluator

* Remove attestation evaluator

* Fix e2e

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-01-25 15:39:56 +08:00
Ivan Martinez
1562d3252b Allow ListBeaconCommittees API to return previous epoch (#4647)
* Allow committees API to request previous epoch

* Fix error log

* Fix previous epoch test
2020-01-24 19:54:18 -06:00
Preston Van Loon
10341cbf7f Add flags to cluster pk manager (#4645)
* Add flags to cluster pk manager
* Merge branch 'master' into cluster-pk-mgr
2020-01-24 19:34:24 +00:00
terence tsao
0f730b5887 Part 10 of proto array fork choice - Add Store (#4644) 2020-01-24 10:58:19 -08:00
terence tsao
b313b46f79 Part 9 of proto array fork choice - get head (#4643) 2020-01-24 10:15:01 -08:00
Jim McDonald
a78defcd26 Move to keymanager/keymanageropts command line parameters (#4590)
* Move to keymanager/keymanageropts command line parameters

* Add help for individual keymanagers

* gazelle

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-01-24 11:21:31 -06:00
terence tsao
86f6a44da6 Part 8 of proto array fork choice - prune (#4642)
* Docs
* Interface definitions
* Fmt and gazelle
* Rename interface to interfaces
* Define all the type for protoarray
* Gaz
* Add error types
* Add compute delta helper
* Compute delta tests
* Gaz
* Add checking if nodes viable
* Test for viable head
* Test for non leaf node can lead to viable head
* Conflict
* Extra space
* Remove fmt print
* Add updateBestChildAndDescendant
* Tests
* Merge branch 'master' into proto-array-forkchoice-6
* Conflict
* Merge branch 'proto-array-forkchoice-6' of git+ssh://github.com/prysmaticlabs/prysm into proto-array-forkchoice-6
* Add applyScoreChanges
* More test
* Rename score to weight
* Conflict
* Add insert function
* Test
* Merge refs/heads/master into proto-array-forkchoice-8
* Merge refs/heads/master into proto-array-forkchoice-8
* Merge branch 'master' of git+ssh://github.com/prysmaticlabs/prysm into proto-array-forkchoice-8
* Merge branch 'proto-array-forkchoice-8' of git+ssh://github.com/prysmaticlabs/prysm into proto-array-forkchoice-8
* Add prune method
* Tests
* Fixed long line
2020-01-24 15:51:11 +00:00
terence tsao
d978c19a41 Part 6 of proto array fork choice - update weight (#4636) 2020-01-23 20:32:27 -08:00
Preston Van Loon
588773cd0c Remove pubkey to validator ID map from validator (#4634)
* Remove map from validator
* remove failure check test
* fix
* Merge refs/heads/master into validator-fix
* Merge refs/heads/master into validator-fix
* Merge refs/heads/master into validator-fix
* Add error log if validator not found in committee
* Merge refs/heads/master into validator-fix
2020-01-24 01:50:07 +00:00
Preston Van Loon
62a5931843 Use a client side rate limit to reduce chance of getting banned (#4637)
* Use a client side rate limit to reduce chance of getting banned

* fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-01-23 17:33:43 -08:00
Preston Van Loon
6d6e8be10a Disable kafka build by default (#4638)
* Disable kafka build by default
2020-01-23 23:44:09 +00:00
terence tsao
144dcc3a69 Part 5 of proto array fork choice - update best child and descendant (#4629) 2020-01-23 14:23:45 -08:00
Jim McDonald
0f27343364 Fix deposit inclusion slot calculation (#4635) 2020-01-23 15:48:51 -05:00
terence tsao
3388ab74cf Part 4 of proto array fork choice - check nodes viable (#4625)
* Docs

* Interface definitions

* Fmt and gazelle

* Rename interface to interfaces

* Define all the type for protoarray

* Gaz

* Add error types

* Add compute delta helper

* Compute delta tests

* Gaz

* Add checking if nodes viable

* Test for viable head

* Test for non leaf node can lead to viable head

* Extra space

* Remove fmt print

* Update beacon-chain/forkchoice/protoarray/nodes.go

Co-Authored-By: Nishant Das <nishdas93@gmail.com>

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Nishant Das <nish1993@hotmail.com>
2020-01-23 08:33:39 -08:00
Andre Miras
663557e44e Fixes broken links to docs.prylabs.network (#4628) 2020-01-23 08:14:35 -06:00
shayzluf
5df77848bb Fix go pbs (#4626)
* fix issues

* fix go pbs

* added services

* added services

* remove unused files

* bring back used files

* bring back db proto files

* gaz

* gaz and bring back faucet

* gaz and bring back rpc

* gaz and bring back rpc

* gaz and bring back rpc

* go imports

* remove unused
2020-01-23 16:03:11 +05:30
Ivan Martinez
ed3ab828a1 Implement attester protection into validator client (#4598)
* Add flag for attester protection

* Remove flags

* Add attestation history DB functions to validator client

* Fix comments

* Update interface to new funcs

* Fix test

* Add flags

* Implement most of attester protection

* Fix tests

* Add test for pruning

* Add more test cases for prunes

* Remove todo comment

* Fix comments

* Rename functions

* Fix logs

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-01-23 00:52:01 -05:00
Nishant Das
ee9b9e69dc Fix Faucet (#4624)
* fix faucet

* preston's review
2020-01-23 12:13:27 +08:00
Nishant Das
460250251d add flag (#4622) 2020-01-23 11:25:10 +08:00
Preston Van Loon
4aa7ebc2b7 Wire voluntary exits pool (#4613)
* Hookup voluntary exits pool
* Merge refs/heads/master into wire-voluntary-exits
* Merge refs/heads/master into wire-voluntary-exits
* Merge refs/heads/master into wire-voluntary-exits
* Merge refs/heads/master into wire-voluntary-exits
* Merge refs/heads/master into wire-voluntary-exits
* Merge refs/heads/master into wire-voluntary-exits
* fix tests
* Merge branch 'wire-voluntary-exits' of github.com:prysmaticlabs/prysm into wire-voluntary-exits
* Merge refs/heads/master into wire-voluntary-exits
* gofmt
* Merge branch 'wire-voluntary-exits' of github.com:prysmaticlabs/prysm into wire-voluntary-exits
* gofmt
* gaz
* Merge refs/heads/master into wire-voluntary-exits
2020-01-22 22:27:44 +00:00
Tim Myers
a1e3c2d47c Add --p2p-host-dns flag to specify p2p external DNS (#4608)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-01-22 16:07:22 -06:00
Preston Van Loon
cc58b5aca6 Refactor block operations for validating exits slightly (#4612)
* Refactor block operations for validating exits slightly so that we don't have to advance state in a pubsub validator

* current slot

* remove duplicated validation for exits

* nil request check

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-01-22 13:49:38 -08:00
Jim McDonald
9a395530b7 Tidy up error logging (#4609)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-01-22 15:12:49 -06:00
terence tsao
5cc6de9e67 Part 3 of proto array fork choice - compute delta helper (#4617)
* Docs

* Interface definitions

* Fmt and gazelle

* Rename interface to interfaces

* Define all the type for protoarray

* Gaz

* Add error types

* Add compute delta helper

* Compute delta tests

* Gaz

* Fix formatting and comments

* Apply suggestions from code review

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
2020-01-22 14:19:52 -06:00
terence tsao
c041403a50 Part 2 of proto array fork choice - proto array types (#4616)
* Docs

* Interface definitions

* Fmt and gazelle

* Rename interface to interfaces

* Define all the type for protoarray

* Gaz
2020-01-22 13:12:41 -05:00
terence tsao
8d889f169e Part 1 of proto array fork choice - docs and interfaces (#4615)
* Docs

* Interface definitions

* Fmt and gazelle

* Rename interface to interfaces
2020-01-22 10:50:16 -06:00
shayzluf
b030771174 Slasher span cache (#4388)
* first version

* cli context

* fix service

* starting change to ccache

* ristretto cache

* added test

* test on evict

* remove evict test

* test onevict

* comment for exported flag

* update all span maps on load

* fix setup db

* span cache added to help flags

* start save cache on exit

* save cache to db before close

* comment fix

* fix flags

* setup db new

* nishant feedback

* workspace cr

* lint fix

* fix calls

* start db

* fix test

* Update slasher/db/db.go

Co-Authored-By: Nishant Das <nishdas93@gmail.com>

* add flag

* nishant feedback

* export Config

* fix imports

* fix imports

* fix imports

* Update slasher/service/service.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Update slasher/service/service.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Update slasher/service/service.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Update slasher/service/service.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* remove mod print

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nish1993@hotmail.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-01-21 23:39:21 -06:00
Raul Jordan
abe679e90e Create New Beacon State Data Structure (#4602)
* begin state service

* begin on the state trie idea

* created beacon state structure

* add in the full clone getter

* return by value instead

* add all setters

* new state setters are being completed

* arrays roots exposed

*  close to finishing all these headerssss

* functionality complete

* added in proto benchmark test

* test for compatibility

* add test for compat

* comments fixed

* add clone

* add clone

* remove underlying copies

* make it immutable

* integrate it into chainservice

* revert

* wrap up comments for package

* address all comments and godocs

* address all comments

* clone the pending attestation properly

* properly clone remaining items

* tests pass fixed bug

* prevent nil pointer exceptions

* fixed up some bugs in the clone comparisons

Co-authored-by: Nishant Das <nish1993@hotmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-01-21 22:36:12 -06:00
Preston Van Loon
bfda29f2ad Implement voluntary exits pool (#4610) 2020-01-21 15:29:04 -08:00
Nishant Das
e96c2f4949 use proper bound (#4607) 2020-01-21 07:45:06 -08:00
Nishant Das
a52f9d4549 Save Deposit Data at Every Interval (#4606)
* save only every 100 logs
2020-01-21 06:07:12 +00:00
terence tsao
29a7a587cf Fix old markdown links (#4603)
* Fix old MD links
* Revert
* Merge branch 'master' into clean-up-old-mds
2020-01-21 03:30:35 +00:00
Preston Van Loon
27254ad362 Use a better skip slots cache with a lock around it for identical parallel ProcessSlots requests (#4597)
* Use a better skip slots cache with a lock around it for common requests
* Merge refs/heads/master into better-skip-slots-cache
* add test
* Merge branch 'better-skip-slots-cache' of github.com:prysmaticlabs/prysm into better-skip-slots-cache
* Merge refs/heads/master into better-skip-slots-cache
* exit process slots if the context expired
* Revert "exit process slots if the context expired"

This reverts commit 1430d8ab19.
* ensure validation has a pubsub timeout
* Merge refs/heads/master into better-skip-slots-cache
* PR feedback
* Merge branch 'better-skip-slots-cache' of github.com:prysmaticlabs/prysm into better-skip-slots-cache
2020-01-21 02:19:42 +00:00
Raul Jordan
eb5e814eb4 Disable Fork Choice Feature Flag (#4574) 2020-01-20 17:45:37 -08:00
Celeste Ariana Seberras
0a8dbaaabc Updated doc portal links (#4599) 2020-01-20 15:41:02 -08:00
Andre Miras
e65d98925b Updaes README.md expose docker port 13000 (#4596)
Port 13000 also needs to be exposed if to improve connectivity and
receive more peers, refs #4323.
Also updates the "Docker on Windows" instructions for consistency.
2020-01-20 14:30:40 -06:00
Nishant Das
781b7d6870 Don't Panic if 0 Peers are Left (#4594)
* log error
* Merge branch 'master' into dontReturnError
* return blocks
* change back
* Merge branch 'dontReturnError' of https://github.com/prysmaticlabs/geth-sharding into dontReturnError
* use a more static finalized epoch
* Update beacon-chain/sync/initial-sync/round_robin.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>
* Merge refs/heads/master into dontReturnError
* jim's review
* Merge branch 'dontReturnError' of https://github.com/prysmaticlabs/geth-sharding into dontReturnError
* Update beacon-chain/sync/initial-sync/round_robin.go
2020-01-20 17:12:28 +00:00
Jim McDonald
dc4c1ca2b7 Ensure initial sync is initialised (#4587)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-01-20 10:38:27 -06:00
Andre Miras
d72e18ba60 Removes trailing backslash, refs #4562 (#4592)
* Removes trailing backslash, refs #4562
* Merge branch 'master' into feature/minor_documentation_fix
2020-01-19 22:46:19 +00:00
Ivan Martinez
a4db560e55 Prepare validator DB for attester protection implementation (#4584)
* Add flag for attester protection
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into protecc-attester-db
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into protecc-attester-db
* Remove flags
* Add attestation history DB functions to validator client
* Fix comments
* Update interface to new funcs
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into protecc-attester-db
* Fix test
* Merge branch 'master' into protecc-attester-db
2020-01-19 22:05:48 +00:00
Preston Van Loon
e7ecd9329a Fix resync (#4585)
* reset synced to false
* comment
2020-01-19 03:29:08 +00:00
Nishant Das
3e7e447160 Make Status Requests Asynchronous (#4577)
* make rpc status requests async
* make whole block async
* fix nogo
* Update beacon-chain/sync/rpc_status.go
* Merge branch 'master' into makeAsync
* Merge refs/heads/master into makeAsync
2020-01-19 01:37:18 +00:00
Nishant Das
1b62e92159 Reset Status (#4576)
* reset status

* Update beacon-chain/powchain/service.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-01-19 09:24:19 +08:00
Preston Van Loon
aae27749d4 Release save deposits flag (#4581)
* release save deposits flag
* Merge refs/heads/master into release-save-deposits
2020-01-18 20:25:29 +00:00
Ivan Martinez
2ba81193b0 Add proto for attestation protection (#4579) 2020-01-18 14:46:33 -05:00
Preston Van Loon
68c1ca755d Don't mark peer as bad as part of this return. (#4575) 2020-01-18 12:46:12 +08:00
Preston Van Loon
ccfc650375 Better parent block request (#4572)
* Use a good peer instead of a random one, if we know about it
* Exit init sync if there is an issue
* Merge refs/heads/master into better-parent-block-processing
* Merge refs/heads/master into better-parent-block-processing
* Merge refs/heads/master into better-parent-block-processing
* Merge refs/heads/master into better-parent-block-processing
* Update pending_blocks_queue.go
2020-01-17 22:43:32 +00:00
terence tsao
3d3dccbdb4 Enabled proposer sig and randao verifications for init sync (#4573)
* Enabled proposer sig and randao verifications in init sync

* Comments

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-01-17 15:58:25 -06:00
Ivan Martinez
d04399ea96 Refactor generated benchmark files to allow for more general usage (#4436)
* Begin to refactor benchmark files to testutil

* Complete most of refactoring

* Fix file path

* gofmt

* Fix path

* Move generatego to tools/

* Move gen util to tools/benchmark-files-gen

* Add comments to pregen funcs

* Make function names consistent

* Update README

* Redo benchmarks with 16384 validators
2020-01-17 12:25:35 -05:00
Prince Sinha
0605118686 p2p: Added log for --p2p-host-ip (#4553)
* added log for external addr
* Merge branch 'master' into log-p2p-address
* Merge branch 'master' into log-p2p-address
* Merge branch 'master' into log-p2p-address
2020-01-17 11:07:37 +00:00
Jim McDonald
dab87ba252 Add --rpc-host option to beacon chain (#4571) 2020-01-16 20:18:26 -06:00
Raul Jordan
eb429ab719 Include Validator Index in GetDuties Response, Update EthereumAPIs (#4567)
* include new patch
* add patch and validator indices to duties resp
* test passing
* move call to validator index
* Merge branch 'master' into include-val-idx
* do not use wait groups anymore
* Merge branch 'include-val-idx' of github.com:prysmaticlabs/prysm into include-val-idx
* Update beacon-chain/rpc/validator/assignments_test.go
2020-01-16 22:37:51 +00:00
Raul Jordan
ed529965af Fix Up SSZ Cache Branch Recomputation (#4558)
* e2e ssz cache busting
* use ssz cache for e2e
* caching ensure
* fix up the cache even more
* gazelle
* formatting
* formatting
* add back cache for val registry
* sync
* fix up commented item
* add attestations
* Merge branch 'master' into e2e-ssz-cache
* Merge branch 'master' into e2e-ssz-cache
* Merge branch 'e2e-ssz-cache' of github.com:prysmaticlabs/prysm into e2e-ssz-cache
* formatting
* gaz
* Merge branch 'master' into e2e-ssz-cache
* resolve comments
* Merge branch 'master' into e2e-ssz-cache
* naming of test
* Merge refs/heads/master into e2e-ssz-cache
* Merge refs/heads/master into e2e-ssz-cache
2020-01-16 21:40:09 +00:00
Raul Jordan
c6343cac3a Enable RPCMaxPageSize via Beacon Node Flag (#4539)
* add new flag
* enforce max page size via flag
* ensure exists in flag group
* Merge refs/heads/master into custom-max-page
* Merge refs/heads/master into custom-max-page
* Merge refs/heads/master into custom-max-page
* Merge refs/heads/master into custom-max-page
* Merge refs/heads/master into custom-max-page
* Merge refs/heads/master into custom-max-page
* Merge refs/heads/master into custom-max-page
* Merge refs/heads/master into custom-max-page
* Merge refs/heads/master into custom-max-page
* Merge refs/heads/master into custom-max-page
* Merge refs/heads/master into custom-max-page
* conflict with master
* resolved broken tests
* Update beacon-chain/flags/config.go
* Merge refs/heads/master into custom-max-page
2020-01-16 21:19:43 +00:00
Jim McDonald
06bc80d314 Add bad peer count (#4537)
* Add bad peer count
* Merge branch 'master' into badpeercount
* Merge branch 'master' into badpeercount
* Merge branch 'master' into badpeercount
* Merge branch 'master' into badpeercount
2020-01-16 21:07:09 +00:00
Nishant Das
60cab2dc73 update archive (#4443)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-01-16 12:37:04 -08:00
Prince Sinha
63d692a833 Fix deposit block slot before genesis state (#4495)
* before genesis state commit
* Merge branch 'master' into deposit-block-slot
* Merge branch 'master' into deposit-block-slot
* depositBlockSlot test added
* Merge branch 'deposit-block-slot' of https://github.com/princesinha19/prysm into deposit-block-slot
* Merge branch 'master' into deposit-block-slot
* Merge branch 'master' into deposit-block-slot
* resolve conflict
* status test commit
* Merge branch 'master' into deposit-block-slot
* Merge branch 'master' into deposit-block-slot
2020-01-16 19:38:30 +00:00
Jim McDonald
d744aaa2cd Better resync checking and running (#4516)
* Separate out fallen behind/resync check
* Remove hard-coded resync interval
* Merge branch 'master' into resync
* Merge branch 'master' into resync
* Merge branch 'master' into resync
* Merge branch 'master' into resync
* Merge branch 'master' into resync
* Merge branch 'master' into resync
* Merge branch 'master' into resync
* Merge branch 'master' into resync
* Merge branch 'master' into resync
2020-01-16 16:57:38 +00:00
JoshSnider
91d5ffae5b Remove invalid init-sync-no-verify option (#4562)
* Remove invalid `init-sync-no-verify` option

`init-sync-no-verify` was removed from `beacon-chain` in 32245a9062
* Merge branch 'master' into patch-1
2020-01-16 16:01:44 +00:00
terence tsao
cb49544fe3 Efficiently add proposer indices to cache (#4548)
* Use UpdateProposerIndicesInCache
* Merge branch 'master' into improve-proposer-cache
* Merge branch 'master' into improve-proposer-cache
* Merge branch 'master' into improve-proposer-cache
* Merge branch 'master' into improve-proposer-cache
2020-01-16 15:03:49 +00:00
Nishant Das
11731c4afe Fix RPC Panic (#4564) 2020-01-16 06:47:55 -08:00
Jim McDonald
5349b00e19 Tidy up peer logging (#4536)
* Tidy up peer logging
* Merge branch 'master' into peerlogs
* Merge branch 'master' into peerlogs
* Merge branch 'master' into peerlogs
2020-01-16 09:43:10 +00:00
Nishant Das
2e5429c94e Fix Stuck Beacon Node (#4454)
* Revert "Revert #4392 (#4449)"

This reverts commit 67c380b197.
* bound start req
* Merge refs/heads/master into revert-4449-revert-4392
* fix test
* Merge branch 'revert-4449-revert-4392' of https://github.com/prysmaticlabs/geth-sharding into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* add flag for deployment block
* Merge branch 'revert-4449-revert-4392' of https://github.com/prysmaticlabs/geth-sharding into revert-4449-revert-4392
* use constant and comments
* lint
* skip test for now
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Update shared/params/config.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>
* Update beacon-chain/powchain/testing/mock.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>
* preston's review
* Merge branch 'revert-4449-revert-4392' of https://github.com/prysmaticlabs/geth-sharding into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* add flag
* Merge branch 'revert-4449-revert-4392' of https://github.com/prysmaticlabs/geth-sharding into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* use stateutils
* Merge branch 'revert-4449-revert-4392' of https://github.com/prysmaticlabs/geth-sharding into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
* Merge refs/heads/master into revert-4449-revert-4392
2020-01-16 01:46:15 +00:00
Nishant Das
0a632064d4 Fix Powchain Status (#4560)
* reset status
* Merge branch 'master' into fixStatus
* Merge refs/heads/master into fixStatus
2020-01-16 01:34:27 +00:00
Preston Van Loon
129bc763ee Rate limiter for rpc beacon blocks (#4549)
* use rate limiter for rpc beacon blocks

* gofmt

* don't delete empty buckets

* disconnect bad peers

* tell peer they are being rate limited

* defer disconnect

* fix tests

* set burst to x10

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-01-15 17:19:06 -08:00
Ivan Martinez
452cadc286 Cleanup featureconfig, make naming consistent (#4557)
* Cleanup featureconfig, make naming consistent

* Fix rename

* Change test package name
2020-01-15 18:41:40 -05:00
Ivan Martinez
2a4c89827d Add double proposal protection to validator client (#4460)
* Add double proposal protection to client

* Add mock test cases for past proposals, and after pruning

* Fix error

* Add force clear db to val in e2e

* Fix val tests

* Move saving proposal history to after broadcasting block

* Add featureflag

* Goimports

* Unexport flag

* Add flag to tests

* gazelle

* Move conditionals
2020-01-15 17:23:39 -05:00
terence tsao
5ab1efb537 Cached head root retrieve from DB on miss (#4552) 2020-01-14 21:02:02 -08:00
Preston Van Loon
d0793f00c5 Partially revert #4477 (#4550)
* Partially revert #4477
2020-01-15 00:29:02 +00:00
Preston Van Loon
d8d9f4482f p2p: Increment RPC metrics (#4547)
* Increment RPC metrics
* Merge refs/heads/master into rpc-metric
2020-01-14 17:02:50 +00:00
Nishant Das
4835ba7bdf Increment Metric at the Start of Validation (#4546)
* shift metric correctly
2020-01-14 16:49:15 +00:00
terence tsao
6ef1a712c2 OnBlockCacheFilteredTree (#4541) 2020-01-14 08:05:22 -08:00
Nishant Das
0bee1de486 Set Capacity for Slices (#4540)
* set capacities

* make it more accurate

* resolve it
2020-01-14 14:44:24 +08:00
Raul Jordan
d2d4e7e35d Benchmark and Optimize ListValidatorBalances (#4530)
* add balances api bench
* rename
* fix flakey test with sharding
* Merge branch 'master' into optimize-api
* optimizing the reqs for pagination
* Merge branch 'optimize-api' of github.com:prysmaticlabs/prysm into optimize-api
* Merge branch 'master' into optimize-api
* wrap up tests
* Merge branch 'optimize-api' of github.com:prysmaticlabs/prysm into optimize-api
* nishant comment
* Update beacon-chain/rpc/beacon/validators.go
2020-01-14 05:40:20 +00:00
Nishant Das
6e0248429f Fix Activation Queue (#4535)
* change operator
* Merge branch 'master' into fixQueue
* Merge refs/heads/master into fixQueue
* Merge refs/heads/master into fixQueue
* Merge refs/heads/master into fixQueue
* add test and fix issue
* Merge branch 'fixQueue' of https://github.com/prysmaticlabs/geth-sharding into fixQueue
2020-01-14 04:35:51 +00:00
terence tsao
884d2a159d Cache proposer indices (#4528)
* Precompute and plug it into run time
* Run time fix
* Testing
* More logging to debug
* More logging to debug
* This should fix it
* Clean up debug logs
* Removed last bit of debug log
* Comments
* Tests
* Merge branch 'master' into cache-proposer-index
* Gaz
* Merge branch 'cache-proposer-index' of git+ssh://github.com/prysmaticlabs/prysm into cache-proposer-index
* Merge refs/heads/master into cache-proposer-index
* Merge refs/heads/master into cache-proposer-index
* Merge refs/heads/master into cache-proposer-index
2020-01-14 04:08:32 +00:00
Preston Van Loon
415af93ad8 Minor tweaks to GetAttestationData (#4533)
* Maybe bugfix

* Maybe bugfix

* make GetAttestationData cheaper

* clone head state getter return values

* Fix tests

* fix e2e and revert most changes 😩

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-01-13 19:28:08 -08:00
shayzluf
0b35743d2c Attester proposer slashing store (#4315)
* Merge branch 'master' of github.com:prysmaticlabs/prysm into update_validators

# Conflicts:
#	slasher/flags/flags.go
#	slasher/main.go
#	slasher/service/data_update.go
#	slasher/service/service.go
#	slasher/service/service_test.go

* proposal and attester store

* day to status

* comment change

* one bucket

* Merge branch 'master' of github.com:prysmaticlabs/prysm into attester_proposer_slashing_store
# Please enter a commit message to explain why this merge is necessary,
# especially if it merges an updated upstream into a topic branch.
#
# Lines starting with '#' will be ignored, and an empty message aborts
# the commit.

added comments

* comment

* typo fix

* raul review fix

* raul review fix full

* nishant feedback

* test fix

* fix tests and remove update gofmt goimports

* remove blank line in imports

* nishant fixes

* comment and fir delete proposer slashings

* avoid marshal twice

* remove space

* Update slasher/db/attester_slashings.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* terence feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-01-14 07:43:25 +05:30
Preston Van Loon
62811e8f7c Only advance to the correct epoch (#4532) 2020-01-13 15:34:05 -08:00
Preston Van Loon
d4ae063ad7 Revert "Filter attestation with ProcessAttestationNoSignatureVerify" (#4529)
* Revert "Filter attestation with ProcessAttestationNoSignatureVerify (#4513)"

This reverts commit 22e01aa9f2.
2020-01-13 21:02:13 +00:00
Jim McDonald
3d24a85121 Tidy-up of BestFinalized (#4505)
* Tidy up BestFinalized
* Ensure no more than maxPeers returned
* Merge branch 'master' into bestfinalized
* Merge branch 'master' into bestfinalized
* Merge branch 'master' into bestfinalized
* Merge branch 'master' into bestfinalized
* Remove swap file
* Provide potential PIDs array with capacity
* Add test for trimming and ordering in BestFinalized
* Merge branch 'master' into bestfinalized
2020-01-13 18:12:10 +00:00
Prince Sinha
888e8925ee cli: Added flag for GRPC max msg size (#4524)
* added grpc max msg size flag
* Merge branch 'master' into grpc-cli-flag
* Merge branch 'master' into grpc-cli-flag
* Merge branch 'master' into grpc-cli-flag
* Merge branch 'master' into grpc-cli-flag
2020-01-13 17:29:43 +00:00
Preston Van Loon
18333293d0 Refactor database interface to prefer blockchain.HeadFetcher (#4523)
* start refactoring and deprecation round 1
* Merge branch 'master' of github.com:prysmaticlabs/prysm into single-source-of-truth-1
* Refactoring of database interface. Preferring limited access interface
* revert some changes from 008f992993
* Fix tests
* gofmt
* Merge branch 'master' into single-source-of-truth-1
* lint
* Merge refs/heads/master into single-source-of-truth-1
* Apply suggestions from code review

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>
* Merge refs/heads/master into single-source-of-truth-1
* Merge refs/heads/master into single-source-of-truth-1
* Merge refs/heads/master into single-source-of-truth-1
* Clone head block to avoid mutation
2020-01-13 17:02:20 +00:00
Nishant Das
e286069b20 Check Block Before Processing it (#4527)
* fix panic
* Update beacon-chain/sync/pending_blocks_queue.go
2020-01-13 15:09:22 +00:00
Jim McDonald
de2f1fbf5c Ignore VI swapfiles (#4525) 2020-01-13 06:43:14 -08:00
Jim McDonald
44fa2c6371 Only one handshake at a time with active peers (#4519) 2020-01-13 18:15:09 +08:00
terence tsao
a8edfa42cc Cache filtered block tree (#4515)
* Cache filtered block tree
* Merge refs/heads/master into cache-filtered-tree
* Merge refs/heads/master into cache-filtered-tree
* Add locks
* Merge branch 'cache-filtered-tree' of git+ssh://github.com/prysmaticlabs/prysm into cache-filtered-tree
* Confligt
* Merge refs/heads/master into cache-filtered-tree
* Merge refs/heads/master into cache-filtered-tree
* Update shared/featureconfig/flags.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Rlock
* Merge branch 'master' of git+ssh://github.com/prysmaticlabs/prysm into cache-filtered-tree
* Merge branch 'cache-filtered-tree' of git+ssh://github.com/prysmaticlabs/prysm into cache-filtered-tree
2020-01-13 04:12:50 +00:00
terence tsao
7edca61e44 #4506 take two (#4518)
* Samething, testing run time
* Check if the latest processed block root is the same
* Merge branch 'master' into improve-receive-block-reattempt
* Merge refs/heads/master into improve-receive-block-reattempt
* Merge refs/heads/master into improve-receive-block-reattempt
2020-01-12 23:55:28 +00:00
Preston Van Loon
1cb0edac00 PR #4502 take two (#4522)
* PR #4502 take two
2020-01-12 23:46:14 +00:00
Raul Jordan
88bce4af34 Revert "Deprecate new cache feature flag" (#4520)
* Revert "Deprecate new cache feature flag (#4502)"

This reverts commit 5287ddc114.
2020-01-12 23:15:08 +00:00
terence tsao
5287ddc114 Deprecate new cache feature flag (#4502)
* Starting to deprecate new cache flag
* All tests passing
* Fixed minimal test
* Merge branch 'master' of git+ssh://github.com/prysmaticlabs/prysm into deprecate-flag
* Fixed mainnet spec test
* Fixed a typo
* Merge refs/heads/master into deprecate-flag
* Merge refs/heads/master into deprecate-flag
* Merge refs/heads/master into deprecate-flag
2020-01-12 22:46:30 +00:00
terence tsao
22e01aa9f2 Filter attestation with ProcessAttestationNoSignatureVerify (#4513)
* Use no sig verify and comment

* Fixed all tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-01-12 16:32:33 -06:00
terence tsao
a79dab7919 Revert "ReceiveBlock: Only retrieve head block from DB if necessary (#4506)" (#4514)
This reverts commit 9a4bf6c1a2.
2020-01-12 14:08:25 -08:00
Preston Van Loon
9a4bf6c1a2 ReceiveBlock: Only retrieve head block from DB if necessary (#4506)
* Only retrieve head block from DB if necessary

* remove redundant comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-01-12 15:13:13 -06:00
Preston Van Loon
7992375a0e Check if we are already synced to the current epoch before querying all of our peers (#4504) 2020-01-11 18:31:30 -08:00
terence tsao
6c4bf22723 Fix up attestation pool (#4493)
* Update aggregated methods

* Update aggregated methods

* Use improved HasAttestation to check caches

* Add back some validations

* There's no need to save unaggregated att

* Fixed all the tests

* remove TODO for now

* Raul feedback

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-01-11 13:59:06 -08:00
Jim McDonald
ea12ffabba Log Ethereum 1 deposits before chainstart (#4499)
* Log Ethereum 1 deposits before chainstart
* Merge branch 'master' into logdeposits
2020-01-11 19:54:43 +00:00
Preston Van Loon
5077c009c3 Only advance slot when the request is for a future epoch (#4501) 2020-01-11 11:40:36 -08:00
Jim McDonald
7f1900e96c Remove unused function (#4496)
* Remove unused function
* Merge branch 'master' into rmunused
2020-01-11 11:20:05 +00:00
Jim McDonald
3c5d5bfc7b Use helper to calculate epoch (#4497) 2020-01-11 19:06:10 +08:00
Nishant Das
4e6c8c5b1a Batch Save Genesis Validators (#4494)
* save vals
* Merge branch 'master' into batchSaveGenesisValidators
2020-01-11 04:40:31 +00:00
Preston Van Loon
7919074a6a Add a step filter for beacon DB to retrieve blocks (#4488)
* Add a step filter for beacon DB to retrieve blocks
* Add a step filter for beacon DB to retrieve blocks
* gofmt
* Merge branch 'master' into db-step-filter
* Merge refs/heads/master into db-step-filter
* Merge refs/heads/master into db-step-filter
* Merge refs/heads/master into db-step-filter
* Merge refs/heads/master into db-step-filter
* fix tests
* Merge branch 'db-step-filter' of github.com:prysmaticlabs/prysm into db-step-filter
* Merge refs/heads/master into db-step-filter
2020-01-11 01:32:45 +00:00
Nishant Das
f6eea8e1fa Optimize Archival Assignment Retrieval (#4480)
* optimize further
* remove func
* Merge branch 'master' into optimizeArchival
* Merge refs/heads/master into optimizeArchival
* Merge refs/heads/master into optimizeArchival
* Merge refs/heads/master into optimizeArchival
* Merge refs/heads/master into optimizeArchival
* Merge refs/heads/master into optimizeArchival
* Merge refs/heads/master into optimizeArchival
* raul's review
* Merge branch 'optimizeArchival' of https://github.com/prysmaticlabs/geth-sharding into optimizeArchival
* preston's review
2020-01-11 01:19:52 +00:00
terence tsao
45e6eccfb4 Add epoch filter for fork choice attestation (#4487)
* Filter target epoch
* Test
* Comment
* Merge branch 'master' into fix-target-epoch
* Merge refs/heads/master into fix-target-epoch
* Merge refs/heads/master into fix-target-epoch
* Merge refs/heads/master into fix-target-epoch
2020-01-10 23:51:49 +00:00
terence tsao
b6c6b9b776 Filter block tree verifies block root has state (#4490)
* Construct block tree ensures block root has state
* Merge refs/heads/master into filter-tree-has-state
2020-01-10 23:40:46 +00:00
Raul Jordan
1a9b0da9ae Batch Save Validator Indices (#4489)
* add bolt alloc fix
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* const
* add a batch save for indices to speed up sync
* Merge branch 'batch-save-indices' into bolt-alloc-fix
* fix up
* fix broken build
* Merge branch 'batch-save-indices' into bolt-alloc-fix
* Merge branch 'master' into batch-save-indices
* ensure it saves each
* Merge branch 'batch-save-indices' of github.com:prysmaticlabs/prysm into batch-save-indices
* Merge branch 'master' into batch-save-indices
* revert ssz cache
* Merge branch 'batch-save-indices' of github.com:prysmaticlabs/prysm into batch-save-indices
2020-01-10 23:27:01 +00:00
Raul Jordan
025be93492 Allocate More Resources to BoltDB (#4485)
* add bolt alloc fix
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* const
* Merge refs/heads/master into bolt-alloc-fix
2020-01-10 22:17:42 +00:00
Jim McDonald
22a3bf53ad Do not panic if initial sync fails (#4477)
* Do not panic if initial sync fails
* Only consider peers with non-0 finalized epoch
* Additional fixes
* Fix tests
* Merge branch 'master' into mvlog
* Merge branch 'master' into mvlog
2020-01-10 21:36:20 +00:00
terence tsao
37459ee765 Forkchoice att seen cache consider bitfield overlaps (#4483)
* Aggregate with previous aggregated attestations

* Update cache to bitfield as value

* Remove fmt print
2020-01-10 14:44:20 -06:00
terence tsao
e9d63e8dd3 Aggregate with previous aggregated attestations (#4478) 2020-01-10 10:56:28 -06:00
terence tsao
01b8a84e21 Check fork choice attestation's block and state in DB (#4475) 2020-01-10 06:25:43 -08:00
Preston Van Loon
9d8364bdfa only advance state in validate aggregate and proof if the epoch has changed between head state and attestation slot (#4474) 2020-01-09 19:11:02 -08:00
terence tsao
2b6a5aaaf9 Use db head info for request attestation (#4472) 2020-01-09 18:37:55 -08:00
Preston Van Loon
6aa92956f6 RPC: Use db.headBlock in getBlock (#4473)
* use headBlock in getBlock
* Merge branch 'master' into use-db-headblock
2020-01-10 02:00:36 +00:00
Preston Van Loon
eae2268dd1 DB: Prevent encoding a nil message (#4470)
* Prevent encoding a nil message
* Merge refs/heads/master into prevent-saving-nil-msg
2020-01-10 01:38:53 +00:00
Preston Van Loon
6de485c27e Use a longer deadline for processing pubsub messages (#4471)
* Use a longer deadline for processing pubsub messages
2020-01-10 01:27:20 +00:00
Jim McDonald
3839f577ec Change database *Index() to use slice (#4466)
* Change database *Index() to use slice

* Remove underscore from helper name
2020-01-09 14:45:05 -08:00
Jim McDonald
fc38a0413e Fix gauge description (#4468) 2020-01-09 11:39:11 -08:00
terence tsao
0dd0e23155 Optimize ListBeaconCommittees to use committees cache (#4464) 2020-01-09 08:28:11 -08:00
Nishant Das
699e1c8a23 Optimize List Validator Assignments (#4456)
* optimize
* Merge refs/heads/master into listValidatorAssingments
* Merge refs/heads/master into listValidatorAssingments
* Merge refs/heads/master into listValidatorAssingments
* Merge refs/heads/master into listValidatorAssingments
* Merge refs/heads/master into listValidatorAssingments
* Merge refs/heads/master into listValidatorAssingments
* Merge refs/heads/master into listValidatorAssingments
* Merge refs/heads/master into listValidatorAssingments
* Merge refs/heads/master into listValidatorAssingments
* terence's and preston's review
* Merge branch 'listValidatorAssingments' of https://github.com/prysmaticlabs/geth-sharding into listValidatorAssingments
2020-01-09 05:07:24 +00:00
shayzluf
39b2570af5 Beacon node slasher client infrastructure (#4111) 2020-01-08 20:49:32 -08:00
Preston Van Loon
200bb5e42a Docker: Make root user the default (#4461)
* make root user the default
* Merge branch 'master' into root-user
* Merge refs/heads/master into root-user
* Merge refs/heads/master into root-user
2020-01-08 19:52:59 +00:00
Preston Van Loon
d249f78d79 Update tool README.md (#4463)
* Update README.md
* Merge refs/heads/master into prestonvanloon-patch-2
2020-01-08 19:44:34 +00:00
Nishant Das
e110f038bc Add Back Eth1 Block Delay (#4458)
* add delay
* Merge refs/heads/master into addBackDelay
* Merge refs/heads/master into addBackDelay
* Merge refs/heads/master into addBackDelay
* Merge refs/heads/master into addBackDelay
* Merge refs/heads/master into addBackDelay
2020-01-08 19:36:20 +00:00
Jim McDonald
5ee79dc4a8 Log fork version mismatches at debug (#4457)
* Log fork version mismatches at debug
* Merge branch 'master' into synclogerrors
* Merge branch 'master' into synclogerrors
* Merge branch 'master' into synclogerrors
2020-01-08 19:18:21 +00:00
Ivan Martinez
4ab0a91e51 Validator Slashing Protection DB (#4389)
* Begin adding DB to validator client

Begin adding ValidatorProposalHistory

Implement most of proposal history

Finish tests

Fix marking a proposal for the first time

Change proposalhistory to not using bit shifting

Add pb.go

Change after proto/slashing added

Finally fix protos

Fix most tests

Fix all tests for double proposal protection

Start initialiing DB in validator client

Add db to validator struct

Add DB to ProposeBlock

Fix test errors and begin mocking

Fix test formatting and pass test for validator protection!

Fix merge issues

Fix renames

Fix tests

* Fix tests

* Fix first startup on DB

* Fix nil check tests

* Fix E2E

* Fix e2e flag

* Fix comments

* Fix for comments

* Move proposal hepers to validator/client to keep DB clean

* Add clear-db flag to validator client

* Fix formatting

* Clear out unintended changes

* Fix build issues

* Fix build issues

* Gazelle

* Fix mock test

* Remove proposal history

* Add terminal confirmation to DB clearing

* Add interface for validatorDB, add context to DB functions

* Add force-clear-db flag

* Cleanup

* Update validator/node/node.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* Change db to clear file, not whole folder

* Fix db test

* Fix teardown test

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-01-08 13:16:17 -05:00
terence tsao
a69cb5c6e4 Committee cache fuzz tests (#4459)
* Starting TestCommitteeKeyFuzz_OK

* Added fuzz tests for committeees by epoch and active indices
2020-01-08 11:13:39 -06:00
terence tsao
624c42421d Add HasAggregatedAttestation getter for pool (#4451) 2020-01-08 06:59:49 -08:00
Jim McDonald
f3ae67a94b Fix bad update in #4453 (#4455) 2020-01-08 21:54:44 +08:00
Ivan Martinez
b30a7d1e19 Fix typos and inconsistencies (#4453)
* Fix typos and inconsistencies

* igoimports

* Gazelle
2020-01-07 20:36:55 -06:00
Ivan Martinez
0d400faea2 Remove unused parameters and unused code (#4452)
* Remove unused parameters
* Remove unused deposit contract config
2020-01-07 23:45:29 +00:00
Preston Van Loon
67c380b197 Revert #4392 (#4449)
* revert #4392
2020-01-07 21:15:40 +00:00
terence tsao
89eedd2123 Efficient computation of epoch participation (#4430)
* Remove custody (#3986)

* Update proto fields

* Updated block operations

* Fixed all block operation tests

* Fixed tests part 1

* Fixed tests part 1

* All tests pass

* Clean up

* Skip spec test

* Fixed ssz test

* Skip ssz test

* Skip mainnet tests

* Update beacon-chain/operations/attestation.go

* Update beacon-chain/operations/attestation.go
* Decoy flip flop check (#3987)
* Bounce attack check (#3989)

* New store values

* Update process block

* Update process attestation

* Update tests

* Helper

* Fixed blockchain package tests

* Update beacon-chain/blockchain/forkchoice/process_block.go
* Conflict
* Unskip mainnet spec tests (#3998)

* Starting

* Fixed attestation mainnet test

* Unskip ssz static and block processing tests

* Fixed workspace

* fixed workspace

* fixed workspace

* Update beacon-chain/core/blocks/block_operations.go
* Unskip minimal spec tests (#3999)

* Starting

* Fixed attestation mainnet test

* Unskip ssz static and block processing tests

* Fixed workspace

* fixed workspace

* fixed workspace

* Update workspace

* Unskip all minimal spec tests

* Update workspace for general test
* Unskip test (#4001)
* Update minimal seconds per slot to 6 (#3978)
* Bounce attack tests (#3993)

* New store values

* Update process block

* Update process attestation

* Update tests

* Helper

* Fixed blockchain package tests

* Slots since epoch starts tests

* Update justified checkpt tests

* Conflict

* Fixed logic

* Update process_block.go

* Use helper
* Conflict
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.1
* Conflict
* Fixed failed tests
* Lower MinGenesisActiveValidatorCount to 16384 (#4100)
* Fork choice beacon block checks (#4107)

* Prevent future blocks check and test

* Removed old code
* Update aggregation proto (#4121)

* Update def
* Update spec test
* Conflict
* Update workspace
* patch
* Resolve conflict
* Patch
* Change workspace
* Update ethereumapis to a forked branch at commit 6eb1193e47
* Fixed all the tests
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into conflict
* fix patch
* Need to regenerate test data
* Merge branch 'master' into v0.9.2
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Enable snappy compression for all (#4157)

* enable snappy compression for all
* enable snappy compression for all
* enable snappy compression for all
* enable snappy compression for all
* Validate aggregate and proof subscriber (#4159)
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Conflict
* Update workspace
* Conflict
* Conflict
* Conflict
* Merge branch 'master' into v0.9.2
* Merge branch 'master' into v0.9.2
* Conflict
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Remove migrate to snappy  (#4205)
* Feature flag: Deprecate --prune-states, release to all (#4204)

* Deprecated prune-states, release to all

* imports

* remote unused import

* remove unused import

* Rm prune state test

* gaz
* Refactoring for dynamic pubsub subscriptions for non-aggregated attestations (#4189)

* checkpoint progress

* chkpt

* checkpoint progress

* put pipeline in its own file

* remove unused imports

* add test, it's failing though

* fix test

* remove head state issue

* add clear db flag to e2e

* add some more error handling, debug logging

* skip processing if chain has not started

* fix test

* wrap in go routine to see if anything breaks

* remove duplicated topic

* Add a regression test. Thanks @nisdas for finding the original problem. May it never happen again *fingers crossed*

* Comments

* gofmt

* comment out with TODO
* Sync with master
* Sync with master
* RPC servers use attestation pool (#4223)
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Refactor RPC to Fully Utilize Ethereum APIs (#4243)

* include attester as a file in the validator server

* remove old proposer server impl

* include new patch and properly sync changes

* align with public pbs

* ensure matches rpc def

* fix up status tests

* resolve all broken test files in the validator rpc package

* gazelle include

* fix up the duties implementation

* fixed up all get duties functions

* all tests pass

* utilize new ethereum apis

* amend validator client to use the new beacon node validator rpc client

* fix up most of validator items

* added in mock

* fix up test

* readd test

* add chain serv mock

* fix a few more validator methods

* all validator tests passingggg

* fix broken test

* resolve even more broken tests

* all tests passsssss

* fix lint

* try PR

* fix up test

* resolve broken other tests
* Sync with master
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Aggregate and proof subscriber (#4240)

* Added subscribers

* Fixed conflict

* Tests

* fix up patch

* Use upstream pb

* include latest patch

* Fmt

* Save state before head block
* skip tests (#4275)
* Delete block attestations from the pool (#4241)

* Added subscribers
* Clean up
* Fixed conflict
* Delete atts in pool in validate pipeline
* Moved it to subscriber
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into use-att-pool-3
* Test
* Fixed test
* Initial work on voluntary exit (#4207)

* Initial implementation of voluntary exit: RPC call

* Update for recent merges

* Break out validation logic for voluntary exits to core module

* RequestExit -> ProposeExit

* Decrease exit package visibility

* Move to operation feed

* Wrap errors
* Fix critical proposer selection bug #4259 (#4265)

* fix critical proposer selection bug #4259

* gofmt

* add 1 more validator to make it 5

* more tests

* Fixed archivedProposerIndex

* Fixed TestFilterAttestation_OK

* Refactor ComputeProposerIndex, add regression test for potential out of range panic

* handle case of nil validator

* Update validators_test.go
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Leftover merge files, oops
* gaz
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.9.2
* Fixes Duplicate Validator Bug (#4322)

* Update dict

* Test helper

* Regression test

* Comment

* Reset test cache
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* fixes after PR #4328
* Complete attestation pool for run time (#4286)

* Added subscribers

* Fixed conflict

* Delete atts in pool in validate pipeline

* Moved it to subscriber

* Test

* Fixed test

* New curl for forkchoice attestations

* Starting att pool service for fork choice

* Update pool interface

* Update pool interface

* Update sync and node

* Lint

* Gazelle

* Updated servers, filled in missing functionalities

* RPC working with 1 beacon node 64 validators

* Started writing tests. Yay

* Test to aggregate and save multiple fork choice atts

* Tests for BatchAttestations for fork choice

* Fixed exisiting tests

* Minor fixes

* Fmt

* Added batch saves

* Lint

* Mo tests yay

* Delete test

* Fmt

* Update interval

* Fixed aggregation broadcast

* Clean up based on design review comment

* Fixed setupBeaconChain

* Raul's feedback. s/error/err
* resolve conflicts
* Merge branch 'v0.9.2' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge refs/heads/master into v0.9.2
* Removed old protos and fixed tests (#4336)
* Merge refs/heads/master into v0.9.2
* Disallow duplicated indices and test (#4339)
* Explicit use of GENESIS_SLOT in fork choice (#4343)
* Update from 2 to 3 (#4345)
* Remove verify unaggregated attestation when aggregating (#4347)
* use slot ticker instead of run every (#4348)
* Add context check for unbounded loop work (#4346)
* Revert "Explicit use of GENESIS_SLOT in fork choice (#4343)" (#4349)

This reverts commit d3f6753c77.
* Refactor Powchain Service (#4306)

* add data structures

* generate proto

* add in new fields

* add comments

* add new mock state

* add new mock state

* add new methods

* some more changes

* check genesis time properly

* lint

* fix refs

* fix tests

* lint

* lint

* lint

* gaz

* fix lint

* raul's comments

* use one method

* fix test

* raul's comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Ensure best better-justification is stored for fork choice (#4342)

* Ensure best better-justification is stored. Minor refactor
* Tests
* Merge refs/heads/v0.9.2 into better-best-justified
* Merge refs/heads/v0.9.2 into better-best-justified
* Ensure that epoch of attestation slot matches the target epoch (#4341)

* Disallow duplicated indices and test
* Add slot to target epoch check to on_attestation
* Add slot to target epoch check to process_attestation
* Merge branch 'v0.9.2' of git+ssh://github.com/prysmaticlabs/prysm into no-dup-att-indices
* Fixed TestProcessAttestations_PrevEpochFFGDataMismatches
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Update beacon-chain/blockchain/forkchoice/process_attestation_test.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Filter viable branches in fork choice (#4355)
* Only activate upon finality (#4359)

* Updated functions
* Tests
* Merge branch 'v0.9.2' of git+ssh://github.com/prysmaticlabs/prysm into queue-fix-on-finality
* Comment
* Merge refs/heads/v0.9.2 into queue-fix-on-finality
* Fixed failing test from 4359 (#4360)

* Fixed
* Skip registry spec tests
* Wait for state to be initialized at least once before running slot ticker based on genesis time (#4364)
* Sync with master
* Fix checkpoint root to  use genesis block root (#4368)
* Return an error on nil head state in fork choice (#4369)

* Return error if nil head state

* Fixed tests. Saved childen blocks state

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Update metrics every epoch (#4367)
* return empty slice if state is nil (#4365)
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge refs/heads/master into v0.9.2
* Pubsub: Broadcast attestations to committee based subnets (#4316)

* Working on un-aggregated pubsub topics

* update subscriber to call pool

* checkpointing

* fix

* untested message validation

* minor fixes

* rename slotsSinceGenesis to slotsSince

* some progress on a unit test, subscribe is not being called still...

* dont change topic

* need to set the data on the message

* restore topic

* fixes

* some helpful parameter changes for mainnet operations

* lint

* Terence feedback

* unskip e2e

* Unit test for validate committee index beacon attestation

* PR feedbacK

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into resolveConflicts
* remove condition
* Remove unused operation pool (#4361)
* Merge refs/heads/master into v0.9.2
* Aggregate attestations periodically  (#4376)
* Persist ETH1 Data to Disk (#4329)

* add data structures

* generate proto

* add in new fields

* add comments

* add new mock state

* add new mock state

* add new methods

* some more changes

* check genesis time properly

* lint

* fix refs

* fix tests

* lint

* lint

* lint

* gaz

* adding in new proto message

* remove outdated vars

* add new changes

* remove latest eth1data

* continue refactoring

* finally works

* lint

* fix test

* fix all tests

* fix all tests again

* fix build

* change back

* add full eth1 test

* fix logs and test

* add constant

* changes

* fix bug

* lint

* fix another bug

* change back

* Apply suggestions from code review

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Fixed VerifyIndexedAttestation (#4382)
* rm signing root (#4381)

* rm signing root

* Fixed VerifyIndexedAttestation

* Check proposer slashed status inside ProcessBlockHeaderNoVerify

* Fixed TestUpdateJustified_CouldUpdateBest

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Remove Redundant Trie Generation (#4383)

* remove trie generation
* remove deposit hashes
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.9.2
* fix build
* Conflict
* Implement StreamAttestations RPC Endpoint (#4390)

* started attestation stream

* stream attestations test

* on slot tick test passing

* imports

* gaz

* Update beacon-chain/rpc/beacon/attestations_test.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

Co-authored-by: shayzluf <thezluf@gmail.com>
* Fixed goimport (#4394)
* Use custom stateutil ssz for ssz HTR spec tests (#4396)

* Use custom stateutil ssz for ssz HTR spec tests

* gofmt
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge refs/heads/master into v0.9.2
* set mainnet to be the default for build and run (#4398)

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* gracefully handle deduplicated registration of topic validators (#4399)

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* SSZ: temporarily disable roots cache until cache issues can be resolved (#4407)

* temporarily disable roots cache until cache issues can be resolved

* Also use custom ssz for spectests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Remove process block attestations as separate routine (#4408)

* Removed old save/process block atts

* Fixed tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Save Deposit Cache to Disk (#4384)

* change to protos

* fix build

* glue everything together

* fix test

* raul's review

* preston's comments

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Fix activation queue sorting (#4409)

* Removed old save/process block atts

* Fixed tests

* Proper sorting by eligibility epoch then by indices

* Deleted old colde
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge branch 'master' into v0.9.2
* Merge refs/heads/master into v0.9.2
* stop recursive lookup if context is cancelled (#4420)
* Fix proposal bug (#4419)
* Add Pending Deposits Safely (#4422)

* safely prune cache

* use proper method

* preston's,terence's reviews and comments

* revert change to build files

* use as feature config instead
* Release custom state ssz (#4421)

* Release custom state ssz, change all HTR of beacon state to use custom method

* typo

* use mainnet config

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Define framework
* Use participation fetcher
* Build
* Fixed all tests
* Lint
* Update initial sync save justified to align with v0.9.3 (#4432)
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* fix build
* don't blacklist on pubsub (#4435)
* Fix Flakey Slot Ticker Test (#4434)

* use interface instead for the slot ticker

* fixed up flakey tests

* add gen time

* get duties comment

* fix lifecycle test

* more fixes
* Fixed rest of the test
* Pass in correct chain service
* Pass in another chain service
* Run time
* Configurable min genesis delay (#4437)

* Configurable min genesis delay based on https://github.com/ethereum/eth2.0-specs/pull/1557

* remove feature flag for genesis delay

* fix

* demo config feedback
* Current -> Prev
* Tests
* patch readme
* save keys unencrypted for validators (#4439)
* Add new demo configuration targeting mainnet scale (#4397)

* Add new demo configuration targeting mainnet, with 1/10th of the deposit value

* reduce quotant by 1/10th. Use 1/10th mainnet values

* only change the inactivity quotant

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Save justified checkpoint state (#4433)

* Save justified checkpoint state

* Lint

* Feedback

* Fixed test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Update shared/testutil/deposits.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update proto/testing/ssz_regression_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/core/epoch/epoch_processing.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/kv/forkchoice.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/pool.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/pool.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/powchain/log_processing_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/service.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber_beacon_blocks_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber_beacon_blocks_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/proposer.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/prepare_forkchoice.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/powchain/log_processing_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/pool.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/powchain/log_processing_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/aggregator/server.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/cache/depositcache/pending_deposits.go
* Update beacon-chain/cache/depositcache/pending_deposits_test.go
* Update beacon-chain/rpc/validator/proposer.go
* Merge refs/heads/master into v0.9.2
* Update test
* Conflict
* Update beacon-chain/blockchain/chain_info.go
* Conflict
* Merge branch 'efficient-participation' of git+ssh://github.com/prysmaticlabs/prysm into efficient-participation
* Merge refs/heads/master into efficient-participation
2020-01-07 19:28:25 +00:00
Preston Van Loon
2182e1cdc9 Fix pk manager db (#4447)
* fix pk manager db
2020-01-07 19:19:40 +00:00
terence tsao
6d2a2ebadf Update run time to v0.9.3 (#4154)
* Remove custody (#3986)

* Update proto fields

* Updated block operations

* Fixed all block operation tests

* Fixed tests part 1

* Fixed tests part 1

* All tests pass

* Clean up

* Skip spec test

* Fixed ssz test

* Skip ssz test

* Skip mainnet tests

* Update beacon-chain/operations/attestation.go

* Update beacon-chain/operations/attestation.go
* Decoy flip flop check (#3987)
* Bounce attack check (#3989)

* New store values

* Update process block

* Update process attestation

* Update tests

* Helper

* Fixed blockchain package tests

* Update beacon-chain/blockchain/forkchoice/process_block.go
* Conflict
* Unskip mainnet spec tests (#3998)

* Starting

* Fixed attestation mainnet test

* Unskip ssz static and block processing tests

* Fixed workspace

* fixed workspace

* fixed workspace

* Update beacon-chain/core/blocks/block_operations.go
* Unskip minimal spec tests (#3999)

* Starting

* Fixed attestation mainnet test

* Unskip ssz static and block processing tests

* Fixed workspace

* fixed workspace

* fixed workspace

* Update workspace

* Unskip all minimal spec tests

* Update workspace for general test
* Unskip test (#4001)
* Update minimal seconds per slot to 6 (#3978)
* Bounce attack tests (#3993)

* New store values

* Update process block

* Update process attestation

* Update tests

* Helper

* Fixed blockchain package tests

* Slots since epoch starts tests

* Update justified checkpt tests

* Conflict

* Fixed logic

* Update process_block.go

* Use helper
* Conflict
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.1
* Conflict
* Fixed failed tests
* Lower MinGenesisActiveValidatorCount to 16384 (#4100)
* Fork choice beacon block checks (#4107)

* Prevent future blocks check and test

* Removed old code
* Update aggregation proto (#4121)

* Update def
* Update spec test
* Conflict
* Update workspace
* patch
* Resolve conflict
* Patch
* Change workspace
* Update ethereumapis to a forked branch at commit 6eb1193e47
* Fixed all the tests
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into conflict
* fix patch
* Need to regenerate test data
* Merge branch 'master' into v0.9.2
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Enable snappy compression for all (#4157)

* enable snappy compression for all
* enable snappy compression for all
* enable snappy compression for all
* enable snappy compression for all
* Validate aggregate and proof subscriber (#4159)
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Conflict
* Update workspace
* Conflict
* Conflict
* Conflict
* Merge branch 'master' into v0.9.2
* Merge branch 'master' into v0.9.2
* Conflict
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Remove migrate to snappy  (#4205)
* Feature flag: Deprecate --prune-states, release to all (#4204)

* Deprecated prune-states, release to all

* imports

* remote unused import

* remove unused import

* Rm prune state test

* gaz
* Refactoring for dynamic pubsub subscriptions for non-aggregated attestations (#4189)

* checkpoint progress

* chkpt

* checkpoint progress

* put pipeline in its own file

* remove unused imports

* add test, it's failing though

* fix test

* remove head state issue

* add clear db flag to e2e

* add some more error handling, debug logging

* skip processing if chain has not started

* fix test

* wrap in go routine to see if anything breaks

* remove duplicated topic

* Add a regression test. Thanks @nisdas for finding the original problem. May it never happen again *fingers crossed*

* Comments

* gofmt

* comment out with TODO
* Sync with master
* Sync with master
* RPC servers use attestation pool (#4223)
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Refactor RPC to Fully Utilize Ethereum APIs (#4243)

* include attester as a file in the validator server

* remove old proposer server impl

* include new patch and properly sync changes

* align with public pbs

* ensure matches rpc def

* fix up status tests

* resolve all broken test files in the validator rpc package

* gazelle include

* fix up the duties implementation

* fixed up all get duties functions

* all tests pass

* utilize new ethereum apis

* amend validator client to use the new beacon node validator rpc client

* fix up most of validator items

* added in mock

* fix up test

* readd test

* add chain serv mock

* fix a few more validator methods

* all validator tests passingggg

* fix broken test

* resolve even more broken tests

* all tests passsssss

* fix lint

* try PR

* fix up test

* resolve broken other tests
* Sync with master
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Aggregate and proof subscriber (#4240)

* Added subscribers

* Fixed conflict

* Tests

* fix up patch

* Use upstream pb

* include latest patch

* Fmt

* Save state before head block
* skip tests (#4275)
* Delete block attestations from the pool (#4241)

* Added subscribers
* Clean up
* Fixed conflict
* Delete atts in pool in validate pipeline
* Moved it to subscriber
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into use-att-pool-3
* Test
* Fixed test
* Initial work on voluntary exit (#4207)

* Initial implementation of voluntary exit: RPC call

* Update for recent merges

* Break out validation logic for voluntary exits to core module

* RequestExit -> ProposeExit

* Decrease exit package visibility

* Move to operation feed

* Wrap errors
* Fix critical proposer selection bug #4259 (#4265)

* fix critical proposer selection bug #4259

* gofmt

* add 1 more validator to make it 5

* more tests

* Fixed archivedProposerIndex

* Fixed TestFilterAttestation_OK

* Refactor ComputeProposerIndex, add regression test for potential out of range panic

* handle case of nil validator

* Update validators_test.go
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Leftover merge files, oops
* gaz
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.9.2
* Fixes Duplicate Validator Bug (#4322)

* Update dict

* Test helper

* Regression test

* Comment

* Reset test cache
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* fixes after PR #4328
* Complete attestation pool for run time (#4286)

* Added subscribers

* Fixed conflict

* Delete atts in pool in validate pipeline

* Moved it to subscriber

* Test

* Fixed test

* New curl for forkchoice attestations

* Starting att pool service for fork choice

* Update pool interface

* Update pool interface

* Update sync and node

* Lint

* Gazelle

* Updated servers, filled in missing functionalities

* RPC working with 1 beacon node 64 validators

* Started writing tests. Yay

* Test to aggregate and save multiple fork choice atts

* Tests for BatchAttestations for fork choice

* Fixed exisiting tests

* Minor fixes

* Fmt

* Added batch saves

* Lint

* Mo tests yay

* Delete test

* Fmt

* Update interval

* Fixed aggregation broadcast

* Clean up based on design review comment

* Fixed setupBeaconChain

* Raul's feedback. s/error/err
* resolve conflicts
* Merge branch 'v0.9.2' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge refs/heads/master into v0.9.2
* Removed old protos and fixed tests (#4336)
* Merge refs/heads/master into v0.9.2
* Disallow duplicated indices and test (#4339)
* Explicit use of GENESIS_SLOT in fork choice (#4343)
* Update from 2 to 3 (#4345)
* Remove verify unaggregated attestation when aggregating (#4347)
* use slot ticker instead of run every (#4348)
* Add context check for unbounded loop work (#4346)
* Revert "Explicit use of GENESIS_SLOT in fork choice (#4343)" (#4349)

This reverts commit d3f6753c77.
* Refactor Powchain Service (#4306)

* add data structures

* generate proto

* add in new fields

* add comments

* add new mock state

* add new mock state

* add new methods

* some more changes

* check genesis time properly

* lint

* fix refs

* fix tests

* lint

* lint

* lint

* gaz

* fix lint

* raul's comments

* use one method

* fix test

* raul's comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Ensure best better-justification is stored for fork choice (#4342)

* Ensure best better-justification is stored. Minor refactor
* Tests
* Merge refs/heads/v0.9.2 into better-best-justified
* Merge refs/heads/v0.9.2 into better-best-justified
* Ensure that epoch of attestation slot matches the target epoch (#4341)

* Disallow duplicated indices and test
* Add slot to target epoch check to on_attestation
* Add slot to target epoch check to process_attestation
* Merge branch 'v0.9.2' of git+ssh://github.com/prysmaticlabs/prysm into no-dup-att-indices
* Fixed TestProcessAttestations_PrevEpochFFGDataMismatches
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Update beacon-chain/blockchain/forkchoice/process_attestation_test.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Filter viable branches in fork choice (#4355)
* Only activate upon finality (#4359)

* Updated functions
* Tests
* Merge branch 'v0.9.2' of git+ssh://github.com/prysmaticlabs/prysm into queue-fix-on-finality
* Comment
* Merge refs/heads/v0.9.2 into queue-fix-on-finality
* Fixed failing test from 4359 (#4360)

* Fixed
* Skip registry spec tests
* Wait for state to be initialized at least once before running slot ticker based on genesis time (#4364)
* Sync with master
* Fix checkpoint root to  use genesis block root (#4368)
* Return an error on nil head state in fork choice (#4369)

* Return error if nil head state

* Fixed tests. Saved childen blocks state

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Update metrics every epoch (#4367)
* return empty slice if state is nil (#4365)
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge refs/heads/master into v0.9.2
* Pubsub: Broadcast attestations to committee based subnets (#4316)

* Working on un-aggregated pubsub topics

* update subscriber to call pool

* checkpointing

* fix

* untested message validation

* minor fixes

* rename slotsSinceGenesis to slotsSince

* some progress on a unit test, subscribe is not being called still...

* dont change topic

* need to set the data on the message

* restore topic

* fixes

* some helpful parameter changes for mainnet operations

* lint

* Terence feedback

* unskip e2e

* Unit test for validate committee index beacon attestation

* PR feedbacK

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into resolveConflicts
* remove condition
* Remove unused operation pool (#4361)
* Merge refs/heads/master into v0.9.2
* Aggregate attestations periodically  (#4376)
* Persist ETH1 Data to Disk (#4329)

* add data structures

* generate proto

* add in new fields

* add comments

* add new mock state

* add new mock state

* add new methods

* some more changes

* check genesis time properly

* lint

* fix refs

* fix tests

* lint

* lint

* lint

* gaz

* adding in new proto message

* remove outdated vars

* add new changes

* remove latest eth1data

* continue refactoring

* finally works

* lint

* fix test

* fix all tests

* fix all tests again

* fix build

* change back

* add full eth1 test

* fix logs and test

* add constant

* changes

* fix bug

* lint

* fix another bug

* change back

* Apply suggestions from code review

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Fixed VerifyIndexedAttestation (#4382)
* rm signing root (#4381)

* rm signing root

* Fixed VerifyIndexedAttestation

* Check proposer slashed status inside ProcessBlockHeaderNoVerify

* Fixed TestUpdateJustified_CouldUpdateBest

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Remove Redundant Trie Generation (#4383)

* remove trie generation
* remove deposit hashes
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.9.2
* fix build
* Conflict
* Implement StreamAttestations RPC Endpoint (#4390)

* started attestation stream

* stream attestations test

* on slot tick test passing

* imports

* gaz

* Update beacon-chain/rpc/beacon/attestations_test.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

Co-authored-by: shayzluf <thezluf@gmail.com>
* Fixed goimport (#4394)
* Use custom stateutil ssz for ssz HTR spec tests (#4396)

* Use custom stateutil ssz for ssz HTR spec tests

* gofmt
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge refs/heads/master into v0.9.2
* set mainnet to be the default for build and run (#4398)

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* gracefully handle deduplicated registration of topic validators (#4399)

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* SSZ: temporarily disable roots cache until cache issues can be resolved (#4407)

* temporarily disable roots cache until cache issues can be resolved

* Also use custom ssz for spectests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Remove process block attestations as separate routine (#4408)

* Removed old save/process block atts

* Fixed tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Save Deposit Cache to Disk (#4384)

* change to protos

* fix build

* glue everything together

* fix test

* raul's review

* preston's comments

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Fix activation queue sorting (#4409)

* Removed old save/process block atts

* Fixed tests

* Proper sorting by eligibility epoch then by indices

* Deleted old colde
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge branch 'master' into v0.9.2
* Merge refs/heads/master into v0.9.2
* stop recursive lookup if context is cancelled (#4420)
* Fix proposal bug (#4419)
* Add Pending Deposits Safely (#4422)

* safely prune cache

* use proper method

* preston's,terence's reviews and comments

* revert change to build files

* use as feature config instead
* Release custom state ssz (#4421)

* Release custom state ssz, change all HTR of beacon state to use custom method

* typo

* use mainnet config

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Update initial sync save justified to align with v0.9.3 (#4432)
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* fix build
* don't blacklist on pubsub (#4435)
* Fix Flakey Slot Ticker Test (#4434)

* use interface instead for the slot ticker

* fixed up flakey tests

* add gen time

* get duties comment

* fix lifecycle test

* more fixes
* Configurable min genesis delay (#4437)

* Configurable min genesis delay based on https://github.com/ethereum/eth2.0-specs/pull/1557

* remove feature flag for genesis delay

* fix

* demo config feedback
* patch readme
* save keys unencrypted for validators (#4439)
* Add new demo configuration targeting mainnet scale (#4397)

* Add new demo configuration targeting mainnet, with 1/10th of the deposit value

* reduce quotant by 1/10th. Use 1/10th mainnet values

* only change the inactivity quotant

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Save justified checkpoint state (#4433)

* Save justified checkpoint state

* Lint

* Feedback

* Fixed test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Update shared/testutil/deposits.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update proto/testing/ssz_regression_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/core/epoch/epoch_processing.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/kv/forkchoice.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/pool.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/pool.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/powchain/log_processing_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/service.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber_beacon_blocks_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber_beacon_blocks_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/proposer.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/prepare_forkchoice.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/powchain/log_processing_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/pool.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/powchain/log_processing_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/aggregator/server.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/cache/depositcache/pending_deposits.go
* Update beacon-chain/cache/depositcache/pending_deposits_test.go
* Update beacon-chain/rpc/validator/proposer.go
* Merge refs/heads/master into v0.9.2
* Fix e2e genesis delay issues (#4442)

* fix e2e genesis delay issues

* register flag

* typo

* Update shared/featureconfig/config.go

Co-Authored-By: Nishant Das <nishdas93@gmail.com>

* Apply suggestions from code review

Co-Authored-By: Nishant Das <nishdas93@gmail.com>

* skip demo e2e

* fix validator

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Nishant Das <nish1993@hotmail.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
* Batch Eth1 RPC Calls (#4392)

* add new methods

* get it working

* optimize past deposit logs processing

* revert change

* fix all tests

* use mock

* lint

* lint

* check for nil

* stop panics

* Apply suggestions from code review

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Terence's Review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-01-07 18:47:39 +00:00
Ivan Martinez
9aed0034ec Vastly improve E2E logs and add README (#4440)
* Improve E2E logs to help debugging
* Add README to E2E
* Remove newline logs
* Remove removedb
* Try releasing after killing process
* Fix validator output
* Fix e2e
* Solve eth1 issue by clearing eth1 db
* Whoops
* Fix log spacing
2020-01-07 17:00:51 +00:00
terence tsao
f764522cbe Log warn and cont if validator pub key not exist in DB (#4429)
* log warn and cont
* assignment
* fixed
* Merge refs/heads/master into log-warn-cont
2020-01-06 21:16:36 +00:00
Ivan Martinez
c7ae03e1b2 E2E cleanup and fix ETH1 chain startup (#4431)
* E2E cleanup and fixes

* Fix build issue
2020-01-06 14:50:36 -06:00
Preston Van Loon
4efc0f5286 Require a state to exist to save justified checkpoint (#4423)
* Add validation to save justified checkpoint in db
* gofmt
2020-01-06 14:41:51 +00:00
Preston Van Loon
9052620453 Release feature --fast-assignments (#4416)
* Deprecated --fast-assignments
* gaz
* Merge branch 'master' of github.com:prysmaticlabs/prysm into release-fast-assignments
2020-01-06 02:57:52 +00:00
Preston Van Loon
0174397f6e Release --enable-bls-pubkey-cache (#4417)
* Release bls pubkey-cache
2020-01-05 20:08:49 +00:00
terence tsao
9f5caf8fea total and target balances metrics (#4414) 2020-01-05 11:12:48 -08:00
Nishant Das
3b3f2c78e2 unskip test (#4411) 2020-01-05 13:39:14 +08:00
Ivan Martinez
242e4bccbf Move confirmDelete from beacon-chain to shared/cmd (#4410)
* Move confirmDelete to shared/cmd as ConfirmAction

* Finish moving function to shared/cmd

* Pass in both text

* Fix for comments
2020-01-04 23:32:09 -05:00
terence tsao
59ab89c98a Validator caches index (#4406) 2020-01-04 11:50:16 -08:00
terence tsao
ac768207ac Safe delete states (#4401)
* Filter block roots by finalization and head
* Tests
* Comments
* Merge branch 'master' into safe-delete-states
* Fixed exisiting tests
* Merge branch 'safe-delete-states' of git+ssh://github.com/prysmaticlabs/prysm into safe-delete-states
* Merge refs/heads/master into safe-delete-states
* Merge refs/heads/master into safe-delete-states
* Merge refs/heads/master into safe-delete-states
* Merge refs/heads/master into safe-delete-states
2020-01-04 19:20:20 +00:00
terence tsao
77d41024dc Revert "Use poststate for calculating att votes (#4395)" (#4404) 2020-01-04 09:25:42 -08:00
Preston Van Loon
f03083f6c8 PK manager: don't panic on bad key (#4405)
* don't panic on bad key
2020-01-04 05:32:47 +00:00
Jim McDonald
5ff9ae2108 Validator keymanager refactor (#4340)
* Move to keymanager
* Move to keymanager
* Merge branch 'keymanager' of github.com:mcdee/prysm into keymanager
* Lint
* Fix visibility
* Bazel fix
* Merge remote-tracking branch 'upstream/master' into keymanager
* logrus->log
* Merge branch 'master' into keymanager
* Merge remote-tracking branch 'upstream/master' into keymanager
* Merge branch 'master' into keymanager
* Merge branch 'master' into keymanager
* Merge branch 'master' into keymanager
* Merge branch 'master' into keymanager
* Fix test after merge
* Merge branch 'master' into keymanager
* And again
2020-01-04 03:51:53 +00:00
terence tsao
5fa03edb29 Update committee cache prev epoch (#4402)
* Update cache base on input epoch, not state epoch
* Tests
* Fixed benchmarks
* Use epochs
* One more
2020-01-03 23:47:54 +00:00
Preston Van Loon
ebe4c9c971 Use a single lock for arrays cache (#4400)
* use a single lock for arrays cache
* Merge refs/heads/master into one-lock
2020-01-03 20:24:19 +00:00
Preston Van Loon
6efe5ef496 Slot ticker: panic on zero genesis time given (#4366)
* panic on zero genesis time given

* fix test

* fix test

* fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-01-03 13:42:35 -06:00
Celeste Ariana Seberras
fbbf5514d1 Syncing gitbook information with README (#4323)
* Syncing gitbook information with README

Updated to match https://prysmaticlabs.gitbook.io/prysm/
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge branch 'master' into master
* Curl readded
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge branch 'master' into master
2020-01-03 19:05:16 +00:00
terence tsao
220af25bce Use poststate for calculating att votes (#4395)
* Use poststate for votes
* Merge branch 'master' into use-post-state
* Merge branch 'master' into use-post-state
2020-01-03 18:29:26 +00:00
Nishant Das
c9252c06c4 Change Skip Slot Cache Key (#4391)
* use different cache key

* add build

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-01-03 11:44:48 -06:00
Jim McDonald
2c565f5d59 Harden BLS (#4393) 2020-01-03 07:34:15 -08:00
Ivan Martinez
1cb58e859e Add protos for validator proposal slashing protection (#4387)
* Add protos for validator proposal protection
* Fix formatting
* Fix formatting
* Rename protos
* remove extra line
2020-01-03 02:41:31 +00:00
terence tsao
d26839c1f2 Add aggregator indices to logs (#4385)
* Add validator_log.go

* Use new logging scheme

* Go fmt

* Better name

* Tests

* Tests

* Add wg.done, moved logging before span end

* Add aggregator indices to submit attestation log

* Rename

* Fixed test

* Add proposer index

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-01-02 17:45:48 -06:00
terence tsao
2cb8430ad4 Enhance attester logging (#4380)
* Add validator_log.go
* Use new logging scheme
* Go fmt
* Better name
* Tests
* Tests
* Merge refs/heads/master into better-logging
* Merge branch 'master' of git+ssh://github.com/prysmaticlabs/prysm into better-logging
* Add wg.done, moved logging before span end
* Merge branch 'better-logging' of git+ssh://github.com/prysmaticlabs/prysm into better-logging
2020-01-02 17:04:07 +00:00
Nishant Das
03356fc7b5 Add Ability to Resync Node (#4279)
* add resyncing functionality

* add more validation to status message

* lint and build

* jim's review

* preston's review

* clean up

* remove log

* remove no sync

* change again

* change back

* remove spaces

* Update shared/slotutil/slottime.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* Apply suggestions from code review

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* fix refs

* raul's review

* goimports

* goimports

* add counter

* removed condition

* change back

* gaz

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-01-02 16:09:28 +08:00
Preston Van Loon
bdc4045e23 Add eth1data deposit count metric (#4374) 2019-12-30 08:34:46 -08:00
Preston Van Loon
dc1bd1ef62 Revert 4372 and 4373 (#4375)
* Revert "only add chain start deposits up to min genesis active validator count (#4373)"

This reverts commit 35380dd9bf.
* Revert "Return an error if the wrong number of deposits are provided for genesis state (#4372)"

This reverts commit 9674575892.
2019-12-30 01:21:08 +00:00
Preston Van Loon
35380dd9bf only add chain start deposits up to min genesis active validator count (#4373) 2019-12-29 16:27:17 -08:00
Preston Van Loon
9674575892 Return an error if the wrong number of deposits are provided for genesis state (#4372)
* Return an error if the wrong number of deposits are provided for genesis state
* add regression test
2019-12-29 20:10:23 +00:00
Nishant Das
b7d0d7cbb6 Shift Deposit Contract Tools (#4357)
* move tools
* Merge refs/heads/master into shiftTools
* Merge refs/heads/master into shiftTools
2019-12-27 00:41:43 +00:00
Nishant Das
28eadac172 Fix Deposit Log Processing (#4352)
* fix log processing
* Merge branch 'master' into fixLogs
* Merge refs/heads/master into fixLogs
2019-12-26 17:44:56 +00:00
Nishant Das
d5181496c4 Add Docker image for slasher (#4356)
* add docker image for slasher

* load docker rules

* change to c base image

* switch off pure builds
2019-12-26 10:53:27 -06:00
Nishant Das
b337a5720c Handle Pubsub Panics (#4350)
* handle panics
* lint
* gaz
* preston's review
2019-12-24 04:59:08 +00:00
terence tsao
53b8eb57ee Fuzz ProcessFinalUpdates (#4308) 2019-12-23 10:27:16 -08:00
terence tsao
30b4b045f5 Add justified check points to chain info getters (#4335)
* Add justified checkpoint getters

* Use it for chainhead

* Mock

* Fixed tests

* Fixed TestServer_StreamChainHead_OnHeadUpdated

* Caught a run time bug. Fixed

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2019-12-22 00:14:23 -06:00
Ivan Martinez
ec1e7ae005 Remove proto/sharding and move slashing to own dir (#4332)
* Clean proto and move slasher proto to own folder

* Change package name to match files

* Fix typo

* Fix tests

* Undo out of scope changes

* Run gazlle

* Fix build.bazel

* goimports
2019-12-20 21:47:00 -06:00
Preston Van Loon
a949673e33 Pubsub ignore messages from yourself (#4337)
* ignore messages from myself
2019-12-20 19:37:25 +00:00
terence tsao
996f4c7f5a Clean up list beacon committees to not use head state (#4333) 2019-12-20 08:02:12 -08:00
Valentin Mihov
3915a6e15a Fix the links in the TOC (#4334)
The links for running the client, were pointing to wrong sections.
2019-12-20 18:44:03 +08:00
Preston Van Loon
961dd21554 Use libp2p gossipsub upstream validator framework (#4318)
* add reject all pubsub validator to stop automatic propagation of messages
* gaz
* Merge branch 'master' of github.com:prysmaticlabs/prysm into pubsub-validator
* refactor p2p validator pipeline
* add sanity check
* Merge branch 'pubsub-validator' of github.com:prysmaticlabs/prysm into pubsub-validator
* fixed up test
* rem
* gaz
* Merge refs/heads/master into pubsub-validator
* fix from self test
* ensure validator data is set
* resolve todo
* Merge refs/heads/master into pubsub-validator
* gaz
* Merge refs/heads/master into pubsub-validator
* Merge branch 'pubsub-validator' of github.com:prysmaticlabs/prysm into pubsub-validator
* Merge refs/heads/master into pubsub-validator
* remove all of the 'from self' logic. filed https://github.com/libp2p/go-libp2p-pubsub/issues/250
* Merge branch 'pubsub-validator' of github.com:prysmaticlabs/prysm into pubsub-validator
* gaz
* update comment
* Merge refs/heads/master into pubsub-validator
* rename "VaidatorData"
* Merge branch 'pubsub-validator' of github.com:prysmaticlabs/prysm into pubsub-validator
* refactor
* one more bit of refactoring
* Update beacon-chain/sync/validate_beacon_attestation.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>
* skip validation on self messages, add @nisdas feedback to increment failure counter
* Merge branch 'pubsub-validator' of github.com:prysmaticlabs/prysm into pubsub-validator
* remove flakey
2019-12-20 03:18:08 +00:00
terence tsao
2e4908e7c4 Optimize committee helpers (#4328) 2019-12-19 15:40:51 -08:00
Preston Van Loon
da637668a8 Minor fixes to create keys errors (#4330)
* minor fixes
2019-12-19 19:44:06 +00:00
Jim McDonald
20168ad729 More complete validator metrics (#4327)
* More complete validator metrics
* Merge branch 'master' into metrics
* Merge branch 'master' into metrics
2019-12-19 16:14:44 +00:00
Jim McDonald
0b07a9f227 Migrate periodic function to use RunEvery (#4324) 2019-12-19 07:02:10 -08:00
Jim McDonald
5dca662d01 Comment typo (#4325) 2019-12-19 06:01:23 -08:00
Nishant Das
8c28d1080c Revert "Fix same deposits from same validator in same block" (#4321)
* Revert "Fix same deposits from same validator in same block (#4319)"

This reverts commit 908d220eb2.
2019-12-19 05:36:19 +00:00
Raul Jordan
6a54a430e1 Add Filter by Epoch in kv/blocks.go (#4303)
* allow for epoch based filtering
* modify repo to include filter by epoch
* resolve items
* revamped to use epoch filter
* Merge branch 'master' into roots-by-epoch
* Merge refs/heads/master into roots-by-epoch
* Merge refs/heads/master into roots-by-epoch
* Merge refs/heads/master into roots-by-epoch
* gazelle rem unused
* Merge refs/heads/master into roots-by-epoch
* Merge refs/heads/master into roots-by-epoch
* Merge refs/heads/master into roots-by-epoch
* Merge refs/heads/master into roots-by-epoch
* Merge refs/heads/master into roots-by-epoch
* Merge refs/heads/master into roots-by-epoch
* Merge refs/heads/master into roots-by-epoch
* Merge refs/heads/master into roots-by-epoch
* Update beacon-chain/db/kv/blocks_test.go

Co-Authored-By: shayzluf <thezluf@gmail.com>
* Update beacon-chain/db/kv/blocks_test.go

Co-Authored-By: shayzluf <thezluf@gmail.com>
* fmt
* lint res
2019-12-19 00:15:31 +00:00
terence tsao
908d220eb2 Fix same deposits from same validator in same block (#4319)
* Update dict

* Test helper

* Regression test

* Comment

* Reset test cache
2019-12-18 16:53:30 -06:00
Preston Van Loon
ff1fd77425 Build docker images for non-root user (#4320)
* build docker images as non-root user
* search and replace mistake
* buildifer
* Change uid to 1001
2019-12-18 20:52:25 +00:00
Nishant Das
e27bc8312f Persist ETH1 Information (#4305)
* add data structures
* generate proto
* add in new fields
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into saveETH1Data
* add comments
* Merge branch 'master' into saveETH1Data
* remove file
* Merge branch 'saveETH1Data' of https://github.com/prysmaticlabs/geth-sharding into saveETH1Data
* Merge branch 'master' into saveETH1Data
* Merge branch 'master' into saveETH1Data
* Merge refs/heads/master into saveETH1Data
* Merge refs/heads/master into saveETH1Data
2019-12-18 05:57:54 +00:00
Jim McDonald
78968c1e29 Add individual p2p host counts (#4312)
* Add individual p2p host counts
* Merge branch 'master' into p2pmetrics
* Merge branch 'master' into p2pmetrics
* Merge branch 'master' into p2pmetrics
* Merge branch 'master' into p2pmetrics
2019-12-18 05:31:46 +00:00
terence tsao
fb431c11c1 Process slots exit early if same slot (#4314)
* skip process slot if it's same slots
* Merge branch 'master' into optimize-process-slots
* Merge refs/heads/master into optimize-process-slots
2019-12-18 04:37:28 +00:00
Raul Jordan
30ed59e9c8 Make Sure ChainHeadStream Remains Open (#4282)
* do not return from stream
* fix test
* Merge branch 'master' into no-stream-return
* Merge refs/heads/master into no-stream-return
2019-12-18 04:07:11 +00:00
Jim McDonald
2e2d5199e8 Remove ChainStartFeed mocks (#4310)
* Remove ChainStartFeed from interop service
* Remove final ChainStartFeed mocks
* Gazelle
* Merge branch 'master' into coldstart
* Merge branch 'master' into coldstart
* Merge branch 'master' into coldstart
2019-12-18 03:36:07 +00:00
Raul Jordan
4fe31cf1b3 Add Benchmarks for Custom SSZ Hash Tree Root (#4313)
* bench ssz tree root
* more benches
* Merge branch 'master' into ssz-bench
2019-12-18 02:57:40 +00:00
Preston Van Loon
e82e582cdf Config to exclude kafka dep at build time (#4309)
* add flag to exclude kafka
* Add config flag to exclude kafka
* Merge branch 'master' into buildtime-exclude-kafka
2019-12-18 02:07:49 +00:00
Jim McDonald
0b2d9d8576 Tidy up interop commands (#4311) 2019-12-17 15:49:21 -08:00
Preston Van Loon
65e3f3e007 Add pubsub message ID function (#4304)
* add pubsub message ID
* thanks linter
* Update rules_go, gogo protobuf, comment
* Merge branch 'master' into add-msg-fn-id
2019-12-17 05:17:54 +00:00
Preston Van Loon
2c28e4e7a3 Improvements to Committee Assignments for multiple key requests (#4294)
* Add committees helper, benchmark, results show 62ms for 8k validators which was previously 4 minutes
* Add regression test with same data
* fix epoch conversion
* lint
* undo and lint
* Merge branch 'master' of github.com:prysmaticlabs/prysm into zoom-zoom-assignments
* remove validaotr index span
* fix comment, add test to test against spec definition method for consistency.
* Deprecate CommitteeAssignment, delete unused reference to CommitteeAssignment
* Merge branch 'master' of github.com:prysmaticlabs/prysm into zoom-zoom-assignments
* remove new line
* make test be more complicated with validators activated in an epoch transition
* add feature flag for fast-assignments
* Merge branch 'master' of github.com:prysmaticlabs/prysm into zoom-zoom-assignments
* gaz, gofmt, add deprecated code back
* Update beacon-chain/core/helpers/committee.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge refs/heads/master into zoom-zoom-assignments
* Merge refs/heads/master into zoom-zoom-assignments
* Merge refs/heads/master into zoom-zoom-assignments
2019-12-17 03:05:26 +00:00
Raul Jordan
642254daa6 Update README Instructions to Resolve Deposit Contract Address (#4302)
* dep addr resolver
* Merge refs/heads/master into update-read
2019-12-17 02:52:54 +00:00
Nishant Das
c41140e15a Optimize Insertion in Deposit Trie (#4299)
* current changes
* change algorithm for tree insert
* almost done with getting this to pass
* unit test passes
* tests now pass
* fix in repo
* Merge branch 'master' into optimizeDepositLogs
* fix build
* Merge branch 'optimizeDepositLogs' of github.com:prysmaticlabs/prysm into optimizeDepositLogs
* remove tautology
* fix tautology
* fix up sparsity
* Merge branch 'master' into optimizeDepositLogs
* further fixes
* Merge branch 'optimizeDepositLogs' of github.com:prysmaticlabs/prysm into optimizeDepositLogs
* Update shared/trieutil/sparse_merkle.go
* comments
* Merge branch 'optimizeDepositLogs' of github.com:prysmaticlabs/prysm into optimizeDepositLogs
* add bench for optimized
* gaz
* Merge refs/heads/master into optimizeDepositLogs
2019-12-17 02:19:12 +00:00
terence tsao
23a6c20dd4 Service as proper names (#4293) 2019-12-16 19:53:55 -06:00
Preston Van Loon
514f5f904f Add prometheus gRPC time histograms (#4300)
* Add grpc_prometheus.EnableHandlingTimeHistogram()
* Merge refs/heads/master into enable-prom-hist
2019-12-16 22:00:34 +00:00
Preston Van Loon
5844436716 Don't serialize bls signature just to deserialize it again (#4298)
* Don't serialize bls signature just to deserialize it again
* gaz
* Merge branch 'master' into minor-thing
* Merge branch 'master' into minor-thing
2019-12-16 19:01:40 +00:00
terence tsao
5879b26b4b Hardening Committee Cache for Runtime (#4270) 2019-12-16 10:14:21 -08:00
terence tsao
566efaef89 Optimize aggregator process slots (#4297)
* Advance slots up to epoch start
* Merge branch 'master' into opt-process-slots-aggregator
* Merge branch 'master' into opt-process-slots-aggregator
2019-12-16 17:43:33 +00:00
Jim McDonald
d9062a7e30 Use RunEvery in place of custom tickers (#4290) 2019-12-16 11:00:15 -06:00
Preston Van Loon
3f344aee55 add a few fuzz tests (#4291) 2019-12-16 00:52:20 -06:00
Nishant Das
fd93751bf7 Fix Goerli Faucet (#4289)
* fix faucet

* minor fixes
2019-12-15 08:21:29 -06:00
Preston Van Loon
325a2503f7 AttestingIndices: Make beacon committee be an argument (#4284)
* make beacon committee be an argument
* remove state from ConvertToIndexed
* Merge branch 'master' into refactor-AttestingIndices-committee
* Merge branch 'master' into refactor-AttestingIndices-committee
* Merge branch 'master' into refactor-AttestingIndices-committee
* Merge refs/heads/master into refactor-AttestingIndices-committee
2019-12-15 05:02:50 +00:00
Preston Van Loon
2179ac683e Fuzz testing for custom state ssz (#4234)
* Add a random fuzz test to ssz to capture panics and compare the effectiveness of the cache. This comment shows a difference in state root calculation 52% of the time and what is even more concering is that spec tests pass with the flag on.
* added case for one
* bring down failure rate
* prevent caching operations if no cache enabled
* unit test and pretty printer
* identify further sources of problems
* no more panics
* not panicking anymore
* fix lint
* Merge branch 'master' into fuzz-ssz
* Merge branch 'master' into fuzz-ssz
* passing up to 68
* Merge branch 'fuzz-ssz' of github.com:prysmaticlabs/prysm into fuzz-ssz
* need to find the culprit for 100
* 100 passes, now only 16 out of 1000
* state roots being mutated
* one out of 10k
* fuzzing stuff
* fix up lint
* Merge branch 'master' into fuzz-ssz
* cleanup
* fixing more comments
* Merge branch 'master' into fuzz-ssz
2019-12-15 04:32:19 +00:00
terence tsao
0f4dabfad8 Fix cloning target state for check point state cache (#4288) 2019-12-14 16:06:30 -08:00
terence tsao
8724dcd41b Sort received atts by sig (#4287) 2019-12-14 10:51:27 -06:00
Jim McDonald
89e1200b73 Add ticker shared helper (#4285) 2019-12-13 15:14:56 -08:00
metanull-operator
0f677a09b6 Added 'Prysm' to version information. (#4281)
* Added 'Prysm' to version information.
* Merge branch 'master' into versionUpdate
2019-12-13 18:52:28 +00:00
Nishant Das
c5dcf49ded Add Flag For Minimum Handshakes (#4280)
* add flag
* jim and preston's review
* check max peers
* gaz
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into minStatusCount
* remove space
* add references
* add warning log
* change log
* gaz
2019-12-13 15:12:49 +00:00
terence tsao
a5881f924f Deprecate active count and committee cache flags (#4276)
* Deprecate active count and committee cache flags
* Merge branch 'master' into deprecate-flags
2019-12-13 14:00:29 +00:00
shayzluf
d93ec64b21 Slasher Grpc client (#4230)
* grpc connection
* fix order
* Merge branch 'fixInteropGenesis' of https://github.com/prysmaticlabs/prysm into grpc_client
* gaz
* grpc setup
* running version
* added comments
* Merge branch 'master' of github.com:prysmaticlabs/prysm into grpc_client
* fix test
* terence feedback
* terence feedback
* feedback changes
* feedback changes
* comment fix
* Merge branch 'master' of github.com:prysmaticlabs/prysm into grpc_client
* logging when there is no chain head
* rename function
* terence and nishant feedback
* fix imports
* nishant feedback
* fix wait for stop
* fix imports
* fix tests
2019-12-13 07:31:37 +00:00
Raul Jordan
a9a5973b98 Add Getter for Genesis Block (#4271)
* test passing

* kafka
2019-12-12 16:27:22 -06:00
Jim McDonald
570efe3d04 Give peers a chance (#4268)
* Add decay function for peer badresponses count
* Activate peer decay in p2p
2019-12-12 14:34:28 +00:00
Raul Jordan
2e9c3895f4 Bring Back Epoch Filtering for ListBlocks API (#4262)
* bring back the epochs!
* fix up
* Merge refs/heads/master into bring-back-epoch-filter
* add in patch
* Merge branch 'bring-back-epoch-filter' of github.com:prysmaticlabs/prysm into bring-back-epoch-filter
* import spacing
* lint
* build
* gaz
* Merge refs/heads/master into bring-back-epoch-filter
* gaz
* Merge branch 'bring-back-epoch-filter' of github.com:prysmaticlabs/prysm into bring-back-epoch-filter
* move back perf
* update ethapis
* fix build
* Merge refs/heads/master into bring-back-epoch-filter
2019-12-12 02:27:19 +00:00
terence tsao
9033f6801b Removed active count and shuffling cache (#4266)
* Removed
* All tests pass
* Gaz
* Removed new lines
* A few more lines...
* I think i got them all
* and I didnt : )
* Could this be last...
2019-12-12 01:15:44 +00:00
Preston Van Loon
c0b3767757 remove old cache for active indices. this is not used in production and will soon be replaced (#4264) 2019-12-11 15:48:48 -06:00
Preston Van Loon
e72ff1bb4f Add unit test to ActiveValidatorIndices (#4263)
* Add regression test to ActiveValidatorIndices

* fix test, more comments

* imports
2019-12-11 12:27:25 -08:00
Jim McDonald
0cb59bb018 Tidy up "Requesting blocks" log in initial sync (#4256)
* Tidy up log
* Merge branch 'master' into logfix
2019-12-11 17:32:38 +00:00
Preston Van Loon
6e549c90ba Initialize server context for beacon server (#4260)
* Initialize server context for beacon server
* Merge branch 'master' into fix-4254
2019-12-11 16:37:05 +00:00
Jim McDonald
813233373e Advanced peer tracking (#4233)
* Advanced peer status

* Rework errors; add tests

* Gazelle

* time->roughtime

* Update beacon-chain/p2p/handshake.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Update beacon-chain/p2p/interfaces.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Downgrade log

* Tidy up handshaking logic and commentary

* Downgrade log message

* Protect connected peers from disconnection; increase high water level to avoid bad interactions at maxPeers
2019-12-11 18:31:36 +08:00
Preston Van Loon
5757ce8894 Fix flaky TestKV_Aggregated_CanSaveRetrieve (#4253)
* Update aggregated_test.go
2019-12-11 06:08:54 +00:00
Nishant Das
7c11367cd8 Update SSZ (#4252)
* update ssz to latest
* Merge branch 'master' into updateSSZ
* Merge refs/heads/master into updateSSZ
2019-12-11 05:16:37 +00:00
terence tsao
6d2c37caf1 Removed process epoch (#4251) 2019-12-10 20:40:14 -08:00
Ivan Martinez
812311f6f7 Add more detail to README and add benchmark for HashTreeRootState (#4247)
* Add more detail to readme and add benchmark for HashTreeRootState
* Add hashtreerootstate benchmark results to readme
* Merge branch 'master' into benchmarks-readme
2019-12-11 00:14:33 +00:00
terence tsao
22d81ef0ed Update process_epoch benchmark (#4245)
* Update to ProcessEpochPrecompute
* Comment
* Add b.N back
2019-12-10 23:32:11 +00:00
Ivan Martinez
414fcda9a2 Change jaeger default endpoint (#4242)
* Change jaeger default
2019-12-10 19:45:14 +00:00
Nishant Das
bb2fc4cd5e Fix Deposit Sender Utility (#4239)
* add in fix

* change gas limit
2019-12-10 09:35:24 -08:00
Nishant Das
5fd6a92052 Fix DiscoveryV5 (#4237)
* add fallback

* fix test
2019-12-10 13:35:16 +08:00
Nishant Das
7ccbe48f54 fix order (#4228) 2019-12-09 22:31:53 +08:00
Preston Van Loon
7a46cc0681 Enforce stronger head state operations (#4216)
* Enforce stronger head state operations

* fix genesis state generation

* one test left to fix

* all tests passing now

* gofmt

* Update beacon-chain/db/kv/state_test.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* Update beacon-chain/db/kv/state.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* fix tests
2019-12-09 15:35:18 +08:00
Preston Van Loon
92d21c72b8 Skip accessing peer status if it does not exist (#4226)
* skip accessing peer status if it does not exist
* Merge refs/heads/master into fix-panic-rr
2019-12-08 23:30:39 +00:00
Preston Van Loon
0cb681476e Add span for saveCheckpointState (#4227)
* Add span for saveCheckpointState
* Add span for saveCheckpointState
2019-12-08 22:53:09 +00:00
Preston Van Loon
fa7b8ab60d A few improvements to handshake (#4214)
* A few improvements to handshake and exit round robin
* revert beacon-chain/sync/initial-sync/round_robin.go
* Merge refs/heads/master into p2p-fixes
* make handshake non-blocking
* Merge branch 'p2p-fixes' of github.com:prysmaticlabs/prysm into p2p-fixes
* Merge refs/heads/master into p2p-fixes
* Merge refs/heads/master into p2p-fixes
* Merge refs/heads/master into p2p-fixes
* Update handshake.go
2019-12-08 05:12:56 +00:00
Preston Van Loon
bdb80271a3 Allow faucet on prylabs.network (#4220)
* fix faucet hostname issue
2019-12-08 03:01:44 +00:00
terence tsao
1b8eb16fc7 --initial-sync-cache-state don't need to save head root (#4219)
* Test

* Run time works

* Revert
2019-12-07 16:45:39 -06:00
metanull-operator
1222ebb6db Graffiti flag (#4213)
* Implementation of graffiti flag without tests.
* Updated to pass graffiti as string instead of []byte all the way to the ProposeBlock RPC call. This ensures that the ToBytes32() call is handled in ProposeBlock as opposed to relying on the caller to ensure that the value passed is only 32 bytes. This adds work by doing that conversion on each proposed block for a static value of graffiti, but it also helps protect against an RPC call to ProposeBlock that has more than 32 bytes for graffiti.
* Added test case for validator.
* Added GraffitiFlag to validate usage test.
* Updated data structures and logic to convert graffiti flag from string to byte array earlier in the process. Now converting when setting up ValidatorService.
* Updated test case to correctly set up validator using byte array.
* Merge branch 'master' into graffitiFlag
2019-12-07 19:13:56 +00:00
terence tsao
3e15e2fc1e Add operation feed (#4215)
* Events
* Notifiers
* Refactor
* Gaz
* Fixed rest
* Lint
* Lint
* Visibility
* Typo
* Typo
* Apply suggestions from code review

Co-Authored-By: Nishant Das <nishdas93@gmail.com>
2019-12-07 17:57:26 +00:00
Nishant Das
667466020e Change All Caches To Ristretto (#4208)
* new caches
* goimports, gaz
* fix all tests
* Merge branch 'swapP2PCaches' of https://github.com/prysmaticlabs/geth-sharding into swapP2PCaches
* remove from bls
* remove ccache
* fix handshake
* Merge branch 'master' into swapP2PCaches
* gofmt
* Merge branch 'master' into swapP2PCaches
2019-12-06 20:06:37 +00:00
terence tsao
f63ab1e136 Remove formatting error for signature fail to verify (#4211)
* Remove formatting error for sig
* Update beacon-chain/core/blocks/block_operations.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge branch 'master' into sig-error-log
2019-12-06 18:21:57 +00:00
Preston Van Loon
6841d96f36 Remove formatting from error (#4210)
* remove formatting from error
* Fix err
2019-12-06 17:48:38 +00:00
Preston Van Loon
cae24068d4 prevent OR on bitlists of different length (#4209)
* prevent OR on bitlists of different length
* prevent OR on bitlists of different length
2019-12-06 14:33:40 +00:00
terence tsao
dc0b8fad4f Move recently seen roots (#4206)
* Move recently seen roots earlier
* Preston's feedback
2019-12-06 06:30:43 +00:00
Preston Van Loon
d3375d98a8 Kafka exporter (#3840)
* abstract db interface, kafka build, work in progress
* checkpoint
* Merge branch 'master' of github.com:prysmaticlabs/prysm into es-exporter
* feature flag
* move passthrough
* flag change
* gofmt
* Merge branch 'master' of github.com:prysmaticlabs/prysm into es-exporter
* missing db methods
* Merge branch 'master' of github.com:prysmaticlabs/prysm into es-exporter
* fix interface
* Merge branch 'master' of github.com:prysmaticlabs/prysm into es-exporter
* Merge branch 'master' of github.com:prysmaticlabs/prysm into es-exporter
* Merge branch 'master' of github.com:prysmaticlabs/prysm into es-exporter
* try using cmake built from source
* lint godocs
* lint godocs
* lint godocs
* Update BUILD.bazel
* Merge branch 'master' into es-exporter
* Merge branch 'master' into es-exporter
* Merge branch 'master' into es-exporter
* Merge branch 'master' of github.com:prysmaticlabs/prysm into es-exporter
* gaz
2019-12-06 02:05:58 +00:00
terence tsao
9d4c7cb4f7 Use cache state during init sync (#4199)
* Initial sync state cache
* Gaz
* Gaz
* Don't save head root
* Fix config validator
* Uncomment save head
* Merge branch 'master' into initial-sync-no-verify
* Minor refactor
* Merge branch 'initial-sync-no-verify' of https://github.com/prysmaticlabs/prysm into initial-sync-no-verify
* Merge branch 'master' into initial-sync-no-verify
* Tests
* Merge branch 'initial-sync-no-verify' of https://github.com/prysmaticlabs/prysm into initial-sync-no-verify
* Merge branch 'master' into initial-sync-no-verify
* Add lock
* Merge branch 'initial-sync-no-verify' of https://github.com/prysmaticlabs/prysm into initial-sync-no-verify
* Tests
* Removed save head
* One more test
* Merge branch 'master' into initial-sync-no-verify
* Raul's feedback
* Merge branch 'initial-sync-no-verify' of https://github.com/prysmaticlabs/prysm into initial-sync-no-verify
* Comment
* Gazelle
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into initial-sync-no-verify
* revert
* Update beacon-chain/blockchain/service.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into initial-sync-no-verify
* Merge branch 'initial-sync-no-verify' of https://github.com/prysmaticlabs/prysm into initial-sync-no-verify
* Fixed test
* Fixed feature flag
* Merge branch 'master' into initial-sync-no-verify
* Fixed cache gensis state test
* Merge branch 'initial-sync-no-verify' of https://github.com/prysmaticlabs/prysm into initial-sync-no-verify
2019-12-06 00:49:19 +00:00
Raul Jordan
ae2b2e74ca Create Functional Cache for Custom State SSZ (#4197)
* better abstraction
* using ristretto
* begin on custom, cached array roots merkleization
* do cache initialization
* passing with new cache
* works
* fix up test
* fixed up cache
* include proper comments
* remove old hash tree root
* rem validator bottleneck
* gaz
* Merge branch 'master' into caching-ssz
* optimized!!!!
* Merge branch 'caching-ssz' of github.com:prysmaticlabs/prysm into caching-ssz
* add mutex
* Merge branch 'master' into caching-ssz
* add read lock
* fmt
* add mathutil
* Merge branch 'master' into caching-ssz
* Merge refs/heads/master into caching-ssz
* Merge refs/heads/master into caching-ssz
* Merge refs/heads/master into caching-ssz
2019-12-05 20:23:59 +00:00
Ivan Martinez
83179376d4 Cleanup testutil and change name scheme to reference deterministic (#4167)
* Clean testutil, change tool names to Deterministic
* Cleanup errors
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into clean-testutil
* Fix bug with generating deposits
* Fix a few tests
* Fix most tests
* Clean up some tests
* Remove err pt. 1
* Remove err pt. 2
* Change tests to use genesis state util
* Remove err from deposits
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into clean-testutil
* Remove circular dependency
* Remove uncompressed signature test
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into clean-testutil
* Merge branch 'master' into clean-testutil
* Goimports
* gazelle
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into clean-testutil
* Add back error handling
* New attestation pool (#4185)

* New pool
* Better namings
* Fmt
* Gazelle
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into define-pool
* Raul's feedback
* Raul's feedback
* Log peer conected log for incoming connections (#4173)

* Log peer conected log for incoming connections
* Merge branch 'master' into peerconnected
* Merge branch 'master' into peerconnected
* Update handshake.go
* Update handshake.go
* Merge branch 'master' into peerconnected
* Merge branch 'master' into peerconnected
* Attestation pool to use go-cache (#4187)
* Update EthereumAPIs  (#4186)

* include new patch targeting latest ethapis master
* ensure project builds
* Merge branch 'master' into update-all-api
* fix up committees
* Merge branch 'update-all-api' of github.com:prysmaticlabs/prysm into update-all-api
* include latest eth apis
* Merge branch 'master' into update-all-api
* update block tests
* Merge branch 'update-all-api' of github.com:prysmaticlabs/prysm into update-all-api
* Merge branch 'master' into update-all-api
* add todos
* Implement GetValidator RPC Endpoint (#4188)

* include new patch targeting latest ethapis master
* ensure project builds
* Merge branch 'master' into update-all-api
* fix up committees
* Merge branch 'update-all-api' of github.com:prysmaticlabs/prysm into update-all-api
* include latest eth apis
* Merge branch 'master' into update-all-api
* update block tests
* Merge branch 'update-all-api' of github.com:prysmaticlabs/prysm into update-all-api
* Merge branch 'master' into update-all-api
* add todos
* implement get validator rpc
* add test for get validator
* table driven test
* fix up test
* fix confs
* tests for more cases
* fix up tests and add out of range
* Slasher optimization (#4172)

* size

* batching and concurrency improvements

* gaz

* merge fixes

* fix comment

* fix test

* fix test

* fix build

* ethpb

* ethpb

* fix test

* fix comment

* add benchmark

* fix benchmark
* Handle error for all testutil uses
* Fix errors
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into clean-testutil
* Revert error handling

Revert "Fix errors"

This reverts commit db081f5486.

Revert "Handle error for all testutil uses"

This reverts commit bdabef2306.

Revert "Add back error handling"

This reverts commit da7e3d2020.
* Change genesis state func to use testing.T
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into clean-testutil
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into clean-testutil
* Fix conflict
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into clean-testutil
* Merge branch 'master' into clean-testutil
* Merge branch 'master' into clean-testutil
* Captialize other logs
* Merge branch 'clean-testutil' of https://github.com/0xKiwi/Prysm into clean-testutil
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into clean-testutil
* Merge branch 'master' into clean-testutil
2019-12-05 19:51:33 +00:00
Nishant Das
c36a852329 Swap to Ristretto Cache (#4070)
* add new cache
* change to larger size
* Merge branch 'master' into swapCache
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into swapCache
* remove imports
* cache fixes
* Merge branch 'master' into swapCache
* add better costing
* Merge branch 'swapCache' of https://github.com/prysmaticlabs/geth-sharding into swapCache
* comment
* change back to var
* Merge branch 'master' into swapCache
* Merge branch 'master' into swapCache
* Merge branch 'master' into swapCache
* Merge branch 'master' into swapCache
* Merge refs/heads/master into swapCache
2019-12-05 19:13:11 +00:00
Jim McDonald
650a278fee Harden BLS against invalid input (#4203)
* Harden BLS against invalid input
* Merge branch 'master' into blsharden
* Merge branch 'master' into blsharden
* Merge branch 'master' into blsharden
2019-12-05 18:33:29 +00:00
Ivan Martinez
6816337589 Make logs more helpful for E2E (#4198)
* Make logs more helpful for E2E
* gofmt
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into helpful-logs
* Add extra info
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into helpful-logs
* gofmt and fix error output
* Use errors
* Gazelle
* Revert "gofmt and fix error output"

This reverts commit 9fc85f2dd2.
* Formatting and fix
* add f
* Add more details to logs
* Merge branch 'master' into helpful-logs
* Change text a bit
* Merge branch 'helpful-logs' of https://github.com/0xKiwi/Prysm into helpful-logs
* Merge branch 'master' into helpful-logs
2019-12-05 18:00:55 +00:00
Preston Van Loon
2950e4aeb4 Faucet: Add score in error (#4200)
* Update server.go
* Merge refs/heads/master into prestonvanloon-patch-2
2019-12-05 17:37:19 +00:00
Jim McDonald
746cc142d0 Remove erroneous err (#4202) 2019-12-05 05:53:39 -08:00
Ivan Martinez
261428118e Isolate BLS pubkey cache to only when cache is enabled (#4195)
* Only add to cache when cache is enabled
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into isolate-cache
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into isolate-cache
* Merge branch 'master' into isolate-cache
2019-12-05 02:56:36 +00:00
Preston Van Loon
544e5309ad Faucet: Use client IP in captcha requests (#4196)
* Use client IP in captcha requests
2019-12-05 02:18:15 +00:00
terence tsao
23dd951e59 Chain head state return nil instead of err (#4193)
* Return nil instead of err
* Preston's feedback
* Merge branch 'master' into return-nil
2019-12-04 23:48:30 +00:00
Preston Van Loon
498417a8fc Wait for 3 peers to start sync (#4194)
* Update service.go
2019-12-04 23:09:47 +00:00
Preston Van Loon
617325b726 Faucet improvements (#4192)
* Add faucet reCaptcha improvements in verification
* Add faucet reCaptcha improvements in verification
* add roughtime
2019-12-04 20:33:46 +00:00
Raul Jordan
9e5cc81340 Implement Prysm-Specific HashTreeRootState (#4077)
* new ssz hash tree root
* Merge branch 'master' into new-ssz-state
* better comments on func
* add errors instead of panic in state
* utilize errors wrap everywhere
* include bench
* added bench info
* equality test
* dup
* gaz
* use new hash tree root in state transition
* fix build
* separate test package
* three targets failign
* single target fails
* please test targets...pass for me
* revert
* Merge branch 'master' into new-ssz-state
* rev
* Merge branch 'new-ssz-state' of github.com:prysmaticlabs/prysm into new-ssz-state
* broken build
* Merge branch 'master' into new-ssz-state
* gaz
* Merge branch 'new-ssz-state' of github.com:prysmaticlabs/prysm into new-ssz-state
* ssz workspace
* master ssz
* Merge branch 'master' into new-ssz-state
* resolve conf
* resolve some conflicts and fix up broken file
* fix up build file issues and sync
* eth1 data votes included
* further abstractions, simplifications
* Merge branch 'master' into new-ssz-state
* gaz
* Merge branch 'new-ssz-state' of github.com:prysmaticlabs/prysm into new-ssz-state
* feature flag gating
* add field count test
* Merge branch 'master' into new-ssz-state
* resolving ivan feedback
* Merge branch 'new-ssz-state' of github.com:prysmaticlabs/prysm into new-ssz-state
* gaz
* Merge branch 'master' into new-ssz-state
* addressed
* Merge branch 'new-ssz-state' of github.com:prysmaticlabs/prysm into new-ssz-state
2019-12-04 19:20:33 +00:00
terence tsao
f75a5a5df8 Implement Atts Pool Curl Methods (#4191)
* New pool
* Better namings
* Fmt
* Gazelle
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into define-pool
* Raul's feedback
* Raul's feedback
* Update to use go-cache
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into define-pool-1
* Update workspace
* Update workspace
* Update pool to use interface
* Move kv init methods
* Curd for aggregated
* Curd for unaggregated
* Gaz
* Tests for aggregated
* Fixed test
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into curd
* Minor fixes
* Typoe
* pool test
* Added deletions as well
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into curd
* Update beacon-chain/operations/attestations/kv/aggregated.go
* Update beacon-chain/operations/attestations/kv/aggregated.go
* Update beacon-chain/operations/attestations/kv/unaggregated_test.go
* Update beacon-chain/operations/attestations/kv/kv.go
2019-12-04 18:30:45 +00:00
shayzluf
ae8df9c32b Slasher optimization (#4172)
* size

* batching and concurrency improvements

* gaz

* merge fixes

* fix comment

* fix test

* fix test

* fix build

* ethpb

* ethpb

* fix test

* fix comment

* add benchmark

* fix benchmark
2019-12-04 12:09:38 +05:30
Raul Jordan
90cbe49496 Implement GetValidator RPC Endpoint (#4188)
* include new patch targeting latest ethapis master
* ensure project builds
* Merge branch 'master' into update-all-api
* fix up committees
* Merge branch 'update-all-api' of github.com:prysmaticlabs/prysm into update-all-api
* include latest eth apis
* Merge branch 'master' into update-all-api
* update block tests
* Merge branch 'update-all-api' of github.com:prysmaticlabs/prysm into update-all-api
* Merge branch 'master' into update-all-api
* add todos
* implement get validator rpc
* add test for get validator
* table driven test
* fix up test
* fix confs
* tests for more cases
* fix up tests and add out of range
2019-12-04 00:33:34 +00:00
Raul Jordan
c31f46d973 Update EthereumAPIs (#4186)
* include new patch targeting latest ethapis master
* ensure project builds
* Merge branch 'master' into update-all-api
* fix up committees
* Merge branch 'update-all-api' of github.com:prysmaticlabs/prysm into update-all-api
* include latest eth apis
* Merge branch 'master' into update-all-api
* update block tests
* Merge branch 'update-all-api' of github.com:prysmaticlabs/prysm into update-all-api
* Merge branch 'master' into update-all-api
* add todos
2019-12-03 23:44:58 +00:00
terence tsao
83781d0b74 Attestation pool to use go-cache (#4187) 2019-12-03 15:07:44 -08:00
Jim McDonald
6488b0527c Log peer conected log for incoming connections (#4173)
* Log peer conected log for incoming connections
* Merge branch 'master' into peerconnected
* Merge branch 'master' into peerconnected
* Update handshake.go
* Update handshake.go
* Merge branch 'master' into peerconnected
* Merge branch 'master' into peerconnected
2019-12-03 22:37:49 +00:00
terence tsao
eeb8779cfc New attestation pool (#4185)
* New pool
* Better namings
* Fmt
* Gazelle
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into define-pool
* Raul's feedback
* Raul's feedback
2019-12-03 22:04:11 +00:00
Raul Jordan
f40bbb92d1 Resolve Broken Active Changes for Validators (#4182)
* fix issues with exited validator indices
* tests pass for validator active set changes and exited keys
* Merge branch 'master' into cached-active-changes
* resolve archive test
* Merge branch 'cached-active-changes' of github.com:prysmaticlabs/prysm into cached-active-changes
* Merge branch 'master' into cached-active-changes
* Merge branch 'master' into cached-active-changes
2019-12-03 21:34:52 +00:00
Nishant Das
4f0bef929f Change BLS to Herumi Again (#4181)
* change to herumi's bls
* change alias
* change to better
* add benchmark
* build
* change to bazel fork
* fix prefix
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into herumiBLS
* make it work with library
* update to latest
* change again
* add import
* update to latest
* add sha commit
* new static lib with groups swapped
* using herumis new lib
* fix dep paths in c headers
* update again
* new changes
* fix commit
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into herumiBLS
* fix serialization
* comment
* fix test
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into herumiBLS
* fix to herumis latest version
* fix test
* fix benchmarks
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into herumiBLS
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into herumiBLS
* add new workspace
* change commit and remove init
* get test to pass
* remove parameter
* remove reverse byte order
* make gazelle happy
* set pure to off
* fix failing tests
* Merge branch 'master' into herumiBLS
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into herumiBLS
* Merge branch 'herumiBLS' of https://github.com/prysmaticlabs/geth-sharding into herumiBLS
* remove old ref
* use HashWithDomain functions
* update to latest version
* clean up
* gaz
* add back removed code
* switch off pure
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into herumiBLS
* use local repo
* resolve docker issues
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into herumiBLS
* fix build and tests
* gaz
* Merge branch 'master' into herumiBLS
* Merge refs/heads/master into herumiBLS
* Merge refs/heads/master into herumiBLS
2019-12-03 20:29:05 +00:00
Raul Jordan
81a83cf100 Implement Chain Head Stream & Naming Consistency (#4160)
* include stream chain head mock
* uncomment test
* stream chain head implemented
* remove imports
* chain head stream test
* include stream test with mockgen
* test now passes
* checkin items
* stream tests all passing
* rem learn
* fix up fork checker
* add stream ctx
* gaz, fix test
* fix broken test
* Merge branch 'master' into chain-head-stream
* include context in chain head stream happy path test
* Merge branch 'master' into chain-head-stream
* Merge branch 'master' into chain-head-stream
* Merge refs/heads/master into chain-head-stream
* Merge refs/heads/master into chain-head-stream
2019-12-03 19:48:11 +00:00
terence tsao
8bbc589edd Spans and check type (#4164)
* Spans and check type
* Typos
* Remove type checks
* Fixed a test bug
* Merge branch 'master' into subs-fixes
* Merge branch 'master' into subs-fixes
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into subs-fixes
* Revert back type assertions
* Merge branch 'master' into subs-fixes
* Merge branch 'subs-fixes' of https://github.com/prysmaticlabs/prysm into subs-fixes
* Merge branch 'master' into subs-fixes
2019-12-03 19:15:01 +00:00
Preston Van Loon
32245a9062 Deprecate --init-sync-no-verify, make it the default (#4179)
* deprecated --init-sync-no-verify, make it the default
* Merge branch 'master' into deprecate-init-sync-verify-flag
* add more flag info
* Merge branch 'deprecate-init-sync-verify-flag' of github.com:prysmaticlabs/prysm into deprecate-init-sync-verify-flag
* gofmt
* Merge refs/heads/master into deprecate-init-sync-verify-flag
* Merge refs/heads/master into deprecate-init-sync-verify-flag
2019-12-03 18:46:04 +00:00
Nishant Das
28c4f28d32 Add Strict Connection Manager (#4110)
* add forked connMgr
* gaz
* add license header
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into connMgr
* add conn manager test
* gaz
* fix connManager
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into connMgr
* gaz
* remove todo
* add new dep
* lint
* lint
* lint
* space
* visibility
* Merge branch 'master' into connMgr
* Merge branch 'master' into connMgr
* Merge refs/heads/master into connMgr
2019-12-03 18:18:57 +00:00
Peter Pratscher
a2d4701f6e Update API to return empty next page token on last page response (#4176)
* Update API to return empty next page token on last page response
* Update tests
* Merge branch 'master' into api-return-empty-page-token-on-last-page
2019-12-03 17:45:43 +00:00
Nishant Das
8e4022f8aa Remove Outdated Proto Files (#4178)
* remove gateway folder
* update att container
* Merge branch 'master' into removeProtoFiles
* Merge refs/heads/master into removeProtoFiles
2019-12-03 16:45:20 +00:00
terence tsao
42e766e909 Fix on participation RPC return (#4171)
* Edit returning epoch
* Merge branch 'master' into fix-participation-typos
2019-12-03 16:23:03 +00:00
Nishant Das
a686be8bd0 Revert "Revert "Update Pending Queue (#4066)" (#4101)" (#4168)
This reverts commit 7a9c297206.
2019-12-03 07:56:04 -08:00
Nishant Das
e3c3dea5d2 Regenerate Missing Proto Files (#4163)
* regen missing proto files
* Merge branch 'master' into regenProto
2019-12-02 23:19:37 +00:00
Ivan Martinez
7754cfb6c6 Reduce amount of time for benchmark tests (#4166)
* Change benchmarks to use different cache

* Fix bench tests and cache

* Add back sig check for test
2019-12-02 15:54:57 -05:00
Andrei Ivasko
f55a380ade Deposit testing (#4043)
* debugging...
* debugging... feedback required
* moved sendDeposits_test to powchain package
* need some guidance to proceed further
* further guidance needed
* match depositData to depositEvent
* debugging validating merkle root
* fixed compile error
* test passed for a single deposit
* Unable verify deposit merkle branch
* fix test
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into AndreisPR
* ready for review
* Merge branch 'master' into deposit-testing
* Merge branch 'master' into deposit-testing
* applied requested changes
* Merge branch 'master' into deposit-testing
2019-12-01 22:23:55 +00:00
Preston Van Loon
9a317ffc0f Update tracing dependencies (#4158)
* update tracing deps
2019-12-01 06:01:53 +00:00
Raul Jordan
3be4894b8a Update Ethereum APIs, Allow Genesis Data Retrieval for Blocks + Attestations (#4150)
* update apis

* include block filter genesis

* genesis atts

* add in workspace file

* include proper diff targeting master of ethereum apis

* genesis block fetching fixes

* remove fmt

* tests for genesis list blocks passing

* fixed up container tests

* tests now passing

* fix up tests
2019-11-30 22:30:48 -06:00
Jim McDonald
646411b881 Log connections (#4143)
* Log connections
* Merge branch 'master' into logconnection
* Merge branch 'master' into logconnection
* Merge branch 'master' into logconnection
2019-12-01 01:37:42 +00:00
Jim McDonald
0e99e4af4f Update lastUpdated on peerstatus.Set() (#4152)
* Update lastUpdated on Set()
* Merge branch 'master' into lastupdated
* gazelle
* Merge branch 'master' into lastupdated
* Merge branch 'master' into lastupdated
2019-11-30 19:47:08 +00:00
terence tsao
e87337a97a Update forkchoice spec link to v0.9.0 (#4147)
* Update forkchoice doc link to v0.9.0
* Merge refs/heads/master into update-link
2019-11-30 05:48:18 +00:00
Jim McDonald
53523b3eef Implement ListPeers API call (#4151)
* update ethereumapis from https://github.com/prysmaticlabs/ethereumapis/pull/55
* add stub for https://github.com/prysmaticlabs/prysm/issues/4141
* Add ListPeers API call
* Merge
* Add comment for exported method
* Fix visibility of new peers package.
* Merge branch 'master' into peersapi
2019-11-30 05:36:02 +00:00
terence tsao
5ec02b28a5 Remove pruned states check (#4153)
* Removed already pruned check
* Tested run time
2019-11-30 05:07:13 +00:00
Raul Jordan
1620290305 Check in archive.pb.go (#4148)
* gen archive.pb
* Merge branch 'master' into regen-protos
2019-11-29 18:28:21 +00:00
Preston Van Loon
fc171434c5 Update README.md to reflect spec version (#4146)
* Update README.md
2019-11-29 17:15:14 +00:00
Preston Van Loon
b08f3f760d Update ethereumapis (#4142)
* update ethereumapis from https://github.com/prysmaticlabs/ethereumapis/pull/55
* add stub for https://github.com/prysmaticlabs/prysm/issues/4141
2019-11-29 16:44:51 +00:00
terence tsao
7495961d6b Prune boundary state (#4139)
* Delete epoch boundary slot of last finalized epoch
* Case to cover start slot is skipped
* Test
* Feature flag
* feature gate the new functionality only
* Update DB for migration
* Test
* Fmt
* Fixed test
* Gazelle
2019-11-28 23:05:47 +00:00
Preston Van Loon
4dbf68b50c Fix log message from PR #4130 (#4136)
* fix log message from PR #4130
* Merge refs/heads/master into fix-log-2
2019-11-27 23:49:47 +00:00
Raul Jordan
e24b060eb6 README for Third Party Directory (#4134)
* begin readme
* add common bugs
* include more details for third party readme
* patch diff
* add readme
* complete readme
* Merge branch 'master' into third-party-readme
* rev
* Merge branch 'third-party-readme' of github.com:prysmaticlabs/prysm into third-party-readme
* revert
* Update third_party/README.md
* Update third_party/README.md
2019-11-27 23:27:13 +00:00
terence tsao
e90358cd8e Removed unused mocks (#4135)
* Removed unused mocks

* Lint

* Gaz
2019-11-27 14:52:24 -06:00
Nishant Das
80865ff3f2 Account for Skipped Slots When Requesting for Blocks (#4130)
* add check

* Update beacon-chain/sync/initial-sync/round_robin.go
2019-11-27 11:18:18 -06:00
Jim McDonald
60469ec7ee Avoid crash if peer goes missing (#4115)
* Migrate ChainStarted and StateInitialized to state notifier
* Provide state notifier to powchain service
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Remove commented line
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge remote-tracking branch 'upstream/master'
* Merge branch 'master' of github.com:mcdee/prysm
* Accept err from HeadState() as non-fatal
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge remote-tracking branch 'upstream/master'
* Merge branch 'master' of github.com:mcdee/prysm
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Do not crash if peer goes missing
* Additional catches
* Merge branch 'master' into rrfix
* Use single refresh time
* Merge branch 'master' into rrfix
2019-11-27 16:00:59 +00:00
Jim McDonald
67be8bd4f0 Mirror run definitions in build (#4129)
* Migrate ChainStarted and StateInitialized to state notifier
* Provide state notifier to powchain service
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Remove commented line
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge remote-tracking branch 'upstream/master'
* Merge branch 'master' of github.com:mcdee/prysm
* Accept err from HeadState() as non-fatal
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge remote-tracking branch 'upstream/master'
* Merge branch 'master' of github.com:mcdee/prysm
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Mirror run arguments in build
* Reset ssz to mainnet for testing
2019-11-27 15:34:57 +00:00
Nishant Das
3682bf1cda Check Best Peer Before Syncing (#4128)
* add check
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into checkBestPeer
* Merge refs/heads/master into checkBestPeer
2019-11-27 06:56:02 +00:00
Preston Van Loon
e203f66fe0 DB Improvements: Snappy compression, remove some unnecessary batch / goroutines (#4125)
* do not use batch for SaveAttestations
* use snappy compression
* Encode / decode everything with snappy
* Add snappy migration path
* batch is probably fine...
* fix test
* gofmt
* Merge branch 'master' of github.com:prysmaticlabs/prysm into remove-batch-attestations
* add sanity check
* remove that thing
* gaz
* Merge branch 'master' of github.com:prysmaticlabs/prysm into remove-batch-attestations
2019-11-27 06:32:56 +00:00
terence tsao
04df922ac9 Add votes to tree graph (#4127)
* Added block tree tool
* Gaz
* Updated workspace
* Playing around
* Adding votes
* Votes work
* Comments
* Gaz
* Add tools to subpackage
* Merge branch 'master' into block-tree-tool-1
2019-11-27 06:07:52 +00:00
Raul Jordan
0326be86b5 Apply Patch Rules to Use EthereumAPIs Generated Protos in Prysm (#4112)
* starting on patch
* finish determining all required patches
* properly redefine the patch rules
* new patch
* rem double semicolon
* fix patch file
* Merge branch 'master' of github.com:prysmaticlabs/prysm into deprecate-eth-protos
* building the deps
* test target passes using ethereumapis
* compile gateway
* attempting to build everything
* e2e use ethereumapis
* more fixes for slasher
* other item
* getting closer to compiling slasher
* build slasher package
* Merge branch 'master' into deprecate-eth-protos
* Merge branch 'master' into deprecate-eth-protos
* fix benches
* lint gazelle
* Merge branch 'deprecate-eth-protos' of github.com:prysmaticlabs/prysm into deprecate-eth-protos
* proper gateway
* lint
* Merge branch 'master' into deprecate-eth-protos
* fix build
* Merge branch 'deprecate-eth-protos' of github.com:prysmaticlabs/prysm into deprecate-eth-protos
* use swag
* resolve
* ignore change
* include new patch changes
* fix test
* builds
* fix e2e
* gaz
2019-11-27 05:08:18 +00:00
Nishant Das
a7ccd52a95 Save Deposit Contract Address (#4114)
* save contract address
* Update beacon-chain/node/node.go
* Merge branch 'master' into saveContract
* Merge refs/heads/master into saveContract
2019-11-26 21:01:56 +00:00
terence tsao
1ced4754db Add signatures to logs (#4095)
* Enhance logging with sig
* Fixed
* Merge branch 'master' into add-sig
* Merge branch 'master' into add-sig
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into add-sig
* signature via debug
* Merge branch 'add-sig' of https://github.com/prysmaticlabs/prysm into add-sig
* Merge branch 'master' into add-sig
* Merge branch 'master' into add-sig
* Merge branch 'master' into add-sig
2019-11-26 20:36:18 +00:00
terence tsao
b872f74fd3 Do not save duplicated indices (#4118)
* Added a duplication test

* Refactor

* Updated test

* Do not save dups for indices bucket
2019-11-26 13:26:35 -06:00
Ivan Martinez
c1c48a8af5 Create Benchmarks Package for State Transition (#3688)
* Begin benchmarks file for block processing
* Complete block processing benchmarks
* Begin epoch benchmarks
* Write most of epoch benchmarks
* Start config
* Make cases for max conditions
* Begin work on benchmarking doc
* Update benchmark numbers
* Complete epoch benchmarks
* Minor changes
* Make createFullBlock function
* Clean up block benchmarks
* Begin fixing merge issues
* Start adding 4M benchmarks
* Almost finish epoch benchmarks
* Test blocks under real life conditions
* More progress on benchmarks
* Fixes
* Fix benchmark errors
* Begin fixing benchmarks
* More progress on tests
* Complete epoch benchmarks
* More progress on block benches
* Finish epoch benchmarks, get progress on block benchmarks
* Undo unneeded changes
* Fix
* Fix block benchmarks
* Complete block benchmarks
* Finish block benchmarks
* Complete benchmarks
* Increase block benchmarks to 65536
* Fix everything
* Reset configs after benchmarks
* Fix logging and suggestions
* Fix comments
* Fix benchmarks after merge
* Fix merge issues
* Add sanity tests for benchmark
* Make sanity check simpler
* Begin fixing after merge
* Add log
* Remove extra line
* Remove unneeded change
* Finally get block benchmarks to pass
* Begin fixing epoch test
* Finetuning constants
* Revert "Finetuning constants"

This reverts commit a872790d67.
* Finetuning
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Add benches for helper functions
* Abstract block generation to testutil
* Create block generation util in testutil
* Gazelle
* Fix deps
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into block-util
* Fix imports
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into block-util
* Merge branch 'master' into block-util
* Change tests to use config and fix integer division
* Merge branch 'block-util' of https://github.com/0xKiwi/prysm into block-util
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into block-util
* Remove logs
* Fix build
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Merge branch 'master' into block-util
* Add test to ensure finalization occurs
* Add check for finalization
* Merge branch 'block-util' of https://github.com/0xKiwi/prysm into block-util
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into block-util
* Add comment for incrementing the state
* Fix test
* Fix test
* Merge branch 'master' into block-util
* Fix testutil use
* Fix tests
* Change var name
* Merge branch 'master' into block-util
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Merge branch 'block-util' of https://github.com/0xKiwi/prysm into new-benchmarks
* Begin cleaning benchmarks
* Get some numbers going
* Use state saved to disk
* Remove cruft
* Cleanup
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Fix merge arrows
* Set up block util and benchmarks for 128 attestations
* Use intended config for benchmark
* Add more benchmark functions
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Add benchmark epoch and modify block gen config to exclude signing
* Cleanup
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Begin unstaleling
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Update block gen util to v0.9 changes
* Prepare benchmarks to use marshalled files
* Cleanup block gen tool some more
* split up into file generation and benchmarking
* Remove logrus
* Merge branch 'master' into new-benchmarks
* Get benchmarks work, start work on process epoch benchmark
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Merge branch 'new-benchmarks' of https://github.com/0xKiwi/prysm into new-benchmarks
* All benchmarks working
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Fix after merge
* Cleanup
* Add bazel target
* Added TestBenchmarkExecuteStateTransition_WithCache
* Change tests to use SSZ and begin making binary
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Merge branch 'new-benchmarks' of https://github.com/0xKiwi/prysm into new-benchmarks
* bazel binary
* Fully change to binary
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Create go_binary to handle benchmark files
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Gofmt
* Remove genesis state from generated files
* Fix tests
* Gazelle
* Fix tests
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Fix block util
* Allow attestations to be in future for block util
* Fix inclusion delay issue
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Finally fix test
* Add README detailing usage and results
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Change test to run with bazel test
* Fix imports
* Merge branch 'master' into new-benchmarks
* Accidentally removed  config change
* Merge branch 'new-benchmarks' of https://github.com/0xKiwi/prysm into new-benchmarks
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Move to core/state/
* Update readme
* Gazelle
* Remove test for cached block
2019-11-26 18:09:57 +00:00
Nishant Das
b88e6dc918 Speed Up Block Processing In Sync (#4075)
* fix proto
* make them non-batched
* gate behind flag
* fix refs
* fix refs
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into speedUpProcessing
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into speedUpProcessing
* use global archiver flags
* lint
* Merge branch 'master' into speedUpProcessing
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into speedUpProcessing
* preston's review
* Merge branch 'speedUpProcessing' of https://github.com/prysmaticlabs/geth-sharding into speedUpProcessing
* Merge branch 'master' into speedUpProcessing
* Merge branch 'master' into speedUpProcessing
* Merge branch 'master' into speedUpProcessing
2019-11-26 07:15:54 +00:00
terence tsao
3868837471 Tool to dump graphviz data for block tree (#4108) 2019-11-25 21:06:25 -08:00
Preston Van Loon
60b1596c4d Lower conn mgr grace period to 1s (#4109)
* Update options.go
2019-11-26 01:32:10 +00:00
Raul Jordan
4f0dcd5e6e Prevent Requesting Current Epoch for Validator Participation (#4104)
* cannot request current epoch
* test for prev epoch instead
* Merge branch 'master' into no-curr-epoch-participation
* Merge branch 'master' into no-curr-epoch-participation
2019-11-25 23:31:21 +00:00
Preston Van Loon
ac405c714f Enforce --p2p-max-peers (#4106)
* Enforce p2p-max-peers
* high == low
2019-11-25 18:55:20 +00:00
Nishant Das
7d0e5a9dc4 Use Latest Vote Map (#4102)
* add latest vote map
* fix all tests
* remove db crud methods
* Merge branch 'master' into latestVoteMap
* preston's review
* Merge branch 'latestVoteMap' of https://github.com/prysmaticlabs/geth-sharding into latestVoteMap
2019-11-25 16:34:20 +00:00
Raul Jordan
feb1267fee Properly Return Finalized Epoch in GetValidatorParticipation Archival Endpoint (#4091)
* properly handle retrieving archived finalized epochs
* test passes for determining if epoch finalized
* Merge branch 'master' into archive-finality
* Merge refs/heads/master into archive-finality
* Merge refs/heads/master into archive-finality
* Merge refs/heads/master into archive-finality
* Merge refs/heads/master into archive-finality
* Merge refs/heads/master into archive-finality
* Merge branch 'master' into archive-finality
* prevent setup panic
* Merge branch 'archive-finality' of github.com:prysmaticlabs/prysm into archive-finality
* Merge refs/heads/master into archive-finality
2019-11-25 15:26:32 +00:00
Nishant Das
7a9c297206 Revert "Update Pending Queue (#4066)" (#4101)
This reverts commit a264a097cc.
2019-11-24 21:05:51 -08:00
terence tsao
21deed0fb7 Revert "Add Lock When Accessing Checkpoints" (#4094)
* Revert "Add Lock When Accessing Checkpoints (#4086)"

This reverts commit 2f392544a6.
* Merge branch 'master' into revert-4086-checkpointLock
2019-11-24 17:12:56 +00:00
terence tsao
627791c54e Complete finalization metrics (#4096)
* Complete finaliztion metrics

* Fixed test
2019-11-23 17:12:29 -08:00
Jim McDonald
3358bde42d Feedfixes (#4093)
* Migrate ChainStarted and StateInitialized to state notifier
* Provide state notifier to powchain service
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Remove commented line
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge remote-tracking branch 'upstream/master'
* Merge branch 'master' of github.com:mcdee/prysm
* Accept err from HeadState() as non-fatal
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge remote-tracking branch 'upstream/master'
* Merge branch 'master' of github.com:mcdee/prysm
* Explicit unsubscribes from state channels where required
2019-11-23 11:15:02 +00:00
Jim McDonald
9e45cffabc Move StateInitialized and ChainStarted to state feed (#4084)
* Migrate ChainStarted and StateInitialized to state notifier
* Provide state notifier to powchain service
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Remove commented line
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge remote-tracking branch 'upstream/master'
* Merge branch 'master' of github.com:mcdee/prysm
* Accept err from HeadState() as non-fatal
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge branch 'master' into master
2019-11-23 03:35:47 +00:00
terence tsao
2c8ff7b36f Update attester to wait till one-third (#4090)
* Update to 1/3
* Use tag
* Test
* Fixed test
* Merge branch 'master' into update-1/3
* Merge branch 'master' into update-1/3
2019-11-23 02:23:20 +00:00
terence tsao
a7ec0679b5 Implement wait till two thirds (#4089)
* Implement two thirds

* Test
2019-11-22 16:48:40 -06:00
Preston Van Loon
f717c5d852 Release --prune-finalized-states to all (#4082)
* Release --prune-finalized-states to all
* Merge branch 'master' into deprecate-ff-prune-finalized-states
* Merge branch 'master' of github.com:prysmaticlabs/prysm into deprecate-ff-prune-finalized-states
* Merge refs/heads/master into deprecate-ff-prune-finalized-states
* Merge refs/heads/master into deprecate-ff-prune-finalized-states
* Merge refs/heads/master into deprecate-ff-prune-finalized-states
* Merge refs/heads/master into deprecate-ff-prune-finalized-states
* Merge branch 'master' of github.com:prysmaticlabs/prysm into deprecate-ff-prune-finalized-states
2019-11-22 20:29:46 +00:00
Preston Van Loon
0cec0ee6c3 Release --optimize-process-epoch to all (#4080)
* Release optimize-proces-epoch to all
* Merge branch 'master' into deprecate-ff-optimize-process-epoch
* Merge branch 'master' of github.com:prysmaticlabs/prysm into deprecate-ff-optimize-process-epoch
* Merge refs/heads/master into deprecate-ff-optimize-process-epoch
* Merge refs/heads/master into deprecate-ff-optimize-process-epoch
* Merge refs/heads/master into deprecate-ff-optimize-process-epoch
* Merge refs/heads/master into deprecate-ff-optimize-process-epoch
2019-11-22 09:40:53 +00:00
Nishant Das
2f392544a6 Add Lock When Accessing Checkpoints (#4086)
* fix data races
* Merge branch 'master' into checkpointLock
* Merge branch 'master' into checkpointLock
* Merge refs/heads/master into checkpointLock
2019-11-22 06:34:42 +00:00
Preston Van Loon
75ce8359eb Buildkite: Disable failing BES (#4087)
* Disable failing BES
2019-11-22 06:11:41 +00:00
terence tsao
f5cb04012e Aggregator selection from RPC to validator client (#4071)
* Config
* Updated proto
* Updated pool
* Updated RPC
* Updated validator client
* run time works
* Clean ups
* Fix tests
* Visibility
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into aggregator
* Raul's feedback
* Tests for RPC server
* Tests for validator client
* Span
* More tests
* Use go routine for SubmitAggregateAndProof
* Go routines
* Updated comments
* Use array of roles
* Fixed tests
* Build
* Update validator/client/runner.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update validator/client/runner.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* If
* Merge branch 'refactor-validator-roles' of https://github.com/prysmaticlabs/prysm into refactor-validator-roles
* Empty
* Feedback
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into aggregator
* Removed proto/eth/v1alpha1/shard_chain.pb.go?
* Cleaned up
* Revert
* Comments
* Lint
* Comment
* Merge branch 'master' into aggregator
2019-11-22 05:11:38 +00:00
Raul Jordan
f461d1e024 Resolve Panic in ListValidatorBalances RPC (#4051)
* adding default response
* regression test for archival
* include full regression test
* Merge branch 'master' into resolve-panic-rpc
* Merge branch 'master' into resolve-panic-rpc
* Merge branch 'master' into resolve-panic-rpc
* Merge branch 'master' into resolve-panic-rpc
* listbal
* Merge branch 'master' into resolve-panic-rpc
* Merge branch 'master' into resolve-panic-rpc
* Merge branch 'master' into resolve-panic-rpc
* Merge refs/heads/master into resolve-panic-rpc
2019-11-22 04:39:28 +00:00
Preston Van Loon
bdbd0aaeb8 Deprecate feature flag --scatter (#4079)
* deprecate --scatter. issue #4031
* forgot one for #4061
* use deprecatedUsage
* hidden
* Merge branch 'master' into deprecate-ff-scatter
* Merge branch 'master' into deprecate-ff-scatter
2019-11-22 04:08:49 +00:00
Preston Van Loon
715d06a215 gRPC Gateway: Emit JSON empty fields by default (#4085)
* Emit JSON empty fields by default
* Merge branch 'master' into emit-json-empty-fields
2019-11-22 03:36:47 +00:00
terence tsao
976a3af637 Refactor validator roles into an array (#4081) 2019-11-21 14:35:20 -08:00
Raul Jordan
8f8d2d36c0 Filter ListValidators by Active in RPC (#4061)
* update workspace

* include active filter

* fix up latest changes to match naming

* better comments, fix evaluators

* latest master

* filter items

* filter only active validators
2019-11-21 14:29:24 -06:00
Nishant Das
a264a097cc Update Pending Queue (#4066)
* update queue

* fix test

* put this all in validate method

* remove ancestors too

* not needed

* terence's review

* period

* preston's review
2019-11-21 21:24:50 +08:00
shayzluf
4330839bc1 Add surround check to endpoint (#4065)
* first version of the watchtower api

* service files

* Begin work on grpc server

* More changes to server

* REnames and mock setup

* working test

* merge

* double propose detection test

* nishant review

* todo change

* gaz

* fix service

* gaz

* remove unused import

* gaz

* resolve circular dependency

* resolve circular dependency 2nd try

* remove package

* fix package

* fix test

* added tests

* gaz

* remove status check

* gaz

* remove context

* remove context

* change var name

* moved to rpc dir

* gaz

* remove server code

* gaz

* slasher server

* visibility change

* pb

* service update

* gaz

* slasher grpc server

* making it work

* setup db and start

* gaz

* service flags fixes

* grpc service running

* go imports

* remove new initializer

* gaz

* remove feature flags

* change back SetupSlasherDB

* fix SetupSlasherDB calls

* define err

* fix bad merge

* fix test

* fix imports

* fix imports

* fix imports

* add cancel

* comment stop

* fix cancel issue

* remove unneeded code

* bring back bad merge that removed TODO

* remove use of epoch as am input

* fixed slasher to be runable again

* wait for channel close

* gaz

* small test

* flags fix

* fix flag order

* double vote detection

* remove source epoch from indexed attestation indices

* change server method to receive indexed attestation

* start implementation

* double vote detection

* proto

* pb

* fir comment

* add surround detection and retrieval to endpoint

* nishant review

* import fix

* fix miss order

* fix detection 0 case
added tests

* terence review
2019-11-21 12:41:23 +05:30
terence tsao
835418d1e3 Fix UpdateCommitteeCache slot (#4074) 2019-11-20 21:29:43 -08:00
3344 changed files with 484093 additions and 106034 deletions

223
.bazelrc
View File

@@ -5,20 +5,229 @@ test --test_verbose_timeout_warnings
test --build_tests_only
test --test_output=errors
# E2E run with debug gotag
test:e2e --define gotags=debug
# Clearly indicate that coverage is enabled to disable certain nogo checks.
coverage --define=coverage_enabled=1
# Fix for rules_docker. See: https://github.com/bazelbuild/rules_docker/issues/842
build --host_force_python=PY2
test --host_force_python=PY2
run --host_force_python=PY2
# Networking is blocked for tests by default, add "requires-network" tag to your test if networking
# is required within the sandbox. This flag is no longer experimental after 0.29.0.
# Network sandboxing only works on linux.
--experimental_sandbox_default_allow_network=false
# is required within the sandbox. Network sandboxing only works on linux.
build --sandbox_default_allow_network=false
# Use minimal protobufs at runtime
run --define ssz=minimal
# Stamp binaries with git information
build --workspace_status_command=./hack/workspace_status.sh
# Prevent PATH changes from rebuilding when switching from IDE to command line.
build --incompatible_strict_action_env
test --incompatible_strict_action_env
run --incompatible_strict_action_env
build --define blst_disabled=false
run --define blst_disabled=false
build:blst_disabled --define blst_disabled=true
build:blst_disabled --define gotags=blst_disabled
build:minimal --//proto:network=minimal
build:minimal --@io_bazel_rules_go//go/config:tags=minimal
# Release flags
build:release --compilation_mode=opt
build:release --stamp
# LLVM compiler for building C/C++ dependencies.
build:llvm --define compiler=llvm
build:llvm --copt -fno-sanitize=vptr,function
build:llvm --linkopt -fno-sanitize=vptr,function
# --incompatible_enable_cc_toolchain_resolution not needed after this issue is closed https://github.com/bazelbuild/bazel/issues/7260
build:llvm --incompatible_enable_cc_toolchain_resolution
build:asan --copt -fsanitize=address,undefined
build:asan --copt -fno-omit-frame-pointer
build:asan --linkopt -fsanitize=address,undefined
build:asan --copt -fno-sanitize=vptr,function
build:asan --linkopt -fno-sanitize=vptr,function
build:asan --copt -DADDRESS_SANITIZER=1
build:asan --copt -D__SANITIZE_ADDRESS__
build:asan --linkopt -ldl
build:llvm-asan --config=llvm
build:llvm-asan --config=asan
build:llvm-asan --linkopt -fuse-ld=ld.lld
build:fuzz --@io_bazel_rules_go//go/config:tags=fuzz
# Build binary with cgo symbolizer for debugging / profiling.
build:cgo_symbolizer --config=llvm
build:cgo_symbolizer --copt=-g
build:cgo_symbolizer --define=USE_CGO_SYMBOLIZER=true
build:cgo_symbolizer -c dbg
build:cgo_symbolizer --define=gotags=cgosymbolizer_enabled
# multi-arch cross-compiling toolchain configs:
-----------------------------------------------
build:cross --crosstool_top=@prysm_toolchains//:multiarch_toolchain
build:cross --host_platform=@io_bazel_rules_go//go/toolchain:linux_amd64
build:cross --host_crosstool_top=@prysm_toolchains//:hostonly_toolchain
# linux_amd64 config for cross compiler toolchain, not strictly necessary since host/exec env is amd64
build:linux_amd64 --platforms=@io_bazel_rules_go//go/toolchain:linux_amd64_cgo
# osx_amd64 config for cross compiler toolchain
build:osx_amd64 --config=cross
build:osx_amd64 --platforms=@io_bazel_rules_go//go/toolchain:darwin_amd64_cgo
build:osx_amd64 --compiler=osxcross
# osx_arm64 config for cross compiler toolchain
build:osx_arm64 --config=cross
build:osx_arm64 --platforms=@io_bazel_rules_go//go/toolchain:darwin_arm64_cgo
build:osx_arm64 --compiler=osxcross
build:osx_arm64 --cpu=aarch64
# windows
build:windows_amd64 --config=cross
build:windows_amd64 --platforms=@io_bazel_rules_go//go/toolchain:windows_amd64_cgo
build:windows_amd64 --compiler=mingw-w64
# linux_arm64 conifg for cross compiler toolchain
build:linux_arm64 --config=cross
build:linux_arm64 --platforms=@io_bazel_rules_go//go/toolchain:linux_arm64_cgo
build:linux_arm64 --copt=-funsafe-math-optimizations
build:linux_arm64 --copt=-ftree-vectorize
build:linux_arm64 --copt=-fomit-frame-pointer
build:linux_arm64 --cpu=aarch64
build:linux_arm64 --compiler=clang
build:linux_arm64 --copt=-march=armv8-a
# toolchain build debug configs
#------------------------------
build:debug --sandbox_debug
build:debug --toolchain_resolution_debug
build:debug --verbose_failures
build:debug -s
# windows debug
build:windows_amd64_debug --config=windows_amd64
build:windows_amd64_debug --config=debug
# osx_amd64 debug config
build:osx_amd64_debug --config=debug
build:osx_amd64_debug --config=osx_amd64
# osx_arm64 debug config
build:osx_arm64_debug --config=debug
build:osx_arm64_debug --config=osx_arm64
# linux_arm64_debug
build:linux_arm64_debug --config=linux_arm64
build:linux_arm64_debug --config=debug
# linux_amd64_debug
build:linux_amd64_debug --config=linux_amd64
build:linux_amd64_debug --config=debug
# Docker Sandbox Configs
#-----------------------
# Note all docker sandbox configs must run from a linux x86_64 host
# build:docker-sandbox --experimental_docker_image=gcr.io/prysmaticlabs/rbe-worker:latest
build:docker-sandbox --spawn_strategy=docker --strategy=Javac=docker --genrule_strategy=docker
build:docker-sandbox --define=EXECUTOR=remote
build:docker-sandbox --experimental_docker_verbose
build:docker-sandbox --experimental_enable_docker_sandbox
build:docker-sandbox --crosstool_top=@rbe_ubuntu_clang//cc:toolchain
build:docker-sandbox --host_javabase=@rbe_ubuntu_clang//java:jdk
build:docker-sandbox --javabase=@rbe_ubuntu_clang//java:jdk
build:docker-sandbox --host_java_toolchain=@bazel_tools//tools/jdk:toolchain_hostjdk8
build:docker-sandbox --java_toolchain=@bazel_tools//tools/jdk:toolchain_hostjdk8
build:docker-sandbox --extra_execution_platforms=@rbe_ubuntu_clang//config:platform
build:docker-sandbox --host_platform=@rbe_ubuntu_clang//config:platform
build:docker-sandbox --platforms=@rbe_ubuntu_clang//config:platform
build:docker-sandbox --extra_toolchains=@prysm_toolchains//:cc-toolchain-multiarch
# windows_amd64 docker sandbox build config
build:windows_amd64_docker --config=docker-sandbox --config=windows_amd64
build:windows_amd64_docker_debug --config=windows_amd64_docker --config=debug
# osx_amd64 docker sandbox build config
build:osx_amd64_docker --config=docker-sandbox --config=osx_amd64
build:osx_amd64_docker_debug --config=osx_amd64_docker --config=debug
# osx_arm64 docker sandbox build config
build:osx_arm64_docker --config=docker-sandbox --config=osx_arm64
build:osx_arm64_docker_debug --config=osx_arm64_docker --config=debug
# linux_arm64 docker sandbox build config
build:linux_arm64_docker --config=docker-sandbox --config=linux_arm64
build:linux_arm64_docker_debug --config=linux_arm64_docker --config=debug
# linux_amd64 docker sandbox build config
build:linux_amd64_docker --config=docker-sandbox --config=linux_amd64
build:linux_amd64_docker_debug --config=linux_amd64_docker --config=debug
# Remote Build Execution
#-----------------------
# Originally from https://github.com/bazelbuild/bazel-toolchains/blob/master/bazelrc/bazel-2.0.0.bazelrc
#
# Depending on how many machines are in the remote execution instance, setting
# this higher can make builds faster by allowing more jobs to run in parallel.
# Setting it too high can result in jobs that timeout, however, while waiting
# for a remote machine to execute them.
build:remote --jobs=50
# Set several flags related to specifying the platform, toolchain and java
# properties.
# These flags should only be used as is for the rbe-ubuntu16-04 container
# and need to be adapted to work with other toolchain containers.
build:remote --host_javabase=@rbe_ubuntu_clang//java:jdk
build:remote --javabase=@rbe_ubuntu_clang//java:jdk
build:remote --host_java_toolchain=@bazel_tools//tools/jdk:toolchain_hostjdk8
build:remote --java_toolchain=@bazel_tools//tools/jdk:toolchain_hostjdk8
build:remote --crosstool_top=@rbe_ubuntu_clang//cc:toolchain
build:remote --action_env=BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1
# Platform flags:
# The toolchain container used for execution is defined in the target indicated
# by "extra_execution_platforms", "host_platform" and "platforms".
# More about platforms: https://docs.bazel.build/versions/master/platforms.html
build:remote --extra_toolchains=@rbe_ubuntu_clang//config:cc-toolchain
build:remote --extra_execution_platforms=@rbe_ubuntu_clang//config:platform
build:remote --host_platform=@rbe_ubuntu_clang//config:platform
build:remote --platforms=@rbe_ubuntu_clang//config:platform
# Starting with Bazel 0.27.0 strategies do not need to be explicitly
# defined. See https://github.com/bazelbuild/bazel/issues/7480
build:remote --define=EXECUTOR=remote
# Enable remote execution so actions are performed on the remote systems.
# build:remote --remote_executor=grpcs://remotebuildexecution.googleapis.com
# Enforce stricter environment rules, which eliminates some non-hermetic
# behavior and therefore improves both the remote cache hit rate and the
# correctness and repeatability of the build.
build:remote --incompatible_strict_action_env=true
# Set a higher timeout value, just in case.
build:remote --remote_timeout=3600
# Enable authentication. This will pick up application default credentials by
# default. You can use --google_credentials=some_file.json to use a service
# account credential instead.
# build:remote --google_default_credentials=true
# Enable build without the bytes
# See: https://github.com/bazelbuild/bazel/issues/6862
build:remote --experimental_remote_download_outputs=toplevel --experimental_inmemory_jdeps_files --experimental_inmemory_dotd_files
build:remote --remote_local_fallback
# Ignore GoStdLib with remote caching
build --modify_execution_info='GoStdlib.*=+no-remote-cache'
# Set bazel gotag
build --define gotags=bazel

1
.bazelversion Normal file
View File

@@ -0,0 +1 @@
5.3.0

View File

@@ -2,46 +2,45 @@
# across machines, developers, and workspaces.
#
# This config is loaded from https://github.com/bazelbuild/bazel-toolchains/blob/master/bazelrc/latest.bazelrc
build:remote-cache --remote_cache=grpcs://remotebuildexecution.googleapis.com
build:remote-cache --remote_timeout=3600
build:remote-cache --auth_enabled=true
build:remote-cache --spawn_strategy=standalone
build:remote-cache --strategy=Javac=standalone
build:remote-cache --strategy=Closure=standalone
build:remote-cache --strategy=Genrule=standalone
# Build results backend.
build:remote-cache --bes_results_url="https://source.cloud.google.com/results/invocations/"
build:remote-cache --bes_backend=buildeventservice.googleapis.com
build:remote-cache --bes_timeout=60s
build:remote-cache --project_id=prysmaticlabs
#build:remote-cache --remote_timeout=3600
#build:remote-cache --spawn_strategy=standalone
#build:remote-cache --strategy=Javac=standalone
#build:remote-cache --strategy=Closure=standalone
#build:remote-cache --strategy=Genrule=standalone
# Prysm specific remote-cache properties.
build:remote-cache --disk_cache=
build:remote-cache --host_platform_remote_properties_override='properties:{name:\"cache-silo-key\" value:\"prysm\"}'
build:remote-cache --remote_instance_name=projects/prysmaticlabs/instances/default_instance
#build:remote-cache --disk_cache=
build:remote-cache --remote_download_toplevel
build:remote-cache --remote_cache=grpc://bazel-remote-cache:9092
build:remote-cache --experimental_remote_downloader=grpc://bazel-remote-cache:9092
build:remote-cache --remote_local_fallback
build:remote-cache --experimental_remote_cache_async
build:remote-cache --experimental_remote_merkle_tree_cache
build:remote-cache --experimental_action_cache_store_output_metadata
build:remote-cache --experimental_remote_cache_compression
# Enforce stricter environment rules, which eliminates some non-hermetic
# behavior and therefore improves both the remote cache hit rate and the
# correctness and repeatability of the build.
build:remote-cache --incompatible_strict_action_env=true
build:remote-cache --experimental_remote_download_outputs=minimal
build:remote-cache --experimental_inmemory_jdeps_files
build:remote-cache --experimental_inmemory_dotd_files
build --experimental_use_hermetic_linux_sandbox
# Import workspace options.
import %workspace%/.bazelrc
startup --host_jvm_args=-Xmx1000m --host_jvm_args=-Xms1000m
startup --host_jvm_args=-Xmx4g --host_jvm_args=-Xms2g
build --experimental_strict_action_env
build --disk_cache=/tmp/bazelbuilds
build --experimental_multi_threaded_digest
build --sandbox_tmpfs_path=/tmp
build --verbose_failures
build --announce_rc
build --show_progress_rate_limit=5
build --curses=yes --color=yes
build --curses=no --color=no
build --keep_going
build --test_output=errors
build --flaky_test_attempts=5
build --jobs=50
build --stamp
test --local_test_jobs=2
# Disabled race detection due to unstable test results under constrained environment build kite
# build --features=race
# Better caching
build:nostamp --nostamp
build:nostamp --workspace_status_command=./hack/workspace_status_ci.sh

View File

@@ -30,4 +30,4 @@ comment:
ignore:
- "**/*.pb.go"
- "**/*_mock.go"
- "**/testing/**/*"

26
.deepsource.toml Normal file
View File

@@ -0,0 +1,26 @@
version = 1
exclude_patterns = [
"tools/analyzers/**",
"validator/keymanager/remote/keymanager_test.go",
"validator/web/site_data.go"
]
[[analyzers]]
name = "go"
enabled = true
[analyzers.meta]
import_paths = ["github.com/prysmaticlabs/prysm/v3"]
[[analyzers]]
name = "test-coverage"
enabled = true
[[analyzers]]
name = "shell"
enabled = true
[[analyzers]]
name = "secrets"
enabled = true

8
.github/CODEOWNERS vendored Normal file
View File

@@ -0,0 +1,8 @@
# Automatically require code review from core-team.
* @prysmaticlabs/core-team
# Starlark code owners
*.bzl @prestonvanloon
# Anyone on prylabs team can approve dependency updates.
deps.bzl @prysmaticlabs/core-team

59
.github/ISSUE_TEMPLATE/bug_report.md vendored Normal file
View File

@@ -0,0 +1,59 @@
---
name: "\U0001F41EBug report"
about: Report a bug or problem with running Prysm
---
<!--💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎
Hellooo! 😄
To help us tend to your issue faster, please search our currently open issues before submitting a new one.
Existing issues often contain information about workarounds, resolution, or progress updates.
💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎-->
# 🐞 Bug Report
### Description
<!-- ✍️--> A clear and concise description of the problem...
### Has this worked before in a previous version?
<!-- Did this behavior use to work in the previous version? -->
<!-- ✍️--> Yes, the previous version in which this bug was not present was: ....
## 🔬 Minimal Reproduction
<!--
Please let us know how we can reproduce this issue. Include the exact method you used to run Prysm along with any flags used in your beacon chain and/or validator. Make sure you don't upload any confidential files or private keys.
-->
## 🔥 Error
<pre><code>
<!-- If the issue is accompanied by an error, please share the error logs with us below. If you have a lot of logs, place make a paste bin with your logs and share the link with us here: -->
<!-- ✍️-->
</code></pre>
## 🌍 Your Environment
**Operating System:**
<pre>
<code>
</code>
</pre>
**What version of Prysm are you running? (Which release)**
<pre>
<code>
</code>
</pre>
**Anything else relevant (validator index / public key)?**

21
.github/ISSUE_TEMPLATE/discussion.md vendored Normal file
View File

@@ -0,0 +1,21 @@
---
name: "\U0001F48EGeneral issue / discussion"
about: Any other type of general issue or discussion
---
<!--💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎
Hellooo! 😄
To help us tend to your issue faster, please search our currently open issues before submitting a new one.
Existing issues often contain information about workarounds, resolution, or progress updates.
💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎-->
# 💎 Issue
### Background
<!-- ✍️--> Context and background information on the discussion...
### Description

38
.github/ISSUE_TEMPLATE/feature_flag.md vendored Normal file
View File

@@ -0,0 +1,38 @@
---
name: "\U0001F984Feature Flag Tracking"
about: Track a new feature, in development, in Prysm. This issue template should only be used by developers or contributors!
labels: Tracking
---
<!--💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎
Hellooo! 😄
Thanks for taking the time to file a tracking issue for your new feature. These issues really help
us track progress of features as they work their way through development. Be sure to review
shared/featureconfig/README.md for the latest documentation around feature flags.
💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎-->
# 🦄 Feature Tracking
**Current status** (opt-in or opt-out):
**Opt-in feature flag name**:
**Opt-out feature flag name**:
### Feature Description
<!-- ✍️--> A clear and concise description of the new feature. Provide links to your technical
design document or other supporting information.
### Tasks
- [ ] Feature flag created as opt-in
- [ ] Feature implemented / code complete
- [ ] Feature tested in production for adequate amount of time
- [ ] Feature flag is inverted to be opt-out and the opt-in flag is deprecated
- [ ] Feature has made it to a tagged production release as an opt-out flag
- [ ] Opt-out feature flag is deprecated and old code paths are cleaned up
This issue should be closed after all of the above tasks are complete.

View File

@@ -0,0 +1,27 @@
---
name: "\U0001F680Feature request"
about: Suggest a feature for Prysm
---
<!--💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎
Hellooo! 😄
To help us tend to your issue faster, please search our currently open issues before submitting a new one.
Existing issues often contain information about workarounds, resolution, or progress updates.
💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎-->
# 🚀 Feature Request
### Description
<!-- ✍️ A clear and concise description of the problem or missing capability... -->
### Describe the solution you'd like
<!-- ✍️ If you have a solution in mind, please describe it. -->
### Describe alternatives you've considered
<!-- ✍️ Have you considered any alternative solutions or workarounds? -->

30
.github/PULL_REQUEST_TEMPLATE.md vendored Normal file
View File

@@ -0,0 +1,30 @@
<!-- Thanks for sending a PR! Before submitting:
1. If this is your first PR, check out our contribution guide here https://docs.prylabs.network/docs/contribute/contribution-guidelines
You will then need to sign our Contributor License Agreement (CLA), which will show up as a comment from a bot in this pull request after you open it. We cannot review code without a signed CLA.
2. Please file an associated tracking issue if this pull request is non-trivial and requires context for our team to understand. All features and most bug fixes should have
an associated issue with a design discussed and decided upon. Small bug
fixes and documentation improvements don't need issues.
3. New features and bug fixes must have tests. Documentation may need to
be updated. If you're unsure what to update, send the PR, and we'll discuss
in review.
4. Note that PRs updating dependencies and new Go versions are not accepted.
Please file an issue instead.
-->
**What type of PR is this?**
> Uncomment one line below and remove others.
>
> Bug fix
> Feature
> Documentation
> Other
**What does this PR do? Why is it needed?**
**Which issues(s) does this PR fix?**
Fixes #
**Other notes for review**

5
.github/actions/gomodtidy/Dockerfile vendored Normal file
View File

@@ -0,0 +1,5 @@
FROM golang:alpine
COPY entrypoint.sh /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]

5
.github/actions/gomodtidy/action.yml vendored Normal file
View File

@@ -0,0 +1,5 @@
name: 'Go mod tidy checker'
description: 'Checks that `go mod tidy` has been applied.'
runs:
using: 'docker'
image: 'Dockerfile'

34
.github/actions/gomodtidy/entrypoint.sh vendored Executable file
View File

@@ -0,0 +1,34 @@
#!/bin/sh -l
set -e
export PATH="$PATH:/usr/local/go/bin"
cd "$GITHUB_WORKSPACE"
cp go.mod go.mod.orig
cp go.sum go.sum.orig
go mod tidy -compat=1.17
echo "Checking go.mod and go.sum:"
checks=0
if [ "$(diff -s go.mod.orig go.mod | grep -c 'Files go.mod.orig and go.mod are identical')" = 1 ]; then
echo "- go.mod is up to date."
checks=$((checks + 1))
else
echo "- go.mod is NOT up to date."
fi
if [ "$(diff -s go.sum.orig go.sum | grep -c 'Files go.sum.orig and go.sum are identical')" = 1 ]; then
echo "- go.sum is up to date."
checks=$((checks + 1))
else
echo "- go.sum is NOT up to date."
fi
if [ $checks -eq 2 ]; then
exit 0
fi
# Notify of any issues.
echo "Run 'go mod tidy' to update."
exit 1

13
.github/no-response.yml vendored Normal file
View File

@@ -0,0 +1,13 @@
# Configuration for probot-no-response - https://github.com/probot/no-response
# Number of days of inactivity before an Issue is closed for lack of response
daysUntilClose: 14
# Label requiring a response
responseRequiredLabel: Need-Info
# Comment to post when closing an Issue for lack of response. Set to `false` to disable
closeComment: >
This issue has been automatically closed because there has been no response
to our request for more information from the original author. With only the
information that is currently in the issue, we don't have enough information
to take action. Please reach out if you have or find the answers we need so
that we can investigate further.

89
.github/workflows/go.yml vendored Normal file
View File

@@ -0,0 +1,89 @@
name: Go
on:
push:
branches: [ master ]
pull_request:
branches: [ '*' ]
jobs:
formatting:
name: Formatting
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Go mod tidy checker
id: gomodtidy
uses: ./.github/actions/gomodtidy
gosec:
name: Gosec scan
runs-on: ubuntu-latest
env:
GO111MODULE: on
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Set up Go 1.19
uses: actions/setup-go@v3
with:
go-version: 1.19
- name: Run Gosec Security Scanner
run: | # https://github.com/securego/gosec/issues/469
export PATH=$PATH:$(go env GOPATH)/bin
go install github.com/securego/gosec/v2/cmd/gosec@v2.12.0
gosec -exclude=G307 -exclude-dir=crypto/bls/herumi ./...
lint:
name: Lint
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Set up Go 1.19
uses: actions/setup-go@v3
with:
go-version: 1.19
id: go
- name: Golangci-lint
uses: golangci/golangci-lint-action@v3
with:
version: v1.50.1
args: --config=.golangci.yml --out-${NO_FUTURE}format colored-line-number
build:
name: Build
runs-on: ubuntu-latest
steps:
- name: Set up Go 1.x
uses: actions/setup-go@v2
with:
go-version: 1.19
id: go
- name: Check out code into the Go module directory
uses: actions/checkout@v2
- name: Get dependencies
run: |
go get -v -t -d ./...
- name: Build
# Use blst tag to allow go and bazel builds for blst.
run: go build -v ./...
env:
CGO_CFLAGS: "-O -D__BLST_PORTABLE__"
# fuzz leverage go tag based stubs at compile time.
# Building and testing with these tags should be checked and enforced at pre-submit.
- name: Test for fuzzing
run: go test -tags=fuzz,develop ./... -test.run=^Fuzz
env:
CGO_CFLAGS: "-O -D__BLST_PORTABLE__"
# Tests run via Bazel for now...
# - name: Test
# run: go test -v ./...

22
.github/workflows/horusec.yaml vendored Normal file
View File

@@ -0,0 +1,22 @@
name: Horusec Security Scan
on:
schedule:
# Runs cron at 16.00 UTC on
- cron: '0 0 * * SUN'
jobs:
Horusec_Scan:
name: horusec-Scan
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/develop'
steps:
- name: Check out code
uses: actions/checkout@v2
with: # Required when commit authors is enabled
fetch-depth: 0
- name: Running Security Scan
run: |
curl -fsSL https://raw.githubusercontent.com/ZupIT/horusec/main/deployments/scripts/install.sh | bash -s latest
horusec start -t="10000" -p="./" -e="true" -i="**/crypto/bls/herumi/**, **/**/*_test.go, **/third_party/afl/**, **/crypto/keystore/key.go"

21
.gitignore vendored
View File

@@ -4,6 +4,9 @@ bazel-*
.DS_Store
.swp
# Ignore VI/Vim swapfiles
.*.sw?
# IntelliJ
.idea
.ijwb
@@ -14,6 +17,7 @@ bazel-*
# Coverage outputs
coverage.txt
profile.out
profile.grind
# Nodejs
node_modules
@@ -23,6 +27,17 @@ yarn-error.log
# Ignore password file
password.txt
# go dependancy
/go.mod
/go.sum
# Dist files
dist
# deepsource cli
bin
# p2p metaData
metaData
# execution API authentication
jwt.hex
# manual testing
tmp

View File

@@ -1,69 +1,28 @@
linters-settings:
govet:
check-shadowing: true
settings:
printf:
funcs:
- (github.com/golangci/golangci-lint/pkg/logutils.Log).Infof
- (github.com/golangci/golangci-lint/pkg/logutils.Log).Warnf
- (github.com/golangci/golangci-lint/pkg/logutils.Log).Errorf
- (github.com/golangci/golangci-lint/pkg/logutils.Log).Fatalf
golint:
min-confidence: 0
gocyclo:
min-complexity: 10
maligned:
suggest-new: true
dupl:
threshold: 100
goconst:
min-len: 2
min-occurrences: 2
depguard:
list-type: blacklist
packages:
# logging is allowed only by logutils.Log, logrus
# is allowed to use only in logutils package
- github.com/sirupsen/logrus
misspell:
locale: US
lll:
line-length: 140
goimports:
local-prefixes: github.com/golangci/golangci-lint
gocritic:
enabled-tags:
- performance
- style
- experimental
disabled-checks:
- wrapperFunc
run:
skip-files:
- validator/web/site_data.go
- .*_test.go
skip-dirs:
- proto
- tools/analyzers
timeout: 10m
go: '1.19'
linters:
enable:
- deadcode
- goconst
- goimports
- golint
- gosec
- misspell
- structcheck
- typecheck
- unparam
- varcheck
- gofmt
- unused
disable-all: true
enable:
- gofmt
- goimports
- unused
- errcheck
- gosimple
- gocognit
run:
skip-dirs:
- proto/
- ^contracts/
deadline: 10m
linters-settings:
gocognit:
# TODO: We should target for < 50
min-complexity: 65
# golangci.com configuration
# https://github.com/golangci/golangci/wiki/Configuration
service:
golangci-lint-version: 1.15.0 # use the fixed version to not introduce new linters unexpectedly
prepare:
- echo "here I can run custom commands, but no preparation needed for this repo"
output:
print-issued-lines: true
sort-results: true

91
.policy.yml Normal file
View File

@@ -0,0 +1,91 @@
policy:
approval:
- or:
- only test files are changed
- only proto files are changed
- touches consensus critical code
- touches sync/blockchain/p2p code
- large line count
- is critical priority
approval_rules:
- name: only test files are changed
if:
only_changed_files:
paths:
- "*_test.go"
- "*.bazel"
options:
ignore_commits_by:
users: ["bulldozer[bot]"]
requires:
count: 1
teams:
- "prysmaticlabs/core-team"
- name: only proto files are changed
if:
only_changed_files:
paths:
- "*pb.go"
- "*pb.gw.go"
- "*.bazel"
options:
ignore_commits_by:
users: ["bulldozer[bot]"]
requires:
count: 1
teams:
- "prysmaticlabs/core-team"
- name: touches consensus critical code
if:
only_changed_files:
paths:
- "beacon-chain/core/*"
- "beacon-chain/state/*"
options:
ignore_commits_by:
users: ["bulldozer[bot]"]
requires:
count: 2
teams:
- "prysmaticlabs/core-team"
- name: touches sync/blockchain/p2p code
if:
only_changed_files:
paths:
- "beacon-chain/blockchain/*"
- "beacon-chain/sync/*"
- "beacon-chain/p2p/*"
options:
ignore_commits_by:
users: ["bulldozer[bot]"]
requires:
count: 2
teams:
- "prysmaticlabs/core-team"
- name: large line count
if:
modified_lines:
total: "> 1000"
changed_files:
ignore:
- "*pb.go"
- "*pb.gw.go"
- "*.bazel"
options:
ignore_commits_by:
users: ["bulldozer[bot]"]
requires:
count: 2
teams:
- "prysmaticlabs/core-team"
- name: is critical priority
if:
has_labels:
- "Priority: Critical"
options:
ignore_commits_by:
users: ["bulldozer[bot]"]
requires:
count: 3
teams:
- "prysmaticlabs/core-team"

218
.well-known/security.pub Normal file
View File

@@ -0,0 +1,218 @@
-----BEGIN PGP PUBLIC KEY BLOCK-----
mQINBFpZUO0BEAC/tqN6QctJKSOF+A7jrBzvNvs6rVEi9Hn55u/scKNIPkSRJRTy
8kVsDMha+EKXka8oQiVM+sHoHMIMCa3tdicm1/k5f67X8dqadQmFP4DhYRYIKCKT
TkntNG568ynf/yUs/YY9Ce8H17JvgytkLK50mxMxycUlYREMaRPMR8Wt1Arrd9QT
U0I2cZemeqUORKuiVuZjj/7BVDHvSXHvzpi5zKI86sJQTnzcuGqyNsrUU4n4sYrb
I+0TfEHzmSdxoAXSaMYjomPLmbaSdBiK/CeNKF8xuFCKJRd8wOUe5FcIN3DCdk0e
yoFye5RMC+W09Ro+tK/jTQ/sTUuNLJm0VUlddQQeoGlhQdLLwiU4PJqcyeL4KaN1
l8cVml7xr1CdemhGV4wmEqAgxnNFN5mKnS2KcDaHxRz7MoGNdgVVQuxxaE0+OsdJ
zCKtA12Q/OcR24qYVFg5+O+bUNhy23mqIxx0HiQ0DsK+IDvcLLWqta0aP9wd9wxG
eObh9WkCxELTLg8xlbe0d7R3juaRjBLdD5d3UyjqGh+7zUflMsFhpUPraNXdKzvm
AqT25cveadM7q/CNzFXeCwmKjZab8Jic8VB80KcikmX6y9eGOHwjFixBogxowlBq
3KpeNTqJMNHYBzEb0V18P8huKVO0SMfg11Z1/nU4NA4lcjiVImb5xnJS0wARAQAB
tCxQcmVzdG9uIFZhbiBMb29uIDxwcmVzdG9uQHByeXNtYXRpY2xhYnMuY29tPokC
NwQTAQgAIQUCWuZ7uwIbAwULCQgHAgYVCAkKCwIEFgIDAQIeAQIXgAAKCRBy4z5N
8aUDbtchD/4hQDTj7qsccQKpaL8furw+OWuoZtX38smO9LrcSoLaZlk5NRxFNlhQ
FaL6iyDLoDkWUXEPg94x1dMFBUfMdIuh6iEF2pcvCN6WG3Pd2j2kG6OxaIx+rI7a
eaD2Wds312etYATu+7AI6pxlQVqPeZi6kZVvXo8r9qzEnl1nifo7d3L4ut6FerDz
/ptvIuhW++BEgksILkwMA44U7owTllkZ5wSbXRJer0bI/jLPLTaRETVMgWmF5L2n
PWKhBnhBXf/P00zTHVoba0peJ/RGdzlb1JZH+uCQk0wGBl0rBMUmiihIB8DZBZDX
5prwMdoHG9qAT9SuJ8ZOjPKaBHVVVw4JOU6rX2+p9E49CVATsdoPfWVbNyVRGi5f
Oo0bJPZU3uO10Q09CIeyqT6JPDCZYS7po2bpfFjQTDkoIv0uPWOsV5l3cvFxVlcD
Pvir4669xhujmoVBOrp18mn/W/rMft2TJ84inp0seKSKdwBUeEyIZwwu1YTorFe4
bgJghDu/Y+K4y0wq7rpPimoiXSoJOaaIPrOzsKAVu20zJB4eoulXlPwHexix8wwf
xeVH4bGl3wtTFWKja0+EQVdxpt+uUlABvrheQbNlNGz5ROOAtjvwASI3YW/jVVaG
wSbOUuMEHfC1rSnj5Y9fN6FbilxJAZ2UbvZE6iXqqemfRdFuB5yedbQmUHJlc3Rv
biBWYW4gTG9vbiA8cHJlc3RvbjkwQGdtYWlsLmNvbT6JAjcEEwEIACEFAlqrvbMC
GwMFCwkIBwIGFQgJCgsCBBYCAwECHgECF4AACgkQcuM+TfGlA269zg/9FynobfLE
Vh+Aid7l5C6pprHflHpdNVhdHzjvosLKfPMCcBJLyCLAIueKC0vvKbt5kyNV6Adl
6fibdO/vRrNsEdmKXTdfDnN03yVX+VO54rqyuweR09CbLHakECh5awWpk0x9E/Gc
3MlU8YDCVNPcWAw8JPZZ0y6s50/MFFXya+/opw45neH6WXrbqbIapzZ4V1bhkloF
TTz7LYp1dk6qjus/f1jHgujclJ6FazG+OqCpzJ22lnuwNYs8sv1DclW+lEUfIin5
PvzcSRCCd6NilFRSGzfBhM7wxZrAp0JzWXpM1jmd2WHtgErusTaTXRTATjPf9Chg
SE9UT3EJvJ5fPxuxm+qOAowpJwe8Irv+YuMzL8C6C2BhZIV8XID3/cErNeQbPocj
QFksmBEwpe9EG3Yhd5SmSXRhCtBFnyFKyOqPpAE2s+U0b7qsnWc50UFbIytPEm4C
YmdNL6IVilp/+LYFcRWWK/ppOtvtgbj8r7+Foidi/tCauJGt2BzhWH7DkLEdXGxk
PfR/rTgyuAQZowl03DaNwAPKegY2SIuROO9kpQACwTWZNg4l2yrZv09qrKBhohyn
b2i4pwHPZ5/bWLikdzENcJuvbH5qhf6tV0FIaNGbTkWX00uI++bvWAxxE25jZv56
e7VTaWuMt8Ly4DBcwl5JyWuvZ74+yv7QGp20WlByZXN0b24gVmFuIExvb24gKDB4
ZjcxRTlDNzY2Q2RmMTY5ZURGYkUyNzQ5NDkwOTQzQzFEQzZiOEE1NSkgPHByZXN0
b25AbWFjaGluZXBvd2VyZWQuY29tPokCNwQTAQgAIQUCWllQ7QIbAwULCQgHAgYV
CAkKCwIEFgIDAQIeAQIXgAAKCRBy4z5N8aUDbmvdD/4jkT1IXJyj65sgHus0HHYC
MBp6mzvtpDcPljg/5Nl1WXydv4zYGnEnIwWIqDYwwvokOKllxAjgevcplHqZa0cb
7XKCFk5h+56FLPHv9e0WK1fOFeo2ad3imNRstSHgaumGAWHJMg9vWbDisGv1zjQ0
usFkrMKoX34YzMt8KBD7IuQyeBkYNupT8EfByeA9Ta+wkvS+X6BojsFB1UWDAFCY
z8RgTcWIFjWtGZwIkWUKCzebqXbElpJF8ckZU9q4qVhyZ2DT+8BS/Lm0Sf4z6VTC
4EN10ZuN+F+J3DMR2Zuudp4X5OUGkDG4KuU/kvj6EJRWpbTS1D9ReK7hoApIbV72
Um6Mf7kC9EKvxgL1gOfl4aj3Io9M0R8FT/0WSMduQzf/jI3XqdNH0nYo2kTDdZm8
S972cyvt6frKYy6RsOlPz1+S61iCmupsRY+HyKDTa0vnpXg2c4dE1neF5eMI6SVw
viJvCG2cflctJ2PLiINZJYAg87gV5Rm1i/Lt2YG5zdxAXXJt2R0uuU9fv4DcHZLU
yx69Yuh+5UiEaXYU7xoRCdZJYyHbvaC2hPcDGnEa3K1QbgnI5hGAtG3cwdbYC5e3
bcaJb/LdwzkdRnHLgpxa/UTAIfejEI1U2kvVvNoe/HvEXq/OPrhFDvE4rW8AzbX+
ISTWWRY0lLSr8/SD0TDJMbkBDQRam2geAQgA0kESOibOO3CcXrHtqP9lPGmj6rVe
G18fRmPDJiWtx863QqJeByuuUKwrGkPW/sqtIa5Ano+iXVHpk7m955nRjBmz4gd8
xqSGZd9XpNObYPA5iirAO8ztpgQvuvsHH9y9Ge50NnR7hQSMUbGVeCUU/vljwT60
/U+UPnsTAmkhvkEI72x50l5Ih9ihcBcr5jJpi08XktE3sFOoannac0kZQJ6WXFrY
4ILbv8fVqcRf44xMKOlFB9qHhleGW0H9ZUjTKs9quRt7r5D1MOiwrZDjNN3LqMco
oWj37hb+3KkmIIsAYB2DjnWYkMPu2B0O4uSOHYAWfEJtRvA8qW7sWj+q1wARAQAB
iQIlBBgBCAAPBQJam2geAhsgBQkB4TOAAAoJEHLjPk3xpQNulz0P/2m9veAAGGCO
yPsqiNVVa8lrhmLGh/W+BaoszQ/r+pfin4xTOV5K5h3MC5CVJM0JxP/cxNol+Nmr
cYGbZgq4QhLlH6PdQ7cReL5ED6Wi+eHb4ocvXJqUuZ2Gl8Z7gzQzp+WFbLrn8vXj
LcyGGEETV84jy+X5cKu24eAjTFK+unfqwfxXFZP5vhIEVe7+uG6A/pMB5eLDqEUh
FQUcWqCF2Imt08Kcn7HL31GoIY0ABHdD+ICXZE5lTm6iJGzpFBKdTMm/e5igCJ3v
/tgtWZbnvyseLR/T/gU1JyheS9kNXP5sICwicBbY/vnGdEP6OKpDOSfQam5x4BBj
cvsBnsNuonY7OJn4iLY6LviQ0MM91PbaJUUJrp9Uyi4hj9iO/MZBaG0Giu0FKjG6
Vk+RlCYPjYIvHflQLZHe9BWLPN1AvaLKszt3IYaxS5seXCu0ZqHDGCBVqVCNDDJk
MJbHlrOVZ9T6mZhA+aQ1EJvTWk3MNj1AOyCJdiXtOOdg+4Fm5dN7nLpumLIg2yP2
afI7YfrPGA7sm+T0IMnOWxkK+KODC7OV9h/QjyBJDcUYGXLapuK9eP80cr8hKvH7
S3G4Top/rXDpnBNQ2azCqmUwaJWNRhLucC80Vd00d4pWIOwAPWpiV70Fq8OhQFhT
PNXwFXVLwtNfPvPxN1s+Vk+BBXp+M19AuQENBFqbaC8BCADSq89Z9xWRS2fvWI/N
+yEWliIU8XiqC9Ch+/6mS2LEyzB1pPLIIQcRvM6rq2dxXIRveVGpb63ck9vUtuJG
//es+DnDkO7re+ZmWHX+LAqMYNdaobSYxHkkR4CcY2HbPSEUbb//Zwk4BDyp3g3W
bKK9noVbslZuSwWNrxjX/Hieh/dIGYkNFeWOlmNfUYsevzqNDjsziOagyXKxLc++
hUM3GKgzXRQJBvBpgzQc4bRY+YGHXtZurk9AiZ4ZBhWv2Qrb5OYYislE9sdv3KWV
Iv1EBpbkAJ9MM1HgS8bkIOIpNs4XxHY6fTWWdrXi+NgZXQwQRYaWTQsXL3mktarS
fFfTABEBAAGJAiUEGAEIAA8FAlqbaC8CGwwFCQHhM4AACgkQcuM+TfGlA24vqg/8
CsVBHO4mh8SSaxdWNEU+mG4pU230BRCwrfn42wuSKb7WNLTTcWMDNI0KY/JNcWSq
G2qa6lngiAyOS72Jd635ptZ6Wvjd0WtBt90NN2jtn+aRqwQ8uItlYMQscofFzskj
QnBF+NWER+44nxboghuQ041m6aI2XmYUErSOBZi6onbC3svH6coMldkkiueWFbdB
qac3UXue4LUcaGR5T9pCQsLgTl3D8V5icEM+HpTVVGQZkVrOuKMKe6T9N5FS/WFu
T6CuFf/OyU15a6RE4WW9QqKYsaHl8B6+0P7uqPoVIxs8hfJcwaUu9XwIiZYBZg7N
yYCQ7JBapC5KZlIcTCfFSxby8lCJoZbIM3Pk/pgCuKxGaS9rHHUGuIvz8TfnM9FO
2zlxl4E6k3NFgGKF3p6oiKayC74o6EOw50p6DMjrisG7kkWVsTRCMINF7CcfeVme
MI9ljGAMB1hPtHPhl27hMRfq+/iIbR9gb7+Xw2yKQL2QRjMDIGGxP3q4NjD4tV8P
VyTbAAwNARG8oMpNM628v3tsW+WYNot1SPUQuZbIx1pCwMeTqljWsdQxWRVW5UxM
dnXTCqZhQwH0ICG/jbepbP6ciVB/CSa7TVohEK6jiTMorhqynRMMJ6p48Z6HIlyY
0Ss8q9K29eXaqmLB8Oy3HmupqxH95TqMntqivzf6i6e5AQ0EWptoPQEIAL1OdDIl
7E3VKAEWH5GnMzdnl9bju/ovoMjLqGevHS9Gyz4OPyZvCQ2pS8k0sgqwsn4F8vWM
7L3xKTyQTSYOPby3do58+kxUrdTNlqGKEEcZDG+MSzxKyft7m+37fzbg6tcU+O3L
/7m8nYWQSRKJeist7Q8HrQJzehuzcgAnNmpeBqXHnAwRBvtqORvz5cQAIQ4EsEvP
f/unTjw95JtL1LtBOaOaynF9ap/TZ34OvDdARmZSdqPpRtEvjfgIboIYYt1dNmDH
kiSaaKaqBLCJTD2z5KT8ccDeF8lzYHAYzNy8v2lTc9vagCZH+lf3d2d6umNcr4y1
NGEN4ZEhrmt/lP0AEQEAAYkDRAQYAQgADwUCWptoPQIbAgUJAeEzgAEpCRBy4z5N
8aUDbsBdIAQZAQgABgUCWptoPQAKCRD619WmVzqkJDTvB/49MQNQk8YJTwAnbdSH
7stU2uQFBbkljE8Juz5WJK53lL6xUVUp/3gKrQio1F+z9uRVqRh0cQnqX6mPMe5a
2dlHEIDHTJjSlR5GCCBRDbssV6SN72xSVgbxVGZ9L32qUYtiwGnxwXQC9D9KsonD
YfGfUhD1CLAldr6HwhJkOq4QKz5GF4ty8sMKEcpM5UaR2sa2y6Ebn9Vzma10YaVp
X7RlZM/iTiDhTmWuxLh7e21DI95k/eFqHpKA912N0n1BdzZPbwk83nVRxXg793NU
RFpegzlLawtaCW9oVJOYqErMGOYN5nbXAHXsr5X7Or70v1Yo1khgzNOfnirO57T9
VjT0J1QP/j1xobgMuNda7Fpwc0xo2oIKEf+SWY9GQrkUK7wCLDbpgbGVxTmREkyE
6oyDzW1QpQXRjLS9Wvtun0J3Wn6r6Aar0VaGKa7uiiq8UORWtFkWQAzzyBj87Rjx
eWZWV1dzLK7eMJdyN0gsOzcejrsOqf1sydzvhm4K66byjDZ78piv0DdyPIb4OsiQ
QU2GH+QwGRYIkYlU9f9g+hSasAfzvrATHlJZNrOjOCgXbut/yP9ug3DHKj76wmoU
n/Y4rpmskAzIQrZIpimkpNBmZVTGr4bkWcokVzrRFor3NCpl1qA1K9Cd43wARH8t
Zwk4evI4abbqUId0vVNCKSSDyCCjgNwRsmU8RXdn7r0vs4ObKuWfY9Yl2y8tq3qO
YPHr0r50YXWtKqUNy5JUc3Ow9DFR1p4O4yfmiSyTLUyOYbglfvtnO32OJkrrZ8Kq
iSDnyiq9u2nYJAEHk7AchF5TJrTCnd8yWNIjohild+wc+rMKktspoEcxmT6zaK4T
AymmEe3jzQhlxptpBm68t2E5kaYFQh+NnizXlRcmFjBgEhH2CxCoerUpK82Y+MuE
S/ODmF9kULre14aXAAspj4vldQ/LJm0SYEfTkE1sDipJQUzv6oS5AFloQDXZPWTB
15P+Xsj8sJV9hWCfJZVmppWuPg/pCYXwdjUHUYouTz3ZL6qnUbm2uQINBFpZUO0B
EADZkfsbXPpDcMALUgVmB3cJU90tFqc8Q5guK9oSs3ibx4SmyhBVmeF2TF6PQNoK
YvpUR50hAcx2AbwtY51u0XxrAr8kFiL6R5ce/0pVMfXGchcC58CJ3uf+O+OPt080
ftyVSTsY9xEnBohMoJescn0L/IPaM6KkkIFIMAI4Ct3QCHox7WHgPLrqB7Efx2Dj
Qkgjbioqj8zVKDwxTs0S3sknch675gOhsCkaI7KMcRGACDinKRF3wQha4brNa8En
vPTV01Cv0ttCo6NCcbQzQi8QQYpMQcWVkquEJWweZQvL/OvYWdT13JgnIp66pkmo
+JvGTOqQpSnIx6OQnV9yqwqsg4E0dkCE+9rDYAHpwvmRkaI2sItjN4KAEQdTWES8
RgGVgfCtvNH87PqpaPIgarMDY/j5KqTDp/7Nc5oGLCmhwZiYzQa7Bm5uVNnYIyOO
d1IjfclgVdVAzMOmrFytvXFCBcga1khL15taC7s6Vuld89TgMZdSot2RVz8W8Xc+
39ZrBvzrCeYsPq5/U0Z85cnOSw4skwh06wsxTvL1D70SilI2c0YdR1sVgbhq04HN
7FyE7SDQ1GqxyTJAU+OPH3Pk97Bl25vWD43RCCIjSUHhKzQRPno2ItObFepE+QJH
lSO1YMXmZDAfsRts3dca3VSDOdAQely6G7HQu5kXWXGtRQARAQABiQIfBBgBCAAJ
BQJaWVDtAhsMAAoJEHLjPk3xpQNuRlsQAKnkyjjXRvfMB3eKZ1PrWf7DBx5WL8Hk
r2QAnv0diDeRTzotQyzKivf3/W3gQc9kQi/ilaPuuXeW+yKiNQaIbai7EC0DdyHa
qG9s8F7UDoojFOGCc3OVpQvuucVbv4a6EpqXBI6ayrQW0jMXakiuIF/dL5uCGRLn
sPMB87xJfMrkqEziymJMooQTRckgj2S9J2QYDZFxmKChPl1kXMlmx6Y4FhzrsYwo
I47hW9hHG1+8129FOgwYTwELEuX6FKShcKpyy77b4Tar3UvdzNkfaysFPvX3O7Oh
Bes2VgfslEZSgK2CScigvLIz9Ehe9CUw6WEC6LZW3bbC+Cbz624gOsm/GYI9Llsc
5/NMMwQTCoTRm+b0UAYQvzHDS7km9yceNSfFlkpE/lWnIt9E0H+8bOwEbYF8y2qy
yLXIm7MYEm4UnUZ0j8kihP+wSHsZP2xPhjEUcQw1ZWhWLACvzlEC0a3zsovoJ6U8
zrqqfEBtuQgfwwJRWArMLQn/rlEJSrTrFfehwhvH3dPVSU36N5nBA8ls5YlSHBQv
38dChpBXu3wzFhetkSuHxdlfopeZMtDmljnIkBNTEFg01oqJNPbyiX2JQcfKizCt
maCIiJY+hOYIKkJdkt1FyOhADBciebxCQrpIIzWupeyQNuVj3I4g6YaQX00+kgiB
H1XKrAg/dpMNmQENBFrHiEoBCADASg9zHYzaSU0/1q1hcmTHU6H4syCQXHHd3zF7
n/RcsGnt4RBuUi/BUvNp3zYR6uFOyyk6LPV1hq2Ca8e/oSFrDYxqLladESQd9GNN
stDeK3HinsWJCVwSbkzNJbUtyr6SclmWt66vNqBZngMallJRQe8QDqpg0ZSZj/0d
VGxPBR16zc/2ddGnXJFe/V5XAWAap9SEo44pyGK4xf87Bgq8jT33LuQtd8exOk3E
atkK5jLEn9xmiheoSePEhOoQSrJMHfMjFka0PYZlCeaaHw7r7yXb/VoHFOAPxb6k
a1cunbp39b4z7Jy9kLBy0tbDnAs/sLp4LUN3Vx1JLoXBSIsHABEBAAG0JFJhdWwg
Sm9yZGFuIDxyYXVsQHByeXNtYXRpY2xhYnMuY29tPokBNwQTAQoAIQUCWseISgIb
AwULCQgHAwUVCgkICwUWAgMBAAIeAQIXgAAKCRCVRSpwGBD+26lmB/wJxChWwva0
StjMRSk3V7JTBPi5RNQm3VY8UydUNjsXzHVvtjUczOj+zHfo8tYUYa/ypWujXEX9
bVYVMr9JGNjm+rysI1gmW1gcx9pZr9gde4CR4Owfpwjh8oFnSrBBrcaelb0Krrv4
Okms43fEw1bWpllayal5SqknOIxpw1ZiNpG3jVe/C9n1/3Nw4XF5R0RHDxXTu6Hu
H3t7zRQwAJcOod6ccRdzNR9CsGdnOoNPG3pqs9w6zWUp0QETMAMcEpSLCJumaVfQ
24PK+MF92F5JRwNVt80xSLA9KMyNnv/ZvxZXXLjkIhsAmwYkHUs3WSm5HJ4qqOCX
V8VwFbwnea3eiQIzBBABCgAdFiEECuAFHWR7o8GpF69AcuM+TfGlA24FAl9zs2AA
CgkQcuM+TfGlA24dMA/+N31SPAXVgwiNasBknb+YSq2/OQPogxQUc8tupvDcQ62H
u0kvA7pUPUFjITJ1xsm8CRXfb7Hge9rNUw9Y2MNKf4sIDQs3bFeKOiAGVbO8Z055
5VCTWWPhXzjWoR84YWrx0Go8WT/V3Lahy4frGA3Vsza6wzi2P6c7LF4jqX2iBD1l
OpAYNGyt0sX/RLp3s2jOTWJwVyRR4UKSZOgpi9OTGLXrq5oU2dpwEIzBmhaWIs+u
oD5/4TaAt4lEFu3Sxk5w8fJlUXbM4IG8A1l+dnRPF5rtjtfvuX0GeQcDtJ5e1qHj
7PvVj8HjGPwxqsAjYHpg6QjyQCdtHYHdboEIU+OXMYRPRh2Iv0GUoeuMqoSsvFgn
c9PGN8Ai5u+oNzKeLJhpxpLV7s9zAbsripdnvDBn7RwuNx2ziqZayxoYvFRCAQl5
wzr0/eo/wq36D9EI7uJ2I3yt1g/VkwWQsAeE2skuGGbwed263cWTkl6g1w+vlDY3
/jc3a8HcS0zIUX194ChrbbNezGb74mQFoLk6PksLfhXseAKCs8bvPaTwIm0JGTMT
YzkRufrv1+I9KPFy3fTpvnMZTbud4nPCLsK179whk+Jrdv866i+E2WZ1JyzIZ9bB
P1x+ABi82PMdzhTw+pTkR1CVHmrmOiSi2MHRYGMedM9ECGnPIdab5d75r1qkuvO5
AQ0EWseISgEIAKHrgTVeZ0CWZMETk7NFjDTvBpoTMQDT7DDe2vFQHc8D2AvR3Aod
+CUaOeY0Yp0GOemd7BEcUbpYbCQk1B9G8vHNcQqTMLjBgujXwF6w2c+lqEu2/tyI
2/uGCOBDs4wuQZo+akCCMekqYjdZBfZV6nQzf/l7eQHIq+sNtnSiZmHdH46PLi7/
17wR9GxzKvuv1R73DCyekCcz1Puo0b7lfGD28kESJK1Mg9SAOqVjtaS58Oxo+Y1M
ZWRqh0tkAkgOBpdyddGy6TX/9c3a0U3eBQweRpNDh0eCEh5UsYDluL4NtXj7rVYd
4mHONJzI3h1LnKWvpVYmk703MPmtgeJwNzEAEQEAAYkBHwQYAQoACQUCWseISgIb
DAAKCRCVRSpwGBD+25PxCACNBj/2HBSpdYAxEGhNHSaw62y7Jwuf7NbsKIAqygzi
m2+dxae7PbAm64jEXYJA5GaCOs4xO52S/2+1N87e5J86VRNg0vlA9/ivFzERAxEg
rgIUyGXYfS8oA/4r5+PfKt/NvXO2wH3MPakrqZqXhOv+1IPvOt03wWoZKmYyVT6g
YnzutOsvH6cbhUh/D0WpuU8ZgkrFu5Xe7ynZoLm1qQOA23pkxxQbeNs0uoKUja9v
bx4eBtaliLc6rsXt+1WzdXA5ZRNqkdn87Tz5IU0FuQYVblYvPz9QoUlvx5siWVaP
Mmc7dIctWaWyaJmgjkLKdy36ydS5axhqn158yIjOZV3emQINBF8h6lMBEACovC4z
oieJ9iMZpWfylsLQKkeEmfSXnjcjW4RLUjs2CRWj+W6H7eCRy/MBOdx+6zAb8TE/
n/TSlP5l5Xx40BGDAMUZVFrJVEkMPK5kUmNzybg7PiuMd3qZE+pNyHEXXlLU77Dn
VO26TD9RvpKXdjm+ATsnQ6rvDNszkYI2Bj1tPxwZ7bRi96U/upL3WfYOsTLHirM3
pEkI3zfMZj8ufDX9XlmGrQ384E/hTgjpLXmDm/jMRlX3mRzDkV1HO+gEicfePm5+
K4eWlMCNXN5bcGwoEFY2LwAojRcbRaH/UH2S3btkG2eSK2fOFEwQ0G4vIyoLF60R
cEk1s6cYIgk3kVsmNSzA5iJ81nD5bbfTceUKsjTZiw5RmN0Vh5g75pw+3uoXB+55
egabEIt/GTZZlP6zhd/CKjTQR0R4+ZaEbZFOCtABukC5xfWGCRdNmXAWMBe0MYpW
ub2TRLISLfNNc7GWM4Y17d9aRhaql9gY6QLIRNGYuvGPiRMaJADZx46LtbWo8YiG
xLId7H8D+/0MSzMOg7RhELqYScYDigiVEXkTHA3QSprf/ohjm8woRhQY5CjCEZp5
8MvhD40VAo4Gxgx1V8lwB3CD3kDgFEaUZh2H+N/fmRegACN2lzEm1krOiS7sAB48
/Av99/1VXalLoBbzFTnuAYwnmLk8vdparT5LlwARAQABtChUZXJlbmNlIFRzYW8g
PHRlcmVuY2VAcHJ5c21hdGljbGFicy5jb20+iQJOBBMBCAA4FiEEMX1ukQWPjzwj
A7p3VjE+RFgSl6YFAl8h6lMCGwMFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQ
VjE+RFgSl6YJgBAAqIAMzMFf/+qE896kuXYSlmaYtzhrMXgQVD09UgSq4JfVYudE
e8EfamWz8RFV7znxHO/5Fp4yoWeDGc2b40DhivvRb1p4XSJi+F64moeHt+qn4Hay
CDs5wGv40KJk0+2r7iILc3Gw4NP6r4c2HU2EqSW8fn/bYM3yxLsYEXt5zhgipKpR
CB4WmqojW+CxXkj8dv4CyY+HlKJ6nZWstK42h736njM44rjMDSDt3lv0ZYykopTw
09THnHkWOXY4PSrKOzE+OqsPMPOD3Gq3bYPqQ0ZgdS4FeuDesqqHjJabRX+7DgJK
lLEnylfDlvDKf6I/tqrAfqCCdQvEjpdQXG84GSdK//wT1biunHvGeyHM/PZx9V5O
X8YoecswcYjozU1F7x8buk8GxV7ZuJ2pbdXr7jTou1dA7e4Nas9Qu1RcK39MoTYc
Ub467ONFCX+Y3MzELTnEyx4hSwTxlWg7VjCQ63zrFOd8VG0WUydlqiHgIE8WCJVE
pQlrr2v/A6ieM1HY1Ftjt+mk/qncqibCdyIofyV4o5FrVo8arx74jR+Lhvkp1T9N
uRj8V6M3g9vxJdAsy1O5/ZrdwKJ6Yy1n62+9b/uxhklYONK7DpDlWU0ZLWHlKEIV
zzfILCiLf+nxMQdVwuthlcysg8JwYEE8xDXc851ciuj0ygv3Wm0BY44V6/m5Ag0E
XyHqUwEQALuKzOAlo75p2vSYD7TecE/E0hBOS+cjs8kRH+oIzm5fx7sf00SC1jU2
q5QLYLixNeT+l0bD70R7b8r2uFu1aZL7pQqbGIGisLHlxu8611+PCpE5AsQi3Wui
IZ6Y8K7grJ28vviBiZUBY3iCCRH0LuvyZN3R0zgyMGbzouQk5wuGJUkRHJKtV5by
FVEl3CYudRtAp5LPFw6j7UzT5yQqBmY6tXp5WMmmAvOtnu8ohpRhzY21dJMlSykX
Ne9rcARy8mVWNdcXJUIc85z0BmyrdiA4YY0XiZTHD9mslj+af4QHsaS+p3aorTLD
5BKkp5Ek79a1BUxBjrao4W2fljYf129/SHbwds/Dup26zB2vi/fhbfVPvevXLPpi
Vm4uz8fE4D5lPYZAdu5GtO2V9kWbhDtU1R1SJSdFDI9Sev3B+NLrstclGfdbFQKF
shbUxydjSX56OJvh5hee50PcCz+Ab+usoyUPkcOfET/L55AdqJo3cVYtnAKpOJG/
mKP5Ih3LZVm9wCdWTCzboEtDfLlcjCc5kLiE9Pq7sKMnXm/NcXNFWBkRuaHg+Mt5
Yk659/Q6oUG3yUrS2d7cSeuNRWNlaRlnk3hZtB13xpmoHGYmrMOe7wiEE5xEnOju
1ctRRRX6lNUPlSvID83Y9JSYL9hYMbidRmFY28AIddT/PjVTgfi1ABEBAAGJAjYE
GAEIACAWIQQxfW6RBY+PPCMDundWMT5EWBKXpgUCXyHqUwIbDAAKCRBWMT5EWBKX
poMOD/4/sRLnK94pY2qtSSfNZEumOdQVvges08u34saG4mT1iLD9TcRgsFvgwcTV
Pec9Git4iJmoJpFylvOzoBWmtDzJ7TB3JxvtwUfFH2uZ22k12eChyAFHI8TxASqI
TV2YwkIgB2PTmva1GrdNKM6y5bfS1BdS0u+Etgp4zN9nyQECY2sM2accUi2S7JBV
7o6lEJWCRgFUwIfY6fdfxhZo/w0g2u5wftUQYlbXWSQLImFFZBc/13k8EMxsfbMi
LQ/wxKEfe1ZwFoXqIS6cq8CHTdj70QO7K9ah6krGPmL23LVuxlkQIR7mT5yFJtv2
VRT4IYgOUl6rAd08B6MOy6MOMa14FpNrAybPK8Yd/eSdUNciVf17q33Yc15x+trs
En1hHbDS82xaLJ/Ld4mANeJVt3FodgVmzZGpKheFEOFXXeuX1ZZ4c9TgX+0wBKoh
aBb8EGBVo4aV4GbZ/gEa8wls5NNTwd56H4G94hiq+qFM39x4YjhAvkM0hLRYzR05
tSCdlEbkh2K6RbOhBPsNHCbypWRt2fN8/q4uLPJJt7NRQ2zi6H/x04HGblhXdX6H
mmYjJv1LTbem9VptcpvPauNsibeIvIWA2nYM2ncDWt6gJpOH9Zh4fHuaG4aCdNmJ
hPNeJNnmLcpDQvR9wU5w7e5tC/ZSeTZ5ul1zOKa1qZ4lJ50BDQ==
=a30p
-----END PGP PUBLIC KEY BLOCK-----

View File

@@ -3,24 +3,24 @@ Hash: SHA512
Contact: mailto:security@prysmaticlabs.com
Encryption: openpgp4fpr:0AE0051D647BA3C1A917AF4072E33E4DF1A5036E
Encryption: openpgp4fpr:341396BAFACC28C5082327F889725027FC8EC0D4
Encryption: openpgp4fpr:8B7814F1B221A8E8AA465FC7BDBF744ADE1A0033
Encryption: openpgp4fpr:CD08DE68C60B82D3EE2A3F7D95452A701810FEDB
Encryption: openpgp4fpr:317D6E91058F8F3C2303BA7756313E44581297A6
Encryption: openpgp4fpr:79C59A585E3FD3AFFA00F5C22940A6479DA7C9EC
Preferred-Languages: en
Canonical: https://github.com/prysmaticlabs/prysm/tree/master/.well-known/security.txt
-----BEGIN PGP SIGNATURE-----
iQIzBAEBCgAdFiEECuAFHWR7o8GpF69AcuM+TfGlA24FAlzi0WgACgkQcuM+TfGl
A241pw/+Ks3Hxx8eGbjRIeuncuK811FkCiofNJS+MY2p4W2/tIrk48DtLRx8/k5L
Dh1QyypZsqUgofrK7PbGVdEin6oEb2jYbTWUarAVTbhlsUdM4YcxwpgmGVslW7+C
Hm8wMasQZhCkFfakzhfKX5hIQoFaFI/OvtVKIQsodP8dAieCDaGmtfq1Bs1LgFqi
KrpeEdC2XbBQs33ADheC5SdGT1mnatP3VX8cOhLsfoPksYgTSpwK0clkoWs1eZOQ
l1ImfW/FJCpSndBWgBR503ZgaU3Ic+5qxmAIuUP4chl0DFRMlPFEM5OWC6JkkCOd
5kKrXGRmrhgtQg+pA3zqJnFItRj7gxPBA/ypxCkKPrLEkRvbdpdZEl5vAlYkeBL6
iKSLHnMswGKldiYxy7ofam5bM3myhYYNFb25boV5pRptrnoUmWOACHioBGQHwWNt
B0XktD0j7+pCCiJyyYxmOnElsk/Y/u4Tv5pYWvfFuxTF2XOg+P/EH64AIFLWgB1U
VnITxhakxqejCBxZkuVCFNSzt+TXG0NS9EIj/UOYBY+wxrBZ62ITjdA16RS/3n3z
DuIDtxOOwUumbOO32+a5zIb+ARmnocYJviI7FuENb01/U6qb+nm9hQI6oIpSCNsv
Pb4O/ZlOx70U/7mt4Xn/dTKH9bnKOOVhOw00KJWFfAce73AVnLA=
=Uhqg
iQIzBAEBCgAdFiEECuAFHWR7o8GpF69AcuM+TfGlA24FAmGOfiYACgkQcuM+TfGl
A24YwRAAiQk3w6yzqSEggrOlNoNn04iu/rWZdn5ihkQgzACXy8XH2D1gdKLChE/X
7e5bUtgE2aCuHryQjwoKxqZakviBJFstVmHgF64rXv2zKhpqA30Mj4fI+T3zn8I+
+FpFV0TTsxNLDx+AcR1eQ1nSayO7ImUDIfOQNDDnSZZy42Bc+F+QIGKB3aH/8bpG
kT+bDTZrXvX+TE1gZTbAtZG8sH8g/zadoWEHIhfXUuYb0kTz+DRzAxoqU4j4Z4ee
1zSfFAgfJwxJP4kWD7s4xkE1sBbCgGBeD6cW/C2lbcfIei+XSizLpHW3jD9dNqh4
fLkmEspSa/LV/iXFq8nFzu/GLww4q+sQZDzzDKZyws54CrATinRitZMhzoIL0bTn
yFZVOGHosFAMEVZ36dl1Aw2+B2W6tr2CVr9c5zfV+kup5/KZH1EmT5nYY/zFwfg2
jYCFB5wmYeiyWZvuprgJXRArgVZLZaJxwWazlPVk4i/4vPvRgvfHqOwHCBe8DXy0
VHPhpewwb/ECYek1KoaNQflgR8iH2GMHkC5RjhGDAt1S0AQDtite5m4ZYt1kvO9E
k/znkv89dduhL9CKDvZvnI+DICwsTrf//4KJ8PM/qaPAJa4GvtiUU/eS/jKBivtv
OP5dZQtX6KPc9ewqqZgn622uHSezoBidgeTkdZsJ6tw2eIu0lsY=
=V7L0
-----END PGP SIGNATURE-----

View File

@@ -3,8 +3,8 @@ load("@com_github_atlassian_bazel_tools//gometalinter:def.bzl", "gometalinter")
load("@com_github_atlassian_bazel_tools//goimports:def.bzl", "goimports")
load("@io_kubernetes_build//defs:run_in_workspace.bzl", "workspace_binary")
load("@io_bazel_rules_go//go:def.bzl", "nogo")
load("@graknlabs_bazel_distribution//common:rules.bzl", "assemble_targz", "assemble_versioned")
load("//tools:binary_targets.bzl", "binary_targets", "determine_targets")
load("@vaticle_bazel_distribution//common:rules.bzl", "assemble_targz", "assemble_versioned")
load("@bazel_skylib//rules:common_settings.bzl", "string_setting")
prefix = "github.com/prysmaticlabs/prysm"
@@ -12,7 +12,12 @@ exports_files([
"LICENSE.md",
])
# gazelle:prefix github.com/prysmaticlabs/prysm
# gazelle:prefix github.com/prysmaticlabs/prysm/v3
# gazelle:map_kind go_library go_library @prysm//tools/go:def.bzl
# gazelle:map_kind go_test go_test @prysm//tools/go:def.bzl
# gazelle:map_kind go_repository go_repository @prysm//tools/go:def.bzl
# gazelle:build_tags bazel
# gazelle:exclude tools/analyzers/**/testdata/**
gazelle(
name = "gazelle",
prefix = prefix,
@@ -21,7 +26,16 @@ gazelle(
# Protobuf compiler (non-gRPC)
alias(
name = "proto_compiler",
actual = "@io_bazel_rules_go//proto:gogofast_proto",
actual = "@io_bazel_rules_go//proto:go_proto",
visibility = [
"//proto:__subpackages__",
],
)
# Cast protobuf compiler (non-gRPC)
alias(
name = "cast_proto_compiler",
actual = "@com_github_prysmaticlabs_protoc_gen_go_cast//:go_cast",
visibility = [
"//proto:__subpackages__",
],
@@ -30,28 +44,22 @@ alias(
# Protobuf compiler (gRPC)
alias(
name = "grpc_proto_compiler",
actual = "@io_bazel_rules_go//proto:gogofast_grpc",
visibility = [
"//proto:__subpackages__",
],
actual = "@io_bazel_rules_go//proto:go_grpc",
visibility = ["//visibility:public"],
)
# Protobuf gRPC compiler without gogoproto. Required for gRPC gateway.
# Cast protobuf compiler (gRPC)
alias(
name = "grpc_nogogo_proto_compiler",
actual = "@io_bazel_rules_go//proto:go_grpc",
visibility = [
"//proto:__subpackages__",
],
name = "cast_grpc_proto_compiler",
actual = "@com_github_prysmaticlabs_protoc_gen_go_cast//:go_cast_grpc",
visibility = ["//visibility:public"],
)
# Protobuf gRPC gateway compiler
alias(
name = "grpc_gateway_proto_compiler",
actual = "@grpc_ecosystem_grpc_gateway//protoc-gen-grpc-gateway:go_gen_grpc_gateway",
visibility = [
"//proto:__subpackages__",
],
actual = "@com_github_grpc_ecosystem_grpc_gateway_v2//protoc-gen-grpc-gateway:go_gen_grpc_gateway",
visibility = ["//visibility:public"],
)
gometalinter(
@@ -80,50 +88,61 @@ nogo(
config = "nogo_config.json",
visibility = ["//visibility:public"],
deps = [
"@org_golang_x_tools//go/analysis/passes/unsafeptr:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/unreachable:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/unmarshal:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/tests:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/structtag:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/stdmethods:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/shift:go_tool_library",
# "@org_golang_x_tools//go/analysis/passes/shadow:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/printf:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/pkgfact:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/nilness:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/nilfunc:go_tool_library",
# lost cancel ignore doesn't seem to work when running with coverage
#"@org_golang_x_tools//go/analysis/passes/lostcancel:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/loopclosure:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/httpresponse:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/findcall:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/deepequalerrors:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/ctrlflow:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/copylock:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/composite:go_tool_library",
# "@org_golang_x_tools//go/analysis/passes/cgocall:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/buildtag:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/buildssa:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/bools:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/atomicalign:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/atomic:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/assign:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/inspect:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/asmdecl:go_tool_library",
],
"@org_golang_x_tools//go/analysis/passes/unsafeptr:go_default_library",
"@org_golang_x_tools//go/analysis/passes/unreachable:go_default_library",
"@org_golang_x_tools//go/analysis/passes/unmarshal:go_default_library",
"@org_golang_x_tools//go/analysis/passes/tests:go_default_library",
"@org_golang_x_tools//go/analysis/passes/structtag:go_default_library",
"@org_golang_x_tools//go/analysis/passes/stdmethods:go_default_library",
"@org_golang_x_tools//go/analysis/passes/shift:go_default_library",
# "@org_golang_x_tools//go/analysis/passes/shadow:go_default_library",
"@org_golang_x_tools//go/analysis/passes/printf:go_default_library",
"@org_golang_x_tools//go/analysis/passes/pkgfact:go_default_library",
"@org_golang_x_tools//go/analysis/passes/nilness:go_default_library",
"@org_golang_x_tools//go/analysis/passes/nilfunc:go_default_library",
"@org_golang_x_tools//go/analysis/passes/loopclosure:go_default_library",
"@org_golang_x_tools//go/analysis/passes/httpresponse:go_default_library",
"@org_golang_x_tools//go/analysis/passes/findcall:go_default_library",
"@org_golang_x_tools//go/analysis/passes/deepequalerrors:go_default_library",
"@org_golang_x_tools//go/analysis/passes/ctrlflow:go_default_library",
"@org_golang_x_tools//go/analysis/passes/copylock:go_default_library",
# "@org_golang_x_tools//go/analysis/passes/cgocall:go_default_library",
"@org_golang_x_tools//go/analysis/passes/buildtag:go_default_library",
"@org_golang_x_tools//go/analysis/passes/buildssa:go_default_library",
"@org_golang_x_tools//go/analysis/passes/bools:go_default_library",
"@org_golang_x_tools//go/analysis/passes/atomicalign:go_default_library",
"@org_golang_x_tools//go/analysis/passes/atomic:go_default_library",
"@org_golang_x_tools//go/analysis/passes/assign:go_default_library",
"@org_golang_x_tools//go/analysis/passes/inspect:go_default_library",
"@org_golang_x_tools//go/analysis/passes/asmdecl:go_default_library",
"//tools/analyzers/comparesame:go_default_library",
"//tools/analyzers/cryptorand:go_default_library",
"//tools/analyzers/errcheck:go_default_library",
"//tools/analyzers/featureconfig:go_default_library",
"//tools/analyzers/gocognit:go_default_library",
"//tools/analyzers/ineffassign:go_default_library",
"//tools/analyzers/interfacechecker:go_default_library",
"//tools/analyzers/logruswitherror:go_default_library",
"//tools/analyzers/maligned:go_default_library",
"//tools/analyzers/nop:go_default_library",
"//tools/analyzers/properpermissions:go_default_library",
"//tools/analyzers/recursivelock:go_default_library",
"//tools/analyzers/shadowpredecl:go_default_library",
"//tools/analyzers/slicedirect:go_default_library",
"//tools/analyzers/uintcast:go_default_library",
] + select({
# nogo checks that fail with coverage enabled.
":coverage_enabled": [],
"//conditions:default": [
"@org_golang_x_tools//go/analysis/passes/lostcancel:go_default_library",
"@org_golang_x_tools//go/analysis/passes/composite:go_default_library",
],
}),
)
assemble_versioned(
name = "assemble-versioned-all",
tags = ["manual"],
targets = [
":assemble-{}-{}-targz".format(
pair[0],
pair[1],
)
for pair in binary_targets
],
version_file = "//:VERSION",
config_setting(
name = "coverage_enabled",
values = {"define": "coverage_enabled=1"},
)
common_files = {
@@ -131,15 +150,8 @@ common_files = {
"//:README.md": "README.md",
}
[assemble_targz(
name = "assemble-{}-{}-targz".format(
pair[0],
pair[1],
),
additional_files = determine_targets(pair, common_files),
output_filename = "prysm-{}-{}".format(
pair[0],
pair[1],
),
tags = ["manual"],
) for pair in binary_targets]
sh_binary(
name = "prysm_sh",
srcs = ["prysm.sh"],
visibility = ["//visibility:public"],
)

View File

@@ -1,8 +1,10 @@
# Contribution Guidelines
Excited by our work and want to get involved in building out our sharding releases? Or maybe you haven't learned as much about the Ethereum protocol but are a savvy developer? Our [READINGS.md](https://github.com/prysmaticlabs/prysm/blob/master/docs/READINGS.md) doc includes comprehensive information on Ethereum and sharding for both part-time and core contributors to the project.
Note: The latest and most up to date documenation can be found on our [docs portal](https://docs.prylabs.network/docs/contribute/contribution-guidelines).
You can explore our [Open Issues](https://github.com/prysmaticlabs/prysm/issues) in-the works for our different releases. Feel free to fork our repo and start creating PRs after assigning yourself to an issue of interest. We are always chatting on [Discord](https://discord.gg/che9auJ) or [Gitter](https://gitter.im/prysmaticlabs/geth-sharding) drop us a line there if you want to get more involved or have any questions on our implementation!
Excited by our work and want to get involved in building out our sharding releases? Or maybe you haven't learned as much about the Ethereum protocol but are a savvy developer?
You can explore our [Open Issues](https://github.com/prysmaticlabs/prysm/issues) in-the works for our different releases. Feel free to fork our repo and start creating PRs after assigning yourself to an issue of interest. We are always chatting on [Discord](https://discord.gg/CTYGPUJ) drop us a line there if you want to get more involved or have any questions on our implementation!
## Contribution Steps
@@ -60,12 +62,6 @@ Changes that only affect a single file can be tested with
$ go test <file_you_are_working_on>
```
Changes that affect multiple files can be tested with ...
```
$ golangci-lint run && bazel test //...
```
**10. Stage the file or files that you want to commit.**
```
@@ -179,7 +175,7 @@ We consider two types of contributions to our repo and categorize them as follow
### Part-Time Contributors
Anyone can become a part-time contributor and help out on implementing sharding. The responsibilities of a part-time contributor include:
Anyone can become a part-time contributor and help out on implementing Ethereum consensus. The responsibilities of a part-time contributor include:
- Engaging in Gitter conversations, asking the questions on how to begin contributing to the project
- Opening up github issues to express interest in code to implement
@@ -190,13 +186,11 @@ Anyone can become a part-time contributor and help out on implementing sharding.
- Follow up on open PRs
- Have an estimated timeframe to completion and let the core contributors know if a PR will take longer than expected
We do not expect all part-time contributors to be experts on all the latest sharding documentation, but all contributors should at least be familiarized with our sharding [README.md](https://github.com/prysmaticlabs/prysm/blob/master/validator/README.md) and have gone through the required Ethereum readings as posted on our [READINGS.md](https://github.com/prysmaticlabs/prysm/blob/master/docs/READINGS.md) document.
### Core Contributors
Core contributors are remote contractors of Prysmatic Labs, LLC. and are considered critical team members of our organization. Core devs have all of the responsibilities of part-time contributors plus the majority of the following:
- Stay up to date on the latest beacon chain sepcification
- Stay up to date on the latest beacon chain specification
- Monitor github issues and PRs to make sure owner, labels, descriptions are correct
- Formulate independent ideas, suggest new work to do, point out improvements to existing approaches
- Participate in code review, ensure code quality is excellent, and have ensure high code coverage

80
DEPENDENCIES.md Normal file
View File

@@ -0,0 +1,80 @@
# Dependency Management in Prysm
Prysm is go project with many complicated dependencies, including some c++ based libraries. There
are two parts to Prysm's dependency management. Go modules and bazel managed dependencies. Be sure
to read [Why Bazel?](https://github.com/prysmaticlabs/documentation/issues/138) to fully
understand the reasoning behind an additional layer of build tooling via Bazel rather than a pure
"go build" project.
## Go Module support
The Prysm project officially supports go modules with some caveats.
### Caveat 1: Some c++ libraries are precompiled archives
Given some of Prysm's c++ dependencies have very complicated project structures which make building
difficult or impossible with "go build" alone. Additionally, building c++ dependencies with certain
compilers, like clang / LLVM, offer a significant performance improvement. To get around this
issue, c++ dependencies have been precompiled as linkable archives. While there isn't necessarily
anything bad about precompiled archives, these files are a "blackbox" which a 3rd party author
could have compiled anything for this archive and detecting undesired behavior would be nearly
impossible. If your risk tolerance is low, always compile everything from source yourself,
including complicated c++ dependencies.
*Recommendation: Use go build only for local development and use bazel build for production.*
### Caveat 2: Generated gRPC protobuf libraries
One key advantage of Bazel over vanilla `go build` is that Bazel automatically (re)builds generated
pb.go files at build time when file changes are present in any protobuf definition file or after
any updates to the protobuf compiler or other relevant dependencies. Vanilla go users should run
the following scripts often to ensure their generated files are up to date. Furthermore, Prysm
generates SSZ marshal related code based on defined data structures. These generated files must
also be updated and checked in as frequently.
```bash
./hack/update-go-pbs.sh
./hack/update-go-ssz.sh
```
*Recommendation: Use go build only for local development and use bazel build for production.*
### Caveat 3: Compile-time optimizations
When Prysmatic Labs builds production binaries, they use the "release" configuration of bazel to
compile with several compiler optimizations and recommended production build configurations.
Additionally, the release build properly stamps the built binaries to include helpful metadata
about how and when the binary was built.
*Recommendation: Use go build only for local development and use bazel build for production.*
```bash
bazel build //beacon-chain --config=release
```
## Adding / updating dependencies
1. Add your dependency as you would with go modules. I.e. `go get ...`
1. Run `gazelle update-repos -from_file=go.mod` to update the bazel managed dependencies.
Example:
```bash
go get github.com/prysmaticlabs/example@v1.2.3
bazel run //:gazelle -- update-repos -from_file=go.mod -to_macro=deps.bzl%prysm_deps -prune=true
```
The deps.bzl file should have been updated with the dependency and any transitive dependencies.
Do NOT add new `go_repository` to the WORKSPACE file. All dependencies should live in deps.bzl.
## Running tests
To enable conditional compilation and custom configuration for tests (where compiled code has more
debug info, while not being completely optimized), we rely on Go's build tags/constraints mechanism
(see official docs on [build constraints](https://golang.org/pkg/go/build/#hdr-Build_Constraints)).
Therefore, whenever using `go test`, do not forget to pass in extra build tag, eg:
```bash
go test ./beacon-chain/sync/initial-sync -tags develop
```

View File

@@ -1,6 +1,6 @@
# Prysm Client Interoperability Guide
This README details how to setup Prysm for interop testing for usage with other Ethereum 2.0 clients.
This README details how to setup Prysm for interop testing for usage with other Ethereum consensus clients.
## Installation & Setup
@@ -26,15 +26,18 @@ You can use `bazel run //tools/genesis-state-gen` to create a deterministic gene
### Usage
- **--genesis-time** uint: Unix timestamp used as the genesis time in the generated genesis state (defaults to now)
- **--mainnet-config** bool: Select whether genesis state should be generated with mainnet or minimal (default) params
- **--num-validators** int: Number of validators to deterministically include in the generated genesis state
- **--output-ssz** string: Output filename of the SSZ marshaling of the generated genesis state
- **--config-name=interop** string: name of the beacon chain config to use when generating the state. ex mainnet|minimal|interop
**deprecated flag: use --config-name instead**
- **--mainnet-config** bool: Select whether genesis state should be generated with mainnet or minimal (default) params
The example below creates 64 validator keys, instantiates a genesis state with those 64 validators and with genesis unix timestamp 1567542540,
and finally writes a ssz encoded output to ~/Desktop/genesis.ssz. This file can be used to kickstart the beacon chain in the next section.
and finally writes a ssz encoded output to ~/Desktop/genesis.ssz. This file can be used to kickstart the beacon chain in the next section. When using the `--interop-*` flags, the beacon node will assume the `interop` config should be used, unless a different config is specified on the command line.
```
bazel run //tools/genesis-state-gen -- --output-ssz ~/Desktop/genesis.ssz --num-validators 64 --genesis-time 1567542540
bazel run //tools/genesis-state-gen -- --config-name interop --output-ssz ~/Desktop/genesis.ssz --num-validators 64 --genesis-time 1567542540
```
## Launching a Beacon Node + Validator Client
@@ -45,10 +48,11 @@ Open up two terminal windows, run:
```
bazel run //beacon-chain -- \
--no-genesis-delay \
--bootstrap-node= \
--deposit-contract 0xD775140349E6A5D12524C6ccc3d6A1d4519D4029 \
--clear-db \
--deposit-contract 0x8A04d14125D0FDCDc742F4A05C051De07232EDa4 \
--datadir=/tmp/beacon-chain-interop \
--force-clear-db \
--min-sync-peers=0 \
--interop-num-validators 64 \
--interop-eth1data-votes
```
@@ -58,11 +62,10 @@ the system with 64 validators and the genesis time set to the current unix times
Wait a bit until your beacon chain starts, and in the other window:
```
bazel run //validator -- --interop-num-validators 64
bazel run //validator -- --keymanager=interop --keymanageropts='{"keys":64}'
```
This will launch and kickstart the system with your 64 validators performing their duties accordingly.
specify which keys
### Launching from `genesis.ssz`
@@ -70,10 +73,11 @@ Assuming you generated a `genesis.ssz` file with 64 validators, open up two term
```
bazel run //beacon-chain -- \
--no-genesis-delay \
--bootstrap-node= \
--deposit-contract 0xD775140349E6A5D12524C6ccc3d6A1d4519D4029 \
--clear-db \
--deposit-contract 0x8A04d14125D0FDCDc742F4A05C051De07232EDa4 \
--datadir=/tmp/beacon-chain-interop \
--force-clear-db \
--min-sync-peers=0 \
--interop-genesis-state /path/to/genesis.ssz \
--interop-eth1data-votes
```
@@ -81,13 +85,7 @@ Assuming you generated a `genesis.ssz` file with 64 validators, open up two term
Wait a bit until your beacon chain starts, and in the other window:
```
bazel run //validator -- --interop-num-validators 64
bazel run //validator -- --keymanager=interop --keymanageropts='{"keys":64}'
```
This will launch and kickstart the system with your 64 validators performing their duties accordingly.

View File

@@ -1,13 +0,0 @@
TODO(you): choose "part of" or "resolves" and the associated github issue #.
[Part of|Resolves] #531
---
# Description
**Write why you are making the changes in this pull request**
**Write a summary of the changes you are making**
**Link anything that would be helpful or relevant to the reviewers**

256
README.md
View File

@@ -1,245 +1,37 @@
# Prysm: An Ethereum 2.0 Client Written in Go
# Prysm: An Ethereum Consensus Implementation Written in Go
[![Build status](https://badge.buildkite.com/b555891daf3614bae4284dcf365b2340cefc0089839526f096.svg?branch=master)](https://buildkite.com/prysmatic-labs/prysm)
[![ETH2.0_Spec_Version 0.8.1](https://img.shields.io/badge/ETH2.0%20Spec%20Version-v0.8.1-blue.svg)](https://github.com/ethereum/eth2.0-specs/commit/452ecf8e27c7852c7854597f2b1bb4a62b80c7ec)
[![Discord](https://user-images.githubusercontent.com/7288322/34471967-1df7808a-efbb-11e7-9088-ed0b04151291.png)](https://discord.gg/KSA7rPr)
[![Gitter](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/prysmaticlabs/geth-sharding?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge)
[![Go Report Card](https://goreportcard.com/badge/github.com/prysmaticlabs/prysm)](https://goreportcard.com/report/github.com/prysmaticlabs/prysm)
[![Consensus_Spec_Version 1.2.0](https://img.shields.io/badge/Consensus%20Spec%20Version-v1.2.0-blue.svg)](https://github.com/ethereum/consensus-specs/tree/v1.2.0)
[![Execution_API_Version 1.0.0-beta.1](https://img.shields.io/badge/Execution%20API%20Version-v1.0.0.beta.1-blue.svg)](https://github.com/ethereum/execution-apis/tree/v1.0.0-beta.1/src/engine)
[![Discord](https://user-images.githubusercontent.com/7288322/34471967-1df7808a-efbb-11e7-9088-ed0b04151291.png)](https://discord.gg/CTYGPUJ)
[![GitPOAP Badge](https://public-api.gitpoap.io/v1/repo/prysmaticlabs/prysm/badge)](https://www.gitpoap.io/gh/prysmaticlabs/prysm)
This is the core repository for Prysm, a [Golang](https://golang.org/) implementation of the Ethereum 2.0 client specifications developed by [Prysmatic Labs](https://prysmaticlabs.com).
This is the core repository for Prysm, a [Golang](https://golang.org/) implementation of the [Ethereum Consensus](https://ethereum.org/en/eth2/) specification, developed by [Prysmatic Labs](https://prysmaticlabs.com). See the [Changelog](https://github.com/prysmaticlabs/prysm/releases) for details of the latest releases and upcoming breaking changes.
### Need assistance?
A more detailed set of installation and usage instructions as well as breakdowns of each individual component are available in the [official documentation portal](https://prysmaticlabs.gitbook.io/prysm/). If you still have questions, feel free to stop by either our [Discord](https://discord.gg/KSA7rPr) or [Gitter](https://gitter.im/prysmaticlabs/geth-sharding?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge) and a member of the team or our community will be happy to assist you.
### Getting Started
### Come join the testnet!
Participation is now open to the public for our Ethereum 2.0 phase 0 testnet release. Visit [prylabs.net](https://prylabs.net) for more information on the project or to sign up as a validator on the network.
A detailed set of installation and usage instructions as well as breakdowns of each individual component are available in the [official documentation portal](https://docs.prylabs.network). If you still have questions, feel free to stop by our [Discord](https://discord.gg/CTYGPUJ).
# Table of Contents
### Staking on Mainnet
- [Dependencies](#dependencies)
- [Installation](#installation)
- [Build via Docker](#build-via-docker)
- [Build via Bazel](#build-via-bazel)
- [Connecting to the public testnet: running a beacon node](#connecting-to-the-testnet-running-a-beacon-node)
- [Running via Docker](#build-via-docker)
- [Running via Bazel](#build-via-bazel)
- [Staking ETH: running a validator client](#staking-eth-running-a-validator-client)
- [Activating your validator: depositing 3.2 Goerli ETH](#activating-your-validator-depositing-32-goerli-eth)
- [Starting the validator with Bazel](#starting-the-validator-with-bazel)
- [Setting up a local ETH2 development chain](#setting-up-a-local-eth2-development-chain)
- [Installation and dependencies](#installation-and-dependencies)
- [Running a local beacon node and validator client](#running-a-local-beacon-node-and-validator-client)
- [Testing Prysm](#testing-prysm)
- [Contributing](#contributing)
- [License](#license)
To participate in staking, you can join the [official eth2 launchpad](https://launchpad.ethereum.org). The launchpad is the only recommended way to become a validator on mainnet. You can explore validator rewards/penalties via Bitfly's block explorer: [beaconcha.in](https://beaconcha.in), and follow the latest blocks added to the chain on [beaconscan](https://beaconscan.com).
## Dependencies
Prysm can be installed either with Docker **(recommended method)** or using our build tool, Bazel. The below instructions include sections for performing both.
**For Docker installations:**
- The latest release of [Docker](https://docs.docker.com/install/)
**For Bazel installations:**
- The latest release of [Bazel](https://docs.bazel.build/versions/master/install.html)
- A modern UNIX operating system (MacOS included)
## Installation
### Build via Docker
1. Ensure you are running the most recent version of Docker by issuing the command:
```
docker -v
```
2. To pull the Prysm images from the server, issue the following commands:
```
docker pull gcr.io/prysmaticlabs/prysm/validator:latest
docker pull gcr.io/prysmaticlabs/prysm/beacon-chain:latest
```
This process will also install any related dependencies.
### Build via Bazel
1. Open a terminal window. Ensure you are running the most recent version of Bazel by issuing the command:
```
bazel version
```
2. Clone this repository and enter the directory:
```
git clone https://github.com/prysmaticlabs/prysm
cd prysm
```
3. Build both the beacon chain node implementation and the validator client:
```
bazel build //beacon-chain:beacon-chain
bazel build //validator:validator
```
Bazel will automatically pull and install any dependencies as well, including Go and necessary compilers.
4. Build the configuration for the Prysm testnet by issuing the commands:
```
bazel build --define ssz=minimal //beacon-chain:beacon-chain
bazel build --define ssz=minimal //validator:validator
```
The binaries will be built in an architecture-dependent subdirectory of `bazel-bin`, and are supplied as part of Bazel's build process. To fetch the location, issue the command:
```
$ bazel build --define ssz=minimal //beacon-chain:beacon-chain
...
Target //beacon-chain:beacon-chain up-to-date:
bazel-bin/beacon-chain/linux_amd64_stripped/beacon-chain
...
```
In the example above, the beacon chain binary has been created in `bazel-bin/beacon-chain/linux_amd64_stripped/beacon-chain`.
## Connecting to the testnet: running a beacon node
This section contains instructions for initialising a beacon node and connecting to the public testnet. To further understand the role that both the beacon node and validator play in Prysm, see [this section of our documentation](https://prysmaticlabs.gitbook.io/prysm/how-prysm-works/overview-technical).
### Running via Docker
#### Docker on Linux/Mac
To start your beacon node, issue the following command:
```
docker run -v $HOME/prysm-data:/data -p 4000:4000 \
--name beacon-node \
gcr.io/prysmaticlabs/prysm/beacon-chain:latest \
--no-genesis-delay \
--datadir=/data
```
(Optional) If you want to enable gRPC, then run this command instead of the one above:
```
docker run -v $HOME/prysm-data:/data -p 4000:4000 -p 7000:7000 \
--name beacon-node \
gcr.io/prysmaticlabs/prysm/beacon-chain:latest \
--datadir=/data \
--no-genesis-delay \
--grpc-gateway-port=7000
```
You can halt the beacon node using `Ctrl+c` or with the following command:
```
docker stop beacon-node
```
To restart the beacon node, issue the command:
```
docker start -ai beacon-node
```
To delete a corrupted container, issue the command:
```
docker rm beacon-node
```
To recreate a deleted container and refresh the chain database, issue the start command with an additional `--force-clear-db` parameter:
```
docker run -it -v $HOME/prysm-data:/data -p 4000:4000 --name beacon-node \
gcr.io/prysmaticlabs/prysm/beacon-chain:latest \
--datadir=/data \
--force-clear-db
```
#### Docker on Windows
1) You will need to share the local drive you wish to mount to to container (e.g. C:).
1. Enter Docker settings (right click the tray icon)
2. Click 'Shared Drives'
3. Select a drive to share
4. Click 'Apply'
2) You will next need to create a directory named ```/tmp/prysm-data/``` within your selected shared Drive. This folder will be used as a local data directory for Beacon Node chain data as well as account and keystore information required by the validator. Docker will **not** create this directory if it does not exist already. For the purposes of these instructions, it is assumed that ```C:``` is your prior-selected shared Drive.
4) To run the beacon node, issue the command:
```
docker run -it -v c:/tmp/prysm-data:/data -p 4000:4000 gcr.io/prysmaticlabs/prysm/beacon-chain:latest --datadir=/data
```
### Running via Bazel
1) To start your Beacon Node with Bazel, issue the command:
```
bazel run //beacon-chain -- --datadir=/tmp/prysm-data
```
This will sync up the Beacon Node with the latest head block in the network. Note that the beacon node must be **completely synced** before attempting to initialise a validator client, otherwise the validator will not be able to complete the deposit and funds will be lost.
## Staking ETH: running a validator client
Once your beacon node is up, the chain will be waiting for you to deposit 3.2 Goerli ETH into the Validator Deposit Contract to activate your validator (discussed in the section below). First though, you will need to create a validator client to connect to this node in order to stake and participate. Each validator represents 3.2 Goerli ETH being staked in the system, and it is possible to spin up as many as you desire in order to have more stake in the network.
For more information on the functionality of validator clients, see [this section](https://prysmaticlabs.gitbook.io/prysm/how-prysm-works/validator-clients) of our official documentation.
### Activating your validator: depositing 3.2 Goerli ETH
Using your validator deposit data from the previous step, follow the instructions found on https://prylabs.net/participate to make a deposit.
It will take a while for the nodes in the network to process your deposit, but once your node is active, the validator will begin doing its responsibility. In your validator client, you will be able to frequently see your validator balance as it goes up over time. Note that, should your node ever go offline for a long period, you'll start gradually losing your deposit until you are removed from the system.
### Starting the validator with Bazel
1. Open another terminal window. Enter your Prysm directory and run the validator by issuing the following command:
```
cd prysm
bazel run //validator
```
**Congratulations, you are now running Ethereum 2.0 Phase 0!**
## Setting up a local ETH2 development chain
This section outlines the process of setting up Prysm for local interop testing with other Ethereum 2.0 client implementations. See the [INTEROP.md](https://github.com/prysmaticlabs/prysm/blob/master/INTEROP.md) file for advanced configuration options. For more background information on interoperability development, see [this blog post](https://blog.ethereum.org/2019/09/19/eth2-interop-in-review/).
### Installation and dependencies
To begin setting up a local ETH2 development chain, follow the **Bazel** instructions found in the [dependencies](#dependencies) and [installation](#installation) sections respectively.
### Running a local beacon node and validator client
The example below will deterministically generate a beacon genesis state, initiate Prysm with 64 validators and set the genesis time to your local machines current UNIX time.
1. Open up two terminal windows. In the first, issue the command:
```
bazel run //beacon-chain -- \
--no-genesis-delay \
--bootstrap-node= \
--deposit-contract 0xD775140349E6A5D12524C6ccc3d6A1d4519D4029 \
--clear-db \
--interop-num-validators 64 \
--interop-eth1data-votes
```
2. Wait a moment for the beacon chain to start. In the other terminal, issue the command:
```
bazel run //validator -- --interop-num-validators 64
```
This command will kickstart the system with your 64 validators performing their duties accordingly.
## Testing Prysm
To run the unit tests of our system, issue the command:
```
bazel test //...
```
To run the linter, ensure you have [golangci-lint](https://github.com/golangci/golangci-lint) installed, then issue the command:
```
golangci-lint run
```
## Contributing
Want to get involved? Check out our [Contribution Guide](https://prysmaticlabs.gitbook.io/prysm/getting-involved/contribution-guidelines) to learn more!
### Branches
Prysm maintains two permanent branches:
* [master](https://github.com/prysmaticlabs/prysm/tree/master): This points to the latest stable release. It is ideal for most users.
* [develop](https://github.com/prysmaticlabs/prysm/tree/develop): This is used for development, it contains the latest PRs. Developers should base their PRs on this branch.
### Guide
Want to get involved? Check out our [Contribution Guide](https://docs.prylabs.network/docs/contribute/contribution-guidelines/) to learn more!
## License
[GNU General Public License v3.0](https://www.gnu.org/licenses/gpl-3.0.en.html)
## Legal Disclaimer
[Terms of Use](/TERMS_OF_SERVICE.md)

11
SECURITY.md Normal file
View File

@@ -0,0 +1,11 @@
# Security Policy
## Supported Versions
[Releases](https://github.com/prysmaticlabs/prysm/releases/) contains all available releases. We recommend using the [most recently released version](https://github.com/prysmaticlabs/prysm/releases/latest).
## Reporting a Vulnerability
Please see our signed [security.txt](https://github.com/prysmaticlabs/prysm/blob/develop/.well-known/security.txt) for preferred encryption and reporting destination.
**Please do not file a public ticket** mentioning the vulnerability, as doing so could increase the likelihood of the vulnerability being used before a fix has been created, released and installed on the network.

45
TERMS_OF_SERVICE.md Normal file
View File

@@ -0,0 +1,45 @@
## Terms of Use
Effective as of Oct 14, 2020
By downloading, accessing or using the Prysm implementation (“Prysm”), you (referenced herein as “you” or the “user”) certify that you have read and agreed to the terms and conditions below (the “Terms”) which form a binding contract between you and Prysmatic Labs (referenced herein as “we” or “us”). If you do not agree to the Terms, do not download or use Prysm.
### About Prysm
Prysm is a client implementation for Ethereum consensus protocol for a proof-of-stake blockchain. To participate in the network, a user must send ETH from the Eth1.0 chain into a validator deposit contract, which will queue in the user as a validator in the system. Validators participate in proposing and voting on blocks in the protocol, and the network applies rewards/penalties based on their behavior. A detailed set of installation and usage instructions as well as breakdowns of each individual component are available in the official documentation portal, however, we do not warrant the accuracy, completeness or usefulness of this documentation. Any reliance you place on such information is strictly at your own risk.
### Licensing Terms
Prysm is a fully open-source software program licensed pursuant to the GNU General Public License v3.0.
The Prysmatic Labs name, the term “Prysm” and all related names, logos, product and service names, designs and slogans are trademarks of Prysmatic Labs or its affiliates and/or licensors. You must not use such marks without our prior written permission.
### Risks of Operating Prysm
The use of Prysm and acting as a validator on the Ethereum network can lead to loss of money. Ethereum is still an experimental system and ETH remains a risky investment. You alone are responsible for your actions on Prysm including the security of your ETH and meeting any applicable minimum system requirements.
Use of Prysm and the ability to receive rewards or penalties may be affected at any time by mistakes made by the user or other users, software problems such as bugs, errors, incorrectly constructed transactions, unsafe cryptographic libraries or malware affecting the network, technical failures in the hardware of a user, security problems experienced by a user and/or actions or inactions of third parties and/or events experienced by third parties, among other risks. We cannot and do not guarantee that any user of Prysm will make money, that the Prysm network will operate in accordance with the documentation or that transactions will be effective or secure.
We make no claims that Prysm is appropriate or permitted for use in any specific jurisdiction. Access to Prysm may not be legal by certain persons or in certain jurisdictions or countries. If you access Prysm, you do so on your own initiative and are responsible for compliance with local laws.
Some Internet plans will charge an additional amount for any excess upload bandwidth used that isnt included in the plan and may terminate your connection without warning because of overuse. We advise that you check whether your Internet connection is subjected to such limitations and monitor your bandwidth use so that you can stop Prysm before you reach your upload limit.
### Warranty Disclaimer
PRYSM IS PROVIDED ON AN “AS-IS” BASIS AND MAY INCLUDE ERRORS, OMISSIONS, OR OTHER INACCURACIES. PRYSMATIC LABS AND ITS CONTRIBUTORS MAKE NO REPRESENTATIONS OR WARRANTIES ABOUT PRYSM FOR ANY PURPOSE, AND HEREBY EXPRESSLY DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING, WITHOUT LIMITATION, ANY WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT OR ANY OTHER IMPLIED WARRANTY UNDER THE UNIFORM COMPUTER INFORMATION TRANSACTIONS ACT AS ENACTED BY ANY STATE. WE ALSO MAKE NO REPRESENTATIONS OR WARRANTIES THAT PRYSM WILL OPERATE ERROR-FREE, UNINTERRUPTED, OR IN A MANNER THAT WILL MEET YOUR REQUIREMENTS AND/OR NEEDS. THEREFORE, YOU ASSUME THE ENTIRE RISK REGARDING THE QUALITY AND/OR PERFORMANCE OF PRYSM AND ANY TRANSACTIONS ENTERED INTO THEREON.
### Limitation of Liability
In no event will Prysmatic Labs or any of its contributors be liable, whether in contract, warranty, tort (including negligence, whether active, passive or imputed), product liability, strict liability or other theory, breach of statutory duty or otherwise arising out of, or in connection with, your use of Prysm, for any direct, indirect, incidental, special or consequential damages (including any loss of profits or data, business interruption or other pecuniary loss, or damage, loss or other compromise of data, in each case whether direct, indirect, incidental, special or consequential) arising out of use Prysm, even if we or other users have been advised of the possibility of such damages. The foregoing limitations and disclaimers shall apply to the maximum extent permitted by applicable law, even if any remedy fails of its essential purpose. You acknowledge and agree that the limitations of liability afforded us hereunder constitute a material and actual inducement and condition to entering into these Terms, and are reasonable, fair and equitable in scope to protect our legitimate interests in light of the fact that we are not receiving consideration from you for providing Prysm.
### Indemnification
To the maximum extent permitted by law, you will defend, indemnify and hold Prysmatic Labs and its contributors harmless from and against any and all claims, actions, suits, investigations, or proceedings by any third party (including any party or purported party to or beneficiary or purported beneficiary of any transaction on Prysm), as well as any and all losses, liabilities,
damages, costs, and expenses (including reasonable attorneys fees) arising out of, accruing from, or in any way related to (i) your breach of the terms of this Agreement, (ii) any transaction, or the failure to occur of any transaction on Prysm, and (iii) your negligence, fraud, or willful misconduct.
### Compliance with Laws and Tax Obligations
Your use of Prysm is subject to all applicable laws of any governmental authority, including, without limitation, federal, state and foreign securities laws, tax laws, tariff and trade laws, ordinances, judgments, decrees, injunctions, writs and orders or like actions of any governmental authority and rules, regulations, orders, interpretations, licenses, and permits of any federal,
regional, state, county, municipal or other governmental authority and you agree to comply with all such laws in your use of Prysm. The users of Prysm are solely responsible to determinate what, if any, taxes apply to their ETH transactions. The owners of, or contributors to, Prysm are not responsible for determining the taxes that apply to ETH transactions.
### Miscellaneous
These Terms will be construed and enforced in accordance with the laws of the state of Illinois as applied to agreements entered into and completely performed in Illinois. You agree to the personal jurisdiction by and venue in Illinois and waive any objection to such jurisdiction or venue.
We reserve the right to revise these Terms, and your rights and obligations are at all times subject to the then-current Terms provided on Prysm. Your continued use of Prysm constitutes acceptance of such revised Terms.
These Terms constitute the entire agreement between you and Prysmatic Labs regarding use of Prysm and will supersede all prior agreements whether, written or oral. No usage of trade or other regular practice or method of dealing between the parties will be used to modify, interpret, supplement, or alter the terms of these Terms.
If any portion of these Terms is held invalid or unenforceable, such invalidity or enforceability will not affect the other provisions of these Terms, which will remain in full force and effect, and the invalid or unenforceable portion will be given effect to the greatest extent possible. The failure of a party to require performance of any provision will not affect that partys right to require performance at any time thereafter, nor will a waiver of any breach or default of these Terms or any provision of these Terms constitute a waiver of any subsequent breach or default or a waiver of the provision itself.

View File

@@ -1,43 +0,0 @@
# Testnet
The Prysmatic Labs test network is available for anyone to join. The easiest way to participate is by joining through the website, https://prylabs.net.
## Interop
For developers looking to connect a client other than Prysm to the test network, here is the relevant information for compatability.
**Spec version** - [v0.8.3](https://github.com/ethereum/eth2.0-specs/tree/v0.8.3)
**ETH 1 Deposit Contract Address** - See https://prylabs.net/contract. This contract is deployed on the [goerli](https://goerli.net/) network.
**Genesis time** - The ETH1 block time in which the 64th deposit to start ETH2 was included. This is NOT midnight of the next day as required by spec.
### ETH 2 Configuration
Use the [minimal config](https://github.com/ethereum/eth2.0-specs/blob/v0.8.3/configs/minimal.yaml) with the following changes.
| field | value |
|-------|-------|
| MIN_DEPOSIT_AMOUNT | 100 |
| MAX_EFFECTIVE_BALANCE | 3.2 * 1e9 |
| EJECTION_BALANCE | 1.6 * 1e9 |
| EFFECTIVE_BALANCE_INCREMENT | 0.1 * 1e9 |
| ETH1_FOLLOW_DISTANCE | 16 |
| GENESIS_FORK_VERSION | See [latest code](https://github.com/prysmaticlabs/prysm/blob/master/shared/params/config.go#L236) |
These parameters reduce the minimal config to 1/10 of the required ETH.
We have a genesis.ssz file available for download [here](https://prysmaticlabs.com/uploads/genesis.ssz)
### Connecting to the network
We have a libp2p bootstrap node available at `/dns4/prylabs.net/tcp/30001/p2p/16Uiu2HAm7Qwe19vz9WzD2Mxn7fXd1vgHHp4iccuyq7TxwRXoAGfc`.
Some of the Prysmatic Labs hosted nodes are behind a libp2p relay, so your libp2p implementation protocol should understand this functionality.
### Other
Undoubtably, you will have bugs. Reach out to us on [Discord](https://discord.gg/KSA7rPr) and be sure to capture issues on Github at https://github.com/prysmaticlabs/prysm/issues.
If you have instructions for you client, we would love to attempt this on your behalf. Kindly send over the instructions via github issue, PR, email to team@prysmaticlabs.com, or discord.

View File

@@ -1 +0,0 @@
0.2.0

1572
WORKSPACE

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,55 @@
load("@prysm//tools/go:def.bzl", "go_library", "go_test")
go_library(
name = "go_default_library",
srcs = [
"checkpoint.go",
"client.go",
"doc.go",
"errors.go",
],
importpath = "github.com/prysmaticlabs/prysm/v3/api/client/beacon",
visibility = ["//visibility:public"],
deps = [
"//beacon-chain/core/helpers:go_default_library",
"//beacon-chain/rpc/apimiddleware:go_default_library",
"//beacon-chain/state:go_default_library",
"//consensus-types/interfaces:go_default_library",
"//consensus-types/primitives:go_default_library",
"//encoding/bytesutil:go_default_library",
"//encoding/ssz/detect:go_default_library",
"//io/file:go_default_library",
"//network/forks:go_default_library",
"//proto/prysm/v1alpha1:go_default_library",
"//runtime/version:go_default_library",
"//time/slots:go_default_library",
"@com_github_ethereum_go_ethereum//common/hexutil:go_default_library",
"@com_github_pkg_errors//:go_default_library",
"@com_github_sirupsen_logrus//:go_default_library",
"@org_golang_x_mod//semver:go_default_library",
],
)
go_test(
name = "go_default_test",
srcs = [
"checkpoint_test.go",
"client_test.go",
],
embed = [":go_default_library"],
deps = [
"//beacon-chain/state:go_default_library",
"//config/params:go_default_library",
"//consensus-types/blocks:go_default_library",
"//consensus-types/blocks/testing:go_default_library",
"//consensus-types/primitives:go_default_library",
"//encoding/ssz/detect:go_default_library",
"//network/forks:go_default_library",
"//proto/prysm/v1alpha1:go_default_library",
"//runtime/version:go_default_library",
"//testing/require:go_default_library",
"//testing/util:go_default_library",
"//time/slots:go_default_library",
"@com_github_pkg_errors//:go_default_library",
],
)

View File

@@ -0,0 +1,263 @@
package beacon
import (
"context"
"fmt"
"path"
"github.com/pkg/errors"
"github.com/prysmaticlabs/prysm/v3/beacon-chain/core/helpers"
"github.com/prysmaticlabs/prysm/v3/beacon-chain/state"
"github.com/prysmaticlabs/prysm/v3/consensus-types/interfaces"
"github.com/prysmaticlabs/prysm/v3/consensus-types/primitives"
"github.com/prysmaticlabs/prysm/v3/encoding/ssz/detect"
"github.com/prysmaticlabs/prysm/v3/io/file"
"github.com/prysmaticlabs/prysm/v3/runtime/version"
"github.com/prysmaticlabs/prysm/v3/time/slots"
log "github.com/sirupsen/logrus"
"golang.org/x/mod/semver"
)
// OriginData represents the BeaconState and SignedBeaconBlock necessary to start an empty Beacon Node
// using Checkpoint Sync.
type OriginData struct {
sb []byte
bb []byte
st state.BeaconState
b interfaces.SignedBeaconBlock
vu *detect.VersionedUnmarshaler
br [32]byte
sr [32]byte
}
// SaveBlock saves the downloaded block to a unique file in the given path.
// For readability and collision avoidance, the file name includes: type, config name, slot and root
func (o *OriginData) SaveBlock(dir string) (string, error) {
blockPath := path.Join(dir, fname("block", o.vu, o.b.Block().Slot(), o.br))
return blockPath, file.WriteFile(blockPath, o.BlockBytes())
}
// SaveState saves the downloaded state to a unique file in the given path.
// For readability and collision avoidance, the file name includes: type, config name, slot and root
func (o *OriginData) SaveState(dir string) (string, error) {
statePath := path.Join(dir, fname("state", o.vu, o.st.Slot(), o.sr))
return statePath, file.WriteFile(statePath, o.StateBytes())
}
// StateBytes returns the ssz-encoded bytes of the downloaded BeaconState value.
func (o *OriginData) StateBytes() []byte {
return o.sb
}
// BlockBytes returns the ssz-encoded bytes of the downloaded SignedBeaconBlock value.
func (o *OriginData) BlockBytes() []byte {
return o.bb
}
func fname(prefix string, vu *detect.VersionedUnmarshaler, slot primitives.Slot, root [32]byte) string {
return fmt.Sprintf("%s_%s_%s_%d-%#x.ssz", prefix, vu.Config.ConfigName, version.String(vu.Fork), slot, root)
}
// DownloadFinalizedData downloads the most recently finalized state, and the block most recently applied to that state.
// This pair can be used to initialize a new beacon node via checkpoint sync.
func DownloadFinalizedData(ctx context.Context, client *Client) (*OriginData, error) {
sb, err := client.GetState(ctx, IdFinalized)
if err != nil {
return nil, err
}
vu, err := detect.FromState(sb)
if err != nil {
return nil, errors.Wrap(err, "error detecting chain config for finalized state")
}
log.Printf("detected supported config in remote finalized state, name=%s, fork=%s", vu.Config.ConfigName, version.String(vu.Fork))
s, err := vu.UnmarshalBeaconState(sb)
if err != nil {
return nil, errors.Wrap(err, "error unmarshaling finalized state to correct version")
}
if s.Slot() != s.LatestBlockHeader().Slot {
return nil, fmt.Errorf("finalized state slot does not match latest block header slot %d != %d", s.Slot(), s.LatestBlockHeader().Slot)
}
sr, err := s.HashTreeRoot(ctx)
if err != nil {
return nil, errors.Wrapf(err, "failed to compute htr for finalized state at slot=%d", s.Slot())
}
header := s.LatestBlockHeader()
header.StateRoot = sr[:]
br, err := header.HashTreeRoot()
if err != nil {
return nil, errors.Wrap(err, "error while computing block root using state data")
}
bb, err := client.GetBlock(ctx, IdFromRoot(br))
if err != nil {
return nil, errors.Wrapf(err, "error requesting block by root = %#x", br)
}
b, err := vu.UnmarshalBeaconBlock(bb)
if err != nil {
return nil, errors.Wrap(err, "unable to unmarshal block to a supported type using the detected fork schedule")
}
realBlockRoot, err := b.Block().HashTreeRoot()
if err != nil {
return nil, errors.Wrap(err, "error computing hash_tree_root of retrieved block")
}
log.Printf("BeaconState slot=%d, Block slot=%d", s.Slot(), b.Block().Slot())
log.Printf("BeaconState htr=%#xd, Block state_root=%#x", sr, b.Block().StateRoot())
log.Printf("BeaconState latest_block_header htr=%#xd, block htr=%#x", br, realBlockRoot)
return &OriginData{
st: s,
b: b,
sb: sb,
bb: bb,
vu: vu,
br: br,
sr: sr,
}, nil
}
// WeakSubjectivityData represents the state root, block root and epoch of the BeaconState + SignedBeaconBlock
// that falls at the beginning of the current weak subjectivity period. These values can be used to construct
// a weak subjectivity checkpoint beacon node flag to be used for validation.
type WeakSubjectivityData struct {
BlockRoot [32]byte
StateRoot [32]byte
Epoch primitives.Epoch
}
// CheckpointString returns the standard string representation of a Checkpoint.
// The format is a a hex-encoded block root, followed by the epoch of the block, separated by a colon. For example:
// "0x1c35540cac127315fabb6bf29181f2ae0de1a3fc909d2e76ba771e61312cc49a:74888"
func (wsd *WeakSubjectivityData) CheckpointString() string {
return fmt.Sprintf("%#x:%d", wsd.BlockRoot, wsd.Epoch)
}
// ComputeWeakSubjectivityCheckpoint attempts to use the prysm weak_subjectivity api
// to obtain the current weak_subjectivity checkpoint.
// For non-prysm nodes, the same computation will be performed with extra steps,
// using the head state downloaded from the beacon node api.
func ComputeWeakSubjectivityCheckpoint(ctx context.Context, client *Client) (*WeakSubjectivityData, error) {
ws, err := client.GetWeakSubjectivity(ctx)
if err != nil {
// a 404/405 is expected if querying an endpoint that doesn't support the weak subjectivity checkpoint api
if !errors.Is(err, ErrNotOK) {
return nil, errors.Wrap(err, "unexpected API response for prysm-only weak subjectivity checkpoint API")
}
// fall back to vanilla Beacon Node API method
return computeBackwardsCompatible(ctx, client)
}
log.Printf("server weak subjectivity checkpoint response - epoch=%d, block_root=%#x, state_root=%#x", ws.Epoch, ws.BlockRoot, ws.StateRoot)
return ws, nil
}
const (
prysmMinimumVersion = "v2.0.7"
prysmImplementationName = "Prysm"
)
// errUnsupportedPrysmCheckpointVersion indicates remote beacon node can't be used for checkpoint retrieval.
var errUnsupportedPrysmCheckpointVersion = errors.New("node does not meet minimum version requirements for checkpoint retrieval")
// for older endpoints or clients that do not support the weak_subjectivity api method
// we gather the necessary data for a checkpoint sync by:
// - inspecting the remote server's head state and computing the weak subjectivity epoch locally
// - requesting the state at the first slot of the epoch
// - using hash_tree_root(state.latest_block_header) to compute the block the state integrates
// - requesting that block by its root
func computeBackwardsCompatible(ctx context.Context, client *Client) (*WeakSubjectivityData, error) {
log.Print("falling back to generic checkpoint derivation, weak_subjectivity API not supported by server")
nv, err := client.GetNodeVersion(ctx)
if err != nil {
return nil, errors.Wrap(err, "unable to proceed with fallback method without confirming node version")
}
if nv.implementation == prysmImplementationName && semver.Compare(nv.semver, prysmMinimumVersion) < 0 {
return nil, errors.Wrapf(errUnsupportedPrysmCheckpointVersion, "%s < minimum (%s)", nv.semver, prysmMinimumVersion)
}
epoch, err := getWeakSubjectivityEpochFromHead(ctx, client)
if err != nil {
return nil, errors.Wrap(err, "error computing weak subjectivity epoch via head state inspection")
}
// use first slot of the epoch for the state slot
slot, err := slots.EpochStart(epoch)
if err != nil {
return nil, errors.Wrapf(err, "error computing first slot of epoch=%d", epoch)
}
log.Printf("requesting checkpoint state at slot %d", slot)
// get the state at the first slot of the epoch
sb, err := client.GetState(ctx, IdFromSlot(slot))
if err != nil {
return nil, errors.Wrapf(err, "failed to request state by slot from api, slot=%d", slot)
}
// ConfigFork is used to unmarshal the BeaconState so we can read the block root in latest_block_header
vu, err := detect.FromState(sb)
if err != nil {
return nil, errors.Wrap(err, "error detecting chain config for beacon state")
}
log.Printf("detected supported config in checkpoint state, name=%s, fork=%s", vu.Config.ConfigName, version.String(vu.Fork))
s, err := vu.UnmarshalBeaconState(sb)
if err != nil {
return nil, errors.Wrap(err, "error using detected config fork to unmarshal state bytes")
}
// compute state and block roots
sr, err := s.HashTreeRoot(ctx)
if err != nil {
return nil, errors.Wrap(err, "error computing hash_tree_root of state")
}
h := s.LatestBlockHeader()
h.StateRoot = sr[:]
br, err := h.HashTreeRoot()
if err != nil {
return nil, errors.Wrap(err, "error while computing block root using state data")
}
bb, err := client.GetBlock(ctx, IdFromRoot(br))
if err != nil {
return nil, errors.Wrapf(err, "error requesting block by root = %d", br)
}
b, err := vu.UnmarshalBeaconBlock(bb)
if err != nil {
return nil, errors.Wrap(err, "unable to unmarshal block to a supported type using the detected fork schedule")
}
br, err = b.Block().HashTreeRoot()
if err != nil {
return nil, errors.Wrap(err, "error computing hash_tree_root for block obtained via root")
}
return &WeakSubjectivityData{
Epoch: epoch,
BlockRoot: br,
StateRoot: sr,
}, nil
}
// this method downloads the head state, which can be used to find the correct chain config
// and use prysm's helper methods to compute the latest weak subjectivity epoch.
func getWeakSubjectivityEpochFromHead(ctx context.Context, client *Client) (primitives.Epoch, error) {
headBytes, err := client.GetState(ctx, IdHead)
if err != nil {
return 0, err
}
vu, err := detect.FromState(headBytes)
if err != nil {
return 0, errors.Wrap(err, "error detecting chain config for beacon state")
}
log.Printf("detected supported config in remote head state, name=%s, fork=%s", vu.Config.ConfigName, version.String(vu.Fork))
headState, err := vu.UnmarshalBeaconState(headBytes)
if err != nil {
return 0, errors.Wrap(err, "error unmarshaling state to correct version")
}
epoch, err := helpers.LatestWeakSubjectivityEpoch(ctx, headState, vu.Config)
if err != nil {
return 0, errors.Wrap(err, "error computing the weak subjectivity epoch from head state")
}
log.Printf("(computed client-side) weak subjectivity epoch = %d", epoch)
return epoch, nil
}

View File

@@ -0,0 +1,503 @@
package beacon
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
"testing"
"github.com/prysmaticlabs/prysm/v3/beacon-chain/state"
"github.com/prysmaticlabs/prysm/v3/consensus-types/blocks"
blocktest "github.com/prysmaticlabs/prysm/v3/consensus-types/blocks/testing"
"github.com/prysmaticlabs/prysm/v3/network/forks"
ethpb "github.com/prysmaticlabs/prysm/v3/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/v3/testing/util"
"github.com/prysmaticlabs/prysm/v3/time/slots"
"github.com/prysmaticlabs/prysm/v3/config/params"
"github.com/prysmaticlabs/prysm/v3/consensus-types/primitives"
"github.com/prysmaticlabs/prysm/v3/encoding/ssz/detect"
"github.com/prysmaticlabs/prysm/v3/runtime/version"
"github.com/pkg/errors"
"github.com/prysmaticlabs/prysm/v3/testing/require"
)
type testRT struct {
rt func(*http.Request) (*http.Response, error)
}
func (rt *testRT) RoundTrip(req *http.Request) (*http.Response, error) {
if rt.rt != nil {
return rt.rt(req)
}
return nil, errors.New("RoundTripper not implemented")
}
var _ http.RoundTripper = &testRT{}
func marshalToEnvelope(val interface{}) ([]byte, error) {
raw, err := json.Marshal(val)
if err != nil {
return nil, errors.Wrap(err, "error marshaling value to place in data envelope")
}
env := struct {
Data json.RawMessage `json:"data"`
}{
Data: raw,
}
return json.Marshal(env)
}
func TestMarshalToEnvelope(t *testing.T) {
d := struct {
Version string `json:"version"`
}{
Version: "Prysm/v2.0.5 (linux amd64)",
}
encoded, err := marshalToEnvelope(d)
require.NoError(t, err)
expected := `{"data":{"version":"Prysm/v2.0.5 (linux amd64)"}}`
require.Equal(t, expected, string(encoded))
}
func TestFallbackVersionCheck(t *testing.T) {
c := &Client{
hc: &http.Client{},
baseURL: &url.URL{Host: "localhost:3500", Scheme: "http"},
}
c.hc.Transport = &testRT{rt: func(req *http.Request) (*http.Response, error) {
res := &http.Response{Request: req}
switch req.URL.Path {
case getNodeVersionPath:
res.StatusCode = http.StatusOK
b := bytes.NewBuffer(nil)
d := struct {
Version string `json:"version"`
}{
Version: "Prysm/v2.0.5 (linux amd64)",
}
encoded, err := marshalToEnvelope(d)
require.NoError(t, err)
b.Write(encoded)
res.Body = io.NopCloser(b)
case getWeakSubjectivityPath:
res.StatusCode = http.StatusNotFound
}
return res, nil
}}
ctx := context.Background()
_, err := ComputeWeakSubjectivityCheckpoint(ctx, c)
require.ErrorIs(t, err, errUnsupportedPrysmCheckpointVersion)
}
func TestFname(t *testing.T) {
vu := &detect.VersionedUnmarshaler{
Config: params.MainnetConfig(),
Fork: version.Phase0,
}
slot := primitives.Slot(23)
prefix := "block"
var root [32]byte
copy(root[:], []byte{0x23, 0x23, 0x23})
expected := "block_mainnet_phase0_23-0x2323230000000000000000000000000000000000000000000000000000000000.ssz"
actual := fname(prefix, vu, slot, root)
require.Equal(t, expected, actual)
vu.Config = params.MinimalSpecConfig()
vu.Fork = version.Altair
slot = 17
prefix = "state"
copy(root[29:], []byte{0x17, 0x17, 0x17})
expected = "state_minimal_altair_17-0x2323230000000000000000000000000000000000000000000000000000171717.ssz"
actual = fname(prefix, vu, slot, root)
require.Equal(t, expected, actual)
}
func TestDownloadWeakSubjectivityCheckpoint(t *testing.T) {
ctx := context.Background()
cfg := params.MainnetConfig().Copy()
epoch := cfg.AltairForkEpoch - 1
// set up checkpoint state, using the epoch that will be computed as the ws checkpoint state based on the head state
wSlot, err := slots.EpochStart(epoch)
require.NoError(t, err)
wst, err := util.NewBeaconState()
require.NoError(t, err)
fork, err := forkForEpoch(cfg, epoch)
require.NoError(t, wst.SetFork(fork))
// set up checkpoint block
b, err := blocks.NewSignedBeaconBlock(util.NewBeaconBlock())
require.NoError(t, err)
b, err = blocktest.SetBlockParentRoot(b, cfg.ZeroHash)
require.NoError(t, err)
b, err = blocktest.SetBlockSlot(b, wSlot)
require.NoError(t, err)
b, err = blocktest.SetProposerIndex(b, 0)
require.NoError(t, err)
// set up state header pointing at checkpoint block - this is how the block is downloaded by root
header, err := b.Header()
require.NoError(t, err)
require.NoError(t, wst.SetLatestBlockHeader(header.Header))
// order of operations can be confusing here:
// - when computing the state root, make sure block header is complete, EXCEPT the state root should be zero-value
// - before computing the block root (to match the request route), the block should include the state root
// *computed from the state with a header that does not have a state root set yet*
wRoot, err := wst.HashTreeRoot(ctx)
require.NoError(t, err)
b, err = blocktest.SetBlockStateRoot(b, wRoot)
require.NoError(t, err)
serBlock, err := b.MarshalSSZ()
require.NoError(t, err)
bRoot, err := b.Block().HashTreeRoot()
require.NoError(t, err)
wsSerialized, err := wst.MarshalSSZ()
require.NoError(t, err)
expectedWSD := WeakSubjectivityData{
BlockRoot: bRoot,
StateRoot: wRoot,
Epoch: epoch,
}
hc := &http.Client{
Transport: &testRT{rt: func(req *http.Request) (*http.Response, error) {
res := &http.Response{Request: req}
switch req.URL.Path {
case getWeakSubjectivityPath:
res.StatusCode = http.StatusOK
cp := struct {
Epoch string `json:"epoch"`
Root string `json:"root"`
}{
Epoch: fmt.Sprintf("%d", slots.ToEpoch(b.Block().Slot())),
Root: fmt.Sprintf("%#x", bRoot),
}
wsr := struct {
Checkpoint interface{} `json:"ws_checkpoint"`
StateRoot string `json:"state_root"`
}{
Checkpoint: cp,
StateRoot: fmt.Sprintf("%#x", wRoot),
}
rb, err := marshalToEnvelope(wsr)
require.NoError(t, err)
res.Body = io.NopCloser(bytes.NewBuffer(rb))
case renderGetStatePath(IdFromSlot(wSlot)):
res.StatusCode = http.StatusOK
res.Body = io.NopCloser(bytes.NewBuffer(wsSerialized))
case renderGetBlockPath(IdFromRoot(bRoot)):
res.StatusCode = http.StatusOK
res.Body = io.NopCloser(bytes.NewBuffer(serBlock))
}
return res, nil
}},
}
c := &Client{
hc: hc,
baseURL: &url.URL{Host: "localhost:3500", Scheme: "http"},
}
wsd, err := ComputeWeakSubjectivityCheckpoint(ctx, c)
require.NoError(t, err)
require.Equal(t, expectedWSD.Epoch, wsd.Epoch)
require.Equal(t, expectedWSD.StateRoot, wsd.StateRoot)
require.Equal(t, expectedWSD.BlockRoot, wsd.BlockRoot)
}
// runs computeBackwardsCompatible directly
// and via ComputeWeakSubjectivityCheckpoint with a round tripper that triggers the backwards compatible code path
func TestDownloadBackwardsCompatibleCombined(t *testing.T) {
ctx := context.Background()
cfg := params.MainnetConfig()
st, expectedEpoch := defaultTestHeadState(t, cfg)
serialized, err := st.MarshalSSZ()
require.NoError(t, err)
// set up checkpoint state, using the epoch that will be computed as the ws checkpoint state based on the head state
wSlot, err := slots.EpochStart(expectedEpoch)
require.NoError(t, err)
wst, err := util.NewBeaconState()
require.NoError(t, err)
fork, err := forkForEpoch(cfg, cfg.GenesisEpoch)
require.NoError(t, wst.SetFork(fork))
// set up checkpoint block
b, err := blocks.NewSignedBeaconBlock(util.NewBeaconBlock())
require.NoError(t, err)
b, err = blocktest.SetBlockParentRoot(b, cfg.ZeroHash)
require.NoError(t, err)
b, err = blocktest.SetBlockSlot(b, wSlot)
require.NoError(t, err)
b, err = blocktest.SetProposerIndex(b, 0)
require.NoError(t, err)
// set up state header pointing at checkpoint block - this is how the block is downloaded by root
header, err := b.Header()
require.NoError(t, err)
require.NoError(t, wst.SetLatestBlockHeader(header.Header))
// order of operations can be confusing here:
// - when computing the state root, make sure block header is complete, EXCEPT the state root should be zero-value
// - before computing the block root (to match the request route), the block should include the state root
// *computed from the state with a header that does not have a state root set yet*
wRoot, err := wst.HashTreeRoot(ctx)
require.NoError(t, err)
b, err = blocktest.SetBlockStateRoot(b, wRoot)
require.NoError(t, err)
serBlock, err := b.MarshalSSZ()
require.NoError(t, err)
bRoot, err := b.Block().HashTreeRoot()
require.NoError(t, err)
wsSerialized, err := wst.MarshalSSZ()
require.NoError(t, err)
hc := &http.Client{
Transport: &testRT{rt: func(req *http.Request) (*http.Response, error) {
res := &http.Response{Request: req}
switch req.URL.Path {
case getNodeVersionPath:
res.StatusCode = http.StatusOK
b := bytes.NewBuffer(nil)
d := struct {
Version string `json:"version"`
}{
Version: "Lighthouse/v0.1.5 (Linux x86_64)",
}
encoded, err := marshalToEnvelope(d)
require.NoError(t, err)
b.Write(encoded)
res.Body = io.NopCloser(b)
case getWeakSubjectivityPath:
res.StatusCode = http.StatusNotFound
case renderGetStatePath(IdHead):
res.StatusCode = http.StatusOK
res.Body = io.NopCloser(bytes.NewBuffer(serialized))
case renderGetStatePath(IdFromSlot(wSlot)):
res.StatusCode = http.StatusOK
res.Body = io.NopCloser(bytes.NewBuffer(wsSerialized))
case renderGetBlockPath(IdFromRoot(bRoot)):
res.StatusCode = http.StatusOK
res.Body = io.NopCloser(bytes.NewBuffer(serBlock))
}
return res, nil
}},
}
c := &Client{
hc: hc,
baseURL: &url.URL{Host: "localhost:3500", Scheme: "http"},
}
wsPub, err := ComputeWeakSubjectivityCheckpoint(ctx, c)
require.NoError(t, err)
wsPriv, err := computeBackwardsCompatible(ctx, c)
require.NoError(t, err)
require.DeepEqual(t, wsPriv, wsPub)
}
func TestGetWeakSubjectivityEpochFromHead(t *testing.T) {
st, expectedEpoch := defaultTestHeadState(t, params.MainnetConfig())
serialized, err := st.MarshalSSZ()
require.NoError(t, err)
hc := &http.Client{
Transport: &testRT{rt: func(req *http.Request) (*http.Response, error) {
res := &http.Response{Request: req}
switch req.URL.Path {
case renderGetStatePath(IdHead):
res.StatusCode = http.StatusOK
res.Body = io.NopCloser(bytes.NewBuffer(serialized))
}
return res, nil
}},
}
c := &Client{
hc: hc,
baseURL: &url.URL{Host: "localhost:3500", Scheme: "http"},
}
actualEpoch, err := getWeakSubjectivityEpochFromHead(context.Background(), c)
require.NoError(t, err)
require.Equal(t, expectedEpoch, actualEpoch)
}
func forkForEpoch(cfg *params.BeaconChainConfig, epoch primitives.Epoch) (*ethpb.Fork, error) {
os := forks.NewOrderedSchedule(cfg)
currentVersion, err := os.VersionForEpoch(epoch)
if err != nil {
return nil, err
}
prevVersion, err := os.Previous(currentVersion)
if err != nil {
if !errors.Is(err, forks.ErrNoPreviousVersion) {
return nil, err
}
// use same version for both in the case of genesis
prevVersion = currentVersion
}
forkEpoch := cfg.ForkVersionSchedule[currentVersion]
return &ethpb.Fork{
PreviousVersion: prevVersion[:],
CurrentVersion: currentVersion[:],
Epoch: forkEpoch,
}, nil
}
func defaultTestHeadState(t *testing.T, cfg *params.BeaconChainConfig) (state.BeaconState, primitives.Epoch) {
st, err := util.NewBeaconStateAltair()
require.NoError(t, err)
fork, err := forkForEpoch(cfg, cfg.AltairForkEpoch)
require.NoError(t, err)
require.NoError(t, st.SetFork(fork))
slot, err := slots.EpochStart(cfg.AltairForkEpoch)
require.NoError(t, err)
require.NoError(t, st.SetSlot(slot))
var validatorCount, avgBalance uint64 = 100, 35
require.NoError(t, populateValidators(cfg, st, validatorCount, avgBalance))
require.NoError(t, st.SetFinalizedCheckpoint(&ethpb.Checkpoint{
Epoch: fork.Epoch - 10,
Root: make([]byte, 32),
}))
// to see the math for this, look at helpers.LatestWeakSubjectivityEpoch
// and for the values use mainnet config values, the validatorCount and avgBalance above, and altair fork epoch
expectedEpoch := slots.ToEpoch(st.Slot()) - 224
return st, expectedEpoch
}
// TODO(10429): refactor beacon state options in testing/util to take a state.BeaconState so this can become an option
func populateValidators(cfg *params.BeaconChainConfig, st state.BeaconState, valCount, avgBalance uint64) error {
validators := make([]*ethpb.Validator, valCount)
balances := make([]uint64, len(validators))
for i := uint64(0); i < valCount; i++ {
validators[i] = &ethpb.Validator{
PublicKey: make([]byte, cfg.BLSPubkeyLength),
WithdrawalCredentials: make([]byte, 32),
EffectiveBalance: avgBalance * 1e9,
ExitEpoch: cfg.FarFutureEpoch,
}
balances[i] = validators[i].EffectiveBalance
}
if err := st.SetValidators(validators); err != nil {
return err
}
if err := st.SetBalances(balances); err != nil {
return err
}
return nil
}
func TestDownloadFinalizedData(t *testing.T) {
ctx := context.Background()
cfg := params.MainnetConfig().Copy()
// avoid the altair zone because genesis tests are easier to set up
epoch := cfg.AltairForkEpoch - 1
// set up checkpoint state, using the epoch that will be computed as the ws checkpoint state based on the head state
slot, err := slots.EpochStart(epoch)
require.NoError(t, err)
st, err := util.NewBeaconState()
require.NoError(t, err)
fork, err := forkForEpoch(cfg, epoch)
require.NoError(t, st.SetFork(fork))
require.NoError(t, st.SetSlot(slot))
// set up checkpoint block
b, err := blocks.NewSignedBeaconBlock(util.NewBeaconBlock())
require.NoError(t, err)
b, err = blocktest.SetBlockParentRoot(b, cfg.ZeroHash)
require.NoError(t, err)
b, err = blocktest.SetBlockSlot(b, slot)
require.NoError(t, err)
b, err = blocktest.SetProposerIndex(b, 0)
require.NoError(t, err)
// set up state header pointing at checkpoint block - this is how the block is downloaded by root
header, err := b.Header()
require.NoError(t, err)
require.NoError(t, st.SetLatestBlockHeader(header.Header))
// order of operations can be confusing here:
// - when computing the state root, make sure block header is complete, EXCEPT the state root should be zero-value
// - before computing the block root (to match the request route), the block should include the state root
// *computed from the state with a header that does not have a state root set yet*
sr, err := st.HashTreeRoot(ctx)
require.NoError(t, err)
b, err = blocktest.SetBlockStateRoot(b, sr)
require.NoError(t, err)
mb, err := b.MarshalSSZ()
require.NoError(t, err)
br, err := b.Block().HashTreeRoot()
require.NoError(t, err)
ms, err := st.MarshalSSZ()
require.NoError(t, err)
hc := &http.Client{
Transport: &testRT{rt: func(req *http.Request) (*http.Response, error) {
res := &http.Response{Request: req}
switch req.URL.Path {
case renderGetStatePath(IdFinalized):
res.StatusCode = http.StatusOK
res.Body = io.NopCloser(bytes.NewBuffer(ms))
case renderGetBlockPath(IdFromRoot(br)):
res.StatusCode = http.StatusOK
res.Body = io.NopCloser(bytes.NewBuffer(mb))
default:
res.StatusCode = http.StatusInternalServerError
res.Body = io.NopCloser(bytes.NewBufferString(""))
}
return res, nil
}},
}
c := &Client{
hc: hc,
baseURL: &url.URL{Host: "localhost:3500", Scheme: "http"},
}
// sanity check before we go through checkpoint
// make sure we can download the state and unmarshal it with the VersionedUnmarshaler
sb, err := c.GetState(ctx, IdFinalized)
require.NoError(t, err)
require.Equal(t, true, bytes.Equal(sb, ms))
vu, err := detect.FromState(sb)
require.NoError(t, err)
us, err := vu.UnmarshalBeaconState(sb)
require.NoError(t, err)
ushtr, err := us.HashTreeRoot(ctx)
require.NoError(t, err)
require.Equal(t, sr, ushtr)
expected := &OriginData{
sb: ms,
bb: mb,
br: br,
sr: sr,
}
od, err := DownloadFinalizedData(ctx, c)
require.NoError(t, err)
require.Equal(t, true, bytes.Equal(expected.sb, od.sb))
require.Equal(t, true, bytes.Equal(expected.bb, od.bb))
require.Equal(t, expected.br, od.br)
require.Equal(t, expected.sr, od.sr)
}

475
api/client/beacon/client.go Normal file
View File

@@ -0,0 +1,475 @@
package beacon
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net"
"net/http"
"net/url"
"path"
"regexp"
"sort"
"strconv"
"text/template"
"time"
"github.com/prysmaticlabs/prysm/v3/network/forks"
"github.com/ethereum/go-ethereum/common/hexutil"
"github.com/pkg/errors"
"github.com/prysmaticlabs/prysm/v3/beacon-chain/rpc/apimiddleware"
"github.com/prysmaticlabs/prysm/v3/consensus-types/primitives"
"github.com/prysmaticlabs/prysm/v3/encoding/bytesutil"
ethpb "github.com/prysmaticlabs/prysm/v3/proto/prysm/v1alpha1"
log "github.com/sirupsen/logrus"
)
const (
getSignedBlockPath = "/eth/v2/beacon/blocks"
getBlockRootPath = "/eth/v1/beacon/blocks/{{.Id}}/root"
getForkForStatePath = "/eth/v1/beacon/states/{{.Id}}/fork"
getWeakSubjectivityPath = "/eth/v1/beacon/weak_subjectivity"
getForkSchedulePath = "/eth/v1/config/fork_schedule"
getStatePath = "/eth/v2/debug/beacon/states"
getNodeVersionPath = "/eth/v1/node/version"
changeBLStoExecutionPath = "/eth/v1/beacon/pool/bls_to_execution_changes"
)
// StateOrBlockId represents the block_id / state_id parameters that several of the Eth Beacon API methods accept.
// StateOrBlockId constants are defined for named identifiers, and helper methods are provided
// for slot and root identifiers. Example text from the Eth Beacon Node API documentation:
//
// "Block identifier can be one of: "head" (canonical head in node's view), "genesis", "finalized",
// <slot>, <hex encoded blockRoot with 0x prefix>."
type StateOrBlockId string
const (
IdGenesis StateOrBlockId = "genesis"
IdHead StateOrBlockId = "head"
IdFinalized StateOrBlockId = "finalized"
)
var ErrMalformedHostname = errors.New("hostname must include port, separated by one colon, like example.com:3500")
// IdFromRoot encodes a block root in the format expected by the API in places where a root can be used to identify
// a BeaconState or SignedBeaconBlock.
func IdFromRoot(r [32]byte) StateOrBlockId {
return StateOrBlockId(fmt.Sprintf("%#x", r))
}
// IdFromSlot encodes a Slot in the format expected by the API in places where a slot can be used to identify
// a BeaconState or SignedBeaconBlock.
func IdFromSlot(s primitives.Slot) StateOrBlockId {
return StateOrBlockId(strconv.FormatUint(uint64(s), 10))
}
// idTemplate is used to create template functions that can interpolate StateOrBlockId values.
func idTemplate(ts string) func(StateOrBlockId) string {
t := template.Must(template.New("").Parse(ts))
f := func(id StateOrBlockId) string {
b := bytes.NewBuffer(nil)
err := t.Execute(b, struct{ Id string }{Id: string(id)})
if err != nil {
panic(fmt.Sprintf("invalid idTemplate: %s", ts))
}
return b.String()
}
// run the template to ensure that it is valid
// this should happen load time (using package scoped vars) to ensure runtime errors aren't possible
_ = f(IdGenesis)
return f
}
// ClientOpt is a functional option for the Client type (http.Client wrapper)
type ClientOpt func(*Client)
// WithTimeout sets the .Timeout attribute of the wrapped http.Client.
func WithTimeout(timeout time.Duration) ClientOpt {
return func(c *Client) {
c.hc.Timeout = timeout
}
}
// Client provides a collection of helper methods for calling the Eth Beacon Node API endpoints.
type Client struct {
hc *http.Client
baseURL *url.URL
}
// NewClient constructs a new client with the provided options (ex WithTimeout).
// `host` is the base host + port used to construct request urls. This value can be
// a URL string, or NewClient will assume an http endpoint if just `host:port` is used.
func NewClient(host string, opts ...ClientOpt) (*Client, error) {
u, err := urlForHost(host)
if err != nil {
return nil, err
}
c := &Client{
hc: &http.Client{},
baseURL: u,
}
for _, o := range opts {
o(c)
}
return c, nil
}
func urlForHost(h string) (*url.URL, error) {
// try to parse as url (being permissive)
u, err := url.Parse(h)
if err == nil && u.Host != "" {
return u, nil
}
// try to parse as host:port
host, port, err := net.SplitHostPort(h)
if err != nil {
return nil, ErrMalformedHostname
}
return &url.URL{Host: fmt.Sprintf("%s:%s", host, port), Scheme: "http"}, nil
}
// NodeURL returns a human-readable string representation of the beacon node base url.
func (c *Client) NodeURL() string {
return c.baseURL.String()
}
type reqOption func(*http.Request)
func withSSZEncoding() reqOption {
return func(req *http.Request) {
req.Header.Set("Accept", "application/octet-stream")
}
}
// get is a generic, opinionated GET function to reduce boilerplate amongst the getters in this package.
func (c *Client) get(ctx context.Context, path string, opts ...reqOption) ([]byte, error) {
u := c.baseURL.ResolveReference(&url.URL{Path: path})
req, err := http.NewRequestWithContext(ctx, http.MethodGet, u.String(), nil)
if err != nil {
return nil, err
}
for _, o := range opts {
o(req)
}
r, err := c.hc.Do(req)
if err != nil {
return nil, err
}
defer func() {
err = r.Body.Close()
}()
if r.StatusCode != http.StatusOK {
return nil, non200Err(r)
}
b, err := io.ReadAll(r.Body)
if err != nil {
return nil, errors.Wrap(err, "error reading http response body from GetBlock")
}
return b, nil
}
func renderGetBlockPath(id StateOrBlockId) string {
return path.Join(getSignedBlockPath, string(id))
}
// GetBlock retrieves the SignedBeaconBlock for the given block id.
// Block identifier can be one of: "head" (canonical head in node's view), "genesis", "finalized",
// <slot>, <hex encoded blockRoot with 0x prefix>. Variables of type StateOrBlockId are exported by this package
// for the named identifiers.
// The return value contains the ssz-encoded bytes.
func (c *Client) GetBlock(ctx context.Context, blockId StateOrBlockId) ([]byte, error) {
blockPath := renderGetBlockPath(blockId)
b, err := c.get(ctx, blockPath, withSSZEncoding())
if err != nil {
return nil, errors.Wrapf(err, "error requesting state by id = %s", blockId)
}
return b, nil
}
var getBlockRootTpl = idTemplate(getBlockRootPath)
// GetBlockRoot retrieves the hash_tree_root of the BeaconBlock for the given block id.
// Block identifier can be one of: "head" (canonical head in node's view), "genesis", "finalized",
// <slot>, <hex encoded blockRoot with 0x prefix>. Variables of type StateOrBlockId are exported by this package
// for the named identifiers.
func (c *Client) GetBlockRoot(ctx context.Context, blockId StateOrBlockId) ([32]byte, error) {
rootPath := getBlockRootTpl(blockId)
b, err := c.get(ctx, rootPath)
if err != nil {
return [32]byte{}, errors.Wrapf(err, "error requesting block root by id = %s", blockId)
}
jsonr := &struct{ Data struct{ Root string } }{}
err = json.Unmarshal(b, jsonr)
if err != nil {
return [32]byte{}, errors.Wrap(err, "error decoding json data from get block root response")
}
rs, err := hexutil.Decode(jsonr.Data.Root)
if err != nil {
return [32]byte{}, errors.Wrap(err, fmt.Sprintf("error decoding hex-encoded value %s", jsonr.Data.Root))
}
return bytesutil.ToBytes32(rs), nil
}
var getForkTpl = idTemplate(getForkForStatePath)
// GetFork queries the Beacon Node API for the Fork from the state identified by stateId.
// Block identifier can be one of: "head" (canonical head in node's view), "genesis", "finalized",
// <slot>, <hex encoded blockRoot with 0x prefix>. Variables of type StateOrBlockId are exported by this package
// for the named identifiers.
func (c *Client) GetFork(ctx context.Context, stateId StateOrBlockId) (*ethpb.Fork, error) {
body, err := c.get(ctx, getForkTpl(stateId))
if err != nil {
return nil, errors.Wrapf(err, "error requesting fork by state id = %s", stateId)
}
fr := &forkResponse{}
dataWrapper := &struct{ Data *forkResponse }{Data: fr}
err = json.Unmarshal(body, dataWrapper)
if err != nil {
return nil, errors.Wrap(err, "error decoding json response in GetFork")
}
return fr.Fork()
}
// GetForkSchedule retrieve all forks, past present and future, of which this node is aware.
func (c *Client) GetForkSchedule(ctx context.Context) (forks.OrderedSchedule, error) {
body, err := c.get(ctx, getForkSchedulePath)
if err != nil {
return nil, errors.Wrap(err, "error requesting fork schedule")
}
fsr := &forkScheduleResponse{}
err = json.Unmarshal(body, fsr)
if err != nil {
return nil, err
}
ofs, err := fsr.OrderedForkSchedule()
if err != nil {
return nil, errors.Wrap(err, fmt.Sprintf("problem unmarshaling %s response", getForkSchedulePath))
}
return ofs, nil
}
type NodeVersion struct {
implementation string
semver string
systemInfo string
}
var versionRE = regexp.MustCompile(`^(\w+)/(v\d+\.\d+\.\d+[-a-zA-Z0-9]*)\s*/?(.*)$`)
func parseNodeVersion(v string) (*NodeVersion, error) {
groups := versionRE.FindStringSubmatch(v)
if len(groups) != 4 {
return nil, errors.Wrapf(ErrInvalidNodeVersion, "could not be parsed: %s", v)
}
return &NodeVersion{
implementation: groups[1],
semver: groups[2],
systemInfo: groups[3],
}, nil
}
// GetNodeVersion requests that the beacon node identify information about its implementation in a format
// similar to a HTTP User-Agent field. ex: Lighthouse/v0.1.5 (Linux x86_64)
func (c *Client) GetNodeVersion(ctx context.Context) (*NodeVersion, error) {
b, err := c.get(ctx, getNodeVersionPath)
if err != nil {
return nil, errors.Wrap(err, "error requesting node version")
}
d := struct {
Data struct {
Version string `json:"version"`
} `json:"data"`
}{}
err = json.Unmarshal(b, &d)
if err != nil {
return nil, errors.Wrapf(err, "error unmarshaling response body: %s", string(b))
}
return parseNodeVersion(d.Data.Version)
}
func renderGetStatePath(id StateOrBlockId) string {
return path.Join(getStatePath, string(id))
}
// GetState retrieves the BeaconState for the given state id.
// State identifier can be one of: "head" (canonical head in node's view), "genesis", "finalized",
// <slot>, <hex encoded stateRoot with 0x prefix>. Variables of type StateOrBlockId are exported by this package
// for the named identifiers.
// The return value contains the ssz-encoded bytes.
func (c *Client) GetState(ctx context.Context, stateId StateOrBlockId) ([]byte, error) {
statePath := path.Join(getStatePath, string(stateId))
b, err := c.get(ctx, statePath, withSSZEncoding())
if err != nil {
return nil, errors.Wrapf(err, "error requesting state by id = %s", stateId)
}
return b, nil
}
// GetWeakSubjectivity calls a proposed API endpoint that is unique to prysm
// This api method does the following:
// - computes weak subjectivity epoch
// - finds the highest non-skipped block preceding the epoch
// - returns the htr of the found block and returns this + the value of state_root from the block
func (c *Client) GetWeakSubjectivity(ctx context.Context) (*WeakSubjectivityData, error) {
body, err := c.get(ctx, getWeakSubjectivityPath)
if err != nil {
return nil, err
}
v := &apimiddleware.WeakSubjectivityResponse{}
err = json.Unmarshal(body, v)
if err != nil {
return nil, err
}
epoch, err := strconv.ParseUint(v.Data.Checkpoint.Epoch, 10, 64)
if err != nil {
return nil, err
}
blockRoot, err := hexutil.Decode(v.Data.Checkpoint.Root)
if err != nil {
return nil, err
}
stateRoot, err := hexutil.Decode(v.Data.StateRoot)
if err != nil {
return nil, err
}
return &WeakSubjectivityData{
Epoch: primitives.Epoch(epoch),
BlockRoot: bytesutil.ToBytes32(blockRoot),
StateRoot: bytesutil.ToBytes32(stateRoot),
}, nil
}
// SubmitChangeBLStoExecution calls a beacon API endpoint to set the withdrawal addresses based on the given signed messages.
// If the API responds with something other than OK there will be failure messages associated to the corresponding request message.
func (c *Client) SubmitChangeBLStoExecution(ctx context.Context, request []*apimiddleware.SignedBLSToExecutionChangeJson) error {
u := c.baseURL.ResolveReference(&url.URL{Path: changeBLStoExecutionPath})
body, err := json.Marshal(request)
if err != nil {
return errors.Wrap(err, "failed to marshal JSON")
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, u.String(), bytes.NewBuffer(body))
if err != nil {
return errors.Wrap(err, "invalid format, failed to create new POST request object")
}
req.Header.Set("Content-Type", "application/json")
resp, err := c.hc.Do(req)
if err != nil {
return err
}
defer func() {
err = resp.Body.Close()
}()
if resp.StatusCode != http.StatusOK {
decoder := json.NewDecoder(resp.Body)
decoder.DisallowUnknownFields()
errorJson := &apimiddleware.IndexedVerificationFailureErrorJson{}
if err := decoder.Decode(errorJson); err != nil {
return errors.Wrapf(err, "failed to decode error JSON for %s", resp.Request.URL)
}
for _, failure := range errorJson.Failures {
w := request[failure.Index].Message
log.WithFields(log.Fields{
"validator_index": w.ValidatorIndex,
"withdrawal_address": w.ToExecutionAddress,
}).Error(failure.Message)
}
return errors.Errorf("POST error %d: %s", errorJson.Code, errorJson.Message)
}
return nil
}
// GetBLStoExecutionChanges gets all the set withdrawal messages in the node's operation pool.
// Returns a struct representation of json response.
func (c *Client) GetBLStoExecutionChanges(ctx context.Context) (*apimiddleware.BLSToExecutionChangesPoolResponseJson, error) {
body, err := c.get(ctx, changeBLStoExecutionPath)
if err != nil {
return nil, err
}
poolResponse := &apimiddleware.BLSToExecutionChangesPoolResponseJson{}
err = json.Unmarshal(body, poolResponse)
if err != nil {
return nil, err
}
return poolResponse, nil
}
func non200Err(response *http.Response) error {
bodyBytes, err := io.ReadAll(response.Body)
var body string
if err != nil {
body = "(Unable to read response body.)"
} else {
body = "response body:\n" + string(bodyBytes)
}
msg := fmt.Sprintf("code=%d, url=%s, body=%s", response.StatusCode, response.Request.URL, body)
switch response.StatusCode {
case 404:
return errors.Wrap(ErrNotFound, msg)
default:
return errors.Wrap(ErrNotOK, msg)
}
}
type forkResponse struct {
PreviousVersion string `json:"previous_version"`
CurrentVersion string `json:"current_version"`
Epoch string `json:"epoch"`
}
func (f *forkResponse) Fork() (*ethpb.Fork, error) {
epoch, err := strconv.ParseUint(f.Epoch, 10, 64)
if err != nil {
return nil, err
}
cSlice, err := hexutil.Decode(f.CurrentVersion)
if err != nil {
return nil, err
}
if len(cSlice) != 4 {
return nil, fmt.Errorf("got %d byte version for CurrentVersion, expected 4 bytes. hex=%s", len(cSlice), f.CurrentVersion)
}
pSlice, err := hexutil.Decode(f.PreviousVersion)
if err != nil {
return nil, err
}
if len(pSlice) != 4 {
return nil, fmt.Errorf("got %d byte version, expected 4 bytes. version hex=%s", len(pSlice), f.PreviousVersion)
}
return &ethpb.Fork{
CurrentVersion: cSlice,
PreviousVersion: pSlice,
Epoch: primitives.Epoch(epoch),
}, nil
}
type forkScheduleResponse struct {
Data []forkResponse
}
func (fsr *forkScheduleResponse) OrderedForkSchedule() (forks.OrderedSchedule, error) {
ofs := make(forks.OrderedSchedule, 0)
for _, d := range fsr.Data {
epoch, err := strconv.Atoi(d.Epoch)
if err != nil {
return nil, err
}
vSlice, err := hexutil.Decode(d.CurrentVersion)
if err != nil {
return nil, err
}
if len(vSlice) != 4 {
return nil, fmt.Errorf("got %d byte version, expected 4 bytes. version hex=%s", len(vSlice), d.CurrentVersion)
}
version := bytesutil.ToBytes4(vSlice)
ofs = append(ofs, forks.ForkScheduleEntry{
Version: version,
Epoch: primitives.Epoch(uint64(epoch)),
})
}
sort.Sort(ofs)
return ofs, nil
}

View File

@@ -0,0 +1,138 @@
package beacon
import (
"net/url"
"testing"
"github.com/prysmaticlabs/prysm/v3/testing/require"
)
func TestParseNodeVersion(t *testing.T) {
cases := []struct {
name string
v string
err error
nv *NodeVersion
}{
{
name: "empty string",
v: "",
err: ErrInvalidNodeVersion,
},
{
name: "Prysm as the version string",
v: "Prysm",
err: ErrInvalidNodeVersion,
},
{
name: "semver only",
v: "v2.0.6",
err: ErrInvalidNodeVersion,
},
{
name: "complete version",
v: "Prysm/v2.0.6 (linux amd64)",
nv: &NodeVersion{
implementation: "Prysm",
semver: "v2.0.6",
systemInfo: "(linux amd64)",
},
},
{
name: "nimbus version",
v: "Nimbus/v22.4.0-039bec-stateofus",
nv: &NodeVersion{
implementation: "Nimbus",
semver: "v22.4.0-039bec-stateofus",
systemInfo: "",
},
},
{
name: "teku version",
v: "teku/v22.3.2/linux-x86_64/oracle-java-11",
nv: &NodeVersion{
implementation: "teku",
semver: "v22.3.2",
systemInfo: "linux-x86_64/oracle-java-11",
},
},
{
name: "lighthouse version",
v: "Lighthouse/v2.1.1-5f628a7/x86_64-linux",
nv: &NodeVersion{
implementation: "Lighthouse",
semver: "v2.1.1-5f628a7",
systemInfo: "x86_64-linux",
},
},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
nv, err := parseNodeVersion(c.v)
if c.err != nil {
require.ErrorIs(t, err, c.err)
} else {
require.NoError(t, err)
require.DeepEqual(t, c.nv, nv)
}
})
}
}
func TestValidHostname(t *testing.T) {
cases := []struct {
name string
hostArg string
path string
joined string
err error
}{
{
name: "hostname without port",
hostArg: "mydomain.org",
err: ErrMalformedHostname,
},
{
name: "hostname with port",
hostArg: "mydomain.org:3500",
path: getNodeVersionPath,
joined: "http://mydomain.org:3500/eth/v1/node/version",
},
{
name: "https scheme, hostname with port",
hostArg: "https://mydomain.org:3500",
path: getNodeVersionPath,
joined: "https://mydomain.org:3500/eth/v1/node/version",
},
{
name: "http scheme, hostname without port",
hostArg: "http://mydomain.org",
path: getNodeVersionPath,
joined: "http://mydomain.org/eth/v1/node/version",
},
{
name: "http scheme, trailing slash, hostname without port",
hostArg: "http://mydomain.org/",
path: getNodeVersionPath,
joined: "http://mydomain.org/eth/v1/node/version",
},
{
name: "http scheme, hostname with basic auth creds and no port",
hostArg: "http://username:pass@mydomain.org/",
path: getNodeVersionPath,
joined: "http://username:pass@mydomain.org/eth/v1/node/version",
},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
cl, err := NewClient(c.hostArg)
if c.err != nil {
require.ErrorIs(t, err, c.err)
return
}
require.NoError(t, err)
require.Equal(t, c.joined, cl.baseURL.ResolveReference(&url.URL{Path: c.path}).String())
})
}
}

5
api/client/beacon/doc.go Normal file
View File

@@ -0,0 +1,5 @@
/*
Package beacon provides a client for interacting with the standard Eth Beacon Node API.
Interactive swagger documentation for the API is available here: https://ethereum.github.io/beacon-APIs/
*/
package beacon

View File

@@ -0,0 +1,13 @@
package beacon
import "github.com/pkg/errors"
// ErrNotOK is used to indicate when an HTTP request to the Beacon Node API failed with any non-2xx response code.
// More specific errors may be returned, but an error in reaction to a non-2xx response will always wrap ErrNotOK.
var ErrNotOK = errors.New("did not receive 2xx response from API")
// ErrNotFound specifically means that a '404 - NOT FOUND' response was received from the API.
var ErrNotFound = errors.Wrap(ErrNotOK, "recv 404 NotFound response from API")
// ErrInvalidNodeVersion indicates that the /eth/v1/node/version api response format was not recognized.
var ErrInvalidNodeVersion = errors.New("invalid node version response")

View File

@@ -0,0 +1,53 @@
load("@prysm//tools/go:def.bzl", "go_library", "go_test")
go_library(
name = "go_default_library",
srcs = [
"bid.go",
"client.go",
"errors.go",
"types.go",
],
importpath = "github.com/prysmaticlabs/prysm/v3/api/client/builder",
visibility = ["//visibility:public"],
deps = [
"//consensus-types/blocks:go_default_library",
"//consensus-types/interfaces:go_default_library",
"//consensus-types/primitives:go_default_library",
"//encoding/bytesutil:go_default_library",
"//monitoring/tracing:go_default_library",
"//network:go_default_library",
"//network/authorization:go_default_library",
"//proto/engine/v1:go_default_library",
"//proto/prysm/v1alpha1:go_default_library",
"//runtime/version:go_default_library",
"@com_github_ethereum_go_ethereum//common/hexutil:go_default_library",
"@com_github_pkg_errors//:go_default_library",
"@com_github_prysmaticlabs_fastssz//:go_default_library",
"@com_github_sirupsen_logrus//:go_default_library",
"@io_opencensus_go//trace:go_default_library",
],
)
go_test(
name = "go_default_test",
srcs = [
"client_test.go",
"types_test.go",
],
data = glob(["testdata/**"]),
embed = [":go_default_library"],
deps = [
"//config/params:go_default_library",
"//consensus-types/blocks:go_default_library",
"//consensus-types/primitives:go_default_library",
"//encoding/bytesutil:go_default_library",
"//proto/engine/v1:go_default_library",
"//proto/prysm/v1alpha1:go_default_library",
"//testing/assert:go_default_library",
"//testing/require:go_default_library",
"@com_github_ethereum_go_ethereum//common/hexutil:go_default_library",
"@com_github_golang_protobuf//proto:go_default_library",
"@com_github_prysmaticlabs_go_bitfield//:go_default_library",
],
)

190
api/client/builder/bid.go Normal file
View File

@@ -0,0 +1,190 @@
package builder
import (
ssz "github.com/prysmaticlabs/fastssz"
"github.com/prysmaticlabs/prysm/v3/consensus-types/blocks"
"github.com/prysmaticlabs/prysm/v3/consensus-types/interfaces"
ethpb "github.com/prysmaticlabs/prysm/v3/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/v3/runtime/version"
)
// SignedBid is an interface describing the method set of a signed builder bid.
type SignedBid interface {
Message() (Bid, error)
Signature() []byte
Version() int
IsNil() bool
}
// Bid is an interface describing the method set of a builder bid.
type Bid interface {
Header() (interfaces.ExecutionData, error)
Value() []byte
Pubkey() []byte
Version() int
IsNil() bool
HashTreeRoot() ([32]byte, error)
HashTreeRootWith(hh *ssz.Hasher) error
}
type signedBuilderBid struct {
p *ethpb.SignedBuilderBid
}
// WrappedSignedBuilderBid is a constructor which wraps a protobuf signed bit into an interface.
func WrappedSignedBuilderBid(p *ethpb.SignedBuilderBid) (SignedBid, error) {
w := signedBuilderBid{p: p}
if w.IsNil() {
return nil, blocks.ErrNilObjectWrapped
}
return w, nil
}
// Message --
func (b signedBuilderBid) Message() (Bid, error) {
return WrappedBuilderBid(b.p.Message)
}
// Signature --
func (b signedBuilderBid) Signature() []byte {
return b.p.Signature
}
// Version --
func (b signedBuilderBid) Version() int {
return version.Bellatrix
}
// IsNil --
func (b signedBuilderBid) IsNil() bool {
return b.p == nil
}
type signedBuilderBidCapella struct {
p *ethpb.SignedBuilderBidCapella
}
// WrappedSignedBuilderBidCapella is a constructor which wraps a protobuf signed bit into an interface.
func WrappedSignedBuilderBidCapella(p *ethpb.SignedBuilderBidCapella) (SignedBid, error) {
w := signedBuilderBidCapella{p: p}
if w.IsNil() {
return nil, blocks.ErrNilObjectWrapped
}
return w, nil
}
// Message --
func (b signedBuilderBidCapella) Message() (Bid, error) {
return WrappedBuilderBidCapella(b.p.Message)
}
// Signature --
func (b signedBuilderBidCapella) Signature() []byte {
return b.p.Signature
}
// Version --
func (b signedBuilderBidCapella) Version() int {
return version.Capella
}
// IsNil --
func (b signedBuilderBidCapella) IsNil() bool {
return b.p == nil
}
type builderBid struct {
p *ethpb.BuilderBid
}
// WrappedBuilderBid is a constructor which wraps a protobuf bid into an interface.
func WrappedBuilderBid(p *ethpb.BuilderBid) (Bid, error) {
w := builderBid{p: p}
if w.IsNil() {
return nil, blocks.ErrNilObjectWrapped
}
return w, nil
}
// Header --
func (b builderBid) Header() (interfaces.ExecutionData, error) {
return blocks.WrappedExecutionPayloadHeader(b.p.Header)
}
// Version --
func (b builderBid) Version() int {
return version.Bellatrix
}
// Value --
func (b builderBid) Value() []byte {
return b.p.Value
}
// Pubkey --
func (b builderBid) Pubkey() []byte {
return b.p.Pubkey
}
// IsNil --
func (b builderBid) IsNil() bool {
return b.p == nil
}
// HashTreeRoot --
func (b builderBid) HashTreeRoot() ([32]byte, error) {
return b.p.HashTreeRoot()
}
// HashTreeRootWith --
func (b builderBid) HashTreeRootWith(hh *ssz.Hasher) error {
return b.p.HashTreeRootWith(hh)
}
type builderBidCapella struct {
p *ethpb.BuilderBidCapella
}
// WrappedBuilderBidCapella is a constructor which wraps a protobuf bid into an interface.
func WrappedBuilderBidCapella(p *ethpb.BuilderBidCapella) (Bid, error) {
w := builderBidCapella{p: p}
if w.IsNil() {
return nil, blocks.ErrNilObjectWrapped
}
return w, nil
}
// Header --
func (b builderBidCapella) Header() (interfaces.ExecutionData, error) {
return blocks.WrappedExecutionPayloadHeaderCapella(b.p.Header)
}
// Version --
func (b builderBidCapella) Version() int {
return version.Capella
}
// Value --
func (b builderBidCapella) Value() []byte {
return b.p.Value
}
// Pubkey --
func (b builderBidCapella) Pubkey() []byte {
return b.p.Pubkey
}
// IsNil --
func (b builderBidCapella) IsNil() bool {
return b.p == nil
}
// HashTreeRoot --
func (b builderBidCapella) HashTreeRoot() ([32]byte, error) {
return b.p.HashTreeRoot()
}
// HashTreeRootWith --
func (b builderBidCapella) HashTreeRootWith(hh *ssz.Hasher) error {
return b.p.HashTreeRootWith(hh)
}

View File

@@ -0,0 +1,420 @@
package builder
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net"
"net/http"
"net/url"
"strings"
"text/template"
"time"
"github.com/pkg/errors"
"github.com/prysmaticlabs/prysm/v3/consensus-types/blocks"
"github.com/prysmaticlabs/prysm/v3/consensus-types/interfaces"
"github.com/prysmaticlabs/prysm/v3/consensus-types/primitives"
"github.com/prysmaticlabs/prysm/v3/monitoring/tracing"
"github.com/prysmaticlabs/prysm/v3/network"
"github.com/prysmaticlabs/prysm/v3/network/authorization"
v1 "github.com/prysmaticlabs/prysm/v3/proto/engine/v1"
ethpb "github.com/prysmaticlabs/prysm/v3/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/v3/runtime/version"
log "github.com/sirupsen/logrus"
"go.opencensus.io/trace"
)
const (
getExecHeaderPath = "/eth/v1/builder/header/{{.Slot}}/{{.ParentHash}}/{{.Pubkey}}"
getStatus = "/eth/v1/builder/status"
postBlindedBeaconBlockPath = "/eth/v1/builder/blinded_blocks"
postRegisterValidatorPath = "/eth/v1/builder/validators"
)
var errMalformedHostname = errors.New("hostname must include port, separated by one colon, like example.com:3500")
var errMalformedRequest = errors.New("required request data are missing")
var errNotBlinded = errors.New("submitted block is not blinded")
var submitBlindedBlockTimeout = 3 * time.Second
// ClientOpt is a functional option for the Client type (http.Client wrapper)
type ClientOpt func(*Client)
type observer interface {
observe(r *http.Request) error
}
func WithObserver(m observer) ClientOpt {
return func(c *Client) {
c.obvs = append(c.obvs, m)
}
}
type requestLogger struct{}
func (*requestLogger) observe(r *http.Request) (e error) {
b := bytes.NewBuffer(nil)
if r.Body == nil {
log.WithFields(log.Fields{
"body-base64": "(nil value)",
"url": r.URL.String(),
}).Info("builder http request")
return nil
}
t := io.TeeReader(r.Body, b)
defer func() {
if r.Body != nil {
e = r.Body.Close()
}
}()
body, err := io.ReadAll(t)
if err != nil {
return err
}
r.Body = io.NopCloser(b)
log.WithFields(log.Fields{
"body-base64": string(body),
"url": r.URL.String(),
}).Info("builder http request")
return nil
}
var _ observer = &requestLogger{}
// BuilderClient provides a collection of helper methods for calling Builder API endpoints.
type BuilderClient interface {
NodeURL() string
GetHeader(ctx context.Context, slot primitives.Slot, parentHash [32]byte, pubkey [48]byte) (SignedBid, error)
RegisterValidator(ctx context.Context, svr []*ethpb.SignedValidatorRegistrationV1) error
SubmitBlindedBlock(ctx context.Context, sb interfaces.SignedBeaconBlock) (interfaces.ExecutionData, error)
Status(ctx context.Context) error
}
// Client provides a collection of helper methods for calling Builder API endpoints.
type Client struct {
hc *http.Client
baseURL *url.URL
obvs []observer
}
// NewClient constructs a new client with the provided options (ex WithTimeout).
// `host` is the base host + port used to construct request urls. This value can be
// a URL string, or NewClient will assume an http endpoint if just `host:port` is used.
func NewClient(host string, opts ...ClientOpt) (*Client, error) {
endpoint := covertEndPoint(host)
u, err := urlForHost(endpoint.Url)
if err != nil {
return nil, err
}
c := &Client{
hc: &http.Client{},
baseURL: u,
}
for _, o := range opts {
o(c)
}
return c, nil
}
func urlForHost(h string) (*url.URL, error) {
// try to parse as url (being permissive)
u, err := url.Parse(h)
if err == nil && u.Host != "" {
return u, nil
}
// try to parse as host:port
host, port, err := net.SplitHostPort(h)
if err != nil {
return nil, errMalformedHostname
}
return &url.URL{Host: net.JoinHostPort(host, port), Scheme: "http"}, nil
}
// NodeURL returns a human-readable string representation of the beacon node base url.
func (c *Client) NodeURL() string {
return c.baseURL.String()
}
type reqOption func(*http.Request)
// do is a generic, opinionated request function to reduce boilerplate amongst the methods in this package api/client/builder/types.go.
func (c *Client) do(ctx context.Context, method string, path string, body io.Reader, opts ...reqOption) (res []byte, err error) {
ctx, span := trace.StartSpan(ctx, "builder.client.do")
defer func() {
tracing.AnnotateError(span, err)
span.End()
}()
u := c.baseURL.ResolveReference(&url.URL{Path: path})
span.AddAttributes(trace.StringAttribute("url", u.String()),
trace.StringAttribute("method", method))
req, err := http.NewRequestWithContext(ctx, method, u.String(), body)
if err != nil {
return
}
for _, o := range opts {
o(req)
}
for _, o := range c.obvs {
if err = o.observe(req); err != nil {
return
}
}
r, err := c.hc.Do(req)
if err != nil {
return
}
defer func() {
closeErr := r.Body.Close()
if closeErr != nil {
log.WithError(closeErr).Error("Failed to close response body")
}
}()
if r.StatusCode != http.StatusOK {
err = non200Err(r)
return
}
res, err = io.ReadAll(r.Body)
if err != nil {
err = errors.Wrap(err, "error reading http response body from builder server")
return
}
return
}
var execHeaderTemplate = template.Must(template.New("").Parse(getExecHeaderPath))
func execHeaderPath(slot primitives.Slot, parentHash [32]byte, pubkey [48]byte) (string, error) {
v := struct {
Slot primitives.Slot
ParentHash string
Pubkey string
}{
Slot: slot,
ParentHash: fmt.Sprintf("%#x", parentHash),
Pubkey: fmt.Sprintf("%#x", pubkey),
}
b := bytes.NewBuffer(nil)
err := execHeaderTemplate.Execute(b, v)
if err != nil {
return "", errors.Wrapf(err, "error rendering exec header template with slot=%d, parentHash=%#x, pubkey=%#x", slot, parentHash, pubkey)
}
return b.String(), nil
}
// GetHeader is used by a proposing validator to request an execution payload header from the Builder node.
func (c *Client) GetHeader(ctx context.Context, slot primitives.Slot, parentHash [32]byte, pubkey [48]byte) (SignedBid, error) {
path, err := execHeaderPath(slot, parentHash, pubkey)
if err != nil {
return nil, err
}
hb, err := c.do(ctx, http.MethodGet, path, nil)
if err != nil {
return nil, err
}
v := &VersionResponse{}
if err := json.Unmarshal(hb, v); err != nil {
return nil, errors.Wrapf(err, "error unmarshaling the builder GetHeader response, using slot=%d, parentHash=%#x, pubkey=%#x", slot, parentHash, pubkey)
}
switch strings.ToLower(v.Version) {
case strings.ToLower(version.String(version.Capella)):
hr := &ExecHeaderResponseCapella{}
if err := json.Unmarshal(hb, hr); err != nil {
return nil, errors.Wrapf(err, "error unmarshaling the builder GetHeader response, using slot=%d, parentHash=%#x, pubkey=%#x", slot, parentHash, pubkey)
}
p, err := hr.ToProto()
if err != nil {
return nil, errors.Wrapf(err, "could not extract proto message from header")
}
return WrappedSignedBuilderBidCapella(p)
case strings.ToLower(version.String(version.Bellatrix)):
hr := &ExecHeaderResponse{}
if err := json.Unmarshal(hb, hr); err != nil {
return nil, errors.Wrapf(err, "error unmarshaling the builder GetHeader response, using slot=%d, parentHash=%#x, pubkey=%#x", slot, parentHash, pubkey)
}
p, err := hr.ToProto()
if err != nil {
return nil, errors.Wrap(err, "could not extract proto message from header")
}
return WrappedSignedBuilderBid(p)
default:
return nil, fmt.Errorf("unsupported header version %s", strings.ToLower(v.Version))
}
}
// RegisterValidator encodes the SignedValidatorRegistrationV1 message to json (including hex-encoding the byte
// fields with 0x prefixes) and posts to the builder validator registration endpoint.
func (c *Client) RegisterValidator(ctx context.Context, svr []*ethpb.SignedValidatorRegistrationV1) error {
ctx, span := trace.StartSpan(ctx, "builder.client.RegisterValidator")
defer span.End()
span.AddAttributes(trace.Int64Attribute("num_reqs", int64(len(svr))))
if len(svr) == 0 {
err := errors.Wrap(errMalformedRequest, "empty validator registration list")
tracing.AnnotateError(span, err)
return err
}
vs := make([]*SignedValidatorRegistration, len(svr))
for i := 0; i < len(svr); i++ {
vs[i] = &SignedValidatorRegistration{SignedValidatorRegistrationV1: svr[i]}
}
body, err := json.Marshal(vs)
if err != nil {
err := errors.Wrap(err, "error encoding the SignedValidatorRegistration value body in RegisterValidator")
tracing.AnnotateError(span, err)
return err
}
_, err = c.do(ctx, http.MethodPost, postRegisterValidatorPath, bytes.NewBuffer(body))
return err
}
// SubmitBlindedBlock calls the builder API endpoint that binds the validator to the builder and submits the block.
// The response is the full execution payload used to create the blinded block.
func (c *Client) SubmitBlindedBlock(ctx context.Context, sb interfaces.SignedBeaconBlock) (interfaces.ExecutionData, error) {
if !sb.IsBlinded() {
return nil, errNotBlinded
}
switch sb.Version() {
case version.Bellatrix:
psb, err := sb.PbBlindedBellatrixBlock()
if err != nil {
return nil, errors.Wrapf(err, "could not get protobuf block")
}
b := &SignedBlindedBeaconBlockBellatrix{SignedBlindedBeaconBlockBellatrix: psb}
body, err := json.Marshal(b)
if err != nil {
return nil, errors.Wrap(err, "error encoding the SignedBlindedBeaconBlockBellatrix value body in SubmitBlindedBlock")
}
ctx, cancel := context.WithTimeout(ctx, submitBlindedBlockTimeout)
defer cancel()
rb, err := c.do(ctx, http.MethodPost, postBlindedBeaconBlockPath, bytes.NewBuffer(body))
if err != nil {
return nil, errors.Wrap(err, "error posting the SignedBlindedBeaconBlockBellatrix to the builder api")
}
ep := &ExecPayloadResponse{}
if err := json.Unmarshal(rb, ep); err != nil {
return nil, errors.Wrap(err, "error unmarshaling the builder SubmitBlindedBlock response")
}
p, err := ep.ToProto()
if err != nil {
return nil, errors.Wrapf(err, "could not extract proto message from payload")
}
return blocks.WrappedExecutionPayload(p)
case version.Capella:
psb, err := sb.PbBlindedCapellaBlock()
if err != nil {
return nil, errors.Wrapf(err, "could not get protobuf block")
}
b := &SignedBlindedBeaconBlockCapella{SignedBlindedBeaconBlockCapella: psb}
body, err := json.Marshal(b)
if err != nil {
return nil, errors.Wrap(err, "error encoding the SignedBlindedBeaconBlockCapella value body in SubmitBlindedBlockCapella")
}
ctx, cancel := context.WithTimeout(ctx, submitBlindedBlockTimeout)
defer cancel()
rb, err := c.do(ctx, http.MethodPost, postBlindedBeaconBlockPath, bytes.NewBuffer(body))
if err != nil {
return nil, errors.Wrap(err, "error posting the SignedBlindedBeaconBlockCapella to the builder api")
}
ep := &ExecPayloadResponseCapella{}
if err := json.Unmarshal(rb, ep); err != nil {
return nil, errors.Wrap(err, "error unmarshaling the builder SubmitBlindedBlockCapella response")
}
p, err := ep.ToProto()
if err != nil {
return nil, errors.Wrapf(err, "could not extract proto message from payload")
}
return blocks.WrappedExecutionPayloadCapella(p)
default:
return nil, fmt.Errorf("unsupported block version %s", version.String(sb.Version()))
}
}
// SubmitBlindedBlockCapella calls the builder API endpoint that binds the validator to the builder and submits the block.
// The response is the full ExecutionPayloadCapella used to create the blinded block.
func (c *Client) SubmitBlindedBlockCapella(ctx context.Context, sb *ethpb.SignedBlindedBeaconBlockCapella) (*v1.ExecutionPayloadCapella, error) {
v := &SignedBlindedBeaconBlockCapella{SignedBlindedBeaconBlockCapella: sb}
body, err := json.Marshal(v)
if err != nil {
return nil, errors.Wrap(err, "error encoding the SignedBlindedBeaconBlockCapella value body in SubmitBlindedBlockCapella")
}
ctx, cancel := context.WithTimeout(ctx, submitBlindedBlockTimeout)
defer cancel()
rb, err := c.do(ctx, http.MethodPost, postBlindedBeaconBlockPath, bytes.NewBuffer(body))
if err != nil {
return nil, errors.Wrap(err, "error posting the SignedBlindedBeaconBlockBellatrix to the builder api")
}
ep := &ExecPayloadResponseCapella{}
if err := json.Unmarshal(rb, ep); err != nil {
return nil, errors.Wrap(err, "error unmarshaling the builder SubmitBlindedBlockCapella response")
}
return ep.ToProto()
}
// Status asks the remote builder server for a health check. A response of 200 with an empty body is the success/healthy
// response, and an error response may have an error message. This method will return a nil value for error in the
// happy path, and an error with information about the server response body for a non-200 response.
func (c *Client) Status(ctx context.Context) error {
_, err := c.do(ctx, http.MethodGet, getStatus, nil)
return err
}
func non200Err(response *http.Response) error {
bodyBytes, err := io.ReadAll(response.Body)
var errMessage ErrorMessage
var body string
if err != nil {
body = "(Unable to read response body.)"
} else {
body = "response body:\n" + string(bodyBytes)
}
msg := fmt.Sprintf("code=%d, url=%s, body=%s", response.StatusCode, response.Request.URL, body)
switch response.StatusCode {
case 204:
log.WithError(ErrNoContent).Debug(msg)
return ErrNoContent
case 400:
if jsonErr := json.Unmarshal(bodyBytes, &errMessage); jsonErr != nil {
return errors.Wrap(jsonErr, "unable to read response body")
}
log.WithError(ErrBadRequest).Debug(msg)
return errors.Wrap(ErrBadRequest, errMessage.Message)
case 404:
if jsonErr := json.Unmarshal(bodyBytes, &errMessage); jsonErr != nil {
return errors.Wrap(jsonErr, "unable to read response body")
}
log.WithError(ErrNotFound).Debug(msg)
return errors.Wrap(ErrNotFound, errMessage.Message)
case 500:
if jsonErr := json.Unmarshal(bodyBytes, &errMessage); jsonErr != nil {
return errors.Wrap(jsonErr, "unable to read response body")
}
log.WithError(ErrNotOK).Debug(msg)
return errors.Wrap(ErrNotOK, errMessage.Message)
default:
log.WithError(ErrNotOK).Debug(msg)
return errors.Wrap(ErrNotOK, fmt.Sprintf("unsupported error code: %d", response.StatusCode))
}
}
func covertEndPoint(ep string) network.Endpoint {
return network.Endpoint{
Url: ep,
Auth: network.AuthorizationData{ // Auth is not used for builder.
Method: authorization.None,
Value: "",
}}
}

View File

@@ -0,0 +1,616 @@
package builder
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
"strconv"
"testing"
"github.com/prysmaticlabs/go-bitfield"
"github.com/prysmaticlabs/prysm/v3/config/params"
"github.com/prysmaticlabs/prysm/v3/consensus-types/blocks"
types "github.com/prysmaticlabs/prysm/v3/consensus-types/primitives"
"github.com/prysmaticlabs/prysm/v3/encoding/bytesutil"
v1 "github.com/prysmaticlabs/prysm/v3/proto/engine/v1"
eth "github.com/prysmaticlabs/prysm/v3/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/v3/testing/assert"
"github.com/prysmaticlabs/prysm/v3/testing/require"
)
type roundtrip func(*http.Request) (*http.Response, error)
func (fn roundtrip) RoundTrip(r *http.Request) (*http.Response, error) {
return fn(r)
}
func TestClient_Status(t *testing.T) {
ctx := context.Background()
statusPath := "/eth/v1/builder/status"
hc := &http.Client{
Transport: roundtrip(func(r *http.Request) (*http.Response, error) {
defer func() {
if r.Body == nil {
return
}
require.NoError(t, r.Body.Close())
}()
require.Equal(t, statusPath, r.URL.Path)
return &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(bytes.NewBuffer(nil)),
Request: r.Clone(ctx),
}, nil
}),
}
c := &Client{
hc: hc,
baseURL: &url.URL{Host: "localhost:3500", Scheme: "http"},
}
require.NoError(t, c.Status(ctx))
hc = &http.Client{
Transport: roundtrip(func(r *http.Request) (*http.Response, error) {
defer func() {
if r.Body == nil {
return
}
require.NoError(t, r.Body.Close())
}()
require.Equal(t, statusPath, r.URL.Path)
message := ErrorMessage{
Code: 500,
Message: "Internal server error",
}
resp, err := json.Marshal(message)
require.NoError(t, err)
return &http.Response{
StatusCode: http.StatusInternalServerError,
Body: io.NopCloser(bytes.NewBuffer(resp)),
Request: r.Clone(ctx),
}, nil
}),
}
c = &Client{
hc: hc,
baseURL: &url.URL{Host: "localhost:3500", Scheme: "http"},
}
require.ErrorIs(t, c.Status(ctx), ErrNotOK)
}
func TestClient_RegisterValidator(t *testing.T) {
ctx := context.Background()
expectedBody := `[{"message":{"fee_recipient":"0x0000000000000000000000000000000000000000","gas_limit":"23","timestamp":"42","pubkey":"0x93247f2209abcacf57b75a51dafae777f9dd38bc7053d1af526f220a7489a6d3a2753e5f3e8b1cfe39b56f43611df74a"},"signature":"0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"}]`
expectedPath := "/eth/v1/builder/validators"
hc := &http.Client{
Transport: roundtrip(func(r *http.Request) (*http.Response, error) {
body, err := io.ReadAll(r.Body)
defer func() {
require.NoError(t, r.Body.Close())
}()
require.NoError(t, err)
require.Equal(t, expectedBody, string(body))
require.Equal(t, expectedPath, r.URL.Path)
require.Equal(t, http.MethodPost, r.Method)
return &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(bytes.NewBuffer(nil)),
Request: r.Clone(ctx),
}, nil
}),
}
c := &Client{
hc: hc,
baseURL: &url.URL{Host: "localhost:3500", Scheme: "http"},
}
reg := &eth.SignedValidatorRegistrationV1{
Message: &eth.ValidatorRegistrationV1{
FeeRecipient: ezDecode(t, params.BeaconConfig().EthBurnAddressHex),
GasLimit: 23,
Timestamp: 42,
Pubkey: ezDecode(t, "0x93247f2209abcacf57b75a51dafae777f9dd38bc7053d1af526f220a7489a6d3a2753e5f3e8b1cfe39b56f43611df74a"),
},
Signature: ezDecode(t, "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"),
}
require.NoError(t, c.RegisterValidator(ctx, []*eth.SignedValidatorRegistrationV1{reg}))
}
func TestClient_GetHeader(t *testing.T) {
ctx := context.Background()
expectedPath := "/eth/v1/builder/header/23/0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2/0x93247f2209abcacf57b75a51dafae777f9dd38bc7053d1af526f220a7489a6d3a2753e5f3e8b1cfe39b56f43611df74a"
var slot types.Slot = 23
parentHash := ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2")
pubkey := ezDecode(t, "0x93247f2209abcacf57b75a51dafae777f9dd38bc7053d1af526f220a7489a6d3a2753e5f3e8b1cfe39b56f43611df74a")
t.Run("server error", func(t *testing.T) {
hc := &http.Client{
Transport: roundtrip(func(r *http.Request) (*http.Response, error) {
require.Equal(t, expectedPath, r.URL.Path)
message := ErrorMessage{
Code: 500,
Message: "Internal server error",
}
resp, err := json.Marshal(message)
require.NoError(t, err)
return &http.Response{
StatusCode: http.StatusInternalServerError,
Body: io.NopCloser(bytes.NewBuffer(resp)),
Request: r.Clone(ctx),
}, nil
}),
}
c := &Client{
hc: hc,
baseURL: &url.URL{Host: "localhost:3500", Scheme: "http"},
}
_, err := c.GetHeader(ctx, slot, bytesutil.ToBytes32(parentHash), bytesutil.ToBytes48(pubkey))
require.ErrorIs(t, err, ErrNotOK)
})
t.Run("header not available", func(t *testing.T) {
hc := &http.Client{
Transport: roundtrip(func(r *http.Request) (*http.Response, error) {
require.Equal(t, expectedPath, r.URL.Path)
return &http.Response{
StatusCode: http.StatusNoContent,
Body: io.NopCloser(bytes.NewBuffer([]byte("No header is available."))),
Request: r.Clone(ctx),
}, nil
}),
}
c := &Client{
hc: hc,
baseURL: &url.URL{Host: "localhost:3500", Scheme: "http"},
}
_, err := c.GetHeader(ctx, slot, bytesutil.ToBytes32(parentHash), bytesutil.ToBytes48(pubkey))
require.ErrorIs(t, err, ErrNoContent)
})
t.Run("bellatrix", func(t *testing.T) {
hc := &http.Client{
Transport: roundtrip(func(r *http.Request) (*http.Response, error) {
require.Equal(t, expectedPath, r.URL.Path)
return &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(bytes.NewBufferString(testExampleHeaderResponse)),
Request: r.Clone(ctx),
}, nil
}),
}
c := &Client{
hc: hc,
baseURL: &url.URL{Host: "localhost:3500", Scheme: "http"},
}
h, err := c.GetHeader(ctx, slot, bytesutil.ToBytes32(parentHash), bytesutil.ToBytes48(pubkey))
require.NoError(t, err)
expectedSig := ezDecode(t, "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505")
require.Equal(t, true, bytes.Equal(expectedSig, h.Signature()))
expectedTxRoot := ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2")
bid, err := h.Message()
require.NoError(t, err)
bidHeader, err := bid.Header()
require.NoError(t, err)
withdrawalsRoot, err := bidHeader.TransactionsRoot()
require.NoError(t, err)
require.Equal(t, true, bytes.Equal(expectedTxRoot, withdrawalsRoot))
require.Equal(t, uint64(1), bidHeader.GasUsed())
value, err := stringToUint256("652312848583266388373324160190187140051835877600158453279131187530910662656")
require.NoError(t, err)
require.Equal(t, fmt.Sprintf("%#x", value.SSZBytes()), fmt.Sprintf("%#x", bid.Value()))
})
t.Run("capella", func(t *testing.T) {
hc := &http.Client{
Transport: roundtrip(func(r *http.Request) (*http.Response, error) {
require.Equal(t, expectedPath, r.URL.Path)
return &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(bytes.NewBufferString(testExampleHeaderResponseCapella)),
Request: r.Clone(ctx),
}, nil
}),
}
c := &Client{
hc: hc,
baseURL: &url.URL{Host: "localhost:3500", Scheme: "http"},
}
h, err := c.GetHeader(ctx, slot, bytesutil.ToBytes32(parentHash), bytesutil.ToBytes48(pubkey))
require.NoError(t, err)
expectedWithdrawalsRoot := ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2")
bid, err := h.Message()
require.NoError(t, err)
bidHeader, err := bid.Header()
require.NoError(t, err)
withdrawalsRoot, err := bidHeader.WithdrawalsRoot()
require.NoError(t, err)
require.Equal(t, true, bytes.Equal(expectedWithdrawalsRoot, withdrawalsRoot))
})
t.Run("unsupported version", func(t *testing.T) {
hc := &http.Client{
Transport: roundtrip(func(r *http.Request) (*http.Response, error) {
require.Equal(t, expectedPath, r.URL.Path)
return &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(bytes.NewBufferString(testExampleHeaderResponseUnknownVersion)),
Request: r.Clone(ctx),
}, nil
}),
}
c := &Client{
hc: hc,
baseURL: &url.URL{Host: "localhost:3500", Scheme: "http"},
}
_, err := c.GetHeader(ctx, slot, bytesutil.ToBytes32(parentHash), bytesutil.ToBytes48(pubkey))
require.ErrorContains(t, "unsupported header version", err)
})
}
func TestSubmitBlindedBlock(t *testing.T) {
ctx := context.Background()
t.Run("bellatrix", func(t *testing.T) {
hc := &http.Client{
Transport: roundtrip(func(r *http.Request) (*http.Response, error) {
require.Equal(t, postBlindedBeaconBlockPath, r.URL.Path)
return &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(bytes.NewBufferString(testExampleExecutionPayload)),
Request: r.Clone(ctx),
}, nil
}),
}
c := &Client{
hc: hc,
baseURL: &url.URL{Host: "localhost:3500", Scheme: "http"},
}
sbbb, err := blocks.NewSignedBeaconBlock(testSignedBlindedBeaconBlockBellatrix(t))
require.NoError(t, err)
ep, err := c.SubmitBlindedBlock(ctx, sbbb)
require.NoError(t, err)
require.Equal(t, true, bytes.Equal(ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"), ep.ParentHash()))
bfpg, err := stringToUint256("452312848583266388373324160190187140051835877600158453279131187530910662656")
require.NoError(t, err)
require.Equal(t, fmt.Sprintf("%#x", bfpg.SSZBytes()), fmt.Sprintf("%#x", ep.BaseFeePerGas()))
require.Equal(t, uint64(1), ep.GasLimit())
})
t.Run("capella", func(t *testing.T) {
hc := &http.Client{
Transport: roundtrip(func(r *http.Request) (*http.Response, error) {
require.Equal(t, postBlindedBeaconBlockPath, r.URL.Path)
return &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(bytes.NewBufferString(testExampleExecutionPayloadCapella)),
Request: r.Clone(ctx),
}, nil
}),
}
c := &Client{
hc: hc,
baseURL: &url.URL{Host: "localhost:3500", Scheme: "http"},
}
sbb, err := blocks.NewSignedBeaconBlock(testSignedBlindedBeaconBlockCapella(t))
require.NoError(t, err)
ep, err := c.SubmitBlindedBlock(ctx, sbb)
require.NoError(t, err)
withdrawals, err := ep.Withdrawals()
require.NoError(t, err)
require.Equal(t, 1, len(withdrawals))
assert.Equal(t, uint64(1), withdrawals[0].Index)
assert.Equal(t, types.ValidatorIndex(1), withdrawals[0].ValidatorIndex)
assert.DeepEqual(t, ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943"), withdrawals[0].Address)
assert.Equal(t, uint64(1), withdrawals[0].Amount)
})
t.Run("not blinded", func(t *testing.T) {
sbb, err := blocks.NewSignedBeaconBlock(&eth.SignedBeaconBlockBellatrix{Block: &eth.BeaconBlockBellatrix{Body: &eth.BeaconBlockBodyBellatrix{}}})
require.NoError(t, err)
_, err = (&Client{}).SubmitBlindedBlock(ctx, sbb)
require.ErrorIs(t, err, errNotBlinded)
})
}
func testSignedBlindedBeaconBlockBellatrix(t *testing.T) *eth.SignedBlindedBeaconBlockBellatrix {
return &eth.SignedBlindedBeaconBlockBellatrix{
Block: &eth.BlindedBeaconBlockBellatrix{
Slot: 1,
ProposerIndex: 1,
ParentRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
StateRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
Body: &eth.BlindedBeaconBlockBodyBellatrix{
RandaoReveal: ezDecode(t, "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"),
Eth1Data: &eth.Eth1Data{
DepositRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
DepositCount: 1,
BlockHash: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
},
Graffiti: ezDecode(t, "0xdeadbeefc0ffee"),
ProposerSlashings: []*eth.ProposerSlashing{
{
Header_1: &eth.SignedBeaconBlockHeader{
Header: &eth.BeaconBlockHeader{
Slot: 1,
ProposerIndex: 1,
ParentRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
StateRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
BodyRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
},
Signature: ezDecode(t, "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"),
},
Header_2: &eth.SignedBeaconBlockHeader{
Header: &eth.BeaconBlockHeader{
Slot: 1,
ProposerIndex: 1,
ParentRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
StateRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
BodyRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
},
Signature: ezDecode(t, "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"),
},
},
},
AttesterSlashings: []*eth.AttesterSlashing{
{
Attestation_1: &eth.IndexedAttestation{
AttestingIndices: []uint64{1},
Data: &eth.AttestationData{
Slot: 1,
CommitteeIndex: 1,
BeaconBlockRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
Source: &eth.Checkpoint{
Epoch: 1,
Root: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
},
Target: &eth.Checkpoint{
Epoch: 1,
Root: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
},
},
Signature: ezDecode(t, "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"),
},
Attestation_2: &eth.IndexedAttestation{
AttestingIndices: []uint64{1},
Data: &eth.AttestationData{
Slot: 1,
CommitteeIndex: 1,
BeaconBlockRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
Source: &eth.Checkpoint{
Epoch: 1,
Root: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
},
Target: &eth.Checkpoint{
Epoch: 1,
Root: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
},
},
Signature: ezDecode(t, "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"),
},
},
},
Attestations: []*eth.Attestation{
{
AggregationBits: bitfield.Bitlist{0x01},
Data: &eth.AttestationData{
Slot: 1,
CommitteeIndex: 1,
BeaconBlockRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
Source: &eth.Checkpoint{
Epoch: 1,
Root: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
},
Target: &eth.Checkpoint{
Epoch: 1,
Root: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
},
},
Signature: ezDecode(t, "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"),
},
},
Deposits: []*eth.Deposit{
{
Proof: [][]byte{ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2")},
Data: &eth.Deposit_Data{
PublicKey: ezDecode(t, "0x93247f2209abcacf57b75a51dafae777f9dd38bc7053d1af526f220a7489a6d3a2753e5f3e8b1cfe39b56f43611df74a"),
WithdrawalCredentials: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
Amount: 1,
Signature: ezDecode(t, "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"),
},
},
},
VoluntaryExits: []*eth.SignedVoluntaryExit{
{
Exit: &eth.VoluntaryExit{
Epoch: 1,
ValidatorIndex: 1,
},
Signature: ezDecode(t, "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"),
},
},
SyncAggregate: &eth.SyncAggregate{
SyncCommitteeSignature: make([]byte, 48),
SyncCommitteeBits: bitfield.Bitvector512{0x01},
},
ExecutionPayloadHeader: &v1.ExecutionPayloadHeader{
ParentHash: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
FeeRecipient: ezDecode(t, "0xabcf8e0d4e9587369b2301d0790347320302cc09"),
StateRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
ReceiptsRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
LogsBloom: ezDecode(t, "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"),
PrevRandao: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
BlockNumber: 1,
GasLimit: 1,
GasUsed: 1,
Timestamp: 1,
ExtraData: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
BaseFeePerGas: []byte(strconv.FormatUint(1, 10)),
BlockHash: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
TransactionsRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
},
},
},
Signature: ezDecode(t, "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"),
}
}
func testSignedBlindedBeaconBlockCapella(t *testing.T) *eth.SignedBlindedBeaconBlockCapella {
return &eth.SignedBlindedBeaconBlockCapella{
Block: &eth.BlindedBeaconBlockCapella{
Slot: 1,
ProposerIndex: 1,
ParentRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
StateRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
Body: &eth.BlindedBeaconBlockBodyCapella{
RandaoReveal: ezDecode(t, "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"),
Eth1Data: &eth.Eth1Data{
DepositRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
DepositCount: 1,
BlockHash: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
},
Graffiti: ezDecode(t, "0xdeadbeefc0ffee"),
ProposerSlashings: []*eth.ProposerSlashing{
{
Header_1: &eth.SignedBeaconBlockHeader{
Header: &eth.BeaconBlockHeader{
Slot: 1,
ProposerIndex: 1,
ParentRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
StateRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
BodyRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
},
Signature: ezDecode(t, "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"),
},
Header_2: &eth.SignedBeaconBlockHeader{
Header: &eth.BeaconBlockHeader{
Slot: 1,
ProposerIndex: 1,
ParentRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
StateRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
BodyRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
},
Signature: ezDecode(t, "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"),
},
},
},
AttesterSlashings: []*eth.AttesterSlashing{
{
Attestation_1: &eth.IndexedAttestation{
AttestingIndices: []uint64{1},
Data: &eth.AttestationData{
Slot: 1,
CommitteeIndex: 1,
BeaconBlockRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
Source: &eth.Checkpoint{
Epoch: 1,
Root: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
},
Target: &eth.Checkpoint{
Epoch: 1,
Root: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
},
},
Signature: ezDecode(t, "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"),
},
Attestation_2: &eth.IndexedAttestation{
AttestingIndices: []uint64{1},
Data: &eth.AttestationData{
Slot: 1,
CommitteeIndex: 1,
BeaconBlockRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
Source: &eth.Checkpoint{
Epoch: 1,
Root: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
},
Target: &eth.Checkpoint{
Epoch: 1,
Root: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
},
},
Signature: ezDecode(t, "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"),
},
},
},
Attestations: []*eth.Attestation{
{
AggregationBits: bitfield.Bitlist{0x01},
Data: &eth.AttestationData{
Slot: 1,
CommitteeIndex: 1,
BeaconBlockRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
Source: &eth.Checkpoint{
Epoch: 1,
Root: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
},
Target: &eth.Checkpoint{
Epoch: 1,
Root: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
},
},
Signature: ezDecode(t, "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"),
},
},
Deposits: []*eth.Deposit{
{
Proof: [][]byte{ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2")},
Data: &eth.Deposit_Data{
PublicKey: ezDecode(t, "0x93247f2209abcacf57b75a51dafae777f9dd38bc7053d1af526f220a7489a6d3a2753e5f3e8b1cfe39b56f43611df74a"),
WithdrawalCredentials: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
Amount: 1,
Signature: ezDecode(t, "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"),
},
},
},
VoluntaryExits: []*eth.SignedVoluntaryExit{
{
Exit: &eth.VoluntaryExit{
Epoch: 1,
ValidatorIndex: 1,
},
Signature: ezDecode(t, "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"),
},
},
SyncAggregate: &eth.SyncAggregate{
SyncCommitteeSignature: make([]byte, 48),
SyncCommitteeBits: bitfield.Bitvector512{0x01},
},
ExecutionPayloadHeader: &v1.ExecutionPayloadHeaderCapella{
ParentHash: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
FeeRecipient: ezDecode(t, "0xabcf8e0d4e9587369b2301d0790347320302cc09"),
StateRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
ReceiptsRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
LogsBloom: ezDecode(t, "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"),
PrevRandao: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
BlockNumber: 1,
GasLimit: 1,
GasUsed: 1,
Timestamp: 1,
ExtraData: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
BaseFeePerGas: []byte(strconv.FormatUint(1, 10)),
BlockHash: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
TransactionsRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
WithdrawalsRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
},
},
},
Signature: ezDecode(t, "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"),
}
}
func TestRequestLogger(t *testing.T) {
wo := WithObserver(&requestLogger{})
c, err := NewClient("localhost:3500", wo)
require.NoError(t, err)
ctx := context.Background()
hc := &http.Client{
Transport: roundtrip(func(r *http.Request) (*http.Response, error) {
require.Equal(t, getStatus, r.URL.Path)
return &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(bytes.NewBufferString(testExampleExecutionPayload)),
Request: r.Clone(ctx),
}, nil
}),
}
c.hc = hc
err = c.Status(ctx)
require.NoError(t, err)
}

View File

@@ -0,0 +1,17 @@
package builder
import "github.com/pkg/errors"
// ErrNotOK is used to indicate when an HTTP request to the Beacon Node API failed with any non-2xx response code.
// More specific errors may be returned, but an error in reaction to a non-2xx response will always wrap ErrNotOK.
var ErrNotOK = errors.New("did not receive 200 response from API")
// ErrNotFound specifically means that a '404 - NOT FOUND' response was received from the API.
var ErrNotFound = errors.Wrap(ErrNotOK, "recv 404 NotFound response from API")
// ErrBadRequest specifically means that a '400 - BAD REQUEST' response was received from the API.
var ErrBadRequest = errors.Wrap(ErrNotOK, "recv 400 BadRequest response from API")
// ErrNoContent specifically means that a '204 - No Content' response was received from the API.
// Typically, a 204 is a success but in this case for the Header API means No header is available
var ErrNoContent = errors.New("recv 204 no content response from API, No header is available")

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1 @@
{"parent_hash":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","fee_recipient":"0xabcf8e0d4e9587369b2301d0790347320302cc09","state_root":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","receipts_root":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","logs_bloom":"0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","prev_randao":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","block_number":"1","gas_limit":"1","gas_used":"1","timestamp":"1","extra_data":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","base_fee_per_gas":"14074904626401341155369551180448584754667373453244490859944217516317499064576","block_hash":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","transactions_root":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","withdrawals_root":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"}

View File

@@ -0,0 +1 @@
{"parent_hash":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","fee_recipient":"0xabcf8e0d4e9587369b2301d0790347320302cc09","state_root":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","receipts_root":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","logs_bloom":"0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","prev_randao":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","block_number":"1","gas_limit":"1","gas_used":"1","timestamp":"1","extra_data":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","base_fee_per_gas":"14074904626401341155369551180448584754667373453244490859944217516317499064576","block_hash":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","transactions_root":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"}

View File

@@ -0,0 +1,15 @@
load("@prysm//tools/go:def.bzl", "go_library")
go_library(
name = "go_default_library",
srcs = ["mock.go"],
importpath = "github.com/prysmaticlabs/prysm/v3/api/client/builder/testing",
visibility = ["//visibility:public"],
deps = [
"//api/client/builder:go_default_library",
"//consensus-types/interfaces:go_default_library",
"//consensus-types/primitives:go_default_library",
"//encoding/bytesutil:go_default_library",
"//proto/prysm/v1alpha1:go_default_library",
],
)

View File

@@ -0,0 +1,50 @@
package testing
import (
"context"
"github.com/prysmaticlabs/prysm/v3/api/client/builder"
"github.com/prysmaticlabs/prysm/v3/consensus-types/interfaces"
"github.com/prysmaticlabs/prysm/v3/consensus-types/primitives"
"github.com/prysmaticlabs/prysm/v3/encoding/bytesutil"
ethpb "github.com/prysmaticlabs/prysm/v3/proto/prysm/v1alpha1"
)
// MockClient is a mock implementation of BuilderClient.
type MockClient struct {
RegisteredVals map[[48]byte]bool
}
// NewClient creates a new, correctly initialized mock.
func NewClient() MockClient {
return MockClient{RegisteredVals: map[[48]byte]bool{}}
}
// NodeURL --
func (MockClient) NodeURL() string {
return ""
}
// GetHeader --
func (MockClient) GetHeader(_ context.Context, _ primitives.Slot, _ [32]byte, _ [48]byte) (builder.SignedBid, error) {
return nil, nil
}
// RegisterValidator --
func (m MockClient) RegisterValidator(_ context.Context, svr []*ethpb.SignedValidatorRegistrationV1) error {
for _, r := range svr {
b := bytesutil.ToBytes48(r.Message.Pubkey)
m.RegisteredVals[b] = true
}
return nil
}
// SubmitBlindedBlock --
func (MockClient) SubmitBlindedBlock(_ context.Context, _ interfaces.SignedBeaconBlock) (interfaces.ExecutionData, error) {
return nil, nil
}
// Status --
func (MockClient) Status(_ context.Context) error {
return nil
}

963
api/client/builder/types.go Normal file
View File

@@ -0,0 +1,963 @@
package builder
import (
"encoding/json"
"fmt"
"math/big"
"strconv"
"github.com/ethereum/go-ethereum/common/hexutil"
"github.com/pkg/errors"
types "github.com/prysmaticlabs/prysm/v3/consensus-types/primitives"
"github.com/prysmaticlabs/prysm/v3/encoding/bytesutil"
v1 "github.com/prysmaticlabs/prysm/v3/proto/engine/v1"
eth "github.com/prysmaticlabs/prysm/v3/proto/prysm/v1alpha1"
)
type SignedValidatorRegistration struct {
*eth.SignedValidatorRegistrationV1
}
type ValidatorRegistration struct {
*eth.ValidatorRegistrationV1
}
func (r *SignedValidatorRegistration) MarshalJSON() ([]byte, error) {
return json.Marshal(struct {
Message *ValidatorRegistration `json:"message"`
Signature hexutil.Bytes `json:"signature"`
}{
Message: &ValidatorRegistration{r.Message},
Signature: r.SignedValidatorRegistrationV1.Signature,
})
}
func (r *SignedValidatorRegistration) UnmarshalJSON(b []byte) error {
if r.SignedValidatorRegistrationV1 == nil {
r.SignedValidatorRegistrationV1 = &eth.SignedValidatorRegistrationV1{}
}
o := struct {
Message *ValidatorRegistration `json:"message"`
Signature hexutil.Bytes `json:"signature"`
}{}
if err := json.Unmarshal(b, &o); err != nil {
return err
}
r.Message = o.Message.ValidatorRegistrationV1
r.Signature = o.Signature
return nil
}
func (r *ValidatorRegistration) MarshalJSON() ([]byte, error) {
return json.Marshal(struct {
FeeRecipient hexutil.Bytes `json:"fee_recipient"`
GasLimit string `json:"gas_limit"`
Timestamp string `json:"timestamp"`
Pubkey hexutil.Bytes `json:"pubkey"`
}{
FeeRecipient: r.FeeRecipient,
GasLimit: fmt.Sprintf("%d", r.GasLimit),
Timestamp: fmt.Sprintf("%d", r.Timestamp),
Pubkey: r.Pubkey,
})
}
func (r *ValidatorRegistration) UnmarshalJSON(b []byte) error {
if r.ValidatorRegistrationV1 == nil {
r.ValidatorRegistrationV1 = &eth.ValidatorRegistrationV1{}
}
o := struct {
FeeRecipient hexutil.Bytes `json:"fee_recipient"`
GasLimit string `json:"gas_limit"`
Timestamp string `json:"timestamp"`
Pubkey hexutil.Bytes `json:"pubkey"`
}{}
if err := json.Unmarshal(b, &o); err != nil {
return err
}
r.FeeRecipient = o.FeeRecipient
r.Pubkey = o.Pubkey
var err error
if r.GasLimit, err = strconv.ParseUint(o.GasLimit, 10, 64); err != nil {
return errors.Wrap(err, "failed to parse gas limit")
}
if r.Timestamp, err = strconv.ParseUint(o.Timestamp, 10, 64); err != nil {
return errors.Wrap(err, "failed to parse timestamp")
}
return nil
}
var errInvalidUint256 = errors.New("invalid Uint256")
var errDecodeUint256 = errors.New("unable to decode into Uint256")
type Uint256 struct {
*big.Int
}
func isValidUint256(bi *big.Int) bool {
return bi.Cmp(big.NewInt(0)) >= 0 && bi.BitLen() <= 256
}
func stringToUint256(s string) (Uint256, error) {
bi := new(big.Int)
_, ok := bi.SetString(s, 10)
if !ok || !isValidUint256(bi) {
return Uint256{}, errors.Wrapf(errDecodeUint256, "value=%s", s)
}
return Uint256{Int: bi}, nil
}
// sszBytesToUint256 creates a Uint256 from a ssz-style (little-endian byte slice) representation.
func sszBytesToUint256(b []byte) (Uint256, error) {
bi := bytesutil.LittleEndianBytesToBigInt(b)
if !isValidUint256(bi) {
return Uint256{}, errors.Wrapf(errDecodeUint256, "value=%s", b)
}
return Uint256{Int: bi}, nil
}
// SSZBytes creates an ssz-style (little-endian byte slice) representation of the Uint256
func (s Uint256) SSZBytes() []byte {
if !isValidUint256(s.Int) {
return []byte{}
}
return bytesutil.PadTo(bytesutil.ReverseByteOrder(s.Int.Bytes()), 32)
}
func (s *Uint256) UnmarshalJSON(t []byte) error {
start := 0
end := len(t)
if t[0] == '"' {
start += 1
}
if t[end-1] == '"' {
end -= 1
}
return s.UnmarshalText(t[start:end])
}
func (s *Uint256) UnmarshalText(t []byte) error {
if s.Int == nil {
s.Int = big.NewInt(0)
}
z, ok := s.SetString(string(t), 10)
if !ok {
return errors.Wrapf(errDecodeUint256, "value=%s", t)
}
if !isValidUint256(z) {
return errors.Wrapf(errDecodeUint256, "value=%s", t)
}
s.Int = z
return nil
}
func (s Uint256) MarshalJSON() ([]byte, error) {
t, err := s.MarshalText()
if err != nil {
return nil, err
}
t = append([]byte{'"'}, t...)
t = append(t, '"')
return t, nil
}
func (s Uint256) MarshalText() ([]byte, error) {
if !isValidUint256(s.Int) {
return nil, errors.Wrapf(errInvalidUint256, "value=%s", s.Int)
}
return []byte(s.String()), nil
}
type Uint64String uint64
func (s *Uint64String) UnmarshalText(t []byte) error {
u, err := strconv.ParseUint(string(t), 10, 64)
*s = Uint64String(u)
return err
}
func (s Uint64String) MarshalText() ([]byte, error) {
return []byte(fmt.Sprintf("%d", s)), nil
}
type VersionResponse struct {
Version string `json:"version"`
}
type ExecHeaderResponse struct {
Version string `json:"version"`
Data struct {
Signature hexutil.Bytes `json:"signature"`
Message *BuilderBid `json:"message"`
} `json:"data"`
}
func (ehr *ExecHeaderResponse) ToProto() (*eth.SignedBuilderBid, error) {
bb, err := ehr.Data.Message.ToProto()
if err != nil {
return nil, err
}
return &eth.SignedBuilderBid{
Message: bb,
Signature: ehr.Data.Signature,
}, nil
}
func (bb *BuilderBid) ToProto() (*eth.BuilderBid, error) {
header, err := bb.Header.ToProto()
if err != nil {
return nil, err
}
return &eth.BuilderBid{
Header: header,
Value: bb.Value.SSZBytes(),
Pubkey: bb.Pubkey,
}, nil
}
func (h *ExecutionPayloadHeader) ToProto() (*v1.ExecutionPayloadHeader, error) {
return &v1.ExecutionPayloadHeader{
ParentHash: h.ParentHash,
FeeRecipient: h.FeeRecipient,
StateRoot: h.StateRoot,
ReceiptsRoot: h.ReceiptsRoot,
LogsBloom: h.LogsBloom,
PrevRandao: h.PrevRandao,
BlockNumber: uint64(h.BlockNumber),
GasLimit: uint64(h.GasLimit),
GasUsed: uint64(h.GasUsed),
Timestamp: uint64(h.Timestamp),
ExtraData: h.ExtraData,
BaseFeePerGas: h.BaseFeePerGas.SSZBytes(),
BlockHash: h.BlockHash,
TransactionsRoot: h.TransactionsRoot,
}, nil
}
type BuilderBid struct {
Header *ExecutionPayloadHeader `json:"header"`
Value Uint256 `json:"value"`
Pubkey hexutil.Bytes `json:"pubkey"`
}
type ExecutionPayloadHeader struct {
ParentHash hexutil.Bytes `json:"parent_hash"`
FeeRecipient hexutil.Bytes `json:"fee_recipient"`
StateRoot hexutil.Bytes `json:"state_root"`
ReceiptsRoot hexutil.Bytes `json:"receipts_root"`
LogsBloom hexutil.Bytes `json:"logs_bloom"`
PrevRandao hexutil.Bytes `json:"prev_randao"`
BlockNumber Uint64String `json:"block_number"`
GasLimit Uint64String `json:"gas_limit"`
GasUsed Uint64String `json:"gas_used"`
Timestamp Uint64String `json:"timestamp"`
ExtraData hexutil.Bytes `json:"extra_data"`
BaseFeePerGas Uint256 `json:"base_fee_per_gas"`
BlockHash hexutil.Bytes `json:"block_hash"`
TransactionsRoot hexutil.Bytes `json:"transactions_root"`
*v1.ExecutionPayloadHeader
}
func (h *ExecutionPayloadHeader) MarshalJSON() ([]byte, error) {
type MarshalCaller ExecutionPayloadHeader
baseFeePerGas, err := sszBytesToUint256(h.ExecutionPayloadHeader.BaseFeePerGas)
if err != nil {
return []byte{}, errors.Wrapf(err, "invalid BaseFeePerGas")
}
return json.Marshal(&MarshalCaller{
ParentHash: h.ExecutionPayloadHeader.ParentHash,
FeeRecipient: h.ExecutionPayloadHeader.FeeRecipient,
StateRoot: h.ExecutionPayloadHeader.StateRoot,
ReceiptsRoot: h.ExecutionPayloadHeader.ReceiptsRoot,
LogsBloom: h.ExecutionPayloadHeader.LogsBloom,
PrevRandao: h.ExecutionPayloadHeader.PrevRandao,
BlockNumber: Uint64String(h.ExecutionPayloadHeader.BlockNumber),
GasLimit: Uint64String(h.ExecutionPayloadHeader.GasLimit),
GasUsed: Uint64String(h.ExecutionPayloadHeader.GasUsed),
Timestamp: Uint64String(h.ExecutionPayloadHeader.Timestamp),
ExtraData: h.ExecutionPayloadHeader.ExtraData,
BaseFeePerGas: baseFeePerGas,
BlockHash: h.ExecutionPayloadHeader.BlockHash,
TransactionsRoot: h.ExecutionPayloadHeader.TransactionsRoot,
})
}
func (h *ExecutionPayloadHeader) UnmarshalJSON(b []byte) error {
type UnmarshalCaller ExecutionPayloadHeader
uc := &UnmarshalCaller{}
if err := json.Unmarshal(b, uc); err != nil {
return err
}
ep := ExecutionPayloadHeader(*uc)
*h = ep
var err error
h.ExecutionPayloadHeader, err = h.ToProto()
return err
}
type ExecPayloadResponse struct {
Version string `json:"version"`
Data ExecutionPayload `json:"data"`
}
type ExecutionPayload struct {
ParentHash hexutil.Bytes `json:"parent_hash"`
FeeRecipient hexutil.Bytes `json:"fee_recipient"`
StateRoot hexutil.Bytes `json:"state_root"`
ReceiptsRoot hexutil.Bytes `json:"receipts_root"`
LogsBloom hexutil.Bytes `json:"logs_bloom"`
PrevRandao hexutil.Bytes `json:"prev_randao"`
BlockNumber Uint64String `json:"block_number"`
GasLimit Uint64String `json:"gas_limit"`
GasUsed Uint64String `json:"gas_used"`
Timestamp Uint64String `json:"timestamp"`
ExtraData hexutil.Bytes `json:"extra_data"`
BaseFeePerGas Uint256 `json:"base_fee_per_gas"`
BlockHash hexutil.Bytes `json:"block_hash"`
Transactions []hexutil.Bytes `json:"transactions"`
}
func (r *ExecPayloadResponse) ToProto() (*v1.ExecutionPayload, error) {
return r.Data.ToProto()
}
func (p *ExecutionPayload) ToProto() (*v1.ExecutionPayload, error) {
txs := make([][]byte, len(p.Transactions))
for i := range p.Transactions {
txs[i] = p.Transactions[i]
}
return &v1.ExecutionPayload{
ParentHash: p.ParentHash,
FeeRecipient: p.FeeRecipient,
StateRoot: p.StateRoot,
ReceiptsRoot: p.ReceiptsRoot,
LogsBloom: p.LogsBloom,
PrevRandao: p.PrevRandao,
BlockNumber: uint64(p.BlockNumber),
GasLimit: uint64(p.GasLimit),
GasUsed: uint64(p.GasUsed),
Timestamp: uint64(p.Timestamp),
ExtraData: p.ExtraData,
BaseFeePerGas: p.BaseFeePerGas.SSZBytes(),
BlockHash: p.BlockHash,
Transactions: txs,
}, nil
}
type ExecHeaderResponseCapella struct {
Data struct {
Signature hexutil.Bytes `json:"signature"`
Message *BuilderBidCapella `json:"message"`
} `json:"data"`
}
func (ehr *ExecHeaderResponseCapella) ToProto() (*eth.SignedBuilderBidCapella, error) {
bb, err := ehr.Data.Message.ToProto()
if err != nil {
return nil, err
}
return &eth.SignedBuilderBidCapella{
Message: bb,
Signature: ehr.Data.Signature,
}, nil
}
func (bb *BuilderBidCapella) ToProto() (*eth.BuilderBidCapella, error) {
header, err := bb.Header.ToProto()
if err != nil {
return nil, err
}
return &eth.BuilderBidCapella{
Header: header,
Value: bb.Value.SSZBytes(),
Pubkey: bb.Pubkey,
}, nil
}
func (h *ExecutionPayloadHeaderCapella) ToProto() (*v1.ExecutionPayloadHeaderCapella, error) {
return &v1.ExecutionPayloadHeaderCapella{
ParentHash: h.ParentHash,
FeeRecipient: h.FeeRecipient,
StateRoot: h.StateRoot,
ReceiptsRoot: h.ReceiptsRoot,
LogsBloom: h.LogsBloom,
PrevRandao: h.PrevRandao,
BlockNumber: uint64(h.BlockNumber),
GasLimit: uint64(h.GasLimit),
GasUsed: uint64(h.GasUsed),
Timestamp: uint64(h.Timestamp),
ExtraData: h.ExtraData,
BaseFeePerGas: h.BaseFeePerGas.SSZBytes(),
BlockHash: h.BlockHash,
TransactionsRoot: h.TransactionsRoot,
WithdrawalsRoot: h.WithdrawalsRoot,
}, nil
}
type BuilderBidCapella struct {
Header *ExecutionPayloadHeaderCapella `json:"header"`
Value Uint256 `json:"value"`
Pubkey hexutil.Bytes `json:"pubkey"`
}
type ExecutionPayloadHeaderCapella struct {
ParentHash hexutil.Bytes `json:"parent_hash"`
FeeRecipient hexutil.Bytes `json:"fee_recipient"`
StateRoot hexutil.Bytes `json:"state_root"`
ReceiptsRoot hexutil.Bytes `json:"receipts_root"`
LogsBloom hexutil.Bytes `json:"logs_bloom"`
PrevRandao hexutil.Bytes `json:"prev_randao"`
BlockNumber Uint64String `json:"block_number"`
GasLimit Uint64String `json:"gas_limit"`
GasUsed Uint64String `json:"gas_used"`
Timestamp Uint64String `json:"timestamp"`
ExtraData hexutil.Bytes `json:"extra_data"`
BaseFeePerGas Uint256 `json:"base_fee_per_gas"`
BlockHash hexutil.Bytes `json:"block_hash"`
TransactionsRoot hexutil.Bytes `json:"transactions_root"`
WithdrawalsRoot hexutil.Bytes `json:"withdrawals_root"`
*v1.ExecutionPayloadHeaderCapella
}
func (h *ExecutionPayloadHeaderCapella) MarshalJSON() ([]byte, error) {
type MarshalCaller ExecutionPayloadHeaderCapella
baseFeePerGas, err := sszBytesToUint256(h.ExecutionPayloadHeaderCapella.BaseFeePerGas)
if err != nil {
return []byte{}, errors.Wrapf(err, "invalid BaseFeePerGas")
}
return json.Marshal(&MarshalCaller{
ParentHash: h.ExecutionPayloadHeaderCapella.ParentHash,
FeeRecipient: h.ExecutionPayloadHeaderCapella.FeeRecipient,
StateRoot: h.ExecutionPayloadHeaderCapella.StateRoot,
ReceiptsRoot: h.ExecutionPayloadHeaderCapella.ReceiptsRoot,
LogsBloom: h.ExecutionPayloadHeaderCapella.LogsBloom,
PrevRandao: h.ExecutionPayloadHeaderCapella.PrevRandao,
BlockNumber: Uint64String(h.ExecutionPayloadHeaderCapella.BlockNumber),
GasLimit: Uint64String(h.ExecutionPayloadHeaderCapella.GasLimit),
GasUsed: Uint64String(h.ExecutionPayloadHeaderCapella.GasUsed),
Timestamp: Uint64String(h.ExecutionPayloadHeaderCapella.Timestamp),
ExtraData: h.ExecutionPayloadHeaderCapella.ExtraData,
BaseFeePerGas: baseFeePerGas,
BlockHash: h.ExecutionPayloadHeaderCapella.BlockHash,
TransactionsRoot: h.ExecutionPayloadHeaderCapella.TransactionsRoot,
WithdrawalsRoot: h.ExecutionPayloadHeaderCapella.WithdrawalsRoot,
})
}
func (h *ExecutionPayloadHeaderCapella) UnmarshalJSON(b []byte) error {
type UnmarshalCaller ExecutionPayloadHeaderCapella
uc := &UnmarshalCaller{}
if err := json.Unmarshal(b, uc); err != nil {
return err
}
ep := ExecutionPayloadHeaderCapella(*uc)
*h = ep
var err error
h.ExecutionPayloadHeaderCapella, err = h.ToProto()
return err
}
type ExecPayloadResponseCapella struct {
Version string `json:"version"`
Data ExecutionPayloadCapella `json:"data"`
}
type ExecutionPayloadCapella struct {
ParentHash hexutil.Bytes `json:"parent_hash"`
FeeRecipient hexutil.Bytes `json:"fee_recipient"`
StateRoot hexutil.Bytes `json:"state_root"`
ReceiptsRoot hexutil.Bytes `json:"receipts_root"`
LogsBloom hexutil.Bytes `json:"logs_bloom"`
PrevRandao hexutil.Bytes `json:"prev_randao"`
BlockNumber Uint64String `json:"block_number"`
GasLimit Uint64String `json:"gas_limit"`
GasUsed Uint64String `json:"gas_used"`
Timestamp Uint64String `json:"timestamp"`
ExtraData hexutil.Bytes `json:"extra_data"`
BaseFeePerGas Uint256 `json:"base_fee_per_gas"`
BlockHash hexutil.Bytes `json:"block_hash"`
Transactions []hexutil.Bytes `json:"transactions"`
Withdrawals []Withdrawal `json:"withdrawals"`
}
func (r *ExecPayloadResponseCapella) ToProto() (*v1.ExecutionPayloadCapella, error) {
return r.Data.ToProto()
}
func (p *ExecutionPayloadCapella) ToProto() (*v1.ExecutionPayloadCapella, error) {
txs := make([][]byte, len(p.Transactions))
for i := range p.Transactions {
txs[i] = p.Transactions[i]
}
withdrawals := make([]*v1.Withdrawal, len(p.Withdrawals))
for i, w := range p.Withdrawals {
withdrawals[i] = &v1.Withdrawal{
Index: w.Index.Uint64(),
ValidatorIndex: types.ValidatorIndex(w.ValidatorIndex.Uint64()),
Address: w.Address,
Amount: w.Amount.Uint64(),
}
}
return &v1.ExecutionPayloadCapella{
ParentHash: p.ParentHash,
FeeRecipient: p.FeeRecipient,
StateRoot: p.StateRoot,
ReceiptsRoot: p.ReceiptsRoot,
LogsBloom: p.LogsBloom,
PrevRandao: p.PrevRandao,
BlockNumber: uint64(p.BlockNumber),
GasLimit: uint64(p.GasLimit),
GasUsed: uint64(p.GasUsed),
Timestamp: uint64(p.Timestamp),
ExtraData: p.ExtraData,
BaseFeePerGas: p.BaseFeePerGas.SSZBytes(),
BlockHash: p.BlockHash,
Transactions: txs,
Withdrawals: withdrawals,
}, nil
}
type Withdrawal struct {
Index Uint256 `json:"index"`
ValidatorIndex Uint256 `json:"validator_index"`
Address hexutil.Bytes `json:"address"`
Amount Uint256 `json:"amount"`
}
type SignedBlindedBeaconBlockBellatrix struct {
*eth.SignedBlindedBeaconBlockBellatrix
}
type BlindedBeaconBlockBellatrix struct {
*eth.BlindedBeaconBlockBellatrix
}
type BlindedBeaconBlockBodyBellatrix struct {
*eth.BlindedBeaconBlockBodyBellatrix
}
func (r *SignedBlindedBeaconBlockBellatrix) MarshalJSON() ([]byte, error) {
return json.Marshal(struct {
Message *BlindedBeaconBlockBellatrix `json:"message"`
Signature hexutil.Bytes `json:"signature"`
}{
Message: &BlindedBeaconBlockBellatrix{r.SignedBlindedBeaconBlockBellatrix.Block},
Signature: r.SignedBlindedBeaconBlockBellatrix.Signature,
})
}
func (b *BlindedBeaconBlockBellatrix) MarshalJSON() ([]byte, error) {
return json.Marshal(struct {
Slot string `json:"slot"`
ProposerIndex string `json:"proposer_index"`
ParentRoot hexutil.Bytes `json:"parent_root"`
StateRoot hexutil.Bytes `json:"state_root"`
Body *BlindedBeaconBlockBodyBellatrix `json:"body"`
}{
Slot: fmt.Sprintf("%d", b.Slot),
ProposerIndex: fmt.Sprintf("%d", b.ProposerIndex),
ParentRoot: b.ParentRoot,
StateRoot: b.StateRoot,
Body: &BlindedBeaconBlockBodyBellatrix{b.BlindedBeaconBlockBellatrix.Body},
})
}
type ProposerSlashing struct {
*eth.ProposerSlashing
}
func (s *ProposerSlashing) MarshalJSON() ([]byte, error) {
return json.Marshal(struct {
SignedHeader1 *SignedBeaconBlockHeader `json:"signed_header_1"`
SignedHeader2 *SignedBeaconBlockHeader `json:"signed_header_2"`
}{
SignedHeader1: &SignedBeaconBlockHeader{s.ProposerSlashing.Header_1},
SignedHeader2: &SignedBeaconBlockHeader{s.ProposerSlashing.Header_2},
})
}
type SignedBeaconBlockHeader struct {
*eth.SignedBeaconBlockHeader
}
func (h *SignedBeaconBlockHeader) MarshalJSON() ([]byte, error) {
return json.Marshal(struct {
Header *BeaconBlockHeader `json:"message"`
Signature hexutil.Bytes `json:"signature"`
}{
Header: &BeaconBlockHeader{h.SignedBeaconBlockHeader.Header},
Signature: h.SignedBeaconBlockHeader.Signature,
})
}
type BeaconBlockHeader struct {
*eth.BeaconBlockHeader
}
func (h *BeaconBlockHeader) MarshalJSON() ([]byte, error) {
return json.Marshal(struct {
Slot string `json:"slot"`
ProposerIndex string `json:"proposer_index"`
ParentRoot hexutil.Bytes `json:"parent_root"`
StateRoot hexutil.Bytes `json:"state_root"`
BodyRoot hexutil.Bytes `json:"body_root"`
}{
Slot: fmt.Sprintf("%d", h.BeaconBlockHeader.Slot),
ProposerIndex: fmt.Sprintf("%d", h.BeaconBlockHeader.ProposerIndex),
ParentRoot: h.BeaconBlockHeader.ParentRoot,
StateRoot: h.BeaconBlockHeader.StateRoot,
BodyRoot: h.BeaconBlockHeader.BodyRoot,
})
}
type IndexedAttestation struct {
*eth.IndexedAttestation
}
func (a *IndexedAttestation) MarshalJSON() ([]byte, error) {
indices := make([]string, len(a.IndexedAttestation.AttestingIndices))
for i := range a.IndexedAttestation.AttestingIndices {
indices[i] = fmt.Sprintf("%d", a.AttestingIndices[i])
}
return json.Marshal(struct {
AttestingIndices []string `json:"attesting_indices"`
Data *AttestationData `json:"data"`
Signature hexutil.Bytes `json:"signature"`
}{
AttestingIndices: indices,
Data: &AttestationData{a.IndexedAttestation.Data},
Signature: a.IndexedAttestation.Signature,
})
}
type AttesterSlashing struct {
*eth.AttesterSlashing
}
func (s *AttesterSlashing) MarshalJSON() ([]byte, error) {
return json.Marshal(struct {
Attestation1 *IndexedAttestation `json:"attestation_1"`
Attestation2 *IndexedAttestation `json:"attestation_2"`
}{
Attestation1: &IndexedAttestation{s.Attestation_1},
Attestation2: &IndexedAttestation{s.Attestation_2},
})
}
type Checkpoint struct {
*eth.Checkpoint
}
func (c *Checkpoint) MarshalJSON() ([]byte, error) {
return json.Marshal(struct {
Epoch string `json:"epoch"`
Root hexutil.Bytes `json:"root"`
}{
Epoch: fmt.Sprintf("%d", c.Checkpoint.Epoch),
Root: c.Checkpoint.Root,
})
}
type AttestationData struct {
*eth.AttestationData
}
func (a *AttestationData) MarshalJSON() ([]byte, error) {
return json.Marshal(struct {
Slot string `json:"slot"`
Index string `json:"index"`
BeaconBlockRoot hexutil.Bytes `json:"beacon_block_root"`
Source *Checkpoint `json:"source"`
Target *Checkpoint `json:"target"`
}{
Slot: fmt.Sprintf("%d", a.AttestationData.Slot),
Index: fmt.Sprintf("%d", a.AttestationData.CommitteeIndex),
BeaconBlockRoot: a.AttestationData.BeaconBlockRoot,
Source: &Checkpoint{a.AttestationData.Source},
Target: &Checkpoint{a.AttestationData.Target},
})
}
type Attestation struct {
*eth.Attestation
}
func (a *Attestation) MarshalJSON() ([]byte, error) {
return json.Marshal(struct {
AggregationBits hexutil.Bytes `json:"aggregation_bits"`
Data *AttestationData `json:"data"`
Signature hexutil.Bytes `json:"signature" ssz-size:"96"`
}{
AggregationBits: hexutil.Bytes(a.Attestation.AggregationBits),
Data: &AttestationData{a.Attestation.Data},
Signature: a.Attestation.Signature,
})
}
type DepositData struct {
*eth.Deposit_Data
}
func (d *DepositData) MarshalJSON() ([]byte, error) {
return json.Marshal(struct {
PublicKey hexutil.Bytes `json:"pubkey"`
WithdrawalCredentials hexutil.Bytes `json:"withdrawal_credentials"`
Amount string `json:"amount"`
Signature hexutil.Bytes `json:"signature"`
}{
PublicKey: d.PublicKey,
WithdrawalCredentials: d.WithdrawalCredentials,
Amount: fmt.Sprintf("%d", d.Amount),
Signature: d.Signature,
})
}
type Deposit struct {
*eth.Deposit
}
func (d *Deposit) MarshalJSON() ([]byte, error) {
proof := make([]hexutil.Bytes, len(d.Proof))
for i := range d.Proof {
proof[i] = d.Proof[i]
}
return json.Marshal(struct {
Proof []hexutil.Bytes `json:"proof"`
Data *DepositData `json:"data"`
}{
Proof: proof,
Data: &DepositData{Deposit_Data: d.Deposit.Data},
})
}
type SignedVoluntaryExit struct {
*eth.SignedVoluntaryExit
}
func (sve *SignedVoluntaryExit) MarshalJSON() ([]byte, error) {
return json.Marshal(struct {
Message *VoluntaryExit `json:"message"`
Signature hexutil.Bytes `json:"signature"`
}{
Signature: sve.SignedVoluntaryExit.Signature,
Message: &VoluntaryExit{sve.SignedVoluntaryExit.Exit},
})
}
type VoluntaryExit struct {
*eth.VoluntaryExit
}
func (ve *VoluntaryExit) MarshalJSON() ([]byte, error) {
return json.Marshal(struct {
Epoch string `json:"epoch"`
ValidatorIndex string `json:"validator_index"`
}{
Epoch: fmt.Sprintf("%d", ve.Epoch),
ValidatorIndex: fmt.Sprintf("%d", ve.ValidatorIndex),
})
}
type SyncAggregate struct {
*eth.SyncAggregate
}
func (s *SyncAggregate) MarshalJSON() ([]byte, error) {
return json.Marshal(struct {
SyncCommitteeBits hexutil.Bytes `json:"sync_committee_bits"`
SyncCommitteeSignature hexutil.Bytes `json:"sync_committee_signature"`
}{
SyncCommitteeBits: hexutil.Bytes(s.SyncAggregate.SyncCommitteeBits),
SyncCommitteeSignature: s.SyncAggregate.SyncCommitteeSignature,
})
}
type Eth1Data struct {
*eth.Eth1Data
}
func (e *Eth1Data) MarshalJSON() ([]byte, error) {
return json.Marshal(struct {
DepositRoot hexutil.Bytes `json:"deposit_root"`
DepositCount string `json:"deposit_count"`
BlockHash hexutil.Bytes `json:"block_hash"`
}{
DepositRoot: e.DepositRoot,
DepositCount: fmt.Sprintf("%d", e.DepositCount),
BlockHash: e.BlockHash,
})
}
func (b *BlindedBeaconBlockBodyBellatrix) MarshalJSON() ([]byte, error) {
sve := make([]*SignedVoluntaryExit, len(b.BlindedBeaconBlockBodyBellatrix.VoluntaryExits))
for i := range b.BlindedBeaconBlockBodyBellatrix.VoluntaryExits {
sve[i] = &SignedVoluntaryExit{SignedVoluntaryExit: b.BlindedBeaconBlockBodyBellatrix.VoluntaryExits[i]}
}
deps := make([]*Deposit, len(b.BlindedBeaconBlockBodyBellatrix.Deposits))
for i := range b.BlindedBeaconBlockBodyBellatrix.Deposits {
deps[i] = &Deposit{Deposit: b.BlindedBeaconBlockBodyBellatrix.Deposits[i]}
}
atts := make([]*Attestation, len(b.BlindedBeaconBlockBodyBellatrix.Attestations))
for i := range b.BlindedBeaconBlockBodyBellatrix.Attestations {
atts[i] = &Attestation{Attestation: b.BlindedBeaconBlockBodyBellatrix.Attestations[i]}
}
atsl := make([]*AttesterSlashing, len(b.BlindedBeaconBlockBodyBellatrix.AttesterSlashings))
for i := range b.BlindedBeaconBlockBodyBellatrix.AttesterSlashings {
atsl[i] = &AttesterSlashing{AttesterSlashing: b.BlindedBeaconBlockBodyBellatrix.AttesterSlashings[i]}
}
pros := make([]*ProposerSlashing, len(b.BlindedBeaconBlockBodyBellatrix.ProposerSlashings))
for i := range b.BlindedBeaconBlockBodyBellatrix.ProposerSlashings {
pros[i] = &ProposerSlashing{ProposerSlashing: b.BlindedBeaconBlockBodyBellatrix.ProposerSlashings[i]}
}
return json.Marshal(struct {
RandaoReveal hexutil.Bytes `json:"randao_reveal"`
Eth1Data *Eth1Data `json:"eth1_data"`
Graffiti hexutil.Bytes `json:"graffiti"`
ProposerSlashings []*ProposerSlashing `json:"proposer_slashings"`
AttesterSlashings []*AttesterSlashing `json:"attester_slashings"`
Attestations []*Attestation `json:"attestations"`
Deposits []*Deposit `json:"deposits"`
VoluntaryExits []*SignedVoluntaryExit `json:"voluntary_exits"`
SyncAggregate *SyncAggregate `json:"sync_aggregate"`
ExecutionPayloadHeader *ExecutionPayloadHeader `json:"execution_payload_header"`
}{
RandaoReveal: b.RandaoReveal,
Eth1Data: &Eth1Data{b.BlindedBeaconBlockBodyBellatrix.Eth1Data},
Graffiti: b.BlindedBeaconBlockBodyBellatrix.Graffiti,
ProposerSlashings: pros,
AttesterSlashings: atsl,
Attestations: atts,
Deposits: deps,
VoluntaryExits: sve,
SyncAggregate: &SyncAggregate{b.BlindedBeaconBlockBodyBellatrix.SyncAggregate},
ExecutionPayloadHeader: &ExecutionPayloadHeader{ExecutionPayloadHeader: b.BlindedBeaconBlockBodyBellatrix.ExecutionPayloadHeader},
})
}
type SignedBLSToExecutionChange struct {
*eth.SignedBLSToExecutionChange
}
func (ch *SignedBLSToExecutionChange) MarshalJSON() ([]byte, error) {
return json.Marshal(struct {
Message *BLSToExecutionChange `json:"message"`
Signature hexutil.Bytes `json:"signature"`
}{
Signature: ch.Signature,
Message: &BLSToExecutionChange{ch.Message},
})
}
type BLSToExecutionChange struct {
*eth.BLSToExecutionChange
}
func (ch *BLSToExecutionChange) MarshalJSON() ([]byte, error) {
return json.Marshal(struct {
ValidatorIndex string `json:"validator_index"`
FromBlsPubkey hexutil.Bytes `json:"from_bls_pubkey"`
ToExecutionAddress hexutil.Bytes `json:"to_execution_address"`
}{
ValidatorIndex: fmt.Sprintf("%d", ch.ValidatorIndex),
FromBlsPubkey: ch.FromBlsPubkey,
ToExecutionAddress: ch.ToExecutionAddress,
})
}
type SignedBlindedBeaconBlockCapella struct {
*eth.SignedBlindedBeaconBlockCapella
}
type BlindedBeaconBlockCapella struct {
*eth.BlindedBeaconBlockCapella
}
type BlindedBeaconBlockBodyCapella struct {
*eth.BlindedBeaconBlockBodyCapella
}
func (b *SignedBlindedBeaconBlockCapella) MarshalJSON() ([]byte, error) {
return json.Marshal(struct {
Message *BlindedBeaconBlockCapella `json:"message"`
Signature hexutil.Bytes `json:"signature"`
}{
Message: &BlindedBeaconBlockCapella{b.Block},
Signature: b.Signature,
})
}
func (b *BlindedBeaconBlockCapella) MarshalJSON() ([]byte, error) {
return json.Marshal(struct {
Slot string `json:"slot"`
ProposerIndex string `json:"proposer_index"`
ParentRoot hexutil.Bytes `json:"parent_root"`
StateRoot hexutil.Bytes `json:"state_root"`
Body *BlindedBeaconBlockBodyCapella `json:"body"`
}{
Slot: fmt.Sprintf("%d", b.Slot),
ProposerIndex: fmt.Sprintf("%d", b.ProposerIndex),
ParentRoot: b.ParentRoot,
StateRoot: b.StateRoot,
Body: &BlindedBeaconBlockBodyCapella{b.Body},
})
}
func (b *BlindedBeaconBlockBodyCapella) MarshalJSON() ([]byte, error) {
sve := make([]*SignedVoluntaryExit, len(b.VoluntaryExits))
for i := range b.VoluntaryExits {
sve[i] = &SignedVoluntaryExit{SignedVoluntaryExit: b.VoluntaryExits[i]}
}
deps := make([]*Deposit, len(b.Deposits))
for i := range b.Deposits {
deps[i] = &Deposit{Deposit: b.Deposits[i]}
}
atts := make([]*Attestation, len(b.Attestations))
for i := range b.Attestations {
atts[i] = &Attestation{Attestation: b.Attestations[i]}
}
atsl := make([]*AttesterSlashing, len(b.AttesterSlashings))
for i := range b.AttesterSlashings {
atsl[i] = &AttesterSlashing{AttesterSlashing: b.AttesterSlashings[i]}
}
pros := make([]*ProposerSlashing, len(b.ProposerSlashings))
for i := range b.ProposerSlashings {
pros[i] = &ProposerSlashing{ProposerSlashing: b.ProposerSlashings[i]}
}
chs := make([]*SignedBLSToExecutionChange, len(b.BlsToExecutionChanges))
for i := range b.BlsToExecutionChanges {
chs[i] = &SignedBLSToExecutionChange{SignedBLSToExecutionChange: b.BlsToExecutionChanges[i]}
}
return json.Marshal(struct {
RandaoReveal hexutil.Bytes `json:"randao_reveal"`
Eth1Data *Eth1Data `json:"eth1_data"`
Graffiti hexutil.Bytes `json:"graffiti"`
ProposerSlashings []*ProposerSlashing `json:"proposer_slashings"`
AttesterSlashings []*AttesterSlashing `json:"attester_slashings"`
Attestations []*Attestation `json:"attestations"`
Deposits []*Deposit `json:"deposits"`
VoluntaryExits []*SignedVoluntaryExit `json:"voluntary_exits"`
BLSToExecutionChanges []*SignedBLSToExecutionChange `json:"bls_to_execution_changes"`
SyncAggregate *SyncAggregate `json:"sync_aggregate"`
ExecutionPayloadHeader *ExecutionPayloadHeaderCapella `json:"execution_payload_header"`
}{
RandaoReveal: b.RandaoReveal,
Eth1Data: &Eth1Data{b.Eth1Data},
Graffiti: b.Graffiti,
ProposerSlashings: pros,
AttesterSlashings: atsl,
Attestations: atts,
Deposits: deps,
VoluntaryExits: sve,
BLSToExecutionChanges: chs,
SyncAggregate: &SyncAggregate{b.SyncAggregate},
ExecutionPayloadHeader: &ExecutionPayloadHeaderCapella{ExecutionPayloadHeaderCapella: b.ExecutionPayloadHeader},
})
}
type ErrorMessage struct {
Code int `json:"code"`
Message string `json:"message"`
Stacktraces []string `json:"stacktraces,omitempty"`
}

File diff suppressed because it is too large Load Diff

44
api/gateway/BUILD.bazel Normal file
View File

@@ -0,0 +1,44 @@
load("@prysm//tools/go:def.bzl", "go_library", "go_test")
go_library(
name = "go_default_library",
srcs = [
"gateway.go",
"log.go",
"modifiers.go",
"options.go",
],
importpath = "github.com/prysmaticlabs/prysm/v3/api/gateway",
visibility = [
"//beacon-chain:__subpackages__",
"//validator:__subpackages__",
],
deps = [
"//api/gateway/apimiddleware:go_default_library",
"//runtime:go_default_library",
"@com_github_gorilla_mux//:go_default_library",
"@com_github_grpc_ecosystem_grpc_gateway_v2//runtime:go_default_library",
"@com_github_pkg_errors//:go_default_library",
"@com_github_rs_cors//:go_default_library",
"@com_github_sirupsen_logrus//:go_default_library",
"@org_golang_google_grpc//:go_default_library",
"@org_golang_google_grpc//connectivity:go_default_library",
"@org_golang_google_grpc//credentials:go_default_library",
"@org_golang_google_protobuf//proto:go_default_library",
],
)
go_test(
name = "go_default_test",
srcs = ["gateway_test.go"],
embed = [":go_default_library"],
deps = [
"//api/gateway/apimiddleware:go_default_library",
"//cmd/beacon-chain/flags:go_default_library",
"//testing/assert:go_default_library",
"//testing/require:go_default_library",
"@com_github_gorilla_mux//:go_default_library",
"@com_github_sirupsen_logrus//hooks/test:go_default_library",
"@com_github_urfave_cli_v2//:go_default_library",
],
)

View File

@@ -0,0 +1,41 @@
load("@prysm//tools/go:def.bzl", "go_library", "go_test")
go_library(
name = "go_default_library",
srcs = [
"api_middleware.go",
"log.go",
"param_handling.go",
"process_field.go",
"process_request.go",
"structs.go",
],
importpath = "github.com/prysmaticlabs/prysm/v3/api/gateway/apimiddleware",
visibility = ["//visibility:public"],
deps = [
"//api/grpc:go_default_library",
"//encoding/bytesutil:go_default_library",
"@com_github_ethereum_go_ethereum//common:go_default_library",
"@com_github_ethereum_go_ethereum//common/hexutil:go_default_library",
"@com_github_gorilla_mux//:go_default_library",
"@com_github_pkg_errors//:go_default_library",
"@com_github_sirupsen_logrus//:go_default_library",
"@com_github_wealdtech_go_bytesutil//:go_default_library",
],
)
go_test(
name = "go_default_test",
srcs = [
"param_handling_test.go",
"process_request_test.go",
],
embed = [":go_default_library"],
deps = [
"//api/grpc:go_default_library",
"//testing/assert:go_default_library",
"//testing/require:go_default_library",
"@com_github_gorilla_mux//:go_default_library",
"@com_github_sirupsen_logrus//hooks/test:go_default_library",
],
)

View File

@@ -0,0 +1,265 @@
package apimiddleware
import (
"net/http"
"reflect"
"time"
"github.com/gorilla/mux"
)
// ApiProxyMiddleware is a proxy between an Ethereum consensus API HTTP client and grpc-gateway.
// The purpose of the proxy is to handle HTTP requests and gRPC responses in such a way that:
// - Ethereum consensus API requests can be handled by grpc-gateway correctly
// - gRPC responses can be returned as spec-compliant Ethereum consensus API responses
type ApiProxyMiddleware struct {
GatewayAddress string
EndpointCreator EndpointFactory
Timeout time.Duration
router *mux.Router
}
// EndpointFactory is responsible for creating new instances of Endpoint values.
type EndpointFactory interface {
Create(path string) (*Endpoint, error)
Paths() []string
IsNil() bool
}
// Endpoint is a representation of an API HTTP endpoint that should be proxied by the middleware.
type Endpoint struct {
Path string // The path of the HTTP endpoint.
GetResponse interface{} // The struct corresponding to the JSON structure used in a GET response.
PostRequest interface{} // The struct corresponding to the JSON structure used in a POST request.
PostResponse interface{} // The struct corresponding to the JSON structure used in a POST response.
DeleteRequest interface{} // The struct corresponding to the JSON structure used in a DELETE request.
DeleteResponse interface{} // The struct corresponding to the JSON structure used in a DELETE response.
RequestURLLiterals []string // Names of URL parameters that should not be base64-encoded.
RequestQueryParams []QueryParam // Query parameters of the request.
Err ErrorJson // The struct corresponding to the error that should be returned in case of a request failure.
Hooks HookCollection // A collection of functions that can be invoked at various stages of the request/response cycle.
CustomHandlers []CustomHandler // Functions that will be executed instead of the default request/response behaviour.
}
// RunDefault expresses whether the default processing logic should be carried out after running a pre hook.
type RunDefault bool
// DefaultEndpoint returns an Endpoint with default configuration, e.g. DefaultErrorJson for error handling.
func DefaultEndpoint() Endpoint {
return Endpoint{
Err: &DefaultErrorJson{},
}
}
// QueryParam represents a single query parameter's metadata.
type QueryParam struct {
Name string
Hex bool
Enum bool
}
// CustomHandler is a function that can be invoked at the very beginning of the request,
// essentially replacing the whole default request/response logic with custom logic for a specific endpoint.
type CustomHandler = func(m *ApiProxyMiddleware, endpoint Endpoint, w http.ResponseWriter, req *http.Request) (handled bool)
// HookCollection contains hooks that can be used to amend the default request/response cycle with custom logic for a specific endpoint.
type HookCollection struct {
OnPreDeserializeRequestBodyIntoContainer func(endpoint *Endpoint, w http.ResponseWriter, req *http.Request) (RunDefault, ErrorJson)
OnPostDeserializeRequestBodyIntoContainer func(endpoint *Endpoint, w http.ResponseWriter, req *http.Request) ErrorJson
OnPreDeserializeGrpcResponseBodyIntoContainer func([]byte, interface{}) (RunDefault, ErrorJson)
OnPreSerializeMiddlewareResponseIntoJson func(interface{}) (RunDefault, []byte, ErrorJson)
}
// fieldProcessor applies the processing function f to a value when the tag is present on the field.
type fieldProcessor struct {
tag string
f func(value reflect.Value) error
}
// Run starts the proxy, registering all proxy endpoints.
func (m *ApiProxyMiddleware) Run(gatewayRouter *mux.Router) {
for _, path := range m.EndpointCreator.Paths() {
gatewayRouter.HandleFunc(path, m.WithMiddleware(path))
}
m.router = gatewayRouter
}
// ServeHTTP for the proxy middleware.
func (m *ApiProxyMiddleware) ServeHTTP(w http.ResponseWriter, req *http.Request) {
m.router.ServeHTTP(w, req)
}
// WithMiddleware wraps the given endpoint handler with the middleware logic.
func (m *ApiProxyMiddleware) WithMiddleware(path string) http.HandlerFunc {
return func(w http.ResponseWriter, req *http.Request) {
endpoint, err := m.EndpointCreator.Create(path)
if err != nil {
log.WithError(err).Errorf("Could not create endpoint for path: %s", path)
return
}
for _, handler := range endpoint.CustomHandlers {
if handler(m, *endpoint, w, req) {
return
}
}
if req.Method == "POST" {
if errJson := handlePostRequestForEndpoint(endpoint, w, req); errJson != nil {
WriteError(w, errJson, nil)
return
}
}
if req.Method == "DELETE" && req.Body != http.NoBody {
if errJson := handleDeleteRequestForEndpoint(endpoint, req); errJson != nil {
WriteError(w, errJson, nil)
return
}
}
if errJson := m.PrepareRequestForProxying(*endpoint, req); errJson != nil {
WriteError(w, errJson, nil)
return
}
grpcResp, errJson := m.ProxyRequest(req)
if errJson != nil {
WriteError(w, errJson, nil)
return
}
grpcRespBody, errJson := ReadGrpcResponseBody(grpcResp.Body)
if errJson != nil {
WriteError(w, errJson, nil)
return
}
var respJson []byte
if !GrpcResponseIsEmpty(grpcRespBody) {
respHasError, errJson := HandleGrpcResponseError(endpoint.Err, grpcResp, grpcRespBody, w)
if errJson != nil {
WriteError(w, errJson, nil)
return
}
if respHasError {
return
}
var resp interface{}
if req.Method == "GET" {
resp = endpoint.GetResponse
} else if req.Method == "DELETE" {
resp = endpoint.DeleteResponse
} else {
resp = endpoint.PostResponse
}
if errJson := deserializeGrpcResponseBodyIntoContainerWrapped(endpoint, grpcRespBody, resp); errJson != nil {
WriteError(w, errJson, nil)
return
}
if errJson := ProcessMiddlewareResponseFields(resp); errJson != nil {
WriteError(w, errJson, nil)
return
}
respJson, errJson = serializeMiddlewareResponseIntoJsonWrapped(endpoint, respJson, resp)
if errJson != nil {
WriteError(w, errJson, nil)
return
}
}
if errJson := WriteMiddlewareResponseHeadersAndBody(grpcResp, respJson, w); errJson != nil {
WriteError(w, errJson, nil)
return
}
if errJson := Cleanup(grpcResp.Body); errJson != nil {
WriteError(w, errJson, nil)
return
}
}
}
func handlePostRequestForEndpoint(endpoint *Endpoint, w http.ResponseWriter, req *http.Request) ErrorJson {
if errJson := deserializeRequestBodyIntoContainerWrapped(endpoint, req, w); errJson != nil {
return errJson
}
if errJson := ProcessRequestContainerFields(endpoint.PostRequest); errJson != nil {
return errJson
}
return SetRequestBodyToRequestContainer(endpoint.PostRequest, req)
}
func handleDeleteRequestForEndpoint(endpoint *Endpoint, req *http.Request) ErrorJson {
if errJson := DeserializeRequestBodyIntoContainer(req.Body, endpoint.DeleteRequest); errJson != nil {
return errJson
}
if errJson := ProcessRequestContainerFields(endpoint.DeleteRequest); errJson != nil {
return errJson
}
return SetRequestBodyToRequestContainer(endpoint.DeleteRequest, req)
}
func deserializeRequestBodyIntoContainerWrapped(endpoint *Endpoint, req *http.Request, w http.ResponseWriter) ErrorJson {
runDefault := true
if endpoint.Hooks.OnPreDeserializeRequestBodyIntoContainer != nil {
run, errJson := endpoint.Hooks.OnPreDeserializeRequestBodyIntoContainer(endpoint, w, req)
if errJson != nil {
return errJson
}
if !run {
runDefault = false
}
}
if runDefault {
if errJson := DeserializeRequestBodyIntoContainer(req.Body, endpoint.PostRequest); errJson != nil {
return errJson
}
}
if endpoint.Hooks.OnPostDeserializeRequestBodyIntoContainer != nil {
if errJson := endpoint.Hooks.OnPostDeserializeRequestBodyIntoContainer(endpoint, w, req); errJson != nil {
return errJson
}
}
return nil
}
func deserializeGrpcResponseBodyIntoContainerWrapped(endpoint *Endpoint, grpcResponseBody []byte, resp interface{}) ErrorJson {
runDefault := true
if endpoint.Hooks.OnPreDeserializeGrpcResponseBodyIntoContainer != nil {
run, errJson := endpoint.Hooks.OnPreDeserializeGrpcResponseBodyIntoContainer(grpcResponseBody, resp)
if errJson != nil {
return errJson
}
if !run {
runDefault = false
}
}
if runDefault {
if errJson := DeserializeGrpcResponseBodyIntoContainer(grpcResponseBody, resp); errJson != nil {
return errJson
}
}
return nil
}
func serializeMiddlewareResponseIntoJsonWrapped(endpoint *Endpoint, respJson []byte, resp interface{}) ([]byte, ErrorJson) {
runDefault := true
var errJson ErrorJson
if endpoint.Hooks.OnPreSerializeMiddlewareResponseIntoJson != nil {
var run RunDefault
run, respJson, errJson = endpoint.Hooks.OnPreSerializeMiddlewareResponseIntoJson(resp)
if errJson != nil {
return nil, errJson
}
if !run {
runDefault = false
}
}
if runDefault {
respJson, errJson = SerializeMiddlewareResponseIntoJson(resp)
if errJson != nil {
return nil, errJson
}
}
return respJson, nil
}

View File

@@ -0,0 +1,5 @@
package apimiddleware
import "github.com/sirupsen/logrus"
var log = logrus.WithField("prefix", "apimiddleware")

View File

@@ -0,0 +1,103 @@
package apimiddleware
import (
"encoding/base64"
"net/http"
"net/url"
"strings"
"github.com/gorilla/mux"
butil "github.com/prysmaticlabs/prysm/v3/encoding/bytesutil"
"github.com/wealdtech/go-bytesutil"
)
// HandleURLParameters processes URL parameters, allowing parameterized URLs to be safely and correctly proxied to grpc-gateway.
func HandleURLParameters(url string, req *http.Request, literals []string) ErrorJson {
segments := strings.Split(url, "/")
segmentsLoop:
for i, s := range segments {
// We only care about segments which are parameterized.
if isRequestParam(s) {
// Don't do anything with parameters which should be forwarded literally to gRPC.
for _, l := range literals {
if s == "{"+l+"}" {
continue segmentsLoop
}
}
routeVar := mux.Vars(req)[s[1:len(s)-1]]
bRouteVar := []byte(routeVar)
if butil.IsHex(bRouteVar) {
var err error
bRouteVar, err = bytesutil.FromHexString(string(bRouteVar))
if err != nil {
return InternalServerErrorWithMessage(err, "could not process URL parameter")
}
}
// Converting hex to base64 may result in a value which malforms the URL.
// We use URLEncoding to safely escape such values.
base64RouteVar := base64.URLEncoding.EncodeToString(bRouteVar)
// Merge segments back into the full URL.
splitPath := strings.Split(req.URL.Path, "/")
splitPath[i] = base64RouteVar
req.URL.Path = strings.Join(splitPath, "/")
}
}
return nil
}
// HandleQueryParameters processes query parameters, allowing them to be safely and correctly proxied to grpc-gateway.
func HandleQueryParameters(req *http.Request, params []QueryParam) ErrorJson {
queryParams := req.URL.Query()
normalizeQueryValues(queryParams)
for key, vals := range queryParams {
for _, p := range params {
if key == p.Name {
if p.Hex {
queryParams.Del(key)
for _, v := range vals {
b := []byte(v)
if butil.IsHex(b) {
var err error
b, err = bytesutil.FromHexString(v)
if err != nil {
return InternalServerErrorWithMessage(err, "could not process query parameter")
}
}
queryParams.Add(key, base64.URLEncoding.EncodeToString(b))
}
}
if p.Enum {
queryParams.Del(key)
for _, v := range vals {
// gRPC expects uppercase enum values.
queryParams.Add(key, strings.ToUpper(v))
}
}
}
}
}
req.URL.RawQuery = queryParams.Encode()
return nil
}
// isRequestParam verifies whether the passed string is a request parameter.
// Request parameters are enclosed in { and }.
func isRequestParam(s string) bool {
return len(s) > 2 && s[0] == '{' && s[len(s)-1] == '}'
}
func normalizeQueryValues(queryParams url.Values) {
// Replace comma-separated values with individual values.
for key, vals := range queryParams {
splitVals := make([]string, 0)
for _, v := range vals {
splitVals = append(splitVals, strings.Split(v, ",")...)
}
queryParams[key] = splitVals
}
}

View File

@@ -0,0 +1,124 @@
package apimiddleware
import (
"bytes"
"net/http/httptest"
"testing"
"github.com/gorilla/mux"
"github.com/prysmaticlabs/prysm/v3/testing/assert"
"github.com/prysmaticlabs/prysm/v3/testing/require"
)
func TestHandleURLParameters(t *testing.T) {
var body bytes.Buffer
t.Run("no_params", func(t *testing.T) {
request := httptest.NewRequest("GET", "http://foo.example/bar", &body)
errJson := HandleURLParameters("/not_param", request, []string{})
require.Equal(t, true, errJson == nil)
assert.Equal(t, "/bar", request.URL.Path)
})
t.Run("with_params", func(t *testing.T) {
muxVars := make(map[string]string)
muxVars["bar_param"] = "bar"
muxVars["quux_param"] = "quux"
request := httptest.NewRequest("GET", "http://foo.example/bar/baz/quux", &body)
request = mux.SetURLVars(request, muxVars)
errJson := HandleURLParameters("/{bar_param}/not_param/{quux_param}", request, []string{})
require.Equal(t, true, errJson == nil)
assert.Equal(t, "/YmFy/baz/cXV1eA==", request.URL.Path)
})
t.Run("with_literal", func(t *testing.T) {
muxVars := make(map[string]string)
muxVars["bar_param"] = "bar"
request := httptest.NewRequest("GET", "http://foo.example/bar/baz", &body)
request = mux.SetURLVars(request, muxVars)
errJson := HandleURLParameters("/{bar_param}/not_param/", request, []string{"bar_param"})
require.Equal(t, true, errJson == nil)
assert.Equal(t, "/bar/baz", request.URL.Path)
})
t.Run("with_hex", func(t *testing.T) {
muxVars := make(map[string]string)
muxVars["hex_param"] = "0x626172"
request := httptest.NewRequest("GET", "http://foo.example/0x626172/baz", &body)
request = mux.SetURLVars(request, muxVars)
errJson := HandleURLParameters("/{hex_param}/not_param/", request, []string{})
require.Equal(t, true, errJson == nil)
assert.Equal(t, "/YmFy/baz", request.URL.Path)
})
}
func TestHandleQueryParameters(t *testing.T) {
var body bytes.Buffer
t.Run("regular_params", func(t *testing.T) {
request := httptest.NewRequest("GET", "http://foo.example?bar=bar&baz=baz", &body)
errJson := HandleQueryParameters(request, []QueryParam{{Name: "bar"}, {Name: "baz"}})
require.Equal(t, true, errJson == nil)
query := request.URL.Query()
v, ok := query["bar"]
require.Equal(t, true, ok, "query param not found")
require.Equal(t, 1, len(v), "wrong number of query param values")
assert.Equal(t, "bar", v[0])
v, ok = query["baz"]
require.Equal(t, true, ok, "query param not found")
require.Equal(t, 1, len(v), "wrong number of query param values")
assert.Equal(t, "baz", v[0])
})
t.Run("hex_and_enum_params", func(t *testing.T) {
request := httptest.NewRequest("GET", "http://foo.example?hex=0x626172&baz=baz", &body)
errJson := HandleQueryParameters(request, []QueryParam{{Name: "hex", Hex: true}, {Name: "baz", Enum: true}})
require.Equal(t, true, errJson == nil)
query := request.URL.Query()
v, ok := query["hex"]
require.Equal(t, true, ok, "query param not found")
require.Equal(t, 1, len(v), "wrong number of query param values")
assert.Equal(t, "YmFy", v[0])
v, ok = query["baz"]
require.Equal(t, true, ok, "query param not found")
require.Equal(t, 1, len(v), "wrong number of query param values")
assert.Equal(t, "BAZ", v[0])
})
}
func TestIsRequestParam(t *testing.T) {
tests := []struct {
s string
b bool
}{
{"", false},
{"{", false},
{"}", false},
{"{}", false},
{"{x}", true},
{"{very_long_parameter_name_with_underscores}", true},
}
for _, tt := range tests {
b := isRequestParam(tt.s)
assert.Equal(t, tt.b, b)
}
}
func TestNormalizeQueryValues(t *testing.T) {
input := make(map[string][]string)
input["key"] = []string{"value1", "value2,value3,value4", "value5"}
normalizeQueryValues(input)
require.Equal(t, 5, len(input["key"]))
assert.Equal(t, "value1", input["key"][0])
assert.Equal(t, "value2", input["key"][1])
assert.Equal(t, "value3", input["key"][2])
assert.Equal(t, "value4", input["key"][3])
assert.Equal(t, "value5", input["key"][4])
}

View File

@@ -0,0 +1,179 @@
package apimiddleware
import (
"encoding/base64"
"fmt"
"math/big"
"reflect"
"strconv"
"strings"
"time"
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/common/hexutil"
"github.com/pkg/errors"
"github.com/wealdtech/go-bytesutil"
)
// processField calls each processor function on any field that has the matching tag set.
// It is a recursive function.
func processField(s interface{}, processors []fieldProcessor) error {
kind := reflect.TypeOf(s).Kind()
if kind != reflect.Ptr && kind != reflect.Slice && kind != reflect.Array {
return fmt.Errorf("processing fields of kind '%v' is unsupported", kind)
}
t := reflect.TypeOf(s).Elem()
v := reflect.Indirect(reflect.ValueOf(s))
for i := 0; i < t.NumField(); i++ {
switch v.Field(i).Kind() {
case reflect.Slice:
sliceElem := t.Field(i).Type.Elem()
kind := sliceElem.Kind()
// Recursively process slices to struct pointers.
switch {
case kind == reflect.Ptr && sliceElem.Elem().Kind() == reflect.Struct:
for j := 0; j < v.Field(i).Len(); j++ {
if err := processField(v.Field(i).Index(j).Interface(), processors); err != nil {
return errors.Wrapf(err, "could not process field '%s'", t.Field(i).Name)
}
}
// Process each string in string slices.
case kind == reflect.String:
for _, proc := range processors {
_, hasTag := t.Field(i).Tag.Lookup(proc.tag)
if !hasTag {
continue
}
for j := 0; j < v.Field(i).Len(); j++ {
if err := proc.f(v.Field(i).Index(j)); err != nil {
return errors.Wrapf(err, "could not process field '%s'", t.Field(i).Name)
}
}
}
}
// Recursively process struct pointers.
case reflect.Ptr:
if v.Field(i).Elem().Kind() == reflect.Struct {
if err := processField(v.Field(i).Interface(), processors); err != nil {
return errors.Wrapf(err, "could not process field '%s'", t.Field(i).Name)
}
}
default:
field := t.Field(i)
for _, proc := range processors {
if _, hasTag := field.Tag.Lookup(proc.tag); hasTag {
if err := proc.f(v.Field(i)); err != nil {
return errors.Wrapf(err, "could not process field '%s'", t.Field(i).Name)
}
}
}
}
}
return nil
}
func hexToBase64Processor(v reflect.Value) error {
if v.String() == "0x" {
v.SetString("")
return nil
}
b, err := bytesutil.FromHexString(v.String())
if err != nil {
return err
}
v.SetString(base64.StdEncoding.EncodeToString(b))
return nil
}
func base64ToHexProcessor(v reflect.Value) error {
if v.String() == "" {
// Empty hex values are represented as "0x".
v.SetString("0x")
return nil
}
b, err := base64.StdEncoding.DecodeString(v.String())
if err != nil {
return err
}
v.SetString(hexutil.Encode(b))
return nil
}
func base64ToChecksumAddressProcessor(v reflect.Value) error {
if v.String() == "" {
// Empty hex values are represented as "0x".
v.SetString("0x")
return nil
}
b, err := base64.StdEncoding.DecodeString(v.String())
if err != nil {
return err
}
v.SetString(common.BytesToAddress(b).Hex())
return nil
}
func base64ToUint256Processor(v reflect.Value) error {
if v.String() == "" {
return nil
}
littleEndian, err := base64.StdEncoding.DecodeString(v.String())
if err != nil {
return err
}
if len(littleEndian) != 32 {
return errors.New("invalid length for Uint256")
}
// Integers are stored as little-endian, but
// big.Int expects big-endian. So we need to reverse
// the byte order before decoding.
var bigEndian [32]byte
for i := 0; i < len(littleEndian); i++ {
bigEndian[i] = littleEndian[len(littleEndian)-1-i]
}
var uint256 big.Int
uint256.SetBytes(bigEndian[:])
v.SetString(uint256.String())
return nil
}
func uint256ToBase64Processor(v reflect.Value) error {
if v.String() == "" {
return nil
}
uint256, ok := new(big.Int).SetString(v.String(), 10)
if !ok {
return fmt.Errorf("could not parse Uint256")
}
bigEndian := uint256.Bytes()
if len(bigEndian) > 32 {
return fmt.Errorf("number too big for Uint256")
}
// Integers are stored as little-endian, but
// big.Int gives big-endian. So we need to reverse
// the byte order before encoding.
var littleEndian [32]byte
for i := 0; i < len(bigEndian); i++ {
littleEndian[i] = bigEndian[len(bigEndian)-1-i]
}
v.SetString(base64.StdEncoding.EncodeToString(littleEndian[:]))
return nil
}
func enumToLowercaseProcessor(v reflect.Value) error {
v.SetString(strings.ToLower(v.String()))
return nil
}
func timeToUnixProcessor(v reflect.Value) error {
t, err := time.Parse(time.RFC3339, v.String())
if err != nil {
return err
}
v.SetString(strconv.FormatUint(uint64(t.Unix()), 10))
return nil
}

View File

@@ -0,0 +1,276 @@
package apimiddleware
import (
"bytes"
"encoding/json"
"io"
"net"
"net/http"
"strconv"
"strings"
"github.com/pkg/errors"
"github.com/prysmaticlabs/prysm/v3/api/grpc"
)
// DeserializeRequestBodyIntoContainer deserializes the request's body into an endpoint-specific struct.
func DeserializeRequestBodyIntoContainer(body io.Reader, requestContainer interface{}) ErrorJson {
decoder := json.NewDecoder(body)
decoder.DisallowUnknownFields()
if err := decoder.Decode(&requestContainer); err != nil {
if strings.Contains(err.Error(), "json: unknown field") {
e := errors.Wrap(err, "could not decode request body")
return &DefaultErrorJson{
Message: e.Error(),
Code: http.StatusBadRequest,
}
}
return InternalServerErrorWithMessage(err, "could not decode request body")
}
return nil
}
// ProcessRequestContainerFields processes fields of an endpoint-specific container according to field tags.
func ProcessRequestContainerFields(requestContainer interface{}) ErrorJson {
if err := processField(requestContainer, []fieldProcessor{
{
tag: "hex",
f: hexToBase64Processor,
},
{
tag: "uint256",
f: uint256ToBase64Processor,
},
}); err != nil {
return InternalServerErrorWithMessage(err, "could not process request data")
}
return nil
}
// SetRequestBodyToRequestContainer makes the endpoint-specific container the new body of the request.
func SetRequestBodyToRequestContainer(requestContainer interface{}, req *http.Request) ErrorJson {
// Serialize the struct, which now includes a base64-encoded value, into JSON.
j, err := json.Marshal(requestContainer)
if err != nil {
return InternalServerErrorWithMessage(err, "could not marshal request")
}
// Set the body to the new JSON.
req.Body = io.NopCloser(bytes.NewReader(j))
req.Header.Set("Content-Length", strconv.Itoa(len(j)))
req.ContentLength = int64(len(j))
return nil
}
// PrepareRequestForProxying applies additional logic to the request so that it can be correctly proxied to grpc-gateway.
func (m *ApiProxyMiddleware) PrepareRequestForProxying(endpoint Endpoint, req *http.Request) ErrorJson {
req.URL.Scheme = "http"
req.URL.Host = m.GatewayAddress
req.RequestURI = ""
if errJson := HandleURLParameters(endpoint.Path, req, endpoint.RequestURLLiterals); errJson != nil {
return errJson
}
if errJson := HandleQueryParameters(req, endpoint.RequestQueryParams); errJson != nil {
return errJson
}
// We have to add the prefix after handling parameters because adding the prefix changes URL segment indexing.
req.URL.Path = "/internal" + req.URL.Path
return nil
}
// ProxyRequest proxies the request to grpc-gateway.
func (m *ApiProxyMiddleware) ProxyRequest(req *http.Request) (*http.Response, ErrorJson) {
// We do not use http.DefaultClient because it does not have any timeout.
netClient := &http.Client{Timeout: m.Timeout}
grpcResp, err := netClient.Do(req)
if err != nil {
if err, ok := err.(net.Error); ok && err.Timeout() {
return nil, TimeoutError()
}
return nil, InternalServerErrorWithMessage(err, "could not proxy request")
}
if grpcResp == nil {
return nil, &DefaultErrorJson{Message: "nil response from gRPC-gateway", Code: http.StatusInternalServerError}
}
return grpcResp, nil
}
// ReadGrpcResponseBody reads the body from the grpc-gateway's response.
func ReadGrpcResponseBody(r io.Reader) ([]byte, ErrorJson) {
body, err := io.ReadAll(r)
if err != nil {
return nil, InternalServerErrorWithMessage(err, "could not read response body")
}
return body, nil
}
// HandleGrpcResponseError acts on an error that resulted from a grpc-gateway's response.
// Whether there was an error is indicated by the bool return value. In case of an error,
// there is no need to write to the response because it's taken care of by the function.
func HandleGrpcResponseError(errJson ErrorJson, resp *http.Response, respBody []byte, w http.ResponseWriter) (bool, ErrorJson) {
responseHasError := false
if err := json.Unmarshal(respBody, errJson); err != nil {
return false, InternalServerErrorWithMessage(err, "could not unmarshal error")
}
if errJson.Msg() != "" {
responseHasError = true
// Something went wrong, but the request completed, meaning we can write headers and the error message.
for h, vs := range resp.Header {
for _, v := range vs {
w.Header().Set(h, v)
}
}
// Handle gRPC timeout.
if resp.StatusCode == http.StatusGatewayTimeout {
WriteError(w, TimeoutError(), resp.Header)
} else {
// Set code to HTTP code because unmarshalled body contained gRPC code.
errJson.SetCode(resp.StatusCode)
WriteError(w, errJson, resp.Header)
}
}
return responseHasError, nil
}
// GrpcResponseIsEmpty determines whether the grpc-gateway's response body contains no data.
func GrpcResponseIsEmpty(grpcResponseBody []byte) bool {
return len(grpcResponseBody) == 0 || string(grpcResponseBody) == "{}"
}
// DeserializeGrpcResponseBodyIntoContainer deserializes the grpc-gateway's response body into an endpoint-specific struct.
func DeserializeGrpcResponseBodyIntoContainer(body []byte, responseContainer interface{}) ErrorJson {
if err := json.Unmarshal(body, &responseContainer); err != nil {
return InternalServerErrorWithMessage(err, "could not unmarshal response")
}
return nil
}
// ProcessMiddlewareResponseFields processes fields of an endpoint-specific container according to field tags.
func ProcessMiddlewareResponseFields(responseContainer interface{}) ErrorJson {
if err := processField(responseContainer, []fieldProcessor{
{
tag: "hex",
f: base64ToHexProcessor,
},
{
tag: "address",
f: base64ToChecksumAddressProcessor,
},
{
tag: "enum",
f: enumToLowercaseProcessor,
},
{
tag: "time",
f: timeToUnixProcessor,
},
{
tag: "uint256",
f: base64ToUint256Processor,
},
}); err != nil {
return InternalServerErrorWithMessage(err, "could not process response data")
}
return nil
}
// SerializeMiddlewareResponseIntoJson serializes the endpoint-specific response struct into a JSON representation.
func SerializeMiddlewareResponseIntoJson(responseContainer interface{}) (jsonResponse []byte, errJson ErrorJson) {
j, err := json.Marshal(responseContainer)
if err != nil {
return nil, InternalServerErrorWithMessage(err, "could not marshal response")
}
return j, nil
}
// WriteMiddlewareResponseHeadersAndBody populates headers and the body of the final response.
func WriteMiddlewareResponseHeadersAndBody(grpcResp *http.Response, responseJson []byte, w http.ResponseWriter) ErrorJson {
var statusCodeHeader string
for h, vs := range grpcResp.Header {
// We don't want to expose any gRPC metadata in the HTTP response, so we skip forwarding metadata headers.
if strings.HasPrefix(h, "Grpc-Metadata") {
if h == "Grpc-Metadata-"+grpc.HttpCodeMetadataKey {
statusCodeHeader = vs[0]
}
} else {
for _, v := range vs {
w.Header().Set(h, v)
}
}
}
if !GrpcResponseIsEmpty(responseJson) {
w.Header().Set("Content-Length", strconv.Itoa(len(responseJson)))
if statusCodeHeader != "" {
code, err := strconv.Atoi(statusCodeHeader)
if err != nil {
return InternalServerErrorWithMessage(err, "could not parse status code")
}
w.WriteHeader(code)
} else {
w.WriteHeader(grpcResp.StatusCode)
}
if _, err := io.Copy(w, io.NopCloser(bytes.NewReader(responseJson))); err != nil {
return InternalServerErrorWithMessage(err, "could not write response message")
}
} else {
w.Header().Set("Content-Length", "0")
w.WriteHeader(grpcResp.StatusCode)
}
return nil
}
// WriteError writes the error by manipulating headers and the body of the final response.
func WriteError(w http.ResponseWriter, errJson ErrorJson, responseHeader http.Header) {
// Include custom error in the error JSON.
hasCustomError := false
if responseHeader != nil {
customError, ok := responseHeader["Grpc-Metadata-"+grpc.CustomErrorMetadataKey]
if ok {
hasCustomError = true
// Assume header has only one value and read the 0 index.
if err := json.Unmarshal([]byte(customError[0]), errJson); err != nil {
log.WithError(err).Error("Could not unmarshal custom error message")
return
}
}
}
var j []byte
if hasCustomError {
var err error
j, err = json.Marshal(errJson)
if err != nil {
log.WithError(err).Error("Could not marshal error message")
return
}
} else {
var err error
// We marshal the response body into a DefaultErrorJson if the custom error is not present.
// This is because the ErrorJson argument is the endpoint's error definition, which may contain custom fields.
// In such a scenario marhaling the endpoint's error would populate the resulting JSON
// with these fields even if they are not present in the gRPC header.
d := &DefaultErrorJson{
Message: errJson.Msg(),
Code: errJson.StatusCode(),
}
j, err = json.Marshal(d)
if err != nil {
log.WithError(err).Error("Could not marshal error message")
return
}
}
w.Header().Set("Content-Length", strconv.Itoa(len(j)))
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(errJson.StatusCode())
if _, err := io.Copy(w, io.NopCloser(bytes.NewReader(j))); err != nil {
log.WithError(err).Error("Could not write error message")
}
}
// Cleanup performs final cleanup on the initial response from grpc-gateway.
func Cleanup(grpcResponseBody io.ReadCloser) ErrorJson {
if err := grpcResponseBody.Close(); err != nil {
return InternalServerErrorWithMessage(err, "could not close response body")
}
return nil
}

View File

@@ -0,0 +1,429 @@
package apimiddleware
import (
"bytes"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/prysmaticlabs/prysm/v3/api/grpc"
"github.com/prysmaticlabs/prysm/v3/testing/assert"
"github.com/prysmaticlabs/prysm/v3/testing/require"
"github.com/sirupsen/logrus/hooks/test"
)
type testRequestContainer struct {
TestString string
TestHexString string `hex:"true"`
TestEmptyHexString string `hex:"true"`
TestUint256String string `uint256:"true"`
}
func defaultRequestContainer() *testRequestContainer {
return &testRequestContainer{
TestString: "test string",
TestHexString: "0x666F6F", // hex encoding of "foo"
TestEmptyHexString: "0x",
TestUint256String: "4196",
}
}
type testResponseContainer struct {
TestString string
TestHex string `hex:"true"`
TestEmptyHex string `hex:"true"`
TestAddress string `address:"true"`
TestEmptyAddress string `address:"true"`
TestUint256 string `uint256:"true"`
TestEnum string `enum:"true"`
TestTime string `time:"true"`
}
func defaultResponseContainer() *testResponseContainer {
return &testResponseContainer{
TestString: "test string",
TestHex: "Zm9v", // base64 encoding of "foo"
TestEmptyHex: "",
TestAddress: "Zm9v",
TestEmptyAddress: "",
TestEnum: "Test Enum",
TestTime: "2006-01-02T15:04:05Z",
// base64 encoding of 4196 in little-endian
TestUint256: "ZBAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=",
}
}
type testErrorJson struct {
Message string
Code int
CustomField string
}
// StatusCode returns the error's underlying error code.
func (e *testErrorJson) StatusCode() int {
return e.Code
}
// Msg returns the error's underlying message.
func (e *testErrorJson) Msg() string {
return e.Message
}
// SetCode sets the error's underlying error code.
func (e *testErrorJson) SetCode(code int) {
e.Code = code
}
// SetMsg sets the error's underlying message.
func (e *testErrorJson) SetMsg(msg string) {
e.Message = msg
}
func TestDeserializeRequestBodyIntoContainer(t *testing.T) {
t.Run("ok", func(t *testing.T) {
var bodyJson bytes.Buffer
err := json.NewEncoder(&bodyJson).Encode(defaultRequestContainer())
require.NoError(t, err)
container := &testRequestContainer{}
errJson := DeserializeRequestBodyIntoContainer(&bodyJson, container)
require.Equal(t, true, errJson == nil)
assert.Equal(t, "test string", container.TestString)
})
t.Run("error", func(t *testing.T) {
var bodyJson bytes.Buffer
bodyJson.Write([]byte("foo"))
errJson := DeserializeRequestBodyIntoContainer(&bodyJson, &testRequestContainer{})
require.NotNil(t, errJson)
assert.Equal(t, true, strings.Contains(errJson.Msg(), "could not decode request body"))
assert.Equal(t, http.StatusInternalServerError, errJson.StatusCode())
})
t.Run("unknown field", func(t *testing.T) {
var bodyJson bytes.Buffer
bodyJson.Write([]byte("{\"foo\":\"foo\"}"))
errJson := DeserializeRequestBodyIntoContainer(&bodyJson, &testRequestContainer{})
require.NotNil(t, errJson)
assert.Equal(t, true, strings.Contains(errJson.Msg(), "could not decode request body"))
assert.Equal(t, http.StatusBadRequest, errJson.StatusCode())
})
}
func TestProcessRequestContainerFields(t *testing.T) {
t.Run("ok", func(t *testing.T) {
container := defaultRequestContainer()
errJson := ProcessRequestContainerFields(container)
require.Equal(t, true, errJson == nil)
assert.Equal(t, "Zm9v", container.TestHexString)
assert.Equal(t, "", container.TestEmptyHexString)
assert.Equal(t, "ZBAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=", container.TestUint256String)
})
t.Run("error", func(t *testing.T) {
errJson := ProcessRequestContainerFields("foo")
require.NotNil(t, errJson)
assert.Equal(t, true, strings.Contains(errJson.Msg(), "could not process request data"))
assert.Equal(t, http.StatusInternalServerError, errJson.StatusCode())
})
}
func TestSetRequestBodyToRequestContainer(t *testing.T) {
var body bytes.Buffer
request := httptest.NewRequest("GET", "http://foo.example", &body)
errJson := SetRequestBodyToRequestContainer(defaultRequestContainer(), request)
require.Equal(t, true, errJson == nil)
container := &testRequestContainer{}
require.NoError(t, json.NewDecoder(request.Body).Decode(container))
assert.Equal(t, "test string", container.TestString)
contentLengthHeader, ok := request.Header["Content-Length"]
require.Equal(t, true, ok)
require.Equal(t, 1, len(contentLengthHeader), "wrong number of header values")
assert.Equal(t, "108", contentLengthHeader[0])
assert.Equal(t, int64(108), request.ContentLength)
}
func TestPrepareRequestForProxying(t *testing.T) {
middleware := &ApiProxyMiddleware{
GatewayAddress: "http://gateway.example",
}
// We will set some params to make the request more interesting.
endpoint := Endpoint{
Path: "/{url_param}",
RequestURLLiterals: []string{"url_param"},
RequestQueryParams: []QueryParam{{Name: "query_param"}},
}
var body bytes.Buffer
request := httptest.NewRequest("GET", "http://foo.example?query_param=bar", &body)
errJson := middleware.PrepareRequestForProxying(endpoint, request)
require.Equal(t, true, errJson == nil)
assert.Equal(t, "http", request.URL.Scheme)
assert.Equal(t, middleware.GatewayAddress, request.URL.Host)
assert.Equal(t, "", request.RequestURI)
}
func TestReadGrpcResponseBody(t *testing.T) {
var b bytes.Buffer
b.Write([]byte("foo"))
body, jsonErr := ReadGrpcResponseBody(&b)
require.Equal(t, true, jsonErr == nil)
assert.Equal(t, "foo", string(body))
}
func TestHandleGrpcResponseError(t *testing.T) {
response := &http.Response{
StatusCode: 400,
Header: http.Header{
"Foo": []string{"foo"},
"Bar": []string{"bar"},
},
}
writer := httptest.NewRecorder()
errJson := &testErrorJson{
Message: "foo",
Code: 400,
}
b, err := json.Marshal(errJson)
require.NoError(t, err)
hasError, e := HandleGrpcResponseError(errJson, response, b, writer)
require.Equal(t, true, e == nil)
assert.Equal(t, true, hasError)
v, ok := writer.Header()["Foo"]
require.Equal(t, true, ok, "header not found")
require.Equal(t, 1, len(v), "wrong number of header values")
assert.Equal(t, "foo", v[0])
v, ok = writer.Header()["Bar"]
require.Equal(t, true, ok, "header not found")
require.Equal(t, 1, len(v), "wrong number of header values")
assert.Equal(t, "bar", v[0])
assert.Equal(t, 400, errJson.StatusCode())
}
func TestGrpcResponseIsEmpty(t *testing.T) {
t.Run("nil", func(t *testing.T) {
assert.Equal(t, true, GrpcResponseIsEmpty(nil))
})
t.Run("empty_slice", func(t *testing.T) {
assert.Equal(t, true, GrpcResponseIsEmpty(make([]byte, 0)))
})
t.Run("empty_brackets", func(t *testing.T) {
assert.Equal(t, true, GrpcResponseIsEmpty([]byte("{}")))
})
t.Run("non_empty", func(t *testing.T) {
assert.Equal(t, false, GrpcResponseIsEmpty([]byte("{\"foo\":\"bar\"})")))
})
}
func TestDeserializeGrpcResponseBodyIntoContainer(t *testing.T) {
t.Run("ok", func(t *testing.T) {
body, err := json.Marshal(defaultRequestContainer())
require.NoError(t, err)
container := &testRequestContainer{}
errJson := DeserializeGrpcResponseBodyIntoContainer(body, container)
require.Equal(t, true, errJson == nil)
assert.Equal(t, "test string", container.TestString)
})
t.Run("error", func(t *testing.T) {
var bodyJson bytes.Buffer
bodyJson.Write([]byte("foo"))
errJson := DeserializeGrpcResponseBodyIntoContainer(bodyJson.Bytes(), &testRequestContainer{})
require.NotNil(t, errJson)
assert.Equal(t, true, strings.Contains(errJson.Msg(), "could not unmarshal response"))
assert.Equal(t, http.StatusInternalServerError, errJson.StatusCode())
})
}
func TestProcessMiddlewareResponseFields(t *testing.T) {
t.Run("Ok", func(t *testing.T) {
container := defaultResponseContainer()
errJson := ProcessMiddlewareResponseFields(container)
require.Equal(t, true, errJson == nil)
assert.Equal(t, "0x666f6f", container.TestHex)
assert.Equal(t, "0x", container.TestEmptyHex)
assert.Equal(t, "0x0000000000000000000000000000000000666F6f", container.TestAddress)
assert.Equal(t, "0x", container.TestEmptyAddress)
assert.Equal(t, "4196", container.TestUint256)
assert.Equal(t, "test enum", container.TestEnum)
assert.Equal(t, "1136214245", container.TestTime)
})
t.Run("error", func(t *testing.T) {
errJson := ProcessMiddlewareResponseFields("foo")
require.NotNil(t, errJson)
assert.Equal(t, true, strings.Contains(errJson.Msg(), "could not process response data"))
assert.Equal(t, http.StatusInternalServerError, errJson.StatusCode())
})
}
func TestSerializeMiddlewareResponseIntoJson(t *testing.T) {
container := defaultResponseContainer()
j, errJson := SerializeMiddlewareResponseIntoJson(container)
assert.Equal(t, true, errJson == nil)
cToDeserialize := &testResponseContainer{}
require.NoError(t, json.Unmarshal(j, cToDeserialize))
assert.Equal(t, "test string", cToDeserialize.TestString)
}
func TestWriteMiddlewareResponseHeadersAndBody(t *testing.T) {
t.Run("GET", func(t *testing.T) {
response := &http.Response{
Header: http.Header{
"Foo": []string{"foo"},
"Grpc-Metadata-" + grpc.HttpCodeMetadataKey: []string{"204"},
},
}
container := defaultResponseContainer()
responseJson, err := json.Marshal(container)
require.NoError(t, err)
writer := httptest.NewRecorder()
writer.Body = &bytes.Buffer{}
errJson := WriteMiddlewareResponseHeadersAndBody(response, responseJson, writer)
require.Equal(t, true, errJson == nil)
v, ok := writer.Header()["Foo"]
require.Equal(t, true, ok, "header not found")
require.Equal(t, 1, len(v), "wrong number of header values")
assert.Equal(t, "foo", v[0])
v, ok = writer.Header()["Content-Length"]
require.Equal(t, true, ok, "header not found")
require.Equal(t, 1, len(v), "wrong number of header values")
assert.Equal(t, "224", v[0])
assert.Equal(t, 204, writer.Code)
assert.DeepEqual(t, responseJson, writer.Body.Bytes())
})
t.Run("GET_no_grpc_status_code_header", func(t *testing.T) {
response := &http.Response{
Header: http.Header{},
StatusCode: 204,
}
container := defaultResponseContainer()
responseJson, err := json.Marshal(container)
require.NoError(t, err)
writer := httptest.NewRecorder()
errJson := WriteMiddlewareResponseHeadersAndBody(response, responseJson, writer)
require.Equal(t, true, errJson == nil)
assert.Equal(t, 204, writer.Code)
})
t.Run("GET_invalid_status_code", func(t *testing.T) {
response := &http.Response{
Header: http.Header{},
}
// Set invalid status code.
response.Header["Grpc-Metadata-"+grpc.HttpCodeMetadataKey] = []string{"invalid"}
container := defaultResponseContainer()
responseJson, err := json.Marshal(container)
require.NoError(t, err)
writer := httptest.NewRecorder()
errJson := WriteMiddlewareResponseHeadersAndBody(response, responseJson, writer)
require.Equal(t, false, errJson == nil)
assert.Equal(t, true, strings.Contains(errJson.Msg(), "could not parse status code"))
assert.Equal(t, http.StatusInternalServerError, errJson.StatusCode())
})
t.Run("POST", func(t *testing.T) {
response := &http.Response{
Header: http.Header{},
StatusCode: 204,
}
container := defaultResponseContainer()
responseJson, err := json.Marshal(container)
require.NoError(t, err)
writer := httptest.NewRecorder()
errJson := WriteMiddlewareResponseHeadersAndBody(response, responseJson, writer)
require.Equal(t, true, errJson == nil)
assert.Equal(t, 204, writer.Code)
})
t.Run("POST_with_response_body", func(t *testing.T) {
response := &http.Response{
Header: http.Header{},
StatusCode: 204,
}
container := defaultResponseContainer()
responseJson, err := json.Marshal(container)
require.NoError(t, err)
writer := httptest.NewRecorder()
writer.Body = &bytes.Buffer{}
errJson := WriteMiddlewareResponseHeadersAndBody(response, responseJson, writer)
require.Equal(t, true, errJson == nil)
assert.Equal(t, 204, writer.Code)
assert.DeepEqual(t, responseJson, writer.Body.Bytes())
})
t.Run("POST_with_empty_json_body", func(t *testing.T) {
response := &http.Response{
Header: http.Header{},
StatusCode: 204,
}
responseJson, err := json.Marshal(struct{}{})
require.NoError(t, err)
writer := httptest.NewRecorder()
writer.Body = &bytes.Buffer{}
errJson := WriteMiddlewareResponseHeadersAndBody(response, responseJson, writer)
require.Equal(t, true, errJson == nil)
assert.Equal(t, 204, writer.Code)
assert.DeepEqual(t, []byte(nil), writer.Body.Bytes())
assert.Equal(t, "0", writer.Header()["Content-Length"][0])
})
}
func TestWriteError(t *testing.T) {
t.Run("ok", func(t *testing.T) {
responseHeader := http.Header{
"Grpc-Metadata-" + grpc.CustomErrorMetadataKey: []string{"{\"CustomField\":\"bar\"}"},
}
errJson := &testErrorJson{
Message: "foo",
Code: 500,
}
writer := httptest.NewRecorder()
writer.Body = &bytes.Buffer{}
WriteError(writer, errJson, responseHeader)
v, ok := writer.Header()["Content-Length"]
require.Equal(t, true, ok, "header not found")
require.Equal(t, 1, len(v), "wrong number of header values")
assert.Equal(t, "48", v[0])
v, ok = writer.Header()["Content-Type"]
require.Equal(t, true, ok, "header not found")
require.Equal(t, 1, len(v), "wrong number of header values")
assert.Equal(t, "application/json", v[0])
assert.Equal(t, 500, writer.Code)
eDeserialize := &testErrorJson{}
require.NoError(t, json.Unmarshal(writer.Body.Bytes(), eDeserialize))
assert.Equal(t, "foo", eDeserialize.Message)
assert.Equal(t, 500, eDeserialize.Code)
assert.Equal(t, "bar", eDeserialize.CustomField)
})
t.Run("invalid_custom_error_header", func(t *testing.T) {
logHook := test.NewGlobal()
responseHeader := http.Header{
"Grpc-Metadata-" + grpc.CustomErrorMetadataKey: []string{"invalid"},
}
WriteError(httptest.NewRecorder(), &testErrorJson{}, responseHeader)
assert.LogsContain(t, logHook, "Could not unmarshal custom error message")
})
}

View File

@@ -0,0 +1,69 @@
package apimiddleware
import (
"net/http"
"github.com/pkg/errors"
)
// ---------------
// Error handling.
// ---------------
// ErrorJson describes common functionality of all JSON error representations.
type ErrorJson interface {
StatusCode() int
SetCode(code int)
Msg() string
SetMsg(msg string)
}
// DefaultErrorJson is a JSON representation of a simple error value, containing only a message and an error code.
type DefaultErrorJson struct {
Message string `json:"message"`
Code int `json:"code"`
}
// InternalServerErrorWithMessage returns a DefaultErrorJson with 500 code and a custom message.
func InternalServerErrorWithMessage(err error, message string) *DefaultErrorJson {
e := errors.Wrapf(err, message)
return &DefaultErrorJson{
Message: e.Error(),
Code: http.StatusInternalServerError,
}
}
// InternalServerError returns a DefaultErrorJson with 500 code.
func InternalServerError(err error) *DefaultErrorJson {
return &DefaultErrorJson{
Message: err.Error(),
Code: http.StatusInternalServerError,
}
}
func TimeoutError() *DefaultErrorJson {
return &DefaultErrorJson{
Message: "Request timeout",
Code: http.StatusRequestTimeout,
}
}
// StatusCode returns the error's underlying error code.
func (e *DefaultErrorJson) StatusCode() int {
return e.Code
}
// Msg returns the error's underlying message.
func (e *DefaultErrorJson) Msg() string {
return e.Message
}
// SetCode sets the error's underlying error code.
func (e *DefaultErrorJson) SetCode(code int) {
e.Code = code
}
// SetMsg sets the error's underlying message.
func (e *DefaultErrorJson) SetMsg(msg string) {
e.Message = msg
}

257
api/gateway/gateway.go Normal file
View File

@@ -0,0 +1,257 @@
// Package gateway defines a grpc-gateway server that serves HTTP-JSON traffic and acts a proxy between HTTP and gRPC.
package gateway
import (
"context"
"fmt"
"net"
"net/http"
"path"
"strings"
"time"
"github.com/gorilla/mux"
gwruntime "github.com/grpc-ecosystem/grpc-gateway/v2/runtime"
"github.com/pkg/errors"
"github.com/prysmaticlabs/prysm/v3/api/gateway/apimiddleware"
"github.com/prysmaticlabs/prysm/v3/runtime"
"github.com/rs/cors"
"google.golang.org/grpc"
"google.golang.org/grpc/connectivity"
"google.golang.org/grpc/credentials"
)
var _ runtime.Service = (*Gateway)(nil)
// PbMux serves grpc-gateway requests for selected patterns using registered protobuf handlers.
type PbMux struct {
Registrations []PbHandlerRegistration // Protobuf registrations to be registered in Mux.
Patterns []string // URL patterns that will be handled by Mux.
Mux *gwruntime.ServeMux // The router that will be used for grpc-gateway requests.
}
// PbHandlerRegistration is a function that registers a protobuf handler.
type PbHandlerRegistration func(context.Context, *gwruntime.ServeMux, *grpc.ClientConn) error
// MuxHandler is a function that implements the mux handler functionality.
type MuxHandler func(
apiMiddlewareHandler *apimiddleware.ApiProxyMiddleware,
h http.HandlerFunc,
w http.ResponseWriter,
req *http.Request,
)
// Config parameters for setting up the gateway service.
type config struct {
maxCallRecvMsgSize uint64
remoteCert string
gatewayAddr string
remoteAddr string
allowedOrigins []string
apiMiddlewareEndpointFactory apimiddleware.EndpointFactory
muxHandler MuxHandler
pbHandlers []*PbMux
router *mux.Router
timeout time.Duration
}
// Gateway is the gRPC gateway to serve HTTP JSON traffic as a proxy and forward it to the gRPC server.
type Gateway struct {
cfg *config
conn *grpc.ClientConn
server *http.Server
cancel context.CancelFunc
proxy *apimiddleware.ApiProxyMiddleware
ctx context.Context
startFailure error
}
// New returns a new instance of the Gateway.
func New(ctx context.Context, opts ...Option) (*Gateway, error) {
g := &Gateway{
ctx: ctx,
cfg: &config{
router: mux.NewRouter(),
},
}
for _, opt := range opts {
if err := opt(g); err != nil {
return nil, err
}
}
return g, nil
}
// Start the gateway service.
func (g *Gateway) Start() {
ctx, cancel := context.WithCancel(g.ctx)
g.cancel = cancel
conn, err := g.dial(ctx, "tcp", g.cfg.remoteAddr)
if err != nil {
log.WithError(err).Error("Failed to connect to gRPC server")
g.startFailure = err
return
}
g.conn = conn
for _, h := range g.cfg.pbHandlers {
for _, r := range h.Registrations {
if err := r(ctx, h.Mux, g.conn); err != nil {
log.WithError(err).Error("Failed to register handler")
g.startFailure = err
return
}
}
for _, p := range h.Patterns {
g.cfg.router.PathPrefix(p).Handler(h.Mux)
}
}
corsMux := g.corsMiddleware(g.cfg.router)
if g.cfg.apiMiddlewareEndpointFactory != nil && !g.cfg.apiMiddlewareEndpointFactory.IsNil() {
g.registerApiMiddleware()
}
if g.cfg.muxHandler != nil {
g.cfg.router.PathPrefix("/").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
g.cfg.muxHandler(g.proxy, corsMux.ServeHTTP, w, r)
})
}
g.server = &http.Server{
Addr: g.cfg.gatewayAddr,
Handler: corsMux,
ReadHeaderTimeout: time.Second,
}
go func() {
log.WithField("address", g.cfg.gatewayAddr).Info("Starting gRPC gateway")
if err := g.server.ListenAndServe(); err != http.ErrServerClosed {
log.WithError(err).Error("Failed to start gRPC gateway")
g.startFailure = err
return
}
}()
}
// Status of grpc gateway. Returns an error if this service is unhealthy.
func (g *Gateway) Status() error {
if g.startFailure != nil {
return g.startFailure
}
if s := g.conn.GetState(); s != connectivity.Ready {
return fmt.Errorf("grpc server is %s", s)
}
return nil
}
// Stop the gateway with a graceful shutdown.
func (g *Gateway) Stop() error {
if g.server != nil {
shutdownCtx, shutdownCancel := context.WithTimeout(g.ctx, 2*time.Second)
defer shutdownCancel()
if err := g.server.Shutdown(shutdownCtx); err != nil {
if errors.Is(err, context.DeadlineExceeded) {
log.Warn("Existing connections terminated")
} else {
log.WithError(err).Error("Failed to gracefully shut down server")
}
}
}
if g.cancel != nil {
g.cancel()
}
return nil
}
func (g *Gateway) corsMiddleware(h http.Handler) http.Handler {
c := cors.New(cors.Options{
AllowedOrigins: g.cfg.allowedOrigins,
AllowedMethods: []string{http.MethodPost, http.MethodGet, http.MethodDelete, http.MethodOptions},
AllowCredentials: true,
MaxAge: 600,
AllowedHeaders: []string{"*"},
})
return c.Handler(h)
}
const swaggerDir = "proto/prysm/v1alpha1/"
// SwaggerServer returns swagger specification files located under "/swagger/"
func SwaggerServer() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
if !strings.HasSuffix(r.URL.Path, ".swagger.json") {
log.Debugf("Not found: %s", r.URL.Path)
http.NotFound(w, r)
return
}
log.Debugf("Serving %s\n", r.URL.Path)
p := strings.TrimPrefix(r.URL.Path, "/swagger/")
p = path.Join(swaggerDir, p)
http.ServeFile(w, r, p)
}
}
// dial the gRPC server.
func (g *Gateway) dial(ctx context.Context, network, addr string) (*grpc.ClientConn, error) {
switch network {
case "tcp":
return g.dialTCP(ctx, addr)
case "unix":
return g.dialUnix(ctx, addr)
default:
return nil, fmt.Errorf("unsupported network type %q", network)
}
}
// dialTCP creates a client connection via TCP.
// "addr" must be a valid TCP address with a port number.
func (g *Gateway) dialTCP(ctx context.Context, addr string) (*grpc.ClientConn, error) {
security := grpc.WithInsecure()
if len(g.cfg.remoteCert) > 0 {
creds, err := credentials.NewClientTLSFromFile(g.cfg.remoteCert, "")
if err != nil {
return nil, err
}
security = grpc.WithTransportCredentials(creds)
}
opts := []grpc.DialOption{
security,
grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(int(g.cfg.maxCallRecvMsgSize))),
}
return grpc.DialContext(ctx, addr, opts...)
}
// dialUnix creates a client connection via a unix domain socket.
// "addr" must be a valid path to the socket.
func (g *Gateway) dialUnix(ctx context.Context, addr string) (*grpc.ClientConn, error) {
d := func(addr string, timeout time.Duration) (net.Conn, error) {
return net.DialTimeout("unix", addr, timeout)
}
f := func(ctx context.Context, addr string) (net.Conn, error) {
if deadline, ok := ctx.Deadline(); ok {
return d(addr, time.Until(deadline))
}
return d(addr, 0)
}
opts := []grpc.DialOption{
grpc.WithInsecure(),
grpc.WithContextDialer(f),
grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(int(g.cfg.maxCallRecvMsgSize))),
}
return grpc.DialContext(ctx, addr, opts...)
}
func (g *Gateway) registerApiMiddleware() {
g.proxy = &apimiddleware.ApiProxyMiddleware{
GatewayAddress: g.cfg.gatewayAddr,
EndpointCreator: g.cfg.apiMiddlewareEndpointFactory,
Timeout: g.cfg.timeout,
}
log.Info("Starting API middleware")
g.proxy.Run(g.cfg.router)
}

128
api/gateway/gateway_test.go Normal file
View File

@@ -0,0 +1,128 @@
package gateway
import (
"context"
"flag"
"fmt"
"net/http"
"net/http/httptest"
"net/url"
"testing"
"github.com/gorilla/mux"
"github.com/prysmaticlabs/prysm/v3/api/gateway/apimiddleware"
"github.com/prysmaticlabs/prysm/v3/cmd/beacon-chain/flags"
"github.com/prysmaticlabs/prysm/v3/testing/assert"
"github.com/prysmaticlabs/prysm/v3/testing/require"
logTest "github.com/sirupsen/logrus/hooks/test"
"github.com/urfave/cli/v2"
)
type mockEndpointFactory struct {
}
func (*mockEndpointFactory) Paths() []string {
return []string{}
}
func (*mockEndpointFactory) Create(_ string) (*apimiddleware.Endpoint, error) {
return nil, nil
}
func (*mockEndpointFactory) IsNil() bool {
return false
}
func TestGateway_Customized(t *testing.T) {
r := mux.NewRouter()
cert := "cert"
origins := []string{"origin"}
size := uint64(100)
endpointFactory := &mockEndpointFactory{}
opts := []Option{
WithRouter(r),
WithRemoteCert(cert),
WithAllowedOrigins(origins),
WithMaxCallRecvMsgSize(size),
WithApiMiddleware(endpointFactory),
WithMuxHandler(func(
_ *apimiddleware.ApiProxyMiddleware,
_ http.HandlerFunc,
_ http.ResponseWriter,
_ *http.Request,
) {
}),
}
g, err := New(context.Background(), opts...)
require.NoError(t, err)
assert.Equal(t, r, g.cfg.router)
assert.Equal(t, cert, g.cfg.remoteCert)
require.Equal(t, 1, len(g.cfg.allowedOrigins))
assert.Equal(t, origins[0], g.cfg.allowedOrigins[0])
assert.Equal(t, size, g.cfg.maxCallRecvMsgSize)
assert.Equal(t, endpointFactory, g.cfg.apiMiddlewareEndpointFactory)
}
func TestGateway_StartStop(t *testing.T) {
hook := logTest.NewGlobal()
app := cli.App{}
set := flag.NewFlagSet("test", 0)
ctx := cli.NewContext(&app, set, nil)
gatewayPort := ctx.Int(flags.GRPCGatewayPort.Name)
gatewayHost := ctx.String(flags.GRPCGatewayHost.Name)
rpcHost := ctx.String(flags.RPCHost.Name)
selfAddress := fmt.Sprintf("%s:%d", rpcHost, ctx.Int(flags.RPCPort.Name))
gatewayAddress := fmt.Sprintf("%s:%d", gatewayHost, gatewayPort)
opts := []Option{
WithGatewayAddr(gatewayAddress),
WithRemoteAddr(selfAddress),
WithMuxHandler(func(
_ *apimiddleware.ApiProxyMiddleware,
_ http.HandlerFunc,
_ http.ResponseWriter,
_ *http.Request,
) {
}),
}
g, err := New(context.Background(), opts...)
require.NoError(t, err)
g.Start()
go func() {
require.LogsContain(t, hook, "Starting gRPC gateway")
require.LogsDoNotContain(t, hook, "Starting API middleware")
}()
err = g.Stop()
require.NoError(t, err)
}
func TestGateway_NilHandler_NotFoundHandlerRegistered(t *testing.T) {
app := cli.App{}
set := flag.NewFlagSet("test", 0)
ctx := cli.NewContext(&app, set, nil)
gatewayPort := ctx.Int(flags.GRPCGatewayPort.Name)
gatewayHost := ctx.String(flags.GRPCGatewayHost.Name)
rpcHost := ctx.String(flags.RPCHost.Name)
selfAddress := fmt.Sprintf("%s:%d", rpcHost, ctx.Int(flags.RPCPort.Name))
gatewayAddress := fmt.Sprintf("%s:%d", gatewayHost, gatewayPort)
opts := []Option{
WithGatewayAddr(gatewayAddress),
WithRemoteAddr(selfAddress),
}
g, err := New(context.Background(), opts...)
require.NoError(t, err)
writer := httptest.NewRecorder()
g.cfg.router.ServeHTTP(writer, &http.Request{Method: "GET", Host: "localhost", URL: &url.URL{Path: "/foo"}})
assert.Equal(t, http.StatusNotFound, writer.Code)
}

30
api/gateway/modifiers.go Normal file
View File

@@ -0,0 +1,30 @@
package gateway
import (
"context"
"net/http"
"strconv"
gwruntime "github.com/grpc-ecosystem/grpc-gateway/v2/runtime"
"google.golang.org/protobuf/proto"
)
func HttpResponseModifier(ctx context.Context, w http.ResponseWriter, _ proto.Message) error {
md, ok := gwruntime.ServerMetadataFromContext(ctx)
if !ok {
return nil
}
// set http status code
if vals := md.HeaderMD.Get("x-http-code"); len(vals) > 0 {
code, err := strconv.Atoi(vals[0])
if err != nil {
return err
}
// delete the headers to not expose any grpc-metadata in http response
delete(md.HeaderMD, "x-http-code")
delete(w.Header(), "Grpc-Metadata-X-Http-Code")
w.WriteHeader(code)
}
return nil
}

93
api/gateway/options.go Normal file
View File

@@ -0,0 +1,93 @@
package gateway
import (
"time"
"github.com/gorilla/mux"
gwruntime "github.com/grpc-ecosystem/grpc-gateway/v2/runtime"
"github.com/prysmaticlabs/prysm/v3/api/gateway/apimiddleware"
)
type Option func(g *Gateway) error
func (g *Gateway) SetRouter(r *mux.Router) *Gateway {
g.cfg.router = r
return g
}
func WithPbHandlers(handlers []*PbMux) Option {
return func(g *Gateway) error {
g.cfg.pbHandlers = handlers
return nil
}
}
func WithMuxHandler(m MuxHandler) Option {
return func(g *Gateway) error {
g.cfg.muxHandler = m
return nil
}
}
func WithGatewayAddr(addr string) Option {
return func(g *Gateway) error {
g.cfg.gatewayAddr = addr
return nil
}
}
func WithRemoteAddr(addr string) Option {
return func(g *Gateway) error {
g.cfg.remoteAddr = addr
return nil
}
}
// WithRouter allows adding a custom mux router to the gateway.
func WithRouter(r *mux.Router) Option {
return func(g *Gateway) error {
g.cfg.router = r
return nil
}
}
// WithAllowedOrigins allows adding a set of allowed origins to the gateway.
func WithAllowedOrigins(origins []string) Option {
return func(g *Gateway) error {
g.cfg.allowedOrigins = origins
return nil
}
}
// WithRemoteCert allows adding a custom certificate to the gateway,
func WithRemoteCert(cert string) Option {
return func(g *Gateway) error {
g.cfg.remoteCert = cert
return nil
}
}
// WithMaxCallRecvMsgSize allows specifying the maximum allowed gRPC message size.
func WithMaxCallRecvMsgSize(size uint64) Option {
return func(g *Gateway) error {
g.cfg.maxCallRecvMsgSize = size
return nil
}
}
// WithApiMiddleware allows adding an API middleware proxy to the gateway.
func WithApiMiddleware(endpointFactory apimiddleware.EndpointFactory) Option {
return func(g *Gateway) error {
g.cfg.apiMiddlewareEndpointFactory = endpointFactory
return nil
}
}
// WithTimeout allows changing the timeout value for API calls.
func WithTimeout(seconds uint64) Option {
return func(g *Gateway) error {
g.cfg.timeout = time.Second * time.Duration(seconds)
gwruntime.DefaultContextTimeout = time.Second * time.Duration(seconds)
return nil
}
}

30
api/grpc/BUILD.bazel Normal file
View File

@@ -0,0 +1,30 @@
load("@prysm//tools/go:def.bzl", "go_library", "go_test")
go_library(
name = "go_default_library",
srcs = [
"grpcutils.go",
"parameters.go",
],
importpath = "github.com/prysmaticlabs/prysm/v3/api/grpc",
visibility = ["//visibility:public"],
deps = [
"@com_github_sirupsen_logrus//:go_default_library",
"@org_golang_google_grpc//:go_default_library",
"@org_golang_google_grpc//metadata:go_default_library",
],
)
go_test(
name = "go_default_test",
srcs = ["grpcutils_test.go"],
embed = [":go_default_library"],
deps = [
"//testing/assert:go_default_library",
"//testing/require:go_default_library",
"@com_github_grpc_ecosystem_grpc_gateway_v2//runtime:go_default_library",
"@com_github_sirupsen_logrus//hooks/test:go_default_library",
"@org_golang_google_grpc//:go_default_library",
"@org_golang_google_grpc//metadata:go_default_library",
],
)

96
api/grpc/grpcutils.go Normal file
View File

@@ -0,0 +1,96 @@
package grpc
import (
"context"
"encoding/json"
"fmt"
"strings"
"time"
"github.com/sirupsen/logrus"
"google.golang.org/grpc"
"google.golang.org/grpc/metadata"
)
// LogRequests logs the gRPC backend as well as request duration when the log level is set to debug
// or higher.
func LogRequests(
ctx context.Context,
method string, req,
reply interface{},
cc *grpc.ClientConn,
invoker grpc.UnaryInvoker,
opts ...grpc.CallOption,
) error {
// Shortcut when debug logging is not enabled.
if logrus.GetLevel() < logrus.DebugLevel {
return invoker(ctx, method, req, reply, cc, opts...)
}
var header metadata.MD
opts = append(
opts,
grpc.Header(&header),
)
start := time.Now()
err := invoker(ctx, method, req, reply, cc, opts...)
logrus.WithField("backend", header["x-backend"]).
WithField("method", method).WithField("duration", time.Since(start)).
Debug("gRPC request finished.")
return err
}
// LogStream prints the method at DEBUG level at the start of the stream.
func LogStream(
ctx context.Context,
sd *grpc.StreamDesc,
conn *grpc.ClientConn,
method string,
streamer grpc.Streamer,
opts ...grpc.CallOption,
) (grpc.ClientStream, error) {
// Shortcut when debug logging is not enabled.
if logrus.GetLevel() < logrus.DebugLevel {
return streamer(ctx, sd, conn, method, opts...)
}
var header metadata.MD
opts = append(
opts,
grpc.Header(&header),
)
strm, err := streamer(ctx, sd, conn, method, opts...)
logrus.WithField("backend", header["x-backend"]).
WithField("method", method).
Debug("gRPC stream started.")
return strm, err
}
// AppendHeaders parses the provided GRPC headers
// and attaches them to the provided context.
func AppendHeaders(parent context.Context, headers []string) context.Context {
for _, h := range headers {
if h != "" {
keyValue := strings.Split(h, "=")
if len(keyValue) < 2 {
logrus.Warnf("Incorrect gRPC header flag format. Skipping %v", keyValue[0])
continue
}
parent = metadata.AppendToOutgoingContext(parent, keyValue[0], strings.Join(keyValue[1:], "="))
}
}
return parent
}
// AppendCustomErrorHeader sets a CustomErrorMetadataKey gRPC header on the passed in context,
// using the passed in error data as the header's value. The data is serialized as JSON.
func AppendCustomErrorHeader(ctx context.Context, errorData interface{}) error {
j, err := json.Marshal(errorData)
if err != nil {
return fmt.Errorf("could not marshal error data into JSON: %w", err)
}
if err := grpc.SetHeader(ctx, metadata.Pairs(CustomErrorMetadataKey, string(j))); err != nil {
return fmt.Errorf("could not set custom error header: %w", err)
}
return nil
}

View File

@@ -0,0 +1,78 @@
package grpc
import (
"context"
"encoding/json"
"strings"
"testing"
"github.com/grpc-ecosystem/grpc-gateway/v2/runtime"
"github.com/prysmaticlabs/prysm/v3/testing/assert"
"github.com/prysmaticlabs/prysm/v3/testing/require"
logTest "github.com/sirupsen/logrus/hooks/test"
"google.golang.org/grpc"
"google.golang.org/grpc/metadata"
)
type customErrorData struct {
Message string `json:"message"`
}
func TestAppendHeaders(t *testing.T) {
t.Run("one_header", func(t *testing.T) {
ctx := AppendHeaders(context.Background(), []string{"first=value1"})
md, ok := metadata.FromOutgoingContext(ctx)
require.Equal(t, true, ok, "Failed to read context metadata")
require.Equal(t, 1, md.Len(), "MetadataV0 contains wrong number of values")
assert.Equal(t, "value1", md.Get("first")[0])
})
t.Run("multiple_headers", func(t *testing.T) {
ctx := AppendHeaders(context.Background(), []string{"first=value1", "second=value2"})
md, ok := metadata.FromOutgoingContext(ctx)
require.Equal(t, true, ok, "Failed to read context metadata")
require.Equal(t, 2, md.Len(), "MetadataV0 contains wrong number of values")
assert.Equal(t, "value1", md.Get("first")[0])
assert.Equal(t, "value2", md.Get("second")[0])
})
t.Run("one_empty_header", func(t *testing.T) {
ctx := AppendHeaders(context.Background(), []string{"first=value1", ""})
md, ok := metadata.FromOutgoingContext(ctx)
require.Equal(t, true, ok, "Failed to read context metadata")
require.Equal(t, 1, md.Len(), "MetadataV0 contains wrong number of values")
assert.Equal(t, "value1", md.Get("first")[0])
})
t.Run("incorrect_header", func(t *testing.T) {
logHook := logTest.NewGlobal()
ctx := AppendHeaders(context.Background(), []string{"first=value1", "second"})
md, ok := metadata.FromOutgoingContext(ctx)
require.Equal(t, true, ok, "Failed to read context metadata")
require.Equal(t, 1, md.Len(), "MetadataV0 contains wrong number of values")
assert.Equal(t, "value1", md.Get("first")[0])
assert.LogsContain(t, logHook, "Skipping second")
})
t.Run("header_value_with_equal_sign", func(t *testing.T) {
ctx := AppendHeaders(context.Background(), []string{"first=value=1"})
md, ok := metadata.FromOutgoingContext(ctx)
require.Equal(t, true, ok, "Failed to read context metadata")
require.Equal(t, 1, md.Len(), "MetadataV0 contains wrong number of values")
assert.Equal(t, "value=1", md.Get("first")[0])
})
}
func TestAppendCustomErrorHeader(t *testing.T) {
stream := &runtime.ServerTransportStream{}
ctx := grpc.NewContextWithServerTransportStream(context.Background(), stream)
data := &customErrorData{Message: "foo"}
require.NoError(t, AppendCustomErrorHeader(ctx, data))
// The stream used in test setup sets the metadata key in lowercase.
value, ok := stream.Header()[strings.ToLower(CustomErrorMetadataKey)]
require.Equal(t, true, ok, "Failed to retrieve custom error metadata value")
expected, err := json.Marshal(data)
require.NoError(t, err)
assert.Equal(t, string(expected), value[0])
}

8
api/grpc/parameters.go Normal file
View File

@@ -0,0 +1,8 @@
package grpc
// CustomErrorMetadataKey is the name of the metadata key storing additional error information.
// Metadata value is expected to be a byte-encoded JSON object.
const CustomErrorMetadataKey = "Custom-Error"
// HttpCodeMetadataKey is the key to use when setting custom HTTP status codes in gRPC metadata.
const HttpCodeMetadataKey = "X-Http-Code"

View File

@@ -0,0 +1,22 @@
load("@prysm//tools/go:def.bzl", "go_library", "go_test")
go_library(
name = "go_default_library",
srcs = ["pagination.go"],
importpath = "github.com/prysmaticlabs/prysm/v3/api/pagination",
visibility = ["//visibility:public"],
deps = [
"//config/params:go_default_library",
"@com_github_pkg_errors//:go_default_library",
],
)
go_test(
name = "go_default_test",
srcs = ["pagination_test.go"],
deps = [
":go_default_library",
"//testing/assert:go_default_library",
"//testing/require:go_default_library",
],
)

View File

@@ -0,0 +1,43 @@
// Package pagination contains useful pagination-related helpers.
package pagination
import (
"fmt"
"strconv"
"github.com/pkg/errors"
"github.com/prysmaticlabs/prysm/v3/config/params"
)
// StartAndEndPage takes in the requested page token, wanted page size, total page size.
// It returns start, end page and the next page token.
func StartAndEndPage(pageToken string, pageSize, totalSize int) (int, int, string, error) {
if pageToken == "" {
pageToken = "0"
}
if pageSize == 0 {
pageSize = params.BeaconConfig().DefaultPageSize
}
token, err := strconv.Atoi(pageToken)
if err != nil {
return 0, 0, "", errors.Wrap(err, "could not convert page token")
}
// Start page can not be greater than set size.
start := token * pageSize
if start >= totalSize {
return 0, 0, "", fmt.Errorf("page start %d >= list %d", start, totalSize)
}
// End page can not go out of bound.
end := start + pageSize
nextPageToken := strconv.Itoa(token + 1)
if end >= totalSize {
end = totalSize
nextPageToken = "" // Return an empty next page token for the last page of a set
}
return start, end, nextPageToken, nil
}

View File

@@ -0,0 +1,87 @@
package pagination_test
import (
"testing"
"github.com/prysmaticlabs/prysm/v3/api/pagination"
"github.com/prysmaticlabs/prysm/v3/testing/assert"
"github.com/prysmaticlabs/prysm/v3/testing/require"
)
func TestStartAndEndPage(t *testing.T) {
tests := []struct {
token string
pageSize int
totalSize int
nextToken string
start int
end int
}{
{
token: "0",
pageSize: 9,
totalSize: 100,
nextToken: "1",
start: 0,
end: 9,
},
{
token: "10",
pageSize: 4,
totalSize: 100,
nextToken: "11",
start: 40,
end: 44,
},
{
token: "100",
pageSize: 5,
totalSize: 1000,
nextToken: "101",
start: 500,
end: 505,
},
{
token: "3",
pageSize: 33,
totalSize: 100,
nextToken: "",
start: 99,
end: 100,
},
{
token: "34",
pageSize: 500,
totalSize: 17500,
nextToken: "",
start: 17000,
end: 17500,
},
}
for _, test := range tests {
start, end, next, err := pagination.StartAndEndPage(test.token, test.pageSize, test.totalSize)
require.NoError(t, err)
if test.start != start {
t.Errorf("expected start and computed start are not equal %d, %d", test.start, start)
}
if test.end != end {
t.Errorf("expected end and computed end are not equal %d, %d", test.end, end)
}
if test.nextToken != next {
t.Errorf("expected next token and computed next token are not equal %v, %v", test.nextToken, next)
}
}
}
func TestStartAndEndPage_CannotConvertPage(t *testing.T) {
wanted := "could not convert page token: strconv.Atoi: parsing"
_, _, _, err := pagination.StartAndEndPage("bad", 0, 0)
assert.ErrorContains(t, wanted, err)
}
func TestStartAndEndPage_ExceedsMaxPage(t *testing.T) {
wanted := "page start 0 >= list 0"
_, _, _, err := pagination.StartAndEndPage("", 0, 0)
assert.ErrorContains(t, wanted, err)
}

33
async/BUILD.bazel Normal file
View File

@@ -0,0 +1,33 @@
load("@prysm//tools/go:def.bzl", "go_library", "go_test")
go_library(
name = "go_default_library",
srcs = [
"debounce.go",
"every.go",
"multilock.go",
"scatter.go",
],
importpath = "github.com/prysmaticlabs/prysm/v3/async",
visibility = ["//visibility:public"],
deps = ["@com_github_sirupsen_logrus//:go_default_library"],
)
go_test(
name = "go_default_test",
srcs = [
"benchmark_test.go",
"debounce_test.go",
"every_test.go",
"multilock_test.go",
"scatter_test.go",
],
embed = [":go_default_library"],
deps = [
"//testing/assert:go_default_library",
"//testing/require:go_default_library",
"//testing/util:go_default_library",
"@com_github_sirupsen_logrus//:go_default_library",
"@com_github_stretchr_testify//assert:go_default_library",
],
)

14
async/abool/BUILD.bazel Normal file
View File

@@ -0,0 +1,14 @@
load("@prysm//tools/go:def.bzl", "go_library", "go_test")
go_library(
name = "go_default_library",
srcs = ["abool.go"],
importpath = "github.com/prysmaticlabs/prysm/v3/async/abool",
visibility = ["//visibility:public"],
)
go_test(
name = "go_default_test",
srcs = ["abool_test.go"],
embed = [":go_default_library"],
)

72
async/abool/abool.go Normal file
View File

@@ -0,0 +1,72 @@
// Package abool provides atomic Boolean type for cleaner code and
// better performance.
// All rights reserved to https://github.com/tevino/abool.
package abool
import "sync/atomic"
// New creates an AtomicBool with default set to false.
func New() *AtomicBool {
return new(AtomicBool)
}
// NewBool creates an AtomicBool with given default value.
func NewBool(ok bool) *AtomicBool {
ab := New()
if ok {
ab.Set()
}
return ab
}
// AtomicBool is an atomic Boolean.
// Its methods are all atomic, thus safe to be called by multiple goroutines simultaneously.
// Note: When embedding into a struct one should always use *AtomicBool to avoid copy.
type AtomicBool int32
// Set sets the Boolean to true.
func (ab *AtomicBool) Set() {
atomic.StoreInt32((*int32)(ab), 1)
}
// UnSet sets the Boolean to false.
func (ab *AtomicBool) UnSet() {
atomic.StoreInt32((*int32)(ab), 0)
}
// IsSet returns whether the Boolean is true.
func (ab *AtomicBool) IsSet() bool {
return atomic.LoadInt32((*int32)(ab))&1 == 1
}
// IsNotSet returns whether the Boolean is false.
func (ab *AtomicBool) IsNotSet() bool {
return !ab.IsSet()
}
// SetTo sets the boolean with given Boolean.
func (ab *AtomicBool) SetTo(yes bool) {
if yes {
atomic.StoreInt32((*int32)(ab), 1)
} else {
atomic.StoreInt32((*int32)(ab), 0)
}
}
// Toggle inverts the Boolean then returns the value before inverting.
func (ab *AtomicBool) Toggle() bool {
return atomic.AddInt32((*int32)(ab), 1)&1 == 0
}
// SetToIf sets the Boolean to new only if the Boolean matches the old.
// Returns whether the set was done.
func (ab *AtomicBool) SetToIf(old, curr bool) (set bool) {
var o, n int32
if old {
o = 1
}
if curr {
n = 1
}
return atomic.CompareAndSwapInt32((*int32)(ab), o, n)
}

298
async/abool/abool_test.go Normal file
View File

@@ -0,0 +1,298 @@
// All rights reserved to https://github.com/tevino/abool.
package abool
import (
"math"
"sync"
"sync/atomic"
"testing"
)
func TestDefaultValue(t *testing.T) {
t.Parallel()
v := New()
if v.IsSet() {
t.Fatal("Empty value of AtomicBool should be false")
}
v = NewBool(true)
if !v.IsSet() {
t.Fatal("NewValue(true) should be true")
}
v = NewBool(false)
if v.IsSet() {
t.Fatal("NewValue(false) should be false")
}
}
func TestIsNotSet(t *testing.T) {
t.Parallel()
v := New()
if v.IsSet() == v.IsNotSet() {
t.Fatal("AtomicBool.IsNotSet() should be the opposite of IsSet()")
}
}
func TestSetUnSet(t *testing.T) {
t.Parallel()
v := New()
v.Set()
if !v.IsSet() {
t.Fatal("AtomicBool.Set() failed")
}
v.UnSet()
if v.IsSet() {
t.Fatal("AtomicBool.UnSet() failed")
}
}
func TestSetTo(t *testing.T) {
t.Parallel()
v := New()
v.SetTo(true)
if !v.IsSet() {
t.Fatal("AtomicBool.SetTo(true) failed")
}
v.SetTo(false)
if v.IsSet() {
t.Fatal("AtomicBool.SetTo(false) failed")
}
if set := v.SetToIf(true, false); set || v.IsSet() {
t.Fatal("AtomicBool.SetTo(true, false) failed")
}
if set := v.SetToIf(false, true); !set || !v.IsSet() {
t.Fatal("AtomicBool.SetTo(false, true) failed")
}
}
func TestToggle(t *testing.T) {
t.Parallel()
v := New()
_ = v.Toggle()
if !v.IsSet() {
t.Fatal("AtomicBool.Toggle() to true failed")
}
prev := v.Toggle()
if v.IsSet() == prev {
t.Fatal("AtomicBool.Toggle() to false failed")
}
}
func TestToogleMultipleTimes(t *testing.T) {
t.Parallel()
v := New()
pre := !v.IsSet()
for i := 0; i < 100; i++ {
v.SetTo(false)
for j := 0; j < i; j++ {
pre = v.Toggle()
}
expected := i%2 != 0
if v.IsSet() != expected {
t.Fatalf("AtomicBool.Toogle() doesn't work after %d calls, expected: %v, got %v", i, expected, v.IsSet())
}
if pre == v.IsSet() {
t.Fatalf("AtomicBool.Toogle() returned wrong value at the %dth calls, expected: %v, got %v", i, !v.IsSet(), pre)
}
}
}
func TestToogleAfterOverflow(t *testing.T) {
t.Parallel()
var value int32 = math.MaxInt32
v := (*AtomicBool)(&value)
valueBeforeToggle := *(*int32)(v)
// test first toggle after overflow
v.Toggle()
expected := math.MaxInt32%2 == 0
if v.IsSet() != expected {
t.Fatalf("AtomicBool.Toogle() doesn't work after overflow, expected: %v, got %v", expected, v.IsSet())
}
// make sure overflow happened
var valueAfterToggle = *(*int32)(v)
if valueAfterToggle >= valueBeforeToggle {
t.Fatalf("Overflow does not happen as expected, before %d, after: %d", valueBeforeToggle, valueAfterToggle)
}
// test second toggle after overflow
v.Toggle()
expected = !expected
if v.IsSet() != expected {
t.Fatalf("AtomicBool.Toogle() doesn't work after the second call after overflow, expected: %v, got %v", expected, v.IsSet())
}
}
func TestRace(t *testing.T) {
t.Parallel()
repeat := 10000
var wg sync.WaitGroup
wg.Add(repeat * 4)
v := New()
// Writer
go func() {
for i := 0; i < repeat; i++ {
v.Set()
wg.Done()
}
}()
// Reader
go func() {
for i := 0; i < repeat; i++ {
v.IsSet()
wg.Done()
}
}()
// Writer
go func() {
for i := 0; i < repeat; i++ {
v.UnSet()
wg.Done()
}
}()
// Reader And Writer
go func() {
for i := 0; i < repeat; i++ {
v.Toggle()
wg.Done()
}
}()
wg.Wait()
}
func ExampleAtomicBool() {
cond := New() // default to false
cond.Set() // Sets to true
cond.IsSet() // Returns true
cond.UnSet() // Sets to false
cond.IsNotSet() // Returns true
cond.SetTo(true) // Sets to whatever you want
cond.SetToIf(true, false) // Sets to `new` only if the Boolean matches the `old`, returns whether succeeded
cond.Toggle() // Inverts the boolean then returns the value before inverting
}
// Benchmark Read
func BenchmarkMutexRead(b *testing.B) {
var m sync.RWMutex
var v bool
b.ResetTimer()
for i := 0; i < b.N; i++ {
m.RLock()
_ = v
m.RUnlock()
}
}
func BenchmarkAtomicValueRead(b *testing.B) {
var v atomic.Value
b.ResetTimer()
for i := 0; i < b.N; i++ {
_ = v.Load() != nil
}
}
func BenchmarkAtomicBoolRead(b *testing.B) {
v := New()
b.ResetTimer()
for i := 0; i < b.N; i++ {
_ = v.IsSet()
}
}
// Benchmark Write
func BenchmarkMutexWrite(b *testing.B) {
var m sync.RWMutex
var v bool
b.ResetTimer()
for i := 0; i < b.N; i++ {
m.RLock()
v = true
m.RUnlock()
}
b.StopTimer()
_ = v
}
func BenchmarkAtomicValueWrite(b *testing.B) {
var v atomic.Value
b.ResetTimer()
for i := 0; i < b.N; i++ {
v.Store(true)
}
}
func BenchmarkAtomicBoolWrite(b *testing.B) {
v := New()
b.ResetTimer()
for i := 0; i < b.N; i++ {
v.Set()
}
}
// Benchmark CAS
func BenchmarkMutexCAS(b *testing.B) {
var m sync.RWMutex
var v bool
b.ResetTimer()
for i := 0; i < b.N; i++ {
m.Lock()
if !v {
v = true
}
m.Unlock()
}
}
func BenchmarkAtomicBoolCAS(b *testing.B) {
v := New()
b.ResetTimer()
for i := 0; i < b.N; i++ {
v.SetToIf(false, true)
}
}
// Benchmark toggle
func BenchmarkMutexToggle(b *testing.B) {
var m sync.RWMutex
var v bool
b.ResetTimer()
for i := 0; i < b.N; i++ {
m.Lock()
v = !v
m.Unlock()
}
}
func BenchmarkAtomicBoolToggle(b *testing.B) {
v := New()
b.ResetTimer()
for i := 0; i < b.N; i++ {
v.Toggle()
}
}

63
async/benchmark_test.go Normal file
View File

@@ -0,0 +1,63 @@
package async_test
import (
"crypto/rand"
"crypto/sha256"
"sync"
"testing"
"github.com/prysmaticlabs/prysm/v3/async"
"github.com/prysmaticlabs/prysm/v3/testing/require"
log "github.com/sirupsen/logrus"
)
var input [][]byte
const (
benchmarkElements = 65536
benchmarkElementSize = 32
benchmarkHashRuns = 128
)
func init() {
input = make([][]byte, benchmarkElements)
for i := 0; i < benchmarkElements; i++ {
input[i] = make([]byte, benchmarkElementSize)
_, err := rand.Read(input[i])
if err != nil {
log.WithError(err).Debug("Cannot read from rand")
}
}
}
// hash repeatedly hashes the data passed to it
func hash(input [][]byte) [][]byte {
output := make([][]byte, len(input))
for i := range input {
copy(output, input)
for j := 0; j < benchmarkHashRuns; j++ {
hash := sha256.Sum256(output[i])
output[i] = hash[:]
}
}
return output
}
func BenchmarkHash(b *testing.B) {
for i := 0; i < b.N; i++ {
hash(input)
}
}
func BenchmarkHashMP(b *testing.B) {
output := make([][]byte, len(input))
for i := 0; i < b.N; i++ {
workerResults, err := async.Scatter(len(input), func(offset int, entries int, _ *sync.RWMutex) (interface{}, error) {
return hash(input[offset : offset+entries]), nil
})
require.NoError(b, err)
for _, result := range workerResults {
copy(output[result.Offset:], result.Extent.([][]byte))
}
}
}

45
async/debounce.go Normal file
View File

@@ -0,0 +1,45 @@
package async
import (
"context"
"time"
)
// Debounce events fired over a channel by a specified duration, ensuring no events
// are handled until a certain interval of time has passed.
func Debounce(ctx context.Context, interval time.Duration, eventsChan <-chan interface{}, handler func(interface{})) {
var timer *time.Timer
defer func() {
if timer != nil {
timer.Stop()
}
}()
for {
select {
// Wait until an event is triggered.
case event := <-eventsChan:
timer = time.NewTimer(interval)
loop:
for {
// If an event is received, wait the specified interval before calling the handler.
// If another event is received before the interval has passed, store
// it and reset the timer.
select {
case event = <-eventsChan:
// Reset timer.
timer.Stop()
timer = time.NewTimer(interval)
case <-timer.C:
// Stop the current timer, handle the request, and wait for more events.
timer.Stop()
handler(event)
break loop
case <-ctx.Done():
return
}
}
case <-ctx.Done():
return
}
}
}

117
async/debounce_test.go Normal file
View File

@@ -0,0 +1,117 @@
package async_test
import (
"context"
"sync"
"sync/atomic"
"testing"
"time"
"github.com/prysmaticlabs/prysm/v3/async"
"github.com/prysmaticlabs/prysm/v3/testing/assert"
"github.com/prysmaticlabs/prysm/v3/testing/require"
"github.com/prysmaticlabs/prysm/v3/testing/util"
)
func TestDebounce_NoEvents(t *testing.T) {
eventsChan := make(chan interface{}, 100)
ctx, cancel := context.WithCancel(context.Background())
interval := time.Second
timesHandled := int32(0)
wg := &sync.WaitGroup{}
wg.Add(1)
go func() {
time.AfterFunc(interval, func() {
cancel()
})
}()
go func() {
async.Debounce(ctx, interval, eventsChan, func(event interface{}) {
atomic.AddInt32(&timesHandled, 1)
})
wg.Done()
}()
if util.WaitTimeout(wg, interval*2) {
t.Fatalf("Test should have exited by now, timed out")
}
assert.Equal(t, int32(0), atomic.LoadInt32(&timesHandled), "Wrong number of handled calls")
}
func TestDebounce_CtxClosing(t *testing.T) {
eventsChan := make(chan interface{}, 100)
ctx, cancel := context.WithCancel(context.Background())
interval := time.Second
timesHandled := int32(0)
wg := &sync.WaitGroup{}
wg.Add(1)
go func() {
ticker := time.NewTicker(time.Millisecond * 100)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
eventsChan <- struct{}{}
}
}
}()
go func() {
time.AfterFunc(interval, func() {
cancel()
})
}()
go func() {
async.Debounce(ctx, interval, eventsChan, func(event interface{}) {
atomic.AddInt32(&timesHandled, 1)
})
wg.Done()
}()
if util.WaitTimeout(wg, interval*2) {
t.Fatalf("Test should have exited by now, timed out")
}
assert.Equal(t, int32(0), atomic.LoadInt32(&timesHandled), "Wrong number of handled calls")
}
func TestDebounce_SingleHandlerInvocation(t *testing.T) {
eventsChan := make(chan interface{}, 100)
ctx, cancel := context.WithCancel(context.Background())
interval := time.Second
timesHandled := int32(0)
go async.Debounce(ctx, interval, eventsChan, func(event interface{}) {
atomic.AddInt32(&timesHandled, 1)
})
for i := 0; i < 100; i++ {
eventsChan <- struct{}{}
}
// We should expect 100 rapid fire changes to only have caused
// 1 handler to trigger after the debouncing period.
time.Sleep(interval * 2)
assert.Equal(t, int32(1), atomic.LoadInt32(&timesHandled), "Wrong number of handled calls")
cancel()
}
func TestDebounce_MultipleHandlerInvocation(t *testing.T) {
eventsChan := make(chan interface{}, 100)
ctx, cancel := context.WithCancel(context.Background())
interval := time.Second
timesHandled := int32(0)
go async.Debounce(ctx, interval, eventsChan, func(event interface{}) {
atomic.AddInt32(&timesHandled, 1)
})
for i := 0; i < 100; i++ {
eventsChan <- struct{}{}
}
require.Equal(t, int32(0), atomic.LoadInt32(&timesHandled), "Events must prevent from handler execution")
// By this time the first event should be triggered.
time.Sleep(2 * time.Second)
assert.Equal(t, int32(1), atomic.LoadInt32(&timesHandled), "Wrong number of handled calls")
// Second event.
eventsChan <- struct{}{}
time.Sleep(2 * time.Second)
assert.Equal(t, int32(2), atomic.LoadInt32(&timesHandled), "Wrong number of handled calls")
cancel()
}

29
async/event/BUILD.bazel Normal file
View File

@@ -0,0 +1,29 @@
load("@prysm//tools/go:def.bzl", "go_library", "go_test")
go_library(
name = "go_default_library",
srcs = [
"feed.go",
"subscription.go",
],
importpath = "github.com/prysmaticlabs/prysm/v3/async/event",
visibility = ["//visibility:public"],
deps = ["//time/mclock:go_default_library"],
)
go_test(
name = "go_default_test",
size = "small",
srcs = [
"example_feed_test.go",
"example_scope_test.go",
"example_subscription_test.go",
"feed_test.go",
"subscription_test.go",
],
embed = [":go_default_library"],
deps = [
"//testing/assert:go_default_library",
"//testing/require:go_default_library",
],
)

View File

@@ -19,7 +19,7 @@ package event_test
import (
"fmt"
"github.com/prysmaticlabs/prysm/shared/event"
"github.com/prysmaticlabs/prysm/v3/async/event"
)
func ExampleFeed_acknowledgedEvents() {

View File

@@ -20,7 +20,7 @@ import (
"fmt"
"sync"
"github.com/prysmaticlabs/prysm/shared/event"
"github.com/prysmaticlabs/prysm/v3/async/event"
)
// This example demonstrates how SubscriptionScope can be used to control the lifetime of

View File

@@ -19,7 +19,7 @@ package event_test
import (
"fmt"
"github.com/prysmaticlabs/prysm/shared/event"
"github.com/prysmaticlabs/prysm/v3/async/event"
)
func ExampleNewSubscription() {

View File

@@ -14,6 +14,7 @@
// You should have received a copy of the GNU Lesser General Public License
// along with the go-ethereum library. If not, see <http://www.gnu.org/licenses/>.
// Package event contains an event feed implementation for process communication.
package event
import (
@@ -97,6 +98,11 @@ func (f *Feed) typecheck(typ reflect.Type) bool {
f.etype = typ
return true
}
// In the event the feed's type is an actual interface, we
// perform an interface conformance check here.
if f.etype.Kind() == reflect.Interface && typ.Implements(f.etype) {
return true
}
return f.etype == typ
}
@@ -195,6 +201,7 @@ type feedSub struct {
err chan error
}
// Unsubscribe remove feed subscription.
func (sub *feedSub) Unsubscribe() {
sub.errOnce.Do(func() {
sub.feed.remove(sub)
@@ -202,6 +209,7 @@ func (sub *feedSub) Unsubscribe() {
})
}
// Err returns error channel.
func (sub *feedSub) Err() <-chan error {
return sub.err
}

507
async/event/feed_test.go Normal file
View File

@@ -0,0 +1,507 @@
// Copyright 2016 The go-ethereum Authors
// This file is part of the go-ethereum library.
//
// The go-ethereum library is free software: you can redistribute it and/or modify
// it under the terms of the GNU Lesser General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// The go-ethereum library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public License
// along with the go-ethereum library. If not, see <http://www.gnu.org/licenses/>.
package event
import (
"fmt"
"reflect"
"sync"
"testing"
"time"
"github.com/prysmaticlabs/prysm/v3/testing/assert"
)
func TestFeedPanics(t *testing.T) {
{
var f Feed
f.Send(2)
want := feedTypeError{op: "Send", got: reflect.TypeOf(uint64(0)), want: reflect.TypeOf(0)}
assert.NoError(t, checkPanic(want, func() { f.Send(uint64(2)) }))
}
{
var f Feed
ch := make(chan int)
f.Subscribe(ch)
want := feedTypeError{op: "Send", got: reflect.TypeOf(uint64(0)), want: reflect.TypeOf(0)}
assert.NoError(t, checkPanic(want, func() { f.Send(uint64(2)) }))
}
{
var f Feed
f.Send(2)
want := feedTypeError{op: "Subscribe", got: reflect.TypeOf(make(chan uint64)), want: reflect.TypeOf(make(chan<- int))}
assert.NoError(t, checkPanic(want, func() { f.Subscribe(make(chan uint64)) }))
}
{
var f Feed
assert.NoError(t, checkPanic(errBadChannel, func() { f.Subscribe(make(<-chan int)) }))
}
{
var f Feed
assert.NoError(t, checkPanic(errBadChannel, func() { f.Subscribe(0) }))
}
}
func checkPanic(want error, fn func()) (err error) {
defer func() {
panicResult := recover()
if panicResult == nil {
err = fmt.Errorf("didn't panic")
} else if !reflect.DeepEqual(panicResult, want) {
err = fmt.Errorf("panicked with wrong error: got %q, want %q", panicResult, want)
}
}()
fn()
return nil
}
func TestFeed(t *testing.T) {
var feed Feed
var done, subscribed sync.WaitGroup
subscriber := func(i int) {
defer done.Done()
subchan := make(chan int)
sub := feed.Subscribe(subchan)
timeout := time.NewTimer(2 * time.Second)
subscribed.Done()
select {
case v := <-subchan:
if v != 1 {
t.Errorf("%d: received value %d, want 1", i, v)
}
case <-timeout.C:
t.Errorf("%d: receive timeout", i)
}
sub.Unsubscribe()
select {
case _, ok := <-sub.Err():
if ok {
t.Errorf("%d: error channel not closed after unsubscribe", i)
}
case <-timeout.C:
t.Errorf("%d: unsubscribe timeout", i)
}
}
const n = 1000
done.Add(n)
subscribed.Add(n)
for i := 0; i < n; i++ {
go subscriber(i)
}
subscribed.Wait()
if nsent := feed.Send(1); nsent != n {
t.Errorf("first send delivered %d times, want %d", nsent, n)
}
if nsent := feed.Send(2); nsent != 0 {
t.Errorf("second send delivered %d times, want 0", nsent)
}
done.Wait()
}
func TestFeedSubscribeSameChannel(t *testing.T) {
var (
feed Feed
done sync.WaitGroup
ch = make(chan int)
sub1 = feed.Subscribe(ch)
sub2 = feed.Subscribe(ch)
_ = feed.Subscribe(ch)
)
expectSends := func(value, n int) {
if nsent := feed.Send(value); nsent != n {
t.Errorf("send delivered %d times, want %d", nsent, n)
}
done.Done()
}
expectRecv := func(wantValue, n int) {
for i := 0; i < n; i++ {
if v := <-ch; v != wantValue {
t.Errorf("received %d, want %d", v, wantValue)
}
}
}
done.Add(1)
go expectSends(1, 3)
expectRecv(1, 3)
done.Wait()
sub1.Unsubscribe()
done.Add(1)
go expectSends(2, 2)
expectRecv(2, 2)
done.Wait()
sub2.Unsubscribe()
done.Add(1)
go expectSends(3, 1)
expectRecv(3, 1)
done.Wait()
}
func TestFeedSubscribeBlockedPost(_ *testing.T) {
var (
feed Feed
nsends = 2000
ch1 = make(chan int)
ch2 = make(chan int)
wg sync.WaitGroup
)
defer wg.Wait()
feed.Subscribe(ch1)
wg.Add(nsends)
for i := 0; i < nsends; i++ {
go func() {
feed.Send(99)
wg.Done()
}()
}
sub2 := feed.Subscribe(ch2)
defer sub2.Unsubscribe()
// We're done when ch1 has received N times.
// The number of receives on ch2 depends on scheduling.
for i := 0; i < nsends; {
select {
case <-ch1:
i++
case <-ch2:
}
}
}
func TestFeedUnsubscribeBlockedPost(_ *testing.T) {
var (
feed Feed
nsends = 200
chans = make([]chan int, 2000)
subs = make([]Subscription, len(chans))
bchan = make(chan int)
bsub = feed.Subscribe(bchan)
wg sync.WaitGroup
)
for i := range chans {
chans[i] = make(chan int, nsends)
}
// Queue up some Sends. None of these can make progress while bchan isn't read.
wg.Add(nsends)
for i := 0; i < nsends; i++ {
go func() {
feed.Send(99)
wg.Done()
}()
}
// Subscribe the other channels.
for i, ch := range chans {
subs[i] = feed.Subscribe(ch)
}
// Unsubscribe them again.
for _, sub := range subs {
sub.Unsubscribe()
}
// Unblock the Sends.
bsub.Unsubscribe()
wg.Wait()
}
// Checks that unsubscribing a channel during Send works even if that
// channel has already been sent on.
func TestFeedUnsubscribeSentChan(_ *testing.T) {
var (
feed Feed
ch1 = make(chan int)
ch2 = make(chan int)
sub1 = feed.Subscribe(ch1)
sub2 = feed.Subscribe(ch2)
wg sync.WaitGroup
)
defer sub2.Unsubscribe()
wg.Add(1)
go func() {
feed.Send(0)
wg.Done()
}()
// Wait for the value on ch1.
<-ch1
// Unsubscribe ch1, removing it from the send cases.
sub1.Unsubscribe()
// Receive ch2, finishing Send.
<-ch2
wg.Wait()
// Send again. This should send to ch2 only, so the wait group will unblock
// as soon as a value is received on ch2.
wg.Add(1)
go func() {
feed.Send(0)
wg.Done()
}()
<-ch2
wg.Wait()
}
func TestFeedUnsubscribeFromInbox(t *testing.T) {
var (
feed Feed
ch1 = make(chan int)
ch2 = make(chan int)
sub1 = feed.Subscribe(ch1)
sub2 = feed.Subscribe(ch1)
sub3 = feed.Subscribe(ch2)
)
assert.Equal(t, 3, len(feed.inbox))
assert.Equal(t, 1, len(feed.sendCases), "sendCases is non-empty after unsubscribe")
sub1.Unsubscribe()
sub2.Unsubscribe()
sub3.Unsubscribe()
assert.Equal(t, 0, len(feed.inbox), "Inbox is non-empty after unsubscribe")
assert.Equal(t, 1, len(feed.sendCases), "sendCases is non-empty after unsubscribe")
}
func BenchmarkFeedSend1000(b *testing.B) {
var (
done sync.WaitGroup
feed Feed
nsubs = 1000
)
subscriber := func(ch <-chan int) {
for i := 0; i < b.N; i++ {
<-ch
}
done.Done()
}
done.Add(nsubs)
for i := 0; i < nsubs; i++ {
ch := make(chan int, 200)
feed.Subscribe(ch)
go subscriber(ch)
}
// The actual benchmark.
b.ResetTimer()
for i := 0; i < b.N; i++ {
if feed.Send(i) != nsubs {
panic("wrong number of sends")
}
}
b.StopTimer()
done.Wait()
}
func TestFeed_Send(t *testing.T) {
tests := []struct {
name string
evFeed *Feed
testSetup func(fd *Feed, t *testing.T, o interface{})
obj interface{}
expectPanic bool
}{
{
name: "normal struct",
evFeed: new(Feed),
testSetup: func(fd *Feed, t *testing.T, o interface{}) {
testChan := make(chan testFeedWithPointer, 1)
fd.Subscribe(testChan)
},
obj: testFeedWithPointer{
a: new(uint64),
b: new(string),
},
expectPanic: false,
},
{
name: "un-implemented interface",
evFeed: new(Feed),
testSetup: func(fd *Feed, t *testing.T, o interface{}) {
testChan := make(chan testFeedIface, 1)
fd.Subscribe(testChan)
},
obj: testFeedWithPointer{
a: new(uint64),
b: new(string),
},
expectPanic: true,
},
{
name: "semi-implemented interface",
evFeed: new(Feed),
testSetup: func(fd *Feed, t *testing.T, o interface{}) {
testChan := make(chan testFeedIface, 1)
fd.Subscribe(testChan)
},
obj: testFeed2{
a: 0,
b: "",
c: []byte{'A'},
},
expectPanic: true,
},
{
name: "fully-implemented interface",
evFeed: new(Feed),
testSetup: func(fd *Feed, t *testing.T, o interface{}) {
testChan := make(chan testFeedIface)
// Make it unbuffered to allow message to
// pass through
go func() {
a := <-testChan
if !reflect.DeepEqual(a, o) {
t.Errorf("Got = %v, want = %v", a, o)
}
}()
fd.Subscribe(testChan)
},
obj: testFeed{
a: 0,
b: "",
},
expectPanic: false,
},
{
name: "fully-implemented interface with additional methods",
evFeed: new(Feed),
testSetup: func(fd *Feed, t *testing.T, o interface{}) {
testChan := make(chan testFeedIface)
// Make it unbuffered to allow message to
// pass through
go func() {
a := <-testChan
if !reflect.DeepEqual(a, o) {
t.Errorf("Got = %v, want = %v", a, o)
}
}()
fd.Subscribe(testChan)
},
obj: testFeed3{
a: 0,
b: "",
c: []byte{'A'},
d: []byte{'B'},
},
expectPanic: false,
},
{
name: "concrete types implementing the same interface",
evFeed: new(Feed),
testSetup: func(fd *Feed, t *testing.T, o interface{}) {
testChan := make(chan testFeed, 1)
// Make it unbuffered to allow message to
// pass through
go func() {
a := <-testChan
if !reflect.DeepEqual(a, o) {
t.Errorf("Got = %v, want = %v", a, o)
}
}()
fd.Subscribe(testChan)
},
obj: testFeed3{
a: 0,
b: "",
c: []byte{'A'},
d: []byte{'B'},
},
expectPanic: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
defer func() {
if r := recover(); r != nil {
if !tt.expectPanic {
t.Errorf("panic triggered when unexpected: %v", r)
}
} else {
if tt.expectPanic {
t.Error("panic not triggered when expected")
}
}
}()
tt.testSetup(tt.evFeed, t, tt.obj)
if gotNsent := tt.evFeed.Send(tt.obj); gotNsent != 1 {
t.Errorf("Send() = %v, want %v", gotNsent, 1)
}
})
}
}
// The following objects below are a collection of different
// struct types to test with.
type testFeed struct {
a uint64
b string
}
func (testFeed) method1() {
}
func (testFeed) method2() {
}
type testFeedWithPointer struct {
a *uint64
b *string
}
type testFeed2 struct {
a uint64
b string
c []byte
}
func (testFeed2) method1() {
}
type testFeed3 struct {
a uint64
b string
c, d []byte
}
func (testFeed3) method1() {
}
func (testFeed3) method2() {
}
func (testFeed3) method3() {
}
type testFeedIface interface {
method1()
method2()
}

View File

@@ -21,9 +21,13 @@ import (
"sync"
"time"
"github.com/prysmaticlabs/prysm/shared/mclockutil"
"github.com/prysmaticlabs/prysm/v3/time/mclock"
)
// waitQuotient is divided against the max backoff time, in order to have N requests based on the full
// request backoff time.
const waitQuotient = 10
// Subscription represents a stream of events. The carrier of the events is typically a
// channel, but isn't part of the interface.
//
@@ -70,6 +74,7 @@ type funcSub struct {
unsubscribed bool
}
// Unsubscribe unsubscribes from subscription.
func (s *funcSub) Unsubscribe() {
s.mu.Lock()
if s.unsubscribed {
@@ -83,6 +88,7 @@ func (s *funcSub) Unsubscribe() {
<-s.err
}
// Err exposes error channel.
func (s *funcSub) Err() <-chan error {
return s.err
}
@@ -96,7 +102,7 @@ func (s *funcSub) Err() <-chan error {
// based on the error rate, but will never exceed backoffMax.
func Resubscribe(backoffMax time.Duration, fn ResubscribeFunc) Subscription {
s := &resubscribeSub{
waitTime: backoffMax / 10,
waitTime: backoffMax / waitQuotient,
backoffMax: backoffMax,
fn: fn,
err: make(chan error),
@@ -114,10 +120,11 @@ type resubscribeSub struct {
err chan error
unsub chan struct{}
unsubOnce sync.Once
lastTry mclockutil.AbsTime
lastTry mclock.AbsTime
waitTime, backoffMax time.Duration
}
// Unsubscribe unsubscribes from subscription.
func (s *resubscribeSub) Unsubscribe() {
s.unsubOnce.Do(func() {
s.unsub <- struct{}{}
@@ -125,6 +132,7 @@ func (s *resubscribeSub) Unsubscribe() {
})
}
// Err exposes error channel.
func (s *resubscribeSub) Err() <-chan error {
return s.err
}
@@ -143,12 +151,12 @@ func (s *resubscribeSub) loop() {
}
func (s *resubscribeSub) subscribe() Subscription {
subscribed := make(chan error)
subscribed := make(chan error, 1)
var sub Subscription
retry:
for {
s.lastTry = mclockutil.Now()
ctx, cancel := context.WithCancel(context.Background())
s.lastTry = mclock.Now()
ctx, cancel := context.WithCancel(context.TODO())
go func() {
rsub, err := s.fn(ctx)
sub = rsub
@@ -186,8 +194,8 @@ func (s *resubscribeSub) waitForError(sub Subscription) bool {
}
func (s *resubscribeSub) backoffWait() bool {
if time.Duration(mclockutil.Now()-s.lastTry) > s.backoffMax {
s.waitTime = s.backoffMax / 10
if time.Duration(mclock.Now()-s.lastTry) > s.backoffMax {
s.waitTime = s.backoffMax / waitQuotient
} else {
s.waitTime *= 2
if s.waitTime > s.backoffMax {
@@ -263,6 +271,7 @@ func (sc *SubscriptionScope) Count() int {
return len(sc.subs)
}
// Unsubscribe unsubscribes from subscription.
func (s *scopeSub) Unsubscribe() {
s.s.Unsubscribe()
s.sc.mu.Lock()
@@ -270,6 +279,7 @@ func (s *scopeSub) Unsubscribe() {
delete(s.sc.subs, s)
}
// Err exposes error channel.
func (s *scopeSub) Err() <-chan error {
return s.s.Err()
}

View File

@@ -19,8 +19,11 @@ package event
import (
"context"
"errors"
"runtime"
"testing"
"time"
"github.com/prysmaticlabs/prysm/v3/testing/require"
)
var errInts = errors.New("error in subscribeInts")
@@ -50,25 +53,17 @@ loop:
for want := 0; want < 10; want++ {
select {
case got := <-channel:
if got != want {
t.Fatalf("wrong int %d, want %d", got, want)
}
require.Equal(t, want, got)
case err := <-sub.Err():
if err != errInts {
t.Fatalf("wrong error: got %q, want %q", err, errInts)
}
if want != 2 {
t.Fatalf("got errInts at int %d, should be received at 2", want)
}
require.Equal(t, errInts, err)
require.Equal(t, 2, want)
break loop
}
}
sub.Unsubscribe()
err, ok := <-sub.Err()
if err != nil {
t.Fatal("got non-nil error after Unsubscribe")
}
require.NoError(t, err)
if ok {
t.Fatal("channel still open after Unsubscribe")
}
@@ -94,9 +89,7 @@ func TestResubscribe(t *testing.T) {
})
<-sub.Err()
if i != nfails {
t.Fatalf("resubscribe function called %d times, want %d times", i, nfails)
}
require.Equal(t, nfails, i)
}
func TestResubscribeAbort(t *testing.T) {
@@ -114,7 +107,20 @@ func TestResubscribeAbort(t *testing.T) {
})
sub.Unsubscribe()
if err := <-done; err != nil {
t.Fatal(err)
}
require.NoError(t, <-done)
}
func TestResubscribeNonBlocking(t *testing.T) {
done := make(chan struct{})
sub := Resubscribe(0, func(ctx context.Context) (Subscription, error) {
<-done
return nil, nil
})
resub, ok := sub.(*resubscribeSub)
require.Equal(t, true, ok)
currNum := runtime.NumGoroutine()
resub.unsub <- struct{}{}
done <- struct{}{}
require.Equal(t, currNum-1, runtime.NumGoroutine())
}

31
async/every.go Normal file
View File

@@ -0,0 +1,31 @@
// Package async includes helpers for scheduling runnable, periodic functions and contains useful helpers for converting multi-processor computation.
package async
import (
"context"
"reflect"
"runtime"
"time"
log "github.com/sirupsen/logrus"
)
// RunEvery runs the provided command periodically.
// It runs in a goroutine, and can be cancelled by finishing the supplied context.
func RunEvery(ctx context.Context, period time.Duration, f func()) {
funcName := runtime.FuncForPC(reflect.ValueOf(f).Pointer()).Name()
ticker := time.NewTicker(period)
go func() {
for {
select {
case <-ticker.C:
log.WithField("function", funcName).Trace("running")
f()
case <-ctx.Done():
log.WithField("function", funcName).Debug("context is closed, exiting")
ticker.Stop()
return
}
}
}()
}

40
async/every_test.go Normal file
View File

@@ -0,0 +1,40 @@
package async_test
import (
"context"
"sync/atomic"
"testing"
"time"
"github.com/prysmaticlabs/prysm/v3/async"
)
func TestEveryRuns(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
i := int32(0)
async.RunEvery(ctx, 100*time.Millisecond, func() {
atomic.AddInt32(&i, 1)
})
// Sleep for a bit and ensure the value has increased.
time.Sleep(200 * time.Millisecond)
if atomic.LoadInt32(&i) == 0 {
t.Error("Counter failed to increment with ticker")
}
cancel()
// Sleep for a bit to let the cancel take place.
time.Sleep(100 * time.Millisecond)
last := atomic.LoadInt32(&i)
// Sleep for a bit and ensure the value has not increased.
time.Sleep(200 * time.Millisecond)
if atomic.LoadInt32(&i) != last {
t.Error("Counter incremented after stop")
}
}

141
async/multilock.go Normal file
View File

@@ -0,0 +1,141 @@
/*
Copyright 2017 Albert Tedja
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package async
import (
"runtime"
"sort"
)
var locks = struct {
lock chan byte
list map[string]chan byte
}{
lock: make(chan byte, 1),
list: make(map[string]chan byte),
}
type Lock struct {
keys []string
chans []chan byte
lock chan byte
unlock chan byte
}
func (lk *Lock) Lock() {
lk.lock <- 1
// get the channels and attempt to acquire them
lk.chans = make([]chan byte, 0, len(lk.keys))
for i := 0; i < len(lk.keys); {
ch := getChan(lk.keys[i])
_, ok := <-ch
if ok {
lk.chans = append(lk.chans, ch)
i++
}
}
lk.unlock <- 1
}
// Unlock unlocks this lock. Must be called after Lock.
// Can only be invoked if there is a previous call to Lock.
func (lk *Lock) Unlock() {
<-lk.unlock
if lk.chans != nil {
for _, ch := range lk.chans {
ch <- 1
}
lk.chans = nil
}
// Clean unused channels after the unlock.
Clean()
<-lk.lock
}
// Yield temporarily unlocks, gives up the cpu time to other goroutine, and attempts to lock again.
func (lk *Lock) Yield() {
lk.Unlock()
runtime.Gosched()
lk.Lock()
}
// NewMultilock creates a new multilock for the specified keys
func NewMultilock(locks ...string) *Lock {
if len(locks) == 0 {
return nil
}
locks = unique(locks)
sort.Strings(locks)
return &Lock{
keys: locks,
lock: make(chan byte, 1),
unlock: make(chan byte, 1),
}
}
// Clean cleans old unused locks. Returns removed keys.
func Clean() []string {
locks.lock <- 1
defer func() { <-locks.lock }()
toDelete := make([]string, 0, len(locks.list))
for key, ch := range locks.list {
select {
case <-ch:
close(ch)
toDelete = append(toDelete, key)
default:
}
}
for _, del := range toDelete {
delete(locks.list, del)
}
return toDelete
}
// Create and get the channel for the specified key.
func getChan(key string) chan byte {
locks.lock <- 1
defer func() { <-locks.lock }()
if locks.list[key] == nil {
locks.list[key] = make(chan byte, 1)
locks.list[key] <- 1
}
return locks.list[key]
}
// Return a new string with unique elements.
func unique(arr []string) []string {
if arr == nil || len(arr) <= 1 {
return arr
}
found := map[string]bool{}
result := make([]string, 0, len(arr))
for _, v := range arr {
if !found[v] {
found[v] = true
result = append(result, v)
}
}
return result
}

343
async/multilock_test.go Normal file
View File

@@ -0,0 +1,343 @@
/*
Copyright 2017 Albert Tedja
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package async
import (
"sync"
"testing"
"time"
"github.com/stretchr/testify/assert"
)
func TestUnique(t *testing.T) {
var arr []string
a := assert.New(t)
arr = []string{"a", "b", "c"}
a.Equal(arr, unique(arr))
arr = []string{"a", "a", "a"}
a.Equal([]string{"a"}, unique(arr))
arr = []string{"a", "a", "b"}
a.Equal([]string{"a", "b"}, unique(arr))
arr = []string{"a", "b", "a"}
a.Equal([]string{"a", "b"}, unique(arr))
arr = []string{"a", "b", "c", "b", "d"}
a.Equal([]string{"a", "b", "c", "d"}, unique(arr))
}
func TestGetChan(t *testing.T) {
ch1 := getChan("a")
ch2 := getChan("aa")
ch3 := getChan("a")
a := assert.New(t)
a.NotEqual(ch1, ch2)
a.Equal(ch1, ch3)
}
func TestLockUnlock(_ *testing.T) {
var wg sync.WaitGroup
wg.Add(5)
go func() {
lock := NewMultilock("dog", "cat", "owl")
lock.Lock()
defer lock.Unlock()
<-time.After(100 * time.Millisecond)
wg.Done()
}()
go func() {
lock := NewMultilock("cat", "dog", "bird")
lock.Lock()
defer lock.Unlock()
<-time.After(100 * time.Millisecond)
wg.Done()
}()
go func() {
lock := NewMultilock("cat", "bird", "owl")
lock.Lock()
defer lock.Unlock()
<-time.After(100 * time.Millisecond)
wg.Done()
}()
go func() {
lock := NewMultilock("bird", "owl", "snake")
lock.Lock()
defer lock.Unlock()
<-time.After(100 * time.Millisecond)
wg.Done()
}()
go func() {
lock := NewMultilock("owl", "snake")
lock.Lock()
defer lock.Unlock()
<-time.After(1 * time.Second)
wg.Done()
}()
wg.Wait()
}
func TestLockUnlock_CleansUnused(t *testing.T) {
var wg sync.WaitGroup
wg.Add(1)
go func() {
lock := NewMultilock("dog", "cat", "owl")
lock.Lock()
assert.Equal(t, 3, len(locks.list))
lock.Unlock()
wg.Done()
}()
wg.Wait()
// We expect that unlocking completely cleared the locks list
// given all 3 lock keys were unused at time of unlock.
assert.Equal(t, 0, len(locks.list))
}
func TestLockUnlock_DoesNotCleanIfHeldElsewhere(t *testing.T) {
var wg sync.WaitGroup
wg.Add(2)
go func() {
lock := NewMultilock("cat")
lock.Lock()
// We take 200 milliseconds to release the lock on "cat"
<-time.After(200 * time.Millisecond)
lock.Unlock()
// Assert that at the end of this goroutine, all locks are cleared.
assert.Equal(t, 0, len(locks.list))
wg.Done()
}()
go func() {
lock := NewMultilock("dog", "cat", "owl")
lock.Lock()
// We release the locks after 100 milliseconds, and check that "cat" is not
// cleared as a lock for it is still held by the previous goroutine.
<-time.After(100 * time.Millisecond)
lock.Unlock()
assert.Equal(t, 1, len(locks.list))
_, ok := locks.list["cat"]
assert.Equal(t, true, ok)
wg.Done()
}()
wg.Wait()
// We expect that at the end of this test, all locks are cleared.
assert.Equal(t, 0, len(locks.list))
}
func TestYield(t *testing.T) {
var wg sync.WaitGroup
wg.Add(2)
var resources = map[string]int{}
go func() {
lock := NewMultilock("A", "C")
lock.Lock()
defer lock.Unlock()
for resources["ac"] == 0 {
lock.Yield()
}
resources["dc"] = 10
wg.Done()
}()
go func() {
lock := NewMultilock("D", "C")
lock.Lock()
defer lock.Unlock()
resources["ac"] = 5
for resources["dc"] == 0 {
lock.Yield()
}
wg.Done()
}()
wg.Wait()
assert.Equal(t, 5, resources["ac"])
assert.Equal(t, 10, resources["dc"])
}
func TestClean(t *testing.T) {
var wg sync.WaitGroup
wg.Add(3)
// some goroutine that holds multiple locks
go1done := make(chan bool, 1)
go func() {
Loop:
for {
select {
case <-go1done:
break Loop
default:
lock := NewMultilock("A", "B", "C", "E", "Z")
lock.Lock()
<-time.After(30 * time.Millisecond)
lock.Unlock()
}
}
wg.Done()
}()
// another goroutine
go2done := make(chan bool, 1)
go func() {
Loop:
for {
select {
case <-go2done:
break Loop
default:
lock := NewMultilock("B", "C", "K", "L", "Z")
lock.Lock()
<-time.After(200 * time.Millisecond)
lock.Unlock()
}
}
wg.Done()
}()
// this one cleans up the locks every 100 ms
done := make(chan bool, 1)
go func() {
c := time.Tick(100 * time.Millisecond)
Loop:
for {
select {
case <-done:
break Loop
case <-c:
Clean()
}
}
wg.Done()
}()
<-time.After(2 * time.Second)
go1done <- true
go2done <- true
<-time.After(1 * time.Second)
done <- true
wg.Wait()
assert.Equal(t, []string{}, Clean())
}
func TestBankAccountProblem(t *testing.T) {
var wg sync.WaitGroup
wg.Add(3)
joe := 50.0
susan := 100.0
// withdraw $80 from joe, only if balance is sufficient
go func() {
lock := NewMultilock("joe")
lock.Lock()
defer lock.Unlock()
for joe < 80.0 {
lock.Yield()
}
joe -= 80.0
wg.Done()
}()
// transfer $200 from susan to joe, only if balance is sufficient
go func() {
lock := NewMultilock("joe", "susan")
lock.Lock()
defer lock.Unlock()
for susan < 200.0 {
lock.Yield()
}
susan -= 200.0
joe += 200.0
wg.Done()
}()
// susan deposit $300 to cover balance
go func() {
lock := NewMultilock("susan")
lock.Lock()
defer lock.Unlock()
susan += 300.0
wg.Done()
}()
wg.Wait()
assert.Equal(t, 170.0, joe)
assert.Equal(t, 200.0, susan)
}
func TestSyncCondCompatibility(t *testing.T) {
var wg sync.WaitGroup
wg.Add(2)
cond := sync.NewCond(NewMultilock("A", "C"))
var testValues = [3]string{"foo", "bar", "fizz!"}
sharedRsc := testValues[0]
go func() {
cond.L.Lock()
for sharedRsc == testValues[0] {
cond.Wait()
}
sharedRsc = testValues[2]
cond.Broadcast()
cond.L.Unlock()
wg.Done()
}()
go func() {
cond.L.Lock()
sharedRsc = testValues[1]
cond.Broadcast()
for sharedRsc == testValues[1] {
cond.Wait()
}
cond.L.Unlock()
wg.Done()
}()
wg.Wait()
assert.Equal(t, testValues[2], sharedRsc)
}

View File

@@ -1,4 +1,4 @@
package mputil
package async
import (
"errors"

108
async/scatter_test.go Normal file
View File

@@ -0,0 +1,108 @@
package async_test
import (
"errors"
"sync"
"testing"
"github.com/prysmaticlabs/prysm/v3/async"
"github.com/prysmaticlabs/prysm/v3/testing/assert"
"github.com/prysmaticlabs/prysm/v3/testing/require"
)
func TestDouble(t *testing.T) {
tests := []struct {
name string
inValues int
err error
}{
{
name: "0",
inValues: 0,
err: errors.New("input length must be greater than 0"),
},
{
name: "1",
inValues: 1,
},
{
name: "1023",
inValues: 1023,
},
{
name: "1024",
inValues: 1024,
},
{
name: "1025",
inValues: 1025,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
inValues := make([]int, test.inValues)
for i := 0; i < test.inValues; i++ {
inValues[i] = i
}
outValues := make([]int, test.inValues)
workerResults, err := async.Scatter(len(inValues), func(offset int, entries int, _ *sync.RWMutex) (interface{}, error) {
extent := make([]int, entries)
for i := 0; i < entries; i++ {
extent[i] = inValues[offset+i] * 2
}
return extent, nil
})
if test.err != nil {
assert.ErrorContains(t, test.err.Error(), err)
} else {
require.NoError(t, err)
for _, result := range workerResults {
copy(outValues[result.Offset:], result.Extent.([]int))
}
for i := 0; i < test.inValues; i++ {
require.Equal(t, inValues[i]*2, outValues[i], "Outvalue at %d mismatch", i)
}
}
})
}
}
func TestMutex(t *testing.T) {
totalRuns := 1048576
val := 0
_, err := async.Scatter(totalRuns, func(offset int, entries int, mu *sync.RWMutex) (interface{}, error) {
for i := 0; i < entries; i++ {
mu.Lock()
val++
mu.Unlock()
}
return nil, nil
})
require.NoError(t, err)
if val != totalRuns {
t.Fatalf("Unexpected value: expected \"%v\", found \"%v\"", totalRuns, val)
}
}
func TestError(t *testing.T) {
totalRuns := 1024
val := 0
_, err := async.Scatter(totalRuns, func(offset int, entries int, mu *sync.RWMutex) (interface{}, error) {
for i := 0; i < entries; i++ {
mu.Lock()
val++
if val == 1011 {
mu.Unlock()
return nil, errors.New("bad number")
}
mu.Unlock()
}
return nil, nil
})
if err == nil {
t.Fatalf("Missing expected error")
}
}

11
bazel.sh Executable file
View File

@@ -0,0 +1,11 @@
#!/bin/bash
# This script serves as a wrapper around bazel to limit the scope of environment variables that
# may change the action output. Using this script should result in a higher cache hit ratio for
# cached actions with a more heremtic build.
env -i \
PATH=/usr/bin:/bin \
HOME="$HOME" \
GOOGLE_APPLICATION_CREDENTIALS="$GOOGLE_APPLICATION_CREDENTIALS" \
bazel "$@"

View File

@@ -1,106 +1,19 @@
load("@io_bazel_rules_go//go:def.bzl", "go_binary", "go_library", "go_test")
load("@io_bazel_rules_docker//go:image.bzl", "go_image")
load("@io_bazel_rules_docker//container:container.bzl", "container_bundle")
load("//tools:binary_targets.bzl", "binary_targets")
load("@io_bazel_rules_docker//contrib:push-all.bzl", "docker_push")
load("//tools:target_migration.bzl", "moved_targets")
go_library(
name = "go_default_library",
srcs = [
"main.go",
"usage.go",
],
importpath = "github.com/prysmaticlabs/prysm/beacon-chain",
visibility = ["//beacon-chain:__subpackages__"],
deps = [
"//beacon-chain/flags:go_default_library",
"//beacon-chain/node:go_default_library",
"//shared/cmd:go_default_library",
"//shared/debug:go_default_library",
"//shared/featureconfig:go_default_library",
"//shared/logutil:go_default_library",
"//shared/version:go_default_library",
"@com_github_ipfs_go_log//:go_default_library",
"@com_github_joonix_log//:go_default_library",
"@com_github_sirupsen_logrus//:go_default_library",
"@com_github_urfave_cli//:go_default_library",
"@com_github_whyrusleeping_go_logging//:go_default_library",
"@com_github_x_cray_logrus_prefixed_formatter//:go_default_library",
"@org_uber_go_automaxprocs//:go_default_library",
moved_targets(
[
":push_images_debug",
":push_images_alpine",
":push_images",
":image_bundle_debug",
":image_debug",
":image_bundle_alpine",
":image_bundle",
":image_with_creation_time",
":image_alpine",
":image",
":go_default_test",
":beacon-chain",
],
"//cmd/beacon-chain",
)
go_image(
name = "image",
srcs = [
"main.go",
"usage.go",
],
goarch = "amd64",
goos = "linux",
importpath = "github.com/prysmaticlabs/prysm/beacon-chain",
race = "off",
static = "off", # Static enabled binary seems to cause issues with DNS lookup with cgo.
tags = ["manual"],
visibility = ["//visibility:private"],
deps = [
"//beacon-chain/flags:go_default_library",
"//beacon-chain/node:go_default_library",
"//shared/cmd:go_default_library",
"//shared/debug:go_default_library",
"//shared/featureconfig:go_default_library",
"//shared/logutil:go_default_library",
"//shared/version:go_default_library",
"@com_github_ipfs_go_log//:go_default_library",
"@com_github_joonix_log//:go_default_library",
"@com_github_sirupsen_logrus//:go_default_library",
"@com_github_urfave_cli//:go_default_library",
"@com_github_whyrusleeping_go_logging//:go_default_library",
"@com_github_x_cray_logrus_prefixed_formatter//:go_default_library",
"@org_uber_go_automaxprocs//:go_default_library",
],
)
container_bundle(
name = "image_bundle",
images = {
"gcr.io/prysmaticlabs/prysm/beacon-chain:latest": ":image",
"gcr.io/prysmaticlabs/prysm/beacon-chain:{DOCKER_TAG}": ":image",
},
tags = ["manual"],
)
docker_push(
name = "push_images",
bundle = ":image_bundle",
tags = ["manual"],
)
go_binary(
name = "beacon-chain",
embed = [":go_default_library"],
visibility = [
"//beacon-chain:__subpackages__",
"//endtoend:__pkg__",
],
)
go_test(
name = "go_default_test",
size = "small",
srcs = ["usage_test.go"],
embed = [":go_default_library"],
deps = ["@com_github_urfave_cli//:go_default_library"],
)
[go_binary(
name = "beacon-chain-{}-{}".format(
pair[0],
pair[1],
),
embed = [":go_default_library"],
goarch = pair[1],
goos = pair[0],
tags = ["manual"],
visibility = ["//visibility:public"],
) for pair in binary_targets]

View File

@@ -1,10 +1,9 @@
# Prysmatic Labs Beacon Chain Implementation
This is the main project folder for the beacon chain implementation of Ethereum Serenity in Golang by [Prysmatic Labs](https://prysmaticlabs.com). Before you begin, check out our [Contribution Guidelines](https://github.com/prysmaticlabs/prysm/blob/master/CONTRIBUTING.md) and join our active chat room on Discord or Gitter below:
This is the main project folder for the beacon chain implementation of Ethereum written in Go by [Prysmatic Labs](https://prysmaticlabs.com).
[![Discord](https://user-images.githubusercontent.com/7288322/34471967-1df7808a-efbb-11e7-9088-ed0b04151291.png)](https://discord.gg/KSA7rPr)
[![Gitter](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/prysmaticlabs/prysm?badge&utm_medium=badge&utm_campaign=pr-badge)
You can also read our main [README](https://github.com/prysmaticlabs/prysm/blob/master/README.md) and join our active chat room on Discord.
Also, read the latest beacon chain [design spec](https://github.com/ethereum/eth2.0-specs/blob/dev/specs/core/0_beacon-chain.md), this design spec serves as a source of truth for the beacon chain implementation we follow at prysmatic labs.
Check out the [FAQs](https://notes.ethereum.org/9MMuzWeFTTSg-3Tz_YeiBA?view). Refer this page on [why](http://email.mg2.substack.com/c/eJwlj9GOhCAMRb9G3jRQQPGBh5mM8xsbhKrsDGIAM9m_X9xN2qZtbpt7rCm4xvSjj5gLOTOmL-809CMbKXFaOKakIl4DZYr2AGyQIGjHOnWH22OiYnoIxmDijaBhhS6fcy7GvjobA9m0mSXOcnZq5GBqLkilXBZhBsus5ZK89VbKkRt-a-BZI6DzZ7iur1lQ953KJ9bemnxgahuQU9XJu6pFPdu8meT8vragzEjpMCwMGLlgLo6h5z1JumQTu4IJd4v15xqMf_8ZLP_Y1bSLdbnrD-LL71i2Kj7DLxaWWF4)
we are combining sharding and casper together.
[![Discord](https://user-images.githubusercontent.com/7288322/34471967-1df7808a-efbb-11e7-9088-ed0b04151291.png)](https://discord.gg/CTYGPUJ)
Also, read the official beacon chain [specification](https://github.com/ethereum/consensus-specs/blob/master/specs/phase0/beacon-chain.md), this design spec serves as a source of truth for the beacon chain implementation we follow at Prysmatic Labs.

View File

@@ -1,42 +0,0 @@
load("@io_bazel_rules_go//go:def.bzl", "go_library", "go_test")
go_library(
name = "go_default_library",
srcs = ["service.go"],
importpath = "github.com/prysmaticlabs/prysm/beacon-chain/archiver",
visibility = ["//beacon-chain:__subpackages__"],
deps = [
"//beacon-chain/blockchain:go_default_library",
"//beacon-chain/core/epoch:go_default_library",
"//beacon-chain/core/helpers:go_default_library",
"//beacon-chain/core/statefeed:go_default_library",
"//beacon-chain/core/validators:go_default_library",
"//beacon-chain/db:go_default_library",
"//proto/beacon/p2p/v1:go_default_library",
"//proto/eth/v1alpha1:go_default_library",
"//shared/params:go_default_library",
"@com_github_pkg_errors//:go_default_library",
"@com_github_sirupsen_logrus//:go_default_library",
],
)
go_test(
name = "go_default_test",
srcs = ["service_test.go"],
embed = [":go_default_library"],
deps = [
"//beacon-chain/blockchain/testing:go_default_library",
"//beacon-chain/core/helpers:go_default_library",
"//beacon-chain/core/statefeed:go_default_library",
"//beacon-chain/db:go_default_library",
"//beacon-chain/db/testing:go_default_library",
"//proto/beacon/p2p/v1:go_default_library",
"//proto/eth/v1alpha1:go_default_library",
"//shared/params:go_default_library",
"//shared/testutil:go_default_library",
"@com_github_gogo_protobuf//proto:go_default_library",
"@com_github_prysmaticlabs_go_bitfield//:go_default_library",
"@com_github_sirupsen_logrus//:go_default_library",
"@com_github_sirupsen_logrus//hooks/test:go_default_library",
],
)

Some files were not shown because too many files have changed in this diff Show More