Compare commits

...

835 Commits

Author SHA1 Message Date
terence tsao
9abc24243b Verifier and processor 2022-02-13 21:20:37 -08:00
terence tsao
eae50b180c gen pbs 2022-02-10 21:18:12 -08:00
terence tsao
b295514900 proto changes 2022-02-10 20:01:25 -08:00
Potuz
9f419bee7d Remove unnecessary reassignement (#10224) 2022-02-10 23:51:20 -03:00
james-prysm
d04eaf8fa0 Better DB Logging (#10215)
* initial commit

* initial commit

* reverting change

* reverting change

* Update beacon-chain/node/node.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/db/kv/kv.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-10 23:15:01 +00:00
Potuz
82ceb51548 Load Synced Tips when starting from Finalized State (#10196)
* Load Synced Tips when starting from Finalized State

* Terence's review #1

* Fix tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-10 22:18:42 +00:00
Potuz
6a2ef13b87 add metrics for synced tips (#10221)
* add metrics for synced tips

* Update beacon-chain/forkchoice/protoarray/metrics.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-10 18:48:46 +00:00
Potuz
72a2dd004b Fix integer overflow (#10222)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-02-10 15:20:36 -03:00
Raul Jordan
1a0e16a48b Initialize the Engine API Client in the Beacon Chain's Powchain Service (#10217)
* define engine caller

* gaz

* Update cmd/beacon-chain/flags/base.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-10 16:46:31 +00:00
Nishant Das
bff5c1e1a9 Handle Bellatrix Fork Epochs Correctly (#10219)
* handle it properly

* Update beacon-chain/sync/fork_watcher.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/sync/fork_watcher.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-10 16:15:55 +00:00
Nishant Das
3f5ce0cdca Handle a ZeroHash as our Eth1 Genesis Block Root (#10218) 2022-02-10 11:28:57 +00:00
terence tsao
d51b52d432 Add pow block protobuf (#10205)
* Add pow_block protobuf

* Revert hash changes

This reverts commit 4afb280385.

* Revert "Add pow_block protobuf"

This reverts commit 4afb280385.

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-09 22:27:30 +00:00
Raul Jordan
7b5a821c81 Implement Execution Engine Client (#10185)
* add in fixtures

* handle real fixtures

* marshaling issues

* ensure error handling works

* begin http client test

* custom type marshaling

* gaz

* custom type

* marshalers

* builds

* builds

* add to build file

* regen

* only root types

* bytes hash

* regen

* bytes hash

* regen

* regen

* alias magic

* revert proto

* more alias magic

* revert build files

* regen

* define marshal methods

* tests passing

* hexbytes

* rem

* rem

* comment

* gaz

* data

* fix conf

* http tests

* sync

* client tests passing

* deepsource

* gaz

* item

* exec block by hash

* begin tests

* http tests passing

* payload id bytes

* proper json encode decode for payload id

* gaz

* proper marshal

* merge engine client

* use big ints

* passes

* confirmed passing

* add in functional methods

* deepsource

* pb

* Update beacon-chain/powchain/engine-api-client/v1/client_test.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* latest

* tests passing again

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-09 20:54:56 +00:00
Mohamed Zahoor
f89265f2f7 Adding a new method in HeaderFetcher (#10212)
* Is optimisyic for root

* add the function in mock chainservice also

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-09 20:21:15 +00:00
james-prysm
c731b715ed Keymanager-Kind: fix cli kind parsing and allow for backward compatability (#10211)
* initial commit with unit tests

* reverting and adding in comments
2022-02-09 14:40:52 -05:00
Radosław Kapka
2a68c69d8f Beacon Node API Postman collection (#10213)
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-02-09 18:42:23 +00:00
terence tsao
a82fd7bf68 Fix merge transition configs (#10206)
* Add proper merge configs

* Switch to bytes

* Use string

* Update mainnet_config.go

* Rm unused

* Fix test

* Update config_test.go

* Update beacon-chain/blockchain/pow_block.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update pow_block_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-09 18:15:29 +00:00
james-prysm
9c540627ab default allowList for validator (#10210)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-02-09 17:18:49 +00:00
james-prysm
11f50453fc Keymanager api postman test collection (#10209)
* initial commit

* adding keymanager api repo url

* updated postman collection

* removing unused folder

* removing validator build

* Update hack/keymanager-api/README.md

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update hack/keymanager-api/README.md

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update hack/keymanager-api/README.md

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-02-09 16:48:08 +00:00
Potuz
5f8b01ccda Make UpdateSyncedTips safe for context cancellation (#10174)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-09 14:54:27 +00:00
Nishant Das
d800107927 Add Peer Scorer to Dev Mode (#10207) 2022-02-09 13:51:59 +01:00
Nishant Das
d3d5cfca0b Peer Score Evaluator (#9717)
* eval stuff

* change up

* update dep

* go mod tidy

* gaz
2022-02-09 09:53:25 +00:00
Potuz
18ef760ee5 Add DeleteBlock() (#10203)
* Add DeleteBlock()

* Raul review #1

* Update comments

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-02-08 23:26:26 +00:00
james-prysm
14e3e80df3 Keymanager api alignment (#10176)
* keymanager-api

* import updates to standards

* adding in more unit tests and code fixes to improve api experience

* deleting accidently generated files

* deleting accidently generated files

* adding in more test coverage

* fixing linter issue

* removing incorrect unit tests

* improving logic for accounts import

* linter fix

* addressing review comments

* fixing based on comments

* fixing nonzero root

* fixing protos

* regen protos

* adjusting protos again

* updating api specs

* fixing code and unit tests after specs changed

* fixing imports

* adding in required changes for api middleware also adding unit tests to catch changes for protos

* fixing deepsource issues

* fixing linting

* seeing if using pointers helps

* addressing comments

* updating bazel build

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-02-08 13:13:36 -06:00
Raul Jordan
3bea0e7896 ExecutionBlock Protobuf Type for the Merge (#10201)
* exec block

* exec block

* base fee

* regen

* exec block test

* fmt

* bytes for big ints

* engine pb

* passing

* add size

* bytes

* size

* size bytes

* passing
2022-02-08 18:06:02 +00:00
Radosław Kapka
8455656597 Use beacon state interfaces instead of concrete structs (#10195)
* return interface from testing/util

* remove usages of v1

* return interface from InitializeFromProto

* return interface from InitializeFromProto

* fix test

* fix interface visibility

* more fixes

* use InitializeFromProtoUnsafe in testing/util

* return early error from mock

* v2

* fix tests

* remove unnecessary assertion

* use struct in nil state test

* Revert "Auxiliary commit to revert individual files from 6bb528c2c5df2446ad18450009f63f44318d41a9"

This reverts commit 7d70238a301209f6dbfc8ff1d81b16e33b0bd67d.

* use struct in sync committee test

* v3

* use InitializeFromProtoUnsafe in mock

* use version information

* Revert "Auxiliary commit to revert individual files from 6bb528c2c5df2446ad18450009f63f44318d41a9"

This reverts commit 5d5e6f2884d21caec7530c16ad2a0d0d27c44aa1.

* revert changes to ClearPreGenesisData

* fix build error

* remove error from PreGenesisState

* bzl
2022-02-08 09:30:06 +00:00
terence tsao
af8bafd895 Spectest 1.1.9 (#10199) 2022-02-07 20:50:11 -08:00
terence tsao
c538a6c068 Add propose bellatrix block (#10189)
* Add propose bellatrix block

* Update BUILD.bazel
2022-02-08 01:29:11 +00:00
Raul Jordan
d5eb8392b6 Engine API Client Custom Marshaling (#10188)
* custom type marshaling

* gaz

* custom type

* marshalers

* builds

* builds

* add to build file

* regen

* only root types

* bytes hash

* regen

* bytes hash

* regen

* regen

* alias magic

* revert proto

* more alias magic

* revert build files

* regen

* define marshal methods

* tests passing

* hexbytes

* rem

* rem

* comment

* gaz

* data

* builds

* passing
2022-02-07 17:13:01 +00:00
Potuz
4906a0e6de add isOptimisticCandidateBlock and IsExecutionBlock (#10191)
* add isOptimisticCandidateBlock and IsExecutionBlock

* remove check

* fix tests

* Align with spec, reuse helper `ExecutionBlock`

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-02-07 01:26:20 +00:00
terence tsao
168e06e607 Save sync tips to DB (#10171)
* Save sync tips to DB

* Fix build

* Update process_block_test.go

* Copy map

* Revert back to nil

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-06 18:00:47 +00:00
Potuz
cf18d5dd42 Add SafeSlotsToImportOptimistically beacon configuration flag (#10190)
* Add SafeSlotsToImportOptimistically beacon configuration flag

* fix e2e

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-06 03:14:56 +00:00
Potuz
aeb6940935 Prune synced tips (#10143)
* Prune synced tips

* Terence's minor cleanups

* Update optimistic_sync.go

* go fmt

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-02-05 20:24:02 +00:00
Potuz
4020a603b6 Invalid synced tips (#10154)
* add synced_tips structure

* create optimisticStore type

* Add UpdateSyncedTips

* use context

* update comment

* use better context cancellation

* reinsert removed error from different PR

* Minor clean ups

* remove invalid nodes from forkchoice

* Update beacon-chain/forkchoice/protoarray/optimistic_sync.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Add check for tip status

* Terence's minor cleanups

* Potus' feedbacks

* Update optimistic_sync.go

* Add erorr

* Synced

* update weights

* Add context error check

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-03 22:57:15 +00:00
terence tsao
3a5cfab5f2 Add IsOptimistic chain info getter (#10184)
* Add  chain info getter

* Interface

* Test

* Update mock.go

* Go fmt

* Update beacon-chain/blockchain/testing/mock.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-02-03 21:30:07 +00:00
james-prysm
8e5ba13352 Fix Small Bug in Genesis Validators Root Validation Func (#10183)
* initial commit

* switching to table driven test
2022-02-03 20:58:25 +00:00
Nishant Das
e7e2f7850d Cleanup Multiclient E2E (#10180) 2022-02-03 14:14:26 +01:00
terence tsao
c8748260fd Implement is_valid_terminal_pow_block (#10172)
* add validator terminal block difficulties

* Fix test

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-03 05:28:23 +00:00
Nishant Das
a70febbf56 E2E Config Cleanup (#10178) 2022-02-03 04:53:09 +00:00
Raul Jordan
55ce1ccc33 Engine API Client Skeleton and Configuration (#10177) 2022-02-02 19:46:27 -08:00
Preston Van Loon
28e0dc5d09 Tool to print current eth1data votes (#9335)
* Initial progress on eth1data votes reporting tool

* gofmt

* add pyrmont genesis time to help txt

* Fix eth1voting tool

* correct Prater genesis time

* Update so it builds

* Add a README

* gofmt

Co-authored-by: rkapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-02 20:41:14 +00:00
Nishant Das
2b0f74904e Multiclient E2E With Lighthouse (#10020)
* save work

* current progress

* fix it more

* save progress

* fixes so far

* add signature test

* fix up changes so far

* change to latest

* update lighthouse version

* fix build

* fix again

* do one

* clean up

* fix build

* fix it

* fix test

* change tag

* remove e2e

* Update config/params/testnet_e2e_config.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* update

* Update sha

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
2022-02-02 19:13:52 +00:00
Nishant Das
5cf976e492 Merge State Cleanup (#10011)
* state related cleanup

* remove unused imports

* radek's review

* fix for native state

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-02 17:51:24 +00:00
Nishant Das
80aa4f483c Make Opt Max Cover Default (#10109)
* make default

* radek's review

* clean up

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-02 16:26:48 +00:00
terence tsao
fb92d518f9 Remove merge setting cli flags (#10162)
* Rm merge client setting

* Fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-02 15:33:48 +00:00
Nishant Das
1774188a17 Fix Certain Edge Cases With the Analyzer (#10164)
* fix analyzer

* fix other deadlock

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-02-02 14:11:31 +00:00
Radosław Kapka
ea31f096b5 Native beacon state: fieldtrie package (#10146)
* Native beacon state: `fieldtrie` package

* reuse ProofFromMerkleLayers

* more reuse + fixes

* more reuse + fixes

* move validateIndices to helpers

* remove TestAppendBeyondIndicesLimit

* do not duplicate package

* correct errors

* handle32ByteArrays function

* type switch

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-02-02 12:18:06 +00:00
Nishant Das
ea1698c1b3 Make Next Slot Cache Default (#10110)
* deprecate all of it

* handle empty slice

* fix bad bug

* fix it

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-02 11:40:46 +00:00
Nishant Das
0802c8fe66 add new methods in (#10173) 2022-02-02 14:56:07 +08:00
Raul Jordan
4322185d80 Use Execution Payload Defined in Engine V1 Namespace (#10170)
* payload use

* incl

* cloners

* regen pb

* gen ssz

* use proper payload

* broken imports

* build

* state

* builds

* proper engine

* builds

* fmt

* baz

* ssz

* baz

* fmt

* regen ssz

* receipts root

* regen pb

* ssz

* receipts root

* ci lint

* proto

* test pkg

* bazel
2022-02-02 03:30:35 +00:00
Hugo Masclet
cab0a4c9df Fix typo in CONTRIBUTING.md (#10168)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-02-02 00:01:12 +00:00
Michael Neuder
8b2b109939 Rename imported keymanager to local. (#10155)
* imported -> local

* reverting to the state of the PR at eb1e3c3d1

* use changes from develop

* del

* rem patch

* patch

* rename to local

* gazelle

* add back build

* imported rename

* gaz

* local

* merge fix + remove proto changes

* comment revert

* build

* gofmt and one new reference

* gofmt pt2

* Update validator/accounts/wallet_edit.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update validator/rpc/accounts.go

* rename

* gaz

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-02-01 19:54:19 +00:00
james-prysm
59be30e9f4 Web3Signer: metrics, tracing, and code coverage (#10161)
* initial commit for metrics, tracing, and code coverage

* fixing unit test, logs, and naming conventions

* Update validator/keymanager/remote-web3signer/internal/client.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/internal/client.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* adding required dependency

* adding more error annotations

* adding in negative test

* fixing bazel build

* gaz

* Update validator/client/validator_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/validator_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/service.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/internal/client.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/validator_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/validator_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* addressing review comments

* Update validator/keymanager/remote-web3signer/internal/client_test.go

* reverting channel direction

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-02-01 18:28:41 +00:00
Mohamed Zahoor
b27d535ade Storing synced tips in DB (#10136)
* DB support for storing synced tips

* gazel fix

* changed the synced_tips to map

* import formatting

* added function docs

* fix potuz comments

* fix prestons comment

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-02-01 16:17:28 +00:00
Nishant Das
e83c9d5862 Revert "Replace a Few IntFlags with Uint64Flags (#9959)" (#10163)
* Revert "Replace a Few IntFlags with Uint64Flags (#9959)"

This reverts commit 790bf03123.

* fix

* fix
2022-02-01 08:51:17 +00:00
Raul Jordan
eef2122a9e Engine API Protobuf Definitions (#10150)
* define proto types for engine api

* proto lint

* prettify

* build bazel

* generated pb

* txs field

* regen

* ssz gen marshal

* fastssz

* ssz

* fmt

* json marshal unmarshal custom

* exec json tests

* passing tests

* deepsource

* gaz

* gaz

* deps

* ssz max

* regen proto

* regen ssz
2022-02-01 01:47:34 +00:00
Radosław Kapka
6406afc6cf Native beacon state: v3 (#10139)
* Native beacon state: v3

* nogo_config

* remove duplicated bazel rule

* gzl

* fix unnecessary assignment

* review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-02-01 08:32:39 +08:00
terence tsao
8a01d412f5 Implement fork choice spec tests (#10159)
* starting

* Reimplement store

* begin on proposer boost

* implement fork choice proposer boost algorithm

* boosting

* gaz

* add mutexes and previous root

* comment on compute proposer boost

* safe

* rem todo

* reset and add tests

* unit test for proposer boost score

* boost works

* Can process block

* Basic test case passing

* ex ante

* test

* propoer test

* More progresses

* More fixes

* rm unused pieces

* Refactor, add phase 0

* locks

* vanilla ex-ante attack

* test similar to spec test

* works works works

* boost test working for num votes > proposer boost weight

* commentary fixes

* rem unused

* comments

* Proposer boost use store time

* Reset proposer root

* debugging

* passing

* Rm unused visibility imports

* Move update head to better place

* Fix deepsrc complains

* Fix more complains

* Raul's feedback

* Use correct byte lengths

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-01-31 19:03:48 +00:00
james-prysm
aed6e13498 Web3Signer: CLI implementation (#10056)
* initial commit for cli integration of web3signer

* resolving conflicts and execution

* remove aggregation slot from proto

* rem aggregation slot

* define a sync message block root struct

* fix sync message name

* sync message block root struct

* amend where sync committee block root is used

* altered switch statement to return correct json request by type

* fixing fork data import, types, and unit tests

* reverting unwanted changes

* reverting more unwanted changes

* fixing deepsource issues

* fixing formatting

* more fixes for deepsource and code clean up

* only want to fetch once for fetch validating public keys

* adding more comments

* new unit tests for requests and fixing a mapper issue

* Update validator/client/validator.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/accounts/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* adjusting comment

* adjusting comment

* fixing import organization

* including more unit tests

* adding new cli edit

* adding in checks for wallet initialize

* adding web3signer flags to main.go

* some how resolved files did not save correctly

* adding in check to make sure web flag only works with types imported and derived

* Update validator/client/sync_committee.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/aggregate.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/accounts/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/main.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/flags/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/flags/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* reverting changes that accidently got checked in

* reverting

* reverting

* continuing to revert unintenteded changes

* reverting

* removing more unneeded changes

* addressing review comment

* initial refactor

* adding in more clarifying comments

* fixing mock

* resolving desource issues

* addressing gosec scan for helper go file

* addressing gosec

* trying to fix bazel build

* removal of interface to fix build

* fixing maligned struct

* addressing deepsource

* fixing deepsource

* addressing efficiency of type checking

* fixing bazel test failure

* fixing go linter errors

* gaz

* web changes

* add w3signer

* new kind

* proper use

* align

* adding prysm validator flags to help flags list

* addressing root comment

* ci lint

* fixing standardapi tests

* fixing accounts_test after removal of keymanager from rpc server

* fixing more unit tests

* Update cmd/validator/flags/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/flags/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/service.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/service.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* addressing missed err checks

* fixing mock tests

* fixing gofmt

* unskipping minimal e2e test and removing related TODOs

* Update testing/endtoend/components/validator.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update testing/endtoend/components/validator.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* adding some error wrapers to clarify failure point

* fixing bazel build with new error checks

* taking preston's advice to make test fail faster to understand what's going on with the test

* checking if genesis validators root is not zero hash

* adding check for genesis validators root giving zero hash

* fixing missing dependency

* adding check for wallet

* log all

* fixing errors for http responses

* switching marshal to pretty print

* adding pretty sign request test

* fixing base url setting

* adding in check for web3signer and temporary wallet instead of having to open the wallet

* refactoring web3signer to not require wallet

* bazel build fix

* fixing gazelle build

* adding content type of request

* fixing more bazel

* removing unused code

* removing unused comments

* adding skip test back in

* addressing a validation and error message

* fix parse

* body

* fixing logic for datadir

* improving error handling

* show resp

* fix

* sign resp as str

* point of pointer remove

* sign resp

* unmarshal sig resp

* read body as str

* adding more verbose logging

* removing unused result

* fixing unit test

* reconfiguring files to properly nest code and mocks

* fix build issue

* using context when using client function calls

* fixing based on suggestion

* addressing comments

* gaz

* removing defined max timeout

* reverting json print pretty

* Update validator/accounts/wallet_edit.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* removing unneeded code restrictions

* should not introduce new code that may impact existing key manager types

* adjusting comments

* adding in json validation

* running go mod tidy

* some logging

* more logs

* fixing typo

* remove logs

* testing without byte trim

* fixing order or properties

* gaz

* tidy

* reverting some logs

* removing the confusing comments

* Update validator/client/aggregate.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/aggregate.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* addressing pr comments

* editing bytes test

* Run gazelle update-repos

* run gazelle

* improving unit test coverage

* fixing text

* fixing a potential escaped error

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-01-31 10:44:17 -06:00
Potuz
26ec352bdc Update synced tips (#10129)
* add synced_tips structure

* create optimisticStore type

* Add UpdateSyncedTips

* use context

* update comment

* use better context cancellation

* reinsert removed error from different PR

* Minor clean ups

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-01-31 15:05:05 +00:00
Nishant Das
ce712a1a3e Fix P2P Test (#10157)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-31 12:23:25 +00:00
Preston Van Loon
98470fdb13 Update rules_go to v0.29.0 (#10064)
* Update rules_go, delete patch, fix analyzers, remove old deployContract code, and prune old dependency which was broken in rules_go update

* Undo some WORKSPACE diffs

* Gazelle

* remove nofuzz, fix build issue

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-31 07:06:07 +00:00
Preston Van Loon
ef906cf704 Update fuzzbuzz.yaml (#10151)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-29 16:55:31 +00:00
Raul Jordan
65ec437ce5 Fork Choice Proposer Boost Changes (#10083)
* begin on proposer boost

* implement fork choice proposer boost algorithm

* boosting

* gaz

* add mutexes and previous root

* comment on compute proposer boost

* safe

* rem todo

* reset and add tests

* unit test for proposer boost score

* boost works

* ex ante

* test

* propoer test

* vanilla ex-ante attack

* test similar to spec test

* works works works

* boost test working for num votes > proposer boost weight

* commentary fixes

* rem unused

* comments

* boost necessary to sandwich unit test

* sad path

* gaz

* Update beacon-chain/forkchoice/protoarray/proposer_boost.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/forkchoice/protoarray/proposer_boost.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/forkchoice/protoarray/proposer_boost_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/forkchoice/protoarray/proposer_boost_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/forkchoice/protoarray/proposer_boost_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/forkchoice/protoarray/proposer_boost_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/forkchoice/protoarray/proposer_boost_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/forkchoice/protoarray/proposer_boost_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* gaz

* radek comments

* reset boosted root in on new slot

* pass

* broken tests

* cfg test

* modify e2e config

* Revert e2e changes

* Test

* Test again

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-01-29 16:32:01 +00:00
Preston Van Loon
58733a9f40 go 1.18 fuzz tests: initial testing (#10063)
* Add basic fuzz test

* Add a few tests for HTR utils

* gofmt

* Add requirement for fuzz gotag

* Run gaz and fix tags

* Update pubsub_fuzz_test.go

I'll fix this later

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-29 08:01:58 +00:00
terence tsao
762ea6dce1 State balance cache: handle input zeros (#10144)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-28 17:14:13 +00:00
terence tsao
eef1730f9e Bellatrix sync changes (#10097)
* Add bellatrix sync changes

* Fix tests

* Update validate_beacon_blocks.go

* check state and block versions

* Sync with develop

* Update rpc_chunked_response_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-28 16:26:52 +00:00
Potuz
de7f7d783e Remove unused code (#10142)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-01-28 14:52:30 +00:00
Nishant Das
5eafb80091 fix it (#10145) 2022-01-28 12:07:41 +08:00
Potuz
cb0c382bf3 Fix Fork Choice prunning (#10137)
* Fix Fork Choice prunning

* Use copy node. Update comments and var names.

* rm unused err var

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-01-27 19:17:19 +00:00
Radosław Kapka
15485f50c3 Fix native state imports (#10140) 2022-01-27 11:02:16 +00:00
Radosław Kapka
9ee00f09a1 Native beacon state: v2 (#10121)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-01-27 09:42:32 +01:00
Radosław Kapka
72817a6d0e Unify validator status response in RPC (#10128) 2022-01-26 14:23:53 -05:00
Preston Van Loon
943dec525c Refactor mock test helpers (#10133)
* Delete deploy contract tool. Move mock to its own package as testonly with some helper functions

* gofmt contracts/deposit/mock/mock.go

* move stategen mock.go to its on testonly pkg

* move password_reader_mock.go to mock testonly package

* move mock keymanager to its own testonly package

* move attestations mock

* move voluntaryexits mock

* Move slashings mock to mock pkg

* move the slasher mock

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-26 14:48:20 +00:00
Leo Lara
d9799e6b6c #10036 Replace codename Merge with Bellatrix (2nd step) (#10116)
* Rename BeaconBlockBodyMerge to BeaconBlockBodyBellatrix

* Rename SignedBeaconBlockMerge to SignedBeaconBlockBellatrix

* Rename CopyBeaconBlockMerge to CopyBeaconBlockMerge

* Rename NewBeaconBlockMerge to NewBeaconBlockBellatrix

* Rename BeaconBlockMerge to BeaconBlockBellatrix

* Rename some comments and strings in pkg proto: Merge -> Bellatrix

* Rename PbMergeBlock to PbBellatrixBlock

* Many renames of merge -> bellatrix in proto package

* Rename some Merge -> Bellatrix in beacon chain package

* More names

* Fix formating in config/params/config.go

* Rename Merge -> Bellatrix in proto/prysm/storage

* Several renames and corrections Merge -> Bellatrix

Co-authored-by: Potuz <potuz@potuz.net>

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Potuz <potuz@potuz.net>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-01-26 07:24:47 +00:00
Potuz
f0fd29d367 Is optimistic (#10124)
* add synced_tips structure

* create optimisticStore type

* Add ForkChoice.Optimistic()

Adds the logic implementing whether a synced block is optimistic or not.

* Terence's bug catch

* Reinforce warning

* Preston's review

* add comment on optimistic sync
2022-01-26 01:42:18 +00:00
terence tsao
22982e5221 Add on_tick from spec (#10130)
* Add `on_tick` from spec

* Update BUILD.bazel
2022-01-25 22:07:57 +00:00
terence tsao
5a505509c3 Use store in blockchain service (#10125)
* Use store in blockchain service

* handle nil cases
2022-01-25 18:47:39 +00:00
Radosław Kapka
588658a649 Extract conditional slot processing to new function (#10114)
* Extract conditional slot processing to new function

* rename function
2022-01-25 14:52:18 +00:00
terence tsao
ae31eed013 Add blockchain store for checkpoints (#10118)
* Add store with tests

* Rm genesis time

* Previous finalized

* Go fmt

* Update beacon-chain/blockchain/store/type.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-01-24 19:53:05 +00:00
Nishant Das
e26cde5e09 Make Balance Trie Computation Default (#10112)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-24 11:36:10 +00:00
Radosław Kapka
5cc201288d Native beacon state: v1 (#10086) 2022-01-24 11:24:38 +01:00
Nishant Das
b4fa626aff Use Batch Decompression For Signatures (#10105)
* use batch decompress method

* terence's review

* gosimple

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
2022-01-21 22:57:29 +00:00
Nishant Das
498e5efed1 Make Get Block Optimization Default (#10113)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
2022-01-21 22:22:44 +00:00
Nishant Das
f563d00e79 Filter Duplicates Messages For Sync Contributions (#10106)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-21 18:25:59 +00:00
Nishant Das
ff50448ca2 Make Batch Verification Of Gossip Default (#10111)
* make default

* add it back

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-21 17:26:42 +00:00
Preston Van Loon
26120e26cb Resolve issues with G304 (#10115) 2022-01-21 16:52:31 +00:00
terence tsao
8d610319bc Remove commented code (#10104)
* Update service.go

* go fmt

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-20 18:15:05 +00:00
terence tsao
288b38be8e Bellatrix p2p changes (#10072)
* Add `process_execution_payload` to `process_block`

* bring in bellatrix p2p changes

* lint

* fix testcase to use MaxChunkSize as config to change allowed size instead of config

* add left out config params

* Fix TestForkSchedule_CorrectNumberOfForks

* fixed Nishant's comments

* gofmt

* add switch case

Co-authored-by: Mohamed Zahoor <zahoor@zahoor.in>
Co-authored-by: Zahoor Mohamed <zahoor@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-01-20 22:12:15 +08:00
Nishant Das
af06bb9737 Do Not Copy State During Upgrade (#10102)
* no copy

* no copy

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-20 06:54:21 +00:00
Nishant Das
52f1190b17 Cleanup Up Resources Correctly (#10103)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-20 06:21:06 +00:00
Preston Van Loon
7df01feb58 Remove libfuzzer / afl based builds in favor of go 1.18 fuzz testing (#10065)
* Remove libfuzzer / afl based builds in favor of go 1.18 fuzz testing

* rm beacon-fuzz archive

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-20 04:51:02 +00:00
Nishant Das
49c73e65a1 make it better (#10101) 2022-01-20 10:37:27 +08:00
terence tsao
06c3306ac2 Update stategen for Bellatrix (#10089)
* Update stategen to for Bellatrix

* Update BUILD.bazel

* Update replay.go
2022-01-19 22:18:11 +00:00
Preston Van Loon
c3228cd5a7 e2e: Add web3signer component (#10088)
* Add initial web3signer binary

* Add support for web3signer component in e2e

* revert some changes

* Reference https://github.com/ConsenSys/web3signer/issues/485 in commentary

* gofmt

* Add notice about java 11 requirement

* Sec issue, revert to 0750 permissions

* remove unused param

* remove unused import

* use hexutil

* Fix yaml struct tags. See https://github.com/ConsenSys/web3signer/issues/485\#issuecomment-1015994840

* fmt

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-01-19 16:22:23 +00:00
james-prysm
8da8855ad5 Web3Signer: Sign Method Implementation (#10084)
* breaking up changes from cli pr

* reverting some changes, adding in changes from remote-web3signer

* adding raul's change

* adding fork info to signing calls

* fixing imports

* gaz

* fixing gofmt

* removing unneeded comment

* Update validator/client/aggregate.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update validator/client/sync_committee.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* addressing comments

* revert proto changes

* proto changes

* reserve

* reserve

* switching to passing slot from passing fork, using slot to find the fork

* removing unneeded check

* fixing missed unit test

* optional

* optional

* gaz

* improving some definitions with constants

* improving some definitions with constants

* rem opt

* rem

* gaz

* moving mocks to its own folder

* adding in bazel field to fix

* fixing type check error

* fixing build

* fixing strict imports

* fixing dependencies

* changing bazel build

* changing bazel build

* changing bazel build

* removing testing only dependency

* removing dependency on testing util package

* update bazel build

* Update checktags_test.go

* Update active_balance.go

* Update sync_committee_minimal.go

* Update sync_committee_mainnet.go

* Update active_balance_disabled.go

* Update committee.go

* Update committee_disabled.go

* Update sync_committee.pb.gw.go

* Update powchain.pb.gw.go

* Update proposer_indices.go

* Update proposer_indices_disabled.go

* Update sync_committee.go

* Update mainnet.go

* Update p2p_messages.pb.gw.go

* Update finalized_block_root_container.pb.gw.go

* Update beacon_block.pb.gw.go

* Update attestation.pb.gw.go

* Update secret_key_test.go

* Update beacon_state.pb.gw.go

* Update version.pb.gw.go

* Update sync_committee.pb.gw.go

* Update sync_committee_disabled.go

* Update mainnet_test.go

* Update minimal.go

* Update signature_test.go

* Update gocast.go

* Update cgo_symbolizer.go

* Update validator.pb.gw.go

* Update beacon_state.pb.gw.go

* Update signature.go

* Update public_key_test.go

* Update minimal_test.go

* Update checktags_test.go

* Update bls_benchmark_test.go

* Update public_key.go

* Update secret_key.go

* Update aliases.go

* Update init.go

* Update stub.go

* Update journald_linux.go

* Update attestation.pb.gw.go

* Update config_utils_develop.go

* Update stub.go

* Update stub.go

* Update beacon_block.pb.gw.go

* Update validator.pb.gw.go

* Update node.pb.gw.go

* Update config_utils_prod.go

* Update journald.go

* Update beacon_block.pb.gw.go

* Update beacon_chain.pb.gw.go

* Update beacon_state.pb.gw.go

* Update events.pb.gw.go

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* addressing comments from review

* updating length of comment

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update stub.go

revert changes

* Update validator/client/aggregate_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* addressing final comments

* fixing gofmt

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-01-18 14:31:58 -06:00
Mohamed Zahoor
4ca04dcc4b Add storage wrapper for BeaconState (#9963)
* adding proto wrapper for BeaconState

* remove BUILD.bazel file spelling mistake

* address terence review comments

* remove capitalization

Co-authored-by: Nishant Das <nishdas93@gmail.com>

* renamed as storage, use v1alpha1 ojects

* fix lint in the generated file

* store state as ssz bytes

* added a seperate version for managing storage

* goimports generated files to satisfy lint

* changed proto to oneof

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-01-18 15:21:56 +00:00
Radosław Kapka
68ef8ae99a Remove CanonicalStateChan (#10093)
(cherry picked from commit 69a2ffd7c9)
2022-01-18 14:24:47 +00:00
Leo Lara
64c02c405b Move ErrNilValidatorsInState from one in each state version to a common one (#10074)
To increase DRY and enable DRY in tests and other users of the Beacon Chain state package,
an error that was duplicated unnecessarily in each version of the state is moved to the root
Beacon Chain state package.
2022-01-18 08:19:20 +00:00
terence tsao
33d1ae0792 Eth2api: GetBlockSSZ supports bellatrix block (#10077)
* add rpc and tests

* rename
2022-01-17 02:30:09 +00:00
Preston Van Loon
182bd615ac refactoring: Deduplicate code (#10090)
* Deduplicate sync committee indices from state

* Deduplicate code in listblocks RPC endpoints, which only differ in response object

* Deduplicate test code in config/fieldparams

* Delete stale benchmark target. This is already included in go_default_test

* deduplicate test cases in TestIsSlashableValidator_OK and fix blst
BUILD file

* Deduplicate TestStore_IsFinalizedChildBlock

* Revert crypto/bls/blst/BUILD.bazel

* Deduplicate TestStore_SaveBlock_NoDuplicates

* Use a generic wrapper for beacon blocks to streamline test

* Deduplicate TestStore_BlocksCRUD

* Deduplicate TestStore_BlocksHandleZeroCase

* Deduplicate TestStore_BlocksBatchDelete

* deduplicate TestStore_BlocksHandleInvalidEndSlot

* Deduplicate TestStore_BlocksCRUD_NoCache

* Deduplicate common block test setup. Make TestStore_Blocks_FiltersCorrectly support multiple forks in test

* Deduplicate the rest of these tests

* lint

* Deprecation warning

* Add test for WrappedSignedBeaconBlock

* Test error path
2022-01-16 18:44:42 +00:00
terence tsao
3c54eb1cf6 Add propose block bellatrix (#10085)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-01-14 21:24:22 +00:00
Michael Neuder
9166f40a04 Eth1 connections (#10073)
* Adding Eth1 connections data message to service definition

* PoC implementation for ETH1 connections RPC

* naming revisions

* cleanup

* register powchain info fetcher

* regenerate protos

* gazelle Bazel

* fixing comments

Co-authored-by: Michael Neuder <michaelneuder@Michaels-MacBook-Pro.local>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-01-14 11:13:04 -05:00
Preston Van Loon
98990ba280 p2p: Update libp2p and libp2p-pubsub (#10082)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-14 14:06:20 +00:00
Radosław Kapka
9ace518539 Custom types for the native beacon state (#10080)
* custom-types package

* split types into distinct files
2022-01-14 13:25:41 +00:00
Chase Jeter
0ea4b02b8b Added analyzer for detecting recursive/nested mutex read locks. (#10066)
* Added analyzer for detecting recursive/nested mutex read locks.

* Added type assertion checks and removed unused 'iTypes' directory

* Clean up

* Bazel file fixes

* Cleaned up code and added comments.

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-01-14 17:12:21 +08:00
Radosław Kapka
d73794ce72 Make Radek and Nishant code owners of duplicated state code (#10079)
* Make Radek and Nishant code owners of the native state feature

* newline

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-01-13 21:16:42 +00:00
Preston Van Loon
2ad0ad5877 Spectests: Refactor ssz_static tests to deduplicate code and test custom HTR (#10081)
* Copy over refactoring efforts from PR #9402

* Remove unused type

* godoc commentary

* Better example err msg

* Better example err msg
2022-01-13 19:52:39 +00:00
Radosław Kapka
963acefe12 Split state package into state-proto and state-native (#10069)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: kasey <489222+kasey@users.noreply.github.com>
Co-authored-by: Dan Loewenherz <dloewenherz.adm@gmail.com>
Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
Co-authored-by: Fredrik Svantes <fredrik@ethereum.org>
Co-authored-by: Leo Lara <leolara@users.noreply.github.com>
2022-01-13 12:23:53 +01:00
Radosław Kapka
0ee85bcbab Do not reuse endpoint struct between API Middleware requests (#10076) 2022-01-12 18:58:32 +00:00
Preston Van Loon
5fb0bcfacd tools/deployContract: Remove specific k8s code (#10075) 2022-01-12 18:31:38 +00:00
james-prysm
f52a214cce Web3Signer: Object Mapper and Types (#10061)
* initial commit

* fixing some types and renaming file

* initial map function commit and moving to a v1 folder

* adding in remaining mapper functions and comments for types and initial commit on unit tests

* adding readme

* fixing unit tests and errors caught

* fixed a few missed properties and names

* updating error handling to be more descriptive

* Update validator/keymanager/remote-web3signer/client_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/v1/custom_mappers.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/v1/custom_mappers.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* fixing comments

* fixing comments

* adding more error checks

* Update validator/keymanager/remote-web3signer/client.go

* fmt and bazel build

* fixing unit tests

* lint needed

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-01-11 17:15:40 -05:00
terence tsao
88b94eae18 Export reverse byte order function (#10040)
* Reverse byte order

* Update BUILD.bazel

* Go imports

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-11 18:42:03 +00:00
Potuz
3d02addfe4 Update README.md (#10071)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-01-11 16:32:07 +00:00
Raul Jordan
53f7030871 Initialize Merkle Layers and Recompute Dirty Fields in BeaconState Proofs (#10032)
* handle trie recomputes

* recomp

* state trie

* args

* deep source

* amend test

* proofs test more fixes

* tests pass
2022-01-11 17:36:03 +08:00
terence tsao
84335b0084 Bellatrix spec tests (#10014)
* Add merge spectests

* fix build

* gazelle

* operation

* all tests should pass

* go fmt

* fix test

* fix dependency

* rm unused constants

* rename to bellatrix

* Gaz

* Rm unused files

* Use Bellatrix for test names

* Add more spec tests

* Gaz and fix test

* sync with dev

* Preston's feedback

* Go fmt

* sync with dev

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-10 18:27:08 +00:00
Leo Lara
b1c2454658 #10036 Replace codename Merge with Bellatrix (1st step) (#10044)
* Rename BeaconStateMerge to BeaconStateBellatrix

* Rename version.Merge to version.Bellatrix

* Rename ComputeFieldRootsWithHasherMerge to ComputeFieldRootsWithHasherBellatrix

* Rename test names to Bellatrix

* Rename comments and strings to Bellatrix

* Fix formatting in a few files

* Revert wrong renaming in test name

* Revert renaming to Bellatrix in mainnet_config.go

* Revert renaming of db key without migration

* Regenerate from proto changes

* Rename new use of already renamed symbols

* gofmt and goimports after regenerating protofiles

* revert weird imports

Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
2022-01-10 16:47:30 +00:00
Fredrik Svantes
29a8b6c08f Updating from buster to bullseye (#10025)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-01-10 15:48:43 +00:00
Radosław Kapka
3bce9df382 Allow comma-separated event topics (#10052) 2022-01-07 17:17:29 +00:00
terence tsao
8ee3019954 Can upgrade to Bellatrix in process slots (#10062)
* Add `process_execution_payload` to `process_block`

* Add upgrade bellatrix to process slots

* Add upgrade bellatrix to process slots

* gazelle
2022-01-07 16:47:33 +00:00
terence tsao
ef1227b14c Add process_execution_payload to process_block (#10060) 2022-01-06 19:36:39 +00:00
Preston Van Loon
80b7f691ae p2p: no alloc extract of gossip digest (#10055)
* Extract gossip topic without a full copy via strings.Split

* Add basic benchmark

* gofmt

* Add empty topic test case

* Remove nl

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-01-06 18:17:51 +00:00
terence tsao
c69bce5d84 Use fieldparams for BLS public key (#10042)
* Use fieldparams for pubkey length

* Fix validator tests

* fix more tests

* fix mock validator

* Fix typo

* bunch of typos

* Update bytes.go

* Update BUILD.bazel

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-06 17:33:08 +00:00
Fredrik Svantes
ad06230291 Updating eip3076_spec_tests and eth_networks URLs (#10057)
From old eth2-clients to new eth-clients

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-01-06 16:58:37 +00:00
terence tsao
a1a301ad5d process payload (#10054) 2022-01-06 08:22:38 -08:00
james-prysm
d28ae62d02 Web3signer keymanager: key manager implementation (#10016)
* initial commit for work on web3signer-keymanager

* WIP key manager client

* adding in more valid unit tests

* refactoring some of my code to work better with current implementations

* adding comments, some error situations and fixing unit tests

* updating go.mod

* gaz

* addressing sandbox errors on test

* fixing more sand box debug items

* missed yet another error case on my test

* reverting go.mod

* addressing comments

* missed removal of unit test

* removing some deepsource issues

* adjusting based on review comments

* addressing comments

* adding the missed types

* changing public keys to only pull once

* fixing commented code format

* gaz

* Update validator/keymanager/remote-web3signer/keymanager.go

updating comments

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/client.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* fixing conflicts

* addressing comments

* fixing deepsource issues

* more deep source issue fix

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-01-05 14:32:46 -06:00
terence tsao
d6a02833dc Add HTR for txs (#10030)
* Add htr for txs

* gaz

* minimal

* test pack
2022-01-05 18:36:39 +00:00
terence tsao
fade8aa7a9 Add validate payload helpers (#10028)
* Add merge complete helper

* Update BUILD.bazel

* minimal

* Add validate payload helpers

* gaz

* update tests

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-01-05 15:28:58 +00:00
terence tsao
8604832499 Use prysm require test lib (#10039)
* Use prysm require test lib

* Fix typo

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-05 07:06:51 +00:00
terence tsao
ceabee0198 Update Transactions and ExtraData non empty check (#10049) 2022-01-03 23:53:39 +01:00
terence tsao
ebedf481f8 Update bazel version to 4.2.2 (#10045) 2021-12-27 23:51:23 +00:00
terence tsao
9f27af43cb Spectest 1.1.8 (#10041)
* Update spec test to v1.1.8

* Update names

* Update network repo

* Update WORKSPACE

* Better names

* Update beacon-chain/core/time/slot_epoch.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-12-26 23:18:28 +00:00
terence tsao
09690590f0 Update spec test to v1.1.7 (#10033) 2021-12-23 21:33:41 +00:00
terence tsao
e3c8491177 Add is_execution_enabled helper (#10027) 2021-12-22 15:03:54 -08:00
terence tsao
bfc90e9abb Add is_merge_block helper (#10026)
* Add merge complete helper

* Update BUILD.bazel

* minimal

* Add is_merge_block helper

* rename to `IsMergeBlock`
2021-12-22 15:52:50 +00:00
Raul Jordan
d173a6e695 Merkle Proof Support for BeaconState Finalized Root and Sync Committees (#10029) 2021-12-19 17:02:42 -08:00
Potuz
7f857ae23a Fix sync commitee endpoint (#10015) 2021-12-18 17:04:30 +01:00
terence tsao
23e39d3d64 Fix state alias v3 (#10022) 2021-12-16 22:44:43 +00:00
terence tsao
ec65c442c4 Minor cleanups on monitor service (#10013)
* Minor cleanups on monitor service

* Update tests

* Update process_block_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-15 20:42:20 +00:00
terence tsao
b38b0186b8 Use signature length from fieldparams (#10019)
* Use bls sig length from fieldparams

* fmt

* fix tests

* fix tests

* fix tests

* Update tags_test.go

* fix tests

* Update BUILD.bazel
2021-12-15 15:14:30 -05:00
terence tsao
4bf935928b Use field params root length (#10012)
* Use config length 32

* Add root length to minimal

* Fix build

* Gaz
2021-12-14 18:42:05 +00:00
terence tsao
acc4ee8d9d Add beacon state upgrade to merge version helper function (#9981) 2021-12-14 01:18:27 -08:00
terence tsao
7b7ed87ad8 Add more merge processing (non substrantive) (#10000)
* Add more merge processing

* Update tests

* raul's feedback

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-12-13 18:04:37 +00:00
Nishant Das
5ab88da183 Use Static Sizes for State Fields (#10007) 2021-12-13 10:16:23 +01:00
Raul Jordan
d6338f6042 Update Web UI Version to v1.0.2 (#10009)
* latest web ui

* fmt

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-12-11 11:36:00 -06:00
Antonio Sanso
c84c6ab547 Add Missing spec tests in sss_static.go (#10003)
* Update ssz_static.go

* Update ssz_static.go

* Update testing/spectest/shared/phase0/ssz_static/ssz_static.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Add Missing spec tests in sss_static.go

* Update testing/spectest/shared/altair/ssz_static/ssz_static.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-12-10 15:18:57 -06:00
Radosław Kapka
52d8a1646f Standardize config overriding in tests across repo (#9998)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-10 04:18:47 +00:00
kasey
3c61cc7d8a allow checkpoint or genesis origin; refactoring (#9976)
* allow checkpoint or genesis origin; refactoring

some quick readability improvements and simplifying the logic enforcing
the startup ordering of the attestation processing routine

* address PR feedback

* gofmt

* Update beacon-chain/blockchain/receive_attestation.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Apply suggestions from code review

use log.WithError for aggregation friendliness

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

Co-authored-by: kasey <kasey@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-09 22:23:00 +00:00
Raul Jordan
37ca409cc1 Add More Fields to Remote Signer Sign Request (#10004)
* add in new fields

* sign request added

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-09 21:53:54 +00:00
Raul Jordan
b381ad49b5 Add Justifications for Gosec Ignored (#10005)
* pin gosec

* edit

* go back to master

* justifications

* Update crypto/bls/blst/signature.go

* proper format

* gosec
2021-12-09 19:40:48 +00:00
james-prysm
00c3a7dcaf Web3signer: Add an HTTP Client Wrapper to Interact with Remote Server (#9991)
* initial commit for web3signer code work in progress

* adding more functions for web3signer

* more improvements to unit tests and web3signer functions

* fixing unit test

* removing path construction

* fixing failing unit test

* adding more happy path unit tests

* fixing unit tests

* temp removing keymanagerfiles being wip

* removing some comments

* Update validator/keymanager/remote-web3signer/client.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/client.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* making errors lowercase

* addressing review comments

* missed resolving a conflict

* addressing deepsource issues

* bazel test and gazelle

* no lint

* deadcode

* addressing comments

* fixing comments

* small fix for readability

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-12-09 09:33:53 -06:00
james-prysm
d66edc9670 fix cors middleware (#9999)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-12-08 17:13:28 -05:00
Raul Jordan
4b249607da Return ERROR Status in ImportKeystores standard API endpoint if slashing protection fails (#9995)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-08 19:52:05 +00:00
Radosław Kapka
02483ba89c Add missing config cleanup in tests (#9996)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-08 19:24:16 +00:00
terence tsao
9629c354f1 Use merge configs for processing (#9982) 2021-12-08 11:02:53 -08:00
Raul Jordan
886332c8fa Add Safe Sub64 Method (#9993)
* add sub 64

* add sub

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-08 09:44:54 +00:00
Raul Jordan
424c8f6b46 API Middleware for Keymanager Standard API Endpoints (#9936)
* begin the middleware approach

* attempt middleware

* middleware works in tandem with web ui

* handle delete as well

* delete request

* DELETE working

* tool to perform imports

* functioning

* commentary

* build

* gaz

* smol test

* enable keymanager api use protonames

* edit

* one rule

* rem gw

* Fix custom compiler

(cherry picked from commit 3b1f65919e04ddf7e07c8f60cba1be883a736476)

* gen proto

* imports

* Update validator/node/node.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* remaining comments

* update item

* rpc

* add

* run gateway

* simplify

* rem flag

* deep source

Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-07 20:26:21 +00:00
terence tsao
cee3b626f3 make db merge compatible (#9987)
* make db merge compatible

* util merge.go per @rauljordan

* Go fmt

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-07 18:51:05 +00:00
Radosław Kapka
5569a68452 Code cleanup (#9992)
* Value assigned to a variable is never read before being overwritten

* The result of append is not used anywhere

* Suspicious assignment of range-loop vars detected

* Unused method receiver detected

* Revert "Auxiliary commit to revert individual files from 54edcb445484a2e5d79612e19af8e949b8861253"

This reverts commit bbd1e1beabf7b0c5cfc4f514dcc820062ad6c063.

* Method modifies receiver

* Fix test

* Duplicate imports detected

* Incorrectly formatted error string

* Types of function parameters can be combined

* One more "Unused method receiver detected"

* Unused parameter detected in function
2021-12-07 17:52:39 +00:00
Raul Jordan
1eff00fb33 Display Num Pruned Items in Slasher Only When Actually Prunes (#9989)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-07 14:38:19 +00:00
terence tsao
2bcda2e021 Add more merge related protobufs (#9986)
* Add merge protos

* Update proto/eth/v2/beacon_block.proto

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-12-07 08:08:14 +00:00
Raul Jordan
98fea2e94d Slasher Significant Optimizations (#9833)
* optimizations to slasher runtime

* remove unnecessary code

* test for epoch update

* commentary

* Gaz

* fmt

* amend test

* better logging

* better logs

* log

* div 0

* more logging

* no log

* use map instead

* passing

* comments

* passing

* for select loop wait for init

* sub

* srv

* debug

* fix panic

* gaz

* builds

* sim gen

* ineff

* commentary

* data

* log

* base

* try

* rem logs

* sim logs

* fix wait for sync event

* ev

* init

* init

* Update beacon-chain/slasher/service.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* comments

* elapsed

* Update testing/slasher/simulator/simulator.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* timeout

* inner cancel

* ctx err everywhere

* Add context aware to several potentially long running db operations

* Fix missing param after updating with develop

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-12-06 21:45:38 +00:00
Radosław Kapka
e53be1acbe Do not rely on peer order in TestDebugServer_ListPeers (#9988)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-12-06 19:29:48 +00:00
Nishant Das
c9f5f8dad5 Use Cached Finalized State When Pruning Deposits (#9985)
* use cached finalized state

* Minor grammar edits

* fmt

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-12-06 12:38:23 +00:00
terence tsao
03bf463d0f Remove unused validator proto imports (#9983) 2021-12-04 21:05:54 +01:00
terence tsao
afb49aaeb9 Add and use merge penalty params (#9878)
* Add and use merge penalty params

* add attester slashing test

* Update config_test.go

* Use DeterministicGenesisStateMerge
2021-12-03 22:00:34 +00:00
Potuz
1fd65ad7fd typos (#9979) 2021-12-03 20:18:43 +00:00
Potuz
b1df3b55b0 more typos (#9980) 2021-12-03 16:31:30 -03:00
Preston Van Loon
3767574c77 stateutil: Reduce allocations by specifying slice capacity eager (#9977)
* Reduce allocations by specifying slice capacity eager

* gofmt
2021-12-03 09:06:05 +00:00
Raul Jordan
d3c97da4e1 Ensure Slashing Protection Exports and Keymanager API Work According to Spec (#9938)
* password compliance

* delete keys tests

* changes to slashing protection exports

* export tests pass

* fix up failures

* gaz

* table driven tests for delete keystores

* comment

* rem deletion logic

* look ma, no db

* fix up tests

* ineff

* gaz

* broken test fix

* Update validator/keymanager/imported/delete.go

* rem
2021-12-02 09:58:49 -05:00
Raul Jordan
1d216a8737 Filter Errored Keys from Returned Slashing Protection History in Standard API (#9968)
* add err condition

* naming
2021-12-02 03:32:34 +00:00
Raul Jordan
790bf03123 Replace a Few IntFlags with Uint64Flags (#9959)
* use uints instead of ints

* fix method

* fix

* fix

* builds

* deepsource

* deep source
2021-12-01 23:34:53 +00:00
Preston Van Loon
ab60b1c7b2 Update go-ethereum to v1.10.13 (#9967)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-01 19:31:21 +00:00
Nishant Das
236a5c4167 Cleanup From Deepsource (#9961)
* ds cleanup

* fix

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-01 18:56:07 +00:00
Nishant Das
5e2229ce9d Update Libp2p to v0.15.1 (#9960)
* fix deps

* tidy it all

* fix build

* remove tls patch

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-01 18:09:34 +00:00
Potuz
6ffba5c769 Add v1alpha1_to_v2.go (#9966)
* Add v1alpha1_to_v2.go

* add tests

* gazelle

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-01 17:39:43 +00:00
Potuz
3e61763bd7 fix operation precedence (#9965) 2021-12-01 17:14:08 +00:00
terence tsao
23bdce2354 Fix grpc client connected... logging (#9956)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-12-01 14:45:39 +00:00
Nishant Das
d94bf32dcf Faster Doppelganger Check (#9964)
* faster check

* potuz's review

* potuz's review
2021-12-01 12:37:10 +00:00
Nishant Das
7cbef104b0 Remove Balances Timeout (#9957)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-12-01 04:03:26 +00:00
Potuz
cd6d0d9cf1 Monitor aggregated logs (#9943) 2021-12-01 03:35:55 +00:00
Potuz
afbe02697d Monitor service (#9933)
* Add a service for the monitor

* Do not block service start

* gaz

* move channel subscription outide go routine

* add service start test

* fix panic on node tests

* Radek's first pass

* Radek's take 2

* uncap error messages

* revert reversal

* Terence take 1

* gaz

* Missing locks found by Terence

* Track via bool not empty interface

* Add tests for every function

* fix allocation of slice

* Minor cleanups

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-11-30 19:27:03 -03:00
terence tsao
2c921ec628 Update spec tests to v1.1.6 (#9955)
* Update spec test to v1.1.6

* Update spec test to v1.1.6
2021-11-30 21:21:59 +00:00
terence tsao
0e72938914 Uncap error messages (#9952) 2021-11-30 07:41:07 -08:00
Potuz
71d55d1cff Check for syncstatus before performing a voluntary exit (#9951)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-11-30 10:40:59 +00:00
terence tsao
d8aa0f8827 Alter config filed name to devnet if it's not populated in file (#9949) 2021-11-29 19:27:26 -08:00
Nishant Das
37bc407b56 Refactor States To Allow for Single Cached Hasher (#9922)
* initial changes

* gaz

* unexport and add in godoc

* nocache

* fix edge case

* fix bad implementation

* fix build file

* add it in

* terence's review

* gaz

* fix build

* Apply suggestions from code review

remove assigned ctx

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-11-29 16:30:17 +00:00
Potuz
5983d0a397 Allow requests for next sync committee (#9945)
* Allow requests for next sync committee

* fix deepsource and variable rename

* Minor cleanup

* Potuz's comments

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-11-28 17:34:24 +00:00
terence tsao
85faecf2ca Add test utility merge state (#9944)
* Add test utility merge state

* gaz

* gaz
2021-11-26 15:53:25 +00:00
terence tsao
f42227aa04 Rest of the merge state implementation (#9939)
* Add rest of the state implementations

* Update BUILD.bazel

* Update state_trie_test.go

* fix test

* fix test

* Update beacon-chain/state/v3/state_trie.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

* Update beacon-chain/state/v3/state_trie.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

* add ctx

* go fmt

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2021-11-25 17:41:05 -03:00
terence tsao
c9d5b4ba0e Add merge beacon block wrappers (#9906) 2021-11-24 14:26:17 -08:00
Potuz
fed004686b Add verbosity to aggregation logs (#9937) 2021-11-24 11:35:45 -08:00
Raul Jordan
4ae7513835 Import Keystores Standard API Implementation (#9924)
* begin

* rem deleted code

* delete keystores all tests

* surface errors to user

* add in changes

* del

* tests

* slice

* begin import process

* add import keystores logic

* unit tests for import

* tests for all import keystores keymanager issues

* change proto

* pbs

* renaming works

* use proper request

* pb

* comment

* gaz

* fix up cli cmd

* test

* add gw

* precond

* tests

* radek comments
2021-11-24 10:40:49 -05:00
Nishant Das
1d53fd2fd3 revert change (#9931) 2021-11-24 07:09:15 -08:00
Potuz
a2c1185032 Monitor sync committee (#9923)
* Add sync committeee contributions to monitor

* gaz

* Raul's review

* Added lock around TrackedValidators

* add comment to trackedIndex

* add missing locks because of trackedIndex

* Terence fixes 2

* moved TrackedValidator to service from config

* Terence comment fix

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-11-24 09:56:34 +08:00
terence tsao
448d62d6e3 Add merge beacon chain objects and generate ssz.go (#9929) 2021-11-23 23:34:31 +00:00
terence tsao
4858de7875 Use prysmaticlabs/fastssz (#9928)
* Use prysmaticlabs/fastssz

* Generated code
2021-11-23 21:28:24 +00:00
terence tsao
cd1e3f2b3e Rename coinbase to fee recipient (#9918)
* Rename coinbase to fee recipient

* Fix imports

* Update field name

* Fee receipient

* Fix goimports

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-23 17:49:06 +00:00
Nishant Das
6f20d17d15 Rename To Signature Batch (#9926)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-11-23 17:57:06 +01:00
terence tsao
94fd99f5cd Add getters and setters for beacon state v3 (part 2) (#9916) 2021-11-22 15:56:23 -08:00
Potuz
d4a420ddfd Monitor blocks (#9910)
* Add proposer logging to validator monitor

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-11-22 18:20:31 -03:00
terence tsao
838b19e985 Add getters and setters for beacon state v3 (part 1) (#9915) 2021-11-22 09:37:55 -08:00
Potuz
788338a004 Stop packing deposits early if we reach max allowed (#9806)
* Stop packing deposits early if we reach max allowed

* Add logs to proposals without deposits

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer_deposits.go

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer_deposits.go

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer_deposits.go

* reinsert debug log

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-11-20 14:14:07 +00:00
kasey
39c33b82ad Switch to lazy state balance cache (#9822)
* quick lazy balance cache proof of concept

* WIP refactoring to use lazy cache

* updating tests to use functional opts

* updating the rest of the tests, all passing

* use mock stategen where possible

reduces the number of test cases that require db setup

* rename test opt method for clear link

* Update beacon-chain/blockchain/process_block.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* test assumption that zerohash is in db

* remove unused MockDB (mocking stategen instead)

* fix cache bug, switch to sync.Mutex

* improve test coverage for the state cache

* uncomment failing genesis test for discussion

* gofmt

* remove unused Service struct member

* cleanup unused func input

* combining type declaration in signature

* don't export the state cache constructor

* work around blockchain deps w/ new file

service_test brings in a ton of dependencies that make bazel rules
for blockchain complex, so just sticking these mocks in their own
file simplifies things.

* gofmt

* remove intentionally failing test

this test established that the zero root can't be used to look up the
state, resulting in a change in another PR to update stategen to use the
GenesisState db method instead when the zero root is detected.

* fixed error introduced by develop refresh

* fix import ordering

* appease deepsource

* remove unused function

* godoc comments on new requires/assert

* defensive constructor per terence's PR comment

* more differentiated balance cache metric names

Co-authored-by: kasey <kasey@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-19 15:59:26 +00:00
Potuz
905e0f4c1c Monitor metrics (#9921)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-11-19 15:34:28 +00:00
Nishant Das
0ade1f121d Add Balance Field Trie (#9793)
* save stuff

* fix in v1

* clean up more

* fix bugs

* add comments and clean up

* add flag + test

* add tests

* fmt

* radek's review

* gaz

* kasey's review

* gaz and new conditional

* improve naming
2021-11-19 20:01:15 +08:00
Raul Jordan
ee52f8dff3 Implement Validator Standard Key Manager API Delete Keystores (#9886)
* begin

* implement delete and filter export history

* rem deleted code

* delete keystores all tests

* gaz

* test

* double import fix

* test

* surface errors to user

* add in changes

* edit proto

* edit

* del

* tests

* gaz

* slice

* duplicate key found in request
2021-11-19 04:11:54 +00:00
Potuz
50159c2e48 Monitor attestations (#9901)
Log attestation performance on the validator monitor
2021-11-18 22:14:56 -03:00
Preston Van Loon
cae58bbbd8 Unskip v2 end to end check for prior release (#9920) 2021-11-18 23:17:19 +00:00
Raul Jordan
9b37418761 Warn Users In Case Slashing Protection Exports are Empty (#9919)
* export text

* Update cmd/validator/slashing-protection/export.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2021-11-18 20:49:19 +00:00
Raul Jordan
a78cdf86cc Warn Users if Slashing Protection History is Empty (#9909)
* warning in case not found

* better comment

* fatal
2021-11-17 15:39:43 +00:00
Nishant Das
1c4ea75a18 Prevent Reprocessing of a Block From Our Pending Queue (#9904)
* fix bugs

* test

* raul's review
2021-11-17 10:05:50 -05:00
terence tsao
6f4c80531c Add field roots for beacon state v3 (#9914)
* Add field roots for beacon state

* Update BUILD.bazel

* Adding an exception for state v3

* fix deadcode

Co-authored-by: nisdas <nishdas93@gmail.com>
2021-11-17 09:04:49 +00:00
terence tsao
e3246922eb Add merge state type definitions (#9908) 2021-11-16 08:36:13 -08:00
Nishant Das
5962363847 Add in Deposit Map To Cache (#9885)
* add in map + tests

* tes

* raul's review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-11-16 02:07:42 +00:00
Potuz
5e8cf9cd28 Monitor exits (#9899)
* Validator monitor process slashings

* Preston's comment

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* add test for tracked index

* Prestons requested changes

* Process voluntary exits in validator monitor

* Address Reviewers' comments

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-11-15 19:45:18 +00:00
Raul Jordan
b5ca09bce6 Add Network Flags to Slashing Protection Export Command (#9907) 2021-11-15 19:21:54 +00:00
terence tsao
720ee3f2a4 Add merge state protobuf (#9888)
* Add beacon state protobuf

* Update proto/prysm/v1alpha1/beacon_state.proto

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update proto/prysm/v1alpha1/beacon_state.proto

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Regenerate pbs

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-15 18:53:43 +00:00
Potuz
3d139d35f6 Validator monitor process slashings (#9898)
* Validator monitor process slashings

* Preston's comment

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* add test for tracked index

* Prestons requested changes

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-11-15 17:39:55 +00:00
terence tsao
ea38969af2 Register start-up state when interop mode (#9900)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-15 16:32:54 +00:00
terence tsao
f753ce81cc Add merge block protobuf (#9887)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-15 15:42:18 +00:00
Nishant Das
9d678b0c47 Clean Up Methods In Prysm (#9903)
* clean up

* go simple
2021-11-15 10:13:52 -05:00
Nishant Das
4a4a7e97df handle canceled contexts (#9893)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-11-13 16:03:27 +08:00
Nishant Das
652b1617ed Use Next Slot Cache In More Places (#9884)
* add changes

* terence's review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-11-12 23:46:06 +00:00
Nishant Das
42edc4f8dd Improve RNG Documentation (#9892)
* add comments

* comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-12 20:30:09 +00:00
terence tsao
70d5bc448f Add cli override setting for merge (#9891) 2021-11-12 12:01:14 -08:00
Preston Van Loon
d1159308c8 Update security.txt (#9896)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-12 15:55:45 +00:00
Nishant Das
e01298bd08 Remove Superflous Errors From Parameter Registration (#9894) 2021-11-12 15:28:21 +00:00
terence tsao
2bcb62db28 Remove unused import (#9890) 2021-11-11 22:54:48 +00:00
terence tsao
672fb72a7f Update spec tests to v1.1.5 (#9875)
* Update spec tests to v1.1.5

* Ignore `TERMINAL_BLOCK_HASH_ACTIVATION_EPOCH`

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-10 02:41:16 +00:00
terence tsao
7fbd5b06da time pkg: can upgrade to merge helper (#9879)
* Helper: can upgrade to merge

* Typo
2021-11-09 18:36:50 +00:00
Raul Jordan
0fb91437fc Group Slashing Protection History Packages Idiomatically (#9873)
* rename

* gaz

* gaz

* Gaz

* rename

* edit

* gaz

* gaz

* build

* fix

* build

* fix up

* fix

* gaz

* cli import export

* gaz

* flag

* rev

* comm

* package renames

* radek
2021-11-09 16:49:28 +00:00
Raul Jordan
6e731bdedd Implement List Keystores for Standard API (#9863)
* start api

* keystores list

* gaz

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-11-08 19:08:17 +00:00
Dan Loewenherz
40eb718ba2 Fix typos: repones -> response, attestion -> attestation (#9868)
* Fix typo: repones -> response

* Fix typo: attestion -> attestation
2021-11-07 11:02:01 -06:00
terence tsao
e1840f7523 Share finalized state at start up (#9843)
* Reuse finalized beacon state at startup

* Better logging for replay

* Update tests

* Fix lint

* Add `WithFinalizedStateAtStartup`

* Update service.go

* Remove unused fields

* Update service_test.go

* Update service_test.go

* Update service_test.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-06 13:45:16 +00:00
terence tsao
b07e1ba7a4 GenesisState when zero hash tests (#9866) 2021-11-06 06:57:54 +01:00
Raul Jordan
233171d17c [Service Config Revamp] - Sync Service With Functional Options (#9859)
* sync config refactor

* rem

* rem

* testing

* gaz

* next

* fuzz

* build

* fuzz

* rev

* log

* cfg
2021-11-05 19:08:58 +00:00
kasey
2b0e132201 fix #9851 using GenesisState when zero hash in stategen StateByRoot/StateByRootInitialSync (#9852)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-11-05 18:14:37 +00:00
Raul Jordan
ad9e5331f5 Update Prysm Web to Use v1.0.1 (#9858)
* update web

* site data update

* fmt
2021-11-04 16:12:32 -04:00
terence tsao
341a2f1ea3 Use math.MaxUint64 (#9857)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-04 18:48:09 +00:00
Raul Jordan
7974fe01cd [Service Revamp] - Powchain Service With Functional Options (#9856)
* begin powchain service refactor

* begin refactor

* powchain passes

* options pkg

* gaz

* rev

* rev

* comments

* move to right place

* bazel powchain

* fix test

* log

* contract addr

* happy path and comments

* gaz

* new service
2021-11-04 14:19:44 -04:00
Raul Jordan
ae56f643eb Rename Web UI Performance Endpoint to Summary (#9855)
* rename endpoint

* rename endpoint
2021-11-04 15:13:16 +00:00
terence tsao
2ea09b621e Simplify should update justified (#9837)
* Simplify should update justified

* Update tests

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-03 22:29:03 +00:00
terence tsao
40fedee137 Use prev epoch source naming correctly (#9840)
* Use prev epoch source correctly

* Update epoch_precompute_test.go

* Update type.go

* Update tests

* Gazelle

* Move version to read only

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-11-03 21:50:41 +00:00
kasey
7cdddcb015 Weak subjectivity verification refactor (#9832)
* weak subjectivity verification refactor

This separates weak subjectivity verification into a
distinct type which does not have a dependency on
blockchain.Service

* remove unused variable

* saving enqueued init blocks before ws verify

* remove TODO, handled in previous commit

* accept suggested comment change

start comment w/ name of function NewWeakSubjectivityVerifier

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* update log w/ Raul's suggested language

* explicit zero value for clarity

* add comments clarifying how we adhere to spec

* more clear TODO per Raul's feedback

* gofmt

Co-authored-by: kasey <kasey@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-11-03 15:07:46 +00:00
Raul Jordan
4440ac199f Empty Genesis Validators Root Check in Slashing Protection Export (#9849)
* test for empty genesis validators root

* precod

* fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-03 14:16:57 +00:00
Radosław Kapka
d78428c49e Return proper responses from KeyManagement service (#9846)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-02 15:46:11 +00:00
Raul Jordan
2e45fada34 Validate Password on RPC CreateWallet Request (#9848)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-02 15:20:43 +00:00
Raul Jordan
4c18d291f4 Rename Interop-Cold-Start Package to Deterministic-Genesis (#9841)
* rename interop-cold-start

* rev

* rev

* BUILD

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-02 14:55:36 +00:00
Preston Van Loon
dfe33b0770 stateutil: Remove duplicated MerkleizeTrieLeaves method (#9847) 2021-11-02 14:31:16 +00:00
Raul Jordan
63308239d9 Define Validator Key Management Standard API Schema (#9817)
* begin service for key management

* begin defining schema

* generate bindings

* rev

* add in custom compiler

* use custom plugin with option

* goimports

* fix up proto to take in multiple passwords

* keymanagent proto edit

* rev

* rev

* dev

* builds

* comment

* indent

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-01 17:26:15 +00:00
terence tsao
712cc18ee0 Use BeaconBlockIsNil helper more (#9834)
* Replace manual checks with helper

* Rename to `BeaconBlockIsNil`

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-01 14:13:05 +00:00
Nishant Das
3d318cffa2 Fix Individual Votes RPC Endpoint (#9831)
* fix

* fix tests

* add terence's review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-11-01 18:36:08 +08:00
terence tsao
026207fc16 Remove de-duplication condition for seen aggregates (#9830)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-10-29 21:53:49 +00:00
Raul Jordan
6b7b30ce47 Use GET Request for Slashing Protection Export (#9838) 2021-10-29 18:23:45 +00:00
Yash Bhutwala
105bb70b5e remove extra condition (#9836) 2021-10-29 09:29:33 -07:00
Potuz
9564ab1f7f Fix validator performance logs (#9828)
* Fix validator performance logs

* Add test

* Clean both phase 0 and altair calculations

* Add back total inclusion distance

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-10-28 16:56:27 +00:00
Raul Jordan
0b09e3e955 Refresh Web Backend JWT Secret Upon File Changes With FSNotify (#9810)
* refresh auth file

* refresh auth token from file changes

* gaz

* test for refresh

* rem token

* secret

* refresh

* remove wallet dir on test end
2021-10-28 10:24:39 -04:00
Radosław Kapka
bb319e02e8 Event support for contribution_and_proof and voluntary_exit (#9779)
* Event support for `contribution_and_prrof`

* event test

* fix panic in tests

* fix

* Revert "Auxiliary commit to revert individual files from dc8d01a15f0056c1fb48733219feab6461f71695"

This reverts commit f5f198564079781f80e1a045cefad7c27f89af25.

* remove receiver

* revive test

* move sending events to sync package

* remove receiver

* remove notification test

* build file

* notifier tests

* revert removal of exit event in API

* simplify exit test

* send notification in contribution API method

* test fix

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-10-28 08:56:22 +00:00
terence tsao
53c86429e4 Add altair test for RPC end point (#9829) 2021-10-27 21:00:09 -04:00
Raul Jordan
61172d5007 Add Missing Objects to Keymanager Protobuf (#9827)
* add missing objects

* update keymanager

* fix up missing sign calls

* gaz

* msg block root

* naming
2021-10-27 18:30:53 +00:00
Yash Bhutwala
1507719613 fix altair individual votes endpoint (#9825) 2021-10-27 15:15:51 +00:00
Raul Jordan
4c677e7b40 Remove JWT Expiration (#9813)
* remove expiry from claims

* fix tests

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-10-26 10:24:09 +00:00
Raul Jordan
28f50862cb Goimport All Items in Proto Folder In Bash Script (#9815)
* update goimports to format all, including gw.pb.go files

* update hack

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-10-26 10:07:49 +00:00
Nishant Das
2dfb0696f7 Fix Validator Exit in our V1 Method (#9819) 2021-10-26 10:14:11 +02:00
terence tsao
ae2c883aaf Generate secret key from big number test only (#9816)
* Generate secret key from big number test only

* Gazelle

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-10-25 19:14:19 +00:00
Raul Jordan
ad9ef9d803 Simplify Prysm Backend Password Requirements (#9814)
* simpler reqs

* gaz

* tidy mod
2021-10-25 13:17:47 -05:00
Kirill Fedoseev
b837f90b35 Fix GetDuties (#9811)
* Fix GetDuties

* Add regression test
2021-10-25 09:09:47 -05:00
Håvard Anda Estensen
7f3ec4221f Add errcheck and gosimple linters (#9729)
* Add errcheck linter

* Check unchecked error

* Add gosimple linter

* Remove type assertion to same type

* Omit nil check

len() for nil slices is defined as zero

* Revert "Remove type assertion to same type"

This reverts commit af69ca1ac8.

* Revert "Revert "Remove type assertion to same type""

This reverts commit 5fe8931504.

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-10-22 17:40:03 -05:00
Radosław Kapka
5b3375638a Improve description of datadir flag (#9809)
* Improve description of `datadir` flag

* improve doc
2021-10-22 12:53:13 +00:00
terence tsao
3c721418db Check participation flag offset length (#9784)
* Check participation flag offset length

* Fix test

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-10-21 15:01:09 +00:00
terence tsao
d7cad27cc3 Pool returns empty contribution slice instead of nil (#9808) 2021-10-20 14:45:57 -07:00
terence tsao
290b4273dd InitializePrecomputeValidators check overflows (#9807) 2021-10-20 13:06:43 -05:00
Raul Jordan
a797a7aaac Refactor Web Authentication to Use Auth Token (#9740)
* builds

* initialize auth token from scratch

* change auth

* add improvements

* moar auth fixes

* web auth changes running

* url encode

* tests

* gaz

* navigate auth

* separate line

* auth token

* 304

* persistent auth tokens and jwts

* fixed up test

* auth token path test and integration test

* auth token test

* auth token command

* gaz

* gaz

* Radek feedback

* fix up

* test

* Update validator/rpc/auth_token.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-10-20 16:37:05 +00:00
Raul Jordan
f7c34b0fd6 Validate Keystores Validator Client RPC Endpoint (#9799)
* validate endpoint

* validate keystores proto

* wallet validate

* validate keystores endpoint

* better err message

* added in gaz
2021-10-20 09:23:59 -05:00
Nishant Das
c6874e33f7 fix transitions (#9804) 2021-10-20 19:36:08 +08:00
Radosław Kapka
13ddc171eb Ignore validators without committee assignment when fetching attester duties (#9780)
* Ignore validators without committee assignment when fetching attester duties

* simplify

* fix tests

* slice capacity

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-10-19 17:41:12 +00:00
Radosław Kapka
acde184aa7 Fill out Version for SSZ block (#9801) 2021-10-19 16:45:38 +00:00
Radosław Kapka
5fd6474e56 Allow submitting sync committee subscriptions for next period (#9798)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-10-19 14:04:04 +00:00
Preston Van Loon
65db331eaf Remove bazel-go-ethereum fork, use upstream go-ethereum (#9725)
* Check in go-ethereum crypto/sepc256k1 package with proper build rules

* gaz

* Add karalabe/usb

* viz improvement

* Remove bazel-go-ethereum, use vendored libraries only

* move vendor stuff to third_party so that go mod wont be mad anymore

* fix geth e2e flags

* fix geth e2e flags

* remove old rules_foreign_cc toolchain

* Update cross compile docker image to support os x

* works for geth build

* remove copy of sepc256k1

* revert changes in tools/cross-toolchain

* gaz

* Update go-ethereum to 1.10.10

* Revert "revert changes in tools/cross-toolchain"

This reverts commit 2e8128f7c3.

* revert tags changes

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-10-19 00:00:22 +00:00
Radosław Kapka
7b8aedbfe4 Update Beacon API to v2.1.0 (#9797)
* `Eth-Consensus-Version` header

* rename unused receiver

* omit receiver name

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-10-18 18:55:18 +00:00
terence tsao
cf956c718d Update to Spectest 1.1.3 (#9786)
* Update transition spec test setup

* Update WORKSPACE

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-10-18 18:13:40 +00:00
Raul Jordan
975f0ea1af [Service Config Revamp] - Blockchain Initialization (#9783)
* viz

* unexport config

* builds

* viz

* viz

* register cfg

* fuzz

* blockchain opts

* deepsource

* rename flag opts
2021-10-18 17:48:05 +00:00
terence tsao
a80b1c252a Refactor rpc proposer into smaller files (#9796)
* Refactor rpc proposer into smaller files

* Update BUILD.bazel

* Fix test

* Update BUILD.bazel
2021-10-18 11:35:32 -05:00
Preston Van Loon
1f51e59bfd Bazel: minimal build config alias (#9794) 2021-10-18 14:22:57 +00:00
Nishant Das
bfcb113f78 Fix Doppelganger Protection (#9748)
* add fix

* fix tests

* Update beacon-chain/rpc/prysm/v1alpha1/validator/status.go

* fix tests

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-10-18 13:09:14 +00:00
terence tsao
20c7efda2c Process slashings return error on unknown state (#9778)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-10-18 12:40:47 +00:00
Radosław Kapka
f2990d8fdd Return errors for unknown block/state versions (#9781)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-10-18 12:18:49 +00:00
Nishant Das
d1f3050d20 Ignore G307 for Gosec (#9790) 2021-10-18 14:04:01 +02:00
Raul Jordan
4dbb5d6974 Add WebUI Security Headers (#9775) 2021-10-15 12:40:23 +02:00
Raul Jordan
59547aea66 Deregister Remote Slashing Protection Until Further Notice (#9774)
* avoid registering

* disable endpoints

* remote slasher protection register

* gaz

* fatal on external protection flag call

* radek comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-10-14 19:33:05 +00:00
Radosław Kapka
545424dd09 Fix epoch calculation when fetching sync committee duties (#9767)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-10-14 18:52:26 +00:00
Nishant Das
508b18f1bd Return Error For Batch Verifier (#9766)
* fix it

* make err check happy

* radek's review

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-10-13 23:52:22 +00:00
Radosław Kapka
e644e6b626 Remove special preparation of graffiti (#9770)
* add attestations to migration tests

* remove sepcial preparation of graffiti

* Revert "add attestations to migration tests"

This reverts commit adbe8cf4bf.
2021-10-13 15:32:15 +00:00
Radosław Kapka
280dc4ecf0 Register v1alpha2 endpoints in the gateway (#9768)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-10-12 15:13:52 +00:00
terence tsao
7a825a79ae E2e test: stricter participation check (#9718)
* Stricter participation check

* 0.99 is still better than 0.95...

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-10-12 11:32:36 +00:00
Nishant Das
b81f5fc7a5 Handle Invalid Number Of Leaves (#9761)
* fix bad trie

* handle edge case

* radek's review

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-10-12 10:36:57 +00:00
Nishant Das
06c084ff52 Log out Gossip Handling Errors (#9765) 2021-10-12 07:44:47 +00:00
Preston Van Loon
76e06438e9 Update bazel version to 4.x (#9763)
* Update to bazel 4.0.0

* bazel 4.2.1

* Regenerate crosstool configs

* restore manual tags

* restore manual tags
2021-10-12 06:17:24 +00:00
Nishant Das
f8f037b63d updateInterval (#9764) 2021-10-11 22:12:07 -05:00
terence tsao
f114a47b5b Improve "synced block.." log (#9760) 2021-10-09 15:27:34 +02:00
Nishant Das
65d2df4609 Add in Stronger Length Checks (#9758)
* add changes

* radek's review

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-10-08 17:41:36 +00:00
Radosław Kapka
63349d863b Do not return data for validators without sync duties (#9756)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-10-08 15:20:40 +00:00
Nishant Das
e2a00d3e2e Update to v1.1.2 version of the Spec (#9755) 2021-10-08 15:08:57 +02:00
terence tsao
271ee2ed32 Cleanup slot helpers into intended locations (#9752)
* Refactor slot time helpers to intended locations

* Gazelle

* Update beacon-chain/core/time/slot_epoch.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/core/time/slot_epoch_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Go fmt

* Update transition_fuzz_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-10-07 19:15:35 +00:00
Radosław Kapka
b5f0bd88b0 Add sync details to error messages (#9750) 2021-10-07 20:50:03 +02:00
terence tsao
e7085897ad Use BLS signature length config value (#9746)
* Use `BeaconConfig().BLSSignatureLength`

* Update BUILD.bazel

* Fix build

* Update BUILD.bazel
2021-10-07 10:37:53 -05:00
Raul Jordan
d9b98e9913 Health Endpoints Not Registered in Validator gRPC Gateway Fix (#9747) 2021-10-06 19:07:29 +00:00
terence tsao
a9f9026c78 Minor cleanups (#9743)
* Minor cleanups

* Delete slasher client files

* Revert "Delete slasher client files"

This reverts commit 0c995a1d4a.

* Update slasher_client.go
2021-10-06 13:23:40 -05:00
Nishant Das
b128d446f2 Bring Eth2FastAggregateVerify Inline With the Spec (#9742)
* fix

* tie it closer to the spec
2021-10-06 10:48:56 +08:00
Radosław Kapka
9aa50352b6 Allow fetching sync committee duties for current and next period's epochs (#9728) 2021-10-05 08:46:16 -07:00
Mohamed Zahoor
362dfa691a GetBlock() - Simple optimisations (#9541)
* GetBlock() optimizations

* addressed preston's comments

* fix fmt

* added benchmarks

* fix deepsource

* merge fix

* remove unnesesary check

* fixed terence's comments

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer.go

Co-authored-by: Nishant Das <nishdas93@gmail.com>

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-10-05 17:28:02 +08:00
Raul Jordan
843ed50e0a Register Slashing Protection Client in Validator (#9735) 2021-10-05 00:41:10 +00:00
Preston Van Loon
da58b4e017 Bump github.com/libp2p/go-tcp-transport to v0.2.8 (#9734) 2021-10-04 23:53:22 +00:00
Raul Jordan
800f78e279 Web UI Fix for Prysm V2.0.0 (#9732) 2021-10-04 22:26:55 +00:00
terence tsao
644038ba61 eth2 api: use balance instead of effective balance (#9722)
* Use balance instead of effective balance

* Update test to reflect validator balance

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-10-04 20:57:40 +00:00
Nishant Das
865ef4e948 Pin Base Images (#9727)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-10-04 18:55:56 +00:00
Håvard Anda Estensen
b793d6258f Add Golangci-lint to GitHub Actions and add Deadcode linter (#9597)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-10-04 13:06:21 +02:00
Raul Jordan
f7845afa57 Use Unique Slot Time Tickers for Slasher (#9723) 2021-10-03 07:49:01 +00:00
terence tsao
c21e43e4c5 Refactor: move functions beacon-chain/core/time -> time/slots (#9719)
* Move necessary functions beacon-chain/core/time -> time/slots

* Fix fuzz

* Fix build

* Update slot_epoch.go
2021-10-01 15:17:57 -05:00
Raul Jordan
4f31ba6489 Changes to E2E for Optimized Slasher (#9698)
* remaining slasher e2e changes

* testing gaz

* slash e2e

* deepsource

* gaz

* viz

* revert wait group changes

* comment

* lock around reset cache

* add slashing reason

* revert changes

* is sync

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
2021-09-30 20:28:14 +00:00
terence tsao
2bc3f4bc6a Update sync time error message (#9713)
* Update sync_committee.go

* Update beacon-chain/core/altair/sync_committee.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Fix slot calculation

* Go fmt

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-30 20:03:57 +00:00
Radosław Kapka
601493098b Return error when request JSON contains unknown fields (#9710)
* Return error when request JSON contains unknown fields

* Update api/gateway/apimiddleware/process_request.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-30 19:42:20 +00:00
terence tsao
8219af46e4 Move slot epoch from core to time pkg (#9714)
* Move slot epoch from core to time pkg

* Fix fuzz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-30 19:00:14 +00:00
Raul Jordan
f5234634d6 Prevent Saving Empty Chunks to Disk (#9707)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-30 18:29:40 +00:00
Raul Jordan
26978fcc50 Make Jaeger in E2E Sharding Aware (#9711)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-30 17:35:05 +00:00
Nishant Das
7c67d381f4 Remove Peer Scorer From Dev (#9712)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-30 16:51:13 +00:00
Nishant Das
a196c78bed clean up (#9709) 2021-09-30 08:10:49 -07:00
Nishant Das
3c052d917f Use Scoring Parameters To Enable Global Score (#8794)
* fix score parameters

* add check

* fix deadlock issues

* fix

* fix again

* gaz

* comment

* fix tests and victor's review

* gaz

* clean up

* fix tests

* fix tests

* gate behind flag

* add scorer to dev

Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-30 03:39:53 +00:00
Raul Jordan
7dae66afc9 Highest Attestations Endpoint for Optimized Slasher (#9706)
* begin highest atts

* highest atts

* deep source
2021-09-29 21:33:28 -05:00
Radosław Kapka
62dc74af2f Move head event's epoch transition to epoch start (#9704)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-09-29 17:18:10 -05:00
Raul Jordan
13cdb83591 Validator Changes for Optimized Slasher (#9705)
* val changes

* mock

* mocks

* gomock any

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-29 21:25:45 +00:00
Preston Van Loon
520bc9d955 Update validator reporting logs and metrics for Altair (#9589)
* Mark fields as deprecated due to Altair

* Only print inclusion distance fields before Altair fork

* Report phase0 and altair metrics respectively

* only set phase0 fields in phase0, only set altair fields in altair

* better use of fields

* Update go pbs

* Update individual votes method

* regen go proto files

* formatting

* Feedback from @potuz

* Annotate metrics per @potuz suggestion

* Set previous release e2e to end 1 epoch before altair. Add some out of bounds checks for validator metrics reporting and a panic catch

* gofmt

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-29 20:49:58 +00:00
Raul Jordan
df33ce3309 Remaining Slasher Beacon Node Changes (#9701)
* slasher beacon node changes

* remaining beacon node items

* moar changes

* gaz

* flag fix

* rem slashable

* builds

* imports

* fix up

* pruning faster test

* deepsource

* fix wrong item

* node node feature flags

* broken test

* preston review

* more preston comments

* comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-29 18:17:37 +00:00
terence tsao
86efa87101 Refactor sync and participation field roots (#9703)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-29 17:46:51 +00:00
Potuz
ff625d55df Use target root for pending attestations in tests instead of unrelated one (#9699)
* Use target root for pending attestations instead of unrelated one

* remove unnecessary block saves

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-09-29 12:27:47 -05:00
Mohamed Zahoor
0678e9f718 finalise deposits before we initialise the node (#9639)
* finalize deposits before we initialize the node

* Update beacon-chain/blockchain/service.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Update beacon-chain/blockchain/service.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* moved this when intiializing caches

* added test case

* satisfy deepsource

* fix gazel

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-09-29 15:43:24 +00:00
Nishant Das
10251c4191 increase size (#9702) 2021-09-29 10:01:10 -05:00
Potuz
861c2f5120 remove unnecessary asserts from tests (#9700) 2021-09-29 22:17:06 +08:00
terence tsao
bfc821d03a Can save justified checkpoint to DB (#9697)
* Can save justified checkpoint to DB

* Update beacon-chain/blockchain/process_block_test.go

Co-authored-by: Nishant Das <nishdas93@gmail.com>

* Update beacon-chain/blockchain/process_block.go

Co-authored-by: Nishant Das <nishdas93@gmail.com>

* Update beacon-chain/blockchain/process_block.go

Co-authored-by: Nishant Das <nishdas93@gmail.com>

* Update beacon-chain/blockchain/process_block.go

Co-authored-by: Nishant Das <nishdas93@gmail.com>

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-09-29 03:00:52 +00:00
Raul Jordan
9edba29f64 Slasher Simulator Code for Testing Optimized Slasher Behavior (#9695)
* slashing simulator

* add in necessary items for slasher sim

* sim item

* fix up

* fixed build

* rev

* slasher sim in testing

* testonly

* gaz

* gaz

* fix viz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
2021-09-29 02:27:21 +00:00
Nishant Das
0edb3b9e65 Flatten Attestation Packing (#9683)
* flatten attestation packing

* dedup again

* terence's review

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-29 01:56:55 +00:00
Preston Van Loon
6c5bf70021 CI: fix remote caching (#9696) 2021-09-29 00:18:31 +00:00
Nishant Das
393549ad19 Add in Balance Safety Check (#9419)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-28 22:19:10 +00:00
terence tsao
8f8ccf11e4 Update head more timely before (#9651)
* Move update head closer to transition

* Update process_block.go

* Update tests

* Move savePostStateInfo back

* Update process_block.go

* Update process_block_helpers.go

* Minor clean up for better diff

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-28 21:51:11 +00:00
terence tsao
806bcf1d29 Clean up unused types & function comments (#9691)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-28 21:21:07 +00:00
Raul Jordan
2a2239d937 Detect Slashable Attestations in Optimized Slasher (#9694)
* detect slashable attestations

* Update beacon-chain/slasher/detect_attestations.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-09-28 20:54:17 +00:00
Preston Van Loon
c94ba40db2 config: ensure config matches altair presets (#9690)
* config: ensure config matches altair presets

* gofmt

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-09-28 20:02:12 +00:00
Raul Jordan
1816906bc7 Detect Slashable Blocks in Optimized Slasher (#9693)
* pass

* Update beacon-chain/slasher/detect_blocks.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Update beacon-chain/slasher/detect_blocks.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-28 19:33:45 +00:00
Radosław Kapka
c32090aae5 Allow sending Altair blocks to /eth/v1/beacon/blocks (#9685)
* Allow sending Altar blocks to `/eth/v1/beacon/blocks`

* tests

* add documentation

* fix ineffectual assignment

* change type of sync committee bits

* remove unused import

* fix Altair epoch calculation

* compare slot against slot

* do not publicly export E2E constant

* tests for setInitialPublishBlockPostRequest

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-28 19:07:32 +00:00
Raul Jordan
57f965df50 Include Slasher Receiving Methods (#9692)
* first

* add receive details

* ensure most builds

* add slasherkv changes

* db iface additions

* build

* gaz

* proper todo comment

* terence comments

* sig check

* bad sig checks

* proper lock issue

* fix test

* fix up tests
2021-09-28 18:13:16 +00:00
Radosław Kapka
8a6e2a5c63 Add MIN_SYNC_COMMITTEE_PARTICIPANTS parameter to config (#9689)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-09-28 16:54:12 +00:00
Raul Jordan
6d79f61fda Process Slashings for Newly Optimized Slasher (#9687)
* add process slashings

* verify sig

* add method

* add test

* add in process slashings functionality

* target state for aggregate

* comment

* Radek comments
2021-09-28 10:27:40 -05:00
Nishant Das
bf2d2cf981 make opt max cover default (#9684) 2021-09-28 17:34:29 +08:00
Raul Jordan
f2840c9ffa Slasher Min/Max Chunk Logic (#9673)
* slasher chunks code

* slasher chunks code

* avoid using shared

* testing helper

* slasher gaz

* radek comments

* preston feedback
2021-09-28 02:04:32 +00:00
Preston Van Loon
cbb4361c64 Update spectests to v1.1.0 (#9680)
* Update spectests to v1.1.0

* ignore placeholder fields, fix spec config check
2021-09-28 00:50:06 +00:00
terence tsao
328e3e6caf Update mainnet_config.go (#9678) 2021-09-27 13:39:55 -07:00
Preston Van Loon
ee0a453b7b core: refactor signing and domain methods from helper to core/signing pkg (#9520)
* Move domain function and all signing root functions from beacon-chain/core/helpers to beacon-chain/core

* @terencechain suggestion to put these methods under core/signing
2021-09-27 16:19:20 +00:00
Radosław Kapka
3e640fe79f Remove unused Eth1Data-related code from the proposer (#9670)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-27 14:29:41 +00:00
Radosław Kapka
bf41fd854d Remove fmt.Println from committee cache (#9677) 2021-09-27 11:18:20 +00:00
Nishant Das
6eb158c16a Check Head State For New Head Methods (#9676)
* check head state

* add tests
2021-09-27 08:27:11 +00:00
terence tsao
376d248c22 Add in progress handler to committee cache (#9664)
* Add in progress handler for committee cache

* Remove debug print

* Update validators.go

* Fix all the tests

* More tests

* Update committee_disabled.go

* Update committee_disabled.go

* Update testing util

* Update main.go

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-09-26 23:27:57 +08:00
Nishant Das
6e4c2b4b20 Wrap Gossip Validators With Error (#9660) 2021-09-25 18:06:48 -07:00
Raul Jordan
75936853af Optimized Slasher Docs and Helpers (#9578)
* bring over helpers

* slasher helpers pass tests

* fix dead link

* rem eth2

* gaz

* params

* gaz

* builds

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-09-24 13:38:13 -05:00
terence tsao
ea9ceeff03 Various clean up before v2 (#9672)
* Update package names

* Various clean up

* Gazelle

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-24 17:42:16 +00:00
Radosław Kapka
78ea402d9d Correct the semantics of startEpoch calculation in registerSyncSubnet (#9671) 2021-09-24 16:02:55 +00:00
Radosław Kapka
fae806c73e Small changes in API Middleware (#9666)
* Small changes in API Middleware's custom hooks

* reorder fields in `Endpoint`

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-24 14:34:19 +00:00
Radosław Kapka
45510fafea Fix error handing for block v2 endpoints (#9667)
* Fix error handing for block v2 endpoints

* rename helper func
2021-09-24 13:52:00 +00:00
Radosław Kapka
12480e12b2 Add flags for disabling selected API (#9606)
* Add flags for disabling selected API

* tests

* build file

* Use comma-separated modules

* test fix

* fix gateway tests

* fix import in flag tests
2021-09-24 09:25:42 +00:00
Radosław Kapka
7dd99de69f Restructure API Middleware (#9663)
* Restructure API Middleware

* fix package name in tests

* build file

* gzl

* fix one more test
2021-09-23 20:41:04 +00:00
Raul Jordan
a9a4bb9163 Move Shared/Testutil into Testing (#9659)
* move testutil

* util pkg

* build

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-23 18:53:46 +00:00
Radosław Kapka
2952bb3570 Increase debug gRPC message size to 128MB (#9661)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-23 18:31:24 +00:00
Radosław Kapka
1fbe05292f Fix SubmitPoolSyncCommitteeSignatures API endpoint (#9646)
* Fix `SubmitPoolSyncCommitteeSignatures` API endpoint

* fix test

* fix another test

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-23 18:01:07 +00:00
Radosław Kapka
36b1f322c0 Various bug fixes in Eth API (#9649)
* fix block structure

* correct status code when block is not found

* make `/internal` work with events and SSZ

* test fix

* better block serialize tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-23 17:34:11 +00:00
Nishant Das
5225d97fd4 Remove Update Timely Feature Config (#9655)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-23 17:04:40 +00:00
Nishant Das
57b7bf6ea0 Update Mainnet Bootnodes (#9656)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-23 16:23:01 +00:00
Raul Jordan
29513c804c Create Encoding Bytesutil (#9658)
* bytesutil

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-23 15:23:37 +00:00
Nishant Das
b9ad81db50 bump it up (#9657) 2021-09-23 09:20:19 -05:00
Nishant Das
d694a775d7 Check For Blocks In Our Initial Sync Cache (#9558)
* add change

* test

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-09-23 05:18:19 +00:00
Nishant Das
4f3762f1f7 Clean Up V2 BeaconState (#9648)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-09-23 01:10:25 +00:00
terence tsao
8678610683 Update package name comments (#9653) 2021-09-22 22:38:06 +00:00
Radosław Kapka
8d4cdde07e Clean up code in Eth API (#9650)
* typo fix

* code cleanup

* fix spans

* move state convertion to migration package

* add migration package to v1 visibility
2021-09-22 17:59:06 +00:00
Raul Jordan
191bce3655 Add True Eth2 Deposit Contract, Bytecode, ABI (#9637)
* solidity contract, abi, and deposit util

* gaz

* gaz

* drain contract remove

* build

* fix up deploy

* add readme

* fix e2e

* revert

* revert flag

* fix

* revert test flag

* fix broken test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-22 17:27:13 +00:00
terence tsao
f52f737d2b Review Altair core epoch files with changes (#9556)
* Clean up core epoch for Altair

* Update BUILD.bazel

* Remove check for spec test

* Update validators_test.go

* Update validators_test.go

* Update beacon-chain/core/altair/epoch_precompute.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/core/altair/epoch_precompute_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/core/altair/epoch_spec.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-09-22 15:07:05 +00:00
Preston Van Loon
2cbe10c701 Fix long running e2e altair exception for previous release (#9645)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-22 04:01:43 +00:00
Preston Van Loon
36a7575437 e2e: fix trace_sync http handler registration (#9644)
* Avoid a global http server so that multiple test runs aren't registering global http handlers and causing a panic

* gofmt
2021-09-22 03:34:09 +00:00
terence tsao
8e3e6d0156 Fix package comment typo (#9641)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-21 20:25:12 +00:00
Raul Jordan
f3d6dbcc1e Move Shared/Params Into Config/Params (#9642)
* config params into pkg

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-21 19:59:25 +00:00
Radosław Kapka
26893bcb8c Production version of Altair Eth APIs (#9640)
* internal prefix in proto services

* half-baked inmplementation

* rename v1 to eth

* use router and remove old flag

* uncomment cors

* update v2 methods in proto services

* move adding path prefix after param processing

* remove unneeded code

* remove flag

* fix e2e

* uncomment sync committee e2e

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-21 19:20:57 +00:00
Raul Jordan
eebcd52ee6 Miscellaneous Packages from Shared Into Proper Folders (#9638)
* slashutil

* builds

* interop

* viz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-21 18:11:16 +00:00
Raul Jordan
45bfd82c88 Add Encoding SSZ Package (#9630)
* ssz package

* compile

* htrutils

* rem pkg doc

* fix cloners_test.go

* fix circular dep/build issues

Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-21 15:02:48 +00:00
Nishant Das
b943f7bce5 Add Flag To Parameterize Number of Peers in A Subnet (#9631)
* add flag

* use old value

* gaz

* raul's review
2021-09-21 07:55:52 +00:00
Radosław Kapka
28364d9f33 E2E for API Middleware (#9635)
* rename file

* Enable SSZ-serialization in v2 endpoints

# Conflicts:
#	proto/eth/v2/beacon_state.pb.go

* e2e

* rename file

* improve logging errors

* Revert "improve logging errors"

This reverts commit 796bcf3e97.

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-21 04:31:41 +00:00
deepsource-autofix[bot]
531f05d30d Unused parameter should be replaced by underscore (#9632)
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-09-20 20:51:59 +00:00
Preston Van Loon
728c77cc0c CI: use nostamp for code coverage and fuzzer uploads (#9634)
* Follow up to #9633, nostamp for code coverage

* nostamp for fuzzer uploads too

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-20 18:59:44 +00:00
terence tsao
161a13ac09 Correctly filter sync committee message at period boundary (#9626)
* Add head sync committee indices with slot consideration

* Apply rpc

* Apply sync

* Update tests

* Update mock.go

* Update sync_committee_test.go

* Update validator_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-20 18:06:43 +00:00
Preston Van Loon
cb631360e9 CI: nostamp config for better cache hits (#9633) 2021-09-20 17:29:41 +00:00
terence tsao
2ba29a3cfc Move block and attestation to proto v1alpha1 folder (#9618)
* Add block and attestation to container pkg

* Move aggregation into attestation

* Update attestation_test.go

* Move them to proto

* Gazelle

* fix cycle

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-20 16:17:03 +00:00
Nishant Das
150640305a Support Go Builds For Darwin Arm64 (#9600)
* add changes

* tidy

* stranded dep

* fixed commit

* use real commit

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-18 17:56:05 +00:00
Raul Jordan
11a1f681e0 Move Shared Packages Into Async/ (#9620)
* async packages

* change pkg

* build

* working

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-18 17:26:11 +00:00
Raul Jordan
7dadc780b8 Move Shared Packages into Math/ and IO/ (#9622)
* amend

* building

* build

* userprompt

* imports

* build val

* gaz

* io file

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-17 21:55:24 +00:00
Raul Jordan
d2f74615ab Move Miscellaneous Shared/ Packages Into Semantic Groups (#9624)
* remove shared network and ip util

* forkutil

* gaz

* build

* gaz

* nogo

* genrule

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-17 19:20:50 +00:00
terence tsao
6308678a89 Fail early if block and state are diff versions (#9625) 2021-09-17 13:03:14 -05:00
Radosław Kapka
93a4ebc662 Enable SSZ-serialization in v2 endpoints (#9587) 2021-09-17 14:08:59 +00:00
Nishant Das
4109d4a868 Standardize Metrics To Match Minimal Metric Spec (#9623) 2021-09-17 13:35:12 +02:00
Nishant Das
568797d53d Add MultiLock When Fetching Sync Head State (#9595)
* add lock when fetching sync state

* improvements

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-17 02:03:18 +00:00
Nishant Das
18e1d43360 Update blst to v0.3.5 and add BLS spec tests (#9611)
* add new dep

* add changes so far

* fix all tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-17 01:32:23 +00:00
Raul Jordan
8f0008c45a Move Related Packages Into API/ Pkg (#9619)
* begin on api pkg

* build

* build

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-16 19:55:51 +00:00
terence tsao
f49637b09b Move iputil and httpuil pkgs into network pkg (#9621)
* Add network pkg

* Go fmt

* Update discovery_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-16 19:12:27 +00:00
Raul Jordan
3daa2d8a18 Remove Deprecated Slasher (#9608)
* delete slasher

* rem slasher

* fix build

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-16 18:26:58 +00:00
Raul Jordan
61c24c3870 Move Fuzz/ Into Testing/ (#9617)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-16 17:32:51 +00:00
Raul Jordan
23e2d62c0c Create Container Package for Shared/ Subpackages (#9607)
* ensure run

* amend

* fix initial sync test target

Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-16 12:05:58 -05:00
Raul Jordan
8df96426ef Move Scripts Into Hack/ Directory (#9605)
* ensure build

* moved third party to toplevel:

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-16 16:22:39 +00:00
terence tsao
989daf267a Remove next slot cache usage for fork choice handler (#9601)
* Remove next slot cache usage for fork choice

* Go fmt

* Update process_attestation_helpers.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-16 15:29:59 +00:00
Raul Jordan
495e92ce9f Create Runtime Package for Moving Several Packages in Shared/ (#9588)
* add maxprocs

* add prereqs

* add tos and version pkg

* add in all runtime packages

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-16 09:46:29 +00:00
terence tsao
514d4ef8f5 Invert next slot state cache feature (#9602)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-16 07:48:06 +00:00
Radosław Kapka
d020e1b2d9 Correctly display epoch participation in GetStateV2 (#9598)
* Correctly display epoch participation in `GetStateV2`

* fix file names

* handle null case

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-16 02:51:51 +00:00
terence tsao
ee5d75732d Add pkg crypto (#9603)
* Add pkg crypto

* Update go.yml

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-15 22:55:11 +00:00
Preston Van Loon
fff0208207 Fix long running e2e tests (#9604)
* Revert e2e changes from #9565

* Override ChurnLimitQuotient to reflect minimal config prior to #9565
2021-09-15 22:04:13 +00:00
Radosław Kapka
c47923d20e Properly display sync committees in GetStateV2 (#9599)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-15 17:23:13 +00:00
Raul Jordan
5230af0e0c Move EndtoEnd Tests to Testing/ Folder (#9586)
* endtoend to testing/

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-15 14:42:05 +00:00
Nishant Das
34391fa780 Implement Batch Verifier For Pubsub Objects (#9344)
* add initial commit

* add aggregates in

* fmt

* add new changes

* gaz

* fix copy

* terence's review

* fix broken build
2021-09-15 05:49:50 +00:00
terence tsao
31a78ab22a Create config package for Shared/featureconfig (#9593)
* add config/features

* Gazelle

* Gazelle

* Fix build

* Go mod tidy

* active cache

Co-authored-by: rauljordan <raul@prysmaticlabs.com>
2021-09-15 01:18:39 +00:00
terence tsao
3e71997290 Create time Package for Shared/timeutil, mclock and slotutil (#9594)
* add time pkg

* Go fmt
2021-09-15 00:09:04 +00:00
terence tsao
77de467250 add lru (#9592) 2021-09-14 18:11:25 -05:00
Raul Jordan
9935ca3733 Move Shared/ Subpackages Into Monitoring/ Folder (#9591)
* add tracing

* monitoring pkg

* move prom

* Add client stats

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-14 20:59:51 +00:00
Radosław Kapka
5b37deb1a6 Allow listing attestations for Altair blocks (#9580)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-14 20:03:09 +00:00
Raul Jordan
01a72a9df2 Move Benchutil to Testing/ Folder (#9583)
* benchutil to testing

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-14 19:18:15 +00:00
Radosław Kapka
f98505bfbe Align ProduceBlockV2 to HTTP API spec (#9581)
* Align `ProduceBlockV2` to HTTP API spec

* snake case protos

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-14 17:48:24 +00:00
Raul Jordan
6340e58f36 Move Spectests Into a Testing/ Folder (#9582) 2021-09-14 16:02:58 +00:00
Radosław Kapka
ac3c544e29 Make until_epoch exclusive in PrepareSyncCommitteeSubnets (#9579)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-14 15:14:28 +00:00
Radosław Kapka
25f2ca4159 Align ProduceSyncCommitteeContribution to HTTP API spec (#9572)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-14 14:39:16 +00:00
Radosław Kapka
ae21665b0a Align GetStateV2 to API HTTP spec (#9569) 2021-09-14 09:07:03 +00:00
terence tsao
408392aa06 Enable prune canonical attestation (#9576)
* Enable prune canonical attestation

* Update shared/featureconfig/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-13 20:10:45 +00:00
Radosław Kapka
562e128251 Align GetSyncCommitteeDuties to API HTTP spec (#9570)
* Align `GetSyncCommitteeDuties` to API HTTP spec

* doc

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-13 19:14:05 +00:00
Radosław Kapka
4f86714c2a Align PrepareSyncCommitteeSubnets with API HTTP spec (#9571)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-13 18:16:16 +00:00
Radosław Kapka
1e4063e69c Align PublishContributionAndProofs to HTTP API spec (#9573)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-13 17:47:05 +00:00
terence tsao
e8e62856c8 Enable insert orphaned attestation (#9575)
* Enable insert orphaned att

* Go fmt
2021-09-13 17:05:09 +00:00
Radosław Kapka
ed68dd757f Implement ProduceBlockV2 in the beacon API (#9560)
* Rename service func to ProduceBlockV2

# Conflicts:
#	beacon-chain/rpc/eth/validator/validator.go

* initial implementation

# Conflicts:
#	beacon-chain/rpc/eth/validator/validator.go
#	proto/eth/v2/validator.pb.go

* test first try

* try new signature in test

* Fix test's sync committee signature

* Use 0xAA instead of 0xFF

* properly get altair block

* final implementation

* Revert "Auxiliary commit to revert individual files from 17779d4228b8c65452343fe5c51ef5764766f732"

This reverts commit 3f352a145c70c8edf8dd283f684fecf93f6d6487.

# Conflicts:
#	beacon-chain/rpc/eth/beacon/blocks.go

* revert pb.go changes to block

* add signature to return value

* fix ineffectual assignment

* simplify if/else statement

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-09-13 13:58:42 +00:00
Nishant Das
fd920bb786 Fix Finalized Block Retrieval in Altair (#9561)
* fix

* radek's review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-09-13 06:57:31 +00:00
terence tsao
416840036a Add beta4 spec test (#9565)
* Add beta4 spec tests

* Fix test setup for Altair registry update

* Update WORKSPACE

* Update churn limit calculation for e2e

* Hard set churn limit

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-11 01:33:31 +00:00
Preston Van Loon
1374b6dd4f Enable active balance cache (#9567)
* Enable active balance cache

* gofmt
2021-09-10 21:47:25 +00:00
Raul Jordan
dcc1f7c0ec Move End-to-End Tests for Altair Into Develop (#9564)
* end to end from hf1

* remove duplicate import

* skip sync eval

* conditional sync participation

* altair fork epoch to 6

* preston feedback

* proper fork epoch

* run for 6

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-10 19:59:43 +00:00
Radosław Kapka
2d9ae57378 Align SubmitPoolSyncCommitteeSignatures to API HTTP spec (#9563) 2021-09-10 09:55:56 -07:00
terence tsao
c6e7ab79ef Share ProcessSlashings between upgrades (#9557)
* Reuse process slashing

* Go fmt

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-10 15:57:16 +00:00
Radosław Kapka
006a2620c4 Align GetBlockV2 to API HTTP spec (#9562) 2021-09-10 10:20:52 -05:00
Nishant Das
741e136b22 Run Fork Watcher (#9559) 2021-09-10 11:51:55 +00:00
Nishant Das
1ccbe79209 Add Altair Metadata Changes (#9552)
* add metadata changes

* gaz

* add tests

* remove

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-10 08:10:08 +00:00
Nishant Das
57c2d86da1 Fork Watcher For Sync (#9550)
* add watcher

* raul's review

* preston's and terence's review

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-10 03:52:56 +00:00
Fredrik Svantes
ffd7579476 Add how users should report security vulnerabilities for this repository (#9525)
* Add how users should report security vulnerabilities for this repository

Suggestion to add a notice on how to report security vulnerabilities. This is visible at https://github.com/prysmaticlabs/prysm/security

* Add correct email

Co-authored-by: Nishant Das <nish1993@hotmail.com>

* Update SECURITY.md

Co-authored-by: Nishant Das <nish1993@hotmail.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-09-10 03:13:09 +00:00
Nishant Das
97f6143a43 Add Remaining Gossip Changes (#9553)
* add changes

* add changes here in

* rem duplicate import

* fix topic not being set in test

* terence's review

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-10 02:36:31 +00:00
Radosław Kapka
fc44ecbb16 Align GetEpochSyncCommittees to API HTTP spec (#9555)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-09 21:54:08 +00:00
Kirill Fedoseev
5325558f03 Recompute SqrRootSlotsPerEpoch at startup for custom configs (#9554)
* Recompute SqrRootSlotsPerEpoch at startup for custom configs

* Fix bazel deps

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-09-09 17:03:00 +00:00
Radosław Kapka
a6abfa5dd8 Implement PublishContributionAndProofs in the beacon API (#9526)
* implementation

* tests

* do not reuse pool

* godoc

* Revert "Auxiliary commit to revert individual files from 7f95a794ef9b2e1feac4683e181fc7d0fc289a0a"

This reverts commit 917ed4cc7a457847139d11bfb7c8d163584b48be.

* handle error in test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-09 15:49:11 +00:00
Nishant Das
2faaafb159 add remaining rpc methods (#9551) 2021-09-09 10:12:49 -05:00
Radosław Kapka
ddfa269e4f Add Version field to v2 endpoints (#9546)
* version enum

* get block

* get state

* rename shared to version

* goimports

* Go fmt

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-09-09 13:29:09 +00:00
Radosław Kapka
cb9c1ad0fe Implement Semi-Generic GetBeaconBlock RPC Endpoint for Pre & Post-Fork Blocks (#9517)
* BuildBlockData

* GetBlockAlatair

* build file

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer.go

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer_test.go

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer_test.go

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer_test.go

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer_test.go

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer_test.go

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer_test.go

* review comments

* add alias

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* add get block test

* comment

* implement get beacon block

* commentary

* test setup issues

* Use correct genesis state

* test renames

* pass

* rem commented code

* ctx unused

* proper grpc errors

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-09-09 05:26:49 +00:00
Nishant Das
279cc16c88 Add Sync Subnets Subscribers (#9539)
* add in digest

* add new stuff

* add tests

* terence's review

* Update beacon-chain/sync/subscriber.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* terence's review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-09-09 03:25:02 +00:00
Radosław Kapka
4b90c10cb2 Fix templating in eth protos (#9548)
* 1st try

* working version

* clean up build file

* fix v1 + cleanup

* remove unneeded comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-08 19:37:26 +00:00
Raul Jordan
83a9079700 Implement Generic ProposeBeaconBlock Function for RPC (#9549)
* add in generic propose block method

* propose generic with tests

* ineff assign issue
2021-09-08 18:54:04 +00:00
terence tsao
7c45c5f8cd More misc altair changes (#9547)
* More altair changes

* Gazelle
2021-09-08 17:52:37 +00:00
Nishant Das
90038a6001 Implement Network Encoder using SSZ Interface (#9535)
* add stuff

* unused

* kasey's review

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-08 23:16:53 +08:00
Preston Van Loon
b586d3784b Rename beacon-chain/core/state to beacon-chain/core/transition pkg (#9530)
* Move, refactor, fix

* @rkapka PR feedback: revert irrelevant changes

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-08 10:41:47 +00:00
kasey
bc551b7e30 using upstream fastssz with altair fixes (#9542)
* using upstream fastssz with altair fixes

* update generated ssz code

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-07 19:42:37 +00:00
terence tsao
d7679d2e71 Validator: rest of the Altair changes (#9532)
* Bring in rest of the validator chagnes

* Go fmt

* Deepsource

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-07 18:43:50 +00:00
Radosław Kapka
ddbac85ec4 Allow returning Altair block headers (#9543)
* GetBlockHeader

* ListBlockHeaders
2021-09-07 18:07:30 +00:00
Nishant Das
ee28dc3d4f Add in Fork Digest For Subscribers (#9538)
* add in digest

* radek's review

* raul's review

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-07 16:58:24 +00:00
Raul Jordan
d210dd7691 Maintainable Beacon Block API Endpoints (#9528)
* add new proto endpoints and deprecate old

* regen protos

* regen mocks

* begin fixing tests

* build tests

* e2e build

* go build

* resolve conflicts

* pass tests

* amend generated name

* fix build

* more fixes

* update gateway paths

* radek comments

* fix comment

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-09-07 15:17:50 +00:00
Radosław Kapka
d6416b29b2 Set Content-Length to zero for empty POST reponse (#9540)
* Set `Content-Length` to zero for empty POST reponse

* remove unused function param
2021-09-07 08:54:44 -05:00
Preston Van Loon
dc71f35f4f Remove copyutil pkg, move v1alpha1 copy methods into v1alpha1 pkg (#9533) 2021-09-04 06:55:57 +00:00
Preston Van Loon
4beb352e6f Move Slot and Epoch functions from helpers to core pkg (#9519)
* Move Slot and Epoch functions from helpers to core

* limited viz

* goimports

* fix fuzz build

* fix fuzz build

* fix

* fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-03 20:10:31 +00:00
terence tsao
29b851a2b7 RPC: Stream altair block (#9521)
* Add tests

* Fix tests

* Fix test

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-03 19:11:40 +00:00
terence tsao
ebf3897017 Validator: propose Altair block (#9414)
* Validator can propose block

* Gazelle

* Preston's feedback

* Rename

* Fix deep source

* Fix build

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-09-03 18:26:10 +00:00
Nishant Das
cc790ceb2e Add RPC Changes to Sync For Block Methods (#9497)
* add changes

* radek's review

* kasey's review

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-03 16:00:56 +00:00
Radosław Kapka
e875fe3dfc Implement ProduceSyncCommitteeContribution in the beacon API (#9524)
* implementation

* test

* change func signature

* remove broadcaster from test
2021-09-03 09:47:35 -05:00
Preston Van Loon
265b5feabf p2p: Handle nil topic (#9522)
* fix nil topic

* fmt

Co-authored-by: nisdas <nishdas93@gmail.com>
2021-09-03 14:39:54 +08:00
Nishant Das
30b2adc5d6 Add in Sync Contribution And Proof Pubsub Validator (#9475)
* add contribution handlers

* fix it

* fmt

* fix more

* add clean up

* Apply suggestions from code review

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/sync/validate_sync_contribution_proof.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* use validation pipeline

* deepsource

* fix all tests

* clean up

* comment

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-03 04:06:54 +00:00
terence tsao
924fe20dcd RPC: list block altair end point (#9494)
* Implement list block altair

* Dont export block container

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-02 22:21:33 +00:00
Preston Van Loon
2d41f0885c p2p: Split MsgID into its own file as a public method (#9518)
* p2p: Split MsgID into its own file as a public method

* gofmt

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-02 21:48:14 +00:00
terence tsao
019f0257c2 RPC: sync committee duties (#9501)
* Add get altair duties

* Refactor, clean up, Rahul's feedback

* Gazelle

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-02 21:17:42 +00:00
Kirill Fedoseev
f319535af5 Fix countdown timer (#9484)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-02 20:41:01 +00:00
Radosław Kapka
c45fe5cc1c Implement GetStateV2 in the beacon API (#9506)
* implementation

# Conflicts:
#	beacon-chain/state/v2/BUILD.bazel

* Revert "Auxiliary commit to revert individual files from 2cbe98c88777cac071876fe97f85029fad964e51"

This reverts commit edc4ff52e7796aefd1782e31eaf40231a3134693.

* tests

* fix function call

* make state package visible to migration

* fix span names

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-02 19:28:55 +00:00
terence tsao
1f48accb0e RPC: Extend participation and performance to Altair (#9499)
* Add endpoints and tests

* Update beacon-chain/rpc/prysm/v1alpha1/beacon/validators_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-02 18:42:04 +00:00
terence tsao
b10964514a Implement SubmitPoolSyncCommitteeSignatures in the beacon API (#9464) (#9503)
* Implement `SubmitPoolSyncCommitteeSignatures` in the beacon API (#9464)

* Update server

* remove duplicated field

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-09-02 16:54:53 +00:00
Radosław Kapka
4b9fb1cd1b Add tests to GetBlockV2 (#9486)
* Add tests to `GetBlockV2`

* add missing error assert

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-02 13:08:47 +00:00
Marcin Górzyński
02966e64d8 Feature lru cache wrapper 2 (#9511)
* Add Wrapper to LRU Cache to handle Invalid Parameters #9461

* Regenerate BUILD.bazel and simplify tests using lru.Cache

* Fix: fuzz_exports.go build error

* Fix: block_fuzz.go

* Revert lru.Cache interface

* Remove redundant err check in pending_attestations_queue_test.go

* Add tests for lru wrapper

* Use lru package in prysm/shared instead of lruwrpr

* Fix: goimports

* Fix: BUILD.bazel

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-09-02 18:36:54 +08:00
Nishant Das
43a24e3d8b Add Subscription Handler (#9508)
* use subscriber

* fix up tests

* fix

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-02 06:10:02 +00:00
Nishant Das
59ed552c64 Buffer Errors For Resubscribers (#9513)
* fix blocker

* make test better

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-02 03:33:25 +00:00
terence tsao
95e07963fb Stategen: fix state by slot off by 1 (#9509)
* Fix state by slot off by 1

* Comment

* Go fmt

* Add err

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-02 03:05:08 +00:00
Raul Jordan
2cd2bc87d0 Sync Committee Message P2P Validations (#9481)
* add in handlers

* add topic switch

* begin refactor

* build

* revert change to sub file

* reordered

* add in handlers

* add topic switch

* fmt

* begin refactor

* build

* revert change to sub file

* reordered

* rationale for cache size

* simplify

* deep source

* kasey feedback

* ignore if not in committee

* add more unit tests

* more cov

* code review commentary

* pass tests

* commentary

* terence comments

* terence feedback on key for cache

Co-authored-by: nisdas <nishdas93@gmail.com>
2021-09-02 02:26:38 +00:00
Radosław Kapka
69ed46af0f Add RPC validator sync committee code (#9510) 2021-09-01 23:08:05 +00:00
Radosław Kapka
28e472aa5f Implement PrepareSyncCommitteeSubnets in the beacon API (#9498)
* initial implementation

* final implementation

* review

* fix logic

* refactor duration calculation

* tests

* revert alias change

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-01 17:19:29 +00:00
terence tsao
46b22bb649 Clean up committee helpers (#9504)
* Clean up committee related helper funcs

* Go fmt

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-09-01 15:17:10 +00:00
Nishant Das
44375a3d67 Handle Missing Status Message Better (#9505)
* throw error in status method

* fix up

* fix up

* gaz
2021-09-01 22:25:22 +08:00
Nishant Das
eb690d3737 Add Subscription Topic Handler (#9490)
* add methods

* zahoor's review
2021-09-01 12:07:20 +08:00
Preston Van Loon
dad03ade77 p2p: Return error if no ChainState found for a peer (#9502)
* p2p: Return error if no ChainState found for a peer

* add commentary
2021-08-31 22:25:54 +00:00
terence tsao
6ee0a7e811 Spectest: increase sizes from small to medium (#9500) 2021-08-31 21:28:09 +00:00
Nishant Das
0ea4e9a71c fix trie (#9495) 2021-08-31 08:01:26 -07:00
terence tsao
da5f37fc71 Clean up from hf1 (#9493)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-08-31 10:47:44 +00:00
Preston Van Loon
59e8523272 fix spectest timeouts and sizing (#9492)
* fix spectest timeouts and sizing

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-31 09:36:56 +00:00
Nishant Das
8d4b92962b Add in Remaining Altair Core Tests (#9489)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-08-31 07:18:45 +00:00
terence tsao
8f88d574f5 Stategen: replay altair epoch (#9468)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-30 23:57:08 +00:00
terence tsao
921acc300b Clean up ProcessInactivityScores (#9483) 2021-08-30 22:50:51 +00:00
terence tsao
516401537a Update to spec test beta3 (#9491)
* Update to beta3 spec tests

* `ProcessEpochParticipation` processes only active indices

* Go fmt

* Update epoch_precompute_test.go

* Update WORKSPACE

* Fix current and previous epoch switch up
2021-08-30 19:32:11 +00:00
terence tsao
572498f954 Fix deep source complains (#9485) 2021-08-29 20:57:37 +02:00
Raul Jordan
20a1a58860 Proposer Sync Aggregate RPC Methods (#9479)
* sync agg

* more readable

* pass tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-28 20:47:45 +00:00
Radosław Kapka
822522a3af Implement GetSyncCommitteeDuties in the beacon API (#9478)
* implementation

* tests

* review feedback

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-28 22:00:46 +02:00
Nishant Das
4d02329cd5 Add in P2P Changes (#9390)
* add in initial changes

* add test method

* raul's review

* Update beacon-chain/p2p/gossip_scoring_params.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/p2p/gossip_scoring_params.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* preston's review

* kasey's review

* only 1

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-08-27 09:34:20 +08:00
Raul Jordan
031830baa4 Remove Kafka from Prysm (#9470)
* remove kafka

* gaz

* rem foreign rules

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-26 23:59:00 +00:00
terence tsao
2cc9fc9e0e Add altair spec tests (#9477)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-26 23:08:04 +00:00
terence tsao
1284496648 Increase seen attestation cache sizes (#9476)
* Increase attestation cache sizes

* Update WORKSPACE

* Go fmt

* Fix tests

* Go fmt

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-26 21:37:35 +00:00
Raul Jordan
c1f841d6d4 Allow Sending Interfaces Over Application Event Feeds (#9458)
* add event feed changes

* deep source issues

* omit receiver name

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-08-26 15:21:29 -05:00
Raul Jordan
7d5f30b01e Delete Deprecated Flags and Old Slashing Protection for V2 (#9453)
* delete deprecated flags and old slashing protection schema

* remove toledo mentions

* build

* do not remove migration
2021-08-26 13:48:25 -05:00
Radosław Kapka
89941d4be2 Implement GetBlockV2 in the beacon API (#9433)
* Implement `GetBlockV2` in the beacon API

* fix gateway config test

* Revert "fix gateway config test"

This reverts commit 8179400b2a.

* unregister v2

* review feedback

* improve comment

* reduce duplication

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-26 15:22:06 +00:00
terence tsao
74df479bd3 Use active balance cache (#9469)
* Use active balance cache

* Feature flag and test

* Gazelle

* Fix fuzz

* Fix test

* Mix in validator count

* Update test to reflect validator count

* Add eth1 data block hash

* Fix assign

* Use eth1 data root

* Update active_balance_test.go

* Remove eth1 data root

* Remove eth1 data usage in tests

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-26 14:32:40 +00:00
Nishant Das
d22552944d Update Blockchain Package From Altair (#9422)
* add latest changes

* fix up

* raul's review

* pass cache tests

* Update beacon-chain/blockchain/head_sync_committee_info.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* kasey's review

* rename cache err

* add nil checks and error path checks

* gaz

* fix test

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-08-26 05:01:09 +00:00
terence tsao
2903275e7e Altair process slots (#9455)
* Update process slots

* More test scenarios

* Update transition_test.go

* Add CanUpgradeToAltair

* Go fmt

* Gaze

* Update beacon-chain/core/state/transition.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update transition_test.go

* Update altair fork epoch for e2e

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-26 09:59:23 +08:00
Preston Van Loon
3d6fad3121 Update Prater config to include Altair fork epoch (#9467)
* Update Prater config after https://github.com/eth2-clients/eth2-networks/pull/58. Add tests to enforce compliance.

* gofmt

* spec: true

* fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-25 17:18:29 +00:00
terence tsao
ffbb722777 Add active balance cache (#9456)
* Add active balance cache

* Better comment

* Fuzz

* ,

* go fmt

* Fix err

* Raul's feedback

* A few renames to effective
2021-08-25 16:39:12 +00:00
Radosław Kapka
8b10adc92a Fix struct type used when sending beacon committee subscriptions (#9466) 2021-08-25 15:35:28 +00:00
Nishant Das
0d60863bed Add Stategen Log (#9460)
* add log

* amend it

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-08-25 14:34:09 +00:00
Nishant Das
0208d9b328 Change P2P Logs to Trace Logs (#9459) 2021-08-25 15:35:50 +02:00
Raul Jordan
e150e29710 Add In Generated Mocks for Altair RPC (#9457) 2021-08-24 21:38:44 +00:00
terence tsao
45b6a80a30 Validator: perform sync committee duties (#9411)
* Validator sync committee methods

* Gazelle

* Update visibility

* Add setupWithKey

* Refactor selection proofs

* Fix build

* Refactor compute and sign

* Fix sign request

* Fix test

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-24 17:33:42 +00:00
terence tsao
cd4bb6a9c8 Add altair process epoch (#9449)
* Add altair process epoch

* Verify balance

* Gaz

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-24 16:07:11 +00:00
Nishant Das
114a14a4b6 Patch Initial Sync For Non Finalized Blocks (#9452)
* fix for now

* off by 1

* preston's review
2021-08-24 10:20:45 -05:00
Raul Jordan
a49c0f19ae Add Sync Committee Head State Cache (#9448)
* add sync committee head state cache

* handle err in init

* include file in build file

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-24 07:54:04 +00:00
Preston Van Loon
362212b5ed Fix 'headSlot' log statement in init-sync (#9451)
* Fix 'headSlot' log statement in init-sync

* gofmt -w beacon-chain/sync/initial-sync/round_robin.go
2021-08-24 07:00:18 +00:00
terence tsao
5c96a2713d Prevent Reinsertion of Orphaned Attestations Into Pool (#9442)
* Can re-save orphaned attestation

* Go fmt

* Fix tests

* Go fmt

* Bug fix flag

* Bug fix flag

* Go fmt

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-24 03:05:08 +00:00
terence tsao
6457ec17bf Process epoch: precompute previous and finalized epoch (#9445)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-24 01:42:05 +00:00
terence tsao
d3475a563d Remove Canonical Attestations From Pool Correctly (#9444)
* Remove canonical attestaitons

* Rm debug log

* Remove deleted test

* Tests and fmt

* Tests

* Bug fix flag

* Move feature flag

* Go fmt

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-24 00:33:17 +00:00
Nishant Das
ab101e8a4a Fix Validator Backups (#9446)
* fix val backups

* fix

* debug

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-23 21:46:04 +00:00
Preston Van Loon
5442d0f486 Altair: Merge passing hf1 tests and shared code (#9447)
* Altair: Merge passing hf1 tests and shared code

* Remove weird files

* del commented code

* Add dynamic file fetching, remove hardcoded list of files

* Remove hard coded paths

* remove hardcoded paths from phase0 too

* nosec fixes
2021-08-23 20:09:24 +00:00
Raul Jordan
0ee203fc2e Deduplicate Copy Functions In State Packages (#9437)
* remove duplicated copy functions

* add all unit tests

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-08-23 16:53:50 +00:00
terence tsao
63e0a4de84 Transition: Altair process block (#9430)
* Add altair transition

* Add tests

* refactor exit into operation funcs

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-08-21 00:32:37 +00:00
terence tsao
cac1e94f04 Add altair epoch processing functions (#9404) 2021-08-20 16:55:42 -07:00
Raul Jordan
bdfdba1e23 Remove Unused Trieutil Helpers (#9435)
* remove unused trieutil helpers

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-20 23:00:04 +00:00
Raul Jordan
e2a3dc13fb Less Flakey E2E API Checks (#9439)
* compare list attestations instead

* add query filter

* fix failing

* fix response

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-20 22:30:56 +00:00
Radosław Kapka
af2801f215 Implement GetEpochSyncCommittees in the beacon API (#9434)
* initial implementation

* get committee for epoch

* test

* fix failing test

# Conflicts:
#	beacon-chain/rpc/prysm/v1alpha1/validator/assignments_test.go

* combine function arguments

* review feedback

* use sync committee setter

* fix failing test

* fix build

* remove bad test

* refactor for single responsibilities

* radek suggestions

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-20 21:32:56 +00:00
Raul Jordan
fa4a4225a7 Improve DB.SaveBlocks Performance (#9431)
* improve save blocks performance

* comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-20 20:01:38 +00:00
terence tsao
005ce7e238 Remove float64 usage for fork choice (#9438) 2021-08-20 14:15:18 -05:00
terence tsao
25bc3f2aeb More altair spec tests (#9436)
* Add fork and ssz static spec tests

* Fix build

* Remove minimal tags
2021-08-20 16:47:37 +00:00
terence tsao
67c3ae0117 Add signature verification using current fork (#9405) 2021-08-20 07:38:35 -07:00
Raul Jordan
6368da2716 Remove Nested BoltDB View Within Update Transaction in SaveHeadBlockRoot (#9428)
* no nested bolt txs

* remove the nested view in update calls

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-19 21:42:30 +00:00
terence tsao
3764b7dca3 ValidatorIndexByPubkey length check (#9424)
* `ValidatorIndexByPubkey` length check

* gaz

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-19 19:33:05 +00:00
Raul Jordan
eac542a8ac Change Eth2 Repository Names (#9425)
* eth2 repo name changes

* rem sha

* use consensus spec terminology and pin sha
2021-08-19 13:00:57 -05:00
Nishant Das
9f36116c2f Update Core Helpers (#9393)
* add helpers

* naming

* remove comments

* Terence's feedback

* Revert `UpdateCommitteeCache` changes

* Update beacon-chain/core/helpers/committee.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-08-19 12:18:27 +00:00
terence tsao
d874b9140f Optimize process attestations (#9418) 2021-08-18 15:41:09 -07:00
Preston Van Loon
dfd740414f Altair: operations spectests (#9413)
* Update spectests to v1.1.0-beta.1 from hf1 branch

* fix params loading

* Altair: operations spectests

* gazelle and nosec on trusted variable inputs

* remove legacy statement from spectest/minimal/altair/operations/BUILD.bazel
2021-08-18 22:01:05 +00:00
Raul Jordan
72cfe50eff Move Validator Accounts Protos Into Proper Place (#9416)
* move into proper namespace

* package naming

* add strict deps

* regen protos

* add web api

* regen

* builds

* rename

* build
2021-08-18 21:24:01 +00:00
terence tsao
61acf6f1d1 Slash validator with altair config (#9415) 2021-08-18 18:44:17 +00:00
Radosław Kapka
0d8ceb18b0 Improve v2 protos (#9412)
* Generate SSZ code for `BeaconBlockContainerV2`

* new line at end of file

* remove unneeded V2 suffix

* more fun stuff

# Conflicts:
#	proto/eth/v1/generated.ssz.go

* rename Altair to V2
2021-08-18 15:50:37 +00:00
Nishant Das
5b7b9f9ac9 Refactor Subnet Methods For Altair (#9386)
* change subnets

* add in missing one

* add comment

* Update beacon-chain/p2p/subnets.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* terence's review

* terence's review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-18 04:55:48 +00:00
Nishant Das
8e503d0980 Change Back Read-Only Validator Declaration (#9410)
* remove indirection

* fix
2021-08-18 11:59:55 +08:00
Preston Van Loon
4966300c96 Update spectests to v1.1.0-beta.1 from hf1 branch (#9403)
* Update spectests to v1.1.0-beta.1 from hf1 branch

* fix params loading
2021-08-17 21:23:38 +00:00
Raul Jordan
bdb09ca9ea V1Alpha1 gRPC Gateway End-to-End Test Evaluators (#9375)
* add in initial evaluator for gateway v1alpha1

* add gazelle

* build

* compare

* functional options and tool for local verification

* chain head response compare base64

* compare validators request

* add validators compare

* more compare

* passing validator comparisons

* compare attestations pool

* compare atts

* begin peers test

* e2e verifiers

* correct port

* renaming

* evaluators

* rem api compare tool

* peer compare tests

* deepsource

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-16 20:31:21 +00:00
terence tsao
b28d65dc34 Add pyrmont hard fork config (#9394)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-16 19:51:43 +00:00
Preston Van Loon
643e20c50a p2p: report status failure when no genesis time is set after start (#9395) 2021-08-16 19:15:43 +00:00
Radosław Kapka
bb68c591f2 Remove beacon-chain/rpc/eth/v1 directory (#9392) 2021-08-16 13:11:25 +02:00
Nishant Das
f8355bb241 Add in Database Support For Altair State (#9388)
* add in schema

* add in altair state support

* nil check

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-16 12:39:52 +08:00
Raul Jordan
8122da6c97 Add Gosec Github Action (#9332)
* add gosec security scan

* add gosec ignores first batch

* more nosec for exec

* add filepath clean

* more nosec

* file inclusion nosec

* build

* herumi

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-15 15:24:13 +00:00
Radosław Kapka
1936f991eb Prepare protos for Ethereum API v2.0.0 (#9384)
* Move proto services to a different package

# Conflicts:
#	beacon-chain/rpc/service.go

* fix tests

* goimports

* fix java class name

* without aggregation_bits

* aggregation_bits

* v2 endpoints

* generate new code and define function stubs

* build files

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-14 16:41:03 +00:00
Nishant Das
49440c90ba Add in Database Support For Altair Blocks (#9387)
* add in schema

* gaz

* fix panic in rpc

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-14 15:50:49 +00:00
Nishant Das
47ee6fa17e Add in Altair Message ID Function (#9382)
* add in changes

* add method call

* gaz

* rauls review

* rauls review

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-14 14:36:02 +00:00
Radosław Kapka
de907fc362 Fix java classname in beacon_debug_service.proto (#9380)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-13 17:59:55 +00:00
Nishant Das
a1c0bee397 Add in Object Mapping For Types (#9381)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-13 17:28:26 +00:00
Raul Jordan
b42465a7dd Optimized Slasher Parameters (#9361)
* add in parameters and helpers files

* add in small set of changes

* build file

* viz
2021-08-13 16:53:04 +00:00
Nishant Das
8fe22422a6 add it in (#9383) 2021-08-13 10:53:51 -05:00
Radosław Kapka
f516e71167 Move proto services to a different package (#9379)
* Move proto services to a different package

# Conflicts:
#	beacon-chain/rpc/service.go

* fix tests

* goimports

* fix java class name
2021-08-13 10:55:24 +00:00
Radosław Kapka
ede85c21e2 Clean up v1 protos (#9377)
* Clean up v1 protos

# Conflicts:
#	proto/eth/v1/generated.ssz.go

* A little more cleanup

# Conflicts:
#	proto/eth/v1/generated.ssz.go

* fix java class names
2021-08-13 09:12:18 +00:00
Nishant Das
a9f2170657 Update P2PUtils And Parameters From Altair (#9373)
* add changes

* add p2putils changes

* clean up

* fix build

* remove test

* raul's review
2021-08-13 13:11:11 +08:00
Nishant Das
3a528147a2 Change to Proto Check For Blocks (#9372)
* check inner objects

* add test in here.

* verify better
2021-08-13 00:35:23 +00:00
Nishant Das
c8cffbbf02 Update Dependencies For Prysm (#9374)
* update deps

* tidy
2021-08-12 11:07:13 -05:00
Raul Jordan
1f102c256d Altair Committee Cache Changes (#9366)
* changes to hf1 caches

* suggested

* Update beacon-chain/cache/sync_subnet_ids.go

* add in proper changes

* committee.go

* committee test and disabled

* nil active balance check
2021-08-12 00:04:40 -05:00
terence tsao
e3492698f8 Core: modified process attestation for Altair (#9367)
* Add epoch participation and proposer base reward helpers

* Update beacon-chain/core/altair/attestation.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Check length

* Go fmt

* Add process att

* Refactor and tests

* Update attestation.go

* Gazelle

* Under flow error message

* Update beacon-chain/core/altair/attestation.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-11 21:58:25 -05:00
Jim McDonald
5256751e8b Trim SSE event names. (#9342)
* Trim SSE event names.

When sending SSE events the event name contains trailing whitespace.
This patch removes the whitespace before processing the event, ensuring
it is easily parsed by clients.

* Separate test.

* Test receiveEvents, not writeEvent.

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-08-11 23:00:39 +00:00
terence tsao
225f9a74f6 Add epoch participation and proposer base reward helpers (#9356)
* Add epoch participation and proposer base reward helpers

* Update beacon-chain/core/altair/attestation.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Check length

* Go fmt

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-11 20:54:45 +00:00
terence tsao
d77616f705 Fixes Issues Found by Goland IDE Code Inspect (#9368)
* Ran code inspect

* Update shared/testutil/altair.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-11 20:12:22 +00:00
Raul Jordan
a217d71d08 Sync Protobufs Between Hard Fork Branch and Develop (#9362)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-11 19:14:49 +00:00
Raul Jordan
efa1f29311 Use Different Golang JWT Library (#9360)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-11 18:10:53 +00:00
Raul Jordan
136b157d00 Better Slashing Protection Import/Export User Experience (#9355)
* failure mode if command goes wrong

* improve export

* fix struct orderw

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-11 17:24:24 +00:00
Raul Jordan
c1280867ed Add Tracing Request Sink to E2E (#9341)
* add tracing request sink

* fix struct order

* add in base64 encode and gzip

* add encoding and gzip

* tracing sink and replay tool

* post

* replay

* include latest sink and replay tool

* capture the gzout file instead

* rem time sleep

* handle err

* better handling

* add documentation

* changes

* working sync

* working

* added more logging
2021-08-11 15:38:00 +00:00
Nishant Das
0dd7a8b078 Fixes Altair Bug Introduced For State Representation (#9358)
* add test

* test

* test
2021-08-11 15:02:55 +00:00
terence tsao
41ab27402f Sync committee subnet cache (#9350)
* Add sync subnet id cache

* Update BUILD.bazel

* Update BUILD.bazel

* Update beacon-chain/cache/sync_subnet_ids.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Add comments

* add more comments

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
2021-08-11 00:26:20 +00:00
terence tsao
9668ab0fea Clean up duplicated imports (#9354) 2021-08-10 14:55:24 -05:00
Radosław Kapka
7a9c717ae0 Reduce locking time in recomputeFieldTrie (#9353)
* Reduce locking time in `recomputeFieldTrie`

* add comment

* reuse variable and improve comment
2021-08-10 09:54:35 +00:00
Nishant Das
a5998fab2f Clean Up State Representation (#9339)
* checkpoint all work

* add new fixes

* fixes

* gaz

* terence's review

* terence's review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-09 21:54:24 +00:00
terence tsao
8df6854a74 Core: upgrade state to Altair (#9349)
* Add upgrade to altair

* Update BUILD.bazel

* Update beacon-chain/core/altair/upgrade.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Move getter and setter outside the loop

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-09 21:27:51 +00:00
Jim McDonald
3de168b806 Fix state state transition logging. (#9343)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-09 20:05:46 +00:00
Preston Van Loon
bfdaf8a3f5 Fix config loader tests (#9348)
* don't mutate mainnet config

* add missing minimal config params

* Ensure all yaml fields were set and are correct

* gofmt

* Add sanity check that some empty yaml wasn't given

* Gazelle and deepsource feedback
2021-08-09 18:40:06 +00:00
terence tsao
38787353a2 Attestation: get participation status (#9340)
* Add participation flag indics matching helpers

* Gazelle

* Tests

* Update BUILD.bazel

* Update attestation.go
2021-08-09 16:35:46 +00:00
Nishant Das
170e2e115b Fix Flaky Sync Committee Test (#9337)
* fix flake

* Update beacon-chain/core/altair/sync_committee_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-09 14:49:13 +00:00
terence tsao
be85b7f909 Cleanup: remove test only process block/attestation (#9345)
* Remove test only process block attestation code

* Rm fuzz process block

* Revert "Rm fuzz process block"

This reverts commit 5d1b0f0741.

* Still can test fuzz

* Fix return

* More unused funcs
2021-08-08 22:01:06 -05:00
terence tsao
ad048a8750 Rewards/penalties: epoch processing for Altair (#9334)
* Add epoch precompute methods

* Better name

* Bring expensive computation outside

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-08-06 20:02:35 +00:00
Raul Jordan
90ea158301 Add Some More Traces to Blockchain Service (#9336)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-06 15:02:36 +00:00
Radosław Kapka
ef9eff29b8 Tiny state management improvements (#9338)
* Tiny state management improvements

* fix indices length calculation

* review comments

* revert function rename
2021-08-06 10:47:50 +00:00
terence tsao
4cce4f1ee5 Refactor slash validator func's signature (#9321)
* Refactor slash validator func's signature

* Named arguments

* Update beacon-chain/core/blocks/proposer_slashing.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-06 04:06:49 +00:00
Raul Jordan
ce10be9a16 Add Latest Slasher KV Methods to DB Subpackage (#9328)
* tests pass

* use end slot for epoch pruning

* amended to endslot of epoch

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-08-05 21:27:05 +00:00
Raul Jordan
4f0b00894b More Precise Metrics E2E (#9331)
* metrics test copute slots since genesis

* gazelle

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-05 20:44:56 +00:00
Radosław Kapka
57fac9b74f Improve API Middleware design (#9324)
* remove superfluous WriteHeader call

* move custom handlers outside hooks

* rename OnPostStart hook

* function that returns a default endpoint

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-05 20:21:43 +00:00
Mohamed Zahoor
395feeb35d Cross check db state bucket after migration (#9311)
* added state db check after migration

* satisfy deepsource

* added db compare after migration sub-command

* fomt issue

* point ot bolt issue

* gazel fix

* maxUint84

* revert accidental deletion of WORKSPACE file

* added sorting support

* cleanup debug

* gazel fix

* satisfy deepsource

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-05 18:30:05 +00:00
terence tsao
7bc22e4ddf Cache: sync committee index position (#9317)
* Add sync committee index cache

* Update BUILD.bazel

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-05 17:44:13 +00:00
Radosław Kapka
016beb51c0 Make SubmitAttestations v1 implementation similar to v1alpha1 (#9319)
* Make `SubmitAttestations` v1 implementation similar to v1alpha1

* fix test

* fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-05 08:01:20 +00:00
Radosław Kapka
456b4d463e Implement PrepareBeaconCommitteeSubnet in the validator API (#9318)
* validator migration

* status helpers

* implementation

* fix existing tests

* build file

* tests

* run endpoint through API Middleware

* reduce helper package visibility

* empty caches before each test

* assign all validators to subnets

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-08-05 07:23:52 +00:00
terence tsao
4d33068496 Core helper: validate sync message time (#9325)
* Add validate sync message time

* Update BUILD.bazel
2021-08-05 12:39:32 +08:00
terence tsao
6e9faa3618 Epoch: precompute functions for Altair part 1 (#9309)
* Add epoch precompute methods

* Split into part 1 for better review

* Update epoch_precompute_test.go

* Preston's feedback

* Skip if the valiator is not eligible

* Add regression test for non eligible

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-04 15:13:05 +00:00
Nishant Das
fd42cb712f Fix Incorrect Conditional Check (#9323)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-04 13:50:27 +00:00
Radosław Kapka
c5b68b4f03 Implement PublishAggregateAndProofs in the validator API (#9294)
* grpc implementation

* build file

* full implementation

* remove unused function

* tests

* wrap top-level array

* Update beacon-chain/rpc/eth/v1/validator/validator.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-04 09:01:38 +00:00
Mohamed Zahoor
239d904d27 Fix memory usability (#9310)
* fix cache usability

* fix the name of the prometheus flag

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-04 02:33:28 +00:00
terence tsao
461be99fac Block: process sync aggregate (#9308)
* Add altair base reward funcs

* Update BUILD.bazel

* Add process sync aggregate

* Update BUILD.bazel

* Update block.go

* Update block_test.go

* Feedback, refactor and more tests

* Further refactor sync committee votes

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-08-03 16:14:15 +00:00
Preston Van Loon
9572add171 Fail fast when using old define=ssz=<value> (#9316)
* Fail fast when using old define=ssz=<value>

* remove print statement
2021-08-02 19:04:30 +00:00
ahadda5
17cf2e5377 Fixing .bat file issues (#9266)
* replacing [] with "" for ifs
using !! vs %% for var with delayed expansion
Check first for 404 then download.

* remove the goto after if are fixed

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-08-02 16:09:09 +00:00
Radosław Kapka
e4388c7fce Update grpc-gateway dependency to proper version (#9315)
* logging

* properly update grpc-gateway dependency

* Revert "logging"

This reverts commit 6c59a87ac0.

# Conflicts:
#	go.mod

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-02 14:52:54 +00:00
Mohamed Mansour
47c28e6cd6 Show error in logs if passing invalid flags in yaml (#9033)
* Show error in logs if passing invalid flags in yaml

The YAML configs pass if an invalid flag is set, this causes frustration
because the flag could have been mispelled causing a huge mystery.

This will strictly check if the flags are deserialized, if a flag
doesn't exist, it will error out as a Fatal error with appropriate
reasoning which flag is wrong.

* Fix review comments to make some of them non-fatal

* Make yaml.TypeError a pointer

* Remove UnmarshalStrict from spectests since they use a different YAML package

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-08-01 03:26:24 +00:00
Preston Van Loon
0c1fc65604 Core/deposit: refactor blocks.ProcessDeposit for less complex altair implementation (#9305)
* Refactor blocks.ProcessDeposit to return whether or not the deposit resulted in a new validator entry.

* Only mark validator as new after successfully appending it to the state

* fix merge conflict

* fix merge conflict

* gofmt

* fix merge conflict

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-31 23:42:07 +00:00
terence tsao
5c36d59672 Reward: altair base reward (#9307)
* Add altair base reward funcs

* Update BUILD.bazel

* Better tests

* Update BUILD.bazel
2021-07-30 19:17:20 +00:00
Nishant Das
7576442ef0 Only Create Deposit Proofs Pre-Genesis (#9289)
* remove proof gen

* fix pre-genesis case

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-30 18:33:31 +00:00
terence tsao
d130bf0035 Fix more duplicated import prefix (#9306) 2021-07-30 18:05:38 +00:00
Mohamed Zahoor
0d818a5709 Reduce memory overhead during DB migrations (#9298)
* wip

* migration happening

* wip

* fix memory utilization, add the testcases

* migration issues

* the gazel

* remove debug logging

* goimport

* gazel
2021-07-30 08:38:54 -05:00
terence tsao
560fe69425 Core: sync committee helpers (#9269)
* Add sync committee helpers

* Create BUILD.bazel

* Update BUILD.bazel

* Update BUILD.bazel

* Fix visibility

* Preston's review
2021-07-29 23:19:32 +00:00
terence tsao
012d279663 Fix duplicated imports (#9304)
* Fix duplicated imports

* Fix metrics test
2021-07-29 16:45:17 -05:00
terence tsao
fcd4938ec0 Head: validator index getters (#9303) 2021-07-29 20:05:48 +00:00
Radosław Kapka
4db0dfc4f1 Properly broadcast attestations (#9301)
* better handling of gateway response

* empty response tests

* build file

* broadcast attestations properly

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-07-29 18:33:18 +02:00
terence tsao
cf4ff3d6f9 Clean up phase 0 spec tests (#9299) 2021-07-29 09:01:18 -05:00
Raul Jordan
9145310647 Eliminate Proto V2 Namespace (#9297)
* get rid of v2 in prysm codebase

* replace block2

* builds

* terence comments

* gazelle
2021-07-28 21:23:44 +00:00
Mohamed Zahoor
21c79f4dfa Protect the state objects in memory when saving them (#9291)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-28 18:27:27 +00:00
Raul Jordan
98f7f82b2f Copy Altair Changes Into v1alpha1 Namespace (#9296)
* move the v2 changes into v1alpha1

* v1alpha1 builds

* regenerate pbs

* regenerate ssz

* top level builds
2021-07-28 18:05:07 +00:00
terence tsao
f16b9859e7 Test: add altair test utility (#9287)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-07-28 17:15:23 +00:00
Nishant Das
a372e30c8f Fix Spaced Arguments for Subcommands (#9292)
* fix args for commands

* fix nasty edge case
2021-07-28 14:36:48 +00:00
Nishant Das
415973fca3 remove flag (#9290) 2021-07-28 16:11:22 +08:00
Raul Jordan
2d9915d2d9 Define Unimplemented V2 Services (#9286)
* generate ssz bindings for protos

* define services and unimplemented

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-27 20:42:14 +00:00
Mohamed Zahoor
c6bd7a2f83 Efficient Historical State Representation (#9155)
* added migration in to new schema

* gazel fix

* goimports fix

* construct state with validator entries from a seperate bucket

* save state with validator entry indirection

* fixed save and retreieve issues

* fixed more test cases related to DeepEqual

* added save benchmark

* gazel fix

* organize benchmarks

* add cache and improve state construction  performance

* gazel fix

* check type

* remove cache checking from Has

* fix decoding when cache is off

* fix slice bugs

* add migration testcases

* linter fix

* fix few review comments

* fix review feedback

* gazel fix

* satisfy deepsource

* added the feature flag

* lint fix

* fix usage of featureconfig Inti in testcases

* call resetConfig method

* add feature flag in  migration testcases

* fix formatting

* change bucket name from blockRootValidatorKeysIndexBucket to blockRootValidatorHashesBucket

* remove from cache when state deleted

* fixed few more comments

* added to devModeFlags

* added cache delete under the feature flag

* fix lint

* change cache sizes and improve documentation

* fiexed few more review coments

* not using hash anymore and using a new SaveStates function

* satisfu deepsource

* run gazel

* fix feature flag related stuff

* fixing merge conflict fix

* few goodies

* improve UX and dont swallow error

* merge fix

* import format fix

* fix migrationion when flag not given issue

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-07-27 20:02:53 +00:00
terence tsao
7129af5ccf Core/deposit: altair processing methods and tests (#9227) 2021-07-27 12:10:08 -07:00
Raul Jordan
f700808c8b Generate SSZ Bindings for Latest Protos (#9285) 2021-07-27 17:57:11 +00:00
Raul Jordan
1ea1c69174 Copy V1Alpha1 Protos and API Endpoints To V2 Namespace (#9284)
* v2 builds

* generate go code

* fix build
2021-07-27 16:54:49 +00:00
terence tsao
54772596e0 State/V2: rest of pkg (#9283)
* Add rest of state v2

* Update BUILD.bazel
2021-07-27 15:47:03 +00:00
Nishant Das
a556cf27c4 Use Only Checkpoint Cache for Processing Attestations (#9282)
* use only one cache

* clean up
2021-07-27 07:26:53 -07:00
terence tsao
5b5c8f4d3f State/V2: setters (#9279)
* Add state v2 getters

* Update BUILD.bazel

* Add state v2 setters

* Update BUILD.bazel
2021-07-26 17:27:05 -05:00
terence tsao
245dc23cde State/V2: getters (#9228)
* Add state v2 getters

* Update BUILD.bazel
2021-07-26 20:55:30 +00:00
Radosław Kapka
837cd4eb8f Implement ProduceAttestationData in the validator API (#9271)
* functionality

* test

* build file

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-07-26 19:22:58 +00:00
Radosław Kapka
8c8f1bb9c1 Regression test for #9264 (#9277) 2021-07-26 11:48:51 +00:00
Kaan
a8c49c50ad v1alpha1 validators: catch possible reqState nil case (#9275)
* v1alpha1 validators: catch possible reqState nil case

* Minor format of error log

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-07-26 13:50:40 +08:00
Lorem Ipsum
14d0d9195f Fixing wrong node information in fallback log (#9258) (#9272) 2021-07-25 23:46:54 +02:00
terence tsao
8db0c9a0e6 Move proto/interfaces -> proto/prysm/v2 (#9270)
* Move proto/interfaces -> proto/prysm

* Update paths

* Fix block2

* Update blocks_fetcher_utils.go

* Update BUILD.bazel

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-23 20:10:15 +00:00
Preston Van Loon
938a038c42 Fix //... for darwin (#9273)
* Fixes

* More manual tags

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-23 18:26:49 +00:00
Raul Jordan
30e93f5763 Move Slashing Protos to V2 (#9256)
* proto slashing

* move slashing related protos

* gaz

* include slasher pbs consolidated

* slasher pb fmt imports

* imports

* beacon builds

* build validator

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-23 18:01:01 +00:00
Radosław Kapka
be82c8714f Do not require a handler function in the gateway (#9264)
* Do not require a handler function in the gateway

* test fix

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-23 16:48:22 +00:00
Preston Van Loon
c0076cc7a2 State: move state interfaces into state package (#9268)
* Move interfaces from github.com/prysmaticlabs/prysm/beacon-chain/state/interface to github.com/prysmaticlabs/prysm/beacon-chain/state

* remove/rename state2
2021-07-23 16:11:21 +00:00
Radosław Kapka
a8363405f8 Implement ProduceBlock in the validator API (#9259)
* server-side implementation

* build file

* run through api middleware

* tests

* build file

* gofmt

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-07-23 16:29:38 +02:00
Raul Jordan
2d10bcf179 Move State Protos Into V2 Namespace (#9257)
* move state protos

* regen ssz

* edit v1 code

* fix imports

* building

* beacon chain builds

* validator and shared builds

* fuzz builds

* changes

* spectest builds

* tools build

* remove import cycle

* generate ssz

* pcli

* gaz

* kafka

* gaz
2021-07-23 08:07:40 -05:00
terence tsao
526596a679 State/v1: add atts and block roots getter tests (#9255)
* Add att and block getter tests

* Update BUILD.bazel

* Update getters_attestation_test.go

* Update beacon-chain/state/v1/getters_block_test.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Fix mutation tests

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-07-22 22:31:14 +00:00
Raul Jordan
5864795ca5 Move RPC Protos Into V2 Namespace (#9254)
* moved rpc protos

* gazelle

* change pb

* validator health

* edit name

* naming
2021-07-22 21:00:28 +00:00
Raul Jordan
b7919b3115 Move DB Protos Into V2 Namespace (#9253)
* move db protos into prysm v2

* build

* build

* imports gaz

* v2

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-22 20:05:06 +00:00
Raul Jordan
715aa408e3 Move Web Protos Into Prysm V2 (#9249)
* move web protos

* move web to v2

* update web protos

* gaz

* proto build

* replace mentions of validator v2

* gaz
2021-07-22 19:19:24 +00:00
Raul Jordan
ceb1ec451a Remove Faucet Protos (#9252)
* rem faucet protos

* tidy
2021-07-22 18:27:31 +00:00
Radosław Kapka
b35c3ab2e7 Implement GetAggregateAttestation in the validator API (#9243)
* server-side implementation

* run through api middleware

* tests

* build file

* build file fix

* return attestation with most aggregation bits

* build file

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-22 17:48:32 +00:00
terence tsao
c8685d256c Lint: add v1 as import alias (#9250)
* Add v1 as import alias

* Update helpers.go

* More

* Update skip_slot_cache_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-22 17:13:18 +00:00
Raul Jordan
0fb43aec34 Remove Unused Cluster Protos (#9251)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-22 16:39:27 +00:00
terence tsao
9addb5f7c3 Core/epoch: sync committee period (#9248)
* Combine function parameters

* Add sync committee epoch calculations

* Update BUILD.bazel
2021-07-22 16:13:42 +00:00
Nishant Das
33f7582d25 Add Doppelganger E2E (#9239)
* add current fixes

* fix prev release

* Update endtoend/endtoend_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update endtoend/endtoend_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update endtoend/endtoend_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* review comments for radek and raul

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-07-21 23:00:12 -05:00
Raul Jordan
6dadb80cc4 Move v1alpha1 into Prysm API namespace (#9245) 2021-07-21 21:34:07 +00:00
Radosław Kapka
ae140073e7 Remove proposer_sync_aggregate.go (#9231)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-07-21 19:35:18 +00:00
terence tsao
b667d30d3f Feature/UpdateHeadTimely: cache finalized checkpoint when initial sync (#9244) 2021-07-21 11:55:27 -07:00
terence tsao
e7bdf35721 Fix unused variables (#9234)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-21 16:06:57 +00:00
Nishant Das
c780301096 Remove Validator DB Cache (#9238)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-21 15:26:07 +00:00
terence tsao
e1d543a77b Combine function parameters (#9242) 2021-07-21 14:55:37 +00:00
Nishant Das
2ffe8336fc Fix Parsing of Nested Subcommands (#9236) 2021-07-21 13:01:33 +02:00
Nishant Das
11d9e7da9b Update Libp2p to v0.14.4 (#9108)
* fix

* tidy

* fix deps

* tidy

* fix pubsub
2021-07-20 23:50:47 +00:00
Radosław Kapka
53e02b1601 Log correct warning when disabling flags (#9230)
* Log correct warning when disabling flags

* use interface

Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
2021-07-20 16:54:08 +00:00
Preston Van Loon
3a35a953bf proto/prysm/v2: Introduce Altair pb changes (#9222)
* Copy v2 protos from hf1

* Remove some extra Altair naming

* Build with github.com/ferranbt/fastssz/pull/50

* Regen pb.go

* Update fastssz to master
2021-07-19 20:42:06 +00:00
Preston Van Loon
8d1c9fe1e2 epoch/precompute: Add changes from Altair (#9219)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-19 19:39:23 +00:00
terence tsao
4229b466ae Add sync committee pool pkg (#9203)
* Add sync committee pkg

* Fix err variable namings
2021-07-19 19:06:49 +00:00
Preston Van Loon
412ddbb29e beacon-chain/state/v1: ReadOnlyValidator wrapper constructor method (#9221)
* Introduce changes from Altair hf1 branch

* go pkg viz changes

* Fix test

* goimports for some reason

* Use a more safe method of wrapping validators with regards to nil validators. Add basic tests for this wrapped validator

* Use a more safe method of wrapping validators with regards to nil validators. Add basic tests for this wrapped validator

* Panic fixes

* Fix tests

* remove nil validator test as it is no longer possible

* goimports for some reason

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-19 18:26:39 +00:00
Radosław Kapka
8a7010f5aa Implement GetProposerDuties in the validator API (#9223)
* endpoint implementation

* extract common code

* tests

* remove named return values

* correct test log

* sort duties by slot

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-07-19 18:00:15 +00:00
Nishant Das
40a96bc2b6 Invert Update Head Timely Flag (#9224)
* invert

* Update shared/featureconfig/flags.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-19 17:23:43 +00:00
terence tsao
1beb0071b5 State/v2: Add InitializeFromProto and InitializeFromProtoUnsafe (#9226)
* Add `InitializeFromProto`

* Update BUILD.bazel
2021-07-19 16:47:42 +00:00
terence tsao
2a0c4e0d5f Use block wrapper and interface for propose block (#9218) 2021-07-19 15:51:38 +00:00
Nishant Das
dad205bd04 Invert Optimized Balance Update Feature (#9225) 2021-07-19 14:27:20 +00:00
Preston Van Loon
15bfcf8ff6 Add tests for wrapped Altair objects (#9213)
* start tests for wrapped methods

* Add a few tests and skeletons for interface impl methods

* Add latest interfaces update from hf1 and stub the appropriate tests

* Add a few more tests

* More progress

* Migitate nil pointer issues in construct. Complete tests

* Copy changes from hf1 to v1alpha1 wrapper

* Use testutils from #9201

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-18 19:32:52 +00:00
terence tsao
15704053e1 Add Altair configs (#9150)
* Add altair configs

* Update minimal_config.go

* Update workspace and config tests

* Revert workspace changes

* Update WORKSPACE

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-07-18 16:18:43 +00:00
terence tsao
5be1ccd3cf Hydrate altair block (#9201)
* Hydrate altair blocks

* Update BUILD.bazel

* HTR test

* Update BUILD.bazel

* Comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-16 23:35:37 +00:00
terence tsao
97e6eb14e2 Altair block wrapper: update return to interfaces (#9212) 2021-07-16 23:09:45 +00:00
Mohamed Zahoor
6f126c92c0 Make exportdb tool faster (#9184)
* added prodecer consumer pattern for reading and printing

* make state interface visible in exploredb

* bazel magic for visibility

* fix manual visibilithy marking

* linter fix

* parallelized bucket stats

* add log.WithErr and log.Infof

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-07-16 22:24:16 +00:00
Radosław Kapka
223d5309a0 Implement GetAttesterDuties in the validator API (#9207)
* remove irrelevant comment

* fix proto documentation

* first implementation

* register validator server

* first working version

* add sync checker and test file

* first test

* gzl

* final version + tests

* gzl

* run duties through API Middleware

* extract dependent root getter

* dependentRoot docs

* wrap indices in array

* fix static analysis issues

* modify err nil check

* create local variables in slot processing test

* Update proto/eth/v1/validator_service.proto

* review

* simplify index loops

* better calculation of committees at slot

* comment about impossible comndition

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-16 20:55:22 +00:00
Preston Van Loon
31d2482f9c Add proto wrappers for Altair (#9209)
* Starting attestation test

* Commiting checkpoint progress before dinner

* Add remainder of tests

* Copy hf1 cloners and add tests

* Introduce wrapper changes from hf1

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-16 20:30:26 +00:00
terence tsao
3ef0c5d6e5 Fix deepsource: shadow of builtin detected (#9210)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-07-16 20:04:30 +00:00
Preston Van Loon
187a1ca53d shared/copyutil: Add copy methods for Altair (#9208)
* Starting attestation test

* Commiting checkpoint progress before dinner

* Add remainder of tests

* Copy hf1 cloners and add tests

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-07-16 19:30:48 +00:00
Preston Van Loon
17d7fc492e shared/copyutil: Add tests for copy methods (#9204)
* Starting attestation test

* Commiting checkpoint progress before dinner

* Add remainder of tests

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-07-16 16:56:09 +00:00
Preston Van Loon
f029fdd44a beacon-chain/state/stateutil: Fix benchmark target (#9199)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-16 14:39:14 +00:00
Preston Van Loon
44d266313e testutil: Add NotEmpty method (#9202) 2021-07-15 22:32:10 +00:00
terence tsao
1a9a46d042 Block util: block header from block interface (#9196)
* Add beacon block header from block interface

* Update BUILD.bazel

* Update shared/blockutil/block_utils.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Go fmt

* Remove unused bodyroot

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-07-15 16:03:46 +00:00
Preston Van Loon
71c8c75bc3 shared/queue: Add RetrieveByKey method (#9195)
* Checkout shared/queue changes from hf1 branch

* Remove logic that removes and reinserts an item into the queue at the back

* Add test to ensure the correct thing was returned

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-07-15 10:31:19 -05:00
terence tsao
b22935a724 Rename AttestationNotifier (#9197) 2021-07-14 18:32:03 -07:00
Radosław Kapka
3ff5b95945 Remove database field from v1alpha1 validator server (#9191)
* Remove database field from v1alpha1 validator server

* remove field

* gzl

* remove not needed variable

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-14 13:16:21 +00:00
Nishant Das
e6bd5f31b1 Cleanup Fast Aggregate Verify (#9193) 2021-07-14 11:12:50 +00:00
terence tsao
9a4ab41761 BaseReward test only (#9189)
* Move baseward to test only

* Update BUILD.bazel

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-07-13 20:12:53 +00:00
terence tsao
7f0749c398 Clean up shared interface package (#9172)
* Remove shared interface folder

* Rename file names

* Gazelle

* Move interface to /beacon/p2p/

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-07-13 18:58:43 +00:00
Nishant Das
d73ac71ee6 add std out for beacon node (#9183) 2021-07-13 08:45:26 -05:00
terence tsao
6ba741f251 Calculate epoch outside of loop (#9178) 2021-07-12 12:35:36 -07:00
Radosław Kapka
4ebb008bc9 Protos and stubs for validator API endpoints (#9174)
* match proto definitions with spec

* stubs for grpc server's functions
2021-07-12 10:38:55 -05:00
Nishant Das
a9ee3ee06a Update Go-Ethereum (#9157)
* update

* tidy

* fix

* fix e2e

* fix e2e

* fix bad tx opts
2021-07-09 07:58:43 -05:00
Preston Van Loon
7c23d02c3e Remove cluster-pk-manager tool (#9165)
* Remove cluster-pk-manager tool

* Go mod tidy
2021-07-08 13:31:41 -05:00
Preston Van Loon
9dc3dd04c7 Update go-bitfield (#9162) 2021-07-08 08:31:40 -07:00
ahadda5
72886986ea prysm scripts should not save 404 pages (#9072)
* first .sh attempt, checked curl

* removed echo

* more meaningful msg

* prysm.bat changes

* do not save 404, using -f and grep

* removed extra line

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-07-06 23:34:42 +00:00
terence tsao
1d3a9983cc Move block interface next to generated pb (#9146)
* Move block interface next to pb

* Update fuzz build bazel

* Move interface to /proto/interface and wrapper next to generated pb

* Fix fuzz build bazel

* Add //proto/eth/v1alpha1/wrapper

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-06 15:34:05 +00:00
Mohamed Zahoor
cca4a8c821 exploredb tool to display bucket contents (#9116)
* add state bucket content display test case

* save work

* state db disection methods

* fix go.mod

* gazel fix

* pass the testcase if the db is not present. for CI

* get db path from env. optimize code

* update deps.bzl

* go mod tidy

* revery deps.bzl

* moved bucket content inspection to exploredb tool

* satisfy deepsource and tidy go.mod

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-07-06 15:11:11 +00:00
terence tsao
6c8fd745a4 Better import alias names (#9149) 2021-07-06 09:37:33 -05:00
terence tsao
466546bb40 Add sync committee aggregation_bits tags for minimal and mainnet (#9147)
* Add aggregation_bits tags for minimal and mainnet

* Update go-bitfield

* Go mod tidy
2021-07-03 18:59:35 -05:00
terence tsao
1ff4605398 Add priority queue tests (#9142)
* Add priority tests

* Gazelle

* bazel

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-07-02 21:54:28 +00:00
terence tsao
2ae6452cb5 Add Eth2FastAggregateVerify (#9144)
* Add `Eth2FastAggregateVerify`

* Update propose_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-07-02 19:27:08 +00:00
terence tsao
00907fd7d1 Update log test and ChainInfoFetcher interface (#9143) 2021-07-02 14:01:37 -05:00
terence tsao
1edf1f4c87 Restructure RPC packages to be future compatible (#9136)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-02 15:54:52 +00:00
terence tsao
877cb9f6e6 Add priority queue from hf1 (#9138)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-02 15:31:32 +00:00
Nishant Das
5d65ace970 Implement Doppelganger Check (#9120)
* checkpoint changes

* Update beacon-chain/rpc/validator/status.go

Co-authored-by: Potuz <potuz@potuz.net>

* Update beacon-chain/rpc/validator/status.go

Co-authored-by: Potuz <potuz@potuz.net>

* add in client side tests

* add ordering

* add all new test cases

* gate feature

* handle edge case

* add one more test case

* fatal error

* zahoor's review

* Update validator/client/validator.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update validator/client/validator.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* doppelganger not doppleganger

* preston's review

* add in comment

* change comment

* Fix e2e to only run new flags on the current version

* Fix bug where zero byte public keys were always sent in the request when attestation history existed. Still some tests to fix due to another bug in attester protection AttestationHistoryForPubKey.

* go mod tidy, gazelle

* Increase test size

* fix timeout, change size back to small

Co-authored-by: Potuz <potuz@potuz.net>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-07-02 12:11:09 +08:00
Preston Van Loon
2df024afc6 Return pubkey with AttestationHistoryForPubKey AttestationRecord response (#9135)
* Return pubkey with AttestationHistoryForPubKey AttestationRecord response

* Fix tests
2021-07-02 05:50:56 +08:00
Preston Van Loon
ca7b312761 Bazel: set alias targets to manual only (#9134) 2021-07-01 14:48:09 +00:00
Preston Van Loon
d6bd619429 Bazel: minimal test build transitions (#9122)
* Add build transitions for minimal config based tests

* respect existing gotags

* clean up import

* Add some commentary

* gaz

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: rauljordan <raul@prysmaticlabs.com>
2021-06-30 19:22:09 +00:00
Preston Van Loon
24d17a536c Return an error on unrecognized arguments when running default commands in Prysm (#9129)
* Return an error on unrecognized arguments when running default commands in Prysm

* fix cases for subcommands

* Deepsource suggestion
2021-06-30 16:24:04 +00:00
Raul Jordan
b23f63a064 Beacon State V2 Interface Definition With Semantic Version Paths (#9125)
* v2 state initialize and semantic paths

* ensure build

* gaz changes to ignored build files

* gaz
2021-06-30 15:06:19 +00:00
terence tsao
4fb3e05124 Add search for most profitable sync contribution (#9121)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-29 17:26:01 +00:00
terence tsao
cd3a2e87fe Naive aggregation for SyncCommitteeContribution (#9114)
* Copy CopySyncCommitteeContribution

* Update BUILD.bazel

* Add naive aggregation for sync contribution

* Gazelle

* Update deps.bzl

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-06-29 16:11:11 +00:00
Radosław Kapka
0347b4b4af Retrieve validators through v1 API in e2e (#9117)
* Retrieve validators through v1 API in e2e

* fix comment

* simplify boolean comparison
2021-06-29 09:47:59 -05:00
Preston Van Loon
0cfc16ec3b Reorder WORKSPACE go dependency registration (#9113)
* Reorder registration of dependencies so that Prysm go deps are registered first and everything else is registered after.

* Build fixes for gazelle

* Another nofuzz fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-29 00:04:26 +00:00
terence tsao
ea34af4abf Add proposer sync contribution type (#9106)
* Add `proposerSyncContributions`

* Update BUILD.bazel

* Add dedup and tests

* Update BUILD.bazel

* Revert grpc changes
2021-06-28 13:54:33 -05:00
Mohamed Zahoor
0f2e6fb8d5 Tool for Exploring BoltDB in Human-Readable Formats (#9063)
* exploredb tool

* bucket stats tool

* bucket stats tool

* satisfy deepsource

* format error

* add flag to get the db file

* revert go get og humanize

* gazel fix

* fix merge errors

* remove bucket-stats

* satisfy deepsource

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-28 16:59:20 +00:00
Radosław Kapka
43623e1089 Handle comma-separated query parameters in API requests (#9102)
* return empty array when no peers found

* add missing query params to endpoint factory

* fix validator endpoints in endpoint factory

* peers test

* Move API param handling to separate file

* Handle comma-separated query parameters
2021-06-28 15:37:14 +00:00
Radosław Kapka
940ce0c292 Make Eth APIs flag name implementation agnostic (#9112)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-28 15:12:39 +00:00
Radosław Kapka
71f1f19334 Small API Middleware improvements (#9111)
* extract creating DefaultErrorJson objects

* do not use default http client for proxying

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-28 11:36:12 +00:00
Nishant Das
f75548ad1e Protect Against Unsafe Coefficients (#9109) 2021-06-28 11:53:03 +02:00
terence tsao
442f8d1d3c Remove old protos (#9107) 2021-06-28 07:41:37 +00:00
ahadda5
349f832bd3 No length check in AggregatePublicKeys (#9105)
* check of len and nil

* minor edit

* minor edit

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-06-27 16:13:55 +00:00
Raul Jordan
a860648960 Reduce Usage of Eth2 Terminology in Prysm (#9104)
* remove all mentions

* more changes

* folder by folder

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-26 19:00:33 +00:00
Radosław Kapka
dd0ae1bbef Move API param handling to separate file (#9101)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-26 18:42:54 +00:00
Radosław Kapka
9eb3ff6bdf Fix a few bugs in the API (#9099) 2021-06-26 11:17:45 -07:00
Radosław Kapka
b114247836 Move common gateway registration code to new package (#9092)
* Move common gateway registration code to new package

* no shared inside of beaconchain

* beacon gateway

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-25 11:02:11 +00:00
Raul Jordan
8f5c0833ff Block Stream V2 Protos (#9096) 2021-06-25 00:05:38 +00:00
Radosław Kapka
43655e6c04 Remove verbatim strings with flag names on startup (#9093)
* Remove verbatim strings with flag names on startup

* gzl

* fix visibility

* gaz

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-24 17:20:47 +00:00
Radosław Kapka
ff99a2ed40 Run text/event-stream values returned from grpc-gateway through API Middleware (#9080)
* register events handler in v1

* begin `status.Error` messages with uppercase

* something works

* handle errors

* topic comments

* handle the rest of event types

* gzl

* make code more testable + test

* better error handling

* tests

* gzl

* fix deps

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-24 16:18:28 +00:00
Raul Jordan
860716666b Add Altair Protobuf Changes to Prysm (#9087)
* all altair protobuf additions

* add altair changes to deps
2021-06-24 10:32:30 -05:00
Raul Jordan
1e99ec30cb Revert "Improve performance during CheckAttesterDoubleVotes" (#9085)
This reverts commit a958dd246b.
2021-06-23 21:19:22 +00:00
Preston Van Loon
1f6a031630 Add a test for e2e validator to run against prior release (#9042)
* Add a test for e2e validator to run against prior release

* gofmt

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-23 17:07:28 +00:00
Preston Van Loon
942472fc7e Add goimports check (#9075)
* Add goimports check

* lil fixes

* lil fixes

* trying to use someone elses project instead of grinding on it myself
2021-06-22 19:57:26 +00:00
Preston Van Loon
228369e95f Double java heap size for buildkite (#9073)
(cherry picked from commit 9e28840b95cc96620841053902bf22889d69ea1b)
2021-06-22 18:28:34 +00:00
Nishant Das
9ee086feaf Disable No Freelist Sync In Backups (#9069) 2021-06-21 15:14:55 +02:00
Nishant Das
6e42f1bc5d add lock (#9068) 2021-06-21 16:33:06 +08:00
Raul Jordan
4ec0d60911 Prevent Exporting False Positive Slashing Event Due to Old Prysm DB Schema Bug (#9060)
* regression test added

* better comment

* export edge case

* add case for edge test

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-21 05:15:06 +00:00
David Shoots
2583dc1279 Remove noisy / unneeded log.Error() calls (#9066) 2021-06-20 13:31:27 -07:00
Raul Jordan
a747d151ba Fix Small Type Assertion Bug in Slasher (#9061) 2021-06-18 18:50:20 +00:00
Radosław Kapka
4f3c17cf69 Make the shared/gateway package unaware of consuming services (#9050)
* gateway refactor

* test customizations

* change variable names

* fix BUILD file

* remove unused code

* move mux registration out of gateway

* fix gateway tests

* better name for struct

* fix mock endpoint factory

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-18 13:35:13 +00:00
terence tsao
a4be7c4a46 Testutil: Reset deposit cache internally (#9059)
* Reset testutil cache internally

* Fix import

* Fix test to correct error

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-18 04:05:40 +00:00
Alexander Mollohan
ae7e2764e0 Adding ability to override directory permissions for db backup (#8888)
* Adding ability to override directory permissions for db backup

* adding new flag to usage.go

* Backup now has its own directory handling method to reduce footprint of prospective change

* removing unneeded test

* fixing build error

* switching to url param for override

* stripping flag

* adding in unit tests

* fixing a test

* minor changes for testing

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-18 01:37:58 +00:00
Radosław Kapka
fbed11b380 Do not register API Middleware with validator gateway (#9055)
* Do not register API Middleware with validator gateway

* add nil checker

* revert bool

* nil check

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-18 00:38:15 +00:00
terence tsao
4d1b5f42af Better comments for aggregator functions (#9053)
* Better comments for aggregator functions

* Naming returned value
2021-06-17 17:28:19 +00:00
Mohamed Zahoor
e953804239 Process future Blocks and Attestations (#8975)
* queue future blocks

* process early attestations

* fix formatting

* fix testcases

* fix signature testcase and move verify slot time to original place

* Add testcases to test early blocks and slot processing

* added test case to cover failure case while inserting in block pending queue

* satisfy deepsource

* fix review comments

* remove invalid testcase

* do not queue blocks which are within MAXIMUM_GOSSIP_CLOCK_DISPARITY

* format fix

* added a helper to check of the block slot is within MAXIMUM_GOSSIP_CLOCK_DISPARITY

* add helper to check clock disparity and test case for the same

* deepsource fix

* satisfy deepsoruce

* sysc/service.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-17 00:17:52 +00:00
terence tsao
3c0a9455a2 Rename field ops service to att service (#9046)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-16 21:48:56 +00:00
terence tsao
7d4418e2fe Go imports (#9045)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-16 14:28:04 +00:00
Preston Van Loon
9dc4c49fbd Update web-ui to beta.4 (#9047)
* add site data

* gofmt

* fix embed

* fmt

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-16 13:36:43 +00:00
Preston Van Loon
6971245e05 Revert "Invert attest-timely flag" (#9044)
This reverts commit 5fe98ab260.
2021-06-15 20:48:47 +00:00
Radosław Kapka
12403d249f [Feature] - API Middleware (#8926)
* HTTP proxy server for Eth2 APIs (#8904)

* Implement API HTTP proxy server

* cleanup + more comments

* gateway will no longer be dependent on beaconv1

* handle error during ErrorJson type assertion

* simplify handling of endpoint data

* fix mux v1 route

* use URL encoding for all requests

* comment fieldProcessor

* fix failing test

* change proxy port to not interfere with e2e

* gzl

* simplify conditional expression

* Move appending custom error header to grpcutils package

* add api-middleware-port flag

* fix documentation for processField

* modify e2e port

* change field processing error message

* better error message for field processing

* simplify base64ToHexProcessor

* fix json structs

* Run several new endpoints through API middleware (#8922)

* Implement API HTTP proxy server

* cleanup + more comments

* gateway will no longer be dependent on beaconv1

* handle error during ErrorJson type assertion

* simplify handling of endpoint data

* fix mux v1 route

* use URL encoding for all requests

* comment fieldProcessor

* fix failing test

* change proxy port to not interfere with e2e

* gzl

* simplify conditional expression

* Move appending custom error header to grpcutils package

* add api-middleware-port flag

* fix documentation for processField

* modify e2e port

* change field processing error message

* better error message for field processing

* simplify base64ToHexProcessor

* fix json structs

* /eth/v1/beacon/states/{state_id}/validators

* /eth/v1/beacon/states/{state_id}/validators/{validator_id}

* /eth/v1/beacon/states/{state_id}/validator_balances

* /eth/v1/beacon/states/{state_id}/committees

* allow skipping base64-encoding for query params

* /eth/v1/beacon/pool/attestations

* replace break with continue

* Remove unused functions (#8924)

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Process SSZ-serialized beacon state through API middleware (#8925)

* update field names

* Process SSZ-serialized beacon state through API middleware

* revert changes to go.mod and go.sum

* Revert "Merge branch '__develop' into feature/api-middleware"

This reverts commit 7c739a8fd7, reversing
changes made to 2d0f8e012e.

* update ethereumapis

* update validator field name

* update deps.bzl

* update json tags (#8942)

* Run `/node/syncing` through API Middleware (#8944)

* add IsSyncing field to grpc response

* run /node/syncing through the middleware

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Return HTTP status codes other than 200 and 500 from node and debug endpoints (#8937)

* error codes for node endpoints

* error codes for debug endpoints

* better comment about headers

* gzl

* review comments

* comment on return value

* update fakeChecker used for fuzz tests

* fix failing tests

* Allow to pass URL params literally, without encoding to base64 (#8938)

* Allow to pass URL params literally, without encoding to base64

* fix compile error

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Process SSZ-serialized beacon state through API middleware (#8925)

* update field names

* Process SSZ-serialized beacon state through API middleware

* revert changes to go.mod and go.sum

* Revert "Merge branch '__develop' into feature/api-middleware"

This reverts commit 7c739a8fd7, reversing
changes made to 2d0f8e012e.

* update ethereumapis

* update validator field name

* update deps.bzl

* update json tags (#8942)

* Run `/node/syncing` through API Middleware (#8944)

* add IsSyncing field to grpc response

* run /node/syncing through the middleware

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Return HTTP status codes other than 200 and 500 from node and debug endpoints (#8937)

* error codes for node endpoints

* error codes for debug endpoints

* better comment about headers

* gzl

* review comments

* comment on return value

* update fakeChecker used for fuzz tests

* fix failing tests

* Allow to pass URL params literally, without encoding to base64 (#8938)

* Allow to pass URL params literally, without encoding to base64

* fix compile error

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* unused import

* Return correct status codes from beacon endpoints (#8960)

* Various API Middleware fixes (#8963)

* Return correct status codes from `/states` endpoints

* better error messages in debug and node

* better error messages in state

* returning correct error codes from validator endpoints

* correct error codes for getting a block header

* gzl

* fix err variable name

* fix nil block comparison

* test fixes

* make status enum test better

* fix ineffectual assignment

* make PR unstuck

* return proper status codes

* return uppercase keys from /config/spec

* return lowercase validator status

* convert requested enum values to uppercase

* validator fixes

* Implement `/beacon/headers` endpoint (#8966)

* Refactor API Middleware into more manageable code  (#8984)

* move endpoint registration out of shared package

* divide main function into smaller components

* return early on error

* implement hooks

* implement custom handlers and add documentation

* fix test compile error

* restrict package visibility

* remove redundant error checking

* rename file

* API Middleware unit tests (#8998)

* move endpoint registration out of shared package

* divide main function into smaller components

* return early on error

* implement hooks

* implement custom handlers and add documentation

* fix test compile error

* restrict package visibility

* remove redundant error checking

* rename file

* api_middleware_processing

* endpoints

* gzl

* remove gazelle:ignore

* merge

* Implement SSZ version of `/blocks/{block_id}` (#8970)

* Implement SSZ version of `/blocks/{block_id}`

* add dependencies back

* fix indentation in deps.bzl

* parameterize ssz functions

* get block ssz

* update ethereumapis dependency

* gzl

* Do not reuse `Endpoint` structs between API calls (#9007)

* code refactor

* implement endpoint factory

* fix test

* fmt

* include pbs

* gaz

* test naming fixes

* remove unused code

* radek comments

* revert endpoint test

* bring back bytes test case

* move `signedBeaconBlock` to `migration` package

* change `fmt.Errorf` to `errors.Wrap`

* capitalize SSZ

* capitalize URL

* more review feedback

* rename `handleGetBlockSSZ` to `handleGetBeaconBlockSSZ`

* rename `IndexOutOfRangeError` to `ValidatorIndexOutOfRangeError`

* simplify parameter names

* test header

* more corrections

* properly allocate array capacity

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-06-15 10:28:49 -05:00
kasey
19ff25f171 fixes 9029 + a few related client-stats fixes/improvements (#9039)
- fix missing scrape logic for these metrics:
      - sync_eth1_connected
      - sync_eth1_fallback_configured
      - sync_eth1_fallback_connected
    - update readme to add more detailed source info
      and indicate unsupported metrics.
    - fix documentation bug in prometheus help text
      for powchain_sync_eth1_connected

Co-authored-by: kasey <kasey@users.noreply.github.com>
2021-06-14 23:59:15 +00:00
Raul Jordan
4551bb47fc Add Events Streaming Endpoint for Eth2 Standard API (#9001)
* begin event source support

* add in events server and proto definitions

* fmt

* add in stream support and proper building

* add in the protos

* tidy

* add structure and comments

* fmt

* revert proto changes

* only return events we explicitly request

* capture missing events needed for eth2 apis

* handle underflow

* no hash tree roots

* gazelle

* static

* race tests passing

* simplify

* handle all required events

* begin on stream tests

* correctness and abstracting method for forwarding to state feed

* genesis root

* include missing pieces

* logging

* rem and and handle return

* added mock

* add in mock

* add tests for block events

* added comprehensive tests

* radek comments

* add missing unit tests

* updatemockgen file

* viz

* done

* Update beacon-chain/rpc/eventsv1/events.go

* Update beacon-chain/rpc/eventsv1/events.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-14 16:58:08 +00:00
Radosław Kapka
77bf4bd9f0 Create ValidatorIndexOutOfRangeError (#9035) 2021-06-14 16:38:48 +00:00
terence tsao
dfd2cc751c Refactor helpers for Altair (#9026)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-11 19:29:50 +00:00
Nishant Das
fb08014146 Add Better Zero Key Checks (#9022)
* add better checks

* preston's review
2021-06-11 13:48:57 -05:00
Preston Van Loon
6981b16ecd RPC: Surface underlying error when failing to return a state. (#9024) 2021-06-11 13:50:02 +00:00
Preston Van Loon
71fd747dca Proto mirror script fix (#9020)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-11 03:05:50 +00:00
Nishant Das
7a6ce27497 Change To LRU from FIFO Cache (#8996)
* change to lru

* fix tests

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-10 21:53:34 +00:00
Raul Jordan
6eb006175a Capture ChainReorg, FinalizedCheckpoint, and NewHead Events in Beacon Node (#9011)
* capture missing events needed for eth2 apis

* handle underflow

* no hash tree roots

* gazelle

* static

* race tests passing

* simplify

* correctness and abstracting method for forwarding to state feed

* genesis root

* add in slotutil
2021-06-10 16:13:15 -05:00
Preston Van Loon
8f90e91e99 Update jwt-go library to fix CVE-2020-26160 (#9017)
* Replace github.com/dgrijalva/jwt-go with github.com/form3tech-oss/jwt-go, including security fix

* Fix to include go cast in deps.bzl and go mod

* Add gocast stub for go.mod / deps.bzl trick

* revert some changes

* Remove ignore tag

* gaz

* go mod tidy

* add build exclusion

* Go mod?

* go mod tidy
2021-06-10 13:59:34 -05:00
Alexander Mollohan
644d5bbb1a Graffiti hex support (#8894)
* more or less feature complete, needs unit tests

* Test support and concomitant refactors

* Resolving requested changes on PR

* minor fix before trying to figure out bazel

* bazel change

* minor changes requested in pr

* fixing build errors

* fixing build errors

* fixing PR feedback

* fixing PR feedback

* resolving nitpicks

* resolving nitpicks

* build failures REEEEE

* tests fail if its not this way, so this way it is

* getting the tests to pass

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-06-10 16:54:24 +00:00
Preston Van Loon
0d8dd8d280 Update mirror-ethereumapis.sh to include committer name / email (#9009)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-09 17:09:09 +00:00
Raul Jordan
bb6f00fd87 New Proto Messages Required for Event Streams (#9010)
* add event messages

* define all required event messages

* add new proto event messages

* formatting
2021-06-09 16:49:03 +00:00
Preston Van Loon
f2768c1414 Client-stats: Fail gracefully when metrics are missing (#8976)
* Client-stauts: Fail gracefully when metrics are missing

* gazelle
2021-06-07 20:37:14 +00:00
Raul Jordan
a81c4ed6a0 Protobuf Changes for Eth2 APIs EventSource Support (#8999)
* protobuf changes for event source support

* deps update

* tidy

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-07 19:29:36 +00:00
Preston Van Loon
26c4e7b527 Update prysm.sh to support Mac M1 chip (#9000)
Updated for Apple M1 users.

Co-authored-by: mrabino1 <mrabino1@gmail.com>
2021-06-07 18:51:05 +00:00
terence tsao
aa4c21d6df Run go imports (#8995) 2021-06-06 07:39:16 +02:00
Preston Van Loon
833ac0092c Fix repo name in mirror script (#8992) 2021-06-04 22:50:41 +00:00
Raul Jordan
2ee5bd12cf Improve Mirror Bash Script for EthereumAPIs (#8991)
* amend mirror script

* bash lint
2021-06-04 16:15:19 +00:00
Preston Van Loon
d4aa4bc258 Remove unused ClosestPowerOf2 method (#8990) 2021-06-04 15:03:18 +00:00
Radosław Kapka
1ea042dd6e Do not use proto casing in v1alpha1 API JSON fields (#8988) 2021-06-04 16:03:40 +08:00
Raul Jordan
dd10a3ab5d Fix EthereumAPIs Mirror Script (#8986) 2021-06-03 21:30:24 +00:00
Raul Jordan
8d428606a2 Mirror EthereumAPIs Bash Script (#8980)
* mirror script

* revert

* add in final script

* comments
2021-06-03 21:05:33 +00:00
terence tsao
8433a313f0 Enhance att sig set checks (#8978)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-03 15:30:46 +08:00
Preston Van Loon
693fdf87bb Add clientstats spec as a README.md (#8977)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-03 00:13:31 +00:00
Raul Jordan
5aac06f04e Move EthereumAPIs Into Prysm (#8968)
* begin move

* use same import path

* imports

* regen protos

* regen

* no rename

* generate ssz

* gaz

* fmt

* edit build file

* imports

* modify

* remove generated files

* remove protos

* edit imports in prysm

* beacon chain all builds

* edit script

* add generated pbs

* add replace rules

* license for ethereumapis protos

* change visibility

* fmt

* update build files to gaz ignore

* use proper form

* edit imports

* wrap block

* revert scripts

* revert go mod
2021-06-02 18:49:52 -05:00
Raul Jordan
1d2fc60ba7 Checkin Generated Protobufs and Proto Files for EthereumAPIs (#8973)
* generated

* add ignore

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-02 20:20:11 +00:00
kasey
2b6c7e7b37 use urfave defaulting method for correct help text (#8972)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-06-02 19:47:47 +00:00
Preston Van Loon
fc898d541f Update prysm scripts to support client-stats, deprecate slasher (#8971) 2021-06-02 14:06:59 -05:00
Nishant Das
91bd477f4f Add Metadata V2 Object and Interface (#8962)
* checkpoint changes

* gaz

* add error check

* gaz

* fix tests

* fix tests

* terence's review

* terence's review

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-02 13:44:34 +08:00
Preston Van Loon
ef20f2da7e Altair: new protos/state object (#8907)
* Altair: new protos/state

* Undo changes in proto/migration/migration.go

* Change proto/beacon/p2p/v1/generated.ssz.go to 0644

* Update proto/beacon/p2p/v1/types.proto

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* PR Feedback: rename SyncCommitteeSigningData to SyncAggregatorSelectionData

* 644

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-02 03:07:13 +00:00
Nishant Das
8a6e2a33a2 Fix Interop Mode (#8961) 2021-06-02 02:45:10 +00:00
aliask
eb1d122aec Feature/debian packages (#8872)
* Basic build targets for debian packages

* Add comments to config files

* Harden beacon-chain systemd service

* Add install scripts to set users and folders up

* Rename bazel target, fix service files

* Extra service hardening, cleanup install scripts

* Fix linting issues

* //shared:version_file run local only

* gazelle

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-06-01 15:44:24 +00:00
terence tsao
638e76cbd7 Sort imports (#8959) 2021-05-31 20:47:30 +02:00
Ivan Martinez
bb8625ede5 Remove stack trace from eth1 connection errors (#8957) 2021-05-28 21:25:54 +00:00
Ivan Martinez
40b4079924 Update deprecated function usages (#8953)
* Clear deprecated usages within Prysm

* Remove deprecatd usages where possible

* Update
2021-05-28 08:57:47 +02:00
Nishant Das
6bb3cc45ef Add Nested Nil Checks To Wrapper (#8951)
* add checks

* fix
2021-05-27 16:07:51 +00:00
Ivan Martinez
e97f6cfdbf ETH2 APIs: Handle invalid validator status as error (#8946)
* Handle invalid validator status as error

* Change text

* Fix review comments

* Remove nil status check

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-05-27 06:44:36 +00:00
terence tsao
d5f4385525 Copy to return interface (#8950) 2021-05-26 17:54:06 -07:00
kasey
8094c1c4c5 regenerate ssz types w/ variable-len bounds checks (#8936)
* regenerate ssz types w/ variable-len bounds checks

using the code in this PR:
https://github.com/ferranbt/fastssz/pull/45

* updating with improved bounds checking

* updating pinned fastssz dependency to newest

* removing redundant higher bounds checks

* regenerate again w/ new fastssz
2021-05-26 16:47:06 -05:00
terence tsao
276d03553c Block interface clean up (#8947) 2021-05-26 11:33:46 -07:00
Nishant Das
caf9bdbc6f Use Block Interface Across Prysm (#8918)
* commit initial work

* checkpoint current work

* gaz

* checkpoint

* req/resp changes

* initial-sync

* finally works

* fix error

* fix bugs

* fix issue

* fix issues

* fix refs

* tests

* more text fixes

* more text fixes

* more text fixes

* fix tests

* fix tests

* tests

* finally fix builds

* finally

* comments

* fix fuzz

* share common library

* fix

* fix

* add in more defensive nil checks

* add in more defensive nil checks

* imports

* Apply suggestions from code review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Apply suggestions from code review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Update shared/interfaces/block_interface.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Update shared/interfaces/block_wrapper.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Update shared/interfaces/block_interface.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* imports

* fix bad changes

* fix

* terence's review

* terence's review

* fmt

* Update beacon-chain/rpc/beacon/blocks.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* fix tests

* fix

* fix all tests

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-05-26 16:19:54 +00:00
Preston Van Loon
7b55213e6d Refactor attestationDelta to reuse the sqrt value of active current epoch (#8945)
* Refactor attestationDelta to reuse the sqrt value of active current epoch

* use baseRewardsPerEpoch
2021-05-26 09:37:35 -05:00
Ivan Martinez
d5ad15ed80 ETH2 APIs: Implement Validator Status support (#8940)
* Update protos for new changes

* Begin implementing validator status

* Add support for validator status to state/validators

* Fix nil pointer
2021-05-26 00:24:29 -04:00
terence tsao
2e322bdf6d Add missing finality spec tests (#8939)
* Remove unused functions

* Add finality spec tests

* Gazelle

* Add minimal tag to BUILD.bazel
2021-05-26 08:18:51 +08:00
terence tsao
5fe98ab260 Invert attest-timely flag (#8827) 2021-05-25 07:22:12 -07:00
Zahoor Mohamed
a958dd246b Improve performance during CheckAttesterDoubleVotes (#8927)
* de-normalize attestationDataRootsBucket to improve performace in CheckAttesterDoubleVotes

* parallize processing of every attestation

* fix double vote test case to take care of jumbled double votes due to parallel processing

* remove attestationRecordsBucket totally and take care of pruning

* missed out build file

* fix review comments

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-05-25 13:02:18 +05:30
Ivan Martinez
aa0fb058c6 ETH2 APIs: Update protos for new ethapis changes (#8920)
* Update protos for new changes

* Fix

* Fix deps

* Fix build

* Include newer changes to ethapis

* Fix

* Fix import

* Do not return unconnected peers
2021-05-24 20:02:01 -05:00
Preston Van Loon
0cbdce6bd6 Remove unused sszgen dependency (#8934)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-05-24 17:26:57 +00:00
Radosław Kapka
5ae6e83c7f Update deps.bzl after updating grpc-gateway dependency (#8933) 2021-05-24 16:18:26 +00:00
Radosław Kapka
180b8447f7 Update grpc-gateway dependency (#8929) 2021-05-24 09:27:50 -05:00
Nishant Das
503ec126e1 Fix Panic When Accessing State Frequently (#8928)
* fix issue

* dont allocate for existence checks

* radek's review
2021-05-24 06:52:33 +00:00
Nishant Das
76cdbaca22 Interface Analyzer (#8923) 2021-05-23 21:55:42 -07:00
terence tsao
09d7efe964 Remove unused functions (#8924) 2021-05-23 09:36:46 +02:00
Preston Van Loon
7dc3d7fc60 Write generated ssz files as non-executable 644 (#8909)
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-05-21 16:27:33 +00:00
Preston Van Loon
59897b78ff IntegerSquareRoot improvements (#8910)
* Add slower, but more precise sqrt method

* nofuzz
2021-05-21 11:02:16 -05:00
Ivan Martinez
f4d6e91e72 ETH2 APIs: Add spans and remove unused Server fields (#8916)
* Add spans and remove unused Server fields

* Remove more
2021-05-20 14:56:53 -05:00
Ivan Martinez
213369f5d0 Fix testing for attestation pools endpoint (#8914)
* Fix testing for attestation pools endpoint

* Remove
2021-05-20 18:59:25 +00:00
Ivan Martinez
8aa909de2d ETH2 API: Implement v1/state/validator endpoints (#8896)
* Implement validator endpoints

* Import

* Fix errors

* Add comments and fix naming :wq

* Fix functions to access state once

* Optimization for slots

* Test more

* Fix

* test listing committees for specific epoch

* Fix error retrurn

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: rkapka <rkapka@wp.pl>
2021-05-20 07:41:39 +00:00
Preston Van Loon
36e9edd654 Minor changes from hf1 branch (#8906)
* Copy new slot helpers from hf1 branch

* Make HandleEth1DataSlice public for altair
2021-05-19 16:11:48 +00:00
Ivan Martinez
9db4eba72b ETH2 API: Implement ListPoolAttestations endpoint (#8902)
* Implement ListPoolAttestations

* Spacing

* add lil doc

* Fix testing and rework logic
2021-05-19 09:55:24 -05:00
Radosław Kapka
fe440bc0eb Copy slice when exiting all keys with --exit-all (#8897)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-05-18 17:17:42 +00:00
terence tsao
f68e0846ba Ran go imports (#8898) 2021-05-18 16:11:02 +00:00
terence tsao
f2a2d5d43c Fix typo (#8899) 2021-05-18 08:33:52 -07:00
Preston Van Loon
f2fe1f7683 Use ExecuteStateTransition in onBlock (#8865)
* Refactor ExecuteStateTransition to use ExecuteStateTransitionNoVerifyAnySig and then verify the signatures. This reduces duplicated code paths.

* Use refactored method from PR #8864

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-05-17 19:52:15 +00:00
Nishant Das
35dc8c31fa Add in Context For Req/Resp (#8895)
* Add in context

* use better function signatures

* add in chainservice

* fix rpc requests

* add in test

* fix tests

* add in comments

* Update beacon-chain/sync/rpc_chunked_response.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Update beacon-chain/sync/rpc_send_request.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* rename files

* fmt

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-05-17 19:25:59 +00:00
Radosław Kapka
519e1e4f4a Decode peer ID properly when fetching peer (#8883)
# Conflicts:
#	beacon-chain/rpc/nodev1/node_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-05-17 18:58:13 +00:00
Ivan Martinez
2f10b1c7b1 Change gogoproto compiler to protoc-gen-go-cast (#8697)
* Remove gogoproto compiler

* Remove more gogoproto

* Improvements

* Fix gengo

* More scripts

* Gazelle, fix deps

* Fix version and errors

* Fix gocast for arrays

* Fix ethapis

* Fixes

* Fix compile errors

* fix go.mod

* //proto/... builds

* Update for protov2

* temp fix compilation to move on

* Change everything to emptypb.empty

* Add grpc to proto/slashings

* Fix almost all build failures

* Oher build problems

* FIX THIS FUCKING THING

* gaz literally every .bazel

* Final touches

* Final final touches

* Fix proto

* Begin moving proto.Marshal to native

* Fix site_data

* Fixes

* Fix duplicate gateway

* Fix gateway target

* Fix ethapis

* Fixes from review

* Update

* Fix

* Fix status test

* Fix fuzz

* Add isprotoslice to fun

* Change DeepEqual to DeepSSZEqual for proto arrays

* Fix build

* Fix gaz

* Update go

* Fixes

* Fixes

* Add case for nil validators after copy

* Fix cast

* Fix test

* Fix imports

* Go mod

* Only use extension where needed

* Fixes

* Split gateway from gengo

* gaz

* go mod

* Add back hydrated state

* fix hydrate

* Fix proto.clone

* Fies

* Revert "Split gateway from gengo"

This reverts commit 7298bb2054.

* Revert "gaz"

This reverts commit ca95256570.

* Merge all gateway into one target

* go mod

* Gaz

* Add generate v1_gateway files

* run pb again

* goimports

* gaz

* Fix comments

* Fix protos

* Fix PR

* Fix protos

* Update grpc-gateway and ethapis

* Update ethapis and gen-go-cast

* Go tidy

* Reorder

* Fix ethapis

* fix spec tests

* Fix script

* Remove unused import

* Fix fuzz

* Fix gomod

* Update version

* Error if the cloned result is nil

* Handle optional slots

* ADd more empty checks to clone

* Undo fuzz changes

* Fix build.bazel

* Gaz

* Redo fuzz changes

* Undo some eth1data changes

* Update go.mod

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Undo clone beacon state

* Remove gogo proto more and unused v1_gateway

* Add manual fix for nil vals

* Fix gaz

* tidy

* Tidy again

* Add detailed error

* Revert "Add detailed error"

This reverts commit 59bc053dcd.

* Undo varint changes

* Fix nil validators in deposit test

* Commit

* Undo

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-05-17 18:32:04 +00:00
Radosław Kapka
e254e8edcc Simplify handling hex state ID (#8890)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-05-17 09:56:00 +02:00
Nishant Das
e6a1d5b1b9 Refactor RPC Topic Construction (#8892)
* check in changes

* goimports

* export it

* add new rpc type
2021-05-15 08:58:56 +08:00
terence tsao
bc27a73600 Various code cleanups (#8889) 2021-05-13 08:29:14 +02:00
Nishant Das
4ef5c9ad15 add it in (#8885) 2021-05-12 15:59:26 +08:00
Radosław Kapka
ddab82e52a Handle undiscovered peers when listing peers (#8882)
* Handle undiscovered peers when listing peers

* off by 1 fix
2021-05-11 19:11:34 +02:00
Radosław Kapka
8fe26e7c84 Serialize nil ENR record (#8881) 2021-05-11 08:31:43 +00:00
Radosław Kapka
f97c7ee1da Initialize VoluntaryExitsPool and BlockReceiver in rpc server (#8876)
* Initialize `VoluntaryExitsPool` in rpc server

* initialize `BlockReceiver`
2021-05-11 07:16:13 +00:00
terence tsao
ad4fadb36a Minor clean up on mixing length root (#8875)
* Minor clean up on mixing length root

* Update root size to 32

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-05-10 22:39:53 +00:00
3058 changed files with 255294 additions and 147465 deletions

View File

@@ -21,34 +21,24 @@ run --host_force_python=PY2
build --sandbox_default_allow_network=false
# Stamp binaries with git information
build --workspace_status_command=./scripts/workspace_status.sh
build --workspace_status_command=./hack/workspace_status.sh
build --stamp
# Use mainnet protobufs at runtime
run --define ssz=mainnet
test --define ssz=mainnet
build --define ssz=mainnet
# Prevent PATH changes from rebuilding when switching from IDE to command line.
build --incompatible_strict_action_env
test --incompatible_strict_action_env
run --incompatible_strict_action_env
# Disable kafka by default, it takes a long time to build...
build --define kafka_enabled=false
test --define kafka_enabled=false
run --define kafka_enabled=false
build --define blst_disabled=false
test --define blst_disabled=false
run --define blst_disabled=false
build:kafka_enabled --define kafka_enabled=true
build:kafka_enabled --define gotags=kafka_enabled
build:blst_disabled --define blst_disabled=true
build:blst_disabled --define gotags=blst_disabled
build:minimal --//proto:network=minimal
build:minimal --@io_bazel_rules_go//go/config:tags=minimal
# Release flags
build:release --compilation_mode=opt
build:release --config=llvm
@@ -72,20 +62,7 @@ build:llvm-asan --config=llvm
build:llvm-asan --config=asan
build:llvm-asan --linkopt -fuse-ld=ld.lld
build:fuzz --define=gotags=libfuzzer
build:fuzz --config=llvm-asan
build:fuzz --copt=-fsanitize=fuzzer-no-link
build:fuzz --linkopt=-fsanitize=fuzzer
build:fuzz --copt=-fno-omit-frame-pointer
build:fuzz --define=FUZZING_ENGINE=libfuzzer
build:fuzz --copt=-DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
build:fuzz --linkopt -Wl,--no-as-needed
build:fuzz --define=gc_goopts=-d=libfuzzer,checkptr
build:fuzz --run_under=//tools:fuzz_wrapper
build:fuzz --compilation_mode=opt
build:fuzz --define=blst_disabled=true
test:fuzz --local_test_jobs="HOST_CPUS*.5"
build:fuzz --@io_bazel_rules_go//go/config:tags=fuzz
# Build binary with cgo symbolizer for debugging / profiling.
build:cgo_symbolizer --config=llvm

View File

@@ -1 +1 @@
3.7.0
4.2.2

View File

@@ -10,7 +10,7 @@
# Prysm specific remote-cache properties.
#build:remote-cache --disk_cache=
build:remote-cache --remote_download_minimal
build:remote-cache --remote_download_toplevel
build:remote-cache --remote_cache=grpc://bazel-remote-cache:9092
build:remote-cache --experimental_remote_downloader=grpc://bazel-remote-cache:9092
build:remote-cache --remote_local_fallback
@@ -22,7 +22,7 @@ build:remote-cache --incompatible_strict_action_env=true
# Import workspace options.
import %workspace%/.bazelrc
startup --host_jvm_args=-Xmx1000m --host_jvm_args=-Xms1000m
startup --host_jvm_args=-Xmx2g --host_jvm_args=-Xms2g
query --repository_cache=/tmp/repositorycache
query --experimental_repository_cache_hardlinks
build --repository_cache=/tmp/repositorycache
@@ -41,5 +41,6 @@ build --flaky_test_attempts=5
# Disabled race detection due to unstable test results under constrained environment build kite
# build --features=race
# Disable flaky test detection for fuzzing.
test:fuzz --flaky_test_attempts=1
# Better caching
build:nostamp --nostamp
build:nostamp --workspace_status_command=./hack/workspace_status_ci.sh

8
.github/CODEOWNERS vendored
View File

@@ -6,3 +6,11 @@
# Anyone on prylabs team can approve dependency updates.
deps.bzl @prysmaticlabs/core-team
# Radek and Nishant are responsible for changes that can affect the native state feature.
# See https://www.notion.so/prysmaticlabs/Native-Beacon-State-Redesign-6cc9744b4ec1439bb34fa829b36a35c1
/beacon-chain/state/fieldtrie/ @rkapka @nisdas
/beacon-chain/state/v1/ @rkapka @nisdas
/beacon-chain/state/v2/ @rkapka @nisdas
/beacon-chain/state/v3/ @rkapka @nisdas
/beacon-chain/state/state-native/ @rkapka @nisdas

View File

@@ -25,6 +25,22 @@ jobs:
with:
path: ./
- name: GoImports checker
id: goimports
uses: Jerome1337/goimports-action@v1.0.2
with:
goimports-path: ./
- name: Gosec security scanner
uses: securego/gosec@master
with:
args: '-exclude=G307 -exclude-dir=crypto/bls/herumi ./...'
- name: Golangci-lint
uses: golangci/golangci-lint-action@v2
with:
args: --print-issued-lines --sort-results --no-config --timeout=10m --disable-all -E deadcode -E errcheck -E gosimple --skip-files=validator/web/site_data.go --skip-dirs=proto
build:
name: Build
runs-on: ubuntu-latest

4
.gitignore vendored
View File

@@ -30,9 +30,5 @@ password.txt
# Dist files
dist
# libfuzzer
oom-*
crash-*
# deepsource cli
bin

View File

@@ -5,21 +5,22 @@ Contact: mailto:security@prysmaticlabs.com
Encryption: openpgp4fpr:0AE0051D647BA3C1A917AF4072E33E4DF1A5036E
Encryption: openpgp4fpr:CD08DE68C60B82D3EE2A3F7D95452A701810FEDB
Encryption: openpgp4fpr:317D6E91058F8F3C2303BA7756313E44581297A6
Encryption: openpgp4fpr:79C59A585E3FD3AFFA00F5C22940A6479DA7C9EC
Preferred-Languages: en
Canonical: https://github.com/prysmaticlabs/prysm/tree/master/.well-known/security.txt
-----BEGIN PGP SIGNATURE-----
iQIzBAEBCgAdFiEECuAFHWR7o8GpF69AcuM+TfGlA24FAl++klgACgkQcuM+TfGl
A27rQw/6A29p1W20J0v+h218p8XWLSUpTIGLnZTxw6KqdyVXMzlsQK0YG4G2s2AB
0LKh7Ae/Di5E0U+Z4AjUW5nc5eaCxK36GMscH9Ah0rgJwNYxEJw7/2o8ZqVT/Ip2
+56rFihRqxFZfaCNKFVuZFaL9jKewV9FKYP38ID6/SnTcrOHiu2AoAlyZGmB03p+
iT57SPRHatygeY4xb/gwcfREFWEv+VHGyBTv8A+6ABZDxyurboCFMERHzFICrbmk
8UdHxxlWZDnHAbAUyAwpERC5znx6IHXQJwF8TMtu6XY6a6axT2XBOyJDF9/mZOz+
kdkz6loX5uxaQBGLtTv6Kqf1yUGANOZ16VhHvWwL209LmHmigIVQ+qSM6c79PsW/
vrsqdz3GBsiMC5Fq2vYgnbgzpfE8Atjn0y7E+j4R7IvwOAE/Ro/b++nqnc4YqhME
P/yTcfGftaCrdSNnQCXeoV9JxpFM5Xy8KV3eexvNKbcgA/9DtgxL5i+s5ZJkUT9A
+qJvoRrRyIym32ghkHgtFJKB3PLCdobeoOVRk6EnMo9zKSiSK2rZEJW8Ccbo515D
W9qUOn3GF7lNVuUFAU/YKEdmDp/AVaViZ7vH+8aq0LC0HBkZ8XlzWnWoArS8sMhw
fX0R9g/HMgrwNte/d0mwim5lJ2Plgv60Bh4grJqwZJeWbU0zi1U=
=uW+X
iQIzBAEBCgAdFiEECuAFHWR7o8GpF69AcuM+TfGlA24FAmGOfiYACgkQcuM+TfGl
A24YwRAAiQk3w6yzqSEggrOlNoNn04iu/rWZdn5ihkQgzACXy8XH2D1gdKLChE/X
7e5bUtgE2aCuHryQjwoKxqZakviBJFstVmHgF64rXv2zKhpqA30Mj4fI+T3zn8I+
+FpFV0TTsxNLDx+AcR1eQ1nSayO7ImUDIfOQNDDnSZZy42Bc+F+QIGKB3aH/8bpG
kT+bDTZrXvX+TE1gZTbAtZG8sH8g/zadoWEHIhfXUuYb0kTz+DRzAxoqU4j4Z4ee
1zSfFAgfJwxJP4kWD7s4xkE1sBbCgGBeD6cW/C2lbcfIei+XSizLpHW3jD9dNqh4
fLkmEspSa/LV/iXFq8nFzu/GLww4q+sQZDzzDKZyws54CrATinRitZMhzoIL0bTn
yFZVOGHosFAMEVZ36dl1Aw2+B2W6tr2CVr9c5zfV+kup5/KZH1EmT5nYY/zFwfg2
jYCFB5wmYeiyWZvuprgJXRArgVZLZaJxwWazlPVk4i/4vPvRgvfHqOwHCBe8DXy0
VHPhpewwb/ECYek1KoaNQflgR8iH2GMHkC5RjhGDAt1S0AQDtite5m4ZYt1kvO9E
k/znkv89dduhL9CKDvZvnI+DICwsTrf//4KJ8PM/qaPAJa4GvtiUU/eS/jKBivtv
OP5dZQtX6KPc9ewqqZgn622uHSezoBidgeTkdZsJ6tw2eIu0lsY=
=V7L0
-----END PGP SIGNATURE-----

View File

@@ -14,6 +14,7 @@ exports_files([
# gazelle:prefix github.com/prysmaticlabs/prysm
# gazelle:map_kind go_library go_library @prysm//tools/go:def.bzl
# gazelle:map_kind go_test go_test @prysm//tools/go:def.bzl
# gazelle:map_kind go_repository go_repository @prysm//tools/go:def.bzl
gazelle(
name = "gazelle",
@@ -23,7 +24,16 @@ gazelle(
# Protobuf compiler (non-gRPC)
alias(
name = "proto_compiler",
actual = "@io_bazel_rules_go//proto:gogofast_proto",
actual = "@io_bazel_rules_go//proto:go_proto",
visibility = [
"//proto:__subpackages__",
],
)
# Cast protobuf compiler (non-gRPC)
alias(
name = "cast_proto_compiler",
actual = "@com_github_prysmaticlabs_protoc_gen_go_cast//:go_cast",
visibility = [
"//proto:__subpackages__",
],
@@ -32,21 +42,21 @@ alias(
# Protobuf compiler (gRPC)
alias(
name = "grpc_proto_compiler",
actual = "@io_bazel_rules_go//proto:gogofast_grpc",
actual = "@io_bazel_rules_go//proto:go_grpc",
visibility = ["//visibility:public"],
)
# Protobuf gRPC compiler without gogoproto. Required for gRPC gateway.
# Cast protobuf compiler (gRPC)
alias(
name = "grpc_nogogo_proto_compiler",
actual = "@io_bazel_rules_go//proto:go_grpc",
name = "cast_grpc_proto_compiler",
actual = "@com_github_prysmaticlabs_protoc_gen_go_cast//:go_cast_grpc",
visibility = ["//visibility:public"],
)
# Protobuf gRPC gateway compiler
alias(
name = "grpc_gateway_proto_compiler",
actual = "@com_github_grpc_ecosystem_grpc_gateway//protoc-gen-grpc-gateway:go_gen_grpc_gateway",
actual = "@com_github_grpc_ecosystem_grpc_gateway_v2//protoc-gen-grpc-gateway:go_gen_grpc_gateway",
visibility = ["//visibility:public"],
)
@@ -76,49 +86,51 @@ nogo(
config = "nogo_config.json",
visibility = ["//visibility:public"],
deps = [
"@org_golang_x_tools//go/analysis/passes/unsafeptr:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/unreachable:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/unmarshal:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/tests:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/structtag:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/stdmethods:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/shift:go_tool_library",
# "@org_golang_x_tools//go/analysis/passes/shadow:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/printf:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/pkgfact:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/nilness:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/nilfunc:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/loopclosure:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/httpresponse:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/findcall:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/deepequalerrors:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/ctrlflow:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/copylock:go_tool_library",
# "@org_golang_x_tools//go/analysis/passes/cgocall:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/buildtag:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/buildssa:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/bools:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/atomicalign:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/atomic:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/assign:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/inspect:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/asmdecl:go_tool_library",
"//tools/analyzers/maligned:go_tool_library",
"//tools/analyzers/cryptorand:go_tool_library",
"//tools/analyzers/errcheck:go_tool_library",
"//tools/analyzers/featureconfig:go_tool_library",
"//tools/analyzers/comparesame:go_tool_library",
"//tools/analyzers/shadowpredecl:go_tool_library",
"//tools/analyzers/nop:go_tool_library",
"//tools/analyzers/slicedirect:go_tool_library",
"//tools/analyzers/ineffassign:go_tool_library",
"//tools/analyzers/properpermissions:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/unsafeptr:go_default_library",
"@org_golang_x_tools//go/analysis/passes/unreachable:go_default_library",
"@org_golang_x_tools//go/analysis/passes/unmarshal:go_default_library",
"@org_golang_x_tools//go/analysis/passes/tests:go_default_library",
"@org_golang_x_tools//go/analysis/passes/structtag:go_default_library",
"@org_golang_x_tools//go/analysis/passes/stdmethods:go_default_library",
"@org_golang_x_tools//go/analysis/passes/shift:go_default_library",
# "@org_golang_x_tools//go/analysis/passes/shadow:go_default_library",
"@org_golang_x_tools//go/analysis/passes/printf:go_default_library",
"@org_golang_x_tools//go/analysis/passes/pkgfact:go_default_library",
"@org_golang_x_tools//go/analysis/passes/nilness:go_default_library",
"@org_golang_x_tools//go/analysis/passes/nilfunc:go_default_library",
"@org_golang_x_tools//go/analysis/passes/loopclosure:go_default_library",
"@org_golang_x_tools//go/analysis/passes/httpresponse:go_default_library",
"@org_golang_x_tools//go/analysis/passes/findcall:go_default_library",
"@org_golang_x_tools//go/analysis/passes/deepequalerrors:go_default_library",
"@org_golang_x_tools//go/analysis/passes/ctrlflow:go_default_library",
"@org_golang_x_tools//go/analysis/passes/copylock:go_default_library",
# "@org_golang_x_tools//go/analysis/passes/cgocall:go_default_library",
"@org_golang_x_tools//go/analysis/passes/buildtag:go_default_library",
"@org_golang_x_tools//go/analysis/passes/buildssa:go_default_library",
"@org_golang_x_tools//go/analysis/passes/bools:go_default_library",
"@org_golang_x_tools//go/analysis/passes/atomicalign:go_default_library",
"@org_golang_x_tools//go/analysis/passes/atomic:go_default_library",
"@org_golang_x_tools//go/analysis/passes/assign:go_default_library",
"@org_golang_x_tools//go/analysis/passes/inspect:go_default_library",
"@org_golang_x_tools//go/analysis/passes/asmdecl:go_default_library",
"//tools/analyzers/maligned:go_default_library",
"//tools/analyzers/cryptorand:go_default_library",
"//tools/analyzers/errcheck:go_default_library",
"//tools/analyzers/featureconfig:go_default_library",
"//tools/analyzers/comparesame:go_default_library",
"//tools/analyzers/shadowpredecl:go_default_library",
"//tools/analyzers/nop:go_default_library",
"//tools/analyzers/slicedirect:go_default_library",
"//tools/analyzers/interfacechecker:go_default_library",
"//tools/analyzers/ineffassign:go_default_library",
"//tools/analyzers/properpermissions:go_default_library",
"//tools/analyzers/recursivelock:go_default_library",
] + select({
# nogo checks that fail with coverage enabled.
":coverage_enabled": [],
"//conditions:default": [
"@org_golang_x_tools//go/analysis/passes/lostcancel:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/composite:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/lostcancel:go_default_library",
"@org_golang_x_tools//go/analysis/passes/composite:go_default_library",
],
}),
)
@@ -133,17 +145,8 @@ common_files = {
"//:README.md": "README.md",
}
toolchain(
name = "built_cmake_toolchain",
toolchain = "@rules_foreign_cc//tools/build_defs/native_tools:built_cmake",
toolchain_type = "@rules_foreign_cc//tools/build_defs:cmake_toolchain",
)
string_setting(
name = "gotags",
build_setting_default = "",
values = [
"",
"libfuzzer",
],
sh_binary(
name = "prysm_sh",
srcs = ["prysm.sh"],
visibility = ["//visibility:public"],
)

View File

@@ -175,7 +175,7 @@ We consider two types of contributions to our repo and categorize them as follow
### Part-Time Contributors
Anyone can become a part-time contributor and help out on implementing eth2. The responsibilities of a part-time contributor include:
Anyone can become a part-time contributor and help out on implementing Ethereum consensus. The responsibilities of a part-time contributor include:
- Engaging in Gitter conversations, asking the questions on how to begin contributing to the project
- Opening up github issues to express interest in code to implement
@@ -190,7 +190,7 @@ Anyone can become a part-time contributor and help out on implementing eth2. The
Core contributors are remote contractors of Prysmatic Labs, LLC. and are considered critical team members of our organization. Core devs have all of the responsibilities of part-time contributors plus the majority of the following:
- Stay up to date on the latest beacon chain sepcification
- Stay up to date on the latest beacon chain specification
- Monitor github issues and PRs to make sure owner, labels, descriptions are correct
- Formulate independent ideas, suggest new work to do, point out improvements to existing approaches
- Participate in code review, ensure code quality is excellent, and have ensure high code coverage

View File

@@ -33,8 +33,8 @@ generates SSZ marshal related code based on defined data structures. These gener
also be updated and checked in as frequently.
```bash
./scripts/update-go-pbs.sh
./scripts/update-go-ssz.sh
./hack/update-go-pbs.sh
./hack/update-go-ssz.sh
```
*Recommendation: Use go build only for local development and use bazel build for production.*

View File

@@ -1,6 +1,6 @@
# Prysm Client Interoperability Guide
This README details how to setup Prysm for interop testing for usage with other Ethereum 2.0 clients.
This README details how to setup Prysm for interop testing for usage with other Ethereum consensus clients.
## Installation & Setup

View File

@@ -1,11 +1,11 @@
# Prysm: An Ethereum 2.0 Implementation Written in Go
# Prysm: An Ethereum Consensus Implementation Written in Go
[![Build status](https://badge.buildkite.com/b555891daf3614bae4284dcf365b2340cefc0089839526f096.svg?branch=master)](https://buildkite.com/prysmatic-labs/prysm)
[![Go Report Card](https://goreportcard.com/badge/github.com/prysmaticlabs/prysm)](https://goreportcard.com/report/github.com/prysmaticlabs/prysm)
[![ETH2.0_Spec_Version 1.0.0](https://img.shields.io/badge/ETH2.0%20Spec%20Version-v1.0.0-blue.svg)](https://github.com/ethereum/eth2.0-specs/tree/v1.0.0)
[![Consensus_Spec_Version 1.1.8](https://img.shields.io/badge/Consensus%20Spec%20Version-v1.1.8-blue.svg)](https://github.com/ethereum/consensus-specs/tree/v1.1.8)
[![Discord](https://user-images.githubusercontent.com/7288322/34471967-1df7808a-efbb-11e7-9088-ed0b04151291.png)](https://discord.gg/CTYGPUJ)
This is the core repository for Prysm, a [Golang](https://golang.org/) implementation of the [Ethereum 2.0](https://ethereum.org/en/eth2/) specification, developed by [Prysmatic Labs](https://prysmaticlabs.com). See the [Changelog](https://github.com/prysmaticlabs/prysm/releases) for details of the latest releases and upcoming breaking changes.
This is the core repository for Prysm, a [Golang](https://golang.org/) implementation of the [Ethereum Consensus](https://ethereum.org/en/eth2/) specification, developed by [Prysmatic Labs](https://prysmaticlabs.com). See the [Changelog](https://github.com/prysmaticlabs/prysm/releases) for details of the latest releases and upcoming breaking changes.
### Getting Started

11
SECURITY.md Normal file
View File

@@ -0,0 +1,11 @@
# Security Policy
## Supported Versions
[Releases](https://github.com/prysmaticlabs/prysm/releases/) contains all available releases. We recommend using the [most recently released version](https://github.com/prysmaticlabs/prysm/releases/latest).
## Reporting a Vulnerability
Please see our signed [security.txt](https://github.com/prysmaticlabs/prysm/blob/develop/.well-known/security.txt) for preferred encryption and reporting destination.
**Please do not file a public ticket** mentioning the vulnerability, as doing so could increase the likelihood of the vulnerability being used before a fix has been created, released and installed on the network.

View File

@@ -5,7 +5,7 @@ Effective as of Oct 14, 2020
By downloading, accessing or using the Prysm implementation (“Prysm”), you (referenced herein as “you” or the “user”) certify that you have read and agreed to the terms and conditions below (the “Terms”) which form a binding contract between you and Prysmatic Labs (referenced herein as “we” or “us”). If you do not agree to the Terms, do not download or use Prysm.
### About Prysm
Prysm is a client implementation for Ethereum 2.0 Phase 0 protocol for a proof-of-stake blockchain. To participate in the network, a user must send ETH from the Eth1.0 chain into a validator deposit contract, which will queue in the user as a validator in the system. Validators participate in proposing and voting on blocks in the protocol, and the network applies rewards/penalties based on their behavior. A detailed set of installation and usage instructions as well as breakdowns of each individual component are available in the official documentation portal, however, we do not warrant the accuracy, completeness or usefulness of this documentation. Any reliance you place on such information is strictly at your own risk.
Prysm is a client implementation for Ethereum consensus protocol for a proof-of-stake blockchain. To participate in the network, a user must send ETH from the Eth1.0 chain into a validator deposit contract, which will queue in the user as a validator in the system. Validators participate in proposing and voting on blocks in the protocol, and the network applies rewards/penalties based on their behavior. A detailed set of installation and usage instructions as well as breakdowns of each individual component are available in the official documentation portal, however, we do not warrant the accuracy, completeness or usefulness of this documentation. Any reliance you place on such information is strictly at your own risk.
### Licensing Terms
Prysm is a fully open-source software program licensed pursuant to the GNU General Public License v3.0.
@@ -13,7 +13,7 @@ Prysm is a fully open-source software program licensed pursuant to the GNU Gener
The Prysmatic Labs name, the term “Prysm” and all related names, logos, product and service names, designs and slogans are trademarks of Prysmatic Labs or its affiliates and/or licensors. You must not use such marks without our prior written permission.
### Risks of Operating Prysm
The use of Prysm and acting as a validator on the Ethereum 2.0 network can lead to loss of money. Ethereum is still an experimental system and ETH remains a risky investment. You alone are responsible for your actions on Prysm including the security of your ETH and meeting any applicable minimum system requirements.
The use of Prysm and acting as a validator on the Ethereum network can lead to loss of money. Ethereum is still an experimental system and ETH remains a risky investment. You alone are responsible for your actions on Prysm including the security of your ETH and meeting any applicable minimum system requirements.
Use of Prysm and the ability to receive rewards or penalties may be affected at any time by mistakes made by the user or other users, software problems such as bugs, errors, incorrectly constructed transactions, unsafe cryptographic libraries or malware affecting the network, technical failures in the hardware of a user, security problems experienced by a user and/or actions or inactions of third parties and/or events experienced by third parties, among other risks. We cannot and do not guarantee that any user of Prysm will make money, that the Prysm network will operate in accordance with the documentation or that transactions will be effective or secure.

274
WORKSPACE
View File

@@ -60,10 +60,10 @@ bazel_skylib_workspace()
http_archive(
name = "bazel_gazelle",
sha256 = "62ca106be173579c0a167deb23358fdfe71ffa1e4cfdddf5582af26520f1c66f",
sha256 = "de69a09dc70417580aabf20a28619bb3ef60d038470c7cf8442fafcf627c21cb",
urls = [
"https://mirror.bazel.build/github.com/bazelbuild/bazel-gazelle/releases/download/v0.23.0/bazel-gazelle-v0.23.0.tar.gz",
"https://github.com/bazelbuild/bazel-gazelle/releases/download/v0.23.0/bazel-gazelle-v0.23.0.tar.gz",
"https://mirror.bazel.build/github.com/bazelbuild/bazel-gazelle/releases/download/v0.24.0/bazel-gazelle-v0.24.0.tar.gz",
"https://github.com/bazelbuild/bazel-gazelle/releases/download/v0.24.0/bazel-gazelle-v0.24.0.tar.gz",
],
)
@@ -76,23 +76,22 @@ http_archive(
http_archive(
name = "io_bazel_rules_docker",
sha256 = "59d5b42ac315e7eadffa944e86e90c2990110a1c8075f1cd145f487e999d22b3",
strip_prefix = "rules_docker-0.17.0",
urls = ["https://github.com/bazelbuild/rules_docker/releases/download/v0.17.0/rules_docker-v0.17.0.tar.gz"],
sha256 = "1f4e59843b61981a96835dc4ac377ad4da9f8c334ebe5e0bb3f58f80c09735f4",
strip_prefix = "rules_docker-0.19.0",
urls = ["https://github.com/bazelbuild/rules_docker/releases/download/v0.19.0/rules_docker-v0.19.0.tar.gz"],
)
http_archive(
name = "io_bazel_rules_go",
patch_args = ["-p1"],
patches = [
# Required until https://github.com/bazelbuild/rules_go/pull/2450 merges otherwise nilness
# nogo check fails for certain third_party dependencies.
"//third_party:io_bazel_rules_go.patch",
# Expose internals of go_test for custom build transitions.
"//third_party:io_bazel_rules_go_test.patch",
],
sha256 = "7c10271940c6bce577d51a075ae77728964db285dac0a46614a7934dc34303e6",
sha256 = "2b1641428dff9018f9e85c0384f03ec6c10660d935b750e3fa1492a281a53b0f",
urls = [
"https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.26.0/rules_go-v0.26.0.tar.gz",
"https://github.com/bazelbuild/rules_go/releases/download/v0.26.0/rules_go-v0.26.0.tar.gz",
"https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.29.0/rules_go-v0.29.0.zip",
"https://github.com/bazelbuild/rules_go/releases/download/v0.29.0/rules_go-v0.29.0.zip",
],
)
@@ -137,6 +136,34 @@ load(
"container_pull",
)
container_pull(
name = "cc_image_base",
digest = "sha256:2c4bb6b7236db0a55ec54ba8845e4031f5db2be957ac61867872bf42e56c4deb",
registry = "gcr.io",
repository = "distroless/cc",
)
container_pull(
name = "cc_debug_image_base",
digest = "sha256:3680c61e81f68fc00bfb5e1ec65e8e678aaafa7c5f056bc2681c29527ebbb30c",
registry = "gcr.io",
repository = "distroless/cc",
)
container_pull(
name = "go_image_base",
digest = "sha256:ba7a315f86771332e76fa9c3d423ecfdbb8265879c6f1c264d6fff7d4fa460a4",
registry = "gcr.io",
repository = "distroless/base",
)
container_pull(
name = "go_debug_image_base",
digest = "sha256:efd8711717d9e9b5d0dbb20ea10876dab0609c923bc05321b912f9239090ca80",
registry = "gcr.io",
repository = "distroless/base",
)
container_pull(
name = "alpine_cc_linux_amd64",
digest = "sha256:752aa0c9a88461ffc50c5267bb7497ef03a303e38b2c8f7f2ded9bebe5f1f00e",
@@ -160,36 +187,6 @@ go_register_toolchains(
nogo = "@//:nogo",
)
load("@bazel_gazelle//:deps.bzl", "gazelle_dependencies")
gazelle_dependencies()
load("@io_bazel_rules_go//extras:embed_data_deps.bzl", "go_embed_data_dependencies")
go_embed_data_dependencies()
load("@com_github_atlassian_bazel_tools//gometalinter:deps.bzl", "gometalinter_dependencies")
gometalinter_dependencies()
load(
"@io_bazel_rules_docker//go:image.bzl",
_go_image_repos = "repositories",
)
# Golang images
# This is using gcr.io/distroless/base
_go_image_repos()
# CC images
# This is using gcr.io/distroless/base
load(
"@io_bazel_rules_docker//cc:image.bzl",
_cc_image_repos = "repositories",
)
_cc_image_repos()
http_archive(
name = "prysm_testnet_site",
build_file_content = """
@@ -222,11 +219,15 @@ filegroup(
)
""",
sha256 = "91434d5fd5e1c6eb7b0174fed2afe25e09bddf00e1e4c431db931b2cee4e7773",
url = "https://github.com/eth2-clients/slashing-protection-interchange-tests/archive/b8413ca42dc92308019d0d4db52c87e9e125c4e9.tar.gz",
url = "https://github.com/eth-clients/slashing-protection-interchange-tests/archive/b8413ca42dc92308019d0d4db52c87e9e125c4e9.tar.gz",
)
consensus_spec_version = "v1.1.9"
bls_test_version = "v0.1.1"
http_archive(
name = "eth2_spec_tests_general",
name = "consensus_spec_tests_general",
build_file_content = """
filegroup(
name = "test_data",
@@ -237,12 +238,12 @@ filegroup(
visibility = ["//visibility:public"],
)
""",
sha256 = "deacc076365c727d653ac064894ecf0d1b0a675d86704dc8de271259f6a7314b",
url = "https://github.com/ethereum/eth2.0-spec-tests/releases/download/v1.1.0-alpha.3/general.tar.gz",
sha256 = "207d9c326ba4fa1f34bab7b6169201c32f2611755db030909a3405873445e0ba",
url = "https://github.com/ethereum/consensus-spec-tests/releases/download/%s/general.tar.gz" % consensus_spec_version,
)
http_archive(
name = "eth2_spec_tests_minimal",
name = "consensus_spec_tests_minimal",
build_file_content = """
filegroup(
name = "test_data",
@@ -253,12 +254,12 @@ filegroup(
visibility = ["//visibility:public"],
)
""",
sha256 = "6e9886af3d2f024e563249d70388129e28e3e92f742f289238ed9b7ec7a7f930",
url = "https://github.com/ethereum/eth2.0-spec-tests/releases/download/v1.1.0-alpha.3/minimal.tar.gz",
sha256 = "a3995b39f412db236b2f1db909f288218da53cb53b9923b71dda9d144d68f40a",
url = "https://github.com/ethereum/consensus-spec-tests/releases/download/%s/minimal.tar.gz" % consensus_spec_version,
)
http_archive(
name = "eth2_spec_tests_mainnet",
name = "consensus_spec_tests_mainnet",
build_file_content = """
filegroup(
name = "test_data",
@@ -269,95 +270,76 @@ filegroup(
visibility = ["//visibility:public"],
)
""",
sha256 = "a7b3d0ffc02a567250f424d69b2474fdc9477cd56eada60af7474560b46a8527",
url = "https://github.com/ethereum/eth2.0-spec-tests/releases/download/v1.1.0-alpha.3/mainnet.tar.gz",
sha256 = "76cea7a4c8e32d458ad456b54bfbb30bc772481a91954a4cd97e229aa3023b1d",
url = "https://github.com/ethereum/consensus-spec-tests/releases/download/%s/mainnet.tar.gz" % consensus_spec_version,
)
http_archive(
name = "consensus_spec",
build_file_content = """
filegroup(
name = "spec_data",
srcs = glob([
"**/*.yaml",
]),
visibility = ["//visibility:public"],
)
""",
sha256 = "0fc429684775f943250dce1f9c485ac25e26c6395d7f585c8d1317becec2ace7",
strip_prefix = "consensus-specs-" + consensus_spec_version[1:],
url = "https://github.com/ethereum/consensus-specs/archive/refs/tags/%s.tar.gz" % consensus_spec_version,
)
http_archive(
name = "bls_spec_tests",
build_file_content = """
filegroup(
name = "test_data",
srcs = glob([
"**/*.yaml",
]),
visibility = ["//visibility:public"],
)
""",
sha256 = "93c7d006e7c5b882cbd11dc9ec6c5d0e07f4a8c6b27a32f964eb17cf2db9763a",
url = "https://github.com/ethereum/bls12-381-tests/releases/download/%s/bls_tests_yaml.tar.gz" % bls_test_version,
)
http_archive(
name = "eth2_networks",
build_file_content = """
filegroup(
name = "configs",
srcs = glob([
"shared/**/config.yaml",
]),
visibility = ["//visibility:public"],
)
""",
sha256 = "4e8a18b21d056c4032605621b1a6632198eabab57cb90c61e273f344c287f1b2",
strip_prefix = "eth2-networks-791a5369c5981e829698b17fbcdcdacbdaba97c8",
url = "https://github.com/eth-clients/eth2-networks/archive/791a5369c5981e829698b17fbcdcdacbdaba97c8.tar.gz",
)
http_archive(
name = "com_github_bazelbuild_buildtools",
sha256 = "b5d7dbc6832f11b6468328a376de05959a1a9e4e9f5622499d3bab509c26b46a",
strip_prefix = "buildtools-bf564b4925ab5876a3f64d8b90fab7f769013d42",
url = "https://github.com/bazelbuild/buildtools/archive/bf564b4925ab5876a3f64d8b90fab7f769013d42.zip",
sha256 = "7a182df18df1debabd9e36ae07c8edfa1378b8424a04561b674d933b965372b3",
strip_prefix = "buildtools-f2aed9ee205d62d45c55cfabbfd26342f8526862",
url = "https://github.com/bazelbuild/buildtools/archive/f2aed9ee205d62d45c55cfabbfd26342f8526862.zip",
)
load("@com_github_bazelbuild_buildtools//buildifier:deps.bzl", "buildifier_dependencies")
buildifier_dependencies()
git_repository(
name = "com_google_protobuf",
commit = "fde7cf7358ec7cd69e8db9be4f1fa6a5c431386a", # v3.13.0
commit = "436bd7880e458532901c58f4d9d1ea23fa7edd52",
remote = "https://github.com/protocolbuffers/protobuf",
shallow_since = "1597443653 -0700",
shallow_since = "1617835118 -0700",
)
load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps")
protobuf_deps()
# Group the sources of the library so that CMake rule have access to it
all_content = """filegroup(name = "all", srcs = glob(["**"]), visibility = ["//visibility:public"])"""
http_archive(
name = "rules_foreign_cc",
sha256 = "b85ce66a3410f7370d1a9a61dfe3a29c7532b7637caeb2877d8d0dfd41d77abb",
strip_prefix = "rules_foreign_cc-3515b20a2417c4dd51c8a4a8cac1f6ecf3c6d934",
url = "https://github.com/bazelbuild/rules_foreign_cc/archive/3515b20a2417c4dd51c8a4a8cac1f6ecf3c6d934.zip",
)
load("@rules_foreign_cc//:workspace_definitions.bzl", "rules_foreign_cc_dependencies")
rules_foreign_cc_dependencies([
"@prysm//:built_cmake_toolchain",
])
http_archive(
name = "librdkafka",
build_file_content = all_content,
sha256 = "3b99a36c082a67ef6295eabd4fb3e32ab0bff7c6b0d397d6352697335f4e57eb",
strip_prefix = "librdkafka-1.4.2",
urls = ["https://github.com/edenhill/librdkafka/archive/v1.4.2.tar.gz"],
)
http_archive(
name = "sigp_beacon_fuzz_corpora",
build_file = "//third_party:beacon-fuzz/corpora.BUILD",
sha256 = "42993d0901a316afda45b4ba6d53c7c21f30c551dcec290a4ca131c24453d1ef",
strip_prefix = "beacon-fuzz-corpora-bac24ad78d45cc3664c0172241feac969c1ac29b",
urls = [
"https://github.com/sigp/beacon-fuzz-corpora/archive/bac24ad78d45cc3664c0172241feac969c1ac29b.tar.gz",
],
)
# External dependencies
http_archive(
name = "sszgen", # Hack because we don't want to build this binary with libfuzzer, but need it to build.
build_file_content = """
load("@io_bazel_rules_go//go:def.bzl", "go_library", "go_binary")
go_library(
name = "go_default_library",
srcs = [
"sszgen/main.go",
"sszgen/marshal.go",
"sszgen/size.go",
"sszgen/unmarshal.go",
],
importpath = "github.com/ferranbt/fastssz/sszgen",
visibility = ["//visibility:private"],
)
go_binary(
name = "sszgen",
embed = [":go_default_library"],
visibility = ["//visibility:public"],
)
""",
strip_prefix = "fastssz-06015a5d84f9e4eefe2c21377ca678fa8f1a1b09",
urls = ["https://github.com/ferranbt/fastssz/archive/06015a5d84f9e4eefe2c21377ca678fa8f1a1b09.tar.gz"],
)
http_archive(
name = "prysm_web_ui",
build_file_content = """
@@ -367,9 +349,9 @@ filegroup(
visibility = ["//visibility:public"],
)
""",
sha256 = "edb80f3a695d84f6000f0e05abf7a4bbf207c03abb91219780ec97e7d6ad21c8",
sha256 = "f196fe4367c2d2d01d36565c0dc6eecfa4f03adba1fc03a61d62953fce606e1f",
urls = [
"https://github.com/prysmaticlabs/prysm-web-ui/releases/download/v1.0.0-beta.3/prysm-web-ui.tar.gz",
"https://github.com/prysmaticlabs/prysm-web-ui/releases/download/v1.0.2/prysm-web-ui.tar.gz",
],
)
@@ -382,8 +364,46 @@ load("@prysm//third_party/herumi:herumi.bzl", "bls_dependencies")
bls_dependencies()
load("@com_github_ethereum_go_ethereum//:deps.bzl", "geth_dependencies")
load("@prysm//testing/endtoend:deps.bzl", "e2e_deps")
geth_dependencies()
e2e_deps()
load(
"@io_bazel_rules_docker//go:image.bzl",
_go_image_repos = "repositories",
)
# Golang images
# This is using gcr.io/distroless/base
_go_image_repos()
# CC images
# This is using gcr.io/distroless/base
load(
"@io_bazel_rules_docker//cc:image.bzl",
_cc_image_repos = "repositories",
)
_cc_image_repos()
load("@io_bazel_rules_go//extras:embed_data_deps.bzl", "go_embed_data_dependencies")
go_embed_data_dependencies()
load("@com_github_atlassian_bazel_tools//gometalinter:deps.bzl", "gometalinter_dependencies")
gometalinter_dependencies()
load("@bazel_gazelle//:deps.bzl", "gazelle_dependencies")
gazelle_dependencies()
load("@com_github_bazelbuild_buildtools//buildifier:deps.bzl", "buildifier_dependencies")
buildifier_dependencies()
load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps")
protobuf_deps()
# Do NOT add new go dependencies here! Refer to DEPENDENCIES.md!

42
api/gateway/BUILD.bazel Normal file
View File

@@ -0,0 +1,42 @@
load("@prysm//tools/go:def.bzl", "go_library", "go_test")
go_library(
name = "go_default_library",
srcs = [
"gateway.go",
"log.go",
"options.go",
],
importpath = "github.com/prysmaticlabs/prysm/api/gateway",
visibility = [
"//beacon-chain:__subpackages__",
"//validator:__subpackages__",
],
deps = [
"//api/gateway/apimiddleware:go_default_library",
"//runtime:go_default_library",
"@com_github_gorilla_mux//:go_default_library",
"@com_github_grpc_ecosystem_grpc_gateway_v2//runtime:go_default_library",
"@com_github_pkg_errors//:go_default_library",
"@com_github_rs_cors//:go_default_library",
"@com_github_sirupsen_logrus//:go_default_library",
"@org_golang_google_grpc//:go_default_library",
"@org_golang_google_grpc//connectivity:go_default_library",
"@org_golang_google_grpc//credentials:go_default_library",
],
)
go_test(
name = "go_default_test",
srcs = ["gateway_test.go"],
embed = [":go_default_library"],
deps = [
"//api/gateway/apimiddleware:go_default_library",
"//cmd/beacon-chain/flags:go_default_library",
"//testing/assert:go_default_library",
"//testing/require:go_default_library",
"@com_github_gorilla_mux//:go_default_library",
"@com_github_sirupsen_logrus//hooks/test:go_default_library",
"@com_github_urfave_cli_v2//:go_default_library",
],
)

View File

@@ -0,0 +1,40 @@
load("@prysm//tools/go:def.bzl", "go_library", "go_test")
go_library(
name = "go_default_library",
srcs = [
"api_middleware.go",
"log.go",
"param_handling.go",
"process_field.go",
"process_request.go",
"structs.go",
],
importpath = "github.com/prysmaticlabs/prysm/api/gateway/apimiddleware",
visibility = ["//visibility:public"],
deps = [
"//api/grpc:go_default_library",
"//encoding/bytesutil:go_default_library",
"@com_github_ethereum_go_ethereum//common/hexutil:go_default_library",
"@com_github_gorilla_mux//:go_default_library",
"@com_github_pkg_errors//:go_default_library",
"@com_github_sirupsen_logrus//:go_default_library",
"@com_github_wealdtech_go_bytesutil//:go_default_library",
],
)
go_test(
name = "go_default_test",
srcs = [
"param_handling_test.go",
"process_request_test.go",
],
embed = [":go_default_library"],
deps = [
"//api/grpc:go_default_library",
"//testing/assert:go_default_library",
"//testing/require:go_default_library",
"@com_github_gorilla_mux//:go_default_library",
"@com_github_sirupsen_logrus//hooks/test:go_default_library",
],
)

View File

@@ -0,0 +1,263 @@
package apimiddleware
import (
"net/http"
"reflect"
"github.com/gorilla/mux"
)
// ApiProxyMiddleware is a proxy between an Ethereum consensus API HTTP client and grpc-gateway.
// The purpose of the proxy is to handle HTTP requests and gRPC responses in such a way that:
// - Ethereum consensus API requests can be handled by grpc-gateway correctly
// - gRPC responses can be returned as spec-compliant Ethereum consensus API responses
type ApiProxyMiddleware struct {
GatewayAddress string
EndpointCreator EndpointFactory
router *mux.Router
}
// EndpointFactory is responsible for creating new instances of Endpoint values.
type EndpointFactory interface {
Create(path string) (*Endpoint, error)
Paths() []string
IsNil() bool
}
// Endpoint is a representation of an API HTTP endpoint that should be proxied by the middleware.
type Endpoint struct {
Path string // The path of the HTTP endpoint.
GetResponse interface{} // The struct corresponding to the JSON structure used in a GET response.
PostRequest interface{} // The struct corresponding to the JSON structure used in a POST request.
PostResponse interface{} // The struct corresponding to the JSON structure used in a POST response.
DeleteRequest interface{} // The struct corresponding to the JSON structure used in a DELETE request.
DeleteResponse interface{} // The struct corresponding to the JSON structure used in a DELETE response.
RequestURLLiterals []string // Names of URL parameters that should not be base64-encoded.
RequestQueryParams []QueryParam // Query parameters of the request.
Err ErrorJson // The struct corresponding to the error that should be returned in case of a request failure.
Hooks HookCollection // A collection of functions that can be invoked at various stages of the request/response cycle.
CustomHandlers []CustomHandler // Functions that will be executed instead of the default request/response behaviour.
}
// RunDefault expresses whether the default processing logic should be carried out after running a pre hook.
type RunDefault bool
// DefaultEndpoint returns an Endpoint with default configuration, e.g. DefaultErrorJson for error handling.
func DefaultEndpoint() Endpoint {
return Endpoint{
Err: &DefaultErrorJson{},
}
}
// QueryParam represents a single query parameter's metadata.
type QueryParam struct {
Name string
Hex bool
Enum bool
}
// CustomHandler is a function that can be invoked at the very beginning of the request,
// essentially replacing the whole default request/response logic with custom logic for a specific endpoint.
type CustomHandler = func(m *ApiProxyMiddleware, endpoint Endpoint, w http.ResponseWriter, req *http.Request) (handled bool)
// HookCollection contains hooks that can be used to amend the default request/response cycle with custom logic for a specific endpoint.
type HookCollection struct {
OnPreDeserializeRequestBodyIntoContainer func(endpoint *Endpoint, w http.ResponseWriter, req *http.Request) (RunDefault, ErrorJson)
OnPostDeserializeRequestBodyIntoContainer func(endpoint *Endpoint, w http.ResponseWriter, req *http.Request) ErrorJson
OnPreDeserializeGrpcResponseBodyIntoContainer func([]byte, interface{}) (RunDefault, ErrorJson)
OnPreSerializeMiddlewareResponseIntoJson func(interface{}) (RunDefault, []byte, ErrorJson)
}
// fieldProcessor applies the processing function f to a value when the tag is present on the field.
type fieldProcessor struct {
tag string
f func(value reflect.Value) error
}
// Run starts the proxy, registering all proxy endpoints.
func (m *ApiProxyMiddleware) Run(gatewayRouter *mux.Router) {
for _, path := range m.EndpointCreator.Paths() {
gatewayRouter.HandleFunc(path, m.WithMiddleware(path))
}
m.router = gatewayRouter
}
// ServeHTTP for the proxy middleware.
func (m *ApiProxyMiddleware) ServeHTTP(w http.ResponseWriter, req *http.Request) {
m.router.ServeHTTP(w, req)
}
// WithMiddleware wraps the given endpoint handler with the middleware logic.
func (m *ApiProxyMiddleware) WithMiddleware(path string) http.HandlerFunc {
return func(w http.ResponseWriter, req *http.Request) {
endpoint, err := m.EndpointCreator.Create(path)
if err != nil {
log.WithError(err).Errorf("Could not create endpoint for path: %s", path)
return
}
for _, handler := range endpoint.CustomHandlers {
if handler(m, *endpoint, w, req) {
return
}
}
if req.Method == "POST" {
if errJson := handlePostRequestForEndpoint(endpoint, w, req); errJson != nil {
WriteError(w, errJson, nil)
return
}
}
if req.Method == "DELETE" {
if errJson := handleDeleteRequestForEndpoint(endpoint, req); errJson != nil {
WriteError(w, errJson, nil)
return
}
}
if errJson := m.PrepareRequestForProxying(*endpoint, req); errJson != nil {
WriteError(w, errJson, nil)
return
}
grpcResp, errJson := ProxyRequest(req)
if errJson != nil {
WriteError(w, errJson, nil)
return
}
grpcRespBody, errJson := ReadGrpcResponseBody(grpcResp.Body)
if errJson != nil {
WriteError(w, errJson, nil)
return
}
var respJson []byte
if !GrpcResponseIsEmpty(grpcRespBody) {
respHasError, errJson := HandleGrpcResponseError(endpoint.Err, grpcResp, grpcRespBody, w)
if errJson != nil {
WriteError(w, errJson, nil)
return
}
if respHasError {
return
}
var resp interface{}
if req.Method == "GET" {
resp = endpoint.GetResponse
} else if req.Method == "DELETE" {
resp = endpoint.DeleteResponse
} else {
resp = endpoint.PostResponse
}
if errJson := deserializeGrpcResponseBodyIntoContainerWrapped(endpoint, grpcRespBody, resp); errJson != nil {
WriteError(w, errJson, nil)
return
}
if errJson := ProcessMiddlewareResponseFields(resp); errJson != nil {
WriteError(w, errJson, nil)
return
}
respJson, errJson = serializeMiddlewareResponseIntoJsonWrapped(endpoint, respJson, resp)
if errJson != nil {
WriteError(w, errJson, nil)
return
}
}
if errJson := WriteMiddlewareResponseHeadersAndBody(grpcResp, respJson, w); errJson != nil {
WriteError(w, errJson, nil)
return
}
if errJson := Cleanup(grpcResp.Body); errJson != nil {
WriteError(w, errJson, nil)
return
}
}
}
func handlePostRequestForEndpoint(endpoint *Endpoint, w http.ResponseWriter, req *http.Request) ErrorJson {
if errJson := deserializeRequestBodyIntoContainerWrapped(endpoint, req, w); errJson != nil {
return errJson
}
if errJson := ProcessRequestContainerFields(endpoint.PostRequest); errJson != nil {
return errJson
}
return SetRequestBodyToRequestContainer(endpoint.PostRequest, req)
}
func handleDeleteRequestForEndpoint(endpoint *Endpoint, req *http.Request) ErrorJson {
if errJson := DeserializeRequestBodyIntoContainer(req.Body, endpoint.DeleteRequest); errJson != nil {
return errJson
}
if errJson := ProcessRequestContainerFields(endpoint.DeleteRequest); errJson != nil {
return errJson
}
return SetRequestBodyToRequestContainer(endpoint.DeleteRequest, req)
}
func deserializeRequestBodyIntoContainerWrapped(endpoint *Endpoint, req *http.Request, w http.ResponseWriter) ErrorJson {
runDefault := true
if endpoint.Hooks.OnPreDeserializeRequestBodyIntoContainer != nil {
run, errJson := endpoint.Hooks.OnPreDeserializeRequestBodyIntoContainer(endpoint, w, req)
if errJson != nil {
return errJson
}
if !run {
runDefault = false
}
}
if runDefault {
if errJson := DeserializeRequestBodyIntoContainer(req.Body, endpoint.PostRequest); errJson != nil {
return errJson
}
}
if endpoint.Hooks.OnPostDeserializeRequestBodyIntoContainer != nil {
if errJson := endpoint.Hooks.OnPostDeserializeRequestBodyIntoContainer(endpoint, w, req); errJson != nil {
return errJson
}
}
return nil
}
func deserializeGrpcResponseBodyIntoContainerWrapped(endpoint *Endpoint, grpcResponseBody []byte, resp interface{}) ErrorJson {
runDefault := true
if endpoint.Hooks.OnPreDeserializeGrpcResponseBodyIntoContainer != nil {
run, errJson := endpoint.Hooks.OnPreDeserializeGrpcResponseBodyIntoContainer(grpcResponseBody, resp)
if errJson != nil {
return errJson
}
if !run {
runDefault = false
}
}
if runDefault {
if errJson := DeserializeGrpcResponseBodyIntoContainer(grpcResponseBody, resp); errJson != nil {
return errJson
}
}
return nil
}
func serializeMiddlewareResponseIntoJsonWrapped(endpoint *Endpoint, respJson []byte, resp interface{}) ([]byte, ErrorJson) {
runDefault := true
var errJson ErrorJson
if endpoint.Hooks.OnPreSerializeMiddlewareResponseIntoJson != nil {
var run RunDefault
run, respJson, errJson = endpoint.Hooks.OnPreSerializeMiddlewareResponseIntoJson(resp)
if errJson != nil {
return nil, errJson
}
if !run {
runDefault = false
}
}
if runDefault {
respJson, errJson = SerializeMiddlewareResponseIntoJson(resp)
if errJson != nil {
return nil, errJson
}
}
return respJson, nil
}

View File

@@ -0,0 +1,5 @@
package apimiddleware
import "github.com/sirupsen/logrus"
var log = logrus.WithField("prefix", "apimiddleware")

View File

@@ -0,0 +1,103 @@
package apimiddleware
import (
"encoding/base64"
"net/http"
"net/url"
"strings"
"github.com/gorilla/mux"
butil "github.com/prysmaticlabs/prysm/encoding/bytesutil"
"github.com/wealdtech/go-bytesutil"
)
// HandleURLParameters processes URL parameters, allowing parameterized URLs to be safely and correctly proxied to grpc-gateway.
func HandleURLParameters(url string, req *http.Request, literals []string) ErrorJson {
segments := strings.Split(url, "/")
segmentsLoop:
for i, s := range segments {
// We only care about segments which are parameterized.
if isRequestParam(s) {
// Don't do anything with parameters which should be forwarded literally to gRPC.
for _, l := range literals {
if s == "{"+l+"}" {
continue segmentsLoop
}
}
routeVar := mux.Vars(req)[s[1:len(s)-1]]
bRouteVar := []byte(routeVar)
if butil.IsHex(bRouteVar) {
var err error
bRouteVar, err = bytesutil.FromHexString(string(bRouteVar))
if err != nil {
return InternalServerErrorWithMessage(err, "could not process URL parameter")
}
}
// Converting hex to base64 may result in a value which malforms the URL.
// We use URLEncoding to safely escape such values.
base64RouteVar := base64.URLEncoding.EncodeToString(bRouteVar)
// Merge segments back into the full URL.
splitPath := strings.Split(req.URL.Path, "/")
splitPath[i] = base64RouteVar
req.URL.Path = strings.Join(splitPath, "/")
}
}
return nil
}
// HandleQueryParameters processes query parameters, allowing them to be safely and correctly proxied to grpc-gateway.
func HandleQueryParameters(req *http.Request, params []QueryParam) ErrorJson {
queryParams := req.URL.Query()
normalizeQueryValues(queryParams)
for key, vals := range queryParams {
for _, p := range params {
if key == p.Name {
if p.Hex {
queryParams.Del(key)
for _, v := range vals {
b := []byte(v)
if butil.IsHex(b) {
var err error
b, err = bytesutil.FromHexString(v)
if err != nil {
return InternalServerErrorWithMessage(err, "could not process query parameter")
}
}
queryParams.Add(key, base64.URLEncoding.EncodeToString(b))
}
}
if p.Enum {
queryParams.Del(key)
for _, v := range vals {
// gRPC expects uppercase enum values.
queryParams.Add(key, strings.ToUpper(v))
}
}
}
}
}
req.URL.RawQuery = queryParams.Encode()
return nil
}
// isRequestParam verifies whether the passed string is a request parameter.
// Request parameters are enclosed in { and }.
func isRequestParam(s string) bool {
return len(s) > 2 && s[0] == '{' && s[len(s)-1] == '}'
}
func normalizeQueryValues(queryParams url.Values) {
// Replace comma-separated values with individual values.
for key, vals := range queryParams {
splitVals := make([]string, 0)
for _, v := range vals {
splitVals = append(splitVals, strings.Split(v, ",")...)
}
queryParams[key] = splitVals
}
}

View File

@@ -0,0 +1,124 @@
package apimiddleware
import (
"bytes"
"net/http/httptest"
"testing"
"github.com/gorilla/mux"
"github.com/prysmaticlabs/prysm/testing/assert"
"github.com/prysmaticlabs/prysm/testing/require"
)
func TestHandleURLParameters(t *testing.T) {
var body bytes.Buffer
t.Run("no_params", func(t *testing.T) {
request := httptest.NewRequest("GET", "http://foo.example/bar", &body)
errJson := HandleURLParameters("/not_param", request, []string{})
require.Equal(t, true, errJson == nil)
assert.Equal(t, "/bar", request.URL.Path)
})
t.Run("with_params", func(t *testing.T) {
muxVars := make(map[string]string)
muxVars["bar_param"] = "bar"
muxVars["quux_param"] = "quux"
request := httptest.NewRequest("GET", "http://foo.example/bar/baz/quux", &body)
request = mux.SetURLVars(request, muxVars)
errJson := HandleURLParameters("/{bar_param}/not_param/{quux_param}", request, []string{})
require.Equal(t, true, errJson == nil)
assert.Equal(t, "/YmFy/baz/cXV1eA==", request.URL.Path)
})
t.Run("with_literal", func(t *testing.T) {
muxVars := make(map[string]string)
muxVars["bar_param"] = "bar"
request := httptest.NewRequest("GET", "http://foo.example/bar/baz", &body)
request = mux.SetURLVars(request, muxVars)
errJson := HandleURLParameters("/{bar_param}/not_param/", request, []string{"bar_param"})
require.Equal(t, true, errJson == nil)
assert.Equal(t, "/bar/baz", request.URL.Path)
})
t.Run("with_hex", func(t *testing.T) {
muxVars := make(map[string]string)
muxVars["hex_param"] = "0x626172"
request := httptest.NewRequest("GET", "http://foo.example/0x626172/baz", &body)
request = mux.SetURLVars(request, muxVars)
errJson := HandleURLParameters("/{hex_param}/not_param/", request, []string{})
require.Equal(t, true, errJson == nil)
assert.Equal(t, "/YmFy/baz", request.URL.Path)
})
}
func TestHandleQueryParameters(t *testing.T) {
var body bytes.Buffer
t.Run("regular_params", func(t *testing.T) {
request := httptest.NewRequest("GET", "http://foo.example?bar=bar&baz=baz", &body)
errJson := HandleQueryParameters(request, []QueryParam{{Name: "bar"}, {Name: "baz"}})
require.Equal(t, true, errJson == nil)
query := request.URL.Query()
v, ok := query["bar"]
require.Equal(t, true, ok, "query param not found")
require.Equal(t, 1, len(v), "wrong number of query param values")
assert.Equal(t, "bar", v[0])
v, ok = query["baz"]
require.Equal(t, true, ok, "query param not found")
require.Equal(t, 1, len(v), "wrong number of query param values")
assert.Equal(t, "baz", v[0])
})
t.Run("hex_and_enum_params", func(t *testing.T) {
request := httptest.NewRequest("GET", "http://foo.example?hex=0x626172&baz=baz", &body)
errJson := HandleQueryParameters(request, []QueryParam{{Name: "hex", Hex: true}, {Name: "baz", Enum: true}})
require.Equal(t, true, errJson == nil)
query := request.URL.Query()
v, ok := query["hex"]
require.Equal(t, true, ok, "query param not found")
require.Equal(t, 1, len(v), "wrong number of query param values")
assert.Equal(t, "YmFy", v[0])
v, ok = query["baz"]
require.Equal(t, true, ok, "query param not found")
require.Equal(t, 1, len(v), "wrong number of query param values")
assert.Equal(t, "BAZ", v[0])
})
}
func TestIsRequestParam(t *testing.T) {
tests := []struct {
s string
b bool
}{
{"", false},
{"{", false},
{"}", false},
{"{}", false},
{"{x}", true},
{"{very_long_parameter_name_with_underscores}", true},
}
for _, tt := range tests {
b := isRequestParam(tt.s)
assert.Equal(t, tt.b, b)
}
}
func TestNormalizeQueryValues(t *testing.T) {
input := make(map[string][]string)
input["key"] = []string{"value1", "value2,value3,value4", "value5"}
normalizeQueryValues(input)
require.Equal(t, 5, len(input["key"]))
assert.Equal(t, "value1", input["key"][0])
assert.Equal(t, "value2", input["key"][1])
assert.Equal(t, "value3", input["key"][2])
assert.Equal(t, "value4", input["key"][3])
assert.Equal(t, "value5", input["key"][4])
}

View File

@@ -0,0 +1,108 @@
package apimiddleware
import (
"encoding/base64"
"fmt"
"reflect"
"strconv"
"strings"
"time"
"github.com/ethereum/go-ethereum/common/hexutil"
"github.com/pkg/errors"
"github.com/wealdtech/go-bytesutil"
)
// processField calls each processor function on any field that has the matching tag set.
// It is a recursive function.
func processField(s interface{}, processors []fieldProcessor) error {
kind := reflect.TypeOf(s).Kind()
if kind != reflect.Ptr && kind != reflect.Slice && kind != reflect.Array {
return fmt.Errorf("processing fields of kind '%v' is unsupported", kind)
}
t := reflect.TypeOf(s).Elem()
v := reflect.Indirect(reflect.ValueOf(s))
for i := 0; i < t.NumField(); i++ {
switch v.Field(i).Kind() {
case reflect.Slice:
sliceElem := t.Field(i).Type.Elem()
kind := sliceElem.Kind()
// Recursively process slices to struct pointers.
if kind == reflect.Ptr && sliceElem.Elem().Kind() == reflect.Struct {
for j := 0; j < v.Field(i).Len(); j++ {
if err := processField(v.Field(i).Index(j).Interface(), processors); err != nil {
return errors.Wrapf(err, "could not process field '%s'", t.Field(i).Name)
}
}
}
// Process each string in string slices.
if kind == reflect.String {
for _, proc := range processors {
_, hasTag := t.Field(i).Tag.Lookup(proc.tag)
if hasTag {
for j := 0; j < v.Field(i).Len(); j++ {
if err := proc.f(v.Field(i).Index(j)); err != nil {
return errors.Wrapf(err, "could not process field '%s'", t.Field(i).Name)
}
}
}
}
}
// Recursively process struct pointers.
case reflect.Ptr:
if v.Field(i).Elem().Kind() == reflect.Struct {
if err := processField(v.Field(i).Interface(), processors); err != nil {
return errors.Wrapf(err, "could not process field '%s'", t.Field(i).Name)
}
}
default:
field := t.Field(i)
for _, proc := range processors {
if _, hasTag := field.Tag.Lookup(proc.tag); hasTag {
if err := proc.f(v.Field(i)); err != nil {
return errors.Wrapf(err, "could not process field '%s'", t.Field(i).Name)
}
}
}
}
}
return nil
}
func hexToBase64Processor(v reflect.Value) error {
b, err := bytesutil.FromHexString(v.String())
if err != nil {
return err
}
v.SetString(base64.StdEncoding.EncodeToString(b))
return nil
}
func base64ToHexProcessor(v reflect.Value) error {
if v.String() == "" {
return nil
}
b, err := base64.StdEncoding.DecodeString(v.String())
if err != nil {
return err
}
v.SetString(hexutil.Encode(b))
return nil
}
func enumToLowercaseProcessor(v reflect.Value) error {
v.SetString(strings.ToLower(v.String()))
return nil
}
func timeToUnixProcessor(v reflect.Value) error {
t, err := time.Parse(time.RFC3339, v.String())
if err != nil {
return err
}
v.SetString(strconv.FormatUint(uint64(t.Unix()), 10))
return nil
}

View File

@@ -0,0 +1,255 @@
package apimiddleware
import (
"bytes"
"encoding/json"
"io"
"io/ioutil"
"net/http"
"strconv"
"strings"
"time"
"github.com/pkg/errors"
"github.com/prysmaticlabs/prysm/api/grpc"
)
// DeserializeRequestBodyIntoContainer deserializes the request's body into an endpoint-specific struct.
func DeserializeRequestBodyIntoContainer(body io.Reader, requestContainer interface{}) ErrorJson {
decoder := json.NewDecoder(body)
decoder.DisallowUnknownFields()
if err := decoder.Decode(&requestContainer); err != nil {
if strings.Contains(err.Error(), "json: unknown field") {
e := errors.Wrap(err, "could not decode request body")
return &DefaultErrorJson{
Message: e.Error(),
Code: http.StatusBadRequest,
}
}
return InternalServerErrorWithMessage(err, "could not decode request body")
}
return nil
}
// ProcessRequestContainerFields processes fields of an endpoint-specific container according to field tags.
func ProcessRequestContainerFields(requestContainer interface{}) ErrorJson {
if err := processField(requestContainer, []fieldProcessor{
{
tag: "hex",
f: hexToBase64Processor,
},
}); err != nil {
return InternalServerErrorWithMessage(err, "could not process request data")
}
return nil
}
// SetRequestBodyToRequestContainer makes the endpoint-specific container the new body of the request.
func SetRequestBodyToRequestContainer(requestContainer interface{}, req *http.Request) ErrorJson {
// Serialize the struct, which now includes a base64-encoded value, into JSON.
j, err := json.Marshal(requestContainer)
if err != nil {
return InternalServerErrorWithMessage(err, "could not marshal request")
}
// Set the body to the new JSON.
req.Body = ioutil.NopCloser(bytes.NewReader(j))
req.Header.Set("Content-Length", strconv.Itoa(len(j)))
req.ContentLength = int64(len(j))
return nil
}
// PrepareRequestForProxying applies additional logic to the request so that it can be correctly proxied to grpc-gateway.
func (m *ApiProxyMiddleware) PrepareRequestForProxying(endpoint Endpoint, req *http.Request) ErrorJson {
req.URL.Scheme = "http"
req.URL.Host = m.GatewayAddress
req.RequestURI = ""
if errJson := HandleURLParameters(endpoint.Path, req, endpoint.RequestURLLiterals); errJson != nil {
return errJson
}
if errJson := HandleQueryParameters(req, endpoint.RequestQueryParams); errJson != nil {
return errJson
}
// We have to add the prefix after handling parameters because adding the prefix changes URL segment indexing.
req.URL.Path = "/internal" + req.URL.Path
return nil
}
// ProxyRequest proxies the request to grpc-gateway.
func ProxyRequest(req *http.Request) (*http.Response, ErrorJson) {
// We do not use http.DefaultClient because it does not have any timeout.
netClient := &http.Client{Timeout: time.Minute * 2}
grpcResp, err := netClient.Do(req)
if err != nil {
return nil, InternalServerErrorWithMessage(err, "could not proxy request")
}
if grpcResp == nil {
return nil, &DefaultErrorJson{Message: "nil response from gRPC-gateway", Code: http.StatusInternalServerError}
}
return grpcResp, nil
}
// ReadGrpcResponseBody reads the body from the grpc-gateway's response.
func ReadGrpcResponseBody(r io.Reader) ([]byte, ErrorJson) {
body, err := ioutil.ReadAll(r)
if err != nil {
return nil, InternalServerErrorWithMessage(err, "could not read response body")
}
return body, nil
}
// HandleGrpcResponseError acts on an error that resulted from a grpc-gateway's response.
func HandleGrpcResponseError(errJson ErrorJson, resp *http.Response, respBody []byte, w http.ResponseWriter) (bool, ErrorJson) {
responseHasError := false
if err := json.Unmarshal(respBody, errJson); err != nil {
return false, InternalServerErrorWithMessage(err, "could not unmarshal error")
}
if errJson.Msg() != "" {
responseHasError = true
// Something went wrong, but the request completed, meaning we can write headers and the error message.
for h, vs := range resp.Header {
for _, v := range vs {
w.Header().Set(h, v)
}
}
// Set code to HTTP code because unmarshalled body contained gRPC code.
errJson.SetCode(resp.StatusCode)
WriteError(w, errJson, resp.Header)
}
return responseHasError, nil
}
// GrpcResponseIsEmpty determines whether the grpc-gateway's response body contains no data.
func GrpcResponseIsEmpty(grpcResponseBody []byte) bool {
return len(grpcResponseBody) == 0 || string(grpcResponseBody) == "{}"
}
// DeserializeGrpcResponseBodyIntoContainer deserializes the grpc-gateway's response body into an endpoint-specific struct.
func DeserializeGrpcResponseBodyIntoContainer(body []byte, responseContainer interface{}) ErrorJson {
if err := json.Unmarshal(body, &responseContainer); err != nil {
return InternalServerErrorWithMessage(err, "could not unmarshal response")
}
return nil
}
// ProcessMiddlewareResponseFields processes fields of an endpoint-specific container according to field tags.
func ProcessMiddlewareResponseFields(responseContainer interface{}) ErrorJson {
if err := processField(responseContainer, []fieldProcessor{
{
tag: "hex",
f: base64ToHexProcessor,
},
{
tag: "enum",
f: enumToLowercaseProcessor,
},
{
tag: "time",
f: timeToUnixProcessor,
},
}); err != nil {
return InternalServerErrorWithMessage(err, "could not process response data")
}
return nil
}
// SerializeMiddlewareResponseIntoJson serializes the endpoint-specific response struct into a JSON representation.
func SerializeMiddlewareResponseIntoJson(responseContainer interface{}) (jsonResponse []byte, errJson ErrorJson) {
j, err := json.Marshal(responseContainer)
if err != nil {
return nil, InternalServerErrorWithMessage(err, "could not marshal response")
}
return j, nil
}
// WriteMiddlewareResponseHeadersAndBody populates headers and the body of the final response.
func WriteMiddlewareResponseHeadersAndBody(grpcResp *http.Response, responseJson []byte, w http.ResponseWriter) ErrorJson {
var statusCodeHeader string
for h, vs := range grpcResp.Header {
// We don't want to expose any gRPC metadata in the HTTP response, so we skip forwarding metadata headers.
if strings.HasPrefix(h, "Grpc-Metadata") {
if h == "Grpc-Metadata-"+grpc.HttpCodeMetadataKey {
statusCodeHeader = vs[0]
}
} else {
for _, v := range vs {
w.Header().Set(h, v)
}
}
}
if !GrpcResponseIsEmpty(responseJson) {
w.Header().Set("Content-Length", strconv.Itoa(len(responseJson)))
if statusCodeHeader != "" {
code, err := strconv.Atoi(statusCodeHeader)
if err != nil {
return InternalServerErrorWithMessage(err, "could not parse status code")
}
w.WriteHeader(code)
} else {
w.WriteHeader(grpcResp.StatusCode)
}
if _, err := io.Copy(w, ioutil.NopCloser(bytes.NewReader(responseJson))); err != nil {
return InternalServerErrorWithMessage(err, "could not write response message")
}
} else {
w.Header().Set("Content-Length", "0")
w.WriteHeader(grpcResp.StatusCode)
}
return nil
}
// WriteError writes the error by manipulating headers and the body of the final response.
func WriteError(w http.ResponseWriter, errJson ErrorJson, responseHeader http.Header) {
// Include custom error in the error JSON.
hasCustomError := false
if responseHeader != nil {
customError, ok := responseHeader["Grpc-Metadata-"+grpc.CustomErrorMetadataKey]
if ok {
hasCustomError = true
// Assume header has only one value and read the 0 index.
if err := json.Unmarshal([]byte(customError[0]), errJson); err != nil {
log.WithError(err).Error("Could not unmarshal custom error message")
return
}
}
}
var j []byte
if hasCustomError {
var err error
j, err = json.Marshal(errJson)
if err != nil {
log.WithError(err).Error("Could not marshal error message")
return
}
} else {
var err error
// We marshal the response body into a DefaultErrorJson if the custom error is not present.
// This is because the ErrorJson argument is the endpoint's error definition, which may contain custom fields.
// In such a scenario marhaling the endpoint's error would populate the resulting JSON
// with these fields even if they are not present in the gRPC header.
d := &DefaultErrorJson{
Message: errJson.Msg(),
Code: errJson.StatusCode(),
}
j, err = json.Marshal(d)
if err != nil {
log.WithError(err).Error("Could not marshal error message")
return
}
}
w.Header().Set("Content-Length", strconv.Itoa(len(j)))
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(errJson.StatusCode())
if _, err := io.Copy(w, ioutil.NopCloser(bytes.NewReader(j))); err != nil {
log.WithError(err).Error("Could not write error message")
}
}
// Cleanup performs final cleanup on the initial response from grpc-gateway.
func Cleanup(grpcResponseBody io.ReadCloser) ErrorJson {
if err := grpcResponseBody.Close(); err != nil {
return InternalServerErrorWithMessage(err, "could not close response body")
}
return nil
}

View File

@@ -0,0 +1,409 @@
package apimiddleware
import (
"bytes"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/prysmaticlabs/prysm/api/grpc"
"github.com/prysmaticlabs/prysm/testing/assert"
"github.com/prysmaticlabs/prysm/testing/require"
"github.com/sirupsen/logrus/hooks/test"
)
type testRequestContainer struct {
TestString string
TestHexString string `hex:"true"`
}
func defaultRequestContainer() *testRequestContainer {
return &testRequestContainer{
TestString: "test string",
TestHexString: "0x666F6F", // hex encoding of "foo"
}
}
type testResponseContainer struct {
TestString string
TestHex string `hex:"true"`
TestEnum string `enum:"true"`
TestTime string `time:"true"`
}
func defaultResponseContainer() *testResponseContainer {
return &testResponseContainer{
TestString: "test string",
TestHex: "Zm9v", // base64 encoding of "foo"
TestEnum: "Test Enum",
TestTime: "2006-01-02T15:04:05Z",
}
}
type testErrorJson struct {
Message string
Code int
CustomField string
}
// StatusCode returns the error's underlying error code.
func (e *testErrorJson) StatusCode() int {
return e.Code
}
// Msg returns the error's underlying message.
func (e *testErrorJson) Msg() string {
return e.Message
}
// SetCode sets the error's underlying error code.
func (e *testErrorJson) SetCode(code int) {
e.Code = code
}
// SetMsg sets the error's underlying message.
func (e *testErrorJson) SetMsg(msg string) {
e.Message = msg
}
func TestDeserializeRequestBodyIntoContainer(t *testing.T) {
t.Run("ok", func(t *testing.T) {
var bodyJson bytes.Buffer
err := json.NewEncoder(&bodyJson).Encode(defaultRequestContainer())
require.NoError(t, err)
container := &testRequestContainer{}
errJson := DeserializeRequestBodyIntoContainer(&bodyJson, container)
require.Equal(t, true, errJson == nil)
assert.Equal(t, "test string", container.TestString)
})
t.Run("error", func(t *testing.T) {
var bodyJson bytes.Buffer
bodyJson.Write([]byte("foo"))
errJson := DeserializeRequestBodyIntoContainer(&bodyJson, &testRequestContainer{})
require.NotNil(t, errJson)
assert.Equal(t, true, strings.Contains(errJson.Msg(), "could not decode request body"))
assert.Equal(t, http.StatusInternalServerError, errJson.StatusCode())
})
t.Run("unknown field", func(t *testing.T) {
var bodyJson bytes.Buffer
bodyJson.Write([]byte("{\"foo\":\"foo\"}"))
errJson := DeserializeRequestBodyIntoContainer(&bodyJson, &testRequestContainer{})
require.NotNil(t, errJson)
assert.Equal(t, true, strings.Contains(errJson.Msg(), "could not decode request body"))
assert.Equal(t, http.StatusBadRequest, errJson.StatusCode())
})
}
func TestProcessRequestContainerFields(t *testing.T) {
t.Run("ok", func(t *testing.T) {
container := defaultRequestContainer()
errJson := ProcessRequestContainerFields(container)
require.Equal(t, true, errJson == nil)
assert.Equal(t, "Zm9v", container.TestHexString)
})
t.Run("error", func(t *testing.T) {
errJson := ProcessRequestContainerFields("foo")
require.NotNil(t, errJson)
assert.Equal(t, true, strings.Contains(errJson.Msg(), "could not process request data"))
assert.Equal(t, http.StatusInternalServerError, errJson.StatusCode())
})
}
func TestSetRequestBodyToRequestContainer(t *testing.T) {
var body bytes.Buffer
request := httptest.NewRequest("GET", "http://foo.example", &body)
errJson := SetRequestBodyToRequestContainer(defaultRequestContainer(), request)
require.Equal(t, true, errJson == nil)
container := &testRequestContainer{}
require.NoError(t, json.NewDecoder(request.Body).Decode(container))
assert.Equal(t, "test string", container.TestString)
contentLengthHeader, ok := request.Header["Content-Length"]
require.Equal(t, true, ok)
require.Equal(t, 1, len(contentLengthHeader), "wrong number of header values")
assert.Equal(t, "55", contentLengthHeader[0])
assert.Equal(t, int64(55), request.ContentLength)
}
func TestPrepareRequestForProxying(t *testing.T) {
middleware := &ApiProxyMiddleware{
GatewayAddress: "http://gateway.example",
}
// We will set some params to make the request more interesting.
endpoint := Endpoint{
Path: "/{url_param}",
RequestURLLiterals: []string{"url_param"},
RequestQueryParams: []QueryParam{{Name: "query_param"}},
}
var body bytes.Buffer
request := httptest.NewRequest("GET", "http://foo.example?query_param=bar", &body)
errJson := middleware.PrepareRequestForProxying(endpoint, request)
require.Equal(t, true, errJson == nil)
assert.Equal(t, "http", request.URL.Scheme)
assert.Equal(t, middleware.GatewayAddress, request.URL.Host)
assert.Equal(t, "", request.RequestURI)
}
func TestReadGrpcResponseBody(t *testing.T) {
var b bytes.Buffer
b.Write([]byte("foo"))
body, jsonErr := ReadGrpcResponseBody(&b)
require.Equal(t, true, jsonErr == nil)
assert.Equal(t, "foo", string(body))
}
func TestHandleGrpcResponseError(t *testing.T) {
response := &http.Response{
StatusCode: 400,
Header: http.Header{
"Foo": []string{"foo"},
"Bar": []string{"bar"},
},
}
writer := httptest.NewRecorder()
errJson := &testErrorJson{
Message: "foo",
Code: 400,
}
b, err := json.Marshal(errJson)
require.NoError(t, err)
hasError, e := HandleGrpcResponseError(errJson, response, b, writer)
require.Equal(t, true, e == nil)
assert.Equal(t, true, hasError)
v, ok := writer.Header()["Foo"]
require.Equal(t, true, ok, "header not found")
require.Equal(t, 1, len(v), "wrong number of header values")
assert.Equal(t, "foo", v[0])
v, ok = writer.Header()["Bar"]
require.Equal(t, true, ok, "header not found")
require.Equal(t, 1, len(v), "wrong number of header values")
assert.Equal(t, "bar", v[0])
assert.Equal(t, 400, errJson.StatusCode())
}
func TestGrpcResponseIsEmpty(t *testing.T) {
t.Run("nil", func(t *testing.T) {
assert.Equal(t, true, GrpcResponseIsEmpty(nil))
})
t.Run("empty_slice", func(t *testing.T) {
assert.Equal(t, true, GrpcResponseIsEmpty(make([]byte, 0)))
})
t.Run("empty_brackets", func(t *testing.T) {
assert.Equal(t, true, GrpcResponseIsEmpty([]byte("{}")))
})
t.Run("non_empty", func(t *testing.T) {
assert.Equal(t, false, GrpcResponseIsEmpty([]byte("{\"foo\":\"bar\"})")))
})
}
func TestDeserializeGrpcResponseBodyIntoContainer(t *testing.T) {
t.Run("ok", func(t *testing.T) {
body, err := json.Marshal(defaultRequestContainer())
require.NoError(t, err)
container := &testRequestContainer{}
errJson := DeserializeGrpcResponseBodyIntoContainer(body, container)
require.Equal(t, true, errJson == nil)
assert.Equal(t, "test string", container.TestString)
})
t.Run("error", func(t *testing.T) {
var bodyJson bytes.Buffer
bodyJson.Write([]byte("foo"))
errJson := DeserializeGrpcResponseBodyIntoContainer(bodyJson.Bytes(), &testRequestContainer{})
require.NotNil(t, errJson)
assert.Equal(t, true, strings.Contains(errJson.Msg(), "could not unmarshal response"))
assert.Equal(t, http.StatusInternalServerError, errJson.StatusCode())
})
}
func TestProcessMiddlewareResponseFields(t *testing.T) {
t.Run("Ok", func(t *testing.T) {
container := defaultResponseContainer()
errJson := ProcessMiddlewareResponseFields(container)
require.Equal(t, true, errJson == nil)
assert.Equal(t, "0x666f6f", container.TestHex)
assert.Equal(t, "test enum", container.TestEnum)
assert.Equal(t, "1136214245", container.TestTime)
})
t.Run("error", func(t *testing.T) {
errJson := ProcessMiddlewareResponseFields("foo")
require.NotNil(t, errJson)
assert.Equal(t, true, strings.Contains(errJson.Msg(), "could not process response data"))
assert.Equal(t, http.StatusInternalServerError, errJson.StatusCode())
})
}
func TestSerializeMiddlewareResponseIntoJson(t *testing.T) {
container := defaultResponseContainer()
j, errJson := SerializeMiddlewareResponseIntoJson(container)
assert.Equal(t, true, errJson == nil)
cToDeserialize := &testResponseContainer{}
require.NoError(t, json.Unmarshal(j, cToDeserialize))
assert.Equal(t, "test string", cToDeserialize.TestString)
}
func TestWriteMiddlewareResponseHeadersAndBody(t *testing.T) {
t.Run("GET", func(t *testing.T) {
response := &http.Response{
Header: http.Header{
"Foo": []string{"foo"},
"Grpc-Metadata-" + grpc.HttpCodeMetadataKey: []string{"204"},
},
}
container := defaultResponseContainer()
responseJson, err := json.Marshal(container)
require.NoError(t, err)
writer := httptest.NewRecorder()
writer.Body = &bytes.Buffer{}
errJson := WriteMiddlewareResponseHeadersAndBody(response, responseJson, writer)
require.Equal(t, true, errJson == nil)
v, ok := writer.Header()["Foo"]
require.Equal(t, true, ok, "header not found")
require.Equal(t, 1, len(v), "wrong number of header values")
assert.Equal(t, "foo", v[0])
v, ok = writer.Header()["Content-Length"]
require.Equal(t, true, ok, "header not found")
require.Equal(t, 1, len(v), "wrong number of header values")
assert.Equal(t, "102", v[0])
assert.Equal(t, 204, writer.Code)
assert.DeepEqual(t, responseJson, writer.Body.Bytes())
})
t.Run("GET_no_grpc_status_code_header", func(t *testing.T) {
response := &http.Response{
Header: http.Header{},
StatusCode: 204,
}
container := defaultResponseContainer()
responseJson, err := json.Marshal(container)
require.NoError(t, err)
writer := httptest.NewRecorder()
errJson := WriteMiddlewareResponseHeadersAndBody(response, responseJson, writer)
require.Equal(t, true, errJson == nil)
assert.Equal(t, 204, writer.Code)
})
t.Run("GET_invalid_status_code", func(t *testing.T) {
response := &http.Response{
Header: http.Header{},
}
// Set invalid status code.
response.Header["Grpc-Metadata-"+grpc.HttpCodeMetadataKey] = []string{"invalid"}
container := defaultResponseContainer()
responseJson, err := json.Marshal(container)
require.NoError(t, err)
writer := httptest.NewRecorder()
errJson := WriteMiddlewareResponseHeadersAndBody(response, responseJson, writer)
require.Equal(t, false, errJson == nil)
assert.Equal(t, true, strings.Contains(errJson.Msg(), "could not parse status code"))
assert.Equal(t, http.StatusInternalServerError, errJson.StatusCode())
})
t.Run("POST", func(t *testing.T) {
response := &http.Response{
Header: http.Header{},
StatusCode: 204,
}
container := defaultResponseContainer()
responseJson, err := json.Marshal(container)
require.NoError(t, err)
writer := httptest.NewRecorder()
errJson := WriteMiddlewareResponseHeadersAndBody(response, responseJson, writer)
require.Equal(t, true, errJson == nil)
assert.Equal(t, 204, writer.Code)
})
t.Run("POST_with_response_body", func(t *testing.T) {
response := &http.Response{
Header: http.Header{},
StatusCode: 204,
}
container := defaultResponseContainer()
responseJson, err := json.Marshal(container)
require.NoError(t, err)
writer := httptest.NewRecorder()
writer.Body = &bytes.Buffer{}
errJson := WriteMiddlewareResponseHeadersAndBody(response, responseJson, writer)
require.Equal(t, true, errJson == nil)
assert.Equal(t, 204, writer.Code)
assert.DeepEqual(t, responseJson, writer.Body.Bytes())
})
t.Run("POST_with_empty_json_body", func(t *testing.T) {
response := &http.Response{
Header: http.Header{},
StatusCode: 204,
}
responseJson, err := json.Marshal(struct{}{})
require.NoError(t, err)
writer := httptest.NewRecorder()
writer.Body = &bytes.Buffer{}
errJson := WriteMiddlewareResponseHeadersAndBody(response, responseJson, writer)
require.Equal(t, true, errJson == nil)
assert.Equal(t, 204, writer.Code)
assert.DeepEqual(t, []byte(nil), writer.Body.Bytes())
assert.Equal(t, "0", writer.Header()["Content-Length"][0])
})
}
func TestWriteError(t *testing.T) {
t.Run("ok", func(t *testing.T) {
responseHeader := http.Header{
"Grpc-Metadata-" + grpc.CustomErrorMetadataKey: []string{"{\"CustomField\":\"bar\"}"},
}
errJson := &testErrorJson{
Message: "foo",
Code: 500,
}
writer := httptest.NewRecorder()
writer.Body = &bytes.Buffer{}
WriteError(writer, errJson, responseHeader)
v, ok := writer.Header()["Content-Length"]
require.Equal(t, true, ok, "header not found")
require.Equal(t, 1, len(v), "wrong number of header values")
assert.Equal(t, "48", v[0])
v, ok = writer.Header()["Content-Type"]
require.Equal(t, true, ok, "header not found")
require.Equal(t, 1, len(v), "wrong number of header values")
assert.Equal(t, "application/json", v[0])
assert.Equal(t, 500, writer.Code)
eDeserialize := &testErrorJson{}
require.NoError(t, json.Unmarshal(writer.Body.Bytes(), eDeserialize))
assert.Equal(t, "foo", eDeserialize.Message)
assert.Equal(t, 500, eDeserialize.Code)
assert.Equal(t, "bar", eDeserialize.CustomField)
})
t.Run("invalid_custom_error_header", func(t *testing.T) {
logHook := test.NewGlobal()
responseHeader := http.Header{
"Grpc-Metadata-" + grpc.CustomErrorMetadataKey: []string{"invalid"},
}
WriteError(httptest.NewRecorder(), &testErrorJson{}, responseHeader)
assert.LogsContain(t, logHook, "Could not unmarshal custom error message")
})
}

View File

@@ -0,0 +1,62 @@
package apimiddleware
import (
"net/http"
"github.com/pkg/errors"
)
// ---------------
// Error handling.
// ---------------
// ErrorJson describes common functionality of all JSON error representations.
type ErrorJson interface {
StatusCode() int
SetCode(code int)
Msg() string
SetMsg(msg string)
}
// DefaultErrorJson is a JSON representation of a simple error value, containing only a message and an error code.
type DefaultErrorJson struct {
Message string `json:"message"`
Code int `json:"code"`
}
// InternalServerErrorWithMessage returns a DefaultErrorJson with 500 code and a custom message.
func InternalServerErrorWithMessage(err error, message string) *DefaultErrorJson {
e := errors.Wrapf(err, message)
return &DefaultErrorJson{
Message: e.Error(),
Code: http.StatusInternalServerError,
}
}
// InternalServerError returns a DefaultErrorJson with 500 code.
func InternalServerError(err error) *DefaultErrorJson {
return &DefaultErrorJson{
Message: err.Error(),
Code: http.StatusInternalServerError,
}
}
// StatusCode returns the error's underlying error code.
func (e *DefaultErrorJson) StatusCode() int {
return e.Code
}
// Msg returns the error's underlying message.
func (e *DefaultErrorJson) Msg() string {
return e.Message
}
// SetCode sets the error's underlying error code.
func (e *DefaultErrorJson) SetCode(code int) {
e.Code = code
}
// SetMsg sets the error's underlying message.
func (e *DefaultErrorJson) SetMsg(msg string) {
e.Message = msg
}

254
api/gateway/gateway.go Normal file
View File

@@ -0,0 +1,254 @@
// Package gateway defines a grpc-gateway server that serves HTTP-JSON traffic and acts a proxy between HTTP and gRPC.
package gateway
import (
"context"
"fmt"
"net"
"net/http"
"path"
"strings"
"time"
"github.com/gorilla/mux"
gwruntime "github.com/grpc-ecosystem/grpc-gateway/v2/runtime"
"github.com/pkg/errors"
"github.com/prysmaticlabs/prysm/api/gateway/apimiddleware"
"github.com/prysmaticlabs/prysm/runtime"
"github.com/rs/cors"
"google.golang.org/grpc"
"google.golang.org/grpc/connectivity"
"google.golang.org/grpc/credentials"
)
var _ runtime.Service = (*Gateway)(nil)
// PbMux serves grpc-gateway requests for selected patterns using registered protobuf handlers.
type PbMux struct {
Registrations []PbHandlerRegistration // Protobuf registrations to be registered in Mux.
Patterns []string // URL patterns that will be handled by Mux.
Mux *gwruntime.ServeMux // The router that will be used for grpc-gateway requests.
}
// PbHandlerRegistration is a function that registers a protobuf handler.
type PbHandlerRegistration func(context.Context, *gwruntime.ServeMux, *grpc.ClientConn) error
// MuxHandler is a function that implements the mux handler functionality.
type MuxHandler func(
apiMiddlewareHandler *apimiddleware.ApiProxyMiddleware,
h http.HandlerFunc,
w http.ResponseWriter,
req *http.Request,
)
// Config parameters for setting up the gateway service.
type config struct {
maxCallRecvMsgSize uint64
remoteCert string
gatewayAddr string
remoteAddr string
allowedOrigins []string
apiMiddlewareEndpointFactory apimiddleware.EndpointFactory
muxHandler MuxHandler
pbHandlers []*PbMux
router *mux.Router
}
// Gateway is the gRPC gateway to serve HTTP JSON traffic as a proxy and forward it to the gRPC server.
type Gateway struct {
cfg *config
conn *grpc.ClientConn
server *http.Server
cancel context.CancelFunc
proxy *apimiddleware.ApiProxyMiddleware
ctx context.Context
startFailure error
}
// New returns a new instance of the Gateway.
func New(ctx context.Context, opts ...Option) (*Gateway, error) {
g := &Gateway{
ctx: ctx,
cfg: &config{
router: mux.NewRouter(),
},
}
for _, opt := range opts {
if err := opt(g); err != nil {
return nil, err
}
}
return g, nil
}
// Start the gateway service.
func (g *Gateway) Start() {
ctx, cancel := context.WithCancel(g.ctx)
g.cancel = cancel
conn, err := g.dial(ctx, "tcp", g.cfg.remoteAddr)
if err != nil {
log.WithError(err).Error("Failed to connect to gRPC server")
g.startFailure = err
return
}
g.conn = conn
for _, h := range g.cfg.pbHandlers {
for _, r := range h.Registrations {
if err := r(ctx, h.Mux, g.conn); err != nil {
log.WithError(err).Error("Failed to register handler")
g.startFailure = err
return
}
}
for _, p := range h.Patterns {
g.cfg.router.PathPrefix(p).Handler(h.Mux)
}
}
corsMux := g.corsMiddleware(g.cfg.router)
if g.cfg.apiMiddlewareEndpointFactory != nil && !g.cfg.apiMiddlewareEndpointFactory.IsNil() {
g.registerApiMiddleware()
}
if g.cfg.muxHandler != nil {
g.cfg.router.PathPrefix("/").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
g.cfg.muxHandler(g.proxy, corsMux.ServeHTTP, w, r)
})
}
g.server = &http.Server{
Addr: g.cfg.gatewayAddr,
Handler: corsMux,
}
go func() {
log.WithField("address", g.cfg.gatewayAddr).Info("Starting gRPC gateway")
if err := g.server.ListenAndServe(); err != http.ErrServerClosed {
log.WithError(err).Error("Failed to start gRPC gateway")
g.startFailure = err
return
}
}()
}
// Status of grpc gateway. Returns an error if this service is unhealthy.
func (g *Gateway) Status() error {
if g.startFailure != nil {
return g.startFailure
}
if s := g.conn.GetState(); s != connectivity.Ready {
return fmt.Errorf("grpc server is %s", s)
}
return nil
}
// Stop the gateway with a graceful shutdown.
func (g *Gateway) Stop() error {
if g.server != nil {
shutdownCtx, shutdownCancel := context.WithTimeout(g.ctx, 2*time.Second)
defer shutdownCancel()
if err := g.server.Shutdown(shutdownCtx); err != nil {
if errors.Is(err, context.DeadlineExceeded) {
log.Warn("Existing connections terminated")
} else {
log.WithError(err).Error("Failed to gracefully shut down server")
}
}
}
if g.cancel != nil {
g.cancel()
}
return nil
}
func (g *Gateway) corsMiddleware(h http.Handler) http.Handler {
c := cors.New(cors.Options{
AllowedOrigins: g.cfg.allowedOrigins,
AllowedMethods: []string{http.MethodPost, http.MethodGet, http.MethodDelete, http.MethodOptions},
AllowCredentials: true,
MaxAge: 600,
AllowedHeaders: []string{"*"},
})
return c.Handler(h)
}
const swaggerDir = "proto/prysm/v1alpha1/"
// SwaggerServer returns swagger specification files located under "/swagger/"
func SwaggerServer() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
if !strings.HasSuffix(r.URL.Path, ".swagger.json") {
log.Debugf("Not found: %s", r.URL.Path)
http.NotFound(w, r)
return
}
log.Debugf("Serving %s\n", r.URL.Path)
p := strings.TrimPrefix(r.URL.Path, "/swagger/")
p = path.Join(swaggerDir, p)
http.ServeFile(w, r, p)
}
}
// dial the gRPC server.
func (g *Gateway) dial(ctx context.Context, network, addr string) (*grpc.ClientConn, error) {
switch network {
case "tcp":
return g.dialTCP(ctx, addr)
case "unix":
return g.dialUnix(ctx, addr)
default:
return nil, fmt.Errorf("unsupported network type %q", network)
}
}
// dialTCP creates a client connection via TCP.
// "addr" must be a valid TCP address with a port number.
func (g *Gateway) dialTCP(ctx context.Context, addr string) (*grpc.ClientConn, error) {
security := grpc.WithInsecure()
if len(g.cfg.remoteCert) > 0 {
creds, err := credentials.NewClientTLSFromFile(g.cfg.remoteCert, "")
if err != nil {
return nil, err
}
security = grpc.WithTransportCredentials(creds)
}
opts := []grpc.DialOption{
security,
grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(int(g.cfg.maxCallRecvMsgSize))),
}
return grpc.DialContext(ctx, addr, opts...)
}
// dialUnix creates a client connection via a unix domain socket.
// "addr" must be a valid path to the socket.
func (g *Gateway) dialUnix(ctx context.Context, addr string) (*grpc.ClientConn, error) {
d := func(addr string, timeout time.Duration) (net.Conn, error) {
return net.DialTimeout("unix", addr, timeout)
}
f := func(ctx context.Context, addr string) (net.Conn, error) {
if deadline, ok := ctx.Deadline(); ok {
return d(addr, time.Until(deadline))
}
return d(addr, 0)
}
opts := []grpc.DialOption{
grpc.WithInsecure(),
grpc.WithContextDialer(f),
grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(int(g.cfg.maxCallRecvMsgSize))),
}
return grpc.DialContext(ctx, addr, opts...)
}
func (g *Gateway) registerApiMiddleware() {
g.proxy = &apimiddleware.ApiProxyMiddleware{
GatewayAddress: g.cfg.gatewayAddr,
EndpointCreator: g.cfg.apiMiddlewareEndpointFactory,
}
log.Info("Starting API middleware")
g.proxy.Run(g.cfg.router)
}

128
api/gateway/gateway_test.go Normal file
View File

@@ -0,0 +1,128 @@
package gateway
import (
"context"
"flag"
"fmt"
"net/http"
"net/http/httptest"
"net/url"
"testing"
"github.com/gorilla/mux"
"github.com/prysmaticlabs/prysm/api/gateway/apimiddleware"
"github.com/prysmaticlabs/prysm/cmd/beacon-chain/flags"
"github.com/prysmaticlabs/prysm/testing/assert"
"github.com/prysmaticlabs/prysm/testing/require"
logTest "github.com/sirupsen/logrus/hooks/test"
"github.com/urfave/cli/v2"
)
type mockEndpointFactory struct {
}
func (*mockEndpointFactory) Paths() []string {
return []string{}
}
func (*mockEndpointFactory) Create(_ string) (*apimiddleware.Endpoint, error) {
return nil, nil
}
func (*mockEndpointFactory) IsNil() bool {
return false
}
func TestGateway_Customized(t *testing.T) {
r := mux.NewRouter()
cert := "cert"
origins := []string{"origin"}
size := uint64(100)
endpointFactory := &mockEndpointFactory{}
opts := []Option{
WithRouter(r),
WithRemoteCert(cert),
WithAllowedOrigins(origins),
WithMaxCallRecvMsgSize(size),
WithApiMiddleware(endpointFactory),
WithMuxHandler(func(
_ *apimiddleware.ApiProxyMiddleware,
_ http.HandlerFunc,
_ http.ResponseWriter,
_ *http.Request,
) {
}),
}
g, err := New(context.Background(), opts...)
require.NoError(t, err)
assert.Equal(t, r, g.cfg.router)
assert.Equal(t, cert, g.cfg.remoteCert)
require.Equal(t, 1, len(g.cfg.allowedOrigins))
assert.Equal(t, origins[0], g.cfg.allowedOrigins[0])
assert.Equal(t, size, g.cfg.maxCallRecvMsgSize)
assert.Equal(t, endpointFactory, g.cfg.apiMiddlewareEndpointFactory)
}
func TestGateway_StartStop(t *testing.T) {
hook := logTest.NewGlobal()
app := cli.App{}
set := flag.NewFlagSet("test", 0)
ctx := cli.NewContext(&app, set, nil)
gatewayPort := ctx.Int(flags.GRPCGatewayPort.Name)
gatewayHost := ctx.String(flags.GRPCGatewayHost.Name)
rpcHost := ctx.String(flags.RPCHost.Name)
selfAddress := fmt.Sprintf("%s:%d", rpcHost, ctx.Int(flags.RPCPort.Name))
gatewayAddress := fmt.Sprintf("%s:%d", gatewayHost, gatewayPort)
opts := []Option{
WithGatewayAddr(gatewayAddress),
WithRemoteAddr(selfAddress),
WithMuxHandler(func(
_ *apimiddleware.ApiProxyMiddleware,
_ http.HandlerFunc,
_ http.ResponseWriter,
_ *http.Request,
) {
}),
}
g, err := New(context.Background(), opts...)
require.NoError(t, err)
g.Start()
go func() {
require.LogsContain(t, hook, "Starting gRPC gateway")
require.LogsDoNotContain(t, hook, "Starting API middleware")
}()
err = g.Stop()
require.NoError(t, err)
}
func TestGateway_NilHandler_NotFoundHandlerRegistered(t *testing.T) {
app := cli.App{}
set := flag.NewFlagSet("test", 0)
ctx := cli.NewContext(&app, set, nil)
gatewayPort := ctx.Int(flags.GRPCGatewayPort.Name)
gatewayHost := ctx.String(flags.GRPCGatewayHost.Name)
rpcHost := ctx.String(flags.RPCHost.Name)
selfAddress := fmt.Sprintf("%s:%d", rpcHost, ctx.Int(flags.RPCPort.Name))
gatewayAddress := fmt.Sprintf("%s:%d", gatewayHost, gatewayPort)
opts := []Option{
WithGatewayAddr(gatewayAddress),
WithRemoteAddr(selfAddress),
}
g, err := New(context.Background(), opts...)
require.NoError(t, err)
writer := httptest.NewRecorder()
g.cfg.router.ServeHTTP(writer, &http.Request{Method: "GET", Host: "localhost", URL: &url.URL{Path: "/foo"}})
assert.Equal(t, http.StatusNotFound, writer.Code)
}

81
api/gateway/options.go Normal file
View File

@@ -0,0 +1,81 @@
package gateway
import (
"github.com/gorilla/mux"
"github.com/prysmaticlabs/prysm/api/gateway/apimiddleware"
)
type Option func(g *Gateway) error
func (g *Gateway) SetRouter(r *mux.Router) *Gateway {
g.cfg.router = r
return g
}
func WithPbHandlers(handlers []*PbMux) Option {
return func(g *Gateway) error {
g.cfg.pbHandlers = handlers
return nil
}
}
func WithMuxHandler(m MuxHandler) Option {
return func(g *Gateway) error {
g.cfg.muxHandler = m
return nil
}
}
func WithGatewayAddr(addr string) Option {
return func(g *Gateway) error {
g.cfg.gatewayAddr = addr
return nil
}
}
func WithRemoteAddr(addr string) Option {
return func(g *Gateway) error {
g.cfg.remoteAddr = addr
return nil
}
}
// WithRouter allows adding a custom mux router to the gateway.
func WithRouter(r *mux.Router) Option {
return func(g *Gateway) error {
g.cfg.router = r
return nil
}
}
// WithAllowedOrigins allows adding a set of allowed origins to the gateway.
func WithAllowedOrigins(origins []string) Option {
return func(g *Gateway) error {
g.cfg.allowedOrigins = origins
return nil
}
}
// WithRemoteCert allows adding a custom certificate to the gateway,
func WithRemoteCert(cert string) Option {
return func(g *Gateway) error {
g.cfg.remoteCert = cert
return nil
}
}
// WithMaxCallRecvMsgSize allows specifying the maximum allowed gRPC message size.
func WithMaxCallRecvMsgSize(size uint64) Option {
return func(g *Gateway) error {
g.cfg.maxCallRecvMsgSize = size
return nil
}
}
// WithApiMiddleware allows adding an API middleware proxy to the gateway.
func WithApiMiddleware(endpointFactory apimiddleware.EndpointFactory) Option {
return func(g *Gateway) error {
g.cfg.apiMiddlewareEndpointFactory = endpointFactory
return nil
}
}

30
api/grpc/BUILD.bazel Normal file
View File

@@ -0,0 +1,30 @@
load("@prysm//tools/go:def.bzl", "go_library", "go_test")
go_library(
name = "go_default_library",
srcs = [
"grpcutils.go",
"parameters.go",
],
importpath = "github.com/prysmaticlabs/prysm/api/grpc",
visibility = ["//visibility:public"],
deps = [
"@com_github_sirupsen_logrus//:go_default_library",
"@org_golang_google_grpc//:go_default_library",
"@org_golang_google_grpc//metadata:go_default_library",
],
)
go_test(
name = "go_default_test",
srcs = ["grpcutils_test.go"],
embed = [":go_default_library"],
deps = [
"//testing/assert:go_default_library",
"//testing/require:go_default_library",
"@com_github_grpc_ecosystem_grpc_gateway_v2//runtime:go_default_library",
"@com_github_sirupsen_logrus//hooks/test:go_default_library",
"@org_golang_google_grpc//:go_default_library",
"@org_golang_google_grpc//metadata:go_default_library",
],
)

View File

@@ -1,7 +1,9 @@
package grpcutils
package grpc
import (
"context"
"encoding/json"
"fmt"
"strings"
"time"
@@ -79,3 +81,16 @@ func AppendHeaders(parent context.Context, headers []string) context.Context {
}
return parent
}
// AppendCustomErrorHeader sets a CustomErrorMetadataKey gRPC header on the passed in context,
// using the passed in error data as the header's value. The data is serialized as JSON.
func AppendCustomErrorHeader(ctx context.Context, errorData interface{}) error {
j, err := json.Marshal(errorData)
if err != nil {
return fmt.Errorf("could not marshal error data into JSON: %w", err)
}
if err := grpc.SetHeader(ctx, metadata.Pairs(CustomErrorMetadataKey, string(j))); err != nil {
return fmt.Errorf("could not set custom error header: %w", err)
}
return nil
}

View File

@@ -0,0 +1,78 @@
package grpc
import (
"context"
"encoding/json"
"strings"
"testing"
"github.com/grpc-ecosystem/grpc-gateway/v2/runtime"
"github.com/prysmaticlabs/prysm/testing/assert"
"github.com/prysmaticlabs/prysm/testing/require"
logTest "github.com/sirupsen/logrus/hooks/test"
"google.golang.org/grpc"
"google.golang.org/grpc/metadata"
)
type customErrorData struct {
Message string `json:"message"`
}
func TestAppendHeaders(t *testing.T) {
t.Run("one_header", func(t *testing.T) {
ctx := AppendHeaders(context.Background(), []string{"first=value1"})
md, ok := metadata.FromOutgoingContext(ctx)
require.Equal(t, true, ok, "Failed to read context metadata")
require.Equal(t, 1, md.Len(), "MetadataV0 contains wrong number of values")
assert.Equal(t, "value1", md.Get("first")[0])
})
t.Run("multiple_headers", func(t *testing.T) {
ctx := AppendHeaders(context.Background(), []string{"first=value1", "second=value2"})
md, ok := metadata.FromOutgoingContext(ctx)
require.Equal(t, true, ok, "Failed to read context metadata")
require.Equal(t, 2, md.Len(), "MetadataV0 contains wrong number of values")
assert.Equal(t, "value1", md.Get("first")[0])
assert.Equal(t, "value2", md.Get("second")[0])
})
t.Run("one_empty_header", func(t *testing.T) {
ctx := AppendHeaders(context.Background(), []string{"first=value1", ""})
md, ok := metadata.FromOutgoingContext(ctx)
require.Equal(t, true, ok, "Failed to read context metadata")
require.Equal(t, 1, md.Len(), "MetadataV0 contains wrong number of values")
assert.Equal(t, "value1", md.Get("first")[0])
})
t.Run("incorrect_header", func(t *testing.T) {
logHook := logTest.NewGlobal()
ctx := AppendHeaders(context.Background(), []string{"first=value1", "second"})
md, ok := metadata.FromOutgoingContext(ctx)
require.Equal(t, true, ok, "Failed to read context metadata")
require.Equal(t, 1, md.Len(), "MetadataV0 contains wrong number of values")
assert.Equal(t, "value1", md.Get("first")[0])
assert.LogsContain(t, logHook, "Skipping second")
})
t.Run("header_value_with_equal_sign", func(t *testing.T) {
ctx := AppendHeaders(context.Background(), []string{"first=value=1"})
md, ok := metadata.FromOutgoingContext(ctx)
require.Equal(t, true, ok, "Failed to read context metadata")
require.Equal(t, 1, md.Len(), "MetadataV0 contains wrong number of values")
assert.Equal(t, "value=1", md.Get("first")[0])
})
}
func TestAppendCustomErrorHeader(t *testing.T) {
stream := &runtime.ServerTransportStream{}
ctx := grpc.NewContextWithServerTransportStream(context.Background(), stream)
data := &customErrorData{Message: "foo"}
require.NoError(t, AppendCustomErrorHeader(ctx, data))
// The stream used in test setup sets the metadata key in lowercase.
value, ok := stream.Header()[strings.ToLower(CustomErrorMetadataKey)]
require.Equal(t, true, ok, "Failed to retrieve custom error metadata value")
expected, err := json.Marshal(data)
require.NoError(t, err)
assert.Equal(t, string(expected), value[0])
}

8
api/grpc/parameters.go Normal file
View File

@@ -0,0 +1,8 @@
package grpc
// CustomErrorMetadataKey is the name of the metadata key storing additional error information.
// Metadata value is expected to be a byte-encoded JSON object.
const CustomErrorMetadataKey = "Custom-Error"
// HttpCodeMetadataKey is the key to use when setting custom HTTP status codes in gRPC metadata.
const HttpCodeMetadataKey = "X-Http-Code"

View File

@@ -0,0 +1,22 @@
load("@prysm//tools/go:def.bzl", "go_library", "go_test")
go_library(
name = "go_default_library",
srcs = ["pagination.go"],
importpath = "github.com/prysmaticlabs/prysm/api/pagination",
visibility = ["//visibility:public"],
deps = [
"//config/params:go_default_library",
"@com_github_pkg_errors//:go_default_library",
],
)
go_test(
name = "go_default_test",
srcs = ["pagination_test.go"],
deps = [
":go_default_library",
"//testing/assert:go_default_library",
"//testing/require:go_default_library",
],
)

View File

@@ -6,7 +6,7 @@ import (
"strconv"
"github.com/pkg/errors"
"github.com/prysmaticlabs/prysm/shared/params"
"github.com/prysmaticlabs/prysm/config/params"
)
// StartAndEndPage takes in the requested page token, wanted page size, total page size.

View File

@@ -3,9 +3,9 @@ package pagination_test
import (
"testing"
"github.com/prysmaticlabs/prysm/shared/pagination"
"github.com/prysmaticlabs/prysm/shared/testutil/assert"
"github.com/prysmaticlabs/prysm/shared/testutil/require"
"github.com/prysmaticlabs/prysm/api/pagination"
"github.com/prysmaticlabs/prysm/testing/assert"
"github.com/prysmaticlabs/prysm/testing/require"
)
func TestStartAndEndPage(t *testing.T) {

33
async/BUILD.bazel Normal file
View File

@@ -0,0 +1,33 @@
load("@prysm//tools/go:def.bzl", "go_library", "go_test")
go_library(
name = "go_default_library",
srcs = [
"debounce.go",
"every.go",
"multilock.go",
"scatter.go",
],
importpath = "github.com/prysmaticlabs/prysm/async",
visibility = ["//visibility:public"],
deps = ["@com_github_sirupsen_logrus//:go_default_library"],
)
go_test(
name = "go_default_test",
srcs = [
"benchmark_test.go",
"debounce_test.go",
"every_test.go",
"multilock_test.go",
"scatter_test.go",
],
embed = [":go_default_library"],
deps = [
"//testing/assert:go_default_library",
"//testing/require:go_default_library",
"//testing/util:go_default_library",
"@com_github_sirupsen_logrus//:go_default_library",
"@com_github_stretchr_testify//assert:go_default_library",
],
)

14
async/abool/BUILD.bazel Normal file
View File

@@ -0,0 +1,14 @@
load("@prysm//tools/go:def.bzl", "go_library", "go_test")
go_library(
name = "go_default_library",
srcs = ["abool.go"],
importpath = "github.com/prysmaticlabs/prysm/async/abool",
visibility = ["//visibility:public"],
)
go_test(
name = "go_default_test",
srcs = ["abool_test.go"],
embed = [":go_default_library"],
)

63
async/benchmark_test.go Normal file
View File

@@ -0,0 +1,63 @@
package async_test
import (
"crypto/rand"
"crypto/sha256"
"sync"
"testing"
"github.com/prysmaticlabs/prysm/async"
"github.com/prysmaticlabs/prysm/testing/require"
log "github.com/sirupsen/logrus"
)
var input [][]byte
const (
benchmarkElements = 65536
benchmarkElementSize = 32
benchmarkHashRuns = 128
)
func init() {
input = make([][]byte, benchmarkElements)
for i := 0; i < benchmarkElements; i++ {
input[i] = make([]byte, benchmarkElementSize)
_, err := rand.Read(input[i])
if err != nil {
log.WithError(err).Debug("Cannot read from rand")
}
}
}
// hash repeatedly hashes the data passed to it
func hash(input [][]byte) [][]byte {
output := make([][]byte, len(input))
for i := range input {
copy(output, input)
for j := 0; j < benchmarkHashRuns; j++ {
hash := sha256.Sum256(output[i])
output[i] = hash[:]
}
}
return output
}
func BenchmarkHash(b *testing.B) {
for i := 0; i < b.N; i++ {
hash(input)
}
}
func BenchmarkHashMP(b *testing.B) {
output := make([][]byte, len(input))
for i := 0; i < b.N; i++ {
workerResults, err := async.Scatter(len(input), func(offset int, entries int, _ *sync.RWMutex) (interface{}, error) {
return hash(input[offset : offset+entries]), nil
})
require.NoError(b, err)
for _, result := range workerResults {
copy(output[result.Offset:], result.Extent.([][]byte))
}
}
}

View File

@@ -1,4 +1,4 @@
package asyncutil
package async
import (
"context"

View File

@@ -1,4 +1,4 @@
package asyncutil
package async_test
import (
"context"
@@ -6,9 +6,10 @@ import (
"testing"
"time"
"github.com/prysmaticlabs/prysm/shared/testutil"
"github.com/prysmaticlabs/prysm/shared/testutil/assert"
"github.com/prysmaticlabs/prysm/shared/testutil/require"
"github.com/prysmaticlabs/prysm/async"
"github.com/prysmaticlabs/prysm/testing/assert"
"github.com/prysmaticlabs/prysm/testing/require"
"github.com/prysmaticlabs/prysm/testing/util"
)
func TestDebounce_NoEvents(t *testing.T) {
@@ -24,12 +25,12 @@ func TestDebounce_NoEvents(t *testing.T) {
})
}()
go func() {
Debounce(ctx, interval, eventsChan, func(event interface{}) {
async.Debounce(ctx, interval, eventsChan, func(event interface{}) {
timesHandled++
})
wg.Done()
}()
if testutil.WaitTimeout(wg, interval*2) {
if util.WaitTimeout(wg, interval*2) {
t.Fatalf("Test should have exited by now, timed out")
}
assert.Equal(t, 0, timesHandled, "Wrong number of handled calls")
@@ -60,12 +61,12 @@ func TestDebounce_CtxClosing(t *testing.T) {
})
}()
go func() {
Debounce(ctx, interval, eventsChan, func(event interface{}) {
async.Debounce(ctx, interval, eventsChan, func(event interface{}) {
timesHandled++
})
wg.Done()
}()
if testutil.WaitTimeout(wg, interval*2) {
if util.WaitTimeout(wg, interval*2) {
t.Fatalf("Test should have exited by now, timed out")
}
assert.Equal(t, 0, timesHandled, "Wrong number of handled calls")
@@ -76,7 +77,7 @@ func TestDebounce_SingleHandlerInvocation(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
interval := time.Second
timesHandled := 0
go Debounce(ctx, interval, eventsChan, func(event interface{}) {
go async.Debounce(ctx, interval, eventsChan, func(event interface{}) {
timesHandled++
})
for i := 0; i < 100; i++ {
@@ -94,7 +95,7 @@ func TestDebounce_MultipleHandlerInvocation(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
interval := time.Second
timesHandled := 0
go Debounce(ctx, interval, eventsChan, func(event interface{}) {
go async.Debounce(ctx, interval, eventsChan, func(event interface{}) {
timesHandled++
})
for i := 0; i < 100; i++ {

29
async/event/BUILD.bazel Normal file
View File

@@ -0,0 +1,29 @@
load("@prysm//tools/go:def.bzl", "go_library", "go_test")
go_library(
name = "go_default_library",
srcs = [
"feed.go",
"subscription.go",
],
importpath = "github.com/prysmaticlabs/prysm/async/event",
visibility = ["//visibility:public"],
deps = ["//time/mclock:go_default_library"],
)
go_test(
name = "go_default_test",
size = "small",
srcs = [
"example_feed_test.go",
"example_scope_test.go",
"example_subscription_test.go",
"feed_test.go",
"subscription_test.go",
],
embed = [":go_default_library"],
deps = [
"//testing/assert:go_default_library",
"//testing/require:go_default_library",
],
)

View File

@@ -19,7 +19,7 @@ package event_test
import (
"fmt"
"github.com/prysmaticlabs/prysm/shared/event"
"github.com/prysmaticlabs/prysm/async/event"
)
func ExampleFeed_acknowledgedEvents() {

View File

@@ -20,7 +20,7 @@ import (
"fmt"
"sync"
"github.com/prysmaticlabs/prysm/shared/event"
"github.com/prysmaticlabs/prysm/async/event"
)
// This example demonstrates how SubscriptionScope can be used to control the lifetime of

View File

@@ -19,7 +19,7 @@ package event_test
import (
"fmt"
"github.com/prysmaticlabs/prysm/shared/event"
"github.com/prysmaticlabs/prysm/async/event"
)
func ExampleNewSubscription() {

View File

@@ -98,6 +98,11 @@ func (f *Feed) typecheck(typ reflect.Type) bool {
f.etype = typ
return true
}
// In the event the feed's type is an actual interface, we
// perform an interface conformance check here.
if f.etype.Kind() == reflect.Interface && typ.Implements(f.etype) {
return true
}
return f.etype == typ
}

507
async/event/feed_test.go Normal file
View File

@@ -0,0 +1,507 @@
// Copyright 2016 The go-ethereum Authors
// This file is part of the go-ethereum library.
//
// The go-ethereum library is free software: you can redistribute it and/or modify
// it under the terms of the GNU Lesser General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// The go-ethereum library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public License
// along with the go-ethereum library. If not, see <http://www.gnu.org/licenses/>.
package event
import (
"fmt"
"reflect"
"sync"
"testing"
"time"
"github.com/prysmaticlabs/prysm/testing/assert"
)
func TestFeedPanics(t *testing.T) {
{
var f Feed
f.Send(2)
want := feedTypeError{op: "Send", got: reflect.TypeOf(uint64(0)), want: reflect.TypeOf(0)}
assert.NoError(t, checkPanic(want, func() { f.Send(uint64(2)) }))
}
{
var f Feed
ch := make(chan int)
f.Subscribe(ch)
want := feedTypeError{op: "Send", got: reflect.TypeOf(uint64(0)), want: reflect.TypeOf(0)}
assert.NoError(t, checkPanic(want, func() { f.Send(uint64(2)) }))
}
{
var f Feed
f.Send(2)
want := feedTypeError{op: "Subscribe", got: reflect.TypeOf(make(chan uint64)), want: reflect.TypeOf(make(chan<- int))}
assert.NoError(t, checkPanic(want, func() { f.Subscribe(make(chan uint64)) }))
}
{
var f Feed
assert.NoError(t, checkPanic(errBadChannel, func() { f.Subscribe(make(<-chan int)) }))
}
{
var f Feed
assert.NoError(t, checkPanic(errBadChannel, func() { f.Subscribe(0) }))
}
}
func checkPanic(want error, fn func()) (err error) {
defer func() {
panicResult := recover()
if panicResult == nil {
err = fmt.Errorf("didn't panic")
} else if !reflect.DeepEqual(panicResult, want) {
err = fmt.Errorf("panicked with wrong error: got %q, want %q", panicResult, want)
}
}()
fn()
return nil
}
func TestFeed(t *testing.T) {
var feed Feed
var done, subscribed sync.WaitGroup
subscriber := func(i int) {
defer done.Done()
subchan := make(chan int)
sub := feed.Subscribe(subchan)
timeout := time.NewTimer(2 * time.Second)
subscribed.Done()
select {
case v := <-subchan:
if v != 1 {
t.Errorf("%d: received value %d, want 1", i, v)
}
case <-timeout.C:
t.Errorf("%d: receive timeout", i)
}
sub.Unsubscribe()
select {
case _, ok := <-sub.Err():
if ok {
t.Errorf("%d: error channel not closed after unsubscribe", i)
}
case <-timeout.C:
t.Errorf("%d: unsubscribe timeout", i)
}
}
const n = 1000
done.Add(n)
subscribed.Add(n)
for i := 0; i < n; i++ {
go subscriber(i)
}
subscribed.Wait()
if nsent := feed.Send(1); nsent != n {
t.Errorf("first send delivered %d times, want %d", nsent, n)
}
if nsent := feed.Send(2); nsent != 0 {
t.Errorf("second send delivered %d times, want 0", nsent)
}
done.Wait()
}
func TestFeedSubscribeSameChannel(t *testing.T) {
var (
feed Feed
done sync.WaitGroup
ch = make(chan int)
sub1 = feed.Subscribe(ch)
sub2 = feed.Subscribe(ch)
_ = feed.Subscribe(ch)
)
expectSends := func(value, n int) {
if nsent := feed.Send(value); nsent != n {
t.Errorf("send delivered %d times, want %d", nsent, n)
}
done.Done()
}
expectRecv := func(wantValue, n int) {
for i := 0; i < n; i++ {
if v := <-ch; v != wantValue {
t.Errorf("received %d, want %d", v, wantValue)
}
}
}
done.Add(1)
go expectSends(1, 3)
expectRecv(1, 3)
done.Wait()
sub1.Unsubscribe()
done.Add(1)
go expectSends(2, 2)
expectRecv(2, 2)
done.Wait()
sub2.Unsubscribe()
done.Add(1)
go expectSends(3, 1)
expectRecv(3, 1)
done.Wait()
}
func TestFeedSubscribeBlockedPost(_ *testing.T) {
var (
feed Feed
nsends = 2000
ch1 = make(chan int)
ch2 = make(chan int)
wg sync.WaitGroup
)
defer wg.Wait()
feed.Subscribe(ch1)
wg.Add(nsends)
for i := 0; i < nsends; i++ {
go func() {
feed.Send(99)
wg.Done()
}()
}
sub2 := feed.Subscribe(ch2)
defer sub2.Unsubscribe()
// We're done when ch1 has received N times.
// The number of receives on ch2 depends on scheduling.
for i := 0; i < nsends; {
select {
case <-ch1:
i++
case <-ch2:
}
}
}
func TestFeedUnsubscribeBlockedPost(_ *testing.T) {
var (
feed Feed
nsends = 200
chans = make([]chan int, 2000)
subs = make([]Subscription, len(chans))
bchan = make(chan int)
bsub = feed.Subscribe(bchan)
wg sync.WaitGroup
)
for i := range chans {
chans[i] = make(chan int, nsends)
}
// Queue up some Sends. None of these can make progress while bchan isn't read.
wg.Add(nsends)
for i := 0; i < nsends; i++ {
go func() {
feed.Send(99)
wg.Done()
}()
}
// Subscribe the other channels.
for i, ch := range chans {
subs[i] = feed.Subscribe(ch)
}
// Unsubscribe them again.
for _, sub := range subs {
sub.Unsubscribe()
}
// Unblock the Sends.
bsub.Unsubscribe()
wg.Wait()
}
// Checks that unsubscribing a channel during Send works even if that
// channel has already been sent on.
func TestFeedUnsubscribeSentChan(_ *testing.T) {
var (
feed Feed
ch1 = make(chan int)
ch2 = make(chan int)
sub1 = feed.Subscribe(ch1)
sub2 = feed.Subscribe(ch2)
wg sync.WaitGroup
)
defer sub2.Unsubscribe()
wg.Add(1)
go func() {
feed.Send(0)
wg.Done()
}()
// Wait for the value on ch1.
<-ch1
// Unsubscribe ch1, removing it from the send cases.
sub1.Unsubscribe()
// Receive ch2, finishing Send.
<-ch2
wg.Wait()
// Send again. This should send to ch2 only, so the wait group will unblock
// as soon as a value is received on ch2.
wg.Add(1)
go func() {
feed.Send(0)
wg.Done()
}()
<-ch2
wg.Wait()
}
func TestFeedUnsubscribeFromInbox(t *testing.T) {
var (
feed Feed
ch1 = make(chan int)
ch2 = make(chan int)
sub1 = feed.Subscribe(ch1)
sub2 = feed.Subscribe(ch1)
sub3 = feed.Subscribe(ch2)
)
assert.Equal(t, 3, len(feed.inbox))
assert.Equal(t, 1, len(feed.sendCases), "sendCases is non-empty after unsubscribe")
sub1.Unsubscribe()
sub2.Unsubscribe()
sub3.Unsubscribe()
assert.Equal(t, 0, len(feed.inbox), "Inbox is non-empty after unsubscribe")
assert.Equal(t, 1, len(feed.sendCases), "sendCases is non-empty after unsubscribe")
}
func BenchmarkFeedSend1000(b *testing.B) {
var (
done sync.WaitGroup
feed Feed
nsubs = 1000
)
subscriber := func(ch <-chan int) {
for i := 0; i < b.N; i++ {
<-ch
}
done.Done()
}
done.Add(nsubs)
for i := 0; i < nsubs; i++ {
ch := make(chan int, 200)
feed.Subscribe(ch)
go subscriber(ch)
}
// The actual benchmark.
b.ResetTimer()
for i := 0; i < b.N; i++ {
if feed.Send(i) != nsubs {
panic("wrong number of sends")
}
}
b.StopTimer()
done.Wait()
}
func TestFeed_Send(t *testing.T) {
tests := []struct {
name string
evFeed *Feed
testSetup func(fd *Feed, t *testing.T, o interface{})
obj interface{}
expectPanic bool
}{
{
name: "normal struct",
evFeed: new(Feed),
testSetup: func(fd *Feed, t *testing.T, o interface{}) {
testChan := make(chan testFeedWithPointer, 1)
fd.Subscribe(testChan)
},
obj: testFeedWithPointer{
a: new(uint64),
b: new(string),
},
expectPanic: false,
},
{
name: "un-implemented interface",
evFeed: new(Feed),
testSetup: func(fd *Feed, t *testing.T, o interface{}) {
testChan := make(chan testFeedIface, 1)
fd.Subscribe(testChan)
},
obj: testFeedWithPointer{
a: new(uint64),
b: new(string),
},
expectPanic: true,
},
{
name: "semi-implemented interface",
evFeed: new(Feed),
testSetup: func(fd *Feed, t *testing.T, o interface{}) {
testChan := make(chan testFeedIface, 1)
fd.Subscribe(testChan)
},
obj: testFeed2{
a: 0,
b: "",
c: []byte{'A'},
},
expectPanic: true,
},
{
name: "fully-implemented interface",
evFeed: new(Feed),
testSetup: func(fd *Feed, t *testing.T, o interface{}) {
testChan := make(chan testFeedIface)
// Make it unbuffered to allow message to
// pass through
go func() {
a := <-testChan
if !reflect.DeepEqual(a, o) {
t.Errorf("Got = %v, want = %v", a, o)
}
}()
fd.Subscribe(testChan)
},
obj: testFeed{
a: 0,
b: "",
},
expectPanic: false,
},
{
name: "fully-implemented interface with additional methods",
evFeed: new(Feed),
testSetup: func(fd *Feed, t *testing.T, o interface{}) {
testChan := make(chan testFeedIface)
// Make it unbuffered to allow message to
// pass through
go func() {
a := <-testChan
if !reflect.DeepEqual(a, o) {
t.Errorf("Got = %v, want = %v", a, o)
}
}()
fd.Subscribe(testChan)
},
obj: testFeed3{
a: 0,
b: "",
c: []byte{'A'},
d: []byte{'B'},
},
expectPanic: false,
},
{
name: "concrete types implementing the same interface",
evFeed: new(Feed),
testSetup: func(fd *Feed, t *testing.T, o interface{}) {
testChan := make(chan testFeed, 1)
// Make it unbuffered to allow message to
// pass through
go func() {
a := <-testChan
if !reflect.DeepEqual(a, o) {
t.Errorf("Got = %v, want = %v", a, o)
}
}()
fd.Subscribe(testChan)
},
obj: testFeed3{
a: 0,
b: "",
c: []byte{'A'},
d: []byte{'B'},
},
expectPanic: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
defer func() {
if r := recover(); r != nil {
if !tt.expectPanic {
t.Errorf("panic triggered when unexpected: %v", r)
}
} else {
if tt.expectPanic {
t.Error("panic not triggered when expected")
}
}
}()
tt.testSetup(tt.evFeed, t, tt.obj)
if gotNsent := tt.evFeed.Send(tt.obj); gotNsent != 1 {
t.Errorf("Send() = %v, want %v", gotNsent, 1)
}
})
}
}
// The following objects below are a collection of different
// struct types to test with.
type testFeed struct {
a uint64
b string
}
func (testFeed) method1() {
}
func (testFeed) method2() {
}
type testFeedWithPointer struct {
a *uint64
b *string
}
type testFeed2 struct {
a uint64
b string
c []byte
}
func (testFeed2) method1() {
}
type testFeed3 struct {
a uint64
b string
c, d []byte
}
func (testFeed3) method1() {
}
func (testFeed3) method2() {
}
func (testFeed3) method3() {
}
type testFeedIface interface {
method1()
method2()
}

View File

@@ -21,7 +21,7 @@ import (
"sync"
"time"
"github.com/prysmaticlabs/prysm/shared/mclockutil"
"github.com/prysmaticlabs/prysm/time/mclock"
)
// waitQuotient is divided against the max backoff time, in order to have N requests based on the full
@@ -120,7 +120,7 @@ type resubscribeSub struct {
err chan error
unsub chan struct{}
unsubOnce sync.Once
lastTry mclockutil.AbsTime
lastTry mclock.AbsTime
waitTime, backoffMax time.Duration
}
@@ -151,11 +151,11 @@ func (s *resubscribeSub) loop() {
}
func (s *resubscribeSub) subscribe() Subscription {
subscribed := make(chan error)
subscribed := make(chan error, 1)
var sub Subscription
retry:
for {
s.lastTry = mclockutil.Now()
s.lastTry = mclock.Now()
ctx, cancel := context.WithCancel(context.TODO())
go func() {
rsub, err := s.fn(ctx)
@@ -194,7 +194,7 @@ func (s *resubscribeSub) waitForError(sub Subscription) bool {
}
func (s *resubscribeSub) backoffWait() bool {
if time.Duration(mclockutil.Now()-s.lastTry) > s.backoffMax {
if time.Duration(mclock.Now()-s.lastTry) > s.backoffMax {
s.waitTime = s.backoffMax / waitQuotient
} else {
s.waitTime *= 2

View File

@@ -19,10 +19,11 @@ package event
import (
"context"
"errors"
"runtime"
"testing"
"time"
"github.com/prysmaticlabs/prysm/shared/testutil/require"
"github.com/prysmaticlabs/prysm/testing/require"
)
var errInts = errors.New("error in subscribeInts")
@@ -108,3 +109,18 @@ func TestResubscribeAbort(t *testing.T) {
sub.Unsubscribe()
require.NoError(t, <-done)
}
func TestResubscribeNonBlocking(t *testing.T) {
done := make(chan struct{})
sub := Resubscribe(0, func(ctx context.Context) (Subscription, error) {
<-done
return nil, nil
})
resub, ok := sub.(*resubscribeSub)
require.Equal(t, true, ok)
currNum := runtime.NumGoroutine()
resub.unsub <- struct{}{}
done <- struct{}{}
require.Equal(t, currNum-1, runtime.NumGoroutine())
}

View File

@@ -1,5 +1,5 @@
// Package runutil includes helpers for scheduling runnable, periodic functions.
package runutil
// Package async includes helpers for scheduling runnable, periodic functions and contains useful helpers for converting multi-processor computation.
package async
import (
"context"

View File

@@ -1,18 +1,18 @@
package runutil_test
package async_test
import (
"context"
"testing"
"time"
"github.com/prysmaticlabs/prysm/shared/runutil"
"github.com/prysmaticlabs/prysm/async"
)
func TestEveryRuns(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
i := 0
runutil.RunEvery(ctx, 100*time.Millisecond, func() {
async.RunEvery(ctx, 100*time.Millisecond, func() {
i++
})

View File

@@ -10,7 +10,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package mputil
package async
import (
"runtime"

View File

@@ -10,7 +10,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package mputil
package async
import (
"sync"
@@ -50,7 +50,7 @@ func TestGetChan(t *testing.T) {
assert.Equal(ch1, ch3)
}
func TestLockUnlock(t *testing.T) {
func TestLockUnlock(_ *testing.T) {
var wg sync.WaitGroup
wg.Add(5)

View File

@@ -1,6 +1,4 @@
// Package mputil contains useful helpers for converting
// multi-processor computation.
package mputil
package async
import (
"errors"

View File

@@ -1,13 +1,13 @@
package mputil_test
package async_test
import (
"errors"
"sync"
"testing"
"github.com/prysmaticlabs/prysm/shared/mputil"
"github.com/prysmaticlabs/prysm/shared/testutil/assert"
"github.com/prysmaticlabs/prysm/shared/testutil/require"
"github.com/prysmaticlabs/prysm/async"
"github.com/prysmaticlabs/prysm/testing/assert"
"github.com/prysmaticlabs/prysm/testing/require"
)
func TestDouble(t *testing.T) {
@@ -46,7 +46,7 @@ func TestDouble(t *testing.T) {
inValues[i] = i
}
outValues := make([]int, test.inValues)
workerResults, err := mputil.Scatter(len(inValues), func(offset int, entries int, _ *sync.RWMutex) (interface{}, error) {
workerResults, err := async.Scatter(len(inValues), func(offset int, entries int, _ *sync.RWMutex) (interface{}, error) {
extent := make([]int, entries)
for i := 0; i < entries; i++ {
extent[i] = inValues[offset+i] * 2
@@ -72,7 +72,7 @@ func TestDouble(t *testing.T) {
func TestMutex(t *testing.T) {
totalRuns := 1048576
val := 0
_, err := mputil.Scatter(totalRuns, func(offset int, entries int, mu *sync.RWMutex) (interface{}, error) {
_, err := async.Scatter(totalRuns, func(offset int, entries int, mu *sync.RWMutex) (interface{}, error) {
for i := 0; i < entries; i++ {
mu.Lock()
val++
@@ -90,7 +90,7 @@ func TestMutex(t *testing.T) {
func TestError(t *testing.T) {
totalRuns := 1024
val := 0
_, err := mputil.Scatter(totalRuns, func(offset int, entries int, mu *sync.RWMutex) (interface{}, error) {
_, err := async.Scatter(totalRuns, func(offset int, entries int, mu *sync.RWMutex) (interface{}, error) {
for i := 0; i < entries; i++ {
mu.Lock()
val++

View File

@@ -1,9 +1,9 @@
# Prysmatic Labs Beacon Chain Implementation
This is the main project folder for the beacon chain implementation of eth2 written in Go by [Prysmatic Labs](https://prysmaticlabs.com).
This is the main project folder for the beacon chain implementation of Ethereum written in Go by [Prysmatic Labs](https://prysmaticlabs.com).
You can also read our main [README](https://github.com/prysmaticlabs/prysm/blob/master/README.md) and join our active chat room on Discord.
[![Discord](https://user-images.githubusercontent.com/7288322/34471967-1df7808a-efbb-11e7-9088-ed0b04151291.png)](https://discord.gg/CTYGPUJ)
Also, read the official beacon chain [specification](https://github.com/ethereum/eth2.0-specs/blob/master/specs/phase0/beacon-chain.md), this design spec serves as a source of truth for the beacon chain implementation we follow at Prysmatic Labs.
Also, read the official beacon chain [specification](https://github.com/ethereum/consensus-specs/blob/master/specs/phase0/beacon-chain.md), this design spec serves as a source of truth for the beacon chain implementation we follow at Prysmatic Labs.

View File

@@ -1,15 +1,20 @@
load("@prysm//tools/go:def.bzl", "go_library")
load("@io_bazel_rules_go//go:def.bzl", "go_test")
load("@prysm//tools/go:def.bzl", "go_library", "go_test")
go_library(
name = "go_default_library",
srcs = [
"chain_info.go",
"error.go",
"head.go",
"head_sync_committee_info.go",
"info.go",
"init_sync_process_block.go",
"log.go",
"metrics.go",
"new_slot.go",
"optimistic_sync.go",
"options.go",
"pow_block.go",
"process_attestation.go",
"process_attestation_helpers.go",
"process_block.go",
@@ -17,21 +22,31 @@ go_library(
"receive_attestation.go",
"receive_block.go",
"service.go",
"state_balance_cache.go",
"weak_subjectivity_checks.go",
],
importpath = "github.com/prysmaticlabs/prysm/beacon-chain/blockchain",
visibility = [
"//beacon-chain:__subpackages__",
"//fuzz:__pkg__",
"//cmd/beacon-chain:__subpackages__",
"//testing/slasher/simulator:__pkg__",
"//testing/spectest:__subpackages__",
],
deps = [
"//async:go_default_library",
"//async/event:go_default_library",
"//beacon-chain/blockchain/store:go_default_library",
"//beacon-chain/cache:go_default_library",
"//beacon-chain/cache/depositcache:go_default_library",
"//beacon-chain/core/altair:go_default_library",
"//beacon-chain/core/blocks:go_default_library",
"//beacon-chain/core/epoch/precompute:go_default_library",
"//beacon-chain/core/feed:go_default_library",
"//beacon-chain/core/feed/state:go_default_library",
"//beacon-chain/core/helpers:go_default_library",
"//beacon-chain/core/state:go_default_library",
"//beacon-chain/core/signing:go_default_library",
"//beacon-chain/core/time:go_default_library",
"//beacon-chain/core/transition:go_default_library",
"//beacon-chain/db:go_default_library",
"//beacon-chain/db/filters:go_default_library",
"//beacon-chain/forkchoice:go_default_library",
@@ -41,26 +56,28 @@ go_library(
"//beacon-chain/operations/voluntaryexits:go_default_library",
"//beacon-chain/p2p:go_default_library",
"//beacon-chain/powchain:go_default_library",
"//beacon-chain/state/interface:go_default_library",
"//beacon-chain/state/stateV0:go_default_library",
"//beacon-chain/state:go_default_library",
"//beacon-chain/state/stategen:go_default_library",
"//cmd/beacon-chain/flags:go_default_library",
"//proto/beacon/p2p/v1:go_default_library",
"//shared/attestationutil:go_default_library",
"//shared/bls:go_default_library",
"//shared/bytesutil:go_default_library",
"//shared/featureconfig:go_default_library",
"//shared/mputil:go_default_library",
"//shared/params:go_default_library",
"//shared/slotutil:go_default_library",
"//shared/timeutils:go_default_library",
"//shared/traceutil:go_default_library",
"//config/features:go_default_library",
"//config/fieldparams:go_default_library",
"//config/params:go_default_library",
"//crypto/bls:go_default_library",
"//encoding/bytesutil:go_default_library",
"//monitoring/tracing:go_default_library",
"//proto/eth/v1:go_default_library",
"//proto/prysm/v1alpha1:go_default_library",
"//proto/prysm/v1alpha1/attestation:go_default_library",
"//proto/prysm/v1alpha1/block:go_default_library",
"//runtime/version:go_default_library",
"//time:go_default_library",
"//time/slots:go_default_library",
"@com_github_emicklei_dot//:go_default_library",
"@com_github_holiman_uint256//:go_default_library",
"@com_github_pkg_errors//:go_default_library",
"@com_github_prometheus_client_golang//prometheus:go_default_library",
"@com_github_prometheus_client_golang//prometheus/promauto:go_default_library",
"@com_github_prysmaticlabs_eth2_types//:go_default_library",
"@com_github_prysmaticlabs_ethereumapis//eth/v1alpha1:go_default_library",
"@com_github_sirupsen_logrus//:go_default_library",
"@io_opencensus_go//trace:go_default_library",
],
@@ -81,10 +98,15 @@ go_test(
"blockchain_test.go",
"chain_info_test.go",
"checktags_test.go",
"head_sync_committee_info_test.go",
"head_test.go",
"info_test.go",
"init_test.go",
"log_test.go",
"metrics_test.go",
"mock_test.go",
"optimistic_sync_test.go",
"pow_block_test.go",
"process_attestation_test.go",
"process_block_test.go",
"receive_attestation_test.go",
@@ -95,32 +117,33 @@ go_test(
embed = [":go_default_library"],
gotags = ["develop"],
deps = [
"//async/event:go_default_library",
"//beacon-chain/blockchain/testing:go_default_library",
"//beacon-chain/cache/depositcache:go_default_library",
"//beacon-chain/core/blocks:go_default_library",
"//beacon-chain/core/helpers:go_default_library",
"//beacon-chain/core/state:go_default_library",
"//beacon-chain/core/transition:go_default_library",
"//beacon-chain/db:go_default_library",
"//beacon-chain/db/testing:go_default_library",
"//beacon-chain/p2p:go_default_library",
"//beacon-chain/powchain:go_default_library",
"//beacon-chain/state/stateutil:go_default_library",
"//proto/beacon/db:go_default_library",
"//proto/beacon/p2p/v1:go_default_library",
"//shared/bytesutil:go_default_library",
"//shared/event:go_default_library",
"//shared/params:go_default_library",
"//shared/testutil:go_default_library",
"//shared/testutil/assert:go_default_library",
"//shared/testutil/require:go_default_library",
"//beacon-chain/state/v1:go_default_library",
"//config/fieldparams:go_default_library",
"//config/params:go_default_library",
"//encoding/bytesutil:go_default_library",
"//proto/prysm/v1alpha1:go_default_library",
"//proto/prysm/v1alpha1/wrapper:go_default_library",
"//testing/assert:go_default_library",
"//testing/require:go_default_library",
"//testing/util:go_default_library",
"@com_github_ethereum_go_ethereum//:go_default_library",
"@com_github_ethereum_go_ethereum//common:go_default_library",
"@com_github_ethereum_go_ethereum//core/types:go_default_library",
"@com_github_gogo_protobuf//proto:go_default_library",
"@com_github_prysmaticlabs_ethereumapis//eth/v1alpha1:go_default_library",
"@com_github_sirupsen_logrus//:go_default_library",
"@com_github_sirupsen_logrus//hooks/test:go_default_library",
"@in_gopkg_d4l3k_messagediff_v1//:go_default_library",
"@org_golang_google_protobuf//proto:go_default_library",
"@org_golang_x_net//context:go_default_library",
],
)
@@ -131,6 +154,7 @@ go_test(
"chain_info_norace_test.go",
"checktags_test.go",
"init_test.go",
"mock_test.go",
"receive_block_test.go",
"service_norace_test.go",
],
@@ -146,29 +170,29 @@ go_test(
race = "on",
tags = ["race_on"],
deps = [
"//async/event:go_default_library",
"//beacon-chain/blockchain/testing:go_default_library",
"//beacon-chain/cache/depositcache:go_default_library",
"//beacon-chain/core/blocks:go_default_library",
"//beacon-chain/core/helpers:go_default_library",
"//beacon-chain/core/state:go_default_library",
"//beacon-chain/core/transition:go_default_library",
"//beacon-chain/db:go_default_library",
"//beacon-chain/db/testing:go_default_library",
"//beacon-chain/p2p:go_default_library",
"//beacon-chain/powchain:go_default_library",
"//proto/beacon/p2p/v1:go_default_library",
"//shared/bytesutil:go_default_library",
"//shared/event:go_default_library",
"//shared/params:go_default_library",
"//shared/testutil:go_default_library",
"//shared/testutil/assert:go_default_library",
"//shared/testutil/require:go_default_library",
"//config/params:go_default_library",
"//encoding/bytesutil:go_default_library",
"//proto/prysm/v1alpha1:go_default_library",
"//proto/prysm/v1alpha1/wrapper:go_default_library",
"//testing/assert:go_default_library",
"//testing/require:go_default_library",
"//testing/util:go_default_library",
"@com_github_ethereum_go_ethereum//:go_default_library",
"@com_github_ethereum_go_ethereum//common:go_default_library",
"@com_github_ethereum_go_ethereum//core/types:go_default_library",
"@com_github_gogo_protobuf//proto:go_default_library",
"@com_github_prysmaticlabs_ethereumapis//eth/v1alpha1:go_default_library",
"@com_github_sirupsen_logrus//:go_default_library",
"@com_github_sirupsen_logrus//hooks/test:go_default_library",
"@org_golang_google_protobuf//proto:go_default_library",
"@org_golang_x_net//context:go_default_library",
],
)

View File

@@ -5,14 +5,14 @@ import (
"time"
types "github.com/prysmaticlabs/eth2-types"
ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
"github.com/prysmaticlabs/prysm/beacon-chain/forkchoice/protoarray"
iface "github.com/prysmaticlabs/prysm/beacon-chain/state/interface"
"github.com/prysmaticlabs/prysm/beacon-chain/state/stateV0"
pb "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
"github.com/prysmaticlabs/prysm/shared/bytesutil"
"github.com/prysmaticlabs/prysm/shared/params"
"github.com/prysmaticlabs/prysm/beacon-chain/state"
fieldparams "github.com/prysmaticlabs/prysm/config/fieldparams"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/block"
"go.opencensus.io/trace"
)
@@ -23,15 +23,18 @@ type ChainInfoFetcher interface {
FinalizationFetcher
GenesisFetcher
CanonicalFetcher
ForkFetcher
TimeFetcher
HeadDomainFetcher
}
// TimeFetcher retrieves the Eth2 data that's related to time.
// TimeFetcher retrieves the Ethereum consensus data that's related to time.
type TimeFetcher interface {
GenesisTime() time.Time
CurrentSlot() types.Slot
}
// GenesisFetcher retrieves the eth2 data related to its genesis.
// GenesisFetcher retrieves the Ethereum consensus data related to its genesis.
type GenesisFetcher interface {
GenesisValidatorRoot() [32]byte
}
@@ -41,19 +44,25 @@ type GenesisFetcher interface {
type HeadFetcher interface {
HeadSlot() types.Slot
HeadRoot(ctx context.Context) ([]byte, error)
HeadBlock(ctx context.Context) (*ethpb.SignedBeaconBlock, error)
HeadState(ctx context.Context) (iface.BeaconState, error)
HeadBlock(ctx context.Context) (block.SignedBeaconBlock, error)
HeadState(ctx context.Context) (state.BeaconState, error)
HeadValidatorsIndices(ctx context.Context, epoch types.Epoch) ([]types.ValidatorIndex, error)
HeadSeed(ctx context.Context, epoch types.Epoch) ([32]byte, error)
HeadGenesisValidatorRoot() [32]byte
HeadETH1Data() *ethpb.Eth1Data
HeadPublicKeyToValidatorIndex(pubKey [fieldparams.BLSPubkeyLength]byte) (types.ValidatorIndex, bool)
HeadValidatorIndexToPublicKey(ctx context.Context, index types.ValidatorIndex) ([fieldparams.BLSPubkeyLength]byte, error)
ProtoArrayStore() *protoarray.Store
ChainHeads() ([][32]byte, []types.Slot)
IsOptimistic(ctx context.Context) (bool, error)
IsOptimisticForRoot(ctx context.Context, root [32]byte, slot types.Slot) (bool, error)
HeadSyncCommitteeFetcher
HeadDomainFetcher
}
// ForkFetcher retrieves the current fork information of the Ethereum beacon chain.
type ForkFetcher interface {
CurrentFork() *pb.Fork
CurrentFork() *ethpb.Fork
}
// CanonicalFetcher retrieves the current chain's canonical information.
@@ -70,31 +79,45 @@ type FinalizationFetcher interface {
PreviousJustifiedCheckpt() *ethpb.Checkpoint
}
// FinalizedCheckpt returns the latest finalized checkpoint from head state.
// FinalizedCheckpt returns the latest finalized checkpoint from chain store.
func (s *Service) FinalizedCheckpt() *ethpb.Checkpoint {
if s.finalizedCheckpt == nil {
cp := s.store.FinalizedCheckpt()
if cp == nil {
return &ethpb.Checkpoint{Root: params.BeaconConfig().ZeroHash[:]}
}
return stateV0.CopyCheckpoint(s.finalizedCheckpt)
return ethpb.CopyCheckpoint(cp)
}
// CurrentJustifiedCheckpt returns the current justified checkpoint from head state.
// CurrentJustifiedCheckpt returns the current justified checkpoint from chain store.
func (s *Service) CurrentJustifiedCheckpt() *ethpb.Checkpoint {
if s.justifiedCheckpt == nil {
cp := s.store.JustifiedCheckpt()
if cp == nil {
return &ethpb.Checkpoint{Root: params.BeaconConfig().ZeroHash[:]}
}
return stateV0.CopyCheckpoint(s.justifiedCheckpt)
return ethpb.CopyCheckpoint(cp)
}
// PreviousJustifiedCheckpt returns the previous justified checkpoint from head state.
// PreviousJustifiedCheckpt returns the previous justified checkpoint from chain store.
func (s *Service) PreviousJustifiedCheckpt() *ethpb.Checkpoint {
if s.prevJustifiedCheckpt == nil {
cp := s.store.PrevJustifiedCheckpt()
if cp == nil {
return &ethpb.Checkpoint{Root: params.BeaconConfig().ZeroHash[:]}
}
return stateV0.CopyCheckpoint(s.prevJustifiedCheckpt)
return ethpb.CopyCheckpoint(cp)
}
// BestJustifiedCheckpt returns the best justified checkpoint from store.
func (s *Service) BestJustifiedCheckpt() *ethpb.Checkpoint {
cp := s.store.BestJustifiedCheckpt()
// If there is no best justified checkpoint, return the checkpoint with root as zeros to be used for genesis cases.
if cp == nil {
return &ethpb.Checkpoint{Root: params.BeaconConfig().ZeroHash[:]}
}
return ethpb.CopyCheckpoint(cp)
}
// HeadSlot returns the slot of the head of the chain.
@@ -123,11 +146,11 @@ func (s *Service) HeadRoot(ctx context.Context) ([]byte, error) {
if err != nil {
return nil, err
}
if b == nil {
if b == nil || b.IsNil() {
return params.BeaconConfig().ZeroHash[:], nil
}
r, err := b.Block.HashTreeRoot()
r, err := b.Block().HashTreeRoot()
if err != nil {
return nil, err
}
@@ -138,7 +161,7 @@ func (s *Service) HeadRoot(ctx context.Context) ([]byte, error) {
// HeadBlock returns the head block of the chain.
// If the head is nil from service struct,
// it will attempt to get the head block from DB.
func (s *Service) HeadBlock(ctx context.Context) (*ethpb.SignedBeaconBlock, error) {
func (s *Service) HeadBlock(ctx context.Context) (block.SignedBeaconBlock, error) {
s.headLock.RLock()
defer s.headLock.RUnlock()
@@ -152,7 +175,7 @@ func (s *Service) HeadBlock(ctx context.Context) (*ethpb.SignedBeaconBlock, erro
// HeadState returns the head state of the chain.
// If the head is nil from service struct,
// it will attempt to get the head state from DB.
func (s *Service) HeadState(ctx context.Context) (iface.BeaconState, error) {
func (s *Service) HeadState(ctx context.Context) (state.BeaconState, error) {
ctx, span := trace.StartSpan(ctx, "blockChain.HeadState")
defer span.End()
s.headLock.RLock()
@@ -176,7 +199,7 @@ func (s *Service) HeadValidatorsIndices(ctx context.Context, epoch types.Epoch)
if !s.hasHeadState() {
return []types.ValidatorIndex{}, nil
}
return helpers.ActiveValidatorIndices(s.headState(ctx), epoch)
return helpers.ActiveValidatorIndices(ctx, s.headState(ctx), epoch)
}
// HeadSeed returns the seed from the head view of a given epoch.
@@ -237,12 +260,12 @@ func (s *Service) GenesisValidatorRoot() [32]byte {
}
// CurrentFork retrieves the latest fork information of the beacon chain.
func (s *Service) CurrentFork() *pb.Fork {
func (s *Service) CurrentFork() *ethpb.Fork {
s.headLock.RLock()
defer s.headLock.RUnlock()
if !s.hasHeadState() {
return &pb.Fork{
return &ethpb.Fork{
PreviousVersion: params.BeaconConfig().GenesisForkVersion,
CurrentVersion: params.BeaconConfig().GenesisForkVersion,
}
@@ -282,3 +305,45 @@ func (s *Service) ChainHeads() ([][32]byte, []types.Slot) {
return headsRoots, headsSlots
}
// HeadPublicKeyToValidatorIndex returns the validator index of the `pubkey` in current head state.
func (s *Service) HeadPublicKeyToValidatorIndex(pubKey [fieldparams.BLSPubkeyLength]byte) (types.ValidatorIndex, bool) {
s.headLock.RLock()
defer s.headLock.RUnlock()
if !s.hasHeadState() {
return 0, false
}
return s.headValidatorIndexAtPubkey(pubKey)
}
// HeadValidatorIndexToPublicKey returns the pubkey of the validator `index` in current head state.
func (s *Service) HeadValidatorIndexToPublicKey(_ context.Context, index types.ValidatorIndex) ([fieldparams.BLSPubkeyLength]byte, error) {
s.headLock.RLock()
defer s.headLock.RUnlock()
if !s.hasHeadState() {
return [fieldparams.BLSPubkeyLength]byte{}, nil
}
v, err := s.headValidatorAtIndex(index)
if err != nil {
return [fieldparams.BLSPubkeyLength]byte{}, err
}
return v.PublicKey(), nil
}
// IsOptimistic returns true if the current head is optimistic.
func (s *Service) IsOptimistic(ctx context.Context) (bool, error) {
s.headLock.RLock()
defer s.headLock.RUnlock()
return s.cfg.ForkChoiceStore.Optimistic(ctx, s.head.root, s.head.slot)
}
// IsOptimisticForRoot takes the root and slot as aguments instead of the current head
// and returns true if it is optimistic.
func (s *Service) IsOptimisticForRoot(ctx context.Context, root [32]byte, slot types.Slot) (bool, error) {
return s.cfg.ForkChoiceStore.Optimistic(ctx, root, slot)
}
// SetGenesisTime sets the genesis time of beacon chain.
func (s *Service) SetGenesisTime(t time.Time) {
s.genesisTime = t
}

View File

@@ -4,16 +4,17 @@ import (
"context"
"testing"
ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
testDB "github.com/prysmaticlabs/prysm/beacon-chain/db/testing"
"github.com/prysmaticlabs/prysm/beacon-chain/state/stategen"
"github.com/prysmaticlabs/prysm/shared/testutil/require"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/wrapper"
"github.com/prysmaticlabs/prysm/testing/require"
)
func TestHeadSlot_DataRace(t *testing.T) {
beaconDB := testDB.SetupDB(t)
s := &Service{
cfg: &Config{BeaconDB: beaconDB},
cfg: &config{BeaconDB: beaconDB},
}
go func() {
require.NoError(t, s.saveHead(context.Background(), [32]byte{}))
@@ -24,7 +25,7 @@ func TestHeadSlot_DataRace(t *testing.T) {
func TestHeadRoot_DataRace(t *testing.T) {
beaconDB := testDB.SetupDB(t)
s := &Service{
cfg: &Config{BeaconDB: beaconDB, StateGen: stategen.New(beaconDB)},
cfg: &config{BeaconDB: beaconDB, StateGen: stategen.New(beaconDB)},
head: &head{root: [32]byte{'A'}},
}
go func() {
@@ -37,8 +38,8 @@ func TestHeadRoot_DataRace(t *testing.T) {
func TestHeadBlock_DataRace(t *testing.T) {
beaconDB := testDB.SetupDB(t)
s := &Service{
cfg: &Config{BeaconDB: beaconDB, StateGen: stategen.New(beaconDB)},
head: &head{block: &ethpb.SignedBeaconBlock{}},
cfg: &config{BeaconDB: beaconDB, StateGen: stategen.New(beaconDB)},
head: &head{block: wrapper.WrappedPhase0SignedBeaconBlock(&ethpb.SignedBeaconBlock{})},
}
go func() {
require.NoError(t, s.saveHead(context.Background(), [32]byte{}))
@@ -50,7 +51,7 @@ func TestHeadBlock_DataRace(t *testing.T) {
func TestHeadState_DataRace(t *testing.T) {
beaconDB := testDB.SetupDB(t)
s := &Service{
cfg: &Config{BeaconDB: beaconDB, StateGen: stategen.New(beaconDB)},
cfg: &config{BeaconDB: beaconDB, StateGen: stategen.New(beaconDB)},
}
go func() {
require.NoError(t, s.saveHead(context.Background(), [32]byte{}))

View File

@@ -5,19 +5,20 @@ import (
"testing"
"time"
"github.com/gogo/protobuf/proto"
types "github.com/prysmaticlabs/eth2-types"
ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
testDB "github.com/prysmaticlabs/prysm/beacon-chain/db/testing"
"github.com/prysmaticlabs/prysm/beacon-chain/forkchoice/protoarray"
"github.com/prysmaticlabs/prysm/beacon-chain/state/stateV0"
pb "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
"github.com/prysmaticlabs/prysm/shared/bytesutil"
"github.com/prysmaticlabs/prysm/shared/params"
"github.com/prysmaticlabs/prysm/shared/testutil"
"github.com/prysmaticlabs/prysm/shared/testutil/assert"
"github.com/prysmaticlabs/prysm/shared/testutil/require"
v1 "github.com/prysmaticlabs/prysm/beacon-chain/state/v1"
fieldparams "github.com/prysmaticlabs/prysm/config/fieldparams"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/wrapper"
"github.com/prysmaticlabs/prysm/testing/assert"
"github.com/prysmaticlabs/prysm/testing/require"
"github.com/prysmaticlabs/prysm/testing/util"
"google.golang.org/protobuf/proto"
)
// Ensure Service implements chain info interface.
@@ -44,7 +45,7 @@ func TestFinalizedCheckpt_CanRetrieve(t *testing.T) {
cp := &ethpb.Checkpoint{Epoch: 5, Root: bytesutil.PadTo([]byte("foo"), 32)}
c := setupBeaconChain(t, beaconDB)
c.finalizedCheckpt = cp
c.store.SetFinalizedCheckpt(cp)
assert.Equal(t, cp.Epoch, c.FinalizedCheckpt().Epoch, "Unexpected finalized epoch")
}
@@ -55,9 +56,9 @@ func TestFinalizedCheckpt_GenesisRootOk(t *testing.T) {
genesisRoot := [32]byte{'A'}
cp := &ethpb.Checkpoint{Root: genesisRoot[:]}
c := setupBeaconChain(t, beaconDB)
c.finalizedCheckpt = cp
c.genesisRoot = genesisRoot
assert.DeepEqual(t, c.genesisRoot[:], c.FinalizedCheckpt().Root)
c.store.SetFinalizedCheckpt(cp)
c.originBlockRoot = genesisRoot
assert.DeepEqual(t, c.originBlockRoot[:], c.FinalizedCheckpt().Root)
}
func TestCurrentJustifiedCheckpt_CanRetrieve(t *testing.T) {
@@ -66,7 +67,7 @@ func TestCurrentJustifiedCheckpt_CanRetrieve(t *testing.T) {
c := setupBeaconChain(t, beaconDB)
assert.Equal(t, params.BeaconConfig().ZeroHash, bytesutil.ToBytes32(c.CurrentJustifiedCheckpt().Root), "Unexpected justified epoch")
cp := &ethpb.Checkpoint{Epoch: 6, Root: bytesutil.PadTo([]byte("foo"), 32)}
c.justifiedCheckpt = cp
c.store.SetJustifiedCheckpt(cp)
assert.Equal(t, cp.Epoch, c.CurrentJustifiedCheckpt().Epoch, "Unexpected justified epoch")
}
@@ -76,9 +77,9 @@ func TestJustifiedCheckpt_GenesisRootOk(t *testing.T) {
c := setupBeaconChain(t, beaconDB)
genesisRoot := [32]byte{'B'}
cp := &ethpb.Checkpoint{Root: genesisRoot[:]}
c.justifiedCheckpt = cp
c.genesisRoot = genesisRoot
assert.DeepEqual(t, c.genesisRoot[:], c.CurrentJustifiedCheckpt().Root)
c.store.SetJustifiedCheckpt(cp)
c.originBlockRoot = genesisRoot
assert.DeepEqual(t, c.originBlockRoot[:], c.CurrentJustifiedCheckpt().Root)
}
func TestPreviousJustifiedCheckpt_CanRetrieve(t *testing.T) {
@@ -87,7 +88,7 @@ func TestPreviousJustifiedCheckpt_CanRetrieve(t *testing.T) {
cp := &ethpb.Checkpoint{Epoch: 7, Root: bytesutil.PadTo([]byte("foo"), 32)}
c := setupBeaconChain(t, beaconDB)
assert.Equal(t, params.BeaconConfig().ZeroHash, bytesutil.ToBytes32(c.CurrentJustifiedCheckpt().Root), "Unexpected justified epoch")
c.prevJustifiedCheckpt = cp
c.store.SetPrevJustifiedCheckpt(cp)
assert.Equal(t, cp.Epoch, c.PreviousJustifiedCheckpt().Epoch, "Unexpected previous justified epoch")
}
@@ -97,14 +98,14 @@ func TestPrevJustifiedCheckpt_GenesisRootOk(t *testing.T) {
genesisRoot := [32]byte{'C'}
cp := &ethpb.Checkpoint{Root: genesisRoot[:]}
c := setupBeaconChain(t, beaconDB)
c.prevJustifiedCheckpt = cp
c.genesisRoot = genesisRoot
assert.DeepEqual(t, c.genesisRoot[:], c.PreviousJustifiedCheckpt().Root)
c.store.SetPrevJustifiedCheckpt(cp)
c.originBlockRoot = genesisRoot
assert.DeepEqual(t, c.originBlockRoot[:], c.PreviousJustifiedCheckpt().Root)
}
func TestHeadSlot_CanRetrieve(t *testing.T) {
c := &Service{}
s, err := stateV0.InitializeFromProto(&pb.BeaconState{})
s, err := v1.InitializeFromProto(&ethpb.BeaconState{})
require.NoError(t, err)
c.head = &head{slot: 100, state: s}
assert.Equal(t, types.Slot(100), c.HeadSlot())
@@ -120,13 +121,13 @@ func TestHeadRoot_CanRetrieve(t *testing.T) {
func TestHeadRoot_UseDB(t *testing.T) {
beaconDB := testDB.SetupDB(t)
c := &Service{cfg: &Config{BeaconDB: beaconDB}}
c := &Service{cfg: &config{BeaconDB: beaconDB}}
c.head = &head{root: params.BeaconConfig().ZeroHash}
b := testutil.NewBeaconBlock()
b := util.NewBeaconBlock()
br, err := b.Block.HashTreeRoot()
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(context.Background(), b))
require.NoError(t, beaconDB.SaveStateSummary(context.Background(), &pb.StateSummary{Root: br[:]}))
require.NoError(t, beaconDB.SaveBlock(context.Background(), wrapper.WrappedPhase0SignedBeaconBlock(b)))
require.NoError(t, beaconDB.SaveStateSummary(context.Background(), &ethpb.StateSummary{Root: br[:]}))
require.NoError(t, beaconDB.SaveHeadBlockRoot(context.Background(), br))
r, err := c.HeadRoot(context.Background())
require.NoError(t, err)
@@ -134,20 +135,20 @@ func TestHeadRoot_UseDB(t *testing.T) {
}
func TestHeadBlock_CanRetrieve(t *testing.T) {
b := testutil.NewBeaconBlock()
b := util.NewBeaconBlock()
b.Block.Slot = 1
s, err := stateV0.InitializeFromProto(&pb.BeaconState{})
s, err := v1.InitializeFromProto(&ethpb.BeaconState{})
require.NoError(t, err)
c := &Service{}
c.head = &head{block: b, state: s}
c.head = &head{block: wrapper.WrappedPhase0SignedBeaconBlock(b), state: s}
recevied, err := c.HeadBlock(context.Background())
require.NoError(t, err)
assert.DeepEqual(t, b, recevied, "Incorrect head block received")
assert.DeepEqual(t, b, recevied.Proto(), "Incorrect head block received")
}
func TestHeadState_CanRetrieve(t *testing.T) {
s, err := stateV0.InitializeFromProto(&pb.BeaconState{Slot: 2, GenesisValidatorsRoot: params.BeaconConfig().ZeroHash[:]})
s, err := v1.InitializeFromProto(&ethpb.BeaconState{Slot: 2, GenesisValidatorsRoot: params.BeaconConfig().ZeroHash[:]})
require.NoError(t, err)
c := &Service{}
c.head = &head{state: s}
@@ -163,8 +164,8 @@ func TestGenesisTime_CanRetrieve(t *testing.T) {
}
func TestCurrentFork_CanRetrieve(t *testing.T) {
f := &pb.Fork{Epoch: 999}
s, err := stateV0.InitializeFromProto(&pb.BeaconState{Fork: f})
f := &ethpb.Fork{Epoch: 999}
s, err := v1.InitializeFromProto(&ethpb.BeaconState{Fork: f})
require.NoError(t, err)
c := &Service{}
c.head = &head{state: s}
@@ -174,7 +175,7 @@ func TestCurrentFork_CanRetrieve(t *testing.T) {
}
func TestCurrentFork_NilHeadSTate(t *testing.T) {
f := &pb.Fork{
f := &ethpb.Fork{
PreviousVersion: params.BeaconConfig().GenesisForkVersion,
CurrentVersion: params.BeaconConfig().GenesisForkVersion,
}
@@ -189,7 +190,7 @@ func TestGenesisValidatorRoot_CanRetrieve(t *testing.T) {
c := &Service{}
assert.Equal(t, [32]byte{}, c.GenesisValidatorRoot(), "Did not get correct genesis validator root")
s, err := stateV0.InitializeFromProto(&pb.BeaconState{GenesisValidatorsRoot: []byte{'a'}})
s, err := v1.InitializeFromProto(&ethpb.BeaconState{GenesisValidatorsRoot: []byte{'a'}})
require.NoError(t, err)
c.head = &head{state: s}
assert.Equal(t, [32]byte{'a'}, c.GenesisValidatorRoot(), "Did not get correct genesis validator root")
@@ -203,7 +204,7 @@ func TestHeadETH1Data_Nil(t *testing.T) {
func TestHeadETH1Data_CanRetrieve(t *testing.T) {
d := &ethpb.Eth1Data{DepositCount: 999}
s, err := stateV0.InitializeFromProto(&pb.BeaconState{Eth1Data: d})
s, err := v1.InitializeFromProto(&ethpb.BeaconState{Eth1Data: d})
require.NoError(t, err)
c := &Service{}
c.head = &head{state: s}
@@ -217,11 +218,11 @@ func TestIsCanonical_Ok(t *testing.T) {
beaconDB := testDB.SetupDB(t)
c := setupBeaconChain(t, beaconDB)
blk := testutil.NewBeaconBlock()
blk := util.NewBeaconBlock()
blk.Block.Slot = 0
root, err := blk.Block.HashTreeRoot()
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, blk))
require.NoError(t, beaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(blk)))
require.NoError(t, beaconDB.SaveGenesisBlockRoot(ctx, root))
can, err := c.IsCanonical(ctx, root)
require.NoError(t, err)
@@ -233,7 +234,7 @@ func TestIsCanonical_Ok(t *testing.T) {
}
func TestService_HeadValidatorsIndices(t *testing.T) {
s, _ := testutil.DeterministicGenesisState(t, 10)
s, _ := util.DeterministicGenesisState(t, 10)
c := &Service{}
c.head = &head{}
@@ -248,7 +249,7 @@ func TestService_HeadValidatorsIndices(t *testing.T) {
}
func TestService_HeadSeed(t *testing.T) {
s, _ := testutil.DeterministicGenesisState(t, 1)
s, _ := util.DeterministicGenesisState(t, 1)
c := &Service{}
seed, err := helpers.Seed(s, 0, params.BeaconConfig().DomainBeaconAttester)
require.NoError(t, err)
@@ -265,7 +266,7 @@ func TestService_HeadSeed(t *testing.T) {
}
func TestService_HeadGenesisValidatorRoot(t *testing.T) {
s, _ := testutil.DeterministicGenesisState(t, 1)
s, _ := util.DeterministicGenesisState(t, 1)
c := &Service{}
c.head = &head{}
@@ -278,14 +279,14 @@ func TestService_HeadGenesisValidatorRoot(t *testing.T) {
}
func TestService_ProtoArrayStore(t *testing.T) {
c := &Service{cfg: &Config{ForkChoiceStore: protoarray.New(0, 0, [32]byte{})}}
c := &Service{cfg: &config{ForkChoiceStore: protoarray.New(0, 0, [32]byte{})}}
p := c.ProtoArrayStore()
require.Equal(t, 0, int(p.FinalizedEpoch()))
}
func TestService_ChainHeads(t *testing.T) {
ctx := context.Background()
c := &Service{cfg: &Config{ForkChoiceStore: protoarray.New(0, 0, [32]byte{})}}
c := &Service{cfg: &config{ForkChoiceStore: protoarray.New(0, 0, [32]byte{})}}
require.NoError(t, c.cfg.ForkChoiceStore.ProcessBlock(ctx, 100, [32]byte{'a'}, [32]byte{}, [32]byte{}, 0, 0))
require.NoError(t, c.cfg.ForkChoiceStore.ProcessBlock(ctx, 101, [32]byte{'b'}, [32]byte{'a'}, [32]byte{}, 0, 0))
require.NoError(t, c.cfg.ForkChoiceStore.ProcessBlock(ctx, 102, [32]byte{'c'}, [32]byte{'b'}, [32]byte{}, 0, 0))
@@ -296,3 +297,83 @@ func TestService_ChainHeads(t *testing.T) {
require.DeepEqual(t, [][32]byte{{'c'}, {'d'}, {'e'}}, roots)
require.DeepEqual(t, []types.Slot{102, 103, 104}, slots)
}
func TestService_HeadPublicKeyToValidatorIndex(t *testing.T) {
s, _ := util.DeterministicGenesisState(t, 10)
c := &Service{}
c.head = &head{state: s}
_, e := c.HeadPublicKeyToValidatorIndex([fieldparams.BLSPubkeyLength]byte{})
require.Equal(t, false, e)
v, err := s.ValidatorAtIndex(0)
require.NoError(t, err)
i, e := c.HeadPublicKeyToValidatorIndex(bytesutil.ToBytes48(v.PublicKey))
require.Equal(t, true, e)
require.Equal(t, types.ValidatorIndex(0), i)
}
func TestService_HeadPublicKeyToValidatorIndexNil(t *testing.T) {
c := &Service{}
c.head = nil
idx, e := c.HeadPublicKeyToValidatorIndex([fieldparams.BLSPubkeyLength]byte{})
require.Equal(t, false, e)
require.Equal(t, types.ValidatorIndex(0), idx)
c.head = &head{state: nil}
i, e := c.HeadPublicKeyToValidatorIndex([fieldparams.BLSPubkeyLength]byte{})
require.Equal(t, false, e)
require.Equal(t, types.ValidatorIndex(0), i)
}
func TestService_HeadValidatorIndexToPublicKey(t *testing.T) {
s, _ := util.DeterministicGenesisState(t, 10)
c := &Service{}
c.head = &head{state: s}
p, err := c.HeadValidatorIndexToPublicKey(context.Background(), 0)
require.NoError(t, err)
v, err := s.ValidatorAtIndex(0)
require.NoError(t, err)
require.Equal(t, bytesutil.ToBytes48(v.PublicKey), p)
}
func TestService_HeadValidatorIndexToPublicKeyNil(t *testing.T) {
c := &Service{}
c.head = nil
p, err := c.HeadValidatorIndexToPublicKey(context.Background(), 0)
require.NoError(t, err)
require.Equal(t, [fieldparams.BLSPubkeyLength]byte{}, p)
c.head = &head{state: nil}
p, err = c.HeadValidatorIndexToPublicKey(context.Background(), 0)
require.NoError(t, err)
require.Equal(t, [fieldparams.BLSPubkeyLength]byte{}, p)
}
func TestService_IsOptimistic(t *testing.T) {
ctx := context.Background()
c := &Service{cfg: &config{ForkChoiceStore: protoarray.New(0, 0, [32]byte{})}, head: &head{slot: 101, root: [32]byte{'b'}}}
require.NoError(t, c.cfg.ForkChoiceStore.ProcessBlock(ctx, 100, [32]byte{'a'}, [32]byte{}, [32]byte{}, 0, 0))
require.NoError(t, c.cfg.ForkChoiceStore.ProcessBlock(ctx, 101, [32]byte{'b'}, [32]byte{'a'}, [32]byte{}, 0, 0))
opt, err := c.IsOptimistic(ctx)
require.NoError(t, err)
require.Equal(t, true, opt)
}
func TestService_IsOptimisticForRoot(t *testing.T) {
ctx := context.Background()
c := &Service{cfg: &config{ForkChoiceStore: protoarray.New(0, 0, [32]byte{})}, head: &head{slot: 101, root: [32]byte{'b'}}}
require.NoError(t, c.cfg.ForkChoiceStore.ProcessBlock(ctx, 100, [32]byte{'a'}, [32]byte{}, [32]byte{}, 0, 0))
require.NoError(t, c.cfg.ForkChoiceStore.ProcessBlock(ctx, 101, [32]byte{'b'}, [32]byte{'a'}, [32]byte{}, 0, 0))
opt, err := c.IsOptimisticForRoot(ctx, [32]byte{'a'}, 100)
require.NoError(t, err)
require.Equal(t, true, opt)
}

View File

@@ -0,0 +1,12 @@
package blockchain
import "github.com/pkg/errors"
var (
// errNilJustifiedInStore is returned when a nil justified checkpt is returned from store.
errNilJustifiedInStore = errors.New("nil justified checkpoint returned from store")
// errNilBestJustifiedInStore is returned when a nil justified checkpt is returned from store.
errNilBestJustifiedInStore = errors.New("nil best justified checkpoint returned from store")
// errNilFinalizedInStore is returned when a nil finalized checkpt is returned from store.
errNilFinalizedInStore = errors.New("nil finalized checkpoint returned from store")
)

View File

@@ -7,25 +7,43 @@ import (
"github.com/pkg/errors"
types "github.com/prysmaticlabs/eth2-types"
ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
"github.com/prysmaticlabs/prysm/beacon-chain/core/feed"
statefeed "github.com/prysmaticlabs/prysm/beacon-chain/core/feed/state"
"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
"github.com/prysmaticlabs/prysm/beacon-chain/forkchoice/protoarray"
iface "github.com/prysmaticlabs/prysm/beacon-chain/state/interface"
"github.com/prysmaticlabs/prysm/beacon-chain/state/stateV0"
"github.com/prysmaticlabs/prysm/shared/bytesutil"
"github.com/prysmaticlabs/prysm/shared/params"
"github.com/prysmaticlabs/prysm/beacon-chain/state"
"github.com/prysmaticlabs/prysm/config/features"
fieldparams "github.com/prysmaticlabs/prysm/config/fieldparams"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
ethpbv1 "github.com/prysmaticlabs/prysm/proto/eth/v1"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/block"
"github.com/prysmaticlabs/prysm/time/slots"
"github.com/sirupsen/logrus"
"go.opencensus.io/trace"
)
// UpdateHeadWithBalances updates the beacon state head after getting justified balanced from cache.
func (s *Service) UpdateHeadWithBalances(ctx context.Context) error {
cp := s.store.JustifiedCheckpt()
if cp == nil {
return errors.New("no justified checkpoint")
}
balances, err := s.justifiedBalances.get(ctx, bytesutil.ToBytes32(cp.Root))
if err != nil {
msg := fmt.Sprintf("could not read balances for state w/ justified checkpoint %#x", cp.Root)
return errors.Wrap(err, msg)
}
return s.updateHead(ctx, balances)
}
// This defines the current chain service's view of head.
type head struct {
slot types.Slot // current head slot.
root [32]byte // current head root.
block *ethpb.SignedBeaconBlock // current head block.
state iface.BeaconState // current head state.
slot types.Slot // current head slot.
root [32]byte // current head root.
block block.SignedBeaconBlock // current head block.
state state.BeaconState // current head state.
}
// Determined the head from the fork choice service and saves its new data
@@ -34,24 +52,20 @@ func (s *Service) updateHead(ctx context.Context, balances []uint64) error {
ctx, span := trace.StartSpan(ctx, "blockChain.updateHead")
defer span.End()
// To get the proper head update, a node first checks its best justified
// can become justified. This is designed to prevent bounce attack and
// ensure head gets its best justified info.
if s.bestJustifiedCheckpt.Epoch > s.justifiedCheckpt.Epoch {
s.justifiedCheckpt = s.bestJustifiedCheckpt
if err := s.cacheJustifiedStateBalances(ctx, bytesutil.ToBytes32(s.justifiedCheckpt.Root)); err != nil {
return err
}
}
// Get head from the fork choice service.
f := s.finalizedCheckpt
j := s.justifiedCheckpt
// To get head before the first justified epoch, the fork choice will start with genesis root
f := s.store.FinalizedCheckpt()
if f == nil {
return errNilFinalizedInStore
}
j := s.store.JustifiedCheckpt()
if j == nil {
return errNilJustifiedInStore
}
// To get head before the first justified epoch, the fork choice will start with origin root
// instead of zero hashes.
headStartRoot := bytesutil.ToBytes32(j.Root)
if headStartRoot == params.BeaconConfig().ZeroHash {
headStartRoot = s.genesisRoot
headStartRoot = s.originBlockRoot
}
// In order to process head, fork choice store requires justified info.
@@ -64,7 +78,7 @@ func (s *Service) updateHead(ctx context.Context, balances []uint64) error {
return err
}
s.cfg.ForkChoiceStore = protoarray.New(j.Epoch, f.Epoch, bytesutil.ToBytes32(f.Root))
if err := s.insertBlockToForkChoiceStore(ctx, jb.Block, headStartRoot, f, j); err != nil {
if err := s.insertBlockToForkChoiceStore(ctx, jb.Block(), headStartRoot, f, j); err != nil {
return err
}
}
@@ -104,8 +118,8 @@ func (s *Service) saveHead(ctx context.Context, headRoot [32]byte) error {
if err != nil {
return err
}
if newHeadBlock == nil || newHeadBlock.Block == nil {
return errors.New("cannot save nil head block")
if err := helpers.BeaconBlockIsNil(newHeadBlock); err != nil {
return err
}
// Get the new head state from cached state or DB.
@@ -113,25 +127,39 @@ func (s *Service) saveHead(ctx context.Context, headRoot [32]byte) error {
if err != nil {
return errors.Wrap(err, "could not retrieve head state in DB")
}
if newHeadState == nil {
if newHeadState == nil || newHeadState.IsNil() {
return errors.New("cannot save nil head state")
}
// A chain re-org occurred, so we fire an event notifying the rest of the services.
headSlot := s.HeadSlot()
if bytesutil.ToBytes32(newHeadBlock.Block.ParentRoot) != bytesutil.ToBytes32(r) {
newHeadSlot := newHeadBlock.Block().Slot()
oldHeadRoot := s.headRoot()
oldStateRoot := s.headBlock().Block().StateRoot()
newStateRoot := newHeadBlock.Block().StateRoot()
if bytesutil.ToBytes32(newHeadBlock.Block().ParentRoot()) != bytesutil.ToBytes32(r) {
log.WithFields(logrus.Fields{
"newSlot": fmt.Sprintf("%d", newHeadBlock.Block.Slot),
"newSlot": fmt.Sprintf("%d", newHeadSlot),
"oldSlot": fmt.Sprintf("%d", headSlot),
}).Debug("Chain reorg occurred")
absoluteSlotDifference := slots.AbsoluteValueSlotDifference(newHeadSlot, headSlot)
s.cfg.StateNotifier.StateFeed().Send(&feed.Event{
Type: statefeed.Reorg,
Data: &statefeed.ReorgData{
NewSlot: newHeadBlock.Block.Slot,
OldSlot: headSlot,
Data: &ethpbv1.EventChainReorg{
Slot: newHeadSlot,
Depth: absoluteSlotDifference,
OldHeadBlock: oldHeadRoot[:],
NewHeadBlock: headRoot[:],
OldHeadState: oldStateRoot,
NewHeadState: newStateRoot,
Epoch: slots.ToEpoch(newHeadSlot),
},
})
if err := s.saveOrphanedAtts(ctx, bytesutil.ToBytes32(r)); err != nil {
return err
}
reorgCount.Inc()
}
@@ -143,14 +171,22 @@ func (s *Service) saveHead(ctx context.Context, headRoot [32]byte) error {
return errors.Wrap(err, "could not save head root in DB")
}
// Forward an event capturing a new chain head over a common event feed
// done in a goroutine to avoid blocking the critical runtime main routine.
go func() {
if err := s.notifyNewHeadEvent(newHeadSlot, newHeadState, newStateRoot, headRoot[:]); err != nil {
log.WithError(err).Error("Could not notify event feed of new chain head")
}
}()
return nil
}
// This gets called to update canonical root mapping. It does not save head block
// root in DB. With the inception of initial-sync-cache-state flag, it uses finalized
// check point as anchors to resume sync therefore head is no longer needed to be saved on per slot basis.
func (s *Service) saveHeadNoDB(ctx context.Context, b *ethpb.SignedBeaconBlock, r [32]byte, hs iface.BeaconState) error {
if err := helpers.VerifyNilBeaconBlock(b); err != nil {
func (s *Service) saveHeadNoDB(ctx context.Context, b block.SignedBeaconBlock, r [32]byte, hs state.BeaconState) error {
if err := helpers.BeaconBlockIsNil(b); err != nil {
return err
}
cachedHeadRoot, err := s.HeadRoot(ctx)
@@ -161,20 +197,20 @@ func (s *Service) saveHeadNoDB(ctx context.Context, b *ethpb.SignedBeaconBlock,
return nil
}
s.setHeadInitialSync(r, stateV0.CopySignedBeaconBlock(b), hs)
s.setHeadInitialSync(r, b.Copy(), hs)
return nil
}
// This sets head view object which is used to track the head slot, root, block and state.
func (s *Service) setHead(root [32]byte, block *ethpb.SignedBeaconBlock, state iface.BeaconState) {
func (s *Service) setHead(root [32]byte, block block.SignedBeaconBlock, state state.BeaconState) {
s.headLock.Lock()
defer s.headLock.Unlock()
// This does a full copy of the block and state.
s.head = &head{
slot: block.Block.Slot,
slot: block.Block().Slot(),
root: root,
block: stateV0.CopySignedBeaconBlock(block),
block: block.Copy(),
state: state.Copy(),
}
}
@@ -182,15 +218,15 @@ func (s *Service) setHead(root [32]byte, block *ethpb.SignedBeaconBlock, state i
// This sets head view object which is used to track the head slot, root, block and state. The method
// assumes that state being passed into the method will not be modified by any other alternate
// caller which holds the state's reference.
func (s *Service) setHeadInitialSync(root [32]byte, block *ethpb.SignedBeaconBlock, state iface.BeaconState) {
func (s *Service) setHeadInitialSync(root [32]byte, block block.SignedBeaconBlock, state state.BeaconState) {
s.headLock.Lock()
defer s.headLock.Unlock()
// This does a full copy of the block only.
s.head = &head{
slot: block.Block.Slot,
slot: block.Block().Slot(),
root: root,
block: stateV0.CopySignedBeaconBlock(block),
block: block.Copy(),
state: state,
}
}
@@ -215,15 +251,15 @@ func (s *Service) headRoot() [32]byte {
// This returns the head block.
// It does a full copy on head block for immutability.
// This is a lock free version.
func (s *Service) headBlock() *ethpb.SignedBeaconBlock {
return stateV0.CopySignedBeaconBlock(s.head.block)
func (s *Service) headBlock() block.SignedBeaconBlock {
return s.head.block.Copy()
}
// This returns the head state.
// It does a full copy on head state for immutability.
// This is a lock free version.
func (s *Service) headState(ctx context.Context) iface.BeaconState {
ctx, span := trace.StartSpan(ctx, "blockChain.headState")
func (s *Service) headState(ctx context.Context) state.BeaconState {
_, span := trace.StartSpan(ctx, "blockChain.headState")
defer span.End()
return s.head.state.Copy()
@@ -235,59 +271,109 @@ func (s *Service) headGenesisValidatorRoot() [32]byte {
return bytesutil.ToBytes32(s.head.state.GenesisValidatorRoot())
}
// This returns the validator referenced by the provided index in
// the head state.
// This is a lock free version.
func (s *Service) headValidatorAtIndex(index types.ValidatorIndex) (state.ReadOnlyValidator, error) {
return s.head.state.ValidatorAtIndexReadOnly(index)
}
// This returns the validator index referenced by the provided pubkey in
// the head state.
// This is a lock free version.
func (s *Service) headValidatorIndexAtPubkey(pubKey [fieldparams.BLSPubkeyLength]byte) (types.ValidatorIndex, bool) {
return s.head.state.ValidatorIndexByPubkey(pubKey)
}
// Returns true if head state exists.
// This is the lock free version.
func (s *Service) hasHeadState() bool {
return s.head != nil && s.head.state != nil
}
// This caches justified state balances to be used for fork choice.
func (s *Service) cacheJustifiedStateBalances(ctx context.Context, justifiedRoot [32]byte) error {
if err := s.cfg.BeaconDB.SaveBlocks(ctx, s.getInitSyncBlocks()); err != nil {
return err
// Notifies a common event feed of a new chain head event. Called right after a new
// chain head is determined, set, and saved to disk.
func (s *Service) notifyNewHeadEvent(
newHeadSlot types.Slot,
newHeadState state.BeaconState,
newHeadStateRoot,
newHeadRoot []byte,
) error {
previousDutyDependentRoot := s.originBlockRoot[:]
currentDutyDependentRoot := s.originBlockRoot[:]
var previousDutyEpoch types.Epoch
currentDutyEpoch := slots.ToEpoch(newHeadSlot)
if currentDutyEpoch > 0 {
previousDutyEpoch = currentDutyEpoch.Sub(1)
}
s.clearInitSyncBlocks()
var justifiedState iface.BeaconState
var err error
if justifiedRoot == s.genesisRoot {
justifiedState, err = s.cfg.BeaconDB.GenesisState(ctx)
currentDutySlot, err := slots.EpochStart(currentDutyEpoch)
if err != nil {
return errors.Wrap(err, "could not get duty slot")
}
previousDutySlot, err := slots.EpochStart(previousDutyEpoch)
if err != nil {
return errors.Wrap(err, "could not get duty slot")
}
if currentDutySlot > 0 {
currentDutyDependentRoot, err = helpers.BlockRootAtSlot(newHeadState, currentDutySlot-1)
if err != nil {
return err
return errors.Wrap(err, "could not get duty dependent root")
}
} else {
justifiedState, err = s.cfg.StateGen.StateByRoot(ctx, justifiedRoot)
}
if previousDutySlot > 0 {
previousDutyDependentRoot, err = helpers.BlockRootAtSlot(newHeadState, previousDutySlot-1)
if err != nil {
return err
return errors.Wrap(err, "could not get duty dependent root")
}
}
if justifiedState == nil {
return errors.New("justified state can't be nil")
}
epoch := helpers.CurrentEpoch(justifiedState)
justifiedBalances := make([]uint64, justifiedState.NumValidators())
if err := justifiedState.ReadFromEveryValidator(func(idx int, val iface.ReadOnlyValidator) error {
if helpers.IsActiveValidatorUsingTrie(val, epoch) {
justifiedBalances[idx] = val.EffectiveBalance()
} else {
justifiedBalances[idx] = 0
}
return nil
}); err != nil {
return err
}
s.justifiedBalancesLock.Lock()
defer s.justifiedBalancesLock.Unlock()
s.justifiedBalances = justifiedBalances
s.cfg.StateNotifier.StateFeed().Send(&feed.Event{
Type: statefeed.NewHead,
Data: &ethpbv1.EventHead{
Slot: newHeadSlot,
Block: newHeadRoot,
State: newHeadStateRoot,
EpochTransition: slots.IsEpochStart(newHeadSlot),
PreviousDutyDependentRoot: previousDutyDependentRoot,
CurrentDutyDependentRoot: currentDutyDependentRoot,
},
})
return nil
}
func (s *Service) getJustifiedBalances() []uint64 {
s.justifiedBalancesLock.RLock()
defer s.justifiedBalancesLock.RUnlock()
return s.justifiedBalances
// This saves the attestations inside the beacon block with respect to root `orphanedRoot` back into the
// attestation pool. It also filters out the attestations that is one epoch older as a
// defense so invalid attestations don't flow into the attestation pool.
func (s *Service) saveOrphanedAtts(ctx context.Context, orphanedRoot [32]byte) error {
if !features.Get().CorrectlyInsertOrphanedAtts {
return nil
}
orphanedBlk, err := s.cfg.BeaconDB.Block(ctx, orphanedRoot)
if err != nil {
return err
}
if orphanedBlk == nil || orphanedBlk.IsNil() {
return errors.New("orphaned block can't be nil")
}
for _, a := range orphanedBlk.Block().Body().Attestations() {
// Is the attestation one epoch older.
if a.Data.Slot+params.BeaconConfig().SlotsPerEpoch < s.CurrentSlot() {
continue
}
if helpers.IsAggregated(a) {
if err := s.cfg.AttPool.SaveAggregatedAttestation(a); err != nil {
return err
}
} else {
if err := s.cfg.AttPool.SaveUnaggregatedAttestation(a); err != nil {
return err
}
}
saveOrphanedAttCount.Inc()
}
return nil
}

View File

@@ -0,0 +1,172 @@
package blockchain
import (
"context"
"fmt"
"github.com/pkg/errors"
types "github.com/prysmaticlabs/eth2-types"
"github.com/prysmaticlabs/prysm/async"
"github.com/prysmaticlabs/prysm/beacon-chain/cache"
"github.com/prysmaticlabs/prysm/beacon-chain/core/altair"
"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
"github.com/prysmaticlabs/prysm/beacon-chain/core/signing"
"github.com/prysmaticlabs/prysm/beacon-chain/core/transition"
"github.com/prysmaticlabs/prysm/beacon-chain/state"
"github.com/prysmaticlabs/prysm/config/params"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/time/slots"
)
// Initialize the state cache for sync committees.
var syncCommitteeHeadStateCache = cache.NewSyncCommitteeHeadState()
// HeadSyncCommitteeFetcher is the interface that wraps the head sync committee related functions.
// The head sync committee functions return callers sync committee indices and public keys with respect to current head state.
type HeadSyncCommitteeFetcher interface {
HeadSyncCommitteeIndices(ctx context.Context, index types.ValidatorIndex, slot types.Slot) ([]types.CommitteeIndex, error)
HeadSyncCommitteePubKeys(ctx context.Context, slot types.Slot, committeeIndex types.CommitteeIndex) ([][]byte, error)
}
// HeadDomainFetcher is the interface that wraps the head sync domain related functions.
// The head sync committee domain functions return callers domain data with respect to slot and head state.
type HeadDomainFetcher interface {
HeadSyncCommitteeDomain(ctx context.Context, slot types.Slot) ([]byte, error)
HeadSyncSelectionProofDomain(ctx context.Context, slot types.Slot) ([]byte, error)
HeadSyncContributionProofDomain(ctx context.Context, slot types.Slot) ([]byte, error)
}
// HeadSyncCommitteeDomain returns the head sync committee domain using current head state advanced up to `slot`.
func (s *Service) HeadSyncCommitteeDomain(ctx context.Context, slot types.Slot) ([]byte, error) {
return s.domainWithHeadState(ctx, slot, params.BeaconConfig().DomainSyncCommittee)
}
// HeadSyncSelectionProofDomain returns the head sync committee domain using current head state advanced up to `slot`.
func (s *Service) HeadSyncSelectionProofDomain(ctx context.Context, slot types.Slot) ([]byte, error) {
return s.domainWithHeadState(ctx, slot, params.BeaconConfig().DomainSyncCommitteeSelectionProof)
}
// HeadSyncContributionProofDomain returns the head sync committee domain using current head state advanced up to `slot`.
func (s *Service) HeadSyncContributionProofDomain(ctx context.Context, slot types.Slot) ([]byte, error) {
return s.domainWithHeadState(ctx, slot, params.BeaconConfig().DomainContributionAndProof)
}
// HeadSyncCommitteeIndices returns the sync committee index position using the head state. Input `slot` is taken in consideration
// where validator's duty for `slot - 1` is used for block inclusion in `slot`. That means when a validator is at epoch boundary
// across EPOCHS_PER_SYNC_COMMITTEE_PERIOD then the valiator will be considered using next period sync committee.
//
// Spec definition:
// Being assigned to a sync committee for a given slot means that the validator produces and broadcasts signatures for slot - 1 for inclusion in slot.
// This means that when assigned to an epoch sync committee signatures must be produced and broadcast for slots on range
// [compute_start_slot_at_epoch(epoch) - 1, compute_start_slot_at_epoch(epoch) + SLOTS_PER_EPOCH - 1)
// rather than for the range
// [compute_start_slot_at_epoch(epoch), compute_start_slot_at_epoch(epoch) + SLOTS_PER_EPOCH)
func (s *Service) HeadSyncCommitteeIndices(ctx context.Context, index types.ValidatorIndex, slot types.Slot) ([]types.CommitteeIndex, error) {
nextSlotEpoch := slots.ToEpoch(slot + 1)
currentEpoch := slots.ToEpoch(slot)
switch {
case slots.SyncCommitteePeriod(nextSlotEpoch) == slots.SyncCommitteePeriod(currentEpoch):
return s.headCurrentSyncCommitteeIndices(ctx, index, slot)
// At sync committee period boundary, validator should sample the next epoch sync committee.
case slots.SyncCommitteePeriod(nextSlotEpoch) == slots.SyncCommitteePeriod(currentEpoch)+1:
return s.headNextSyncCommitteeIndices(ctx, index, slot)
default:
// Impossible condition.
return nil, errors.New("could get calculate sync subcommittee based on the period")
}
}
// headCurrentSyncCommitteeIndices returns the input validator `index`'s position indices in the current sync committee with respect to `slot`.
// Head state advanced up to `slot` is used for calculation.
func (s *Service) headCurrentSyncCommitteeIndices(ctx context.Context, index types.ValidatorIndex, slot types.Slot) ([]types.CommitteeIndex, error) {
headState, err := s.getSyncCommitteeHeadState(ctx, slot)
if err != nil {
return nil, err
}
return helpers.CurrentPeriodSyncSubcommitteeIndices(headState, index)
}
// headNextSyncCommitteeIndices returns the input validator `index`'s position indices in the next sync committee with respect to `slot`.
// Head state advanced up to `slot` is used for calculation.
func (s *Service) headNextSyncCommitteeIndices(ctx context.Context, index types.ValidatorIndex, slot types.Slot) ([]types.CommitteeIndex, error) {
headState, err := s.getSyncCommitteeHeadState(ctx, slot)
if err != nil {
return nil, err
}
return helpers.NextPeriodSyncSubcommitteeIndices(headState, index)
}
// HeadSyncCommitteePubKeys returns the head sync committee public keys with respect to `slot` and subcommittee index `committeeIndex`.
// Head state advanced up to `slot` is used for calculation.
func (s *Service) HeadSyncCommitteePubKeys(ctx context.Context, slot types.Slot, committeeIndex types.CommitteeIndex) ([][]byte, error) {
headState, err := s.getSyncCommitteeHeadState(ctx, slot)
if err != nil {
return nil, err
}
nextSlotEpoch := slots.ToEpoch(headState.Slot() + 1)
currEpoch := slots.ToEpoch(headState.Slot())
var syncCommittee *ethpb.SyncCommittee
if currEpoch == nextSlotEpoch || slots.SyncCommitteePeriod(currEpoch) == slots.SyncCommitteePeriod(nextSlotEpoch) {
syncCommittee, err = headState.CurrentSyncCommittee()
if err != nil {
return nil, err
}
} else {
syncCommittee, err = headState.NextSyncCommittee()
if err != nil {
return nil, err
}
}
return altair.SyncSubCommitteePubkeys(syncCommittee, committeeIndex)
}
// returns calculated domain using input `domain` and `slot`.
func (s *Service) domainWithHeadState(ctx context.Context, slot types.Slot, domain [4]byte) ([]byte, error) {
headState, err := s.getSyncCommitteeHeadState(ctx, slot)
if err != nil {
return nil, err
}
return signing.Domain(headState.Fork(), slots.ToEpoch(headState.Slot()), domain, headState.GenesisValidatorRoot())
}
// returns the head state that is advanced up to `slot`. It utilizes the cache `syncCommitteeHeadState` by retrieving using `slot` as key.
// For the cache miss, it processes head state up to slot and fill the cache with `slot` as key.
func (s *Service) getSyncCommitteeHeadState(ctx context.Context, slot types.Slot) (state.BeaconState, error) {
var headState state.BeaconState
var err error
mLock := async.NewMultilock(fmt.Sprintf("%s-%d", "syncHeadState", slot))
mLock.Lock()
defer mLock.Unlock()
// If there's already a head state exists with the request slot, we don't need to process slots.
cachedState, err := syncCommitteeHeadStateCache.Get(slot)
switch {
case err == nil:
syncHeadStateHit.Inc()
headState = cachedState
return headState, nil
case errors.Is(err, cache.ErrNotFound):
headState, err = s.HeadState(ctx)
if err != nil {
return nil, err
}
if headState == nil || headState.IsNil() {
return nil, errors.New("nil state")
}
headState, err = transition.ProcessSlotsIfPossible(ctx, headState, slot)
if err != nil {
return nil, err
}
syncHeadStateMiss.Inc()
err = syncCommitteeHeadStateCache.Put(slot, headState)
return headState, err
default:
// In the event, we encounter another error
// we return it.
return nil, err
}
}

View File

@@ -0,0 +1,176 @@
package blockchain
import (
"context"
"testing"
types "github.com/prysmaticlabs/eth2-types"
"github.com/prysmaticlabs/prysm/beacon-chain/cache"
"github.com/prysmaticlabs/prysm/beacon-chain/core/signing"
dbtest "github.com/prysmaticlabs/prysm/beacon-chain/db/testing"
"github.com/prysmaticlabs/prysm/beacon-chain/state/stategen"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/testing/require"
"github.com/prysmaticlabs/prysm/testing/util"
"github.com/prysmaticlabs/prysm/time/slots"
)
func TestService_headSyncCommitteeFetcher_Errors(t *testing.T) {
beaconDB := dbtest.SetupDB(t)
c := &Service{
cfg: &config{
StateGen: stategen.New(beaconDB),
},
}
c.head = &head{}
_, err := c.headCurrentSyncCommitteeIndices(context.Background(), types.ValidatorIndex(0), types.Slot(0))
require.ErrorContains(t, "nil state", err)
_, err = c.headNextSyncCommitteeIndices(context.Background(), types.ValidatorIndex(0), types.Slot(0))
require.ErrorContains(t, "nil state", err)
_, err = c.HeadSyncCommitteePubKeys(context.Background(), types.Slot(0), types.CommitteeIndex(0))
require.ErrorContains(t, "nil state", err)
}
func TestService_HeadDomainFetcher_Errors(t *testing.T) {
beaconDB := dbtest.SetupDB(t)
c := &Service{
cfg: &config{
StateGen: stategen.New(beaconDB),
},
}
c.head = &head{}
_, err := c.HeadSyncCommitteeDomain(context.Background(), types.Slot(0))
require.ErrorContains(t, "nil state", err)
_, err = c.HeadSyncSelectionProofDomain(context.Background(), types.Slot(0))
require.ErrorContains(t, "nil state", err)
_, err = c.HeadSyncSelectionProofDomain(context.Background(), types.Slot(0))
require.ErrorContains(t, "nil state", err)
}
func TestService_HeadSyncCommitteeIndices(t *testing.T) {
s, _ := util.DeterministicGenesisStateAltair(t, params.BeaconConfig().TargetCommitteeSize)
c := &Service{}
c.head = &head{state: s}
// Current period
slot := 2*uint64(params.BeaconConfig().EpochsPerSyncCommitteePeriod)*uint64(params.BeaconConfig().SlotsPerEpoch) + 1
a, err := c.HeadSyncCommitteeIndices(context.Background(), 0, types.Slot(slot))
require.NoError(t, err)
// Current period where slot-2 across EPOCHS_PER_SYNC_COMMITTEE_PERIOD
slot = 3*uint64(params.BeaconConfig().EpochsPerSyncCommitteePeriod)*uint64(params.BeaconConfig().SlotsPerEpoch) - 2
b, err := c.HeadSyncCommitteeIndices(context.Background(), 0, types.Slot(slot))
require.NoError(t, err)
require.DeepEqual(t, a, b)
// Next period where slot-1 across EPOCHS_PER_SYNC_COMMITTEE_PERIOD
slot = 3*uint64(params.BeaconConfig().EpochsPerSyncCommitteePeriod)*uint64(params.BeaconConfig().SlotsPerEpoch) - 1
b, err = c.HeadSyncCommitteeIndices(context.Background(), 0, types.Slot(slot))
require.NoError(t, err)
require.DeepNotEqual(t, a, b)
}
func TestService_headCurrentSyncCommitteeIndices(t *testing.T) {
s, _ := util.DeterministicGenesisStateAltair(t, params.BeaconConfig().TargetCommitteeSize)
c := &Service{}
c.head = &head{state: s}
// Process slot up to `EpochsPerSyncCommitteePeriod` so it can `ProcessSyncCommitteeUpdates`.
slot := uint64(params.BeaconConfig().EpochsPerSyncCommitteePeriod)*uint64(params.BeaconConfig().SlotsPerEpoch) + 1
indices, err := c.headCurrentSyncCommitteeIndices(context.Background(), 0, types.Slot(slot))
require.NoError(t, err)
// NextSyncCommittee becomes CurrentSyncCommittee so it should be empty by default.
require.Equal(t, 0, len(indices))
}
func TestService_headNextSyncCommitteeIndices(t *testing.T) {
s, _ := util.DeterministicGenesisStateAltair(t, params.BeaconConfig().TargetCommitteeSize)
c := &Service{}
c.head = &head{state: s}
// Process slot up to `EpochsPerSyncCommitteePeriod` so it can `ProcessSyncCommitteeUpdates`.
slot := uint64(params.BeaconConfig().EpochsPerSyncCommitteePeriod)*uint64(params.BeaconConfig().SlotsPerEpoch) + 1
indices, err := c.headNextSyncCommitteeIndices(context.Background(), 0, types.Slot(slot))
require.NoError(t, err)
// NextSyncCommittee should be be empty after `ProcessSyncCommitteeUpdates`. Validator should get indices.
require.NotEqual(t, 0, len(indices))
}
func TestService_HeadSyncCommitteePubKeys(t *testing.T) {
s, _ := util.DeterministicGenesisStateAltair(t, params.BeaconConfig().TargetCommitteeSize)
c := &Service{}
c.head = &head{state: s}
// Process slot up to 2 * `EpochsPerSyncCommitteePeriod` so it can run `ProcessSyncCommitteeUpdates` twice.
slot := uint64(2*params.BeaconConfig().EpochsPerSyncCommitteePeriod)*uint64(params.BeaconConfig().SlotsPerEpoch) + 1
pubkeys, err := c.HeadSyncCommitteePubKeys(context.Background(), types.Slot(slot), 0)
require.NoError(t, err)
// Any subcommittee should match the subcommittee size.
subCommitteeSize := params.BeaconConfig().SyncCommitteeSize / params.BeaconConfig().SyncCommitteeSubnetCount
require.Equal(t, int(subCommitteeSize), len(pubkeys))
}
func TestService_HeadSyncCommitteeDomain(t *testing.T) {
s, _ := util.DeterministicGenesisStateAltair(t, params.BeaconConfig().TargetCommitteeSize)
c := &Service{}
c.head = &head{state: s}
wanted, err := signing.Domain(s.Fork(), slots.ToEpoch(s.Slot()), params.BeaconConfig().DomainSyncCommittee, s.GenesisValidatorRoot())
require.NoError(t, err)
d, err := c.HeadSyncCommitteeDomain(context.Background(), 0)
require.NoError(t, err)
require.DeepEqual(t, wanted, d)
}
func TestService_HeadSyncContributionProofDomain(t *testing.T) {
s, _ := util.DeterministicGenesisStateAltair(t, params.BeaconConfig().TargetCommitteeSize)
c := &Service{}
c.head = &head{state: s}
wanted, err := signing.Domain(s.Fork(), slots.ToEpoch(s.Slot()), params.BeaconConfig().DomainContributionAndProof, s.GenesisValidatorRoot())
require.NoError(t, err)
d, err := c.HeadSyncContributionProofDomain(context.Background(), 0)
require.NoError(t, err)
require.DeepEqual(t, wanted, d)
}
func TestService_HeadSyncSelectionProofDomain(t *testing.T) {
s, _ := util.DeterministicGenesisStateAltair(t, params.BeaconConfig().TargetCommitteeSize)
c := &Service{}
c.head = &head{state: s}
wanted, err := signing.Domain(s.Fork(), slots.ToEpoch(s.Slot()), params.BeaconConfig().DomainSyncCommitteeSelectionProof, s.GenesisValidatorRoot())
require.NoError(t, err)
d, err := c.HeadSyncSelectionProofDomain(context.Background(), 0)
require.NoError(t, err)
require.DeepEqual(t, wanted, d)
}
func TestSyncCommitteeHeadStateCache_RoundTrip(t *testing.T) {
c := syncCommitteeHeadStateCache
t.Cleanup(func() {
syncCommitteeHeadStateCache = cache.NewSyncCommitteeHeadState()
})
beaconState, _ := util.DeterministicGenesisStateAltair(t, 100)
require.NoError(t, beaconState.SetSlot(100))
cachedState, err := c.Get(101)
require.ErrorContains(t, cache.ErrNotFound.Error(), err)
require.Equal(t, nil, cachedState)
require.NoError(t, c.Put(101, beaconState))
cachedState, err = c.Get(101)
require.NoError(t, err)
require.DeepEqual(t, beaconState, cachedState)
}

View File

@@ -4,14 +4,20 @@ import (
"bytes"
"context"
"testing"
"time"
types "github.com/prysmaticlabs/eth2-types"
ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
mock "github.com/prysmaticlabs/prysm/beacon-chain/blockchain/testing"
testDB "github.com/prysmaticlabs/prysm/beacon-chain/db/testing"
pb "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
"github.com/prysmaticlabs/prysm/shared/testutil"
"github.com/prysmaticlabs/prysm/shared/testutil/assert"
"github.com/prysmaticlabs/prysm/shared/testutil/require"
"github.com/prysmaticlabs/prysm/config/features"
"github.com/prysmaticlabs/prysm/config/params"
ethpbv1 "github.com/prysmaticlabs/prysm/proto/eth/v1"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/wrapper"
"github.com/prysmaticlabs/prysm/testing/assert"
"github.com/prysmaticlabs/prysm/testing/require"
"github.com/prysmaticlabs/prysm/testing/util"
"github.com/prysmaticlabs/prysm/time/slots"
logTest "github.com/sirupsen/logrus/hooks/test"
)
@@ -32,20 +38,30 @@ func TestSaveHead_Different(t *testing.T) {
beaconDB := testDB.SetupDB(t)
service := setupBeaconChain(t, beaconDB)
oldRoot := [32]byte{'A'}
service.head = &head{slot: 0, root: oldRoot}
util.NewBeaconBlock()
oldBlock := wrapper.WrappedPhase0SignedBeaconBlock(
util.NewBeaconBlock(),
)
require.NoError(t, service.cfg.BeaconDB.SaveBlock(context.Background(), oldBlock))
oldRoot, err := oldBlock.Block().HashTreeRoot()
require.NoError(t, err)
service.head = &head{
slot: 0,
root: oldRoot,
block: oldBlock,
}
newHeadSignedBlock := testutil.NewBeaconBlock()
newHeadSignedBlock := util.NewBeaconBlock()
newHeadSignedBlock.Block.Slot = 1
newHeadBlock := newHeadSignedBlock.Block
require.NoError(t, service.cfg.BeaconDB.SaveBlock(context.Background(), newHeadSignedBlock))
require.NoError(t, service.cfg.BeaconDB.SaveBlock(context.Background(), wrapper.WrappedPhase0SignedBeaconBlock(newHeadSignedBlock)))
newRoot, err := newHeadBlock.HashTreeRoot()
require.NoError(t, err)
headState, err := testutil.NewBeaconState()
headState, err := util.NewBeaconState()
require.NoError(t, err)
require.NoError(t, headState.SetSlot(1))
require.NoError(t, service.cfg.BeaconDB.SaveStateSummary(context.Background(), &pb.StateSummary{Slot: 1, Root: newRoot[:]}))
require.NoError(t, service.cfg.BeaconDB.SaveStateSummary(context.Background(), &ethpb.StateSummary{Slot: 1, Root: newRoot[:]}))
require.NoError(t, service.cfg.BeaconDB.SaveState(context.Background(), headState, newRoot))
require.NoError(t, service.saveHead(context.Background(), newRoot))
@@ -54,7 +70,7 @@ func TestSaveHead_Different(t *testing.T) {
cachedRoot, err := service.HeadRoot(context.Background())
require.NoError(t, err)
assert.DeepEqual(t, cachedRoot, newRoot[:], "Head did not change")
assert.DeepEqual(t, newHeadSignedBlock, service.headBlock(), "Head did not change")
assert.DeepEqual(t, newHeadSignedBlock, service.headBlock().Proto(), "Head did not change")
assert.DeepSSZEqual(t, headState.CloneInnerState(), service.headState(ctx).CloneInnerState(), "Head did not change")
}
@@ -64,22 +80,31 @@ func TestSaveHead_Different_Reorg(t *testing.T) {
beaconDB := testDB.SetupDB(t)
service := setupBeaconChain(t, beaconDB)
oldRoot := [32]byte{'A'}
service.head = &head{slot: 0, root: oldRoot}
oldBlock := wrapper.WrappedPhase0SignedBeaconBlock(
util.NewBeaconBlock(),
)
require.NoError(t, service.cfg.BeaconDB.SaveBlock(context.Background(), oldBlock))
oldRoot, err := oldBlock.Block().HashTreeRoot()
require.NoError(t, err)
service.head = &head{
slot: 0,
root: oldRoot,
block: oldBlock,
}
reorgChainParent := [32]byte{'B'}
newHeadSignedBlock := testutil.NewBeaconBlock()
newHeadSignedBlock := util.NewBeaconBlock()
newHeadSignedBlock.Block.Slot = 1
newHeadSignedBlock.Block.ParentRoot = reorgChainParent[:]
newHeadBlock := newHeadSignedBlock.Block
require.NoError(t, service.cfg.BeaconDB.SaveBlock(context.Background(), newHeadSignedBlock))
require.NoError(t, service.cfg.BeaconDB.SaveBlock(context.Background(), wrapper.WrappedPhase0SignedBeaconBlock(newHeadSignedBlock)))
newRoot, err := newHeadBlock.HashTreeRoot()
require.NoError(t, err)
headState, err := testutil.NewBeaconState()
headState, err := util.NewBeaconState()
require.NoError(t, err)
require.NoError(t, headState.SetSlot(1))
require.NoError(t, service.cfg.BeaconDB.SaveStateSummary(context.Background(), &pb.StateSummary{Slot: 1, Root: newRoot[:]}))
require.NoError(t, service.cfg.BeaconDB.SaveStateSummary(context.Background(), &ethpb.StateSummary{Slot: 1, Root: newRoot[:]}))
require.NoError(t, service.cfg.BeaconDB.SaveState(context.Background(), headState, newRoot))
require.NoError(t, service.saveHead(context.Background(), newRoot))
@@ -90,7 +115,7 @@ func TestSaveHead_Different_Reorg(t *testing.T) {
if !bytes.Equal(cachedRoot, newRoot[:]) {
t.Error("Head did not change")
}
assert.DeepEqual(t, newHeadSignedBlock, service.headBlock(), "Head did not change")
assert.DeepEqual(t, newHeadSignedBlock, service.headBlock().Proto(), "Head did not change")
assert.DeepSSZEqual(t, headState.CloneInnerState(), service.headState(ctx).CloneInnerState(), "Head did not change")
require.LogsContain(t, hook, "Chain reorg occurred")
}
@@ -98,27 +123,147 @@ func TestSaveHead_Different_Reorg(t *testing.T) {
func TestCacheJustifiedStateBalances_CanCache(t *testing.T) {
beaconDB := testDB.SetupDB(t)
service := setupBeaconChain(t, beaconDB)
ctx := context.Background()
state, _ := testutil.DeterministicGenesisState(t, 100)
state, _ := util.DeterministicGenesisState(t, 100)
r := [32]byte{'a'}
require.NoError(t, service.cfg.BeaconDB.SaveStateSummary(context.Background(), &pb.StateSummary{Root: r[:]}))
require.NoError(t, service.cfg.BeaconDB.SaveStateSummary(context.Background(), &ethpb.StateSummary{Root: r[:]}))
require.NoError(t, service.cfg.BeaconDB.SaveState(context.Background(), state, r))
require.NoError(t, service.cacheJustifiedStateBalances(context.Background(), r))
require.DeepEqual(t, service.getJustifiedBalances(), state.Balances(), "Incorrect justified balances")
balances, err := service.justifiedBalances.get(ctx, r)
require.NoError(t, err)
require.DeepEqual(t, balances, state.Balances(), "Incorrect justified balances")
}
func TestUpdateHead_MissingJustifiedRoot(t *testing.T) {
beaconDB := testDB.SetupDB(t)
service := setupBeaconChain(t, beaconDB)
b := testutil.NewBeaconBlock()
require.NoError(t, service.cfg.BeaconDB.SaveBlock(context.Background(), b))
b := util.NewBeaconBlock()
require.NoError(t, service.cfg.BeaconDB.SaveBlock(context.Background(), wrapper.WrappedPhase0SignedBeaconBlock(b)))
r, err := b.Block.HashTreeRoot()
require.NoError(t, err)
service.justifiedCheckpt = &ethpb.Checkpoint{Root: r[:]}
service.finalizedCheckpt = &ethpb.Checkpoint{}
service.bestJustifiedCheckpt = &ethpb.Checkpoint{}
service.store.SetJustifiedCheckpt(&ethpb.Checkpoint{Root: r[:]})
service.store.SetFinalizedCheckpt(&ethpb.Checkpoint{})
service.store.SetBestJustifiedCheckpt(&ethpb.Checkpoint{})
require.NoError(t, service.updateHead(context.Background(), []uint64{}))
}
func Test_notifyNewHeadEvent(t *testing.T) {
t.Run("genesis_state_root", func(t *testing.T) {
bState, _ := util.DeterministicGenesisState(t, 10)
notifier := &mock.MockStateNotifier{RecordEvents: true}
srv := &Service{
cfg: &config{
StateNotifier: notifier,
},
originBlockRoot: [32]byte{1},
}
newHeadStateRoot := [32]byte{2}
newHeadRoot := [32]byte{3}
err := srv.notifyNewHeadEvent(1, bState, newHeadStateRoot[:], newHeadRoot[:])
require.NoError(t, err)
events := notifier.ReceivedEvents()
require.Equal(t, 1, len(events))
eventHead, ok := events[0].Data.(*ethpbv1.EventHead)
require.Equal(t, true, ok)
wanted := &ethpbv1.EventHead{
Slot: 1,
Block: newHeadRoot[:],
State: newHeadStateRoot[:],
EpochTransition: false,
PreviousDutyDependentRoot: srv.originBlockRoot[:],
CurrentDutyDependentRoot: srv.originBlockRoot[:],
}
require.DeepSSZEqual(t, wanted, eventHead)
})
t.Run("non_genesis_values", func(t *testing.T) {
bState, _ := util.DeterministicGenesisState(t, 10)
notifier := &mock.MockStateNotifier{RecordEvents: true}
genesisRoot := [32]byte{1}
srv := &Service{
cfg: &config{
StateNotifier: notifier,
},
originBlockRoot: genesisRoot,
}
epoch1Start, err := slots.EpochStart(1)
require.NoError(t, err)
epoch2Start, err := slots.EpochStart(1)
require.NoError(t, err)
require.NoError(t, bState.SetSlot(epoch1Start))
newHeadStateRoot := [32]byte{2}
newHeadRoot := [32]byte{3}
err = srv.notifyNewHeadEvent(epoch2Start, bState, newHeadStateRoot[:], newHeadRoot[:])
require.NoError(t, err)
events := notifier.ReceivedEvents()
require.Equal(t, 1, len(events))
eventHead, ok := events[0].Data.(*ethpbv1.EventHead)
require.Equal(t, true, ok)
wanted := &ethpbv1.EventHead{
Slot: epoch2Start,
Block: newHeadRoot[:],
State: newHeadStateRoot[:],
EpochTransition: true,
PreviousDutyDependentRoot: genesisRoot[:],
CurrentDutyDependentRoot: make([]byte, 32),
}
require.DeepSSZEqual(t, wanted, eventHead)
})
}
func TestSaveOrphanedAtts(t *testing.T) {
resetCfg := features.InitWithReset(&features.Flags{
CorrectlyInsertOrphanedAtts: true,
})
defer resetCfg()
genesis, keys := util.DeterministicGenesisState(t, 64)
b, err := util.GenerateFullBlock(genesis, keys, util.DefaultBlockGenConfig(), 1)
assert.NoError(t, err)
r, err := b.Block.HashTreeRoot()
require.NoError(t, err)
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
service := setupBeaconChain(t, beaconDB)
service.genesisTime = time.Now()
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(b)))
require.NoError(t, service.saveOrphanedAtts(ctx, r))
require.Equal(t, len(b.Block.Body.Attestations), service.cfg.AttPool.AggregatedAttestationCount())
savedAtts := service.cfg.AttPool.AggregatedAttestations()
atts := b.Block.Body.Attestations
require.DeepSSZEqual(t, atts, savedAtts)
}
func TestSaveOrphanedAtts_CanFilter(t *testing.T) {
resetCfg := features.InitWithReset(&features.Flags{
CorrectlyInsertOrphanedAtts: true,
})
defer resetCfg()
genesis, keys := util.DeterministicGenesisState(t, 64)
b, err := util.GenerateFullBlock(genesis, keys, util.DefaultBlockGenConfig(), 1)
assert.NoError(t, err)
r, err := b.Block.HashTreeRoot()
require.NoError(t, err)
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
service := setupBeaconChain(t, beaconDB)
service.genesisTime = time.Now().Add(time.Duration(-1*int64(params.BeaconConfig().SlotsPerEpoch+1)*int64(params.BeaconConfig().SecondsPerSlot)) * time.Second)
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(b)))
require.NoError(t, service.saveOrphanedAtts(ctx, r))
require.Equal(t, 0, service.cfg.AttPool.AggregatedAttestationCount())
savedAtts := service.cfg.AttPool.AggregatedAttestations()
atts := b.Block.Body.Attestations
require.DeepNotSSZEqual(t, atts, savedAtts)
}

View File

@@ -6,7 +6,7 @@ import (
"net/http"
"github.com/emicklei/dot"
"github.com/prysmaticlabs/prysm/shared/params"
"github.com/prysmaticlabs/prysm/config/params"
)
const template = `<html>
@@ -39,7 +39,7 @@ func (s *Service) TreeHandler(w http.ResponseWriter, r *http.Request) {
log.WithError(err).Error("Could not get head state")
return
}
if headState == nil {
if headState == nil || headState.IsNil() {
if _, err := w.Write([]byte("Unavailable during initial syncing")); err != nil {
log.WithError(err).Error("Failed to render p2p info page")
}

View File

@@ -9,10 +9,11 @@ import (
testDB "github.com/prysmaticlabs/prysm/beacon-chain/db/testing"
"github.com/prysmaticlabs/prysm/beacon-chain/forkchoice/protoarray"
"github.com/prysmaticlabs/prysm/beacon-chain/state/stategen"
"github.com/prysmaticlabs/prysm/shared/params"
"github.com/prysmaticlabs/prysm/shared/testutil"
"github.com/prysmaticlabs/prysm/shared/testutil/assert"
"github.com/prysmaticlabs/prysm/shared/testutil/require"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/wrapper"
"github.com/prysmaticlabs/prysm/testing/assert"
"github.com/prysmaticlabs/prysm/testing/require"
"github.com/prysmaticlabs/prysm/testing/util"
)
func TestService_TreeHandler(t *testing.T) {
@@ -21,23 +22,24 @@ func TestService_TreeHandler(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
headState, err := testutil.NewBeaconState()
headState, err := util.NewBeaconState()
require.NoError(t, err)
require.NoError(t, headState.SetBalances([]uint64{params.BeaconConfig().GweiPerEth}))
cfg := &Config{
BeaconDB: beaconDB,
ForkChoiceStore: protoarray.New(
0, // justifiedEpoch
0, // finalizedEpoch
[32]byte{'a'},
),
StateGen: stategen.New(beaconDB),
fcs := protoarray.New(
0, // justifiedEpoch
0, // finalizedEpoch
[32]byte{'a'},
)
opts := []Option{
WithDatabase(beaconDB),
WithStateGen(stategen.New(beaconDB)),
WithForkChoiceStore(fcs),
}
s, err := NewService(ctx, cfg)
s, err := NewService(ctx, opts...)
require.NoError(t, err)
require.NoError(t, s.cfg.ForkChoiceStore.ProcessBlock(ctx, 0, [32]byte{'a'}, [32]byte{'g'}, [32]byte{'c'}, 0, 0))
require.NoError(t, s.cfg.ForkChoiceStore.ProcessBlock(ctx, 1, [32]byte{'b'}, [32]byte{'a'}, [32]byte{'c'}, 0, 0))
s.setHead([32]byte{'a'}, testutil.NewBeaconBlock(), headState)
s.setHead([32]byte{'a'}, wrapper.WrappedPhase0SignedBeaconBlock(util.NewBeaconBlock()), headState)
rr := httptest.NewRecorder()
handler := http.HandlerFunc(s.TreeHandler)

View File

@@ -1,11 +1,11 @@
package blockchain
import (
ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/block"
)
// This saves a beacon block to the initial sync blocks cache.
func (s *Service) saveInitSyncBlock(r [32]byte, b *ethpb.SignedBeaconBlock) {
func (s *Service) saveInitSyncBlock(r [32]byte, b block.SignedBeaconBlock) {
s.initSyncBlocksLock.Lock()
defer s.initSyncBlocksLock.Unlock()
s.initSyncBlocks[r] = b
@@ -22,7 +22,7 @@ func (s *Service) hasInitSyncBlock(r [32]byte) bool {
// This retrieves a beacon block from the initial sync blocks cache using the root of
// the block.
func (s *Service) getInitSyncBlock(r [32]byte) *ethpb.SignedBeaconBlock {
func (s *Service) getInitSyncBlock(r [32]byte) block.SignedBeaconBlock {
s.initSyncBlocksLock.RLock()
defer s.initSyncBlocksLock.RUnlock()
b := s.initSyncBlocks[r]
@@ -31,11 +31,11 @@ func (s *Service) getInitSyncBlock(r [32]byte) *ethpb.SignedBeaconBlock {
// This retrieves all the beacon blocks from the initial sync blocks cache, the returned
// blocks are unordered.
func (s *Service) getInitSyncBlocks() []*ethpb.SignedBeaconBlock {
func (s *Service) getInitSyncBlocks() []block.SignedBeaconBlock {
s.initSyncBlocksLock.RLock()
defer s.initSyncBlocksLock.RUnlock()
blks := make([]*ethpb.SignedBeaconBlock, 0, len(s.initSyncBlocks))
blks := make([]block.SignedBeaconBlock, 0, len(s.initSyncBlocks))
for _, b := range s.initSyncBlocks {
blks = append(blks, b)
}
@@ -46,5 +46,5 @@ func (s *Service) getInitSyncBlocks() []*ethpb.SignedBeaconBlock {
func (s *Service) clearInitSyncBlocks() {
s.initSyncBlocksLock.Lock()
defer s.initSyncBlocksLock.Unlock()
s.initSyncBlocks = make(map[[32]byte]*ethpb.SignedBeaconBlock)
s.initSyncBlocks = make(map[[32]byte]block.SignedBeaconBlock)
}

View File

@@ -1,7 +1,7 @@
package blockchain
import (
"github.com/prysmaticlabs/prysm/shared/params"
"github.com/prysmaticlabs/prysm/config/params"
)
func init() {

View File

@@ -5,53 +5,63 @@ import (
"fmt"
"time"
ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
"github.com/prysmaticlabs/prysm/shared/params"
"github.com/prysmaticlabs/prysm/shared/timeutils"
"github.com/prysmaticlabs/prysm/config/params"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/block"
"github.com/prysmaticlabs/prysm/runtime/version"
prysmTime "github.com/prysmaticlabs/prysm/time"
"github.com/prysmaticlabs/prysm/time/slots"
"github.com/sirupsen/logrus"
)
var log = logrus.WithField("prefix", "blockchain")
// logs state transition related data every slot.
func logStateTransitionData(b *ethpb.BeaconBlock) {
log := log.WithField("slot", b.Slot)
if len(b.Body.Attestations) > 0 {
log = log.WithField("attestations", len(b.Body.Attestations))
func logStateTransitionData(b block.BeaconBlock) {
log := log.WithField("slot", b.Slot())
if len(b.Body().Attestations()) > 0 {
log = log.WithField("attestations", len(b.Body().Attestations()))
}
if len(b.Body.Deposits) > 0 {
log = log.WithField("deposits", len(b.Body.Deposits))
if len(b.Body().Deposits()) > 0 {
log = log.WithField("deposits", len(b.Body().Deposits()))
}
if len(b.Body.AttesterSlashings) > 0 {
log = log.WithField("attesterSlashings", len(b.Body.AttesterSlashings))
if len(b.Body().AttesterSlashings()) > 0 {
log = log.WithField("attesterSlashings", len(b.Body().AttesterSlashings()))
}
if len(b.Body.ProposerSlashings) > 0 {
log = log.WithField("proposerSlashings", len(b.Body.ProposerSlashings))
if len(b.Body().ProposerSlashings()) > 0 {
log = log.WithField("proposerSlashings", len(b.Body().ProposerSlashings()))
}
if len(b.Body.VoluntaryExits) > 0 {
log = log.WithField("voluntaryExits", len(b.Body.VoluntaryExits))
if len(b.Body().VoluntaryExits()) > 0 {
log = log.WithField("voluntaryExits", len(b.Body().VoluntaryExits()))
}
if b.Version() == version.Altair {
agg, err := b.Body().SyncAggregate()
if err == nil {
log = log.WithField("syncBitsCount", agg.SyncCommitteeBits.Count())
}
}
log.Info("Finished applying state transition")
}
func logBlockSyncStatus(block *ethpb.BeaconBlock, blockRoot [32]byte, finalized *ethpb.Checkpoint, receivedTime time.Time, genesisTime uint64) error {
startTime, err := helpers.SlotToTime(genesisTime, block.Slot)
func logBlockSyncStatus(block block.BeaconBlock, blockRoot [32]byte, finalized *ethpb.Checkpoint, receivedTime time.Time, genesisTime uint64) error {
startTime, err := slots.ToTime(genesisTime, block.Slot())
if err != nil {
return err
}
log.WithFields(logrus.Fields{
"slot": block.Slot,
"slotInEpoch": block.Slot % params.BeaconConfig().SlotsPerEpoch,
"slot": block.Slot(),
"slotInEpoch": block.Slot() % params.BeaconConfig().SlotsPerEpoch,
"block": fmt.Sprintf("0x%s...", hex.EncodeToString(blockRoot[:])[:8]),
"epoch": helpers.SlotToEpoch(block.Slot),
"epoch": slots.ToEpoch(block.Slot()),
"finalizedEpoch": finalized.Epoch,
"finalizedRoot": fmt.Sprintf("0x%s...", hex.EncodeToString(finalized.Root)[:8]),
"parentRoot": fmt.Sprintf("0x%s...", hex.EncodeToString(block.ParentRoot())[:8]),
"version": version.String(block.Version()),
}).Info("Synced new block")
log.WithFields(logrus.Fields{
"slot": block.Slot,
"sinceSlotStartTime": timeutils.Now().Sub(startTime),
"chainServiceProcessedTime": timeutils.Now().Sub(receivedTime),
"sinceSlotStartTime": prysmTime.Now().Sub(startTime),
"chainServiceProcessedTime": prysmTime.Now().Sub(receivedTime),
}).Debug("Sync new block times")
return nil
}

View File

@@ -3,53 +3,56 @@ package blockchain
import (
"testing"
ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
"github.com/prysmaticlabs/prysm/shared/testutil/require"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/block"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/wrapper"
"github.com/prysmaticlabs/prysm/testing/require"
logTest "github.com/sirupsen/logrus/hooks/test"
)
func Test_logStateTransitionData(t *testing.T) {
tests := []struct {
name string
b *ethpb.BeaconBlock
b block.BeaconBlock
want string
}{
{name: "empty block body",
b: &ethpb.BeaconBlock{Body: &ethpb.BeaconBlockBody{}},
b: wrapper.WrappedPhase0BeaconBlock(&ethpb.BeaconBlock{Body: &ethpb.BeaconBlockBody{}}),
want: "\"Finished applying state transition\" prefix=blockchain slot=0",
},
{name: "has attestation",
b: &ethpb.BeaconBlock{Body: &ethpb.BeaconBlockBody{Attestations: []*ethpb.Attestation{{}}}},
b: wrapper.WrappedPhase0BeaconBlock(&ethpb.BeaconBlock{Body: &ethpb.BeaconBlockBody{Attestations: []*ethpb.Attestation{{}}}}),
want: "\"Finished applying state transition\" attestations=1 prefix=blockchain slot=0",
},
{name: "has deposit",
b: &ethpb.BeaconBlock{Body: &ethpb.BeaconBlockBody{
Attestations: []*ethpb.Attestation{{}},
Deposits: []*ethpb.Deposit{{}}}},
b: wrapper.WrappedPhase0BeaconBlock(
&ethpb.BeaconBlock{Body: &ethpb.BeaconBlockBody{
Attestations: []*ethpb.Attestation{{}},
Deposits: []*ethpb.Deposit{{}}}}),
want: "\"Finished applying state transition\" attestations=1 deposits=1 prefix=blockchain slot=0",
},
{name: "has attester slashing",
b: &ethpb.BeaconBlock{Body: &ethpb.BeaconBlockBody{
AttesterSlashings: []*ethpb.AttesterSlashing{{}}}},
b: wrapper.WrappedPhase0BeaconBlock(&ethpb.BeaconBlock{Body: &ethpb.BeaconBlockBody{
AttesterSlashings: []*ethpb.AttesterSlashing{{}}}}),
want: "\"Finished applying state transition\" attesterSlashings=1 prefix=blockchain slot=0",
},
{name: "has proposer slashing",
b: &ethpb.BeaconBlock{Body: &ethpb.BeaconBlockBody{
ProposerSlashings: []*ethpb.ProposerSlashing{{}}}},
b: wrapper.WrappedPhase0BeaconBlock(&ethpb.BeaconBlock{Body: &ethpb.BeaconBlockBody{
ProposerSlashings: []*ethpb.ProposerSlashing{{}}}}),
want: "\"Finished applying state transition\" prefix=blockchain proposerSlashings=1 slot=0",
},
{name: "has exit",
b: &ethpb.BeaconBlock{Body: &ethpb.BeaconBlockBody{
VoluntaryExits: []*ethpb.SignedVoluntaryExit{{}}}},
b: wrapper.WrappedPhase0BeaconBlock(&ethpb.BeaconBlock{Body: &ethpb.BeaconBlockBody{
VoluntaryExits: []*ethpb.SignedVoluntaryExit{{}}}}),
want: "\"Finished applying state transition\" prefix=blockchain slot=0 voluntaryExits=1",
},
{name: "has everything",
b: &ethpb.BeaconBlock{Body: &ethpb.BeaconBlockBody{
b: wrapper.WrappedPhase0BeaconBlock(&ethpb.BeaconBlock{Body: &ethpb.BeaconBlockBody{
Attestations: []*ethpb.Attestation{{}},
Deposits: []*ethpb.Deposit{{}},
AttesterSlashings: []*ethpb.AttesterSlashing{{}},
ProposerSlashings: []*ethpb.ProposerSlashing{{}},
VoluntaryExits: []*ethpb.SignedVoluntaryExit{{}}}},
VoluntaryExits: []*ethpb.SignedVoluntaryExit{{}}}}),
want: "\"Finished applying state transition\" attestations=1 attesterSlashings=1 deposits=1 prefix=blockchain proposerSlashings=1 slot=0 voluntaryExits=1",
},
}

View File

@@ -3,14 +3,18 @@ package blockchain
import (
"context"
"github.com/pkg/errors"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promauto"
types "github.com/prysmaticlabs/eth2-types"
ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
"github.com/prysmaticlabs/prysm/beacon-chain/core/altair"
"github.com/prysmaticlabs/prysm/beacon-chain/core/epoch/precompute"
iface "github.com/prysmaticlabs/prysm/beacon-chain/state/interface"
"github.com/prysmaticlabs/prysm/shared/bytesutil"
"github.com/prysmaticlabs/prysm/shared/params"
"github.com/prysmaticlabs/prysm/beacon-chain/state"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/block"
"github.com/prysmaticlabs/prysm/runtime/version"
)
var (
@@ -59,6 +63,10 @@ var (
Name: "beacon_previous_justified_root",
Help: "Previous justified root of the processed state",
})
activeValidatorCount = promauto.NewGauge(prometheus.GaugeOpts{
Name: "beacon_current_active_validators",
Help: "Current total active validators",
})
validatorsCount = promauto.NewGaugeVec(prometheus.GaugeOpts{
Name: "validator_count",
Help: "The total number of validators",
@@ -75,6 +83,10 @@ var (
Name: "current_eth1_data_deposit_count",
Help: "The current eth1 deposit count in the last processed state eth1data field.",
})
processedDepositsCount = promauto.NewGauge(prometheus.GaugeOpts{
Name: "beacon_processed_deposits_total",
Help: "Total number of deposits processed",
})
stateTrieReferences = promauto.NewGaugeVec(prometheus.GaugeOpts{
Name: "field_references",
Help: "The number of states a particular field is shared with.",
@@ -96,9 +108,13 @@ var (
Help: "The total amount of ether, in gwei, that has been used in voting attestation head of previous epoch",
})
reorgCount = promauto.NewCounter(prometheus.CounterOpts{
Name: "beacon_reorg_total",
Name: "beacon_reorgs_total",
Help: "Count the number of times beacon chain has a reorg",
})
saveOrphanedAttCount = promauto.NewCounter(prometheus.CounterOpts{
Name: "saved_orphaned_att_total",
Help: "Count the number of times an orphaned attestation is saved",
})
attestationInclusionDelay = promauto.NewHistogram(
prometheus.HistogramOpts{
Name: "attestation_inclusion_delay_slots",
@@ -106,6 +122,22 @@ var (
Buckets: []float64{1, 2, 3, 4, 6, 32, 64},
},
)
syncHeadStateMiss = promauto.NewCounter(prometheus.CounterOpts{
Name: "sync_head_state_miss",
Help: "The number of sync head state requests that are not present in the cache.",
})
syncHeadStateHit = promauto.NewCounter(prometheus.CounterOpts{
Name: "sync_head_state_hit",
Help: "The number of sync head state requests that are present in the cache.",
})
stateBalanceCacheHit = promauto.NewCounter(prometheus.CounterOpts{
Name: "state_balance_cache_hit",
Help: "Count the number of state balance cache hits.",
})
stateBalanceCacheMiss = promauto.NewCounter(prometheus.CounterOpts{
Name: "state_balance_cache_miss",
Help: "Count the number of state balance cache hits.",
})
)
// reportSlotMetrics reports slot related metrics.
@@ -120,7 +152,7 @@ func reportSlotMetrics(stateSlot, headSlot, clockSlot types.Slot, finalizedCheck
}
// reportEpochMetrics reports epoch related metrics.
func reportEpochMetrics(ctx context.Context, postState, headState iface.BeaconState) error {
func reportEpochMetrics(ctx context.Context, postState, headState state.BeaconState) error {
currentEpoch := types.Epoch(postState.Slot() / params.BeaconConfig().SlotsPerEpoch)
// Validator instances
@@ -178,6 +210,7 @@ func reportEpochMetrics(ctx context.Context, postState, headState iface.BeaconSt
activeBalance += exitingBalance + slashingBalance
activeEffectiveBalance += exitingEffectiveBalance + slashingEffectiveBalance
activeValidatorCount.Set(float64(activeInstances))
validatorsCount.WithLabelValues("Pending").Set(float64(pendingInstances))
validatorsCount.WithLabelValues("Active").Set(float64(activeInstances))
validatorsCount.WithLabelValues("Exiting").Set(float64(exitingInstances))
@@ -204,15 +237,33 @@ func reportEpochMetrics(ctx context.Context, postState, headState iface.BeaconSt
beaconFinalizedEpoch.Set(float64(postState.FinalizedCheckpointEpoch()))
beaconFinalizedRoot.Set(float64(bytesutil.ToLowInt64(postState.FinalizedCheckpoint().Root)))
currentEth1DataDepositCount.Set(float64(postState.Eth1Data().DepositCount))
processedDepositsCount.Set(float64(postState.Eth1DepositIndex() + 1))
// Validator participation should be viewed on the canonical chain.
v, b, err := precompute.New(ctx, headState)
if err != nil {
return err
}
_, b, err = precompute.ProcessAttestations(ctx, headState, v, b)
if err != nil {
return err
var b *precompute.Balance
var v []*precompute.Validator
var err error
switch headState.Version() {
case version.Phase0:
// Validator participation should be viewed on the canonical chain.
v, b, err = precompute.New(ctx, headState)
if err != nil {
return err
}
_, b, err = precompute.ProcessAttestations(ctx, headState, v, b)
if err != nil {
return err
}
case version.Altair, version.Bellatrix:
v, b, err = altair.InitializePrecomputeValidators(ctx, headState)
if err != nil {
return err
}
_, b, err = altair.ProcessEpochParticipation(ctx, headState, b, v)
if err != nil {
return err
}
default:
return errors.Errorf("invalid state type provided: %T", headState.InnerStateUnsafe())
}
prevEpochActiveBalances.Set(float64(b.ActivePrevEpoch))
prevEpochSourceBalances.Set(float64(b.PrevEpochAttested))
@@ -227,8 +278,8 @@ func reportEpochMetrics(ctx context.Context, postState, headState iface.BeaconSt
return nil
}
func reportAttestationInclusion(blk *ethpb.BeaconBlock) {
for _, att := range blk.Body.Attestations {
attestationInclusionDelay.Observe(float64(blk.Slot - att.Data.Slot))
func reportAttestationInclusion(blk block.BeaconBlock) {
for _, att := range blk.Body().Attestations() {
attestationInclusionDelay.Observe(float64(blk.Slot() - att.Data.Slot))
}
}

View File

@@ -4,16 +4,15 @@ import (
"context"
"testing"
eth "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
pb "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
"github.com/prysmaticlabs/prysm/shared/testutil"
"github.com/prysmaticlabs/prysm/shared/testutil/require"
eth "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/testing/require"
"github.com/prysmaticlabs/prysm/testing/util"
)
func TestReportEpochMetrics_BadHeadState(t *testing.T) {
s, err := testutil.NewBeaconState()
s, err := util.NewBeaconState()
require.NoError(t, err)
h, err := testutil.NewBeaconState()
h, err := util.NewBeaconState()
require.NoError(t, err)
require.NoError(t, h.SetValidators(nil))
err = reportEpochMetrics(context.Background(), s, h)
@@ -21,22 +20,22 @@ func TestReportEpochMetrics_BadHeadState(t *testing.T) {
}
func TestReportEpochMetrics_BadAttestation(t *testing.T) {
s, err := testutil.NewBeaconState()
s, err := util.NewBeaconState()
require.NoError(t, err)
h, err := testutil.NewBeaconState()
h, err := util.NewBeaconState()
require.NoError(t, err)
require.NoError(t, h.AppendCurrentEpochAttestations(&pb.PendingAttestation{InclusionDelay: 0}))
require.NoError(t, h.AppendCurrentEpochAttestations(&eth.PendingAttestation{InclusionDelay: 0}))
err = reportEpochMetrics(context.Background(), s, h)
require.ErrorContains(t, "attestation with inclusion delay of 0", err)
}
func TestReportEpochMetrics_SlashedValidatorOutOfBound(t *testing.T) {
h, _ := testutil.DeterministicGenesisState(t, 1)
h, _ := util.DeterministicGenesisState(t, 1)
v, err := h.ValidatorAtIndex(0)
require.NoError(t, err)
v.Slashed = true
require.NoError(t, h.UpdateValidatorAtIndex(0, v))
require.NoError(t, h.AppendCurrentEpochAttestations(&pb.PendingAttestation{InclusionDelay: 1, Data: testutil.HydrateAttestationData(&eth.AttestationData{})}))
require.NoError(t, h.AppendCurrentEpochAttestations(&eth.PendingAttestation{InclusionDelay: 1, Data: util.HydrateAttestationData(&eth.AttestationData{})}))
err = reportEpochMetrics(context.Background(), h, h)
require.ErrorContains(t, "slot 0 out of bounds", err)
}

View File

@@ -0,0 +1,50 @@
package blockchain
import (
"context"
"errors"
"testing"
testDB "github.com/prysmaticlabs/prysm/beacon-chain/db/testing"
"github.com/prysmaticlabs/prysm/beacon-chain/forkchoice/protoarray"
"github.com/prysmaticlabs/prysm/beacon-chain/state"
"github.com/prysmaticlabs/prysm/beacon-chain/state/stategen"
)
func testServiceOptsWithDB(t *testing.T) []Option {
beaconDB := testDB.SetupDB(t)
fcs := protoarray.New(0, 0, [32]byte{'a'})
return []Option{
WithDatabase(beaconDB),
WithStateGen(stategen.New(beaconDB)),
WithForkChoiceStore(fcs),
}
}
// warning: only use these opts when you are certain there are no db calls
// in your code path. this is a lightweight way to satisfy the stategen/beacondb
// initialization requirements w/o the overhead of db init.
func testServiceOptsNoDB() []Option {
return []Option{
withStateBalanceCache(satisfactoryStateBalanceCache()),
}
}
type mockStateByRooter struct {
state state.BeaconState
err error
}
var _ stateByRooter = &mockStateByRooter{}
func (m mockStateByRooter) StateByRoot(_ context.Context, _ [32]byte) (state.BeaconState, error) {
return m.state, m.err
}
// returns an instance of the state balance cache that can be used
// to satisfy the requirement for one in NewService, but which will
// always return an error if used.
func satisfactoryStateBalanceCache() *stateBalanceCache {
err := errors.New("satisfactoryStateBalanceCache doesn't perform real caching")
return &stateBalanceCache{stateGen: mockStateByRooter{err: err}}
}

View File

@@ -0,0 +1,72 @@
package blockchain
import (
"bytes"
"context"
"github.com/pkg/errors"
types "github.com/prysmaticlabs/eth2-types"
"github.com/prysmaticlabs/prysm/time/slots"
)
// NewSlot mimics the implementation of `on_tick` in fork choice consensus spec.
// It resets the proposer boost root in fork choice, and it updates store's justified checkpoint
// if a better checkpoint on the store's finalized checkpoint chain.
// This should only be called at the start of every slot interval.
//
// Spec pseudocode definition:
// # Reset store.proposer_boost_root if this is a new slot
// if current_slot > previous_slot:
// store.proposer_boost_root = Root()
//
// # Not a new epoch, return
// if not (current_slot > previous_slot and compute_slots_since_epoch_start(current_slot) == 0):
// return
//
// # Update store.justified_checkpoint if a better checkpoint on the store.finalized_checkpoint chain
// if store.best_justified_checkpoint.epoch > store.justified_checkpoint.epoch:
// finalized_slot = compute_start_slot_at_epoch(store.finalized_checkpoint.epoch)
// ancestor_at_finalized_slot = get_ancestor(store, store.best_justified_checkpoint.root, finalized_slot)
// if ancestor_at_finalized_slot == store.finalized_checkpoint.root:
// store.justified_checkpoint = store.best_justified_checkpoint
func (s *Service) NewSlot(ctx context.Context, slot types.Slot) error {
// Reset proposer boost root in fork choice.
if err := s.cfg.ForkChoiceStore.ResetBoostedProposerRoot(ctx); err != nil {
return errors.Wrap(err, "could not reset boosted proposer root in fork choice")
}
// Return if it's not a new epoch.
if !slots.IsEpochStart(slot) {
return nil
}
// Update store.justified_checkpoint if a better checkpoint on the store.finalized_checkpoint chain
bj := s.store.BestJustifiedCheckpt()
if bj == nil {
return errNilBestJustifiedInStore
}
j := s.store.JustifiedCheckpt()
if j == nil {
return errNilJustifiedInStore
}
f := s.store.FinalizedCheckpt()
if f == nil {
return errNilFinalizedInStore
}
if bj.Epoch > j.Epoch {
finalizedSlot, err := slots.EpochStart(f.Epoch)
if err != nil {
return err
}
r, err := s.ancestor(ctx, bj.Root, finalizedSlot)
if err != nil {
return err
}
if bytes.Equal(r, f.Root) {
s.store.SetJustifiedCheckpt(bj)
}
}
return nil
}

View File

@@ -0,0 +1,100 @@
package blockchain
import (
"context"
"testing"
types "github.com/prysmaticlabs/eth2-types"
"github.com/prysmaticlabs/prysm/beacon-chain/blockchain/store"
testDB "github.com/prysmaticlabs/prysm/beacon-chain/db/testing"
"github.com/prysmaticlabs/prysm/beacon-chain/forkchoice/protoarray"
"github.com/prysmaticlabs/prysm/beacon-chain/state/stategen"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/testing/require"
)
func TestService_newSlot(t *testing.T) {
beaconDB := testDB.SetupDB(t)
fcs := protoarray.New(0, 0, [32]byte{})
opts := []Option{
WithDatabase(beaconDB),
WithStateGen(stategen.New(beaconDB)),
WithForkChoiceStore(fcs),
}
ctx := context.Background()
require.NoError(t, fcs.ProcessBlock(ctx, 0, [32]byte{}, [32]byte{}, [32]byte{}, 0, 0)) // genesis
require.NoError(t, fcs.ProcessBlock(ctx, 32, [32]byte{'a'}, [32]byte{}, [32]byte{}, 0, 0)) // finalized
require.NoError(t, fcs.ProcessBlock(ctx, 64, [32]byte{'b'}, [32]byte{'a'}, [32]byte{}, 0, 0)) // justified
require.NoError(t, fcs.ProcessBlock(ctx, 96, [32]byte{'c'}, [32]byte{'a'}, [32]byte{}, 0, 0)) // best justified
require.NoError(t, fcs.ProcessBlock(ctx, 97, [32]byte{'d'}, [32]byte{}, [32]byte{}, 0, 0)) // bad
type args struct {
slot types.Slot
finalized *ethpb.Checkpoint
justified *ethpb.Checkpoint
bestJustified *ethpb.Checkpoint
shouldEqual bool
}
tests := []struct {
name string
args args
}{
{
name: "Not epoch boundary. No change",
args: args{
slot: params.BeaconConfig().SlotsPerEpoch + 1,
finalized: &ethpb.Checkpoint{Epoch: 1, Root: bytesutil.PadTo([]byte{'a'}, 32)},
justified: &ethpb.Checkpoint{Epoch: 2, Root: bytesutil.PadTo([]byte{'b'}, 32)},
bestJustified: &ethpb.Checkpoint{Epoch: 3, Root: bytesutil.PadTo([]byte{'c'}, 32)},
shouldEqual: false,
},
},
{
name: "Justified higher than best justified. No change",
args: args{
slot: params.BeaconConfig().SlotsPerEpoch,
finalized: &ethpb.Checkpoint{Epoch: 1, Root: bytesutil.PadTo([]byte{'a'}, 32)},
justified: &ethpb.Checkpoint{Epoch: 3, Root: bytesutil.PadTo([]byte{'b'}, 32)},
bestJustified: &ethpb.Checkpoint{Epoch: 2, Root: bytesutil.PadTo([]byte{'c'}, 32)},
shouldEqual: false,
},
},
{
name: "Best justified not on the same chain as finalized. No change",
args: args{
slot: params.BeaconConfig().SlotsPerEpoch,
finalized: &ethpb.Checkpoint{Epoch: 1, Root: bytesutil.PadTo([]byte{'a'}, 32)},
justified: &ethpb.Checkpoint{Epoch: 2, Root: bytesutil.PadTo([]byte{'b'}, 32)},
bestJustified: &ethpb.Checkpoint{Epoch: 3, Root: bytesutil.PadTo([]byte{'d'}, 32)},
shouldEqual: false,
},
},
{
name: "Best justified on the same chain as finalized. Yes change",
args: args{
slot: params.BeaconConfig().SlotsPerEpoch,
finalized: &ethpb.Checkpoint{Epoch: 1, Root: bytesutil.PadTo([]byte{'a'}, 32)},
justified: &ethpb.Checkpoint{Epoch: 2, Root: bytesutil.PadTo([]byte{'b'}, 32)},
bestJustified: &ethpb.Checkpoint{Epoch: 3, Root: bytesutil.PadTo([]byte{'c'}, 32)},
shouldEqual: true,
},
},
}
for _, test := range tests {
service, err := NewService(ctx, opts...)
require.NoError(t, err)
store := store.New(test.args.justified, test.args.finalized)
store.SetBestJustifiedCheckpt(test.args.bestJustified)
service.store = store
require.NoError(t, service.NewSlot(ctx, test.args.slot))
if test.args.shouldEqual {
require.DeepSSZEqual(t, service.store.BestJustifiedCheckpt(), service.store.JustifiedCheckpt())
} else {
require.DeepNotSSZEqual(t, service.store.BestJustifiedCheckpt(), service.store.JustifiedCheckpt())
}
}
}

View File

@@ -0,0 +1,53 @@
package blockchain
import (
"context"
"github.com/pkg/errors"
types "github.com/prysmaticlabs/eth2-types"
"github.com/prysmaticlabs/prysm/beacon-chain/core/blocks"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/block"
)
// optimisticCandidateBlock returns true if this block can be optimistically synced.
//
// Spec pseudocode definition:
// def is_optimistic_candidate_block(opt_store: OptimisticStore, current_slot: Slot, block: BeaconBlock) -> bool:
// justified_root = opt_store.block_states[opt_store.head_block_root].current_justified_checkpoint.root
// justified_is_execution_block = is_execution_block(opt_store.blocks[justified_root])
// block_is_deep = block.slot + SAFE_SLOTS_TO_IMPORT_OPTIMISTICALLY <= current_slot
// return justified_is_execution_block or block_is_deep
func (s *Service) optimisticCandidateBlock(ctx context.Context, blk block.BeaconBlock) (bool, error) {
if blk.Slot()+params.BeaconConfig().SafeSlotsToImportOptimistically <= s.CurrentSlot() {
return true, nil
}
j := s.store.JustifiedCheckpt()
if j == nil {
return false, errNilJustifiedInStore
}
jBlock, err := s.cfg.BeaconDB.Block(ctx, bytesutil.ToBytes32(j.Root))
if err != nil {
return false, err
}
return blocks.ExecutionBlock(jBlock.Block().Body())
}
// loadSyncedTips loads a previously saved synced Tips from DB
// if no synced tips are saved, then it creates one from the given
// root and slot number.
func (s *Service) loadSyncedTips(root [32]byte, slot types.Slot) error {
// Initialize synced tips
tips, err := s.cfg.BeaconDB.ValidatedTips(s.ctx)
if err != nil {
return errors.Wrap(err, "could not get synced tips")
}
if len(tips) == 0 {
tips[root] = slot
}
if err := s.cfg.ForkChoiceStore.SetSyncedTips(tips); err != nil {
return errors.Wrap(err, "could not set synced tips")
}
return nil
}

View File

@@ -0,0 +1,141 @@
package blockchain
import (
"context"
"testing"
"time"
testDB "github.com/prysmaticlabs/prysm/beacon-chain/db/testing"
"github.com/prysmaticlabs/prysm/beacon-chain/forkchoice/protoarray"
"github.com/prysmaticlabs/prysm/beacon-chain/state/stategen"
fieldparams "github.com/prysmaticlabs/prysm/config/fieldparams"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/block"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/wrapper"
"github.com/prysmaticlabs/prysm/testing/require"
"github.com/prysmaticlabs/prysm/testing/util"
"github.com/prysmaticlabs/prysm/time/slots"
)
func Test_IsOptimisticCandidateBlock(t *testing.T) {
params.SetupTestConfigCleanup(t)
params.OverrideBeaconConfig(params.MainnetConfig())
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
fcs := protoarray.New(0, 0, [32]byte{'a'})
opts := []Option{
WithDatabase(beaconDB),
WithStateGen(stategen.New(beaconDB)),
WithForkChoiceStore(fcs),
}
service, err := NewService(ctx, opts...)
require.NoError(t, err)
params.BeaconConfig().SafeSlotsToImportOptimistically = 128
service.genesisTime = time.Now().Add(-time.Second * 12 * 2 * 128)
tests := []struct {
name string
blk block.BeaconBlock
justified block.SignedBeaconBlock
want bool
}{
{
name: "deep block",
blk: func(tt *testing.T) block.BeaconBlock {
blk := util.NewBeaconBlockBellatrix()
blk.Block.Slot = 1
wr, err := wrapper.WrappedBellatrixBeaconBlock(blk.Block)
require.NoError(tt, err)
return wr
}(t),
justified: func(tt *testing.T) block.SignedBeaconBlock {
blk := util.NewBeaconBlockBellatrix()
blk.Block.Slot = 32
wr, err := wrapper.WrappedBellatrixSignedBeaconBlock(blk)
require.NoError(tt, err)
return wr
}(t),
want: true,
},
{
name: "shallow block, Altair justified chkpt",
blk: func(tt *testing.T) block.BeaconBlock {
blk := util.NewBeaconBlockAltair()
blk.Block.Slot = 200
wr, err := wrapper.WrappedAltairBeaconBlock(blk.Block)
require.NoError(tt, err)
return wr
}(t),
justified: func(tt *testing.T) block.SignedBeaconBlock {
blk := util.NewBeaconBlockAltair()
blk.Block.Slot = 32
wr, err := wrapper.WrappedAltairSignedBeaconBlock(blk)
require.NoError(tt, err)
return wr
}(t),
want: false,
},
{
name: "shallow block, Bellatrix justified chkpt without execution",
blk: func(tt *testing.T) block.BeaconBlock {
blk := util.NewBeaconBlockBellatrix()
blk.Block.Slot = 200
wr, err := wrapper.WrappedBellatrixBeaconBlock(blk.Block)
require.NoError(tt, err)
return wr
}(t),
justified: func(tt *testing.T) block.SignedBeaconBlock {
blk := util.NewBeaconBlockBellatrix()
blk.Block.Slot = 32
wr, err := wrapper.WrappedBellatrixSignedBeaconBlock(blk)
require.NoError(tt, err)
return wr
}(t),
want: false,
},
{
name: "shallow block, execution enabled justified chkpt",
blk: func(tt *testing.T) block.BeaconBlock {
blk := util.NewBeaconBlockBellatrix()
blk.Block.Slot = 200
wr, err := wrapper.WrappedBellatrixBeaconBlock(blk.Block)
require.NoError(tt, err)
return wr
}(t),
justified: func(tt *testing.T) block.SignedBeaconBlock {
blk := util.NewBeaconBlockBellatrix()
blk.Block.Slot = 32
blk.Block.Body.ExecutionPayload.ParentHash = bytesutil.PadTo([]byte{'a'}, fieldparams.RootLength)
blk.Block.Body.ExecutionPayload.FeeRecipient = bytesutil.PadTo([]byte{'a'}, fieldparams.FeeRecipientLength)
blk.Block.Body.ExecutionPayload.StateRoot = bytesutil.PadTo([]byte{'a'}, fieldparams.RootLength)
blk.Block.Body.ExecutionPayload.ReceiptsRoot = bytesutil.PadTo([]byte{'a'}, fieldparams.RootLength)
blk.Block.Body.ExecutionPayload.LogsBloom = bytesutil.PadTo([]byte{'a'}, fieldparams.LogsBloomLength)
blk.Block.Body.ExecutionPayload.Random = bytesutil.PadTo([]byte{'a'}, fieldparams.RootLength)
blk.Block.Body.ExecutionPayload.BaseFeePerGas = bytesutil.PadTo([]byte{'a'}, fieldparams.RootLength)
blk.Block.Body.ExecutionPayload.BlockHash = bytesutil.PadTo([]byte{'a'}, fieldparams.RootLength)
wr, err := wrapper.WrappedBellatrixSignedBeaconBlock(blk)
require.NoError(tt, err)
return wr
}(t),
want: true,
},
}
for _, tt := range tests {
jroot, err := tt.justified.Block().HashTreeRoot()
require.NoError(t, err)
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, tt.justified))
service.store.SetJustifiedCheckpt(
&ethpb.Checkpoint{
Root: jroot[:],
Epoch: slots.ToEpoch(tt.justified.Block().Slot()),
})
candidate, err := service.optimisticCandidateBlock(ctx, tt.blk)
require.NoError(t, err)
require.Equal(t, tt.want, candidate, tt.name)
}
}

View File

@@ -0,0 +1,146 @@
package blockchain
import (
"github.com/prysmaticlabs/prysm/async/event"
"github.com/prysmaticlabs/prysm/beacon-chain/cache/depositcache"
statefeed "github.com/prysmaticlabs/prysm/beacon-chain/core/feed/state"
"github.com/prysmaticlabs/prysm/beacon-chain/db"
"github.com/prysmaticlabs/prysm/beacon-chain/forkchoice"
"github.com/prysmaticlabs/prysm/beacon-chain/operations/attestations"
"github.com/prysmaticlabs/prysm/beacon-chain/operations/slashings"
"github.com/prysmaticlabs/prysm/beacon-chain/operations/voluntaryexits"
"github.com/prysmaticlabs/prysm/beacon-chain/p2p"
"github.com/prysmaticlabs/prysm/beacon-chain/powchain"
"github.com/prysmaticlabs/prysm/beacon-chain/state"
"github.com/prysmaticlabs/prysm/beacon-chain/state/stategen"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
)
type Option func(s *Service) error
// WithMaxGoroutines to control resource use of the blockchain service.
func WithMaxGoroutines(x int) Option {
return func(s *Service) error {
s.cfg.MaxRoutines = x
return nil
}
}
// WithWeakSubjectivityCheckpoint for checkpoint sync.
func WithWeakSubjectivityCheckpoint(c *ethpb.Checkpoint) Option {
return func(s *Service) error {
s.cfg.WeakSubjectivityCheckpt = c
return nil
}
}
// WithDatabase for head access.
func WithDatabase(beaconDB db.HeadAccessDatabase) Option {
return func(s *Service) error {
s.cfg.BeaconDB = beaconDB
return nil
}
}
// WithChainStartFetcher to retrieve information about genesis.
func WithChainStartFetcher(f powchain.ChainStartFetcher) Option {
return func(s *Service) error {
s.cfg.ChainStartFetcher = f
return nil
}
}
// WithDepositCache for deposit lifecycle after chain inclusion.
func WithDepositCache(c *depositcache.DepositCache) Option {
return func(s *Service) error {
s.cfg.DepositCache = c
return nil
}
}
// WithAttestationPool for attestation lifecycle after chain inclusion.
func WithAttestationPool(p attestations.Pool) Option {
return func(s *Service) error {
s.cfg.AttPool = p
return nil
}
}
// WithExitPool for exits lifecycle after chain inclusion.
func WithExitPool(p voluntaryexits.PoolManager) Option {
return func(s *Service) error {
s.cfg.ExitPool = p
return nil
}
}
// WithSlashingPool for slashings lifecycle after chain inclusion.
func WithSlashingPool(p slashings.PoolManager) Option {
return func(s *Service) error {
s.cfg.SlashingPool = p
return nil
}
}
// WithP2PBroadcaster to broadcast messages after appropriate processing.
func WithP2PBroadcaster(p p2p.Broadcaster) Option {
return func(s *Service) error {
s.cfg.P2p = p
return nil
}
}
// WithStateNotifier to notify an event feed of state processing.
func WithStateNotifier(n statefeed.Notifier) Option {
return func(s *Service) error {
s.cfg.StateNotifier = n
return nil
}
}
// WithForkChoiceStore to update an optimized fork-choice representation.
func WithForkChoiceStore(f forkchoice.ForkChoicer) Option {
return func(s *Service) error {
s.cfg.ForkChoiceStore = f
return nil
}
}
// WithAttestationService for dealing with attestation lifecycles.
func WithAttestationService(srv *attestations.Service) Option {
return func(s *Service) error {
s.cfg.AttService = srv
return nil
}
}
// WithStateGen for managing state regeneration and replay.
func WithStateGen(g *stategen.State) Option {
return func(s *Service) error {
s.cfg.StateGen = g
return nil
}
}
// WithSlasherAttestationsFeed to forward attestations into slasher if enabled.
func WithSlasherAttestationsFeed(f *event.Feed) Option {
return func(s *Service) error {
s.cfg.SlasherAttestationsFeed = f
return nil
}
}
func withStateBalanceCache(c *stateBalanceCache) Option {
return func(s *Service) error {
s.justifiedBalances = c
return nil
}
}
// WithFinalizedStateAtStartUp to store finalized state at start up.
func WithFinalizedStateAtStartUp(st state.BeaconState) Option {
return func(s *Service) error {
s.cfg.FinalizedStateAtStartUp = st
return nil
}
}

View File

@@ -0,0 +1,29 @@
package blockchain
import (
"math/big"
"github.com/holiman/uint256"
"github.com/pkg/errors"
"github.com/prysmaticlabs/prysm/config/params"
)
// validates terminal pow block by comparing own total difficulty with parent's total difficulty.
//
// def is_valid_terminal_pow_block(block: PowBlock, parent: PowBlock) -> bool:
// is_total_difficulty_reached = block.total_difficulty >= TERMINAL_TOTAL_DIFFICULTY
// is_parent_total_difficulty_valid = parent.total_difficulty < TERMINAL_TOTAL_DIFFICULTY
// return is_total_difficulty_reached and is_parent_total_difficulty_valid
func validTerminalPowBlock(currentDifficulty *uint256.Int, parentDifficulty *uint256.Int) (bool, error) {
b, ok := new(big.Int).SetString(params.BeaconConfig().TerminalTotalDifficulty, 10)
if !ok {
return false, errors.New("failed to parse terminal total difficulty")
}
ttd, of := uint256.FromBig(b)
if of {
return false, errors.New("overflow terminal total difficulty")
}
totalDifficultyReached := currentDifficulty.Cmp(ttd) >= 0
parentTotalDifficultyValid := ttd.Cmp(parentDifficulty) > 0
return totalDifficultyReached && parentTotalDifficultyValid, nil
}

View File

@@ -0,0 +1,93 @@
package blockchain
import (
"fmt"
"math/big"
"testing"
"github.com/holiman/uint256"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/testing/require"
)
func Test_validTerminalPowBlock(t *testing.T) {
tests := []struct {
name string
currentDifficulty *uint256.Int
parentDifficulty *uint256.Int
ttd uint64
want bool
}{
{
name: "current > ttd, parent > ttd",
currentDifficulty: uint256.NewInt(2),
parentDifficulty: uint256.NewInt(2),
ttd: 1,
want: false,
},
{
name: "current < ttd, parent < ttd",
currentDifficulty: uint256.NewInt(2),
parentDifficulty: uint256.NewInt(2),
ttd: 3,
want: false,
},
{
name: "current == ttd, parent == ttd",
currentDifficulty: uint256.NewInt(2),
parentDifficulty: uint256.NewInt(2),
ttd: 2,
want: false,
},
{
name: "current > ttd, parent == ttd",
currentDifficulty: uint256.NewInt(2),
parentDifficulty: uint256.NewInt(1),
ttd: 1,
want: false,
},
{
name: "current == ttd, parent < ttd",
currentDifficulty: uint256.NewInt(2),
parentDifficulty: uint256.NewInt(1),
ttd: 2,
want: true,
},
{
name: "current > ttd, parent < ttd",
currentDifficulty: uint256.NewInt(3),
parentDifficulty: uint256.NewInt(1),
ttd: 2,
want: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
cfg := params.BeaconConfig()
cfg.TerminalTotalDifficulty = fmt.Sprint(tt.ttd)
params.OverrideBeaconConfig(cfg)
got, err := validTerminalPowBlock(tt.currentDifficulty, tt.parentDifficulty)
require.NoError(t, err)
if got != tt.want {
t.Errorf("validTerminalPowBlock() = %v, want %v", got, tt.want)
}
})
}
}
func Test_validTerminalPowBlockSpecConfig(t *testing.T) {
cfg := params.BeaconConfig()
cfg.TerminalTotalDifficulty = "115792089237316195423570985008687907853269984665640564039457584007913129638912"
params.OverrideBeaconConfig(cfg)
i, _ := new(big.Int).SetString("115792089237316195423570985008687907853269984665640564039457584007913129638912", 10)
current, of := uint256.FromBig(i)
require.Equal(t, of, false)
i, _ = new(big.Int).SetString("115792089237316195423570985008687907853269984665640564039457584007913129638911", 10)
parent, of := uint256.FromBig(i)
require.Equal(t, of, false)
got, err := validTerminalPowBlock(current, parent)
require.NoError(t, err)
require.Equal(t, true, got)
}

View File

@@ -2,19 +2,19 @@ package blockchain
import (
"context"
"time"
"github.com/pkg/errors"
ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
"github.com/prysmaticlabs/prysm/beacon-chain/state/stateV0"
"github.com/prysmaticlabs/prysm/shared/attestationutil"
"github.com/prysmaticlabs/prysm/shared/bytesutil"
"github.com/prysmaticlabs/prysm/shared/params"
"github.com/prysmaticlabs/prysm/shared/timeutils"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/attestation"
"github.com/prysmaticlabs/prysm/time/slots"
"go.opencensus.io/trace"
)
// onAttestation is called whenever an attestation is received, verifies the attestation is valid and saves
// OnAttestation is called whenever an attestation is received, verifies the attestation is valid and saves
// it to the DB. As a stateless function, this does not hold nor delay attestation based on the spec descriptions.
// The delay is handled by the caller in `processAttestations`.
//
@@ -36,7 +36,7 @@ import (
//
// # Update latest messages for attesting indices
// update_latest_messages(store, indexed_attestation.attesting_indices, attestation)
func (s *Service) onAttestation(ctx context.Context, a *ethpb.Attestation) error {
func (s *Service) OnAttestation(ctx context.Context, a *ethpb.Attestation) error {
ctx, span := trace.StartSpan(ctx, "blockChain.onAttestation")
defer span.End()
@@ -46,7 +46,7 @@ func (s *Service) onAttestation(ctx context.Context, a *ethpb.Attestation) error
if err := helpers.ValidateSlotTargetEpoch(a.Data); err != nil {
return err
}
tgt := stateV0.CopyCheckpoint(a.Data.Target)
tgt := ethpb.CopyCheckpoint(a.Data.Target)
// Note that target root check is ignored here because it was performed in sync's validation pipeline:
// validate_aggregate_proof.go and validate_beacon_attestation.go
@@ -59,10 +59,10 @@ func (s *Service) onAttestation(ctx context.Context, a *ethpb.Attestation) error
return err
}
genesisTime := baseState.GenesisTime()
genesisTime := uint64(s.genesisTime.Unix())
// Verify attestation target is from current epoch or previous epoch.
if err := s.verifyAttTargetEpoch(ctx, genesisTime, uint64(timeutils.Now().Unix()), tgt); err != nil {
if err := verifyAttTargetEpoch(ctx, genesisTime, uint64(time.Now().Unix()), tgt); err != nil {
return err
}
@@ -75,20 +75,20 @@ func (s *Service) onAttestation(ctx context.Context, a *ethpb.Attestation) error
// validate_aggregate_proof.go and validate_beacon_attestation.go
// Verify attestations can only affect the fork choice of subsequent slots.
if err := helpers.VerifySlotTime(genesisTime, a.Data.Slot+1, params.BeaconNetworkConfig().MaximumGossipClockDisparity); err != nil {
if err := slots.VerifyTime(genesisTime, a.Data.Slot+1, params.BeaconNetworkConfig().MaximumGossipClockDisparity); err != nil {
return err
}
// Use the target state to verify attesting indices are valid.
committee, err := helpers.BeaconCommitteeFromState(baseState, a.Data.Slot, a.Data.CommitteeIndex)
committee, err := helpers.BeaconCommitteeFromState(ctx, baseState, a.Data.Slot, a.Data.CommitteeIndex)
if err != nil {
return err
}
indexedAtt, err := attestationutil.ConvertToIndexed(ctx, a, committee)
indexedAtt, err := attestation.ConvertToIndexed(ctx, a, committee)
if err != nil {
return err
}
if err := attestationutil.IsValidAttestationIndices(ctx, indexedAtt); err != nil {
if err := attestation.IsValidAttestationIndices(ctx, indexedAtt); err != nil {
return err
}

View File

@@ -7,29 +7,29 @@ import (
"github.com/pkg/errors"
types "github.com/prysmaticlabs/eth2-types"
ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
"github.com/prysmaticlabs/prysm/async"
"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
"github.com/prysmaticlabs/prysm/beacon-chain/core/state"
iface "github.com/prysmaticlabs/prysm/beacon-chain/state/interface"
"github.com/prysmaticlabs/prysm/shared/bytesutil"
"github.com/prysmaticlabs/prysm/shared/featureconfig"
"github.com/prysmaticlabs/prysm/shared/mputil"
"github.com/prysmaticlabs/prysm/shared/params"
"github.com/prysmaticlabs/prysm/beacon-chain/core/transition"
"github.com/prysmaticlabs/prysm/beacon-chain/state"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/time/slots"
)
// getAttPreState retrieves the att pre state by either from the cache or the DB.
func (s *Service) getAttPreState(ctx context.Context, c *ethpb.Checkpoint) (iface.BeaconState, error) {
func (s *Service) getAttPreState(ctx context.Context, c *ethpb.Checkpoint) (state.BeaconState, error) {
// Use a multilock to allow scoped holding of a mutex by a checkpoint root + epoch
// allowing us to behave smarter in terms of how this function is used concurrently.
epochKey := strconv.FormatUint(uint64(c.Epoch), 10 /* base 10 */)
lock := mputil.NewMultilock(string(c.Root) + epochKey)
lock := async.NewMultilock(string(c.Root) + epochKey)
lock.Lock()
defer lock.Unlock()
cachedState, err := s.checkpointStateCache.StateByCheckpoint(c)
if err != nil {
return nil, errors.Wrap(err, "could not get cached checkpoint state")
}
if cachedState != nil {
if cachedState != nil && !cachedState.IsNil() {
return cachedState, nil
}
@@ -38,47 +38,30 @@ func (s *Service) getAttPreState(ctx context.Context, c *ethpb.Checkpoint) (ifac
return nil, errors.Wrapf(err, "could not get pre state for epoch %d", c.Epoch)
}
epochStartSlot, err := helpers.StartSlot(c.Epoch)
epochStartSlot, err := slots.EpochStart(c.Epoch)
if err != nil {
return nil, err
}
if epochStartSlot > baseState.Slot() {
if featureconfig.Get().EnableNextSlotStateCache {
baseState, err = state.ProcessSlotsUsingNextSlotCache(ctx, baseState, c.Root, epochStartSlot)
if err != nil {
return nil, errors.Wrapf(err, "could not process slots up to epoch %d", c.Epoch)
}
} else {
baseState, err = state.ProcessSlots(ctx, baseState, epochStartSlot)
if err != nil {
return nil, errors.Wrapf(err, "could not process slots up to epoch %d", c.Epoch)
}
}
if err := s.checkpointStateCache.AddCheckpointState(c, baseState); err != nil {
return nil, errors.Wrap(err, "could not saved checkpoint state to cache")
}
return baseState, nil
baseState, err = transition.ProcessSlotsIfPossible(ctx, baseState, epochStartSlot)
if err != nil {
return nil, errors.Wrapf(err, "could not process slots up to epoch %d", c.Epoch)
}
// To avoid sharing the same state across checkpoint state cache and hot state cache,
// we don't add the state to check point cache.
has, err := s.cfg.StateGen.HasStateInCache(ctx, bytesutil.ToBytes32(c.Root))
if err != nil {
return nil, err
}
if !has {
if err := s.checkpointStateCache.AddCheckpointState(c, baseState); err != nil {
return nil, errors.Wrap(err, "could not saved checkpoint state to cache")
}
// Sharing the same state across caches is perfectly fine here, the fetching
// of attestation prestate is by far the most accessed state fetching pattern in
// the beacon node. An extra state instance cached isn't an issue in the bigger
// picture.
if err := s.checkpointStateCache.AddCheckpointState(c, baseState); err != nil {
return nil, errors.Wrap(err, "could not save checkpoint state to cache")
}
return baseState, nil
}
// verifyAttTargetEpoch validates attestation is from the current or previous epoch.
func (s *Service) verifyAttTargetEpoch(_ context.Context, genesisTime, nowTime uint64, c *ethpb.Checkpoint) error {
func verifyAttTargetEpoch(_ context.Context, genesisTime, nowTime uint64, c *ethpb.Checkpoint) error {
currentSlot := types.Slot((nowTime - genesisTime) / params.BeaconConfig().SecondsPerSlot)
currentEpoch := helpers.SlotToEpoch(currentSlot)
currentEpoch := slots.ToEpoch(currentSlot)
var prevEpoch types.Epoch
// Prevents previous epoch under flow
if currentEpoch > 1 {
@@ -99,14 +82,14 @@ func (s *Service) verifyBeaconBlock(ctx context.Context, data *ethpb.Attestation
}
// If the block does not exist in db, check again if block exists in initial sync block cache.
// This could happen as the node first syncs to head.
if b == nil && s.hasInitSyncBlock(r) {
if (b == nil || b.IsNil()) && s.hasInitSyncBlock(r) {
b = s.getInitSyncBlock(r)
}
if err := helpers.VerifyNilBeaconBlock(b); err != nil {
if err := helpers.BeaconBlockIsNil(b); err != nil {
return err
}
if b.Block.Slot > data.Slot {
return fmt.Errorf("could not process attestation for future block, block.Slot=%d > attestation.Data.Slot=%d", b.Block.Slot, data.Slot)
if b.Block().Slot() > data.Slot {
return fmt.Errorf("could not process attestation for future block, block.Slot=%d > attestation.Data.Slot=%d", b.Block().Slot(), data.Slot)
}
return nil
}

View File

@@ -3,64 +3,65 @@ package blockchain
import (
"context"
"testing"
"time"
types "github.com/prysmaticlabs/eth2-types"
ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
"github.com/prysmaticlabs/prysm/beacon-chain/core/state"
"github.com/prysmaticlabs/prysm/beacon-chain/core/transition"
testDB "github.com/prysmaticlabs/prysm/beacon-chain/db/testing"
"github.com/prysmaticlabs/prysm/beacon-chain/forkchoice/protoarray"
"github.com/prysmaticlabs/prysm/beacon-chain/state/stategen"
pb "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
"github.com/prysmaticlabs/prysm/shared/bytesutil"
"github.com/prysmaticlabs/prysm/shared/params"
"github.com/prysmaticlabs/prysm/shared/testutil"
"github.com/prysmaticlabs/prysm/shared/testutil/assert"
"github.com/prysmaticlabs/prysm/shared/testutil/require"
"github.com/prysmaticlabs/prysm/shared/timeutils"
fieldparams "github.com/prysmaticlabs/prysm/config/fieldparams"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/wrapper"
"github.com/prysmaticlabs/prysm/testing/assert"
"github.com/prysmaticlabs/prysm/testing/require"
"github.com/prysmaticlabs/prysm/testing/util"
"github.com/prysmaticlabs/prysm/time/slots"
)
func TestStore_OnAttestation_ErrorConditions(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
cfg := &Config{
BeaconDB: beaconDB,
ForkChoiceStore: protoarray.New(0, 0, [32]byte{}),
StateGen: stategen.New(beaconDB),
opts := []Option{
WithDatabase(beaconDB),
WithForkChoiceStore(protoarray.New(0, 0, [32]byte{})),
WithStateGen(stategen.New(beaconDB)),
}
service, err := NewService(ctx, cfg)
service, err := NewService(ctx, opts...)
require.NoError(t, err)
_, err = blockTree1(t, beaconDB, []byte{'g'})
require.NoError(t, err)
BlkWithOutState := testutil.NewBeaconBlock()
BlkWithOutState := util.NewBeaconBlock()
BlkWithOutState.Block.Slot = 0
require.NoError(t, beaconDB.SaveBlock(ctx, BlkWithOutState))
require.NoError(t, beaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(BlkWithOutState)))
BlkWithOutStateRoot, err := BlkWithOutState.Block.HashTreeRoot()
require.NoError(t, err)
BlkWithStateBadAtt := testutil.NewBeaconBlock()
BlkWithStateBadAtt := util.NewBeaconBlock()
BlkWithStateBadAtt.Block.Slot = 1
require.NoError(t, beaconDB.SaveBlock(ctx, BlkWithStateBadAtt))
require.NoError(t, beaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(BlkWithStateBadAtt)))
BlkWithStateBadAttRoot, err := BlkWithStateBadAtt.Block.HashTreeRoot()
require.NoError(t, err)
s, err := testutil.NewBeaconState()
s, err := util.NewBeaconState()
require.NoError(t, err)
require.NoError(t, s.SetSlot(100*params.BeaconConfig().SlotsPerEpoch))
require.NoError(t, service.cfg.BeaconDB.SaveState(ctx, s, BlkWithStateBadAttRoot))
BlkWithValidState := testutil.NewBeaconBlock()
BlkWithValidState := util.NewBeaconBlock()
BlkWithValidState.Block.Slot = 2
require.NoError(t, beaconDB.SaveBlock(ctx, BlkWithValidState))
require.NoError(t, beaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(BlkWithValidState)))
BlkWithValidStateRoot, err := BlkWithValidState.Block.HashTreeRoot()
require.NoError(t, err)
s, err = testutil.NewBeaconState()
s, err = util.NewBeaconState()
require.NoError(t, err)
err = s.SetFork(&pb.Fork{
err = s.SetFork(&ethpb.Fork{
Epoch: 0,
CurrentVersion: params.BeaconConfig().GenesisForkVersion,
PreviousVersion: params.BeaconConfig().GenesisForkVersion,
@@ -75,17 +76,17 @@ func TestStore_OnAttestation_ErrorConditions(t *testing.T) {
}{
{
name: "attestation's data slot not aligned with target vote",
a: testutil.HydrateAttestation(&ethpb.Attestation{Data: &ethpb.AttestationData{Slot: params.BeaconConfig().SlotsPerEpoch, Target: &ethpb.Checkpoint{Root: make([]byte, 32)}}}),
a: util.HydrateAttestation(&ethpb.Attestation{Data: &ethpb.AttestationData{Slot: params.BeaconConfig().SlotsPerEpoch, Target: &ethpb.Checkpoint{Root: make([]byte, 32)}}}),
wantedErr: "slot 32 does not match target epoch 0",
},
{
name: "no pre state for attestations's target block",
a: testutil.HydrateAttestation(&ethpb.Attestation{Data: &ethpb.AttestationData{Target: &ethpb.Checkpoint{Root: BlkWithOutStateRoot[:]}}}),
a: util.HydrateAttestation(&ethpb.Attestation{Data: &ethpb.AttestationData{Target: &ethpb.Checkpoint{Root: BlkWithOutStateRoot[:]}}}),
wantedErr: "could not get pre state for epoch 0",
},
{
name: "process attestation doesn't match current epoch",
a: testutil.HydrateAttestation(&ethpb.Attestation{Data: &ethpb.AttestationData{Slot: 100 * params.BeaconConfig().SlotsPerEpoch, Target: &ethpb.Checkpoint{Epoch: 100,
a: util.HydrateAttestation(&ethpb.Attestation{Data: &ethpb.AttestationData{Slot: 100 * params.BeaconConfig().SlotsPerEpoch, Target: &ethpb.Checkpoint{Epoch: 100,
Root: BlkWithStateBadAttRoot[:]}}}),
wantedErr: "target epoch 100 does not match current epoch",
},
@@ -103,9 +104,9 @@ func TestStore_OnAttestation_ErrorConditions(t *testing.T) {
name: "process nil field (a.Target) in attestation",
a: &ethpb.Attestation{
Data: &ethpb.AttestationData{
BeaconBlockRoot: make([]byte, 32),
BeaconBlockRoot: make([]byte, fieldparams.RootLength),
Target: nil,
Source: &ethpb.Checkpoint{Root: make([]byte, 32)},
Source: &ethpb.Checkpoint{Root: make([]byte, fieldparams.RootLength)},
},
AggregationBits: make([]byte, 1),
Signature: make([]byte, 96),
@@ -116,7 +117,7 @@ func TestStore_OnAttestation_ErrorConditions(t *testing.T) {
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := service.onAttestation(ctx, tt.a)
err := service.OnAttestation(ctx, tt.a)
if tt.wantedErr != "" {
assert.ErrorContains(t, tt.wantedErr, err)
} else {
@@ -130,45 +131,46 @@ func TestStore_OnAttestation_Ok(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
cfg := &Config{
BeaconDB: beaconDB,
ForkChoiceStore: protoarray.New(0, 0, [32]byte{}),
StateGen: stategen.New(beaconDB),
fcs := protoarray.New(0, 0, [32]byte{'a'})
opts := []Option{
WithDatabase(beaconDB),
WithStateGen(stategen.New(beaconDB)),
WithForkChoiceStore(fcs),
}
service, err := NewService(ctx, cfg)
service, err := NewService(ctx, opts...)
require.NoError(t, err)
genesisState, pks := testutil.DeterministicGenesisState(t, 64)
require.NoError(t, genesisState.SetGenesisTime(uint64(timeutils.Now().Unix())-params.BeaconConfig().SecondsPerSlot))
genesisState, pks := util.DeterministicGenesisState(t, 64)
service.SetGenesisTime(time.Unix(time.Now().Unix()-int64(params.BeaconConfig().SecondsPerSlot), 0))
require.NoError(t, service.saveGenesisData(ctx, genesisState))
att, err := testutil.GenerateAttestations(genesisState, pks, 1, 0, false)
att, err := util.GenerateAttestations(genesisState, pks, 1, 0, false)
require.NoError(t, err)
tRoot := bytesutil.ToBytes32(att[0].Data.Target.Root)
copied := genesisState.Copy()
copied, err = state.ProcessSlots(ctx, copied, 1)
copied, err = transition.ProcessSlots(ctx, copied, 1)
require.NoError(t, err)
require.NoError(t, service.cfg.BeaconDB.SaveState(ctx, copied, tRoot))
require.NoError(t, service.cfg.ForkChoiceStore.ProcessBlock(ctx, 0, tRoot, tRoot, tRoot, 1, 1))
require.NoError(t, service.onAttestation(ctx, att[0]))
require.NoError(t, service.OnAttestation(ctx, att[0]))
}
func TestStore_SaveCheckpointState(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
cfg := &Config{
BeaconDB: beaconDB,
StateGen: stategen.New(beaconDB),
opts := []Option{
WithDatabase(beaconDB),
WithStateGen(stategen.New(beaconDB)),
}
service, err := NewService(ctx, cfg)
service, err := NewService(ctx, opts...)
require.NoError(t, err)
s, err := testutil.NewBeaconState()
s, err := util.NewBeaconState()
require.NoError(t, err)
err = s.SetFinalizedCheckpoint(&ethpb.Checkpoint{Root: bytesutil.PadTo([]byte{'A'}, 32)})
err = s.SetFinalizedCheckpoint(&ethpb.Checkpoint{Root: bytesutil.PadTo([]byte{'A'}, fieldparams.RootLength)})
require.NoError(t, err)
val := &ethpb.Validator{
PublicKey: bytesutil.PadTo([]byte("foo"), 48),
WithdrawalCredentials: bytesutil.PadTo([]byte("bar"), 32),
WithdrawalCredentials: bytesutil.PadTo([]byte("bar"), fieldparams.RootLength),
}
err = s.SetValidators([]*ethpb.Validator{val})
require.NoError(t, err)
@@ -177,23 +179,23 @@ func TestStore_SaveCheckpointState(t *testing.T) {
r := [32]byte{'g'}
require.NoError(t, service.cfg.BeaconDB.SaveState(ctx, s, r))
service.justifiedCheckpt = &ethpb.Checkpoint{Root: r[:]}
service.bestJustifiedCheckpt = &ethpb.Checkpoint{Root: r[:]}
service.finalizedCheckpt = &ethpb.Checkpoint{Root: r[:]}
service.prevFinalizedCheckpt = &ethpb.Checkpoint{Root: r[:]}
service.store.SetJustifiedCheckpt(&ethpb.Checkpoint{Root: r[:]})
service.store.SetBestJustifiedCheckpt(&ethpb.Checkpoint{Root: r[:]})
service.store.SetFinalizedCheckpt(&ethpb.Checkpoint{Root: r[:]})
service.store.SetPrevFinalizedCheckpt(&ethpb.Checkpoint{Root: r[:]})
r = bytesutil.ToBytes32([]byte{'A'})
cp1 := &ethpb.Checkpoint{Epoch: 1, Root: bytesutil.PadTo([]byte{'A'}, 32)}
cp1 := &ethpb.Checkpoint{Epoch: 1, Root: bytesutil.PadTo([]byte{'A'}, fieldparams.RootLength)}
require.NoError(t, service.cfg.BeaconDB.SaveState(ctx, s, bytesutil.ToBytes32([]byte{'A'})))
require.NoError(t, service.cfg.BeaconDB.SaveStateSummary(ctx, &pb.StateSummary{Root: bytesutil.PadTo([]byte{'A'}, 32)}))
require.NoError(t, service.cfg.BeaconDB.SaveStateSummary(ctx, &ethpb.StateSummary{Root: bytesutil.PadTo([]byte{'A'}, fieldparams.RootLength)}))
s1, err := service.getAttPreState(ctx, cp1)
require.NoError(t, err)
assert.Equal(t, 1*params.BeaconConfig().SlotsPerEpoch, s1.Slot(), "Unexpected state slot")
cp2 := &ethpb.Checkpoint{Epoch: 2, Root: bytesutil.PadTo([]byte{'B'}, 32)}
cp2 := &ethpb.Checkpoint{Epoch: 2, Root: bytesutil.PadTo([]byte{'B'}, fieldparams.RootLength)}
require.NoError(t, service.cfg.BeaconDB.SaveState(ctx, s, bytesutil.ToBytes32([]byte{'B'})))
require.NoError(t, service.cfg.BeaconDB.SaveStateSummary(ctx, &pb.StateSummary{Root: bytesutil.PadTo([]byte{'B'}, 32)}))
require.NoError(t, service.cfg.BeaconDB.SaveStateSummary(ctx, &ethpb.StateSummary{Root: bytesutil.PadTo([]byte{'B'}, fieldparams.RootLength)}))
s2, err := service.getAttPreState(ctx, cp2)
require.NoError(t, err)
assert.Equal(t, 2*params.BeaconConfig().SlotsPerEpoch, s2.Slot(), "Unexpected state slot")
@@ -211,13 +213,13 @@ func TestStore_SaveCheckpointState(t *testing.T) {
assert.Equal(t, 2*params.BeaconConfig().SlotsPerEpoch, s2.Slot(), "Unexpected state slot")
require.NoError(t, s.SetSlot(params.BeaconConfig().SlotsPerEpoch+1))
service.justifiedCheckpt = &ethpb.Checkpoint{Root: r[:]}
service.bestJustifiedCheckpt = &ethpb.Checkpoint{Root: r[:]}
service.finalizedCheckpt = &ethpb.Checkpoint{Root: r[:]}
service.prevFinalizedCheckpt = &ethpb.Checkpoint{Root: r[:]}
cp3 := &ethpb.Checkpoint{Epoch: 1, Root: bytesutil.PadTo([]byte{'C'}, 32)}
service.store.SetJustifiedCheckpt(&ethpb.Checkpoint{Root: r[:]})
service.store.SetBestJustifiedCheckpt(&ethpb.Checkpoint{Root: r[:]})
service.store.SetFinalizedCheckpt(&ethpb.Checkpoint{Root: r[:]})
service.store.SetPrevFinalizedCheckpt(&ethpb.Checkpoint{Root: r[:]})
cp3 := &ethpb.Checkpoint{Epoch: 1, Root: bytesutil.PadTo([]byte{'C'}, fieldparams.RootLength)}
require.NoError(t, service.cfg.BeaconDB.SaveState(ctx, s, bytesutil.ToBytes32([]byte{'C'})))
require.NoError(t, service.cfg.BeaconDB.SaveStateSummary(ctx, &pb.StateSummary{Root: bytesutil.PadTo([]byte{'C'}, 32)}))
require.NoError(t, service.cfg.BeaconDB.SaveStateSummary(ctx, &ethpb.StateSummary{Root: bytesutil.PadTo([]byte{'C'}, fieldparams.RootLength)}))
s3, err := service.getAttPreState(ctx, cp3)
require.NoError(t, err)
assert.Equal(t, s.Slot(), s3.Slot(), "Unexpected state slot")
@@ -227,16 +229,16 @@ func TestStore_UpdateCheckpointState(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
cfg := &Config{
BeaconDB: beaconDB,
StateGen: stategen.New(beaconDB),
opts := []Option{
WithDatabase(beaconDB),
WithStateGen(stategen.New(beaconDB)),
}
service, err := NewService(ctx, cfg)
service, err := NewService(ctx, opts...)
require.NoError(t, err)
epoch := types.Epoch(1)
baseState, _ := testutil.DeterministicGenesisState(t, 1)
checkpoint := &ethpb.Checkpoint{Epoch: epoch, Root: bytesutil.PadTo([]byte("hi"), 32)}
baseState, _ := util.DeterministicGenesisState(t, 1)
checkpoint := &ethpb.Checkpoint{Epoch: epoch, Root: bytesutil.PadTo([]byte("hi"), fieldparams.RootLength)}
require.NoError(t, service.cfg.BeaconDB.SaveState(ctx, baseState, bytesutil.ToBytes32(checkpoint.Root)))
returned, err := service.getAttPreState(ctx, checkpoint)
require.NoError(t, err)
@@ -247,13 +249,13 @@ func TestStore_UpdateCheckpointState(t *testing.T) {
assert.Equal(t, returned.Slot(), cached.Slot(), "State should have been cached")
epoch = 2
newCheckpoint := &ethpb.Checkpoint{Epoch: epoch, Root: bytesutil.PadTo([]byte("bye"), 32)}
newCheckpoint := &ethpb.Checkpoint{Epoch: epoch, Root: bytesutil.PadTo([]byte("bye"), fieldparams.RootLength)}
require.NoError(t, service.cfg.BeaconDB.SaveState(ctx, baseState, bytesutil.ToBytes32(newCheckpoint.Root)))
returned, err = service.getAttPreState(ctx, newCheckpoint)
require.NoError(t, err)
s, err := helpers.StartSlot(newCheckpoint.Epoch)
s, err := slots.EpochStart(newCheckpoint.Epoch)
require.NoError(t, err)
baseState, err = state.ProcessSlots(ctx, baseState, s)
baseState, err = transition.ProcessSlots(ctx, baseState, s)
require.NoError(t, err)
assert.Equal(t, returned.Slot(), baseState.Slot(), "Incorrectly returned base state")
@@ -264,64 +266,46 @@ func TestStore_UpdateCheckpointState(t *testing.T) {
func TestAttEpoch_MatchPrevEpoch(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
cfg := &Config{BeaconDB: beaconDB}
service, err := NewService(ctx, cfg)
require.NoError(t, err)
nowTime := uint64(params.BeaconConfig().SlotsPerEpoch) * params.BeaconConfig().SecondsPerSlot
require.NoError(t, service.verifyAttTargetEpoch(ctx, 0, nowTime, &ethpb.Checkpoint{Root: make([]byte, 32)}))
require.NoError(t, verifyAttTargetEpoch(ctx, 0, nowTime, &ethpb.Checkpoint{Root: make([]byte, fieldparams.RootLength)}))
}
func TestAttEpoch_MatchCurrentEpoch(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
cfg := &Config{BeaconDB: beaconDB}
service, err := NewService(ctx, cfg)
require.NoError(t, err)
nowTime := uint64(params.BeaconConfig().SlotsPerEpoch) * params.BeaconConfig().SecondsPerSlot
require.NoError(t, service.verifyAttTargetEpoch(ctx, 0, nowTime, &ethpb.Checkpoint{Epoch: 1}))
require.NoError(t, verifyAttTargetEpoch(ctx, 0, nowTime, &ethpb.Checkpoint{Epoch: 1}))
}
func TestAttEpoch_NotMatch(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
cfg := &Config{BeaconDB: beaconDB}
service, err := NewService(ctx, cfg)
require.NoError(t, err)
nowTime := 2 * uint64(params.BeaconConfig().SlotsPerEpoch) * params.BeaconConfig().SecondsPerSlot
err = service.verifyAttTargetEpoch(ctx, 0, nowTime, &ethpb.Checkpoint{Root: make([]byte, 32)})
err := verifyAttTargetEpoch(ctx, 0, nowTime, &ethpb.Checkpoint{Root: make([]byte, fieldparams.RootLength)})
assert.ErrorContains(t, "target epoch 0 does not match current epoch 2 or prev epoch 1", err)
}
func TestVerifyBeaconBlock_NoBlock(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
cfg := &Config{BeaconDB: beaconDB}
service, err := NewService(ctx, cfg)
opts := testServiceOptsWithDB(t)
service, err := NewService(ctx, opts...)
require.NoError(t, err)
d := testutil.HydrateAttestationData(&ethpb.AttestationData{})
d := util.HydrateAttestationData(&ethpb.AttestationData{})
assert.ErrorContains(t, "signed beacon block can't be nil", service.verifyBeaconBlock(ctx, d))
}
func TestVerifyBeaconBlock_futureBlock(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
cfg := &Config{BeaconDB: beaconDB}
service, err := NewService(ctx, cfg)
opts := testServiceOptsWithDB(t)
service, err := NewService(ctx, opts...)
require.NoError(t, err)
b := testutil.NewBeaconBlock()
b := util.NewBeaconBlock()
b.Block.Slot = 2
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, b))
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(b)))
r, err := b.Block.HashTreeRoot()
require.NoError(t, err)
d := &ethpb.AttestationData{Slot: 1, BeaconBlockRoot: r[:]}
@@ -331,15 +315,14 @@ func TestVerifyBeaconBlock_futureBlock(t *testing.T) {
func TestVerifyBeaconBlock_OK(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
cfg := &Config{BeaconDB: beaconDB}
service, err := NewService(ctx, cfg)
opts := testServiceOptsWithDB(t)
service, err := NewService(ctx, opts...)
require.NoError(t, err)
b := testutil.NewBeaconBlock()
b := util.NewBeaconBlock()
b.Block.Slot = 2
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, b))
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(b)))
r, err := b.Block.HashTreeRoot()
require.NoError(t, err)
d := &ethpb.AttestationData{Slot: 2, BeaconBlockRoot: r[:]}
@@ -351,22 +334,26 @@ func TestVerifyFinalizedConsistency_InconsistentRoot(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
cfg := &Config{BeaconDB: beaconDB, ForkChoiceStore: protoarray.New(0, 0, [32]byte{})}
service, err := NewService(ctx, cfg)
fcs := protoarray.New(0, 0, [32]byte{'a'})
opts := []Option{
WithDatabase(beaconDB),
WithStateGen(stategen.New(beaconDB)),
WithForkChoiceStore(fcs),
}
service, err := NewService(ctx, opts...)
require.NoError(t, err)
b32 := testutil.NewBeaconBlock()
b32 := util.NewBeaconBlock()
b32.Block.Slot = 32
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, b32))
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(b32)))
r32, err := b32.Block.HashTreeRoot()
require.NoError(t, err)
service.finalizedCheckpt = &ethpb.Checkpoint{Epoch: 1}
b33 := testutil.NewBeaconBlock()
service.store.SetFinalizedCheckpt(&ethpb.Checkpoint{Epoch: 1})
b33 := util.NewBeaconBlock()
b33.Block.Slot = 33
b33.Block.ParentRoot = r32[:]
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, b33))
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(b33)))
r33, err := b33.Block.HashTreeRoot()
require.NoError(t, err)
@@ -376,24 +363,23 @@ func TestVerifyFinalizedConsistency_InconsistentRoot(t *testing.T) {
func TestVerifyFinalizedConsistency_OK(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
cfg := &Config{BeaconDB: beaconDB, ForkChoiceStore: protoarray.New(0, 0, [32]byte{})}
service, err := NewService(ctx, cfg)
opts := testServiceOptsWithDB(t)
service, err := NewService(ctx, opts...)
require.NoError(t, err)
b32 := testutil.NewBeaconBlock()
b32 := util.NewBeaconBlock()
b32.Block.Slot = 32
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, b32))
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(b32)))
r32, err := b32.Block.HashTreeRoot()
require.NoError(t, err)
service.finalizedCheckpt = &ethpb.Checkpoint{Epoch: 1, Root: r32[:]}
service.store.SetFinalizedCheckpt(&ethpb.Checkpoint{Root: r32[:], Epoch: 1})
b33 := testutil.NewBeaconBlock()
b33 := util.NewBeaconBlock()
b33.Block.Slot = 33
b33.Block.ParentRoot = r32[:]
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, b33))
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(b33)))
r33, err := b33.Block.HashTreeRoot()
require.NoError(t, err)
@@ -403,20 +389,19 @@ func TestVerifyFinalizedConsistency_OK(t *testing.T) {
func TestVerifyFinalizedConsistency_IsCanonical(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
cfg := &Config{BeaconDB: beaconDB, ForkChoiceStore: protoarray.New(0, 0, [32]byte{})}
service, err := NewService(ctx, cfg)
opts := testServiceOptsWithDB(t)
service, err := NewService(ctx, opts...)
require.NoError(t, err)
b32 := testutil.NewBeaconBlock()
b32 := util.NewBeaconBlock()
b32.Block.Slot = 32
r32, err := b32.Block.HashTreeRoot()
require.NoError(t, err)
service.finalizedCheckpt = &ethpb.Checkpoint{Epoch: 1, Root: r32[:]}
service.store.SetFinalizedCheckpt(&ethpb.Checkpoint{Root: r32[:], Epoch: 1})
b33 := testutil.NewBeaconBlock()
b33 := util.NewBeaconBlock()
b33.Block.Slot = 33
b33.Block.ParentRoot = r32[:]
r33, err := b33.Block.HashTreeRoot()

View File

@@ -6,18 +6,22 @@ import (
"time"
"github.com/pkg/errors"
ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
"github.com/prysmaticlabs/prysm/beacon-chain/core/feed"
statefeed "github.com/prysmaticlabs/prysm/beacon-chain/core/feed/state"
"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
"github.com/prysmaticlabs/prysm/beacon-chain/core/state"
iface "github.com/prysmaticlabs/prysm/beacon-chain/state/interface"
pb "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
"github.com/prysmaticlabs/prysm/shared/attestationutil"
"github.com/prysmaticlabs/prysm/shared/bls"
"github.com/prysmaticlabs/prysm/shared/bytesutil"
"github.com/prysmaticlabs/prysm/shared/featureconfig"
"github.com/prysmaticlabs/prysm/shared/params"
coreTime "github.com/prysmaticlabs/prysm/beacon-chain/core/time"
"github.com/prysmaticlabs/prysm/beacon-chain/core/transition"
"github.com/prysmaticlabs/prysm/beacon-chain/state"
"github.com/prysmaticlabs/prysm/config/features"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/crypto/bls"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
"github.com/prysmaticlabs/prysm/monitoring/tracing"
ethpbv1 "github.com/prysmaticlabs/prysm/proto/eth/v1"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/attestation"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/block"
"github.com/prysmaticlabs/prysm/time/slots"
"go.opencensus.io/trace"
)
@@ -81,81 +85,132 @@ var initialSyncBlockCacheSize = uint64(2 * params.BeaconConfig().SlotsPerEpoch)
// ancestor_at_finalized_slot = get_ancestor(store, store.justified_checkpoint.root, finalized_slot)
// if ancestor_at_finalized_slot != store.finalized_checkpoint.root:
// store.justified_checkpoint = state.current_justified_checkpoint
func (s *Service) onBlock(ctx context.Context, signed *ethpb.SignedBeaconBlock, blockRoot [32]byte) error {
func (s *Service) onBlock(ctx context.Context, signed block.SignedBeaconBlock, blockRoot [32]byte) error {
ctx, span := trace.StartSpan(ctx, "blockChain.onBlock")
defer span.End()
if signed == nil || signed.Block == nil {
return errors.New("nil block")
if err := helpers.BeaconBlockIsNil(signed); err != nil {
return err
}
b := signed.Block
b := signed.Block()
preState, err := s.getBlockPreState(ctx, b)
if err != nil {
return err
}
set, postState, err := state.ExecuteStateTransitionNoVerifyAnySig(ctx, preState, signed)
postState, err := transition.ExecuteStateTransition(ctx, preState, signed)
if err != nil {
return errors.Wrap(err, "could not execute state transition")
return err
}
valid, err := set.Verify()
if err != nil {
return errors.Wrap(err, "could not batch verify signature")
}
if !valid {
return errors.New("signature in block failed to verify")
// We add a proposer score boost to fork choice for the block root if applicable, right after
// running a successful state transition for the block.
if err := s.cfg.ForkChoiceStore.BoostProposerRoot(
ctx, signed.Block().Slot(), blockRoot, s.genesisTime,
); err != nil {
return err
}
if err := s.savePostStateInfo(ctx, blockRoot, signed, postState, false /* reg sync */); err != nil {
return err
}
// Updating next slot state cache can happen in the background. It shouldn't block rest of the process.
if featureconfig.Get().EnableNextSlotStateCache {
// If slasher is configured, forward the attestations in the block via
// an event feed for processing.
if features.Get().EnableSlasher {
// Feed the indexed attestation to slasher if enabled. This action
// is done in the background to avoid adding more load to this critical code path.
go func() {
// Use a custom deadline here, since this method runs asynchronously.
// We ignore the parent method's context and instead create a new one
// with a custom deadline, therefore using the background context instead.
slotCtx, cancel := context.WithTimeout(context.Background(), slotDeadline)
defer cancel()
if err := state.UpdateNextSlotCache(slotCtx, blockRoot[:], postState); err != nil {
log.WithError(err).Debug("could not update next slot state cache")
// Using a different context to prevent timeouts as this operation can be expensive
// and we want to avoid affecting the critical code path.
ctx := context.TODO()
for _, att := range signed.Block().Body().Attestations() {
committee, err := helpers.BeaconCommitteeFromState(ctx, preState, att.Data.Slot, att.Data.CommitteeIndex)
if err != nil {
log.WithError(err).Error("Could not get attestation committee")
tracing.AnnotateError(span, err)
return
}
indexedAtt, err := attestation.ConvertToIndexed(ctx, att, committee)
if err != nil {
log.WithError(err).Error("Could not convert to indexed attestation")
tracing.AnnotateError(span, err)
return
}
s.cfg.SlasherAttestationsFeed.Send(indexedAtt)
}
}()
}
// Update justified check point.
if postState.CurrentJustifiedCheckpoint().Epoch > s.justifiedCheckpt.Epoch {
justified := s.store.JustifiedCheckpt()
if justified == nil {
return errNilJustifiedInStore
}
currJustifiedEpoch := justified.Epoch
if postState.CurrentJustifiedCheckpoint().Epoch > currJustifiedEpoch {
if err := s.updateJustified(ctx, postState); err != nil {
return err
}
}
newFinalized := postState.FinalizedCheckpointEpoch() > s.finalizedCheckpt.Epoch
if featureconfig.Get().UpdateHeadTimely {
if newFinalized {
if err := s.finalizedImpliesNewJustified(ctx, postState); err != nil {
return errors.Wrap(err, "could not save new justified")
}
s.prevFinalizedCheckpt = s.finalizedCheckpt
s.finalizedCheckpt = postState.FinalizedCheckpoint()
}
finalized := s.store.FinalizedCheckpt()
if finalized == nil {
return errNilFinalizedInStore
}
newFinalized := postState.FinalizedCheckpointEpoch() > finalized.Epoch
if newFinalized {
s.store.SetPrevFinalizedCheckpt(finalized)
s.store.SetFinalizedCheckpt(postState.FinalizedCheckpoint())
s.store.SetPrevJustifiedCheckpt(justified)
s.store.SetJustifiedCheckpt(postState.CurrentJustifiedCheckpoint())
}
if err := s.updateHead(ctx, s.getJustifiedBalances()); err != nil {
log.WithError(err).Warn("Could not update head")
}
balances, err := s.justifiedBalances.get(ctx, bytesutil.ToBytes32(justified.Root))
if err != nil {
msg := fmt.Sprintf("could not read balances for state w/ justified checkpoint %#x", justified.Root)
return errors.Wrap(err, msg)
}
if err := s.updateHead(ctx, balances); err != nil {
log.WithError(err).Warn("Could not update head")
}
// Send notification of the processed block to the state feed.
s.cfg.StateNotifier.StateFeed().Send(&feed.Event{
Type: statefeed.BlockProcessed,
Data: &statefeed.BlockProcessedData{
Slot: signed.Block.Slot,
BlockRoot: blockRoot,
SignedBlock: signed,
Verified: true,
},
})
if err := s.saveSyncedTipsDB(ctx); err != nil {
return errors.Wrap(err, "could not save synced tips")
}
if err := s.pruneCanonicalAttsFromPool(ctx, blockRoot, signed); err != nil {
return err
}
// Send notification of the processed block to the state feed.
s.cfg.StateNotifier.StateFeed().Send(&feed.Event{
Type: statefeed.BlockProcessed,
Data: &statefeed.BlockProcessedData{
Slot: signed.Block().Slot(),
BlockRoot: blockRoot,
SignedBlock: signed,
Verified: true,
},
})
// Updating next slot state cache can happen in the background. It shouldn't block rest of the process.
go func() {
// Use a custom deadline here, since this method runs asynchronously.
// We ignore the parent method's context and instead create a new one
// with a custom deadline, therefore using the background context instead.
slotCtx, cancel := context.WithTimeout(context.Background(), slotDeadline)
defer cancel()
if err := transition.UpdateNextSlotCache(slotCtx, blockRoot[:], postState); err != nil {
log.WithError(err).Debug("could not update next slot state cache")
}
}()
// Save justified check point to db.
if postState.CurrentJustifiedCheckpoint().Epoch > currJustifiedEpoch {
if err := s.cfg.BeaconDB.SaveJustifiedCheckpoint(ctx, postState.CurrentJustifiedCheckpoint()); err != nil {
return err
}
}
// Update finalized check point.
@@ -167,12 +222,17 @@ func (s *Service) onBlock(ctx context.Context, signed *ethpb.SignedBeaconBlock,
if err := s.cfg.ForkChoiceStore.Prune(ctx, fRoot); err != nil {
return errors.Wrap(err, "could not prune proto array fork choice nodes")
}
if !featureconfig.Get().UpdateHeadTimely {
if err := s.finalizedImpliesNewJustified(ctx, postState); err != nil {
return errors.Wrap(err, "could not save new justified")
}
}
go func() {
// Send an event regarding the new finalized checkpoint over a common event feed.
s.cfg.StateNotifier.StateFeed().Send(&feed.Event{
Type: statefeed.FinalizedCheckpoint,
Data: &ethpbv1.EventFinalizedCheckpoint{
Epoch: postState.FinalizedCheckpoint().Epoch,
Block: postState.FinalizedCheckpoint().Root,
State: signed.Block().StateRoot(),
},
})
// Use a custom deadline here, since this method runs asynchronously.
// We ignore the parent method's context and instead create a new one
// with a custom deadline, therefore using the background context instead.
@@ -182,6 +242,7 @@ func (s *Service) onBlock(ctx context.Context, signed *ethpb.SignedBeaconBlock,
log.WithError(err).Error("Could not insert finalized deposits.")
}
}()
}
defer reportAttestationInclusion(b)
@@ -189,7 +250,7 @@ func (s *Service) onBlock(ctx context.Context, signed *ethpb.SignedBeaconBlock,
return s.handleEpochBoundary(ctx, postState)
}
func (s *Service) onBlockBatch(ctx context.Context, blks []*ethpb.SignedBeaconBlock,
func (s *Service) onBlockBatch(ctx context.Context, blks []block.SignedBeaconBlock,
blockRoots [][32]byte) ([]*ethpb.Checkpoint, []*ethpb.Checkpoint, error) {
ctx, span := trace.StartSpan(ctx, "blockChain.onBlockBatch")
defer span.End()
@@ -197,39 +258,39 @@ func (s *Service) onBlockBatch(ctx context.Context, blks []*ethpb.SignedBeaconBl
if len(blks) == 0 || len(blockRoots) == 0 {
return nil, nil, errors.New("no blocks provided")
}
if blks[0] == nil || blks[0].Block == nil {
return nil, nil, errors.New("nil block")
if err := helpers.BeaconBlockIsNil(blks[0]); err != nil {
return nil, nil, err
}
b := blks[0].Block
b := blks[0].Block()
// Retrieve incoming block's pre state.
if err := s.verifyBlkPreState(ctx, b); err != nil {
return nil, nil, err
}
preState, err := s.cfg.StateGen.StateByRootInitialSync(ctx, bytesutil.ToBytes32(b.ParentRoot))
preState, err := s.cfg.StateGen.StateByRootInitialSync(ctx, bytesutil.ToBytes32(b.ParentRoot()))
if err != nil {
return nil, nil, err
}
if preState == nil {
return nil, nil, fmt.Errorf("nil pre state for slot %d", b.Slot)
if preState == nil || preState.IsNil() {
return nil, nil, fmt.Errorf("nil pre state for slot %d", b.Slot())
}
jCheckpoints := make([]*ethpb.Checkpoint, len(blks))
fCheckpoints := make([]*ethpb.Checkpoint, len(blks))
sigSet := &bls.SignatureSet{
sigSet := &bls.SignatureBatch{
Signatures: [][]byte{},
PublicKeys: []bls.PublicKey{},
Messages: [][32]byte{},
}
var set *bls.SignatureSet
boundaries := make(map[[32]byte]iface.BeaconState)
var set *bls.SignatureBatch
boundaries := make(map[[32]byte]state.BeaconState)
for i, b := range blks {
set, preState, err = state.ExecuteStateTransitionNoVerifyAnySig(ctx, preState, b)
set, preState, err = transition.ExecuteStateTransitionNoVerifyAnySig(ctx, preState, b)
if err != nil {
return nil, nil, err
}
// Save potential boundary states.
if helpers.IsEpochStart(preState.Slot()) {
if slots.IsEpochStart(preState.Slot()) {
boundaries[blockRoots[i]] = preState.Copy()
if err := s.handleEpochBoundary(ctx, preState); err != nil {
return nil, nil, errors.Wrap(err, "could not handle epoch boundary state")
@@ -265,16 +326,20 @@ func (s *Service) onBlockBatch(ctx context.Context, blks []*ethpb.SignedBeaconBl
// handles a block after the block's batch has been verified, where we can save blocks
// their state summaries and split them off to relative hot/cold storage.
func (s *Service) handleBlockAfterBatchVerify(ctx context.Context, signed *ethpb.SignedBeaconBlock,
func (s *Service) handleBlockAfterBatchVerify(ctx context.Context, signed block.SignedBeaconBlock,
blockRoot [32]byte, fCheckpoint, jCheckpoint *ethpb.Checkpoint) error {
b := signed.Block
b := signed.Block()
s.saveInitSyncBlock(blockRoot, signed)
if err := s.insertBlockToForkChoiceStore(ctx, b, blockRoot, fCheckpoint, jCheckpoint); err != nil {
return err
}
if err := s.cfg.BeaconDB.SaveStateSummary(ctx, &pb.StateSummary{
Slot: signed.Block.Slot,
if err := s.saveSyncedTipsDB(ctx); err != nil {
return errors.Wrap(err, "could not save synced tips")
}
if err := s.cfg.BeaconDB.SaveStateSummary(ctx, &ethpb.StateSummary{
Slot: signed.Block().Slot(),
Root: blockRoot[:],
}); err != nil {
return err
@@ -288,34 +353,47 @@ func (s *Service) handleBlockAfterBatchVerify(ctx context.Context, signed *ethpb
s.clearInitSyncBlocks()
}
if jCheckpoint.Epoch > s.justifiedCheckpt.Epoch {
justified := s.store.JustifiedCheckpt()
if justified == nil {
return errNilJustifiedInStore
}
if jCheckpoint.Epoch > justified.Epoch {
if err := s.updateJustifiedInitSync(ctx, jCheckpoint); err != nil {
return err
}
}
finalized := s.store.FinalizedCheckpt()
if finalized == nil {
return errNilFinalizedInStore
}
// Update finalized check point. Prune the block cache and helper caches on every new finalized epoch.
if fCheckpoint.Epoch > s.finalizedCheckpt.Epoch {
if fCheckpoint.Epoch > finalized.Epoch {
if err := s.updateFinalized(ctx, fCheckpoint); err != nil {
return err
}
s.store.SetPrevFinalizedCheckpt(finalized)
s.store.SetFinalizedCheckpt(fCheckpoint)
}
return nil
}
// Epoch boundary bookkeeping such as logging epoch summaries.
func (s *Service) handleEpochBoundary(ctx context.Context, postState iface.BeaconState) error {
func (s *Service) handleEpochBoundary(ctx context.Context, postState state.BeaconState) error {
ctx, span := trace.StartSpan(ctx, "blockChain.handleEpochBoundary")
defer span.End()
if postState.Slot()+1 == s.nextEpochBoundarySlot {
// Update caches for the next epoch at epoch boundary slot - 1.
if err := helpers.UpdateCommitteeCache(postState, helpers.NextEpoch(postState)); err != nil {
if err := helpers.UpdateCommitteeCache(postState, coreTime.NextEpoch(postState)); err != nil {
return err
}
copied := postState.Copy()
copied, err := state.ProcessSlots(ctx, copied, copied.Slot()+1)
copied, err := transition.ProcessSlots(ctx, copied, copied.Slot()+1)
if err != nil {
return err
}
if err := helpers.UpdateProposerIndicesInCache(copied); err != nil {
if err := helpers.UpdateProposerIndicesInCache(ctx, copied); err != nil {
return err
}
} else if postState.Slot() >= s.nextEpochBoundarySlot {
@@ -323,17 +401,17 @@ func (s *Service) handleEpochBoundary(ctx context.Context, postState iface.Beaco
return err
}
var err error
s.nextEpochBoundarySlot, err = helpers.StartSlot(helpers.NextEpoch(postState))
s.nextEpochBoundarySlot, err = slots.EpochStart(coreTime.NextEpoch(postState))
if err != nil {
return err
}
// Update caches at epoch boundary slot.
// The following updates have short cut to return nil cheaply if fulfilled during boundary slot - 1.
if err := helpers.UpdateCommitteeCache(postState, helpers.CurrentEpoch(postState)); err != nil {
if err := helpers.UpdateCommitteeCache(postState, coreTime.CurrentEpoch(postState)); err != nil {
return err
}
if err := helpers.UpdateProposerIndicesInCache(postState); err != nil {
if err := helpers.UpdateProposerIndicesInCache(ctx, postState); err != nil {
return err
}
}
@@ -343,20 +421,23 @@ func (s *Service) handleEpochBoundary(ctx context.Context, postState iface.Beaco
// This feeds in the block and block's attestations to fork choice store. It's allows fork choice store
// to gain information on the most current chain.
func (s *Service) insertBlockAndAttestationsToForkChoiceStore(ctx context.Context, blk *ethpb.BeaconBlock, root [32]byte,
st iface.BeaconState) error {
func (s *Service) insertBlockAndAttestationsToForkChoiceStore(ctx context.Context, blk block.BeaconBlock, root [32]byte,
st state.BeaconState) error {
ctx, span := trace.StartSpan(ctx, "blockChain.insertBlockAndAttestationsToForkChoiceStore")
defer span.End()
fCheckpoint := st.FinalizedCheckpoint()
jCheckpoint := st.CurrentJustifiedCheckpoint()
if err := s.insertBlockToForkChoiceStore(ctx, blk, root, fCheckpoint, jCheckpoint); err != nil {
return err
}
// Feed in block's attestations to fork choice store.
for _, a := range blk.Body.Attestations {
committee, err := helpers.BeaconCommitteeFromState(st, a.Data.Slot, a.Data.CommitteeIndex)
for _, a := range blk.Body().Attestations() {
committee, err := helpers.BeaconCommitteeFromState(ctx, st, a.Data.Slot, a.Data.CommitteeIndex)
if err != nil {
return err
}
indices, err := attestationutil.AttestingIndices(a.AggregationBits, committee)
indices, err := attestation.AttestingIndices(a.AggregationBits, committee)
if err != nil {
return err
}
@@ -365,14 +446,14 @@ func (s *Service) insertBlockAndAttestationsToForkChoiceStore(ctx context.Contex
return nil
}
func (s *Service) insertBlockToForkChoiceStore(ctx context.Context, blk *ethpb.BeaconBlock,
func (s *Service) insertBlockToForkChoiceStore(ctx context.Context, blk block.BeaconBlock,
root [32]byte, fCheckpoint, jCheckpoint *ethpb.Checkpoint) error {
if err := s.fillInForkChoiceMissingBlocks(ctx, blk, fCheckpoint, jCheckpoint); err != nil {
return err
}
// Feed in block to fork choice store.
if err := s.cfg.ForkChoiceStore.ProcessBlock(ctx,
blk.Slot, root, bytesutil.ToBytes32(blk.ParentRoot), bytesutil.ToBytes32(blk.Body.Graffiti),
blk.Slot(), root, bytesutil.ToBytes32(blk.ParentRoot()), bytesutil.ToBytes32(blk.Body().Graffiti()),
jCheckpoint.Epoch,
fCheckpoint.Epoch); err != nil {
return errors.Wrap(err, "could not process block for proto array fork choice")
@@ -382,19 +463,58 @@ func (s *Service) insertBlockToForkChoiceStore(ctx context.Context, blk *ethpb.B
// This saves post state info to DB or cache. This also saves post state info to fork choice store.
// Post state info consists of processed block and state. Do not call this method unless the block and state are verified.
func (s *Service) savePostStateInfo(ctx context.Context, r [32]byte, b *ethpb.SignedBeaconBlock, st iface.BeaconState, initSync bool) error {
func (s *Service) savePostStateInfo(ctx context.Context, r [32]byte, b block.SignedBeaconBlock, st state.BeaconState, initSync bool) error {
ctx, span := trace.StartSpan(ctx, "blockChain.savePostStateInfo")
defer span.End()
if initSync {
s.saveInitSyncBlock(r, b)
} else if err := s.cfg.BeaconDB.SaveBlock(ctx, b); err != nil {
return errors.Wrapf(err, "could not save block from slot %d", b.Block.Slot)
return errors.Wrapf(err, "could not save block from slot %d", b.Block().Slot())
}
if err := s.cfg.StateGen.SaveState(ctx, r, st); err != nil {
return errors.Wrap(err, "could not save state")
}
if err := s.insertBlockAndAttestationsToForkChoiceStore(ctx, b.Block, r, st); err != nil {
return errors.Wrapf(err, "could not insert block %d to fork choice store", b.Block.Slot)
if err := s.insertBlockAndAttestationsToForkChoiceStore(ctx, b.Block(), r, st); err != nil {
return errors.Wrapf(err, "could not insert block %d to fork choice store", b.Block().Slot())
}
return nil
}
// This removes the attestations from the mem pool. It will only remove the attestations if input root `r` is canonical,
// meaning the block `b` is part of the canonical chain.
func (s *Service) pruneCanonicalAttsFromPool(ctx context.Context, r [32]byte, b block.SignedBeaconBlock) error {
if !features.Get().CorrectlyPruneCanonicalAtts {
return nil
}
canonical, err := s.IsCanonical(ctx, r)
if err != nil {
return err
}
if !canonical {
return nil
}
atts := b.Block().Body().Attestations()
for _, att := range atts {
if helpers.IsAggregated(att) {
if err := s.cfg.AttPool.DeleteAggregatedAttestation(att); err != nil {
return err
}
} else {
if err := s.cfg.AttPool.DeleteUnaggregatedAttestation(att); err != nil {
return err
}
}
}
return nil
}
// Saves synced and validated tips to DB.
func (s *Service) saveSyncedTipsDB(ctx context.Context) error {
tips := s.cfg.ForkChoiceStore.SyncedTips()
if len(tips) == 0 {
return nil
}
return s.cfg.BeaconDB.UpdateValidatedTips(ctx, tips)
}

View File

@@ -7,26 +7,26 @@ import (
"github.com/pkg/errors"
types "github.com/prysmaticlabs/eth2-types"
ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
iface "github.com/prysmaticlabs/prysm/beacon-chain/state/interface"
"github.com/prysmaticlabs/prysm/shared/attestationutil"
"github.com/prysmaticlabs/prysm/shared/bytesutil"
"github.com/prysmaticlabs/prysm/shared/featureconfig"
"github.com/prysmaticlabs/prysm/shared/params"
"github.com/prysmaticlabs/prysm/shared/traceutil"
"github.com/prysmaticlabs/prysm/beacon-chain/state"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
"github.com/prysmaticlabs/prysm/monitoring/tracing"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/block"
"github.com/prysmaticlabs/prysm/time/slots"
"go.opencensus.io/trace"
)
// CurrentSlot returns the current slot based on time.
func (s *Service) CurrentSlot() types.Slot {
return helpers.CurrentSlot(uint64(s.genesisTime.Unix()))
return slots.CurrentSlot(uint64(s.genesisTime.Unix()))
}
// getBlockPreState returns the pre state of an incoming block. It uses the parent root of the block
// to retrieve the state in DB. It verifies the pre state's validity and the incoming block
// is in the correct time window.
func (s *Service) getBlockPreState(ctx context.Context, b *ethpb.BeaconBlock) (iface.BeaconState, error) {
func (s *Service) getBlockPreState(ctx context.Context, b block.BeaconBlock) (state.BeaconState, error) {
ctx, span := trace.StartSpan(ctx, "blockChain.getBlockPreState")
defer span.End()
@@ -35,16 +35,16 @@ func (s *Service) getBlockPreState(ctx context.Context, b *ethpb.BeaconBlock) (i
return nil, err
}
preState, err := s.cfg.StateGen.StateByRoot(ctx, bytesutil.ToBytes32(b.ParentRoot))
preState, err := s.cfg.StateGen.StateByRoot(ctx, bytesutil.ToBytes32(b.ParentRoot()))
if err != nil {
return nil, errors.Wrapf(err, "could not get pre state for slot %d", b.Slot)
return nil, errors.Wrapf(err, "could not get pre state for slot %d", b.Slot())
}
if preState == nil {
return nil, errors.Wrapf(err, "nil pre state for slot %d", b.Slot)
if preState == nil || preState.IsNil() {
return nil, errors.Wrapf(err, "nil pre state for slot %d", b.Slot())
}
// Verify block slot time is not from the future.
if err := helpers.VerifySlotTime(preState.GenesisTime(), b.Slot, params.BeaconNetworkConfig().MaximumGossipClockDisparity); err != nil {
if err := slots.VerifyTime(uint64(s.genesisTime.Unix()), b.Slot(), params.BeaconNetworkConfig().MaximumGossipClockDisparity); err != nil {
return nil, err
}
@@ -57,11 +57,11 @@ func (s *Service) getBlockPreState(ctx context.Context, b *ethpb.BeaconBlock) (i
}
// verifyBlkPreState validates input block has a valid pre-state.
func (s *Service) verifyBlkPreState(ctx context.Context, b *ethpb.BeaconBlock) error {
func (s *Service) verifyBlkPreState(ctx context.Context, b block.BeaconBlock) error {
ctx, span := trace.StartSpan(ctx, "blockChain.verifyBlkPreState")
defer span.End()
parentRoot := bytesutil.ToBytes32(b.ParentRoot)
parentRoot := bytesutil.ToBytes32(b.ParentRoot())
// Loosen the check to HasBlock because state summary gets saved in batches
// during initial syncing. There's no risk given a state summary object is just a
// a subset of the block object.
@@ -69,7 +69,7 @@ func (s *Service) verifyBlkPreState(ctx context.Context, b *ethpb.BeaconBlock) e
return errors.New("could not reconstruct parent state")
}
if err := s.VerifyBlkDescendant(ctx, bytesutil.ToBytes32(b.ParentRoot)); err != nil {
if err := s.VerifyBlkDescendant(ctx, bytesutil.ToBytes32(b.ParentRoot())); err != nil {
return err
}
@@ -91,16 +91,20 @@ func (s *Service) verifyBlkPreState(ctx context.Context, b *ethpb.BeaconBlock) e
func (s *Service) VerifyBlkDescendant(ctx context.Context, root [32]byte) error {
ctx, span := trace.StartSpan(ctx, "blockChain.VerifyBlkDescendant")
defer span.End()
fRoot := s.ensureRootNotZeros(bytesutil.ToBytes32(s.finalizedCheckpt.Root))
finalized := s.store.FinalizedCheckpt()
if finalized == nil {
return errNilFinalizedInStore
}
fRoot := s.ensureRootNotZeros(bytesutil.ToBytes32(finalized.Root))
finalizedBlkSigned, err := s.cfg.BeaconDB.Block(ctx, fRoot)
if err != nil {
return err
}
if finalizedBlkSigned == nil || finalizedBlkSigned.Block == nil {
if finalizedBlkSigned == nil || finalizedBlkSigned.IsNil() || finalizedBlkSigned.Block().IsNil() {
return errors.New("nil finalized block")
}
finalizedBlk := finalizedBlkSigned.Block
bFinalizedRoot, err := s.ancestor(ctx, root[:], finalizedBlk.Slot)
finalizedBlk := finalizedBlkSigned.Block()
bFinalizedRoot, err := s.ancestor(ctx, root[:], finalizedBlk.Slot())
if err != nil {
return errors.Wrap(err, "could not get finalized block root")
}
@@ -110,9 +114,9 @@ func (s *Service) VerifyBlkDescendant(ctx context.Context, root [32]byte) error
if !bytes.Equal(bFinalizedRoot, fRoot[:]) {
err := fmt.Errorf("block %#x is not a descendent of the current finalized block slot %d, %#x != %#x",
bytesutil.Trunc(root[:]), finalizedBlk.Slot, bytesutil.Trunc(bFinalizedRoot),
bytesutil.Trunc(root[:]), finalizedBlk.Slot(), bytesutil.Trunc(bFinalizedRoot),
bytesutil.Trunc(fRoot[:]))
traceutil.AnnotateError(span, err)
tracing.AnnotateError(span, err)
return err
}
return nil
@@ -120,13 +124,17 @@ func (s *Service) VerifyBlkDescendant(ctx context.Context, root [32]byte) error
// verifyBlkFinalizedSlot validates input block is not less than or equal
// to current finalized slot.
func (s *Service) verifyBlkFinalizedSlot(b *ethpb.BeaconBlock) error {
finalizedSlot, err := helpers.StartSlot(s.finalizedCheckpt.Epoch)
func (s *Service) verifyBlkFinalizedSlot(b block.BeaconBlock) error {
finalized := s.store.FinalizedCheckpt()
if finalized == nil {
return errNilFinalizedInStore
}
finalizedSlot, err := slots.EpochStart(finalized.Epoch)
if err != nil {
return err
}
if finalizedSlot >= b.Slot {
return fmt.Errorf("block is equal or earlier than finalized block, slot %d < slot %d", b.Slot, finalizedSlot)
if finalizedSlot >= b.Slot() {
return fmt.Errorf("block is equal or earlier than finalized block, slot %d < slot %d", b.Slot(), finalizedSlot)
}
return nil
}
@@ -134,63 +142,59 @@ func (s *Service) verifyBlkFinalizedSlot(b *ethpb.BeaconBlock) error {
// shouldUpdateCurrentJustified prevents bouncing attack, by only update conflicting justified
// checkpoints in the fork choice if in the early slots of the epoch.
// Otherwise, delay incorporation of new justified checkpoint until next epoch boundary.
// See https://ethresear.ch/t/prevention-of-bouncing-attack-on-ffg/6114 for more detailed analysis and discussion.
//
// Spec code:
// def should_update_justified_checkpoint(store: Store, new_justified_checkpoint: Checkpoint) -> bool:
// """
// To address the bouncing attack, only update conflicting justified
// checkpoints in the fork choice if in the early slots of the epoch.
// Otherwise, delay incorporation of new justified checkpoint until next epoch boundary.
//
// See https://ethresear.ch/t/prevention-of-bouncing-attack-on-ffg/6114 for more detailed analysis and discussion.
// """
// if compute_slots_since_epoch_start(get_current_slot(store)) < SAFE_SLOTS_TO_UPDATE_JUSTIFIED:
// return True
//
// justified_slot = compute_start_slot_at_epoch(store.justified_checkpoint.epoch)
// if not get_ancestor(store, new_justified_checkpoint.root, justified_slot) == store.justified_checkpoint.root:
// return False
//
// return True
func (s *Service) shouldUpdateCurrentJustified(ctx context.Context, newJustifiedCheckpt *ethpb.Checkpoint) (bool, error) {
if helpers.SlotsSinceEpochStarts(s.CurrentSlot()) < params.BeaconConfig().SafeSlotsToUpdateJustified {
ctx, span := trace.StartSpan(ctx, "blockChain.shouldUpdateCurrentJustified")
defer span.End()
if slots.SinceEpochStarts(s.CurrentSlot()) < params.BeaconConfig().SafeSlotsToUpdateJustified {
return true, nil
}
var newJustifiedBlockSigned *ethpb.SignedBeaconBlock
justified := s.store.JustifiedCheckpt()
jSlot, err := slots.EpochStart(justified.Epoch)
if err != nil {
return false, err
}
justifiedRoot := s.ensureRootNotZeros(bytesutil.ToBytes32(newJustifiedCheckpt.Root))
var err error
if s.hasInitSyncBlock(justifiedRoot) {
newJustifiedBlockSigned = s.getInitSyncBlock(justifiedRoot)
} else {
newJustifiedBlockSigned, err = s.cfg.BeaconDB.Block(ctx, justifiedRoot)
if err != nil {
return false, err
}
}
if newJustifiedBlockSigned == nil || newJustifiedBlockSigned.Block == nil {
return false, errors.New("nil new justified block")
}
newJustifiedBlock := newJustifiedBlockSigned.Block
jSlot, err := helpers.StartSlot(s.justifiedCheckpt.Epoch)
b, err := s.ancestor(ctx, justifiedRoot[:], jSlot)
if err != nil {
return false, err
}
if newJustifiedBlock.Slot <= jSlot {
if !bytes.Equal(b, justified.Root) {
return false, nil
}
var justifiedBlockSigned *ethpb.SignedBeaconBlock
cachedJustifiedRoot := s.ensureRootNotZeros(bytesutil.ToBytes32(s.justifiedCheckpt.Root))
if s.hasInitSyncBlock(cachedJustifiedRoot) {
justifiedBlockSigned = s.getInitSyncBlock(cachedJustifiedRoot)
} else {
justifiedBlockSigned, err = s.cfg.BeaconDB.Block(ctx, cachedJustifiedRoot)
if err != nil {
return false, err
}
}
if justifiedBlockSigned == nil || justifiedBlockSigned.Block == nil {
return false, errors.New("nil justified block")
}
justifiedBlock := justifiedBlockSigned.Block
b, err := s.ancestor(ctx, justifiedRoot[:], justifiedBlock.Slot)
if err != nil {
return false, err
}
if !bytes.Equal(b, s.justifiedCheckpt.Root) {
return false, nil
}
return true, nil
}
func (s *Service) updateJustified(ctx context.Context, state iface.ReadOnlyBeaconState) error {
func (s *Service) updateJustified(ctx context.Context, state state.ReadOnlyBeaconState) error {
ctx, span := trace.StartSpan(ctx, "blockChain.updateJustified")
defer span.End()
cpt := state.CurrentJustifiedCheckpoint()
if cpt.Epoch > s.bestJustifiedCheckpt.Epoch {
s.bestJustifiedCheckpt = cpt
bestJustified := s.store.BestJustifiedCheckpt()
if bestJustified == nil {
return errNilBestJustifiedInStore
}
if cpt.Epoch > bestJustified.Epoch {
s.store.SetBestJustifiedCheckpt(cpt)
}
canUpdate, err := s.shouldUpdateCurrentJustified(ctx, cpt)
if err != nil {
@@ -198,30 +202,39 @@ func (s *Service) updateJustified(ctx context.Context, state iface.ReadOnlyBeaco
}
if canUpdate {
s.prevJustifiedCheckpt = s.justifiedCheckpt
s.justifiedCheckpt = cpt
if err := s.cacheJustifiedStateBalances(ctx, bytesutil.ToBytes32(s.justifiedCheckpt.Root)); err != nil {
return err
justified := s.store.JustifiedCheckpt()
if justified == nil {
return errNilJustifiedInStore
}
s.store.SetPrevJustifiedCheckpt(justified)
s.store.SetJustifiedCheckpt(cpt)
}
return s.cfg.BeaconDB.SaveJustifiedCheckpoint(ctx, cpt)
return nil
}
// This caches input checkpoint as justified for the service struct. It rotates current justified to previous justified,
// caches justified checkpoint balances for fork choice and save justified checkpoint in DB.
// This method does not have defense against fork choice bouncing attack, which is why it's only recommend to be used during initial syncing.
func (s *Service) updateJustifiedInitSync(ctx context.Context, cp *ethpb.Checkpoint) error {
s.prevJustifiedCheckpt = s.justifiedCheckpt
s.justifiedCheckpt = cp
if err := s.cacheJustifiedStateBalances(ctx, bytesutil.ToBytes32(s.justifiedCheckpt.Root)); err != nil {
justified := s.store.JustifiedCheckpt()
if justified == nil {
return errNilJustifiedInStore
}
s.store.SetPrevJustifiedCheckpt(justified)
if err := s.cfg.BeaconDB.SaveJustifiedCheckpoint(ctx, cp); err != nil {
return err
}
s.store.SetJustifiedCheckpt(cp)
return s.cfg.BeaconDB.SaveJustifiedCheckpoint(ctx, cp)
return nil
}
func (s *Service) updateFinalized(ctx context.Context, cp *ethpb.Checkpoint) error {
ctx, span := trace.StartSpan(ctx, "blockChain.updateFinalized")
defer span.End()
// Blocks need to be saved so that we can retrieve finalized block from
// DB when migrating states.
if err := s.cfg.BeaconDB.SaveBlocks(ctx, s.getInitSyncBlocks()); err != nil {
@@ -232,10 +245,6 @@ func (s *Service) updateFinalized(ctx context.Context, cp *ethpb.Checkpoint) err
if err := s.cfg.BeaconDB.SaveFinalizedCheckpoint(ctx, cp); err != nil {
return err
}
if !featureconfig.Get().UpdateHeadTimely {
s.prevFinalizedCheckpt = s.finalizedCheckpt
s.finalizedCheckpt = cp
}
fRoot := bytesutil.ToBytes32(cp.Root)
if err := s.cfg.StateGen.MigrateToCold(ctx, fRoot); err != nil {
@@ -307,71 +316,32 @@ func (s *Service) ancestorByDB(ctx context.Context, r [32]byte, slot types.Slot)
signed = s.getInitSyncBlock(r)
}
if signed == nil || signed.Block == nil {
if signed == nil || signed.IsNil() || signed.Block().IsNil() {
return nil, errors.New("nil block")
}
b := signed.Block
if b.Slot == slot || b.Slot < slot {
b := signed.Block()
if b.Slot() == slot || b.Slot() < slot {
return r[:], nil
}
return s.ancestorByDB(ctx, bytesutil.ToBytes32(b.ParentRoot), slot)
}
// This updates justified check point in store, if the new justified is later than stored justified or
// the store's justified is not in chain with finalized check point.
//
// Spec definition:
// # Potentially update justified if different from store
// if store.justified_checkpoint != state.current_justified_checkpoint:
// # Update justified if new justified is later than store justified
// if state.current_justified_checkpoint.epoch > store.justified_checkpoint.epoch:
// store.justified_checkpoint = state.current_justified_checkpoint
// return
// # Update justified if store justified is not in chain with finalized checkpoint
// finalized_slot = compute_start_slot_at_epoch(store.finalized_checkpoint.epoch)
// ancestor_at_finalized_slot = get_ancestor(store, store.justified_checkpoint.root, finalized_slot)
// if ancestor_at_finalized_slot != store.finalized_checkpoint.root:
// store.justified_checkpoint = state.current_justified_checkpoint
func (s *Service) finalizedImpliesNewJustified(ctx context.Context, state iface.BeaconState) error {
// Update justified if it's different than the one cached in the store.
if !attestationutil.CheckPointIsEqual(s.justifiedCheckpt, state.CurrentJustifiedCheckpoint()) {
if state.CurrentJustifiedCheckpoint().Epoch > s.justifiedCheckpt.Epoch {
s.justifiedCheckpt = state.CurrentJustifiedCheckpoint()
return s.cacheJustifiedStateBalances(ctx, bytesutil.ToBytes32(s.justifiedCheckpt.Root))
}
// Update justified if store justified is not in chain with finalized check point.
finalizedSlot, err := helpers.StartSlot(s.finalizedCheckpt.Epoch)
if err != nil {
return err
}
justifiedRoot := s.ensureRootNotZeros(bytesutil.ToBytes32(s.justifiedCheckpt.Root))
anc, err := s.ancestor(ctx, justifiedRoot[:], finalizedSlot)
if err != nil {
return err
}
if !bytes.Equal(anc, s.finalizedCheckpt.Root) {
s.justifiedCheckpt = state.CurrentJustifiedCheckpoint()
if err := s.cacheJustifiedStateBalances(ctx, bytesutil.ToBytes32(s.justifiedCheckpt.Root)); err != nil {
return err
}
}
}
return nil
return s.ancestorByDB(ctx, bytesutil.ToBytes32(b.ParentRoot()), slot)
}
// This retrieves missing blocks from DB (ie. the blocks that couldn't be received over sync) and inserts them to fork choice store.
// This is useful for block tree visualizer and additional vote accounting.
func (s *Service) fillInForkChoiceMissingBlocks(ctx context.Context, blk *ethpb.BeaconBlock,
func (s *Service) fillInForkChoiceMissingBlocks(ctx context.Context, blk block.BeaconBlock,
fCheckpoint, jCheckpoint *ethpb.Checkpoint) error {
pendingNodes := make([]*ethpb.BeaconBlock, 0)
pendingNodes := make([]block.BeaconBlock, 0)
pendingRoots := make([][32]byte, 0)
parentRoot := bytesutil.ToBytes32(blk.ParentRoot)
slot := blk.Slot
parentRoot := bytesutil.ToBytes32(blk.ParentRoot())
slot := blk.Slot()
// Fork choice only matters from last finalized slot.
fSlot, err := helpers.StartSlot(s.finalizedCheckpt.Epoch)
finalized := s.store.FinalizedCheckpt()
if finalized == nil {
return errNilFinalizedInStore
}
fSlot, err := slots.EpochStart(finalized.Epoch)
if err != nil {
return err
}
@@ -383,11 +353,11 @@ func (s *Service) fillInForkChoiceMissingBlocks(ctx context.Context, blk *ethpb.
return err
}
pendingNodes = append(pendingNodes, b.Block)
pendingNodes = append(pendingNodes, b.Block())
copiedRoot := parentRoot
pendingRoots = append(pendingRoots, copiedRoot)
parentRoot = bytesutil.ToBytes32(b.Block.ParentRoot)
slot = b.Block.Slot
parentRoot = bytesutil.ToBytes32(b.Block().ParentRoot())
slot = b.Block().Slot()
higherThanFinalized = slot > fSlot
}
@@ -397,7 +367,7 @@ func (s *Service) fillInForkChoiceMissingBlocks(ctx context.Context, blk *ethpb.
b := pendingNodes[i]
r := pendingRoots[i]
if err := s.cfg.ForkChoiceStore.ProcessBlock(ctx,
b.Slot, r, bytesutil.ToBytes32(b.ParentRoot), bytesutil.ToBytes32(b.Body.Graffiti),
b.Slot(), r, bytesutil.ToBytes32(b.ParentRoot()), bytesutil.ToBytes32(b.Body().Graffiti()),
jCheckpoint.Epoch,
fCheckpoint.Epoch); err != nil {
return errors.Wrap(err, "could not process block for proto array fork choice")
@@ -450,7 +420,7 @@ func (s *Service) deletePoolAtts(atts []*ethpb.Attestation) error {
// fork choice justification routine.
func (s *Service) ensureRootNotZeros(root [32]byte) [32]byte {
if root == params.BeaconConfig().ZeroHash {
return s.genesisRoot
return s.originBlockRoot
}
return root
}

File diff suppressed because it is too large Load Diff

View File

@@ -7,21 +7,28 @@ import (
"time"
"github.com/pkg/errors"
ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
"github.com/prysmaticlabs/prysm/async/event"
"github.com/prysmaticlabs/prysm/beacon-chain/core/feed"
"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
iface "github.com/prysmaticlabs/prysm/beacon-chain/state/interface"
"github.com/prysmaticlabs/prysm/shared/bytesutil"
"github.com/prysmaticlabs/prysm/shared/params"
"github.com/prysmaticlabs/prysm/shared/slotutil"
"github.com/prysmaticlabs/prysm/beacon-chain/state"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/time/slots"
"github.com/sirupsen/logrus"
"go.opencensus.io/trace"
)
// AttestationStateFetcher allows for retrieving a beacon state corresponding to the block
// root of an attestation's target checkpoint.
type AttestationStateFetcher interface {
AttestationTargetState(ctx context.Context, target *ethpb.Checkpoint) (state.BeaconState, error)
}
// AttestationReceiver interface defines the methods of chain service receive and processing new attestations.
type AttestationReceiver interface {
AttestationStateFetcher
ReceiveAttestationNoPubsub(ctx context.Context, att *ethpb.Attestation) error
AttestationPreState(ctx context.Context, att *ethpb.Attestation) (iface.BeaconState, error)
VerifyLmdFfgConsistency(ctx context.Context, att *ethpb.Attestation) error
VerifyFinalizedConsistency(ctx context.Context, root []byte) error
}
@@ -35,28 +42,28 @@ func (s *Service) ReceiveAttestationNoPubsub(ctx context.Context, att *ethpb.Att
ctx, span := trace.StartSpan(ctx, "beacon-chain.blockchain.ReceiveAttestationNoPubsub")
defer span.End()
if err := s.onAttestation(ctx, att); err != nil {
if err := s.OnAttestation(ctx, att); err != nil {
return errors.Wrap(err, "could not process attestation")
}
return nil
}
// AttestationPreState returns the pre state of attestation.
func (s *Service) AttestationPreState(ctx context.Context, att *ethpb.Attestation) (iface.BeaconState, error) {
ss, err := helpers.StartSlot(att.Data.Target.Epoch)
// AttestationTargetState returns the pre state of attestation.
func (s *Service) AttestationTargetState(ctx context.Context, target *ethpb.Checkpoint) (state.BeaconState, error) {
ss, err := slots.EpochStart(target.Epoch)
if err != nil {
return nil, err
}
if err := helpers.ValidateSlotClock(ss, uint64(s.genesisTime.Unix())); err != nil {
if err := slots.ValidateClock(ss, uint64(s.genesisTime.Unix())); err != nil {
return nil, err
}
return s.getAttPreState(ctx, att.Data.Target)
return s.getAttPreState(ctx, target)
}
// VerifyLmdFfgConsistency verifies that attestation's LMD and FFG votes are consistency to each other.
func (s *Service) VerifyLmdFfgConsistency(ctx context.Context, a *ethpb.Attestation) error {
targetSlot, err := helpers.StartSlot(a.Data.Target.Epoch)
targetSlot, err := slots.EpochStart(a.Data.Target.Epoch)
if err != nil {
return err
}
@@ -81,7 +88,10 @@ func (s *Service) VerifyFinalizedConsistency(ctx context.Context, root []byte) e
}
f := s.FinalizedCheckpt()
ss, err := helpers.StartSlot(f.Epoch)
if f == nil {
return errNilFinalizedInStore
}
ss, err := slots.EpochStart(f.Epoch)
if err != nil {
return err
}
@@ -97,39 +107,66 @@ func (s *Service) VerifyFinalizedConsistency(ctx context.Context, root []byte) e
}
// This routine processes fork choice attestations from the pool to account for validator votes and fork choice.
func (s *Service) processAttestationsRoutine(subscribedToStateEvents chan<- struct{}) {
func (s *Service) spawnProcessAttestationsRoutine(stateFeed *event.Feed) {
// Wait for state to be initialized.
stateChannel := make(chan *feed.Event, 1)
stateSub := s.cfg.StateNotifier.StateFeed().Subscribe(stateChannel)
subscribedToStateEvents <- struct{}{}
<-stateChannel
stateSub.Unsubscribe()
if s.genesisTime.IsZero() {
log.Warn("ProcessAttestations routine waiting for genesis time")
for s.genesisTime.IsZero() {
time.Sleep(1 * time.Second)
}
log.Warn("Genesis time received, now available to process attestations")
}
st := slotutil.NewSlotTicker(s.genesisTime, params.BeaconConfig().SecondsPerSlot)
for {
stateSub := stateFeed.Subscribe(stateChannel)
go func() {
select {
case <-s.ctx.Done():
stateSub.Unsubscribe()
return
case <-st.C():
// Continue when there's no fork choice attestation, there's nothing to process and update head.
// This covers the condition when the node is still initial syncing to the head of the chain.
if s.cfg.AttPool.ForkchoiceAttestationCount() == 0 {
continue
case <-stateChannel:
stateSub.Unsubscribe()
break
}
if s.genesisTime.IsZero() {
log.Warn("ProcessAttestations routine waiting for genesis time")
for s.genesisTime.IsZero() {
if err := s.ctx.Err(); err != nil {
log.WithError(err).Error("Giving up waiting for genesis time")
return
}
time.Sleep(1 * time.Second)
}
s.processAttestations(s.ctx)
if err := s.updateHead(s.ctx, s.getJustifiedBalances()); err != nil {
log.Warnf("Resolving fork due to new attestation: %v", err)
log.Warn("Genesis time received, now available to process attestations")
}
st := slots.NewSlotTicker(s.genesisTime, params.BeaconConfig().SecondsPerSlot)
for {
select {
case <-s.ctx.Done():
return
case <-st.C():
if err := s.NewSlot(s.ctx, s.CurrentSlot()); err != nil {
log.WithError(err).Error("Could not process new slot")
return
}
// Continue when there's no fork choice attestation, there's nothing to process and update head.
// This covers the condition when the node is still initial syncing to the head of the chain.
if s.cfg.AttPool.ForkchoiceAttestationCount() == 0 {
continue
}
s.processAttestations(s.ctx)
justified := s.store.JustifiedCheckpt()
if justified == nil {
log.WithError(errNilJustifiedInStore).Error("Could not get justified checkpoint")
continue
}
balances, err := s.justifiedBalances.get(s.ctx, bytesutil.ToBytes32(justified.Root))
if err != nil {
log.WithError(err).Errorf("Unable to get justified balances for root %v", justified.Root)
continue
}
if err := s.updateHead(s.ctx, balances); err != nil {
log.WithError(err).Warn("Resolving fork due to new attestation")
}
}
}
}
}()
}
// This processes fork choice attestations from the pool to account for validator votes and fork choice.
@@ -138,9 +175,9 @@ func (s *Service) processAttestations(ctx context.Context) {
for _, a := range atts {
// Based on the spec, don't process the attestation until the subsequent slot.
// This delays consideration in the fork choice until their slot is in the past.
// https://github.com/ethereum/eth2.0-specs/blob/dev/specs/phase0/fork-choice.md#validate_on_attestation
// https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/fork-choice.md#validate_on_attestation
nextSlot := a.Data.Slot + 1
if err := helpers.VerifySlotTime(uint64(s.genesisTime.Unix()), nextSlot, params.BeaconNetworkConfig().MaximumGossipClockDisparity); err != nil {
if err := slots.VerifyTime(uint64(s.genesisTime.Unix()), nextSlot, params.BeaconNetworkConfig().MaximumGossipClockDisparity); err != nil {
continue
}

View File

@@ -6,21 +6,26 @@ import (
"time"
types "github.com/prysmaticlabs/eth2-types"
ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
"github.com/prysmaticlabs/prysm/beacon-chain/core/state"
"github.com/prysmaticlabs/prysm/beacon-chain/core/transition"
testDB "github.com/prysmaticlabs/prysm/beacon-chain/db/testing"
"github.com/prysmaticlabs/prysm/beacon-chain/forkchoice/protoarray"
"github.com/prysmaticlabs/prysm/beacon-chain/operations/attestations"
"github.com/prysmaticlabs/prysm/beacon-chain/state/stategen"
"github.com/prysmaticlabs/prysm/shared/bytesutil"
"github.com/prysmaticlabs/prysm/shared/params"
"github.com/prysmaticlabs/prysm/shared/testutil"
"github.com/prysmaticlabs/prysm/shared/testutil/require"
"github.com/prysmaticlabs/prysm/shared/timeutils"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/wrapper"
"github.com/prysmaticlabs/prysm/testing/require"
"github.com/prysmaticlabs/prysm/testing/util"
prysmTime "github.com/prysmaticlabs/prysm/time"
"github.com/prysmaticlabs/prysm/time/slots"
logTest "github.com/sirupsen/logrus/hooks/test"
)
var (
_ = AttestationReceiver(&Service{})
_ = AttestationStateFetcher(&Service{})
)
func TestAttestationCheckPtState_FarFutureSlot(t *testing.T) {
helpers.ClearCache()
beaconDB := testDB.SetupDB(t)
@@ -28,33 +33,32 @@ func TestAttestationCheckPtState_FarFutureSlot(t *testing.T) {
chainService := setupBeaconChain(t, beaconDB)
chainService.genesisTime = time.Now()
e := types.Epoch(helpers.MaxSlotBuffer/uint64(params.BeaconConfig().SlotsPerEpoch) + 1)
_, err := chainService.AttestationPreState(context.Background(), &ethpb.Attestation{Data: &ethpb.AttestationData{Target: &ethpb.Checkpoint{Epoch: e}}})
e := types.Epoch(slots.MaxSlotBuffer/uint64(params.BeaconConfig().SlotsPerEpoch) + 1)
_, err := chainService.AttestationTargetState(context.Background(), &ethpb.Checkpoint{Epoch: e})
require.ErrorContains(t, "exceeds max allowed value relative to the local clock", err)
}
func TestVerifyLMDFFGConsistent_NotOK(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
opts := testServiceOptsWithDB(t)
cfg := &Config{BeaconDB: beaconDB, ForkChoiceStore: protoarray.New(0, 0, [32]byte{})}
service, err := NewService(ctx, cfg)
service, err := NewService(ctx, opts...)
require.NoError(t, err)
b32 := testutil.NewBeaconBlock()
b32 := util.NewBeaconBlock()
b32.Block.Slot = 32
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, b32))
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(b32)))
r32, err := b32.Block.HashTreeRoot()
require.NoError(t, err)
b33 := testutil.NewBeaconBlock()
b33 := util.NewBeaconBlock()
b33.Block.Slot = 33
b33.Block.ParentRoot = r32[:]
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, b33))
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(b33)))
r33, err := b33.Block.HashTreeRoot()
require.NoError(t, err)
wanted := "FFG and LMD votes are not consistent"
a := testutil.NewAttestation()
a := util.NewAttestation()
a.Data.Target.Epoch = 1
a.Data.Target.Root = []byte{'a'}
a.Data.BeaconBlockRoot = r33[:]
@@ -63,25 +67,24 @@ func TestVerifyLMDFFGConsistent_NotOK(t *testing.T) {
func TestVerifyLMDFFGConsistent_OK(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
cfg := &Config{BeaconDB: beaconDB, ForkChoiceStore: protoarray.New(0, 0, [32]byte{})}
service, err := NewService(ctx, cfg)
opts := testServiceOptsWithDB(t)
service, err := NewService(ctx, opts...)
require.NoError(t, err)
b32 := testutil.NewBeaconBlock()
b32 := util.NewBeaconBlock()
b32.Block.Slot = 32
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, b32))
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(b32)))
r32, err := b32.Block.HashTreeRoot()
require.NoError(t, err)
b33 := testutil.NewBeaconBlock()
b33 := util.NewBeaconBlock()
b33.Block.Slot = 33
b33.Block.ParentRoot = r32[:]
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, b33))
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(b33)))
r33, err := b33.Block.HashTreeRoot()
require.NoError(t, err)
a := testutil.NewAttestation()
a := util.NewAttestation()
a.Data.Target.Epoch = 1
a.Data.Target.Root = r32[:]
a.Data.BeaconBlockRoot = r33[:]
@@ -92,25 +95,20 @@ func TestVerifyLMDFFGConsistent_OK(t *testing.T) {
func TestProcessAttestations_Ok(t *testing.T) {
hook := logTest.NewGlobal()
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
opts := testServiceOptsWithDB(t)
opts = append(opts, WithAttestationPool(attestations.NewPool()))
cfg := &Config{
BeaconDB: beaconDB,
ForkChoiceStore: protoarray.New(0, 0, [32]byte{}),
StateGen: stategen.New(beaconDB),
AttPool: attestations.NewPool(),
}
service, err := NewService(ctx, cfg)
service.genesisTime = timeutils.Now().Add(-1 * time.Duration(params.BeaconConfig().SecondsPerSlot) * time.Second)
service, err := NewService(ctx, opts...)
require.NoError(t, err)
genesisState, pks := testutil.DeterministicGenesisState(t, 64)
require.NoError(t, genesisState.SetGenesisTime(uint64(timeutils.Now().Unix())-params.BeaconConfig().SecondsPerSlot))
service.genesisTime = prysmTime.Now().Add(-1 * time.Duration(params.BeaconConfig().SecondsPerSlot) * time.Second)
genesisState, pks := util.DeterministicGenesisState(t, 64)
require.NoError(t, genesisState.SetGenesisTime(uint64(prysmTime.Now().Unix())-params.BeaconConfig().SecondsPerSlot))
require.NoError(t, service.saveGenesisData(ctx, genesisState))
atts, err := testutil.GenerateAttestations(genesisState, pks, 1, 0, false)
atts, err := util.GenerateAttestations(genesisState, pks, 1, 0, false)
require.NoError(t, err)
tRoot := bytesutil.ToBytes32(atts[0].Data.Target.Root)
copied := genesisState.Copy()
copied, err = state.ProcessSlots(ctx, copied, 1)
copied, err = transition.ProcessSlots(ctx, copied, 1)
require.NoError(t, err)
require.NoError(t, service.cfg.BeaconDB.SaveState(ctx, copied, tRoot))
require.NoError(t, service.cfg.ForkChoiceStore.ProcessBlock(ctx, 0, tRoot, tRoot, tRoot, 1, 1))

View File

@@ -5,14 +5,12 @@ import (
"github.com/pkg/errors"
types "github.com/prysmaticlabs/eth2-types"
ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
"github.com/prysmaticlabs/prysm/beacon-chain/core/feed"
statefeed "github.com/prysmaticlabs/prysm/beacon-chain/core/feed/state"
"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
"github.com/prysmaticlabs/prysm/beacon-chain/state/stateV0"
"github.com/prysmaticlabs/prysm/shared/featureconfig"
"github.com/prysmaticlabs/prysm/shared/timeutils"
"github.com/prysmaticlabs/prysm/shared/traceutil"
"github.com/prysmaticlabs/prysm/monitoring/tracing"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/block"
"github.com/prysmaticlabs/prysm/time"
"github.com/prysmaticlabs/prysm/time/slots"
"go.opencensus.io/trace"
)
@@ -21,8 +19,8 @@ var epochsSinceFinalitySaveHotStateDB = types.Epoch(100)
// BlockReceiver interface defines the methods of chain service receive and processing new blocks.
type BlockReceiver interface {
ReceiveBlock(ctx context.Context, block *ethpb.SignedBeaconBlock, blockRoot [32]byte) error
ReceiveBlockBatch(ctx context.Context, blocks []*ethpb.SignedBeaconBlock, blkRoots [][32]byte) error
ReceiveBlock(ctx context.Context, block block.SignedBeaconBlock, blockRoot [32]byte) error
ReceiveBlockBatch(ctx context.Context, blocks []block.SignedBeaconBlock, blkRoots [][32]byte) error
HasInitSyncBlock(root [32]byte) bool
}
@@ -31,38 +29,21 @@ type BlockReceiver interface {
// 1. Validate block, apply state transition and update check points
// 2. Apply fork choice to the processed block
// 3. Save latest head info
func (s *Service) ReceiveBlock(ctx context.Context, block *ethpb.SignedBeaconBlock, blockRoot [32]byte) error {
func (s *Service) ReceiveBlock(ctx context.Context, block block.SignedBeaconBlock, blockRoot [32]byte) error {
ctx, span := trace.StartSpan(ctx, "blockChain.ReceiveBlock")
defer span.End()
receivedTime := timeutils.Now()
blockCopy := stateV0.CopySignedBeaconBlock(block)
receivedTime := time.Now()
blockCopy := block.Copy()
// Apply state transition on the new block.
if err := s.onBlock(ctx, blockCopy, blockRoot); err != nil {
err := errors.Wrap(err, "could not process block")
traceutil.AnnotateError(span, err)
tracing.AnnotateError(span, err)
return err
}
// Update and save head block after fork choice.
if !featureconfig.Get().UpdateHeadTimely {
if err := s.updateHead(ctx, s.getJustifiedBalances()); err != nil {
log.WithError(err).Warn("Could not update head")
}
// Send notification of the processed block to the state feed.
s.cfg.StateNotifier.StateFeed().Send(&feed.Event{
Type: statefeed.BlockProcessed,
Data: &statefeed.BlockProcessedData{
Slot: blockCopy.Block.Slot,
BlockRoot: blockRoot,
SignedBlock: blockCopy,
Verified: true,
},
})
}
// Handle post block operations such as attestations and exits.
if err := s.handlePostBlockOperations(blockCopy.Block); err != nil {
if err := s.handlePostBlockOperations(blockCopy.Block()); err != nil {
return err
}
@@ -72,14 +53,18 @@ func (s *Service) ReceiveBlock(ctx context.Context, block *ethpb.SignedBeaconBlo
}
// Reports on block and fork choice metrics.
reportSlotMetrics(blockCopy.Block.Slot, s.HeadSlot(), s.CurrentSlot(), s.finalizedCheckpt)
finalized := s.store.FinalizedCheckpt()
if finalized == nil {
return errNilFinalizedInStore
}
reportSlotMetrics(blockCopy.Block().Slot(), s.HeadSlot(), s.CurrentSlot(), finalized)
// Log block sync status.
if err := logBlockSyncStatus(blockCopy.Block, blockRoot, s.finalizedCheckpt, receivedTime, uint64(s.genesisTime.Unix())); err != nil {
if err := logBlockSyncStatus(blockCopy.Block(), blockRoot, finalized, receivedTime, uint64(s.genesisTime.Unix())); err != nil {
return err
}
// Log state transition data.
logStateTransitionData(blockCopy.Block)
logStateTransitionData(blockCopy.Block())
return nil
}
@@ -87,7 +72,7 @@ func (s *Service) ReceiveBlock(ctx context.Context, block *ethpb.SignedBeaconBlo
// ReceiveBlockBatch processes the whole block batch at once, assuming the block batch is linear ,transitioning
// the state, performing batch verification of all collected signatures and then performing the appropriate
// actions for a block post-transition.
func (s *Service) ReceiveBlockBatch(ctx context.Context, blocks []*ethpb.SignedBeaconBlock, blkRoots [][32]byte) error {
func (s *Service) ReceiveBlockBatch(ctx context.Context, blocks []block.SignedBeaconBlock, blkRoots [][32]byte) error {
ctx, span := trace.StartSpan(ctx, "blockChain.ReceiveBlockBatch")
defer span.End()
@@ -95,21 +80,21 @@ func (s *Service) ReceiveBlockBatch(ctx context.Context, blocks []*ethpb.SignedB
fCheckpoints, jCheckpoints, err := s.onBlockBatch(ctx, blocks, blkRoots)
if err != nil {
err := errors.Wrap(err, "could not process block in batch")
traceutil.AnnotateError(span, err)
tracing.AnnotateError(span, err)
return err
}
for i, b := range blocks {
blockCopy := stateV0.CopySignedBeaconBlock(b)
blockCopy := b.Copy()
if err = s.handleBlockAfterBatchVerify(ctx, blockCopy, blkRoots[i], fCheckpoints[i], jCheckpoints[i]); err != nil {
traceutil.AnnotateError(span, err)
tracing.AnnotateError(span, err)
return err
}
// Send notification of the processed block to the state feed.
s.cfg.StateNotifier.StateFeed().Send(&feed.Event{
Type: statefeed.BlockProcessed,
Data: &statefeed.BlockProcessedData{
Slot: blockCopy.Block.Slot,
Slot: blockCopy.Block().Slot(),
BlockRoot: blkRoots[i],
SignedBlock: blockCopy,
Verified: true,
@@ -117,10 +102,21 @@ func (s *Service) ReceiveBlockBatch(ctx context.Context, blocks []*ethpb.SignedB
})
// Reports on blockCopy and fork choice metrics.
reportSlotMetrics(blockCopy.Block.Slot, s.HeadSlot(), s.CurrentSlot(), s.finalizedCheckpt)
finalized := s.store.FinalizedCheckpt()
if finalized == nil {
return errNilFinalizedInStore
}
reportSlotMetrics(blockCopy.Block().Slot(), s.HeadSlot(), s.CurrentSlot(), finalized)
}
if err := s.VerifyWeakSubjectivityRoot(s.ctx); err != nil {
if err := s.cfg.BeaconDB.SaveBlocks(ctx, s.getInitSyncBlocks()); err != nil {
return err
}
finalized := s.store.FinalizedCheckpt()
if finalized == nil {
return errNilFinalizedInStore
}
if err := s.wsVerifier.VerifyWeakSubjectivity(s.ctx, finalized.Epoch); err != nil {
// log.Fatalf will prevent defer from being called
span.End()
// Exit run time if the node failed to verify weak subjectivity checkpoint.
@@ -135,24 +131,24 @@ func (s *Service) HasInitSyncBlock(root [32]byte) bool {
return s.hasInitSyncBlock(root)
}
func (s *Service) handlePostBlockOperations(b *ethpb.BeaconBlock) error {
func (s *Service) handlePostBlockOperations(b block.BeaconBlock) error {
// Delete the processed block attestations from attestation pool.
if err := s.deletePoolAtts(b.Body.Attestations); err != nil {
if err := s.deletePoolAtts(b.Body().Attestations()); err != nil {
return err
}
// Add block attestations to the fork choice pool to compute head.
if err := s.cfg.AttPool.SaveBlockAttestations(b.Body.Attestations); err != nil {
if err := s.cfg.AttPool.SaveBlockAttestations(b.Body().Attestations()); err != nil {
log.Errorf("Could not save block attestations for fork choice: %v", err)
return nil
}
// Mark block exits as seen so we don't include same ones in future blocks.
for _, e := range b.Body.VoluntaryExits {
for _, e := range b.Body().VoluntaryExits() {
s.cfg.ExitPool.MarkIncluded(e)
}
// Mark attester slashings as seen so we don't include same ones in future blocks.
for _, as := range b.Body.AttesterSlashings {
for _, as := range b.Body().AttesterSlashings() {
s.cfg.SlashingPool.MarkIncludedAttesterSlashing(as)
}
return nil
@@ -161,11 +157,15 @@ func (s *Service) handlePostBlockOperations(b *ethpb.BeaconBlock) error {
// This checks whether it's time to start saving hot state to DB.
// It's time when there's `epochsSinceFinalitySaveHotStateDB` epochs of non-finality.
func (s *Service) checkSaveHotStateDB(ctx context.Context) error {
currentEpoch := helpers.SlotToEpoch(s.CurrentSlot())
currentEpoch := slots.ToEpoch(s.CurrentSlot())
// Prevent `sinceFinality` going underflow.
var sinceFinality types.Epoch
if currentEpoch > s.finalizedCheckpt.Epoch {
sinceFinality = currentEpoch - s.finalizedCheckpt.Epoch
finalized := s.store.FinalizedCheckpt()
if finalized == nil {
return errNilFinalizedInStore
}
if currentEpoch > finalized.Epoch {
sinceFinality = currentEpoch - finalized.Epoch
}
if sinceFinality >= epochsSinceFinalitySaveHotStateDB {

View File

@@ -7,30 +7,33 @@ import (
"time"
types "github.com/prysmaticlabs/eth2-types"
ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
blockchainTesting "github.com/prysmaticlabs/prysm/beacon-chain/blockchain/testing"
testDB "github.com/prysmaticlabs/prysm/beacon-chain/db/testing"
"github.com/prysmaticlabs/prysm/beacon-chain/forkchoice/protoarray"
"github.com/prysmaticlabs/prysm/beacon-chain/operations/attestations"
"github.com/prysmaticlabs/prysm/beacon-chain/operations/voluntaryexits"
"github.com/prysmaticlabs/prysm/beacon-chain/state/stategen"
"github.com/prysmaticlabs/prysm/shared/bytesutil"
"github.com/prysmaticlabs/prysm/shared/params"
"github.com/prysmaticlabs/prysm/shared/testutil"
"github.com/prysmaticlabs/prysm/shared/testutil/assert"
"github.com/prysmaticlabs/prysm/shared/testutil/require"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/block"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/wrapper"
"github.com/prysmaticlabs/prysm/testing/assert"
"github.com/prysmaticlabs/prysm/testing/require"
"github.com/prysmaticlabs/prysm/testing/util"
logTest "github.com/sirupsen/logrus/hooks/test"
)
func TestService_ReceiveBlock(t *testing.T) {
ctx := context.Background()
genesis, keys := testutil.DeterministicGenesisState(t, 64)
genFullBlock := func(t *testing.T, conf *testutil.BlockGenConfig, slot types.Slot) *ethpb.SignedBeaconBlock {
blk, err := testutil.GenerateFullBlock(genesis, keys, conf, slot)
genesis, keys := util.DeterministicGenesisState(t, 64)
genFullBlock := func(t *testing.T, conf *util.BlockGenConfig, slot types.Slot) *ethpb.SignedBeaconBlock {
blk, err := util.GenerateFullBlock(genesis, keys, conf, slot)
assert.NoError(t, err)
return blk
}
params.SetupTestConfigCleanup(t)
bc := params.BeaconConfig()
bc.ShardCommitteePeriod = 0 // Required for voluntary exits test in reasonable time.
params.OverrideBeaconConfig(bc)
@@ -47,13 +50,13 @@ func TestService_ReceiveBlock(t *testing.T) {
{
name: "applies block with state transition",
args: args{
block: genFullBlock(t, testutil.DefaultBlockGenConfig(), 2 /*slot*/),
block: genFullBlock(t, util.DefaultBlockGenConfig(), 2 /*slot*/),
},
check: func(t *testing.T, s *Service) {
if hs := s.head.state.Slot(); hs != 2 {
t.Errorf("Unexpected state slot. Got %d but wanted %d", hs, 2)
}
if bs := s.head.block.Block.Slot; bs != 2 {
if bs := s.head.block.Block().Slot(); bs != 2 {
t.Errorf("Unexpected head block slot. Got %d but wanted %d", bs, 2)
}
},
@@ -62,7 +65,7 @@ func TestService_ReceiveBlock(t *testing.T) {
name: "saves attestations to pool",
args: args{
block: genFullBlock(t,
&testutil.BlockGenConfig{
&util.BlockGenConfig{
NumProposerSlashings: 0,
NumAttesterSlashings: 0,
NumAttestations: 2,
@@ -82,7 +85,7 @@ func TestService_ReceiveBlock(t *testing.T) {
{
name: "updates exit pool",
args: args{
block: genFullBlock(t, &testutil.BlockGenConfig{
block: genFullBlock(t, &util.BlockGenConfig{
NumProposerSlashings: 0,
NumAttesterSlashings: 0,
NumAttestations: 0,
@@ -106,7 +109,7 @@ func TestService_ReceiveBlock(t *testing.T) {
{
name: "notifies block processed on state feed",
args: args{
block: genFullBlock(t, testutil.DefaultBlockGenConfig(), 1 /*slot*/),
block: genFullBlock(t, util.DefaultBlockGenConfig(), 1 /*slot*/),
},
check: func(t *testing.T, s *Service) {
if recvd := len(s.cfg.StateNotifier.(*blockchainTesting.MockStateNotifier).ReceivedEvents()); recvd < 1 {
@@ -122,29 +125,25 @@ func TestService_ReceiveBlock(t *testing.T) {
genesisBlockRoot := bytesutil.ToBytes32(nil)
require.NoError(t, beaconDB.SaveState(ctx, genesis, genesisBlockRoot))
cfg := &Config{
BeaconDB: beaconDB,
ForkChoiceStore: protoarray.New(
0, // justifiedEpoch
0, // finalizedEpoch
genesisBlockRoot,
),
AttPool: attestations.NewPool(),
ExitPool: voluntaryexits.NewPool(),
StateNotifier: &blockchainTesting.MockStateNotifier{RecordEvents: true},
StateGen: stategen.New(beaconDB),
opts := []Option{
WithDatabase(beaconDB),
WithForkChoiceStore(protoarray.New(0, 0, genesisBlockRoot)),
WithAttestationPool(attestations.NewPool()),
WithExitPool(voluntaryexits.NewPool()),
WithStateNotifier(&blockchainTesting.MockStateNotifier{RecordEvents: true}),
WithStateGen(stategen.New(beaconDB)),
}
s, err := NewService(ctx, cfg)
s, err := NewService(ctx, opts...)
require.NoError(t, err)
require.NoError(t, s.saveGenesisData(ctx, genesis))
gBlk, err := s.cfg.BeaconDB.GenesisBlock(ctx)
require.NoError(t, err)
gRoot, err := gBlk.Block.HashTreeRoot()
gRoot, err := gBlk.Block().HashTreeRoot()
require.NoError(t, err)
s.finalizedCheckpt = &ethpb.Checkpoint{Root: gRoot[:]}
s.store.SetFinalizedCheckpt(&ethpb.Checkpoint{Root: gRoot[:]})
root, err := tt.args.block.Block.HashTreeRoot()
require.NoError(t, err)
err = s.ReceiveBlock(ctx, tt.args.block, root)
err = s.ReceiveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(tt.args.block), root)
if tt.wantedErr != "" {
assert.ErrorContains(t, tt.wantedErr, err)
} else {
@@ -157,38 +156,35 @@ func TestService_ReceiveBlock(t *testing.T) {
func TestService_ReceiveBlockUpdateHead(t *testing.T) {
ctx := context.Background()
genesis, keys := testutil.DeterministicGenesisState(t, 64)
b, err := testutil.GenerateFullBlock(genesis, keys, testutil.DefaultBlockGenConfig(), 1)
genesis, keys := util.DeterministicGenesisState(t, 64)
b, err := util.GenerateFullBlock(genesis, keys, util.DefaultBlockGenConfig(), 1)
assert.NoError(t, err)
beaconDB := testDB.SetupDB(t)
genesisBlockRoot := bytesutil.ToBytes32(nil)
require.NoError(t, beaconDB.SaveState(ctx, genesis, genesisBlockRoot))
cfg := &Config{
BeaconDB: beaconDB,
ForkChoiceStore: protoarray.New(
0, // justifiedEpoch
0, // finalizedEpoch
genesisBlockRoot,
),
AttPool: attestations.NewPool(),
ExitPool: voluntaryexits.NewPool(),
StateNotifier: &blockchainTesting.MockStateNotifier{RecordEvents: true},
StateGen: stategen.New(beaconDB),
opts := []Option{
WithDatabase(beaconDB),
WithForkChoiceStore(protoarray.New(0, 0, genesisBlockRoot)),
WithAttestationPool(attestations.NewPool()),
WithExitPool(voluntaryexits.NewPool()),
WithStateNotifier(&blockchainTesting.MockStateNotifier{RecordEvents: true}),
WithStateGen(stategen.New(beaconDB)),
}
s, err := NewService(ctx, cfg)
s, err := NewService(ctx, opts...)
require.NoError(t, err)
require.NoError(t, s.saveGenesisData(ctx, genesis))
gBlk, err := s.cfg.BeaconDB.GenesisBlock(ctx)
require.NoError(t, err)
gRoot, err := gBlk.Block.HashTreeRoot()
gRoot, err := gBlk.Block().HashTreeRoot()
require.NoError(t, err)
s.finalizedCheckpt = &ethpb.Checkpoint{Root: gRoot[:]}
s.store.SetFinalizedCheckpt(&ethpb.Checkpoint{Root: gRoot[:]})
root, err := b.Block.HashTreeRoot()
require.NoError(t, err)
wg := sync.WaitGroup{}
wg.Add(1)
go func() {
require.NoError(t, s.ReceiveBlock(ctx, b, root))
require.NoError(t, s.ReceiveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(b), root))
wg.Done()
}()
wg.Wait()
@@ -202,9 +198,9 @@ func TestService_ReceiveBlockUpdateHead(t *testing.T) {
func TestService_ReceiveBlockBatch(t *testing.T) {
ctx := context.Background()
genesis, keys := testutil.DeterministicGenesisState(t, 64)
genFullBlock := func(t *testing.T, conf *testutil.BlockGenConfig, slot types.Slot) *ethpb.SignedBeaconBlock {
blk, err := testutil.GenerateFullBlock(genesis, keys, conf, slot)
genesis, keys := util.DeterministicGenesisState(t, 64)
genFullBlock := func(t *testing.T, conf *util.BlockGenConfig, slot types.Slot) *ethpb.SignedBeaconBlock {
blk, err := util.GenerateFullBlock(genesis, keys, conf, slot)
assert.NoError(t, err)
return blk
}
@@ -221,17 +217,17 @@ func TestService_ReceiveBlockBatch(t *testing.T) {
{
name: "applies block with state transition",
args: args{
block: genFullBlock(t, testutil.DefaultBlockGenConfig(), 2 /*slot*/),
block: genFullBlock(t, util.DefaultBlockGenConfig(), 2 /*slot*/),
},
check: func(t *testing.T, s *Service) {
assert.Equal(t, types.Slot(2), s.head.state.Slot(), "Incorrect head state slot")
assert.Equal(t, types.Slot(2), s.head.block.Block.Slot, "Incorrect head block slot")
assert.Equal(t, types.Slot(2), s.head.block.Block().Slot(), "Incorrect head block slot")
},
},
{
name: "notifies block processed on state feed",
args: args{
block: genFullBlock(t, testutil.DefaultBlockGenConfig(), 1 /*slot*/),
block: genFullBlock(t, util.DefaultBlockGenConfig(), 1 /*slot*/),
},
check: func(t *testing.T, s *Service) {
if recvd := len(s.cfg.StateNotifier.(*blockchainTesting.MockStateNotifier).ReceivedEvents()); recvd < 1 {
@@ -246,29 +242,25 @@ func TestService_ReceiveBlockBatch(t *testing.T) {
beaconDB := testDB.SetupDB(t)
genesisBlockRoot, err := genesis.HashTreeRoot(ctx)
require.NoError(t, err)
cfg := &Config{
BeaconDB: beaconDB,
ForkChoiceStore: protoarray.New(
0, // justifiedEpoch
0, // finalizedEpoch
genesisBlockRoot,
),
StateNotifier: &blockchainTesting.MockStateNotifier{RecordEvents: true},
StateGen: stategen.New(beaconDB),
opts := []Option{
WithDatabase(beaconDB),
WithForkChoiceStore(protoarray.New(0, 0, genesisBlockRoot)),
WithStateNotifier(&blockchainTesting.MockStateNotifier{RecordEvents: true}),
WithStateGen(stategen.New(beaconDB)),
}
s, err := NewService(ctx, cfg)
s, err := NewService(ctx, opts...)
require.NoError(t, err)
err = s.saveGenesisData(ctx, genesis)
require.NoError(t, err)
gBlk, err := s.cfg.BeaconDB.GenesisBlock(ctx)
require.NoError(t, err)
gRoot, err := gBlk.Block.HashTreeRoot()
gRoot, err := gBlk.Block().HashTreeRoot()
require.NoError(t, err)
s.finalizedCheckpt = &ethpb.Checkpoint{Root: gRoot[:]}
s.store.SetFinalizedCheckpt(&ethpb.Checkpoint{Root: gRoot[:]})
root, err := tt.args.block.Block.HashTreeRoot()
require.NoError(t, err)
blks := []*ethpb.SignedBeaconBlock{tt.args.block}
blks := []block.SignedBeaconBlock{wrapper.WrappedPhase0SignedBeaconBlock(tt.args.block)}
roots := [][32]byte{root}
err = s.ReceiveBlockBatch(ctx, blks, roots)
if tt.wantedErr != "" {
@@ -282,37 +274,39 @@ func TestService_ReceiveBlockBatch(t *testing.T) {
}
func TestService_HasInitSyncBlock(t *testing.T) {
s, err := NewService(context.Background(), &Config{StateNotifier: &blockchainTesting.MockStateNotifier{}})
opts := testServiceOptsNoDB()
opts = append(opts, WithStateNotifier(&blockchainTesting.MockStateNotifier{}))
s, err := NewService(context.Background(), opts...)
require.NoError(t, err)
r := [32]byte{'a'}
if s.HasInitSyncBlock(r) {
t.Error("Should not have block")
}
s.saveInitSyncBlock(r, testutil.NewBeaconBlock())
s.saveInitSyncBlock(r, wrapper.WrappedPhase0SignedBeaconBlock(util.NewBeaconBlock()))
if !s.HasInitSyncBlock(r) {
t.Error("Should have block")
}
}
func TestCheckSaveHotStateDB_Enabling(t *testing.T) {
beaconDB := testDB.SetupDB(t)
opts := testServiceOptsWithDB(t)
hook := logTest.NewGlobal()
s, err := NewService(context.Background(), &Config{StateGen: stategen.New(beaconDB)})
s, err := NewService(context.Background(), opts...)
require.NoError(t, err)
st := params.BeaconConfig().SlotsPerEpoch.Mul(uint64(epochsSinceFinalitySaveHotStateDB))
s.genesisTime = time.Now().Add(time.Duration(-1*int64(st)*int64(params.BeaconConfig().SecondsPerSlot)) * time.Second)
s.finalizedCheckpt = &ethpb.Checkpoint{}
s.store.SetFinalizedCheckpt(&ethpb.Checkpoint{})
require.NoError(t, s.checkSaveHotStateDB(context.Background()))
assert.LogsContain(t, hook, "Entering mode to save hot states in DB")
}
func TestCheckSaveHotStateDB_Disabling(t *testing.T) {
beaconDB := testDB.SetupDB(t)
hook := logTest.NewGlobal()
s, err := NewService(context.Background(), &Config{StateGen: stategen.New(beaconDB)})
opts := testServiceOptsWithDB(t)
s, err := NewService(context.Background(), opts...)
require.NoError(t, err)
s.finalizedCheckpt = &ethpb.Checkpoint{}
s.store.SetFinalizedCheckpt(&ethpb.Checkpoint{})
require.NoError(t, s.checkSaveHotStateDB(context.Background()))
s.genesisTime = time.Now()
@@ -321,11 +315,11 @@ func TestCheckSaveHotStateDB_Disabling(t *testing.T) {
}
func TestCheckSaveHotStateDB_Overflow(t *testing.T) {
beaconDB := testDB.SetupDB(t)
hook := logTest.NewGlobal()
s, err := NewService(context.Background(), &Config{StateGen: stategen.New(beaconDB)})
opts := testServiceOptsWithDB(t)
s, err := NewService(context.Background(), opts...)
require.NoError(t, err)
s.finalizedCheckpt = &ethpb.Checkpoint{Epoch: 10000000}
s.store.SetFinalizedCheckpt(&ethpb.Checkpoint{Epoch: 10000000})
s.genesisTime = time.Now()
require.NoError(t, s.checkSaveHotStateDB(context.Background()))

View File

@@ -1,6 +1,5 @@
// Package blockchain defines the life-cycle of the blockchain at the core of
// eth2, including processing of new blocks and attestations using casper
// proof of stake.
// Ethereum, including processing of new blocks and attestations using proof of stake.
package blockchain
import (
@@ -12,13 +11,14 @@ import (
"github.com/pkg/errors"
types "github.com/prysmaticlabs/eth2-types"
ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
"github.com/prysmaticlabs/prysm/async/event"
"github.com/prysmaticlabs/prysm/beacon-chain/blockchain/store"
"github.com/prysmaticlabs/prysm/beacon-chain/cache"
"github.com/prysmaticlabs/prysm/beacon-chain/cache/depositcache"
"github.com/prysmaticlabs/prysm/beacon-chain/core/feed"
statefeed "github.com/prysmaticlabs/prysm/beacon-chain/core/feed/state"
"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
"github.com/prysmaticlabs/prysm/beacon-chain/core/state"
"github.com/prysmaticlabs/prysm/beacon-chain/core/transition"
"github.com/prysmaticlabs/prysm/beacon-chain/db"
f "github.com/prysmaticlabs/prysm/beacon-chain/forkchoice"
"github.com/prysmaticlabs/prysm/beacon-chain/forkchoice/protoarray"
@@ -27,13 +27,15 @@ import (
"github.com/prysmaticlabs/prysm/beacon-chain/operations/voluntaryexits"
"github.com/prysmaticlabs/prysm/beacon-chain/p2p"
"github.com/prysmaticlabs/prysm/beacon-chain/powchain"
iface "github.com/prysmaticlabs/prysm/beacon-chain/state/interface"
"github.com/prysmaticlabs/prysm/beacon-chain/state/stateV0"
"github.com/prysmaticlabs/prysm/beacon-chain/state"
"github.com/prysmaticlabs/prysm/beacon-chain/state/stategen"
"github.com/prysmaticlabs/prysm/cmd/beacon-chain/flags"
"github.com/prysmaticlabs/prysm/shared/bytesutil"
"github.com/prysmaticlabs/prysm/shared/params"
"github.com/prysmaticlabs/prysm/shared/slotutil"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/block"
prysmTime "github.com/prysmaticlabs/prysm/time"
"github.com/prysmaticlabs/prysm/time/slots"
"github.com/sirupsen/logrus"
"go.opencensus.io/trace"
)
@@ -45,30 +47,26 @@ const headSyncMinEpochsAfterCheckpoint = 128
// Service represents a service that handles the internal
// logic of managing the full PoS beacon chain.
type Service struct {
cfg *Config
ctx context.Context
cancel context.CancelFunc
genesisTime time.Time
head *head
headLock sync.RWMutex
genesisRoot [32]byte
justifiedCheckpt *ethpb.Checkpoint
prevJustifiedCheckpt *ethpb.Checkpoint
bestJustifiedCheckpt *ethpb.Checkpoint
finalizedCheckpt *ethpb.Checkpoint
prevFinalizedCheckpt *ethpb.Checkpoint
cfg *config
ctx context.Context
cancel context.CancelFunc
genesisTime time.Time
head *head
headLock sync.RWMutex
// originBlockRoot is the genesis root, or weak subjectivity checkpoint root, depending on how the node is initialized
originBlockRoot [32]byte
nextEpochBoundarySlot types.Slot
boundaryRoots [][32]byte
checkpointStateCache *cache.CheckpointStateCache
initSyncBlocks map[[32]byte]*ethpb.SignedBeaconBlock
initSyncBlocks map[[32]byte]block.SignedBeaconBlock
initSyncBlocksLock sync.RWMutex
justifiedBalances []uint64
justifiedBalancesLock sync.RWMutex
wsVerified bool
justifiedBalances *stateBalanceCache
wsVerifier *WeakSubjectivityVerifier
store *store.Store
}
// Config options for the service.
type Config struct {
// config options for the service.
type config struct {
BeaconBlockBuf int
ChainStartFetcher powchain.ChainStartFetcher
BeaconDB db.HeadAccessDatabase
@@ -80,229 +78,58 @@ type Config struct {
MaxRoutines int
StateNotifier statefeed.Notifier
ForkChoiceStore f.ForkChoicer
OpsService *attestations.Service
AttService *attestations.Service
StateGen *stategen.State
SlasherAttestationsFeed *event.Feed
WeakSubjectivityCheckpt *ethpb.Checkpoint
FinalizedStateAtStartUp state.BeaconState
}
// NewService instantiates a new block service instance that will
// be registered into a running beacon node.
func NewService(ctx context.Context, cfg *Config) (*Service, error) {
func NewService(ctx context.Context, opts ...Option) (*Service, error) {
ctx, cancel := context.WithCancel(ctx)
return &Service{
cfg: cfg,
srv := &Service{
ctx: ctx,
cancel: cancel,
boundaryRoots: [][32]byte{},
checkpointStateCache: cache.NewCheckpointStateCache(),
initSyncBlocks: make(map[[32]byte]*ethpb.SignedBeaconBlock),
justifiedBalances: make([]uint64, 0),
}, nil
initSyncBlocks: make(map[[32]byte]block.SignedBeaconBlock),
cfg: &config{},
store: &store.Store{},
}
for _, opt := range opts {
if err := opt(srv); err != nil {
return nil, err
}
}
var err error
if srv.justifiedBalances == nil {
srv.justifiedBalances, err = newStateBalanceCache(srv.cfg.StateGen)
if err != nil {
return nil, err
}
}
srv.wsVerifier, err = NewWeakSubjectivityVerifier(srv.cfg.WeakSubjectivityCheckpt, srv.cfg.BeaconDB)
if err != nil {
return nil, err
}
return srv, nil
}
// Start a blockchain service's main event loop.
func (s *Service) Start() {
// For running initial sync with state cache, in an event of restart, we use
// last finalized check point as start point to sync instead of head
// state. This is because we no longer save state every slot during sync.
cp, err := s.cfg.BeaconDB.FinalizedCheckpoint(s.ctx)
if err != nil {
log.Fatalf("Could not fetch finalized cp: %v", err)
}
saved := s.cfg.FinalizedStateAtStartUp
r := bytesutil.ToBytes32(cp.Root)
// Before the first finalized epoch, in the current epoch,
// the finalized root is defined as zero hashes instead of genesis root hash.
// We want to use genesis root to retrieve for state.
if r == params.BeaconConfig().ZeroHash {
genesisBlock, err := s.cfg.BeaconDB.GenesisBlock(s.ctx)
if err != nil {
log.Fatalf("Could not fetch finalized cp: %v", err)
if saved != nil && !saved.IsNil() {
if err := s.startFromSavedState(saved); err != nil {
log.Fatal(err)
}
if genesisBlock != nil {
r, err = genesisBlock.Block.HashTreeRoot()
if err != nil {
log.Fatalf("Could not tree hash genesis block: %v", err)
}
}
}
beaconState, err := s.cfg.StateGen.StateByRoot(s.ctx, r)
if err != nil {
log.Fatalf("Could not fetch beacon state by root: %v", err)
}
// Make sure that attestation processor is subscribed and ready for state initializing event.
attestationProcessorSubscribed := make(chan struct{}, 1)
// If the chain has already been initialized, simply start the block processing routine.
if beaconState != nil {
log.Info("Blockchain data already exists in DB, initializing...")
s.genesisTime = time.Unix(int64(beaconState.GenesisTime()), 0)
s.cfg.OpsService.SetGenesisTime(beaconState.GenesisTime())
if err := s.initializeChainInfo(s.ctx); err != nil {
log.Fatalf("Could not set up chain info: %v", err)
}
// We start a counter to genesis, if needed.
gState, err := s.cfg.BeaconDB.GenesisState(s.ctx)
if err != nil {
log.Fatalf("Could not retrieve genesis state: %v", err)
}
gRoot, err := gState.HashTreeRoot(s.ctx)
if err != nil {
log.Fatalf("Could not hash tree root genesis state: %v", err)
}
go slotutil.CountdownToGenesis(s.ctx, s.genesisTime, uint64(gState.NumValidators()), gRoot)
justifiedCheckpoint, err := s.cfg.BeaconDB.JustifiedCheckpoint(s.ctx)
if err != nil {
log.Fatalf("Could not get justified checkpoint: %v", err)
}
finalizedCheckpoint, err := s.cfg.BeaconDB.FinalizedCheckpoint(s.ctx)
if err != nil {
log.Fatalf("Could not get finalized checkpoint: %v", err)
}
// Resume fork choice.
s.justifiedCheckpt = stateV0.CopyCheckpoint(justifiedCheckpoint)
if err := s.cacheJustifiedStateBalances(s.ctx, s.ensureRootNotZeros(bytesutil.ToBytes32(s.justifiedCheckpt.Root))); err != nil {
log.Fatalf("Could not cache justified state balances: %v", err)
}
s.prevJustifiedCheckpt = stateV0.CopyCheckpoint(justifiedCheckpoint)
s.bestJustifiedCheckpt = stateV0.CopyCheckpoint(justifiedCheckpoint)
s.finalizedCheckpt = stateV0.CopyCheckpoint(finalizedCheckpoint)
s.prevFinalizedCheckpt = stateV0.CopyCheckpoint(finalizedCheckpoint)
s.resumeForkChoice(justifiedCheckpoint, finalizedCheckpoint)
ss, err := helpers.StartSlot(s.finalizedCheckpt.Epoch)
if err != nil {
log.Fatalf("Could not get start slot of finalized epoch: %v", err)
}
h := s.headBlock().Block
if h.Slot > ss {
log.WithFields(logrus.Fields{
"startSlot": ss,
"endSlot": h.Slot,
}).Info("Loading blocks to fork choice store, this may take a while.")
if err := s.fillInForkChoiceMissingBlocks(s.ctx, h, s.finalizedCheckpt, s.justifiedCheckpt); err != nil {
log.Fatalf("Could not fill in fork choice store missing blocks: %v", err)
}
}
if err := s.VerifyWeakSubjectivityRoot(s.ctx); err != nil {
// Exit run time if the node failed to verify weak subjectivity checkpoint.
log.Fatalf("Could not verify weak subjectivity checkpoint: %v", err)
}
s.cfg.StateNotifier.StateFeed().Send(&feed.Event{
Type: statefeed.Initialized,
Data: &statefeed.InitializedData{
StartTime: s.genesisTime,
GenesisValidatorsRoot: beaconState.GenesisValidatorRoot(),
},
})
} else {
log.Info("Waiting to reach the validator deposit threshold to start the beacon chain...")
if s.cfg.ChainStartFetcher == nil {
log.Fatal("Not configured web3Service for POW chain")
return // return need for TestStartUninitializedChainWithoutConfigPOWChain.
if err := s.startFromPOWChain(); err != nil {
log.Fatal(err)
}
go func() {
stateChannel := make(chan *feed.Event, 1)
stateSub := s.cfg.StateNotifier.StateFeed().Subscribe(stateChannel)
defer stateSub.Unsubscribe()
<-attestationProcessorSubscribed
for {
select {
case event := <-stateChannel:
if event.Type == statefeed.ChainStarted {
data, ok := event.Data.(*statefeed.ChainStartedData)
if !ok {
log.Error("event data is not type *statefeed.ChainStartedData")
return
}
log.WithField("starttime", data.StartTime).Debug("Received chain start event")
s.processChainStartTime(s.ctx, data.StartTime)
return
}
case <-s.ctx.Done():
log.Debug("Context closed, exiting goroutine")
return
case err := <-stateSub.Err():
log.WithError(err).Error("Subscription to state notifier failed")
return
}
}
}()
}
go s.processAttestationsRoutine(attestationProcessorSubscribed)
}
// processChainStartTime initializes a series of deposits from the ChainStart deposits in the eth1
// deposit contract, initializes the beacon chain's state, and kicks off the beacon chain.
func (s *Service) processChainStartTime(ctx context.Context, genesisTime time.Time) {
preGenesisState := s.cfg.ChainStartFetcher.PreGenesisState()
initializedState, err := s.initializeBeaconChain(ctx, genesisTime, preGenesisState, s.cfg.ChainStartFetcher.ChainStartEth1Data())
if err != nil {
log.Fatalf("Could not initialize beacon chain: %v", err)
}
// We start a counter to genesis, if needed.
gRoot, err := initializedState.HashTreeRoot(s.ctx)
if err != nil {
log.Fatalf("Could not hash tree root genesis state: %v", err)
}
go slotutil.CountdownToGenesis(ctx, genesisTime, uint64(initializedState.NumValidators()), gRoot)
// We send out a state initialized event to the rest of the services
// running in the beacon node.
s.cfg.StateNotifier.StateFeed().Send(&feed.Event{
Type: statefeed.Initialized,
Data: &statefeed.InitializedData{
StartTime: genesisTime,
GenesisValidatorsRoot: initializedState.GenesisValidatorRoot(),
},
})
}
// initializes the state and genesis block of the beacon chain to persistent storage
// based on a genesis timestamp value obtained from the ChainStart event emitted
// by the ETH1.0 Deposit Contract and the POWChain service of the node.
func (s *Service) initializeBeaconChain(
ctx context.Context,
genesisTime time.Time,
preGenesisState iface.BeaconState,
eth1data *ethpb.Eth1Data) (iface.BeaconState, error) {
ctx, span := trace.StartSpan(ctx, "beacon-chain.Service.initializeBeaconChain")
defer span.End()
s.genesisTime = genesisTime
unixTime := uint64(genesisTime.Unix())
genesisState, err := state.OptimizedGenesisBeaconState(unixTime, preGenesisState, eth1data)
if err != nil {
return nil, errors.Wrap(err, "could not initialize genesis state")
}
if err := s.saveGenesisData(ctx, genesisState); err != nil {
return nil, errors.Wrap(err, "could not save genesis data")
}
log.Info("Initialized beacon chain genesis state")
// Clear out all pre-genesis data now that the state is initialized.
s.cfg.ChainStartFetcher.ClearPreGenesisData()
// Update committee shuffled indices for genesis epoch.
if err := helpers.UpdateCommitteeCache(genesisState, 0 /* genesis epoch */); err != nil {
return nil, err
}
if err := helpers.UpdateProposerIndicesInCache(genesisState); err != nil {
return nil, err
}
s.cfg.OpsService.SetGenesisTime(genesisState.GenesisTime())
return genesisState, nil
}
// Stop the blockchain service's main event loop and associated goroutines.
@@ -322,7 +149,7 @@ func (s *Service) Stop() error {
// Status always returns nil unless there is an error condition that causes
// this service to be unhealthy.
func (s *Service) Status() error {
if s.genesisRoot == params.BeaconConfig().ZeroHash {
if s.originBlockRoot == params.BeaconConfig().ZeroHash {
return errors.New("genesis state has not been created")
}
if runtime.NumGoroutine() > s.cfg.MaxRoutines {
@@ -331,64 +158,104 @@ func (s *Service) Status() error {
return nil
}
// This gets called when beacon chain is first initialized to save genesis data (state, block, and more) in db.
func (s *Service) saveGenesisData(ctx context.Context, genesisState iface.BeaconState) error {
if err := s.cfg.BeaconDB.SaveGenesisData(ctx, genesisState); err != nil {
return errors.Wrap(err, "could not save genesis data")
}
genesisBlk, err := s.cfg.BeaconDB.GenesisBlock(ctx)
if err != nil || genesisBlk == nil {
return fmt.Errorf("could not load genesis block: %v", err)
}
genesisBlkRoot, err := genesisBlk.Block.HashTreeRoot()
func (s *Service) startFromSavedState(saved state.BeaconState) error {
log.Info("Blockchain data already exists in DB, initializing...")
s.genesisTime = time.Unix(int64(saved.GenesisTime()), 0)
s.cfg.AttService.SetGenesisTime(saved.GenesisTime())
originRoot, err := s.originRootFromSavedState(s.ctx)
if err != nil {
return errors.Wrap(err, "could not get genesis block root")
}
s.genesisRoot = genesisBlkRoot
s.cfg.StateGen.SaveFinalizedState(0 /*slot*/, genesisBlkRoot, genesisState)
// Finalized checkpoint at genesis is a zero hash.
genesisCheckpoint := genesisState.FinalizedCheckpoint()
s.justifiedCheckpt = stateV0.CopyCheckpoint(genesisCheckpoint)
if err := s.cacheJustifiedStateBalances(ctx, genesisBlkRoot); err != nil {
return err
}
s.prevJustifiedCheckpt = stateV0.CopyCheckpoint(genesisCheckpoint)
s.bestJustifiedCheckpt = stateV0.CopyCheckpoint(genesisCheckpoint)
s.finalizedCheckpt = stateV0.CopyCheckpoint(genesisCheckpoint)
s.prevFinalizedCheckpt = stateV0.CopyCheckpoint(genesisCheckpoint)
s.originBlockRoot = originRoot
if err := s.cfg.ForkChoiceStore.ProcessBlock(ctx,
genesisBlk.Block.Slot,
genesisBlkRoot,
params.BeaconConfig().ZeroHash,
[32]byte{},
genesisCheckpoint.Epoch,
genesisCheckpoint.Epoch); err != nil {
log.Fatalf("Could not process genesis block for fork choice: %v", err)
if err := s.initializeHeadFromDB(s.ctx); err != nil {
return errors.Wrap(err, "could not set up chain info")
}
spawnCountdownIfPreGenesis(s.ctx, s.genesisTime, s.cfg.BeaconDB)
justified, err := s.cfg.BeaconDB.JustifiedCheckpoint(s.ctx)
if err != nil {
return errors.Wrap(err, "could not get justified checkpoint")
}
finalized, err := s.cfg.BeaconDB.FinalizedCheckpoint(s.ctx)
if err != nil {
return errors.Wrap(err, "could not get finalized checkpoint")
}
s.store = store.New(justified, finalized)
store := protoarray.New(justified.Epoch, finalized.Epoch, bytesutil.ToBytes32(finalized.Root))
s.cfg.ForkChoiceStore = store
if err := s.loadSyncedTips(originRoot, saved.Slot()); err != nil {
return err
}
s.setHead(genesisBlkRoot, genesisBlk, genesisState)
ss, err := slots.EpochStart(finalized.Epoch)
if err != nil {
return errors.Wrap(err, "could not get start slot of finalized epoch")
}
h := s.headBlock().Block()
if h.Slot() > ss {
log.WithFields(logrus.Fields{
"startSlot": ss,
"endSlot": h.Slot(),
}).Info("Loading blocks to fork choice store, this may take a while.")
if err := s.fillInForkChoiceMissingBlocks(s.ctx, h, finalized, justified); err != nil {
return errors.Wrap(err, "could not fill in fork choice store missing blocks")
}
}
// not attempting to save initial sync blocks here, because there shouldn't be any until
// after the statefeed.Initialized event is fired (below)
if err := s.wsVerifier.VerifyWeakSubjectivity(s.ctx, finalized.Epoch); err != nil {
// Exit run time if the node failed to verify weak subjectivity checkpoint.
return errors.Wrap(err, "could not verify initial checkpoint provided for chain sync")
}
s.cfg.StateNotifier.StateFeed().Send(&feed.Event{
Type: statefeed.Initialized,
Data: &statefeed.InitializedData{
StartTime: s.genesisTime,
GenesisValidatorsRoot: saved.GenesisValidatorRoot(),
},
})
s.spawnProcessAttestationsRoutine(s.cfg.StateNotifier.StateFeed())
return nil
}
// This gets called to initialize chain info variables using the finalized checkpoint stored in DB
func (s *Service) initializeChainInfo(ctx context.Context) error {
func (s *Service) originRootFromSavedState(ctx context.Context) ([32]byte, error) {
// first check if we have started from checkpoint sync and have a root
originRoot, err := s.cfg.BeaconDB.OriginBlockRoot(ctx)
if err == nil {
return originRoot, nil
}
if !errors.Is(err, db.ErrNotFound) {
return originRoot, errors.Wrap(err, "could not retrieve checkpoint sync chain origin data from db")
}
// we got here because OriginBlockRoot gave us an ErrNotFound. this means the node was started from a genesis state,
// so we should have a value for GenesisBlock
genesisBlock, err := s.cfg.BeaconDB.GenesisBlock(ctx)
if err != nil {
return errors.Wrap(err, "could not get genesis block from db")
return originRoot, errors.Wrap(err, "could not get genesis block from db")
}
if genesisBlock == nil {
return errors.New("no genesis block in db")
if err := helpers.BeaconBlockIsNil(genesisBlock); err != nil {
return originRoot, err
}
genesisBlkRoot, err := genesisBlock.Block.HashTreeRoot()
genesisBlkRoot, err := genesisBlock.Block().HashTreeRoot()
if err != nil {
return errors.Wrap(err, "could not get signing root of genesis block")
return genesisBlkRoot, errors.Wrap(err, "could not get signing root of genesis block")
}
s.genesisRoot = genesisBlkRoot
return genesisBlkRoot, nil
}
// initializeHeadFromDB uses the finalized checkpoint and head block found in the database to set the current head
// note that this may block until stategen replays blocks between the finalized and head blocks
// if the head sync flag was specified and the gap between the finalized and head blocks is at least 128 epochs long
func (s *Service) initializeHeadFromDB(ctx context.Context) error {
finalized, err := s.cfg.BeaconDB.FinalizedCheckpoint(ctx)
if err != nil {
return errors.Wrap(err, "could not get finalized checkpoint from db")
@@ -399,9 +266,9 @@ func (s *Service) initializeChainInfo(ctx context.Context) error {
return errors.New("no finalized epoch in the database")
}
finalizedRoot := s.ensureRootNotZeros(bytesutil.ToBytes32(finalized.Root))
var finalizedState iface.BeaconState
var finalizedState state.BeaconState
finalizedState, err = s.cfg.StateGen.Resume(ctx)
finalizedState, err = s.cfg.StateGen.Resume(ctx, s.cfg.FinalizedStateAtStartUp)
if err != nil {
return errors.Wrap(err, "could not get finalized state from db")
}
@@ -411,7 +278,7 @@ func (s *Service) initializeChainInfo(ctx context.Context) error {
if err != nil {
return errors.Wrap(err, "could not retrieve head block")
}
headEpoch := helpers.SlotToEpoch(headBlock.Block.Slot)
headEpoch := slots.ToEpoch(headBlock.Block().Slot())
var epochsSinceFinality types.Epoch
if headEpoch > finalized.Epoch {
epochsSinceFinality = headEpoch - finalized.Epoch
@@ -419,16 +286,16 @@ func (s *Service) initializeChainInfo(ctx context.Context) error {
// Head sync when node is far enough beyond known finalized epoch,
// this becomes really useful during long period of non-finality.
if epochsSinceFinality >= headSyncMinEpochsAfterCheckpoint {
headRoot, err := headBlock.Block.HashTreeRoot()
headRoot, err := headBlock.Block().HashTreeRoot()
if err != nil {
return errors.Wrap(err, "could not hash head block")
}
finalizedState, err := s.cfg.StateGen.Resume(ctx)
finalizedState, err := s.cfg.StateGen.Resume(ctx, s.cfg.FinalizedStateAtStartUp)
if err != nil {
return errors.Wrap(err, "could not get finalized state from db")
}
log.Infof("Regenerating state from the last checkpoint at slot %d to current head slot of %d."+
"This process may take a while, please wait.", finalizedState.Slot(), headBlock.Block.Slot)
"This process may take a while, please wait.", finalizedState.Slot(), headBlock.Block().Slot())
headState, err := s.cfg.StateGen.StateByRoot(ctx, headRoot)
if err != nil {
return errors.Wrap(err, "could not retrieve head state")
@@ -447,7 +314,7 @@ func (s *Service) initializeChainInfo(ctx context.Context) error {
return errors.Wrap(err, "could not get finalized block from db")
}
if finalizedState == nil || finalizedBlock == nil {
if finalizedState == nil || finalizedState.IsNil() || finalizedBlock == nil || finalizedBlock.IsNil() {
return errors.New("finalized state and block can't be nil")
}
s.setHead(finalizedRoot, finalizedBlock, finalizedState)
@@ -455,11 +322,141 @@ func (s *Service) initializeChainInfo(ctx context.Context) error {
return nil
}
// This is called when a client starts from non-genesis slot. This passes last justified and finalized
// information to fork choice service to initializes fork choice store.
func (s *Service) resumeForkChoice(justifiedCheckpoint, finalizedCheckpoint *ethpb.Checkpoint) {
store := protoarray.New(justifiedCheckpoint.Epoch, finalizedCheckpoint.Epoch, bytesutil.ToBytes32(finalizedCheckpoint.Root))
s.cfg.ForkChoiceStore = store
func (s *Service) startFromPOWChain() error {
log.Info("Waiting to reach the validator deposit threshold to start the beacon chain...")
if s.cfg.ChainStartFetcher == nil {
return errors.New("not configured web3Service for POW chain")
}
go func() {
stateChannel := make(chan *feed.Event, 1)
stateSub := s.cfg.StateNotifier.StateFeed().Subscribe(stateChannel)
defer stateSub.Unsubscribe()
s.spawnProcessAttestationsRoutine(s.cfg.StateNotifier.StateFeed())
for {
select {
case event := <-stateChannel:
if event.Type == statefeed.ChainStarted {
data, ok := event.Data.(*statefeed.ChainStartedData)
if !ok {
log.Error("event data is not type *statefeed.ChainStartedData")
return
}
log.WithField("starttime", data.StartTime).Debug("Received chain start event")
s.onPowchainStart(s.ctx, data.StartTime)
return
}
case <-s.ctx.Done():
log.Debug("Context closed, exiting goroutine")
return
case err := <-stateSub.Err():
log.WithError(err).Error("Subscription to state notifier failed")
return
}
}
}()
return nil
}
// onPowchainStart initializes a series of deposits from the ChainStart deposits in the eth1
// deposit contract, initializes the beacon chain's state, and kicks off the beacon chain.
func (s *Service) onPowchainStart(ctx context.Context, genesisTime time.Time) {
preGenesisState := s.cfg.ChainStartFetcher.PreGenesisState()
initializedState, err := s.initializeBeaconChain(ctx, genesisTime, preGenesisState, s.cfg.ChainStartFetcher.ChainStartEth1Data())
if err != nil {
log.Fatalf("Could not initialize beacon chain: %v", err)
}
// We start a counter to genesis, if needed.
gRoot, err := initializedState.HashTreeRoot(s.ctx)
if err != nil {
log.Fatalf("Could not hash tree root genesis state: %v", err)
}
go slots.CountdownToGenesis(ctx, genesisTime, uint64(initializedState.NumValidators()), gRoot)
// We send out a state initialized event to the rest of the services
// running in the beacon node.
s.cfg.StateNotifier.StateFeed().Send(&feed.Event{
Type: statefeed.Initialized,
Data: &statefeed.InitializedData{
StartTime: genesisTime,
GenesisValidatorsRoot: initializedState.GenesisValidatorRoot(),
},
})
}
// initializes the state and genesis block of the beacon chain to persistent storage
// based on a genesis timestamp value obtained from the ChainStart event emitted
// by the ETH1.0 Deposit Contract and the POWChain service of the node.
func (s *Service) initializeBeaconChain(
ctx context.Context,
genesisTime time.Time,
preGenesisState state.BeaconState,
eth1data *ethpb.Eth1Data) (state.BeaconState, error) {
ctx, span := trace.StartSpan(ctx, "beacon-chain.Service.initializeBeaconChain")
defer span.End()
s.genesisTime = genesisTime
unixTime := uint64(genesisTime.Unix())
genesisState, err := transition.OptimizedGenesisBeaconState(unixTime, preGenesisState, eth1data)
if err != nil {
return nil, errors.Wrap(err, "could not initialize genesis state")
}
if err := s.saveGenesisData(ctx, genesisState); err != nil {
return nil, errors.Wrap(err, "could not save genesis data")
}
log.Info("Initialized beacon chain genesis state")
// Clear out all pre-genesis data now that the state is initialized.
s.cfg.ChainStartFetcher.ClearPreGenesisData()
// Update committee shuffled indices for genesis epoch.
if err := helpers.UpdateCommitteeCache(genesisState, 0 /* genesis epoch */); err != nil {
return nil, err
}
if err := helpers.UpdateProposerIndicesInCache(ctx, genesisState); err != nil {
return nil, err
}
s.cfg.AttService.SetGenesisTime(genesisState.GenesisTime())
return genesisState, nil
}
// This gets called when beacon chain is first initialized to save genesis data (state, block, and more) in db.
func (s *Service) saveGenesisData(ctx context.Context, genesisState state.BeaconState) error {
if err := s.cfg.BeaconDB.SaveGenesisData(ctx, genesisState); err != nil {
return errors.Wrap(err, "could not save genesis data")
}
genesisBlk, err := s.cfg.BeaconDB.GenesisBlock(ctx)
if err != nil || genesisBlk == nil || genesisBlk.IsNil() {
return fmt.Errorf("could not load genesis block: %v", err)
}
genesisBlkRoot, err := genesisBlk.Block().HashTreeRoot()
if err != nil {
return errors.Wrap(err, "could not get genesis block root")
}
s.originBlockRoot = genesisBlkRoot
s.cfg.StateGen.SaveFinalizedState(0 /*slot*/, genesisBlkRoot, genesisState)
// Finalized checkpoint at genesis is a zero hash.
genesisCheckpoint := genesisState.FinalizedCheckpoint()
s.store = store.New(genesisCheckpoint, genesisCheckpoint)
if err := s.cfg.ForkChoiceStore.ProcessBlock(ctx,
genesisBlk.Block().Slot(),
genesisBlkRoot,
params.BeaconConfig().ZeroHash,
[32]byte{},
genesisCheckpoint.Epoch,
genesisCheckpoint.Epoch); err != nil {
log.Fatalf("Could not process genesis block for fork choice: %v", err)
}
s.setHead(genesisBlkRoot, genesisBlk, genesisState)
return nil
}
// This returns true if block has been processed before. Two ways to verify the block has been processed:
@@ -473,3 +470,20 @@ func (s *Service) hasBlock(ctx context.Context, root [32]byte) bool {
return s.cfg.BeaconDB.HasBlock(ctx, root)
}
func spawnCountdownIfPreGenesis(ctx context.Context, genesisTime time.Time, db db.HeadAccessDatabase) {
currentTime := prysmTime.Now()
if currentTime.After(genesisTime) {
return
}
gState, err := db.GenesisState(ctx)
if err != nil {
log.Fatalf("Could not retrieve genesis state: %v", err)
}
gRoot, err := gState.HashTreeRoot(ctx)
if err != nil {
log.Fatalf("Could not hash tree root genesis state: %v", err)
}
go slots.CountdownToGenesis(ctx, genesisTime, uint64(gState.NumValidators()), gRoot)
}

View File

@@ -6,7 +6,7 @@ import (
"testing"
testDB "github.com/prysmaticlabs/prysm/beacon-chain/db/testing"
"github.com/prysmaticlabs/prysm/shared/testutil/require"
"github.com/prysmaticlabs/prysm/testing/require"
"github.com/sirupsen/logrus"
)
@@ -18,7 +18,7 @@ func init() {
func TestChainService_SaveHead_DataRace(t *testing.T) {
beaconDB := testDB.SetupDB(t)
s := &Service{
cfg: &Config{BeaconDB: beaconDB},
cfg: &config{BeaconDB: beaconDB},
}
go func() {
require.NoError(t, s.saveHead(context.Background(), [32]byte{}))

View File

@@ -8,32 +8,36 @@ import (
"time"
"github.com/ethereum/go-ethereum/common"
"github.com/gogo/protobuf/proto"
ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
types "github.com/prysmaticlabs/eth2-types"
"github.com/prysmaticlabs/prysm/async/event"
"github.com/prysmaticlabs/prysm/beacon-chain/blockchain/store"
mock "github.com/prysmaticlabs/prysm/beacon-chain/blockchain/testing"
"github.com/prysmaticlabs/prysm/beacon-chain/cache/depositcache"
b "github.com/prysmaticlabs/prysm/beacon-chain/core/blocks"
"github.com/prysmaticlabs/prysm/beacon-chain/core/feed"
statefeed "github.com/prysmaticlabs/prysm/beacon-chain/core/feed/state"
"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
"github.com/prysmaticlabs/prysm/beacon-chain/core/state"
"github.com/prysmaticlabs/prysm/beacon-chain/core/transition"
"github.com/prysmaticlabs/prysm/beacon-chain/db"
testDB "github.com/prysmaticlabs/prysm/beacon-chain/db/testing"
"github.com/prysmaticlabs/prysm/beacon-chain/forkchoice/protoarray"
"github.com/prysmaticlabs/prysm/beacon-chain/operations/attestations"
"github.com/prysmaticlabs/prysm/beacon-chain/p2p"
"github.com/prysmaticlabs/prysm/beacon-chain/powchain"
"github.com/prysmaticlabs/prysm/beacon-chain/state/stateV0"
"github.com/prysmaticlabs/prysm/beacon-chain/state/stategen"
v1 "github.com/prysmaticlabs/prysm/beacon-chain/state/v1"
"github.com/prysmaticlabs/prysm/cmd/beacon-chain/flags"
protodb "github.com/prysmaticlabs/prysm/proto/beacon/db"
pb "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
"github.com/prysmaticlabs/prysm/shared/bytesutil"
"github.com/prysmaticlabs/prysm/shared/event"
"github.com/prysmaticlabs/prysm/shared/params"
"github.com/prysmaticlabs/prysm/shared/testutil"
"github.com/prysmaticlabs/prysm/shared/testutil/assert"
"github.com/prysmaticlabs/prysm/shared/testutil/require"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/block"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/wrapper"
"github.com/prysmaticlabs/prysm/testing/assert"
"github.com/prysmaticlabs/prysm/testing/require"
"github.com/prysmaticlabs/prysm/testing/util"
"github.com/prysmaticlabs/prysm/time/slots"
logTest "github.com/sirupsen/logrus/hooks/test"
"google.golang.org/protobuf/proto"
)
type mockBeaconNode struct {
@@ -62,6 +66,11 @@ func (mb *mockBroadcaster) BroadcastAttestation(_ context.Context, _ uint64, _ *
return nil
}
func (mb *mockBroadcaster) BroadcastSyncCommitteeMessage(_ context.Context, _ uint64, _ *ethpb.SyncCommitteeMessage) error {
mb.broadcastCalled = true
return nil
}
var _ p2p.Broadcaster = (*mockBroadcaster)(nil)
func setupBeaconChain(t *testing.T, beaconDB db.Database) *Service {
@@ -69,55 +78,56 @@ func setupBeaconChain(t *testing.T, beaconDB db.Database) *Service {
ctx := context.Background()
var web3Service *powchain.Service
var err error
bState, _ := testutil.DeterministicGenesisState(t, 10)
pbState, err := stateV0.ProtobufBeaconState(bState.InnerStateUnsafe())
bState, _ := util.DeterministicGenesisState(t, 10)
pbState, err := v1.ProtobufBeaconState(bState.InnerStateUnsafe())
require.NoError(t, err)
err = beaconDB.SavePowchainData(ctx, &protodb.ETH1ChainData{
err = beaconDB.SavePowchainData(ctx, &ethpb.ETH1ChainData{
BeaconState: pbState,
Trie: &protodb.SparseMerkleTrie{},
CurrentEth1Data: &protodb.LatestETH1Data{
Trie: &ethpb.SparseMerkleTrie{},
CurrentEth1Data: &ethpb.LatestETH1Data{
BlockHash: make([]byte, 32),
},
ChainstartData: &protodb.ChainStartData{
ChainstartData: &ethpb.ChainStartData{
Eth1Data: &ethpb.Eth1Data{
DepositRoot: make([]byte, 32),
DepositCount: 0,
BlockHash: make([]byte, 32),
},
},
DepositContainers: []*protodb.DepositContainer{},
DepositContainers: []*ethpb.DepositContainer{},
})
require.NoError(t, err)
web3Service, err = powchain.NewService(ctx, &powchain.Web3ServiceConfig{
BeaconDB: beaconDB,
HttpEndpoints: []string{endpoint},
DepositContract: common.Address{},
})
web3Service, err = powchain.NewService(
ctx,
powchain.WithDatabase(beaconDB),
powchain.WithHttpEndpoints([]string{endpoint}),
powchain.WithDepositContractAddress(common.Address{}),
)
require.NoError(t, err, "Unable to set up web3 service")
opsService, err := attestations.NewService(ctx, &attestations.Config{Pool: attestations.NewPool()})
attService, err := attestations.NewService(ctx, &attestations.Config{Pool: attestations.NewPool()})
require.NoError(t, err)
depositCache, err := depositcache.New()
require.NoError(t, err)
cfg := &Config{
BeaconBlockBuf: 0,
BeaconDB: beaconDB,
DepositCache: depositCache,
ChainStartFetcher: web3Service,
P2p: &mockBroadcaster{},
StateNotifier: &mockBeaconNode{},
AttPool: attestations.NewPool(),
StateGen: stategen.New(beaconDB),
ForkChoiceStore: protoarray.New(0, 0, params.BeaconConfig().ZeroHash),
OpsService: opsService,
stateGen := stategen.New(beaconDB)
// Safe a state in stategen to purposes of testing a service stop / shutdown.
require.NoError(t, stateGen.SaveState(ctx, bytesutil.ToBytes32(bState.FinalizedCheckpoint().Root), bState))
opts := []Option{
WithDatabase(beaconDB),
WithDepositCache(depositCache),
WithChainStartFetcher(web3Service),
WithAttestationPool(attestations.NewPool()),
WithP2PBroadcaster(&mockBroadcaster{}),
WithStateNotifier(&mockBeaconNode{}),
WithForkChoiceStore(protoarray.New(0, 0, params.BeaconConfig().ZeroHash)),
WithAttestationService(attService),
WithStateGen(stateGen),
}
// Safe a state in stategen to purposes of testing a service stop / shutdown.
require.NoError(t, cfg.StateGen.SaveState(ctx, bytesutil.ToBytes32(bState.FinalizedCheckpoint().Root), bState))
chainService, err := NewService(ctx, cfg)
chainService, err := NewService(ctx, opts...)
require.NoError(t, err, "Unable to setup chain service")
chainService.genesisTime = time.Unix(1, 0) // non-zero time
@@ -131,11 +141,11 @@ func TestChainStartStop_Initialized(t *testing.T) {
chainService := setupBeaconChain(t, beaconDB)
genesisBlk := testutil.NewBeaconBlock()
genesisBlk := util.NewBeaconBlock()
blkRoot, err := genesisBlk.Block.HashTreeRoot()
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, genesisBlk))
s, err := testutil.NewBeaconState()
require.NoError(t, beaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(genesisBlk)))
s, err := util.NewBeaconState()
require.NoError(t, err)
require.NoError(t, s.SetSlot(1))
require.NoError(t, beaconDB.SaveState(ctx, s, blkRoot))
@@ -143,7 +153,7 @@ func TestChainStartStop_Initialized(t *testing.T) {
require.NoError(t, beaconDB.SaveGenesisBlockRoot(ctx, blkRoot))
require.NoError(t, beaconDB.SaveJustifiedCheckpoint(ctx, &ethpb.Checkpoint{Root: blkRoot[:]}))
require.NoError(t, beaconDB.SaveFinalizedCheckpoint(ctx, &ethpb.Checkpoint{Root: blkRoot[:]}))
chainService.cfg.FinalizedStateAtStartUp = s
// Test the start function.
chainService.Start()
@@ -154,6 +164,74 @@ func TestChainStartStop_Initialized(t *testing.T) {
require.LogsContain(t, hook, "data already exists")
}
func TestChainStart_SyncedTipsInDB(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
chainService := setupBeaconChain(t, beaconDB)
genesisBlk := util.NewBeaconBlock()
blkRoot, err := genesisBlk.Block.HashTreeRoot()
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(genesisBlk)))
s, err := util.NewBeaconState()
require.NoError(t, err)
require.NoError(t, s.SetSlot(1))
require.NoError(t, beaconDB.SaveState(ctx, s, blkRoot))
require.NoError(t, beaconDB.SaveHeadBlockRoot(ctx, blkRoot))
require.NoError(t, beaconDB.SaveGenesisBlockRoot(ctx, blkRoot))
require.NoError(t, beaconDB.SaveJustifiedCheckpoint(ctx, &ethpb.Checkpoint{Root: blkRoot[:]}))
require.NoError(t, beaconDB.SaveFinalizedCheckpoint(ctx, &ethpb.Checkpoint{Root: blkRoot[:]}))
chainService.cfg.FinalizedStateAtStartUp = s
tips := make(map[[32]byte]types.Slot)
tips[bytesutil.ToBytes32([]byte{'a'})] = 1
tips[bytesutil.ToBytes32([]byte{'b'})] = 2
require.NoError(t, beaconDB.UpdateValidatedTips(ctx, tips))
// Test the start function.
chainService.Start()
// Test synced Tips in DB
tips2 := chainService.cfg.ForkChoiceStore.SyncedTips()
require.Equal(t, len(tips2), len(tips))
for k, v := range tips {
v2, ok := tips2[k]
require.Equal(t, true, ok)
require.Equal(t, v, v2)
}
}
func TestChainStart_SyncedTipsNotInDB(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
chainService := setupBeaconChain(t, beaconDB)
genesisBlk := util.NewBeaconBlock()
blkRoot, err := genesisBlk.Block.HashTreeRoot()
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(genesisBlk)))
s, err := util.NewBeaconState()
require.NoError(t, err)
require.NoError(t, s.SetSlot(1))
require.NoError(t, beaconDB.SaveState(ctx, s, blkRoot))
require.NoError(t, beaconDB.SaveHeadBlockRoot(ctx, blkRoot))
require.NoError(t, beaconDB.SaveGenesisBlockRoot(ctx, blkRoot))
require.NoError(t, beaconDB.SaveJustifiedCheckpoint(ctx, &ethpb.Checkpoint{Root: blkRoot[:]}))
require.NoError(t, beaconDB.SaveFinalizedCheckpoint(ctx, &ethpb.Checkpoint{Root: blkRoot[:]}))
chainService.cfg.FinalizedStateAtStartUp = s
// Test the start function.
chainService.Start()
// Test synced Tips in DB
tips := chainService.cfg.ForkChoiceStore.SyncedTips()
require.Equal(t, 1, len(tips))
slot, ok := tips[blkRoot]
require.Equal(t, true, ok)
require.Equal(t, types.Slot(1), slot)
}
func TestChainStartStop_GenesisZeroHashes(t *testing.T) {
hook := logTest.NewGlobal()
ctx := context.Background()
@@ -161,16 +239,16 @@ func TestChainStartStop_GenesisZeroHashes(t *testing.T) {
chainService := setupBeaconChain(t, beaconDB)
genesisBlk := testutil.NewBeaconBlock()
genesisBlk := util.NewBeaconBlock()
blkRoot, err := genesisBlk.Block.HashTreeRoot()
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, genesisBlk))
s, err := testutil.NewBeaconState()
require.NoError(t, beaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(genesisBlk)))
s, err := util.NewBeaconState()
require.NoError(t, err)
require.NoError(t, beaconDB.SaveState(ctx, s, blkRoot))
require.NoError(t, beaconDB.SaveGenesisBlockRoot(ctx, blkRoot))
require.NoError(t, beaconDB.SaveJustifiedCheckpoint(ctx, &ethpb.Checkpoint{Root: params.BeaconConfig().ZeroHash[:]}))
chainService.cfg.FinalizedStateAtStartUp = s
// Test the start function.
chainService.Start()
@@ -191,12 +269,12 @@ func TestChainService_InitializeBeaconChain(t *testing.T) {
// Set up 10 deposits pre chain start for validators to register
count := uint64(10)
deposits, _, err := testutil.DeterministicDepositsAndKeys(count)
deposits, _, err := util.DeterministicDepositsAndKeys(count)
require.NoError(t, err)
trie, _, err := testutil.DepositTrieFromDeposits(deposits)
trie, _, err := util.DepositTrieFromDeposits(deposits)
require.NoError(t, err)
hashTreeRoot := trie.HashTreeRoot()
genState, err := state.EmptyGenesisState()
genState, err := transition.EmptyGenesisState()
require.NoError(t, err)
err = genState.SetEth1Data(&ethpb.Eth1Data{
DepositRoot: hashTreeRoot[:],
@@ -230,23 +308,23 @@ func TestChainService_CorrectGenesisRoots(t *testing.T) {
chainService := setupBeaconChain(t, beaconDB)
genesisBlk := testutil.NewBeaconBlock()
genesisBlk := util.NewBeaconBlock()
blkRoot, err := genesisBlk.Block.HashTreeRoot()
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, genesisBlk))
s, err := testutil.NewBeaconState()
require.NoError(t, beaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(genesisBlk)))
s, err := util.NewBeaconState()
require.NoError(t, err)
require.NoError(t, s.SetSlot(0))
require.NoError(t, beaconDB.SaveState(ctx, s, blkRoot))
require.NoError(t, beaconDB.SaveHeadBlockRoot(ctx, blkRoot))
require.NoError(t, beaconDB.SaveGenesisBlockRoot(ctx, blkRoot))
require.NoError(t, beaconDB.SaveFinalizedCheckpoint(ctx, &ethpb.Checkpoint{Root: blkRoot[:]}))
chainService.cfg.FinalizedStateAtStartUp = s
// Test the start function.
chainService.Start()
require.DeepEqual(t, blkRoot[:], chainService.finalizedCheckpt.Root, "Finalize Checkpoint root is incorrect")
require.DeepEqual(t, params.BeaconConfig().ZeroHash[:], chainService.justifiedCheckpt.Root, "Justified Checkpoint root is incorrect")
require.DeepEqual(t, blkRoot[:], chainService.store.FinalizedCheckpt().Root, "Finalize Checkpoint root is incorrect")
require.DeepEqual(t, params.BeaconConfig().ZeroHash[:], chainService.store.JustifiedCheckpt().Root, "Justified Checkpoint root is incorrect")
require.NoError(t, chainService.Stop(), "Unable to stop chain service")
@@ -256,17 +334,17 @@ func TestChainService_InitializeChainInfo(t *testing.T) {
beaconDB := testDB.SetupDB(t)
ctx := context.Background()
genesis := testutil.NewBeaconBlock()
genesis := util.NewBeaconBlock()
genesisRoot, err := genesis.Block.HashTreeRoot()
require.NoError(t, err)
require.NoError(t, beaconDB.SaveGenesisBlockRoot(ctx, genesisRoot))
require.NoError(t, beaconDB.SaveBlock(ctx, genesis))
require.NoError(t, beaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(genesis)))
finalizedSlot := params.BeaconConfig().SlotsPerEpoch*2 + 1
headBlock := testutil.NewBeaconBlock()
headBlock := util.NewBeaconBlock()
headBlock.Block.Slot = finalizedSlot
headBlock.Block.ParentRoot = bytesutil.PadTo(genesisRoot[:], 32)
headState, err := testutil.NewBeaconState()
headState, err := util.NewBeaconState()
require.NoError(t, err)
require.NoError(t, headState.SetSlot(finalizedSlot))
require.NoError(t, headState.SetGenesisValidatorRoot(params.BeaconConfig().ZeroHash[:]))
@@ -274,13 +352,16 @@ func TestChainService_InitializeChainInfo(t *testing.T) {
require.NoError(t, err)
require.NoError(t, beaconDB.SaveState(ctx, headState, headRoot))
require.NoError(t, beaconDB.SaveState(ctx, headState, genesisRoot))
require.NoError(t, beaconDB.SaveBlock(ctx, headBlock))
require.NoError(t, beaconDB.SaveFinalizedCheckpoint(ctx, &ethpb.Checkpoint{Epoch: helpers.SlotToEpoch(finalizedSlot), Root: headRoot[:]}))
c := &Service{cfg: &Config{BeaconDB: beaconDB, StateGen: stategen.New(beaconDB)}}
require.NoError(t, c.initializeChainInfo(ctx))
require.NoError(t, beaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(headBlock)))
require.NoError(t, beaconDB.SaveFinalizedCheckpoint(ctx, &ethpb.Checkpoint{Epoch: slots.ToEpoch(finalizedSlot), Root: headRoot[:]}))
attSrv, err := attestations.NewService(ctx, &attestations.Config{})
require.NoError(t, err)
c, err := NewService(ctx, WithDatabase(beaconDB), WithStateGen(stategen.New(beaconDB)), WithAttestationService(attSrv), WithStateNotifier(&mock.MockStateNotifier{}), WithFinalizedStateAtStartUp(headState))
require.NoError(t, err)
require.NoError(t, c.startFromSavedState(headState))
headBlk, err := c.HeadBlock(ctx)
require.NoError(t, err)
assert.DeepEqual(t, headBlock, headBlk, "Head block incorrect")
assert.DeepEqual(t, headBlock, headBlk.Proto(), "Head block incorrect")
s, err := c.HeadState(ctx)
require.NoError(t, err)
assert.DeepSSZEqual(t, headState.InnerStateUnsafe(), s.InnerStateUnsafe(), "Head state incorrect")
@@ -290,24 +371,24 @@ func TestChainService_InitializeChainInfo(t *testing.T) {
if !bytes.Equal(headRoot[:], r) {
t.Error("head slot incorrect")
}
assert.Equal(t, genesisRoot, c.genesisRoot, "Genesis block root incorrect")
assert.Equal(t, genesisRoot, c.originBlockRoot, "Genesis block root incorrect")
}
func TestChainService_InitializeChainInfo_SetHeadAtGenesis(t *testing.T) {
beaconDB := testDB.SetupDB(t)
ctx := context.Background()
genesis := testutil.NewBeaconBlock()
genesis := util.NewBeaconBlock()
genesisRoot, err := genesis.Block.HashTreeRoot()
require.NoError(t, err)
require.NoError(t, beaconDB.SaveGenesisBlockRoot(ctx, genesisRoot))
require.NoError(t, beaconDB.SaveBlock(ctx, genesis))
require.NoError(t, beaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(genesis)))
finalizedSlot := params.BeaconConfig().SlotsPerEpoch*2 + 1
headBlock := testutil.NewBeaconBlock()
headBlock := util.NewBeaconBlock()
headBlock.Block.Slot = finalizedSlot
headBlock.Block.ParentRoot = bytesutil.PadTo(genesisRoot[:], 32)
headState, err := testutil.NewBeaconState()
headState, err := util.NewBeaconState()
require.NoError(t, err)
require.NoError(t, headState.SetSlot(finalizedSlot))
require.NoError(t, headState.SetGenesisValidatorRoot(params.BeaconConfig().ZeroHash[:]))
@@ -315,14 +396,17 @@ func TestChainService_InitializeChainInfo_SetHeadAtGenesis(t *testing.T) {
require.NoError(t, err)
require.NoError(t, beaconDB.SaveState(ctx, headState, headRoot))
require.NoError(t, beaconDB.SaveState(ctx, headState, genesisRoot))
require.NoError(t, beaconDB.SaveBlock(ctx, headBlock))
c := &Service{cfg: &Config{BeaconDB: beaconDB, StateGen: stategen.New(beaconDB)}}
require.NoError(t, c.initializeChainInfo(ctx))
require.NoError(t, beaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(headBlock)))
attSrv, err := attestations.NewService(ctx, &attestations.Config{})
require.NoError(t, err)
c, err := NewService(ctx, WithDatabase(beaconDB), WithStateGen(stategen.New(beaconDB)), WithAttestationService(attSrv), WithStateNotifier(&mock.MockStateNotifier{}))
require.NoError(t, err)
require.NoError(t, c.startFromSavedState(headState))
s, err := c.HeadState(ctx)
require.NoError(t, err)
assert.DeepSSZEqual(t, headState.InnerStateUnsafe(), s.InnerStateUnsafe(), "Head state incorrect")
assert.Equal(t, genesisRoot, c.genesisRoot, "Genesis block root incorrect")
assert.DeepEqual(t, genesis, c.head.block)
assert.Equal(t, genesisRoot, c.originBlockRoot, "Genesis block root incorrect")
assert.DeepEqual(t, genesis, c.head.block.Proto())
}
func TestChainService_InitializeChainInfo_HeadSync(t *testing.T) {
@@ -339,28 +423,28 @@ func TestChainService_InitializeChainInfo_HeadSync(t *testing.T) {
beaconDB := testDB.SetupDB(t)
ctx := context.Background()
genesisBlock := testutil.NewBeaconBlock()
genesisBlock := util.NewBeaconBlock()
genesisRoot, err := genesisBlock.Block.HashTreeRoot()
require.NoError(t, err)
require.NoError(t, beaconDB.SaveGenesisBlockRoot(ctx, genesisRoot))
require.NoError(t, beaconDB.SaveBlock(ctx, genesisBlock))
require.NoError(t, beaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(genesisBlock)))
finalizedBlock := testutil.NewBeaconBlock()
finalizedBlock := util.NewBeaconBlock()
finalizedBlock.Block.Slot = finalizedSlot
finalizedBlock.Block.ParentRoot = genesisRoot[:]
finalizedRoot, err := finalizedBlock.Block.HashTreeRoot()
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, finalizedBlock))
require.NoError(t, beaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(finalizedBlock)))
// Set head slot close to the finalization point, no head sync is triggered.
headBlock := testutil.NewBeaconBlock()
headBlock := util.NewBeaconBlock()
headBlock.Block.Slot = finalizedSlot + params.BeaconConfig().SlotsPerEpoch*5
headBlock.Block.ParentRoot = finalizedRoot[:]
headRoot, err := headBlock.Block.HashTreeRoot()
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, headBlock))
require.NoError(t, beaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(headBlock)))
headState, err := testutil.NewBeaconState()
headState, err := util.NewBeaconState()
require.NoError(t, err)
require.NoError(t, headState.SetSlot(headBlock.Block.Slot))
require.NoError(t, headState.SetGenesisValidatorRoot(params.BeaconConfig().ZeroHash[:]))
@@ -369,40 +453,42 @@ func TestChainService_InitializeChainInfo_HeadSync(t *testing.T) {
require.NoError(t, beaconDB.SaveState(ctx, headState, headRoot))
require.NoError(t, beaconDB.SaveHeadBlockRoot(ctx, headRoot))
require.NoError(t, beaconDB.SaveFinalizedCheckpoint(ctx, &ethpb.Checkpoint{
Epoch: helpers.SlotToEpoch(finalizedBlock.Block.Slot),
Epoch: slots.ToEpoch(finalizedBlock.Block.Slot),
Root: finalizedRoot[:],
}))
c := &Service{cfg: &Config{BeaconDB: beaconDB, StateGen: stategen.New(beaconDB)}}
require.NoError(t, c.initializeChainInfo(ctx))
attSrv, err := attestations.NewService(ctx, &attestations.Config{})
require.NoError(t, err)
c, err := NewService(ctx, WithDatabase(beaconDB), WithStateGen(stategen.New(beaconDB)), WithAttestationService(attSrv), WithStateNotifier(&mock.MockStateNotifier{}), WithFinalizedStateAtStartUp(headState))
require.NoError(t, err)
require.NoError(t, c.startFromSavedState(headState))
s, err := c.HeadState(ctx)
require.NoError(t, err)
assert.DeepSSZEqual(t, headState.InnerStateUnsafe(), s.InnerStateUnsafe(), "Head state incorrect")
assert.Equal(t, genesisRoot, c.genesisRoot, "Genesis block root incorrect")
assert.Equal(t, genesisRoot, c.originBlockRoot, "Genesis block root incorrect")
// Since head sync is not triggered, chain is initialized to the last finalization checkpoint.
assert.DeepEqual(t, finalizedBlock, c.head.block)
assert.DeepEqual(t, finalizedBlock, c.head.block.Proto())
assert.LogsContain(t, hook, "resetting head from the checkpoint ('--head-sync' flag is ignored)")
assert.LogsDoNotContain(t, hook, "Regenerating state from the last checkpoint at slot")
// Set head slot far beyond the finalization point, head sync should be triggered.
headBlock = testutil.NewBeaconBlock()
headBlock = util.NewBeaconBlock()
headBlock.Block.Slot = finalizedSlot + params.BeaconConfig().SlotsPerEpoch*headSyncMinEpochsAfterCheckpoint
headBlock.Block.ParentRoot = finalizedRoot[:]
headRoot, err = headBlock.Block.HashTreeRoot()
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, headBlock))
require.NoError(t, beaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(headBlock)))
require.NoError(t, beaconDB.SaveState(ctx, headState, headRoot))
require.NoError(t, beaconDB.SaveHeadBlockRoot(ctx, headRoot))
hook.Reset()
require.NoError(t, c.initializeChainInfo(ctx))
require.NoError(t, c.initializeHeadFromDB(ctx))
s, err = c.HeadState(ctx)
require.NoError(t, err)
assert.DeepSSZEqual(t, headState.InnerStateUnsafe(), s.InnerStateUnsafe(), "Head state incorrect")
assert.Equal(t, genesisRoot, c.genesisRoot, "Genesis block root incorrect")
assert.Equal(t, genesisRoot, c.originBlockRoot, "Genesis block root incorrect")
// Head slot is far beyond the latest finalized checkpoint, head sync is triggered.
assert.DeepEqual(t, headBlock, c.head.block)
assert.DeepEqual(t, headBlock, c.head.block.Proto())
assert.LogsContain(t, hook, "Regenerating state from the last checkpoint at slot 225")
assert.LogsDoNotContain(t, hook, "resetting head from the checkpoint ('--head-sync' flag is ignored)")
}
@@ -411,16 +497,16 @@ func TestChainService_SaveHeadNoDB(t *testing.T) {
beaconDB := testDB.SetupDB(t)
ctx := context.Background()
s := &Service{
cfg: &Config{BeaconDB: beaconDB, StateGen: stategen.New(beaconDB)},
cfg: &config{BeaconDB: beaconDB, StateGen: stategen.New(beaconDB)},
}
blk := testutil.NewBeaconBlock()
blk := util.NewBeaconBlock()
blk.Block.Slot = 1
r, err := blk.HashTreeRoot()
require.NoError(t, err)
newState, err := testutil.NewBeaconState()
newState, err := util.NewBeaconState()
require.NoError(t, err)
require.NoError(t, s.cfg.StateGen.SaveState(ctx, r, newState))
require.NoError(t, s.saveHeadNoDB(ctx, blk, r, newState))
require.NoError(t, s.saveHeadNoDB(ctx, wrapper.WrappedPhase0SignedBeaconBlock(blk), r, newState))
newB, err := s.cfg.BeaconDB.HeadBlock(ctx)
require.NoError(t, err)
@@ -433,15 +519,16 @@ func TestHasBlock_ForkChoiceAndDB(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
s := &Service{
cfg: &Config{ForkChoiceStore: protoarray.New(0, 0, [32]byte{}), BeaconDB: beaconDB},
finalizedCheckpt: &ethpb.Checkpoint{Root: make([]byte, 32)},
cfg: &config{ForkChoiceStore: protoarray.New(0, 0, [32]byte{}), BeaconDB: beaconDB},
store: &store.Store{},
}
block := testutil.NewBeaconBlock()
s.store.SetFinalizedCheckpt(&ethpb.Checkpoint{Epoch: 0, Root: params.BeaconConfig().ZeroHash[:]})
block := util.NewBeaconBlock()
r, err := block.Block.HashTreeRoot()
require.NoError(t, err)
beaconState, err := testutil.NewBeaconState()
beaconState, err := util.NewBeaconState()
require.NoError(t, err)
require.NoError(t, s.insertBlockAndAttestationsToForkChoiceStore(ctx, block.Block, r, beaconState))
require.NoError(t, s.insertBlockAndAttestationsToForkChoiceStore(ctx, wrapper.WrappedPhase0SignedBeaconBlock(block).Block(), r, beaconState))
assert.Equal(t, false, s.hasBlock(ctx, [32]byte{}), "Should not have block")
assert.Equal(t, true, s.hasBlock(ctx, r), "Should have block")
@@ -451,15 +538,15 @@ func TestServiceStop_SaveCachedBlocks(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
beaconDB := testDB.SetupDB(t)
s := &Service{
cfg: &Config{BeaconDB: beaconDB, StateGen: stategen.New(beaconDB)},
cfg: &config{BeaconDB: beaconDB, StateGen: stategen.New(beaconDB)},
ctx: ctx,
cancel: cancel,
initSyncBlocks: make(map[[32]byte]*ethpb.SignedBeaconBlock),
initSyncBlocks: make(map[[32]byte]block.SignedBeaconBlock),
}
b := testutil.NewBeaconBlock()
r, err := b.Block.HashTreeRoot()
block := util.NewBeaconBlock()
r, err := block.Block.HashTreeRoot()
require.NoError(t, err)
s.saveInitSyncBlock(r, b)
s.saveInitSyncBlock(r, wrapper.WrappedPhase0SignedBeaconBlock(block))
require.NoError(t, s.Stop())
require.Equal(t, true, s.cfg.BeaconDB.HasBlock(ctx, r))
}
@@ -470,7 +557,7 @@ func TestProcessChainStartTime_ReceivedFeed(t *testing.T) {
stateChannel := make(chan *feed.Event, 1)
stateSub := service.cfg.StateNotifier.StateFeed().Subscribe(stateChannel)
defer stateSub.Unsubscribe()
service.processChainStartTime(context.Background(), time.Now())
service.onPowchainStart(context.Background(), time.Now())
stateEvent := <-stateChannel
require.Equal(t, int(stateEvent.Type), statefeed.Initialized)
@@ -482,10 +569,10 @@ func BenchmarkHasBlockDB(b *testing.B) {
beaconDB := testDB.SetupDB(b)
ctx := context.Background()
s := &Service{
cfg: &Config{BeaconDB: beaconDB},
cfg: &config{BeaconDB: beaconDB},
}
block := testutil.NewBeaconBlock()
require.NoError(b, s.cfg.BeaconDB.SaveBlock(ctx, block))
block := util.NewBeaconBlock()
require.NoError(b, s.cfg.BeaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(block)))
r, err := block.Block.HashTreeRoot()
require.NoError(b, err)
@@ -499,16 +586,17 @@ func BenchmarkHasBlockForkChoiceStore(b *testing.B) {
ctx := context.Background()
beaconDB := testDB.SetupDB(b)
s := &Service{
cfg: &Config{ForkChoiceStore: protoarray.New(0, 0, [32]byte{}), BeaconDB: beaconDB},
finalizedCheckpt: &ethpb.Checkpoint{Root: make([]byte, 32)},
cfg: &config{ForkChoiceStore: protoarray.New(0, 0, [32]byte{}), BeaconDB: beaconDB},
store: &store.Store{},
}
s.store.SetFinalizedCheckpt(&ethpb.Checkpoint{Epoch: 0, Root: params.BeaconConfig().ZeroHash[:]})
block := &ethpb.SignedBeaconBlock{Block: &ethpb.BeaconBlock{Body: &ethpb.BeaconBlockBody{}}}
r, err := block.Block.HashTreeRoot()
require.NoError(b, err)
bs := &pb.BeaconState{FinalizedCheckpoint: &ethpb.Checkpoint{Root: make([]byte, 32)}, CurrentJustifiedCheckpoint: &ethpb.Checkpoint{Root: make([]byte, 32)}}
beaconState, err := stateV0.InitializeFromProto(bs)
bs := &ethpb.BeaconState{FinalizedCheckpoint: &ethpb.Checkpoint{Root: make([]byte, 32)}, CurrentJustifiedCheckpoint: &ethpb.Checkpoint{Root: make([]byte, 32)}}
beaconState, err := v1.InitializeFromProto(bs)
require.NoError(b, err)
require.NoError(b, s.insertBlockAndAttestationsToForkChoiceStore(ctx, block.Block, r, beaconState))
require.NoError(b, s.insertBlockAndAttestationsToForkChoiceStore(ctx, wrapper.WrappedPhase0SignedBeaconBlock(block).Block(), r, beaconState))
b.ResetTimer()
for i := 0; i < b.N; i++ {

View File

@@ -0,0 +1,83 @@
package blockchain
import (
"context"
"errors"
"sync"
"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
"github.com/prysmaticlabs/prysm/beacon-chain/core/time"
"github.com/prysmaticlabs/prysm/beacon-chain/state"
"github.com/prysmaticlabs/prysm/beacon-chain/state/stategen"
)
var errNilStateFromStategen = errors.New("justified state can't be nil")
type stateBalanceCache struct {
sync.Mutex
balances []uint64
root [32]byte
stateGen stateByRooter
}
type stateByRooter interface {
StateByRoot(context.Context, [32]byte) (state.BeaconState, error)
}
// newStateBalanceCache exists to remind us that stateBalanceCache needs a stagegen
// to avoid nil pointer bugs when updating the cache in the read path (get())
func newStateBalanceCache(sg *stategen.State) (*stateBalanceCache, error) {
if sg == nil {
return nil, errors.New("can't initialize state balance cache without stategen")
}
return &stateBalanceCache{stateGen: sg}, nil
}
// update is called by get() when the requested root doesn't match
// the previously read value. This cache assumes we only want to cache one
// set of balances for a single root (the current justified root).
//
// warning: this is not thread-safe on its own, relies on get() for locking
func (c *stateBalanceCache) update(ctx context.Context, justifiedRoot [32]byte) ([]uint64, error) {
stateBalanceCacheMiss.Inc()
justifiedState, err := c.stateGen.StateByRoot(ctx, justifiedRoot)
if err != nil {
return nil, err
}
if justifiedState == nil || justifiedState.IsNil() {
return nil, errNilStateFromStategen
}
epoch := time.CurrentEpoch(justifiedState)
justifiedBalances := make([]uint64, justifiedState.NumValidators())
var balanceAccumulator = func(idx int, val state.ReadOnlyValidator) error {
if helpers.IsActiveValidatorUsingTrie(val, epoch) {
justifiedBalances[idx] = val.EffectiveBalance()
} else {
justifiedBalances[idx] = 0
}
return nil
}
if err := justifiedState.ReadFromEveryValidator(balanceAccumulator); err != nil {
return nil, err
}
c.balances = justifiedBalances
c.root = justifiedRoot
return c.balances, nil
}
// getBalances takes an explicit justifiedRoot so it can invalidate the singleton cache key
// when the justified root changes, and takes a context so that the long-running stategen
// read path can connect to the upstream cancellation/timeout chain.
func (c *stateBalanceCache) get(ctx context.Context, justifiedRoot [32]byte) ([]uint64, error) {
c.Lock()
defer c.Unlock()
if justifiedRoot != [32]byte{} && justifiedRoot == c.root {
stateBalanceCacheHit.Inc()
return c.balances, nil
}
return c.update(ctx, justifiedRoot)
}

View File

@@ -0,0 +1,236 @@
package blockchain
import (
"context"
"encoding/binary"
"errors"
"testing"
types "github.com/prysmaticlabs/eth2-types"
"github.com/prysmaticlabs/prysm/beacon-chain/state"
v2 "github.com/prysmaticlabs/prysm/beacon-chain/state/v2"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/testing/require"
"github.com/prysmaticlabs/prysm/time/slots"
)
type mockStateByRoot struct {
state state.BeaconState
err error
}
func (m *mockStateByRoot) StateByRoot(context.Context, [32]byte) (state.BeaconState, error) {
return m.state, m.err
}
type testStateOpt func(*ethpb.BeaconStateAltair)
func testStateWithValidators(v []*ethpb.Validator) testStateOpt {
return func(a *ethpb.BeaconStateAltair) {
a.Validators = v
}
}
func testStateWithSlot(slot types.Slot) testStateOpt {
return func(a *ethpb.BeaconStateAltair) {
a.Slot = slot
}
}
func testStateFixture(opts ...testStateOpt) state.BeaconState {
a := &ethpb.BeaconStateAltair{}
for _, o := range opts {
o(a)
}
s, _ := v2.InitializeFromProtoUnsafe(a)
return s
}
func generateTestValidators(count int, opts ...func(*ethpb.Validator)) []*ethpb.Validator {
vs := make([]*ethpb.Validator, count)
var i uint32 = 0
for ; i < uint32(count); i++ {
pk := make([]byte, 48)
binary.LittleEndian.PutUint32(pk, i)
v := &ethpb.Validator{PublicKey: pk}
for _, o := range opts {
o(v)
}
vs[i] = v
}
return vs
}
func oddValidatorsExpired(currentSlot types.Slot) func(*ethpb.Validator) {
return func(v *ethpb.Validator) {
pki := binary.LittleEndian.Uint64(v.PublicKey)
if pki%2 == 0 {
v.ExitEpoch = types.Epoch(int(slots.ToEpoch(currentSlot)) + 1)
} else {
v.ExitEpoch = types.Epoch(int(slots.ToEpoch(currentSlot)) - 1)
}
}
}
func oddValidatorsQueued(currentSlot types.Slot) func(*ethpb.Validator) {
return func(v *ethpb.Validator) {
v.ExitEpoch = types.Epoch(int(slots.ToEpoch(currentSlot)) + 1)
pki := binary.LittleEndian.Uint64(v.PublicKey)
if pki%2 == 0 {
v.ActivationEpoch = types.Epoch(int(slots.ToEpoch(currentSlot)) - 1)
} else {
v.ActivationEpoch = types.Epoch(int(slots.ToEpoch(currentSlot)) + 1)
}
}
}
func allValidatorsValid(currentSlot types.Slot) func(*ethpb.Validator) {
return func(v *ethpb.Validator) {
v.ActivationEpoch = types.Epoch(int(slots.ToEpoch(currentSlot)) - 1)
v.ExitEpoch = types.Epoch(int(slots.ToEpoch(currentSlot)) + 1)
}
}
func balanceIsKeyTimes2(v *ethpb.Validator) {
pki := binary.LittleEndian.Uint64(v.PublicKey)
v.EffectiveBalance = uint64(pki) * 2
}
func testHalfExpiredValidators() ([]*ethpb.Validator, []uint64) {
balances := []uint64{0, 0, 4, 0, 8, 0, 12, 0, 16, 0}
return generateTestValidators(10,
oddValidatorsExpired(types.Slot(99)),
balanceIsKeyTimes2), balances
}
func testHalfQueuedValidators() ([]*ethpb.Validator, []uint64) {
balances := []uint64{0, 0, 4, 0, 8, 0, 12, 0, 16, 0}
return generateTestValidators(10,
oddValidatorsQueued(types.Slot(99)),
balanceIsKeyTimes2), balances
}
func testAllValidValidators() ([]*ethpb.Validator, []uint64) {
balances := []uint64{0, 2, 4, 6, 8, 10, 12, 14, 16, 18}
return generateTestValidators(10,
allValidatorsValid(types.Slot(99)),
balanceIsKeyTimes2), balances
}
func TestStateBalanceCache(t *testing.T) {
type sbcTestCase struct {
err error
root [32]byte
sbc *stateBalanceCache
balances []uint64
name string
}
sentinelCacheMiss := errors.New("Cache missed, as expected!")
sentinelBalances := []uint64{1, 2, 3, 4, 5}
halfExpiredValidators, halfExpiredBalances := testHalfExpiredValidators()
halfQueuedValidators, halfQueuedBalances := testHalfQueuedValidators()
allValidValidators, allValidBalances := testAllValidValidators()
cases := []sbcTestCase{
{
root: bytesutil.ToBytes32([]byte{'A'}),
balances: sentinelBalances,
sbc: &stateBalanceCache{
stateGen: &mockStateByRooter{
err: sentinelCacheMiss,
},
root: bytesutil.ToBytes32([]byte{'A'}),
balances: sentinelBalances,
},
name: "cache hit",
},
// this works by using a staterooter that returns a known error
// so really we're testing the miss by making sure stategen got called
// this also tells us stategen errors are propagated
{
sbc: &stateBalanceCache{
stateGen: &mockStateByRooter{
err: sentinelCacheMiss,
},
root: bytesutil.ToBytes32([]byte{'B'}),
},
err: sentinelCacheMiss,
root: bytesutil.ToBytes32([]byte{'A'}),
name: "cache miss",
},
{
sbc: &stateBalanceCache{
stateGen: &mockStateByRooter{},
root: bytesutil.ToBytes32([]byte{'B'}),
},
err: errNilStateFromStategen,
root: bytesutil.ToBytes32([]byte{'A'}),
name: "error for nil state upon cache miss",
},
{
sbc: &stateBalanceCache{
stateGen: &mockStateByRooter{
state: testStateFixture(
testStateWithSlot(99),
testStateWithValidators(halfExpiredValidators)),
},
},
balances: halfExpiredBalances,
root: bytesutil.ToBytes32([]byte{'A'}),
name: "test filtering by exit epoch",
},
{
sbc: &stateBalanceCache{
stateGen: &mockStateByRooter{
state: testStateFixture(
testStateWithSlot(99),
testStateWithValidators(halfQueuedValidators)),
},
},
balances: halfQueuedBalances,
root: bytesutil.ToBytes32([]byte{'A'}),
name: "test filtering by activation epoch",
},
{
sbc: &stateBalanceCache{
stateGen: &mockStateByRooter{
state: testStateFixture(
testStateWithSlot(99),
testStateWithValidators(allValidValidators)),
},
},
balances: allValidBalances,
root: bytesutil.ToBytes32([]byte{'A'}),
name: "happy path",
},
{
sbc: &stateBalanceCache{
stateGen: &mockStateByRooter{
state: testStateFixture(
testStateWithSlot(99),
testStateWithValidators(allValidValidators)),
},
},
balances: allValidBalances,
root: [32]byte{},
name: "zero root",
},
}
ctx := context.Background()
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
cache := c.sbc
cacheRootStart := cache.root
b, err := cache.get(ctx, c.root)
require.ErrorIs(t, err, c.err)
require.DeepEqual(t, c.balances, b)
if c.err != nil {
// if there was an error somewhere, the root should not have changed (unless it already matched)
require.Equal(t, cacheRootStart, cache.root)
} else {
// when successful, the cache should always end with a root matching the request
require.Equal(t, c.root, cache.root)
}
})
}
}

View File

@@ -0,0 +1,27 @@
load("@prysm//tools/go:def.bzl", "go_library", "go_test")
go_library(
name = "go_default_library",
srcs = [
"doc.go",
"new.go",
"setter_getter.go",
"type.go",
],
importpath = "github.com/prysmaticlabs/prysm/beacon-chain/blockchain/store",
visibility = ["//beacon-chain:__subpackages__"],
deps = ["//proto/prysm/v1alpha1:go_default_library"],
)
go_test(
name = "go_default_test",
srcs = [
"new_test.go",
"setter_getter_test.go",
],
embed = [":go_default_library"],
deps = [
"//proto/prysm/v1alpha1:go_default_library",
"//testing/require:go_default_library",
],
)

View File

@@ -0,0 +1,4 @@
// Package store implements the store object defined in the phase0 fork choice spec.
// It serves as a helpful middleware layer in between blockchain pkg and fork choice protoarray pkg.
// All the getters and setters are concurrent thread safe
package store

Some files were not shown because too many files have changed in this diff Show More