Compare commits

...

467 Commits

Author SHA1 Message Date
Preston Van Loon
9f33f4ed6a Add staticcheck to github actions 2022-03-30 09:55:19 -05:00
Potuz
588605ceeb Remove TODOs that were already taken care of (#10454) 2022-03-30 13:24:02 +00:00
terence tsao
59b9519284 Forkchoice spec test: set boost with deterministic timing (#10428)
* Set boost with deterministic timing

* confs

* gaz

* Update runner.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-03-29 19:48:33 +00:00
Nishant Das
d4038fb752 add rlock (#10444) 2022-03-29 06:55:41 -07:00
Raul Jordan
56ab5872bb Add Histogram Response Buckets to Engine API Methods (#10414)
* gaz

* defer payload func

* amend

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-28 22:17:31 +00:00
Raul Jordan
eb6d68a4b1 Deterministic Proposer Root Boosting in Forkchoice (#10427)
* begin refactor

* fix doubly linked tree tests

* pass

* fix up

* gaz

* buidl

* buidl more

* comment

* args check

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-28 21:34:41 +00:00
kasey
7920528ede Checkpoint Sync 4/5 - enable checkpoint sync to be used by beacon node (#10386)
* enable checkpoint sync in beacon node

* lint fix

* rm unused error

* addressing PR feedback from Radek

* consistent slice -> fixed conversion

Co-authored-by: kasey <kasey@users.noreply.github.com>
2022-03-28 21:01:55 +00:00
Raul Jordan
1af3c07ec5 Stop Checking Transition Configuration Post-Merge (#10413)
* no longer check transition config after the merge

* nil check

* better payload check

* stop checking post merge based on received blocks

* add proper test

* gaz

* fix test

* lint

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-28 17:14:02 +00:00
terence tsao
7279349ae2 Cleanup helpers in beacon-chain/core/blocks/payload.go (#10435)
* less fragile check for execution payload

* gaz

* gaz

* include field

* clean up helpers for checking merge status

* Update beacon-chain/core/blocks/payload.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

* Update payload.go

* Rename

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-03-28 15:25:49 +00:00
Potuz
2d60d04b57 Update LastValidatedCheckpoint on DB (#10412)
* Update LastValidatedCheckpoint on DB

* fix conflicts
2022-03-28 14:48:14 +00:00
Nishant Das
ef8bc97d3e more efficient (#10440) 2022-03-28 20:27:14 +08:00
Nishant Das
071f6de559 Fix Beacon API Responses For Bellatrix (#10436)
* fix bugs

* fix build

* simplify it
2022-03-26 07:34:29 +00:00
Preston Van Loon
b697463da9 wrapper: Remove deprecated fork specific wrapper methods for signed beacon blocks (#10369)
* Remove deprecated wrappers, lots of refactoring

* Revert proto/prysm/v1alpha1/validator.proto

* fix tests

* fix test

* fix conversion in e2e

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-25 23:00:44 +00:00
kasey
bfbf693660 Checkpoint Sync 3/5 - beacon node api client lib and prysmctl cli tool (#10385)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2022-03-25 17:18:03 +00:00
Radosław Kapka
ed07f4bd77 Add execution_optimistic field to API responses (#10389)
* protos

* grpc

* middleware

* event protos

* event backend

* event middleware

* fix tests

* use `IsOptimisticForRoot` with proper block instead of header

* better algorithm

* return proper error

* review

* optimistic return on bellatrix proposal

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-25 16:31:50 +00:00
terence tsao
1922416cac Use correct pre state to call new payload (#10416)
* Update todo strings

* Go fmt

* add merge specific checks when receiving a block from gossip

* Fix beacon chain build

* Interop merge beacon state

* fix finding Transactions size

* Update go commit

* Merge union debugging (#9751)

* changes test cases per ssz changes

* noisy commit, restoring pb field order codegen

* get rid of codegen garbage

* M2 works with Geth  🎉

* Fix bazel build //...

* restoring generated pb field ordering

* defensive nil check

* separate ExecutionPayload/Header from codegen

* tell bazel about this new file

* Merge: support terminal difficulty override (#9769)

* Fix finding terminal block hash calculation

* Update mainnet_config.go

* Update beacon_block.pb.go

* Various fixes to pass all spec tests for Merge (#9777)

* Proper upgrade altair to merge state

* Use uint64 for ttd

* Correctly upgrade to merge state + object mapping fixes

* Use proper receive block path for initial syncing

* Disable contract lookback

* Disable deposit contract lookback

* Go fmt

* Merge: switch from go bindings to raw rpc calls (#9803)

* Disable genesis ETH1.0 chain header logging

* Update htrutils.go

* all gossip tests passing

* Remove gas validations

* Update penalty params for Merge

* Fix gossip and tx size limits for the merge part 1

* Remove extraneous p2p condition

* Add and use

* Add and use TBH_ACTIVATION_EPOCH

* Update WORKSPACE

* Update Kintsugi engine API (#9865)

* Kintsugi ssz (#9867)

* All spec tests pass

* Update spec test shas

* Update Kintsugi consensus implementations (#9872)

* Remove secp256k1

* Remove unused merge genesis state gen tool

* Manually override nil transaction field. M2 works

* Fix bad hex conversion

* Change Gossip message size and Chunk SIze from 1 MB t0 10MB (#9860)

* change gossip size and chunk size after merge

* change ssz to accomodate both changes

* gofmt config file

* add testcase for merge MsgId

* Update beacon-chain/p2p/message_id.go

Change MB to Mib in comment

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* change function name from altairMsgID to postAltairMsgID

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Sync with develop

* Merge branch 'develop' of github.com:prysmaticlabs/prysm into kintsugi

* Update state_trie.go

* Clean up conflicts

* Fix build

* Update config to devnet1

* Fix state merge

* Handle merge test case for update balance

* Fix build

* State pkg cleanup

* Fix a bug with loading mainnet state

* Fix transactions root

* Add v2 endpoint for merge blocks (#9802)

* Add V2 blocks endpoint for merge blocks

* Update beacon-chain/rpc/apimiddleware/structs.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* go mod

* fix transactions

* Terence's comments

* add missing file

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Sync

* Go mod tidy

* change EP field names

* latest kintusgi execution api

* fix conflicts

* converting base fee to big endian format (#10018)

* ReverseByteOrder function does not mess the input

* sync with develop

* use merge gossip sizes

* correct gossip sizes this time

* visibility

* clean ups

* Sync with develop, fix payload nil check bug

* Speed up syncing, hide cosmetic errors

* Sync with develop

* Clean up after sync

* Update generate_keys.go

* sync with develop

* Update mainnet_config.go

* Clean ups

* Sync optimistically candidate blocks (#10193)

* Revert "Sync optimistically candidate blocks (#10193)"

This reverts commit f99a0419ef.

* Sync optimistically candidate blocks (#10193)

* allow optimistic sync

* Fix merge transition block validation

* Update proposer.go

* Sync with develop

* delete deprecated client, update testnet flag

* Change optimistic logic (#10194)

* Logs and err handling

* Fix build

* Clean ups

* Add back get payload

* c

* Done

* Rm uncommented

* Optimistic sync: prysm validator rpcs (#10200)

* Logs to reproduce

* Use pointers

* Use pointers

* Use pointers

* Update json_marshal_unmarshal.go

* Fix marshal

* Update json_marshal_unmarshal.go

* Log

* string total diff

* str

* marshal un

* set string

* json

* gaz

* Comment out optimistic status

* remove kiln flag here (#10269)

* Sync with devleop

* Sync with develop

* clean ups

* refactor engine calls

* Update process_block.go

* Fix deadlock, uncomment duty opt sync

* Update proposer_execution_payload.go

* Sync with develop

* Rm post state check

* Bypass eth1 data checks

* Update proposer_execution_payload.go

* Return early if ttd is not reached

* Sync with devleop

* Update process_block.go

* Update receive_block.go

* Update bzl

* Revert "Update receive_block.go"

This reverts commit 5b4a87c512.

* Fix run time

* Update go.mod

* Fix AltairCompatible to account for future state version

* Update proposer_execution_payload.go

* Handle pre state Altair with valid payload

* Handle pre state Altair with valid payload

* Log bellatrix fields

* Update log.go

* Friendly fee recipient log

* Remove extra SetOptimisticToValid

* Fix base fee per gas

* Fix notifypayload headroot

* clean up with develop branch

* Sync with devleop

* Fix

* Fix

* fix

* Fix

* fix tests

* revert some changes

* fix tests

* Update optimistic_sync_test.go

* Simplify IsExecutionEnabledUsingHeader

Co-authored-by: Zahoor Mohamed <zahoor@prysmaticlabs.com>
Co-authored-by: Zahoor Mohamed <zahoor@zahoor.in>
Co-authored-by: kasey <489222+kasey@users.noreply.github.com>
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-25 16:03:15 +00:00
Potuz
a5dc40393d Allow inner protoarray nodes to become VALID (#10418)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-25 15:35:44 +00:00
Raul Jordan
b238235210 prevent run (#10432) 2022-03-25 11:08:38 -04:00
terence tsao
73a6a85069 Ensure finalized root can't be zeros (#10422) 2022-03-24 23:55:54 +00:00
terence tsao
ac4a30588f Properly initialize replayer builder for rpc (#10426) 2022-03-24 23:12:47 +00:00
Raul Jordan
3579551f15 Use --http-web3provider for Execution Engine Connection (#10307)
* combine endpoints

* initialization in the start function instead

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-03-24 20:53:07 +00:00
Nishant Das
956e7a7563 Update Go-Ethereum (#10417)
* add changes

* fix CI

* wait here

* Fix gazelle with the correct directive for libp2p

Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-24 19:00:53 +00:00
kasey
acd7e62cfb fix issue #10420 (#10421)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2022-03-24 18:32:39 +00:00
Radosław Kapka
165f0667f0 Implement getDebugChainHeadsV2 API endpoint (#10419)
* protos

* grpc

* middleware

* update comments
2022-03-24 13:08:40 +00:00
kasey
49826ebe28 Checkpoint Sync 2/5 - API support for retrieving weak subjectivity data (#10384)
also refactor replayer code for better reuse:
- separately expose stategen's canonical block func
- CanonicalHistory in ws api, NewCanonicalBuilder
- refactor CanonicalHistory into a ReplayerBuilder

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2022-03-23 22:54:07 +00:00
terence tsao
3ad6abd9c0 Add tracings for engine api methods (#10407)
* Add engine call tracings

* Update BUILD.bazel

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-23 22:17:09 +00:00
terence tsao
c61f54176d Ignore attestations voting for the wrong finalized checkpoint (#10295)
* Ignore attestations voting for the wrong finalized checkpoint

* aggregate

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-23 21:48:26 +00:00
Potuz
0165351db3 Setup Optimistic sync when starting the service (#10397)
* Load Last Validated Checkpoint at service startup and initialize
Forkchoice

* fix spectests

* Update service_test.go

* wip

* Fix blockchain tests

* remove unnecessary summary saves

* Terence's suggestion

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-03-23 21:11:36 +00:00
Potuz
8f49167117 return invalid roots from SetOptimisticToInvalid (#10403)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-03-23 15:55:05 -03:00
Nishant Das
b4c27f64de Add Method To Modify Participation Bits (#10409)
* fix up all tests

* Update beacon-chain/state/v2/BUILD.bazel

* add tests
2022-03-23 23:18:02 +08:00
Preston Van Loon
e4ac5e74b7 RPC: Deduplicate block production logic (#10368)
* RPC: Deduplicate block production logic

* Surface error to caller

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-03-23 12:51:20 +00:00
Nishant Das
0b9b635646 Block Processing And State Transition Improvments (#10408)
* cleanup

* clean up more

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-03-23 11:57:48 +00:00
Leo Lara
c8c1d04c07 Complete coverage for in beacon chain state for getters checkpoint (#10373)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-03-23 09:58:45 +00:00
james-prysm
601987faf2 Fee Recipient : ux log fixes and prevent validator from calling beacon node if flags are not provided. (#10406)
* initial commit

* improving logs
2022-03-22 13:13:59 -05:00
james-prysm
a060d765b3 rename prepare beacon proposer to fee recipient (#10404) 2022-03-22 15:00:50 +00:00
Potuz
5550334956 set optimistic status on notify payload (#10398)
* set optimistic status on notify payload

* pre-execution blocks are never optimistic

* add unit test

* add test for lengths

* Update optimistic_sync_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-03-22 03:35:02 +00:00
Mike Neuder
002253bba3 Migrating Keymanager account list functionality into each type (#10382)
* Migrating Keymanager account list functionality into each keymanager type

* Addressing review comments

* Adding newline at end of BUILD.bazel

* bazel run //:gazelle -- fix

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-03-22 03:04:09 +00:00
Radosław Kapka
c055642f79 Return correct header root from API calls (#10401)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-22 01:56:44 +00:00
terence tsao
c7d64c03ac Insert payload hash to forkchoice store nodes (#10400)
* Insert payload hash to forkchoice store nodes

* Fix tests

* Update new_slot_test.go

* Update beacon-chain/blockchain/process_block.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-03-22 01:20:42 +00:00
james-prysm
131fb43ba7 Fee recipient flag rename for beacon node (#10402)
* initial commit

* fixing comment

* fixing bazel
2022-03-21 21:25:33 +00:00
kasey
cf0bd633f0 Checkpoint Sync 1/5 - fork/version detection and unmarshaling support (#10380)
* fork/version detection and unmarshaling support

* Update config/params/config.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update proto/detect/configfork.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* PR feedback

* move ssz initialization into the detect package

* clarify comment

* VersionForEpoch is much simpler/clearer in reverse

* simpler VersionForEpoch; build AllConfigs in init

* use fieldparams for Version

* Update proto/detect/configfork_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* remove custom ForkName type, use runtime/version

* pr cleanup

* random fix from bad gh ui suggestion; privatize

* privatize fieldSpec methods; + unit tests

* Update proto/detect/configfork.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

* fix bad github ui suggestion

* ensure unique versions for simpler config match

* fmt & adding unit test for ByState()

* table-driven unit test for ByState

* TestUnmarshalState

* OrderedSchedule -> network/forks per PR feedback

* goimports

* lint fixes

* move proto/detect -> ssz/encoding/detect

* use typeUndefined in String

* backport config tests from e2e PR

* fix config parity test; make debugging it easier

* lint

* fix fork schedule initialization

* cleanup

* fix build

* fix big ole derp

* anything for you, deep source

* goimportsss

* InitializeForkSchedule in LoadChainConfigFile

* PR feedback

Co-authored-by: kasey <kasey@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-03-21 19:43:41 +00:00
james-prysm
df8da80db8 Implement Fee Recipient: CLI flags and Calling Beacon API (#10312)
* send proposer data to config

* wip implementation with file and url based config import

* improving logic on get validator index

* fix function

* optimizing function for map and address bug

* fixing log

* update cache if it doesn't exist

* updating flags

* initial unit test scaffold

* fixing validator to call rpc call, removed temporary dependency

* adding the API calls for the runner

* fixing broken build

* fixing deepsource

* fixing interface

* fixing fatal

* fixing more deepsource issues

* adding test placeholders

* updating proposer config to add validation

* changing how if statement throws error

* removing unneeded validation, validating in a different way

* wip improving tests

* more unit test work

* fixing unit test

* fixing unit tests and edge cases

* adding unit tests and adjusting how the config is created

* fixing bazel builds

* fixing proto generation

* fixing imports

* fixing unit tests

* Update cmd/validator/flags/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* updating flags based on comments, fixing unit tests

* fixing bazel

* removing unneeded function

* fixing unit tests

* refactors and unit test fixes based on comments

* fixing bazel build

* refactor the cache out fo the fee recipient function

* adding usecase for multiple fee recipient

* refactor burn name

* Update validator/client/validator.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/validator.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/validator.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* fixing bug with validator index based on code review

* edited flag descriptions to better communicate usage

* fixing manual reference to flag name

* fixing code review comments

* fixing linting

* Update validator/client/validator.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* addressing comments and renaming functions

* fixing linting

* Update cmd/validator/flags/flags.go

* Update cmd/validator/flags/flags.go

* Update cmd/validator/flags/flags.go

* improving comments

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-03-21 13:48:02 -05:00
Raul Jordan
dc527a3c80 Remove Visibility Check Script for Bazel (#10391)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-21 17:03:52 +00:00
Potuz
94f80bd208 Call FCU on update head after process attestation (#10394)
* Call FCU on update head after process attestation

* add test

* fix test

* terence review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-03-21 01:50:08 +00:00
terence tsao
c6bfefa0bc Add notifyNewPayload to onBlockBatch (#10392) 2022-03-18 09:36:46 -07:00
Radosław Kapka
3446eaa5f2 Add missing tests to state retrieval (#10390) 2022-03-17 18:40:07 +00:00
terence tsao
193866c731 Add fcu to batch blk processing (#10371) 2022-03-17 07:35:50 -07:00
terence tsao
b90ce1b60f Bellatrix fork choice spec tests (#10381)
* Add bellatrix forkchoice spec tests

* Update service.go

* better conversion
2022-03-17 05:30:44 +00:00
terence tsao
8449d24ed0 Fix BaseFeePerGas endianess (#10376)
* Fix base fee endianess and add more test entropys

* Update json_marshal_unmarshal_test.go

* Update setters_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-16 22:12:08 +00:00
Radosław Kapka
b92b01f379 API middleware support for bellatrix in produceBlockV2 (#10374) 2022-03-16 21:44:15 +00:00
terence tsao
bcd180ee4d Remove deprecated set_optimistic_to_valid calls (#10370) 2022-03-16 21:14:13 +00:00
Raul Jordan
5f10e51a49 Add API Middleware Support for Fee Recipients Endpoint (#10377)
* add in success log

* add in info log

* uint64 for schema

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-16 20:02:26 +00:00
terence tsao
c4f03868ce Update default fee warning (#10378)
* Add fcu to batch blk processing

* Log error only if is burned

* Revert some changes

* Burn address
2022-03-16 19:26:51 +00:00
james-prysm
d55c0d74dd PrepareBeaconProposer API (#10367)
* intial commit implementing prepare beacon proposal

* removing files accidently checked in by goland

* removing files accidently checked in by goland

* Delete BUILD 2

* Delete armeabi_cc_toolchain_config 2.bzl

* Delete builtin_include_directory_paths 2

* Delete cc_toolchain_config 2.bzl

* Delete BUILD 2.bazel

* Delete cc_wrapper 2.sh

* Delete module 2.modulemap

* Delete BUILD 2

* Delete BUILD 2

* Delete WORKSPACE 2

* Delete cc_toolchain_config_linux_arm64 2.bzl

* Delete cc_toolchain_config_osx 2.bzl

* Delete cc_toolchain_config_windows 2.bzl

* Delete 0258716f94e00f9df0da869fd97f9e0d0c6ac83eb528677d918f0ac9be5f4b8d 2

* Delete test-unmarshal-bad 2.json

* Delete test-unmarshal-good 2.json

* adding simple unit tests

* removing duplicate imports

* adding bazel changes

* adding validation and switched to table driven tests

* updating based on comments

* adding to test retrieve from db

* fixing bazel
2022-03-16 11:17:41 -04:00
Potuz
807b71244b Add database methods for optimistic sync (#10357)
* Add database methods for optimistic sync

* Add epoch comparison

* add extra epoch comparison

* Summary instead of block

* fix tests

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-03-15 20:52:59 +00:00
Potuz
2744eba391 Update optimisticCandidateBlock to specs version (#10355)
* Update optimisticCandidateBlock to specs version

* fix double import

* Change error message
2022-03-15 19:12:52 +00:00
Leo Lara
5507558678 Some improvements for the state package testing (#10316)
* Refactor to unify state getters block tests

* Add reference tests to Altair and Bellatrix versions ofthe state

* Fix function naming convetion

* Add state-native/v2/references_test.go and state-native/v3/references_test.go

* Gazelle run

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-03-15 18:32:09 +00:00
terence tsao
8cecd4e8bf Log bellatrix fields (#10365)
* Log bellatrix fields

* Update log.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-15 14:02:39 +00:00
terence tsao
69a9388515 Spec test v1.1.10 (#10298)
* spectest

* uncomment shas

* Fc spec test service shouldn't not spawn attestation routine

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-03-15 13:17:51 +00:00
terence tsao
bd308b6d73 Update default fee recipient error log (#10366) 2022-03-14 20:35:56 -07:00
Radosław Kapka
9abea200a5 Add Bellatrix version to ProduceBlock API (#10361)
* Add Bellatrix version to `ProduceBlock` API

* fix errors

* fix test

* Revert "Auxiliary commit to revert individual files from 8cc6fc9ca3b6ddb67aeafeaddc2a2e1ca6610ded"

This reverts commit 6358ee92d899e8ee6587ad5503cebb40ac92add4.

# Conflicts:
#	beacon-chain/blockchain/optimistic_sync_test.go
#	beacon-chain/blockchain/pow_block_test.go
#	beacon-chain/blockchain/testing/mock_engine.go

* use proper mock

* bzl

* fix build file

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-14 23:08:01 +00:00
terence tsao
730678bf21 Handle pre state Altair with valid payload (#10364)
* Handle pre state Altair with valid payload

* Handle pre state Altair with valid payload
2022-03-14 22:40:08 +00:00
kasey
0b1a777d62 ran gofmt @ v1.17, these are the resulting changes (#10362)
* ran gofmt @ v1.17, these are the resulting changes

* fix some flaky tests that are tripping up this PR

* gofmt

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2022-03-14 15:58:13 -05:00
terence tsao
5461c5b84f Insert stored fee recipient for ForkchoiceUpdated call (#10349)
* Add default

* Add and use fee recipient in db

* Update BUILD.bazel

* Feedback

* Add basic addr check

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-14 18:44:40 +00:00
Preston Van Loon
03e7acbf9f increase bazel java max mem to 4gb (#10363)
* increase bazel java max mem to 4gb

* DEBUG: set nocache and modify nogo to force any disk cache misses. Revert this commit after testing in CI

* Revert "DEBUG: set nocache and modify nogo to force any disk cache misses. Revert this commit after testing in CI"

This reverts commit dc41cce43d.
2022-03-14 17:53:42 +00:00
Potuz
6cc88e6454 enable optimistic sync (#10358)
* enable optimistic sync

* revert computation of HTR in notifyNewPayload

* fix tests

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-03-14 17:25:40 +00:00
terence tsao
65cea17268 Return early if ttd is not reached (#10359) 2022-03-14 16:38:05 +00:00
Preston Van Loon
aa86c94a91 Update bazel to 5.0.0 (#10352)
* Update to bazel 5

* Add new bazel 5 experimental flags to improve remote caching

* regen cross compile for bazel 5

* gazelle

* remove some old flags

* use heremtic builds sandbox

* minimal downloads for better disk usage

* Restore manual tags for darwin builds
2022-03-14 15:05:08 +00:00
Radosław Kapka
75bb25d515 Bellatrix publish block - API Middleware fixes (#10348)
* Bellatrix publish block - API Middleware fixes

* update field name

* correct docs
2022-03-14 13:36:46 +00:00
terence tsao
1e845bc276 Fix sync committee assignment for Bellatrix (#10356)
* Fix AltairCompatible to account for future state version

* Tests

* gaz

* More tests

* Update comments

* Rename to HigherThanAltairVersionAndEpoch

* HigherEqualThanAltairVersionAndEpoch

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-03-14 01:00:33 +00:00
Nishant Das
58f7e942f2 Fix Mnemonic Validation (#10354)
* fix mnemonic validation

* potuz's review

* Add fuzz test for wallet recover

Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
2022-03-14 07:30:11 +08:00
Potuz
3f1e3cf82f remove optimistic control flag (#10340)
* remove optimistic control flag

* Rename

* More

* More

* redundant return

* fix conflict

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-03-12 17:32:04 +00:00
terence tsao
60003c481b Add unlock to IsOptimistic returns (#10341)
* Add unlock to returns

* add regression test

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-11 10:36:47 +00:00
Preston Van Loon
c1197d7881 Add static analysis for unsafe uint casting (#10318)
* Add static analysis for unsafe uint casting

* Fix violations of uintcast

* go mod tidy

* Add exclusion to nogo for darwin build

* Add test for math.Int

* Move some things to const so they are assured not to exceed int64

* Self review

* lint

* fix tests

* fix test

* Add init check for non 64 bit OS

* Move new deps from WORKSPACE to deps.bzl

* fix bazel build for go analysis runs

* Update BUILD.bazel

Remove TODO

* add math.AddInt method

* Add new test casts

* Add case where builtin functions and declared functions are covered

* Fix new findings

* cleanup

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-03-11 09:34:30 +00:00
Nishant Das
693cc79cc9 Minor Cleanup from #10255 (#10337)
* minor cleanup

* kasey's review

* kasey's review
2022-03-11 07:34:15 +00:00
kasey
23778959eb this is safer if a goroutine require fails (#10344)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2022-03-11 05:29:52 +00:00
kasey
92278e2255 fix TestValidator_WaitForKeymanagerInitialization_Web race (#10343)
* TestValidator_WaitForKeymanagerInitialization_Web

* gofmt

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2022-03-10 20:04:27 -06:00
james-prysm
c58ce41b3b Flagutil: utility for json file from directory or url and unmarshal to struct (#10333)
* initial commit

* adding first testcase wip

* fixing test

* adding more unit tests

* adding bazel file

* adding more unit tests and file checks

* addressing comments

* refactoring based on comments

* added bazel

* fixing build
2022-03-10 14:25:26 -06:00
kasey
d3b09d1e9d --curses=no for cleaner logs (no term refresh) (#10334)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-10 18:05:36 +00:00
Michael Neuder
27082e2cd2 Adding ExtractKeystores to the Keymanager interface (#10313)
* adding ExtractKeystores to the Keymanager interface

* adding ExtractKeystore to mockKeymanager struct types

* bazel run //:gazelle -- fix

Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-03-10 17:19:04 +00:00
terence tsao
12080727ea Save and retrieve fee recipients for db (#10336)
* Can save fee recipients in db

* Update BUILD.bazel

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-03-10 15:05:10 +00:00
Potuz
a731b8c0bc Boost proposer later (#10338)
* Boost Proposer after inserting block

* add regression test
2022-03-10 13:13:06 +00:00
Raul Jordan
f6eed74500 PrepareBeaconProposer Protobuf Schema (#10332)
* prepare v1 protos

* add in protos

* prepare beacon proposer

* stubs

* builds

* gaz

* build

* rem

* ssz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-09 21:40:26 +00:00
Potuz
329a4a600c Add USE_PRYSM_MODERN environment variable (#10229)
* Add USE_PRYSM_MODERN environment variable

* fix deepsource

* change naming convention

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-09 21:00:12 +00:00
terence tsao
7ce712bb5e Kiln run time changes (#10321)
* First take

* Update proposer_execution_payload.go

* Update optimistic_sync_test.go

* Add tests

* Update proposer_execution_payload.go

* Fix tests

* Add deprecation

* Fix bad merge

* New test

* Update beacon-chain/core/blocks/payload.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Proposer test can get and compare payload

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-03-09 20:26:23 +00:00
kasey
17a43c1158 Retrieving state by slot to always apply canonical block at slot, when available (#10255)
* new stategen.StateReplayer/ReplayerBuilder to give more fine-grained
  control of replaying state+block history
* all rpc/api methods updated to use the new interface, return post-state

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-03-09 13:33:18 -06:00
Radosław Kapka
77b8b13eff Bellatrix API support for block endpoints (#10324)
* refactor GetBlockV2

* Add bellatrix to GetBlockSSZV2

* Add bellatrix to ListBlockAttestations

* Add bellatrix to SubmitBlock

* gzl

* return error from SubmitBlock

* return nil

* Better code flow when getting blocks

* remove tautology

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-09 17:18:52 +00:00
Potuz
b59484b285 Export SetOptimisticToInvalid (#10330)
* temp

* export remove invalid blocks
2022-03-09 13:50:57 -03:00
Radosław Kapka
1964fb8146 Correct package name in features' README file (#10329) 2022-03-09 14:24:00 +00:00
Nishant Das
63825290cb Handle Port Registration Better (#10325) 2022-03-09 13:52:50 +01:00
kmax.eth
1619d880d4 fix TestSyncHandlers_WaitTillSynced (#10328)
* fix TestLockUnlock_CleansUnused

* fix TestSyncHandlers_WaitTillSynced
The test is failing silently (while go test showing 'PASS') due to panic caused by feed type mismatch. As a result, intended testing logic is not exercised at all. There are a couuple fixes here:
1. fix the feed event type to be pointer
2. add seendCache to avoid nil pointer panic
3. fill block in beaconDB so validateBeaconBlockPubSub gets short cut
4. replace time.Sleep() with deterministic channel waiting

* handle cancel func
2022-03-09 05:42:03 +00:00
kmax.eth
af2b858aa2 fix TestLockUnlock_CleansUnused (#10326) 2022-03-09 04:08:28 +00:00
Potuz
57a323f083 Forkchoice featureflag (#10299)
* Compiling main beacon-chain binary

* Add feature flag

* passing protoarray tests

* passing nodetree tests

* passing blockchain package tests

* passing rpc tests

* go fmt

* re-export forkchoice store from blockchain package

* remove duplicated import

* remove unused var

* add nodetree rpc method

* remove slot from IsOptimisticForRoot

* release lock in IsOptimistic

* change package name

* Revert "change package name"

This reverts commit 679112f9ef.

* rename package

* Update doc

* Fix span names

* Terence + Raul review

* remove go:build flags

* add errors dep

* spec tests

* fix call to IsOptimisticForRoot

* fix test

* Fix conflict

* change name of function

* remove ctx from store.head

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-03-09 03:05:51 +00:00
Nishant Das
738f00129b control test (#10323) 2022-03-08 07:58:19 -08:00
Radosław Kapka
3a03623094 E2E: Cleanup old Eth1 code (#10320) 2022-03-07 23:14:10 +00:00
terence tsao
cde58f6924 Rest of kiln changes (#10319)
* Update configs

* gazelle

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-07 19:47:08 +00:00
Preston Van Loon
78fe712e53 Validator: deduplicate fork logic for block proposals (#10297)
* Deduplicate block proposals

* fix tests, more dedup of tests

* Add godoc to BuildSignedBeaconBlock.

* Rename SignRequest_Object => SignRequestObject

* Fix error messages

* Add tests for new methods

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-07 19:18:54 +00:00
terence tsao
40e5a5d796 RPC: add prepare execution payload (#10311)
* Add prepare execution payload

* Add prepare execution payload

* Add prepare execution payload

* Update client.go

* Update proposer_execution_payload.go

* Update proposer_execution_payload_test.go

* Handle post bellatrix finalied blk

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer_execution_payload.go

* Use BeaconBlockIsNil

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-03-07 18:43:20 +00:00
Radosław Kapka
87507cbfe2 Support Bellatrix state in getStateV2 standard API (#10314) 2022-03-07 17:56:54 +01:00
Mohamed Zahoor
b516cfd998 cherry picked PR of #10233 (#10301)
* fix merge conflicts

* fix more merge conflicts

* fix fmt

* remove unnecessary cfg

* setting bad block if the bellatrix validation fails

* added an edge condition

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-07 13:19:07 +00:00
Radosław Kapka
f98d1ce64b Run multiple go-ethereum nodes in e2e (#10277)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-03-07 13:23:15 +01:00
Mohamed Zahoor
a103dd91c0 cherry picked PR for #10274 (#10300)
* ignore topic messages (except block topic) during optimistic sync

* address review comments

* nit pick fix

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-03-06 08:09:12 +00:00
terence tsao
74fe2cc8d0 Spawn attestation routine at a better place (#10303)
* Spawn attestation routine at a better place

* Revert

* Update service.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-04 22:25:04 +00:00
Nishant Das
a4bbaac262 Add Better Error Logs for Context Deadlines (#10310)
* better log here too

* remove period

* Update beacon-chain/p2p/pubsub.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-03-04 16:21:19 +00:00
Nishant Das
1af11885ee Remove Support for 2d-list Hashers (#10290)
* add changes

* fix logic bug

* fix

* potuz's review

* Update beacon-chain/state/stateutil/eth1_root.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-03-04 15:19:07 +00:00
Nishant Das
1437cb8982 Update GoHashTree Library (#10308) 2022-03-04 14:18:52 +00:00
Nishant Das
0b559afe30 Better Error Log In Context Deadlines (#10309)
* add better error log

* radek's review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-04 09:54:06 +00:00
Radosław Kapka
54915850a2 Refactor E2E port registration (#10306)
* Refactor e2e port registration

* uncomment tests

* explain calculation

* fix things

* change param to pointer

* fix errors

* unit test and constant
2022-03-04 09:26:28 +00:00
terence tsao
69618d157a Bellatrix validator terminal block helpers (#10305)
* Add get transition block

* Update proposer_execution_payload.go
2022-03-03 15:00:11 -05:00
Nishant Das
fa750650ed Disable Vectorized HTR from Our Dev Flag (#10304) 2022-03-03 03:49:26 +00:00
terence tsao
7303985232 Validator RPC: add optimistic status check (#10291)
* Add optimistic status check

* Simplify a bit more

* Update status_test.go

* Add non opt tests

* Update aggregator_test.go

* More tests

* Preston's feedback
2022-03-03 00:16:34 +00:00
Raul Jordan
958dd9d783 Check Engine API Transition Configuration in Background (#10250)
* transition proto

* gen pb

* builds

* impl transition config

* begin tests

* transition config messed up

* amend proto

* use str

* passing

* gaz

* config

* client test

* pb

* set to 0

* rem log

* gaz

* check transition config

* check config differences

* check transition config in background

* gaz

* pass

* redundant

* fix up error handling and healthz

* simplify status

* gazelle

* build

* err config check

* test

* gaz

* Fix run time

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-01 21:38:51 +00:00
Preston Van Loon
a3f8ccd924 validator: Fix flaky test TestServer_RefreshJWTSecretOnFileChange (#10296) 2022-03-01 20:34:05 +00:00
terence tsao
d64f6cb7a8 Add engine methods to block processing (#10285)
* Add notify newPayload and forkchoiceUpdate

* Tests

* Raul's feedback

* Update optimistic_sync.go

* Simplify

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-01 16:43:06 +00:00
terence tsao
a9a75e0004 Release lock before return validatedTips (#10289)
* Release lock before return

* add test

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-03-01 15:36:47 +00:00
Nishant Das
339540274b Integration of Vectorized Sha256 In Prysm (#10166)
* add changes

* fix for vectorize

* fix bug

* add new bench

* use new algorithms

* add latest updates

* save progress

* hack even more

* add more changes

* change library

* go mod

* fix deps

* fix dumb bug

* add flag and remove redundant code

* clean up better

* remove those ones

* clean up benches

* clean up benches

* cleanup

* gaz

* revert change

* potuz's review

* potuz's review

* potuz's review

* gaz

* potuz's review

* remove cyclical import

* revert ide changes

* potuz's review

* return
2022-02-28 21:56:12 +08:00
Raul Jordan
12ba8f3645 Renaming Random in ExecutionPayloads to PrevRandao (#10283)
* rename proto

* p header

* regen

* regen ssz

* fix randao

* random name changes

* bazel builds

* bt

* incorrect prev randao

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-26 03:47:16 +00:00
Raul Jordan
f3a7f399c0 Engine API Client Authentication for the Merge via HTTP (#10236)
* round tripper with claims

* auth

* edit auth

* test out jwt

* passing

* jwt flag

* comment

* passing

* commentary

* fix up jwt parsing

* gaz

* update jwt libs

* tidy

* gaz

* lint

* tidy up

* comment too long

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-02-25 19:08:43 +00:00
james-prysm
6163e091a7 web3signer: fixes for e2e (#10281)
* fixing logs and caught bug in mappers

* Fix schema

* improving logging

* Update validator/keymanager/remote-web3signer/internal/client.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* adding logurus dependency

Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
2022-02-25 02:42:26 +00:00
terence tsao
2fb4ddcbe7 Engine API: add payload status handling and tests (#10282)
* Add status handling and tests

* Update client.go

* Fmt

* Update mock_engine_test.go

* Update client_test.go
2022-02-24 19:35:01 +00:00
Radosław Kapka
1c2e463a30 --api-timeout flag (#10260)
* `--api-timeout` flag

* simplify code

* review feedback

* better error handling

* better docs

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-24 18:01:37 +00:00
james-prysm
01e9125761 web3signer: url parsing bug (#10278)
* adding in fixes for url

* fixing gazelle

* fixing wrong keymanager kind

* adding required scheme to urls

* fixing another unit test

* removing unused file

* adding new commit to retrigger deepsource ci
2022-02-24 10:24:11 -06:00
terence tsao
02a088d93c Add validate_merge_block (#10273)
* Add

* Update pow_block.go

* Update BUILD.bazel

* Update beacon-chain/blockchain/pow_block.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* feedbacks

* Feedbacks

* Fmt

* Update BUILD.bazel

* Update BUILD.bazel

* Update pow_block_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-23 22:41:11 +00:00
terence tsao
8cadb2ac6f Update IsOptimistic to always false (#10276)
* Update  to always false

* Use epoch

* Update chain_info_test.go

* Update chain_info_test.go
2022-02-23 20:16:45 +00:00
Potuz
be722604f7 Fix logarithm of 2 (#10275)
* Fix logarithm of 2

* add regression test

* Update encoding/ssz/merkleize.go

Co-authored-by: Nishant Das <nishdas93@gmail.com>

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-02-23 12:44:04 +00:00
Raul Jordan
3bb2acfc7d Update Web UI to v1.0.3 (#10264)
* update web UI version

* fixing format

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: James He <james@prysmaticlabs.com>
2022-02-22 18:26:15 +00:00
terence tsao
7719356b69 Add opt sync bool to IsOptimistic (#10270)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-22 16:46:42 +00:00
Raul Jordan
75b9bdba7c Small Comment Fix in Exchanging Transition Config (#10271) 2022-02-22 16:16:02 +00:00
Nishant Das
525c818672 Remove SSZ Cache (#10256)
* remove ssz cache

* gaz

* lint

* analyze more

* fix
2022-02-22 17:27:51 +08:00
terence tsao
a55fdf8949 Use type string for total_difficulty (#10265)
* Use string for difficulty

* fix go

* fix test

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-02-21 14:03:12 +00:00
Preston Van Loon
7f41b69281 Wrapper: Update block interface and reorganize fork logic (#10267)
* Add ssz.HashRoot interface composition to BeaconBlock interface, move fork specific logic into it's own files

* Remove needless underscore
2022-02-20 20:53:05 +00:00
Preston Van Loon
c5189a6862 Deduplicate TestProposer_ProposeBlock_OK (#10266)
* Deduplicate tests for TestProposer_ProposeBlock

* remove erroneous save of block

* ensure block root is returned
2022-02-19 22:52:03 +00:00
Leo Lara
b4b976c28b Experimental prototype of Apple M1 processor support (#10192)
* Experimental prototype of Apple M1 processor support

* Enable Apple M1 compilation of herumi MCL by adding a precompiled library

* Renable nogo

* Fix by gazelle

* Update go.mod to reflect go 1.17.6 changes in WORKSPACE

* go mod tidy

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
2022-02-19 18:17:08 +00:00
Nishant Das
ced24892a5 Cleanup Powchain Service (#10259)
* add cleanup

* gaz

* Fix test build
2022-02-18 14:13:31 +00:00
terence tsao
49f989e342 Fix base fee endianness (#10253) 2022-02-16 17:48:55 +00:00
Leo Lara
3003f08770 Remove unnecessary target of third_party/herumi/bls.BUILD (#10252)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-02-16 16:35:19 +00:00
terence tsao
0232b5f8f5 Regen pbs (#10227)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-02-16 10:06:41 -06:00
terence tsao
90a15b2fbe Handle PayloadStatus UnmarshalJSON nil cases (#10249)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-16 01:24:50 +00:00
Raul Jordan
0f58c9a925 Exchange Transition Configuration for Engine API (#10248)
* transition proto

* gen pb

* builds

* impl transition config

* begin tests

* transition config messed up

* amend proto

* use str

* passing

* gaz

* config

* client test

* pb

* set to 0

* rem log

* gaz
2022-02-16 00:56:23 +00:00
terence tsao
6311cfd8ab Fix payloadStatusJSON fields (#10246)
* Use pointers

* Use pointers

* Use pointers

* Update json_marshal_unmarshal.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-15 23:26:58 +00:00
Radosław Kapka
6dcf47675b E2E BellatrixE2EForkEpoch constant (#10240)
* Add `BellatrixE2EForkEpoch` constant

* change epoch

* Config param

* revert e2e update

* disable linter

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-15 21:06:14 +00:00
terence tsao
72aa782849 PayloadStatus reserve 0 enum (#10247)
* Leave 0 as UNKNOWN

* Leave 0 as UNKNOWN
2022-02-15 20:32:39 +00:00
terence tsao
cc637bad4a Fix ForkchoiceUpdatedResponse json tag (#10243) 2022-02-15 15:49:36 +00:00
Potuz
f5719f8c8e Fix updateCanonicalNodes (#10241)
* Fix updateCanonicalNodes

* fix logic

* terence's review
2022-02-14 21:30:26 +00:00
Rootul Patel
f550a964f5 Add path to Keystore (#10152)
* Add failing test

* Add path to keystore

* Assert encoded contains "path"

* Fix lint

Ran `bazel run //:gazelle -- fix`

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-02-14 09:57:15 -06:00
Radosław Kapka
c5c039fd6b Unify GenesisValidator(s)Root throughout the codebase (#10230)
* Unify `GenesisValidator(s)Root` throughout the codebase

* comments and literals

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-02-14 13:34:38 +00:00
Radosław Kapka
4d9947543f Native state feature flag (#10232)
* add flag

* working version

* rename flag

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-14 12:33:02 +00:00
Michael Neuder
7370c42bae Adding nil error check to connection status for ETH1 nodes (#10186)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-02-14 10:21:45 +00:00
Radosław Kapka
3c76cc3af5 Return state interface from native state constructors (#10208)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-14 17:51:22 +08:00
terence tsao
28af5bc601 Fix eth_getBlockByHash call (#10239)
* Update client.go

* Update client_test.go
2022-02-13 16:33:11 +00:00
terence tsao
c9f299b50a Clean up fork choice (#10226)
* Clean up fork choice

* Update beacon-chain/forkchoice/protoarray/proposer_boost_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/forkchoice/protoarray/store_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/forkchoice/protoarray/store_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update store.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-02-12 03:33:46 +00:00
terence tsao
9dfb385160 Pad fields to correct length (#10237)
* Pad fields to correct length

* Use constants

* builds

* Fix test

* Update BUILD.bazel

* tests

* passing

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-12 01:58:09 +00:00
Raul Jordan
b1774efeb7 Fix JSON Marshal/Unmarshal for Execution Block (#10238) 2022-02-12 01:06:28 +00:00
Potuz
4b3a723166 Do not stop on error loading synced tips (#10235)
* Do not stop on error loading synced tips

* Terence's review

* Update beacon-chain/blockchain/optimistic_sync.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-11 21:46:53 +00:00
terence tsao
50f253619e Fix engine client method names (#10234)
* Fix engine client method names

* Typo
2022-02-11 16:12:11 -05:00
james-prysm
753afb4fb2 small fix for web3signer (#10223) 2022-02-11 09:41:31 -05:00
Leo Lara
1835f54197 Refactor for DRY state getters_validator_test and getters_test (#10117)
* Refactor for DRY state getters_validator_test and getters_test

* Add BUILD.bazel to beacon-chain/state/testing

* Handle type assertion error in beacon-chain/state tests

* Fix with gazelle

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-02-11 14:48:29 +08:00
terence tsao
26a2311c82 Clean up outdated bellatrix code (#10225)
* Rm outdated merge code

* Update config_test.go

* Native

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-11 04:40:12 +00:00
Potuz
9f419bee7d Remove unnecessary reassignement (#10224) 2022-02-10 23:51:20 -03:00
james-prysm
d04eaf8fa0 Better DB Logging (#10215)
* initial commit

* initial commit

* reverting change

* reverting change

* Update beacon-chain/node/node.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/db/kv/kv.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-10 23:15:01 +00:00
Potuz
82ceb51548 Load Synced Tips when starting from Finalized State (#10196)
* Load Synced Tips when starting from Finalized State

* Terence's review #1

* Fix tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-10 22:18:42 +00:00
Potuz
6a2ef13b87 add metrics for synced tips (#10221)
* add metrics for synced tips

* Update beacon-chain/forkchoice/protoarray/metrics.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-10 18:48:46 +00:00
Potuz
72a2dd004b Fix integer overflow (#10222)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-02-10 15:20:36 -03:00
Raul Jordan
1a0e16a48b Initialize the Engine API Client in the Beacon Chain's Powchain Service (#10217)
* define engine caller

* gaz

* Update cmd/beacon-chain/flags/base.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-10 16:46:31 +00:00
Nishant Das
bff5c1e1a9 Handle Bellatrix Fork Epochs Correctly (#10219)
* handle it properly

* Update beacon-chain/sync/fork_watcher.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/sync/fork_watcher.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-10 16:15:55 +00:00
Nishant Das
3f5ce0cdca Handle a ZeroHash as our Eth1 Genesis Block Root (#10218) 2022-02-10 11:28:57 +00:00
terence tsao
d51b52d432 Add pow block protobuf (#10205)
* Add pow_block protobuf

* Revert hash changes

This reverts commit 4afb280385.

* Revert "Add pow_block protobuf"

This reverts commit 4afb280385.

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-09 22:27:30 +00:00
Raul Jordan
7b5a821c81 Implement Execution Engine Client (#10185)
* add in fixtures

* handle real fixtures

* marshaling issues

* ensure error handling works

* begin http client test

* custom type marshaling

* gaz

* custom type

* marshalers

* builds

* builds

* add to build file

* regen

* only root types

* bytes hash

* regen

* bytes hash

* regen

* regen

* alias magic

* revert proto

* more alias magic

* revert build files

* regen

* define marshal methods

* tests passing

* hexbytes

* rem

* rem

* comment

* gaz

* data

* fix conf

* http tests

* sync

* client tests passing

* deepsource

* gaz

* item

* exec block by hash

* begin tests

* http tests passing

* payload id bytes

* proper json encode decode for payload id

* gaz

* proper marshal

* merge engine client

* use big ints

* passes

* confirmed passing

* add in functional methods

* deepsource

* pb

* Update beacon-chain/powchain/engine-api-client/v1/client_test.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* latest

* tests passing again

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-09 20:54:56 +00:00
Mohamed Zahoor
f89265f2f7 Adding a new method in HeaderFetcher (#10212)
* Is optimisyic for root

* add the function in mock chainservice also

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-09 20:21:15 +00:00
james-prysm
c731b715ed Keymanager-Kind: fix cli kind parsing and allow for backward compatability (#10211)
* initial commit with unit tests

* reverting and adding in comments
2022-02-09 14:40:52 -05:00
Radosław Kapka
2a68c69d8f Beacon Node API Postman collection (#10213)
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-02-09 18:42:23 +00:00
terence tsao
a82fd7bf68 Fix merge transition configs (#10206)
* Add proper merge configs

* Switch to bytes

* Use string

* Update mainnet_config.go

* Rm unused

* Fix test

* Update config_test.go

* Update beacon-chain/blockchain/pow_block.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update pow_block_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-09 18:15:29 +00:00
james-prysm
9c540627ab default allowList for validator (#10210)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-02-09 17:18:49 +00:00
james-prysm
11f50453fc Keymanager api postman test collection (#10209)
* initial commit

* adding keymanager api repo url

* updated postman collection

* removing unused folder

* removing validator build

* Update hack/keymanager-api/README.md

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update hack/keymanager-api/README.md

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update hack/keymanager-api/README.md

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-02-09 16:48:08 +00:00
Potuz
5f8b01ccda Make UpdateSyncedTips safe for context cancellation (#10174)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-09 14:54:27 +00:00
Nishant Das
d800107927 Add Peer Scorer to Dev Mode (#10207) 2022-02-09 13:51:59 +01:00
Nishant Das
d3d5cfca0b Peer Score Evaluator (#9717)
* eval stuff

* change up

* update dep

* go mod tidy

* gaz
2022-02-09 09:53:25 +00:00
Potuz
18ef760ee5 Add DeleteBlock() (#10203)
* Add DeleteBlock()

* Raul review #1

* Update comments

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-02-08 23:26:26 +00:00
james-prysm
14e3e80df3 Keymanager api alignment (#10176)
* keymanager-api

* import updates to standards

* adding in more unit tests and code fixes to improve api experience

* deleting accidently generated files

* deleting accidently generated files

* adding in more test coverage

* fixing linter issue

* removing incorrect unit tests

* improving logic for accounts import

* linter fix

* addressing review comments

* fixing based on comments

* fixing nonzero root

* fixing protos

* regen protos

* adjusting protos again

* updating api specs

* fixing code and unit tests after specs changed

* fixing imports

* adding in required changes for api middleware also adding unit tests to catch changes for protos

* fixing deepsource issues

* fixing linting

* seeing if using pointers helps

* addressing comments

* updating bazel build

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-02-08 13:13:36 -06:00
Raul Jordan
3bea0e7896 ExecutionBlock Protobuf Type for the Merge (#10201)
* exec block

* exec block

* base fee

* regen

* exec block test

* fmt

* bytes for big ints

* engine pb

* passing

* add size

* bytes

* size

* size bytes

* passing
2022-02-08 18:06:02 +00:00
Radosław Kapka
8455656597 Use beacon state interfaces instead of concrete structs (#10195)
* return interface from testing/util

* remove usages of v1

* return interface from InitializeFromProto

* return interface from InitializeFromProto

* fix test

* fix interface visibility

* more fixes

* use InitializeFromProtoUnsafe in testing/util

* return early error from mock

* v2

* fix tests

* remove unnecessary assertion

* use struct in nil state test

* Revert "Auxiliary commit to revert individual files from 6bb528c2c5df2446ad18450009f63f44318d41a9"

This reverts commit 7d70238a301209f6dbfc8ff1d81b16e33b0bd67d.

* use struct in sync committee test

* v3

* use InitializeFromProtoUnsafe in mock

* use version information

* Revert "Auxiliary commit to revert individual files from 6bb528c2c5df2446ad18450009f63f44318d41a9"

This reverts commit 5d5e6f2884d21caec7530c16ad2a0d0d27c44aa1.

* revert changes to ClearPreGenesisData

* fix build error

* remove error from PreGenesisState

* bzl
2022-02-08 09:30:06 +00:00
terence tsao
af8bafd895 Spectest 1.1.9 (#10199) 2022-02-07 20:50:11 -08:00
terence tsao
c538a6c068 Add propose bellatrix block (#10189)
* Add propose bellatrix block

* Update BUILD.bazel
2022-02-08 01:29:11 +00:00
Raul Jordan
d5eb8392b6 Engine API Client Custom Marshaling (#10188)
* custom type marshaling

* gaz

* custom type

* marshalers

* builds

* builds

* add to build file

* regen

* only root types

* bytes hash

* regen

* bytes hash

* regen

* regen

* alias magic

* revert proto

* more alias magic

* revert build files

* regen

* define marshal methods

* tests passing

* hexbytes

* rem

* rem

* comment

* gaz

* data

* builds

* passing
2022-02-07 17:13:01 +00:00
Potuz
4906a0e6de add isOptimisticCandidateBlock and IsExecutionBlock (#10191)
* add isOptimisticCandidateBlock and IsExecutionBlock

* remove check

* fix tests

* Align with spec, reuse helper `ExecutionBlock`

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-02-07 01:26:20 +00:00
terence tsao
168e06e607 Save sync tips to DB (#10171)
* Save sync tips to DB

* Fix build

* Update process_block_test.go

* Copy map

* Revert back to nil

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-06 18:00:47 +00:00
Potuz
cf18d5dd42 Add SafeSlotsToImportOptimistically beacon configuration flag (#10190)
* Add SafeSlotsToImportOptimistically beacon configuration flag

* fix e2e

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-06 03:14:56 +00:00
Potuz
aeb6940935 Prune synced tips (#10143)
* Prune synced tips

* Terence's minor cleanups

* Update optimistic_sync.go

* go fmt

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-02-05 20:24:02 +00:00
Potuz
4020a603b6 Invalid synced tips (#10154)
* add synced_tips structure

* create optimisticStore type

* Add UpdateSyncedTips

* use context

* update comment

* use better context cancellation

* reinsert removed error from different PR

* Minor clean ups

* remove invalid nodes from forkchoice

* Update beacon-chain/forkchoice/protoarray/optimistic_sync.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Add check for tip status

* Terence's minor cleanups

* Potus' feedbacks

* Update optimistic_sync.go

* Add erorr

* Synced

* update weights

* Add context error check

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-03 22:57:15 +00:00
terence tsao
3a5cfab5f2 Add IsOptimistic chain info getter (#10184)
* Add  chain info getter

* Interface

* Test

* Update mock.go

* Go fmt

* Update beacon-chain/blockchain/testing/mock.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-02-03 21:30:07 +00:00
james-prysm
8e5ba13352 Fix Small Bug in Genesis Validators Root Validation Func (#10183)
* initial commit

* switching to table driven test
2022-02-03 20:58:25 +00:00
Nishant Das
e7e2f7850d Cleanup Multiclient E2E (#10180) 2022-02-03 14:14:26 +01:00
terence tsao
c8748260fd Implement is_valid_terminal_pow_block (#10172)
* add validator terminal block difficulties

* Fix test

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-03 05:28:23 +00:00
Nishant Das
a70febbf56 E2E Config Cleanup (#10178) 2022-02-03 04:53:09 +00:00
Raul Jordan
55ce1ccc33 Engine API Client Skeleton and Configuration (#10177) 2022-02-02 19:46:27 -08:00
Preston Van Loon
28e0dc5d09 Tool to print current eth1data votes (#9335)
* Initial progress on eth1data votes reporting tool

* gofmt

* add pyrmont genesis time to help txt

* Fix eth1voting tool

* correct Prater genesis time

* Update so it builds

* Add a README

* gofmt

Co-authored-by: rkapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-02 20:41:14 +00:00
Nishant Das
2b0f74904e Multiclient E2E With Lighthouse (#10020)
* save work

* current progress

* fix it more

* save progress

* fixes so far

* add signature test

* fix up changes so far

* change to latest

* update lighthouse version

* fix build

* fix again

* do one

* clean up

* fix build

* fix it

* fix test

* change tag

* remove e2e

* Update config/params/testnet_e2e_config.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* update

* Update sha

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
2022-02-02 19:13:52 +00:00
Nishant Das
5cf976e492 Merge State Cleanup (#10011)
* state related cleanup

* remove unused imports

* radek's review

* fix for native state

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-02 17:51:24 +00:00
Nishant Das
80aa4f483c Make Opt Max Cover Default (#10109)
* make default

* radek's review

* clean up

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-02 16:26:48 +00:00
terence tsao
fb92d518f9 Remove merge setting cli flags (#10162)
* Rm merge client setting

* Fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-02 15:33:48 +00:00
Nishant Das
1774188a17 Fix Certain Edge Cases With the Analyzer (#10164)
* fix analyzer

* fix other deadlock

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-02-02 14:11:31 +00:00
Radosław Kapka
ea31f096b5 Native beacon state: fieldtrie package (#10146)
* Native beacon state: `fieldtrie` package

* reuse ProofFromMerkleLayers

* more reuse + fixes

* more reuse + fixes

* move validateIndices to helpers

* remove TestAppendBeyondIndicesLimit

* do not duplicate package

* correct errors

* handle32ByteArrays function

* type switch

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-02-02 12:18:06 +00:00
Nishant Das
ea1698c1b3 Make Next Slot Cache Default (#10110)
* deprecate all of it

* handle empty slice

* fix bad bug

* fix it

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-02 11:40:46 +00:00
Nishant Das
0802c8fe66 add new methods in (#10173) 2022-02-02 14:56:07 +08:00
Raul Jordan
4322185d80 Use Execution Payload Defined in Engine V1 Namespace (#10170)
* payload use

* incl

* cloners

* regen pb

* gen ssz

* use proper payload

* broken imports

* build

* state

* builds

* proper engine

* builds

* fmt

* baz

* ssz

* baz

* fmt

* regen ssz

* receipts root

* regen pb

* ssz

* receipts root

* ci lint

* proto

* test pkg

* bazel
2022-02-02 03:30:35 +00:00
Hugo Masclet
cab0a4c9df Fix typo in CONTRIBUTING.md (#10168)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-02-02 00:01:12 +00:00
Michael Neuder
8b2b109939 Rename imported keymanager to local. (#10155)
* imported -> local

* reverting to the state of the PR at eb1e3c3d1

* use changes from develop

* del

* rem patch

* patch

* rename to local

* gazelle

* add back build

* imported rename

* gaz

* local

* merge fix + remove proto changes

* comment revert

* build

* gofmt and one new reference

* gofmt pt2

* Update validator/accounts/wallet_edit.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update validator/rpc/accounts.go

* rename

* gaz

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-02-01 19:54:19 +00:00
james-prysm
59be30e9f4 Web3Signer: metrics, tracing, and code coverage (#10161)
* initial commit for metrics, tracing, and code coverage

* fixing unit test, logs, and naming conventions

* Update validator/keymanager/remote-web3signer/internal/client.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/internal/client.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* adding required dependency

* adding more error annotations

* adding in negative test

* fixing bazel build

* gaz

* Update validator/client/validator_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/validator_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/service.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/internal/client.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/validator_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/validator_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* addressing review comments

* Update validator/keymanager/remote-web3signer/internal/client_test.go

* reverting channel direction

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-02-01 18:28:41 +00:00
Mohamed Zahoor
b27d535ade Storing synced tips in DB (#10136)
* DB support for storing synced tips

* gazel fix

* changed the synced_tips to map

* import formatting

* added function docs

* fix potuz comments

* fix prestons comment

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-02-01 16:17:28 +00:00
Nishant Das
e83c9d5862 Revert "Replace a Few IntFlags with Uint64Flags (#9959)" (#10163)
* Revert "Replace a Few IntFlags with Uint64Flags (#9959)"

This reverts commit 790bf03123.

* fix

* fix
2022-02-01 08:51:17 +00:00
Raul Jordan
eef2122a9e Engine API Protobuf Definitions (#10150)
* define proto types for engine api

* proto lint

* prettify

* build bazel

* generated pb

* txs field

* regen

* ssz gen marshal

* fastssz

* ssz

* fmt

* json marshal unmarshal custom

* exec json tests

* passing tests

* deepsource

* gaz

* gaz

* deps

* ssz max

* regen proto

* regen ssz
2022-02-01 01:47:34 +00:00
Radosław Kapka
6406afc6cf Native beacon state: v3 (#10139)
* Native beacon state: v3

* nogo_config

* remove duplicated bazel rule

* gzl

* fix unnecessary assignment

* review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-02-01 08:32:39 +08:00
terence tsao
8a01d412f5 Implement fork choice spec tests (#10159)
* starting

* Reimplement store

* begin on proposer boost

* implement fork choice proposer boost algorithm

* boosting

* gaz

* add mutexes and previous root

* comment on compute proposer boost

* safe

* rem todo

* reset and add tests

* unit test for proposer boost score

* boost works

* Can process block

* Basic test case passing

* ex ante

* test

* propoer test

* More progresses

* More fixes

* rm unused pieces

* Refactor, add phase 0

* locks

* vanilla ex-ante attack

* test similar to spec test

* works works works

* boost test working for num votes > proposer boost weight

* commentary fixes

* rem unused

* comments

* Proposer boost use store time

* Reset proposer root

* debugging

* passing

* Rm unused visibility imports

* Move update head to better place

* Fix deepsrc complains

* Fix more complains

* Raul's feedback

* Use correct byte lengths

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-01-31 19:03:48 +00:00
james-prysm
aed6e13498 Web3Signer: CLI implementation (#10056)
* initial commit for cli integration of web3signer

* resolving conflicts and execution

* remove aggregation slot from proto

* rem aggregation slot

* define a sync message block root struct

* fix sync message name

* sync message block root struct

* amend where sync committee block root is used

* altered switch statement to return correct json request by type

* fixing fork data import, types, and unit tests

* reverting unwanted changes

* reverting more unwanted changes

* fixing deepsource issues

* fixing formatting

* more fixes for deepsource and code clean up

* only want to fetch once for fetch validating public keys

* adding more comments

* new unit tests for requests and fixing a mapper issue

* Update validator/client/validator.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/accounts/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* adjusting comment

* adjusting comment

* fixing import organization

* including more unit tests

* adding new cli edit

* adding in checks for wallet initialize

* adding web3signer flags to main.go

* some how resolved files did not save correctly

* adding in check to make sure web flag only works with types imported and derived

* Update validator/client/sync_committee.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/aggregate.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/accounts/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/main.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/flags/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/flags/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* reverting changes that accidently got checked in

* reverting

* reverting

* continuing to revert unintenteded changes

* reverting

* removing more unneeded changes

* addressing review comment

* initial refactor

* adding in more clarifying comments

* fixing mock

* resolving desource issues

* addressing gosec scan for helper go file

* addressing gosec

* trying to fix bazel build

* removal of interface to fix build

* fixing maligned struct

* addressing deepsource

* fixing deepsource

* addressing efficiency of type checking

* fixing bazel test failure

* fixing go linter errors

* gaz

* web changes

* add w3signer

* new kind

* proper use

* align

* adding prysm validator flags to help flags list

* addressing root comment

* ci lint

* fixing standardapi tests

* fixing accounts_test after removal of keymanager from rpc server

* fixing more unit tests

* Update cmd/validator/flags/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/flags/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/service.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/service.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* addressing missed err checks

* fixing mock tests

* fixing gofmt

* unskipping minimal e2e test and removing related TODOs

* Update testing/endtoend/components/validator.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update testing/endtoend/components/validator.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* adding some error wrapers to clarify failure point

* fixing bazel build with new error checks

* taking preston's advice to make test fail faster to understand what's going on with the test

* checking if genesis validators root is not zero hash

* adding check for genesis validators root giving zero hash

* fixing missing dependency

* adding check for wallet

* log all

* fixing errors for http responses

* switching marshal to pretty print

* adding pretty sign request test

* fixing base url setting

* adding in check for web3signer and temporary wallet instead of having to open the wallet

* refactoring web3signer to not require wallet

* bazel build fix

* fixing gazelle build

* adding content type of request

* fixing more bazel

* removing unused code

* removing unused comments

* adding skip test back in

* addressing a validation and error message

* fix parse

* body

* fixing logic for datadir

* improving error handling

* show resp

* fix

* sign resp as str

* point of pointer remove

* sign resp

* unmarshal sig resp

* read body as str

* adding more verbose logging

* removing unused result

* fixing unit test

* reconfiguring files to properly nest code and mocks

* fix build issue

* using context when using client function calls

* fixing based on suggestion

* addressing comments

* gaz

* removing defined max timeout

* reverting json print pretty

* Update validator/accounts/wallet_edit.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* removing unneeded code restrictions

* should not introduce new code that may impact existing key manager types

* adjusting comments

* adding in json validation

* running go mod tidy

* some logging

* more logs

* fixing typo

* remove logs

* testing without byte trim

* fixing order or properties

* gaz

* tidy

* reverting some logs

* removing the confusing comments

* Update validator/client/aggregate.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/aggregate.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* addressing pr comments

* editing bytes test

* Run gazelle update-repos

* run gazelle

* improving unit test coverage

* fixing text

* fixing a potential escaped error

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-01-31 10:44:17 -06:00
Potuz
26ec352bdc Update synced tips (#10129)
* add synced_tips structure

* create optimisticStore type

* Add UpdateSyncedTips

* use context

* update comment

* use better context cancellation

* reinsert removed error from different PR

* Minor clean ups

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-01-31 15:05:05 +00:00
Nishant Das
ce712a1a3e Fix P2P Test (#10157)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-31 12:23:25 +00:00
Preston Van Loon
98470fdb13 Update rules_go to v0.29.0 (#10064)
* Update rules_go, delete patch, fix analyzers, remove old deployContract code, and prune old dependency which was broken in rules_go update

* Undo some WORKSPACE diffs

* Gazelle

* remove nofuzz, fix build issue

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-31 07:06:07 +00:00
Preston Van Loon
ef906cf704 Update fuzzbuzz.yaml (#10151)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-29 16:55:31 +00:00
Raul Jordan
65ec437ce5 Fork Choice Proposer Boost Changes (#10083)
* begin on proposer boost

* implement fork choice proposer boost algorithm

* boosting

* gaz

* add mutexes and previous root

* comment on compute proposer boost

* safe

* rem todo

* reset and add tests

* unit test for proposer boost score

* boost works

* ex ante

* test

* propoer test

* vanilla ex-ante attack

* test similar to spec test

* works works works

* boost test working for num votes > proposer boost weight

* commentary fixes

* rem unused

* comments

* boost necessary to sandwich unit test

* sad path

* gaz

* Update beacon-chain/forkchoice/protoarray/proposer_boost.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/forkchoice/protoarray/proposer_boost.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/forkchoice/protoarray/proposer_boost_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/forkchoice/protoarray/proposer_boost_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/forkchoice/protoarray/proposer_boost_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/forkchoice/protoarray/proposer_boost_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/forkchoice/protoarray/proposer_boost_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/forkchoice/protoarray/proposer_boost_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* gaz

* radek comments

* reset boosted root in on new slot

* pass

* broken tests

* cfg test

* modify e2e config

* Revert e2e changes

* Test

* Test again

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-01-29 16:32:01 +00:00
Preston Van Loon
58733a9f40 go 1.18 fuzz tests: initial testing (#10063)
* Add basic fuzz test

* Add a few tests for HTR utils

* gofmt

* Add requirement for fuzz gotag

* Run gaz and fix tags

* Update pubsub_fuzz_test.go

I'll fix this later

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-29 08:01:58 +00:00
terence tsao
762ea6dce1 State balance cache: handle input zeros (#10144)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-28 17:14:13 +00:00
terence tsao
eef1730f9e Bellatrix sync changes (#10097)
* Add bellatrix sync changes

* Fix tests

* Update validate_beacon_blocks.go

* check state and block versions

* Sync with develop

* Update rpc_chunked_response_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-28 16:26:52 +00:00
Potuz
de7f7d783e Remove unused code (#10142)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-01-28 14:52:30 +00:00
Nishant Das
5eafb80091 fix it (#10145) 2022-01-28 12:07:41 +08:00
Potuz
cb0c382bf3 Fix Fork Choice prunning (#10137)
* Fix Fork Choice prunning

* Use copy node. Update comments and var names.

* rm unused err var

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-01-27 19:17:19 +00:00
Radosław Kapka
15485f50c3 Fix native state imports (#10140) 2022-01-27 11:02:16 +00:00
Radosław Kapka
9ee00f09a1 Native beacon state: v2 (#10121)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-01-27 09:42:32 +01:00
Radosław Kapka
72817a6d0e Unify validator status response in RPC (#10128) 2022-01-26 14:23:53 -05:00
Preston Van Loon
943dec525c Refactor mock test helpers (#10133)
* Delete deploy contract tool. Move mock to its own package as testonly with some helper functions

* gofmt contracts/deposit/mock/mock.go

* move stategen mock.go to its on testonly pkg

* move password_reader_mock.go to mock testonly package

* move mock keymanager to its own testonly package

* move attestations mock

* move voluntaryexits mock

* Move slashings mock to mock pkg

* move the slasher mock

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-26 14:48:20 +00:00
Leo Lara
d9799e6b6c #10036 Replace codename Merge with Bellatrix (2nd step) (#10116)
* Rename BeaconBlockBodyMerge to BeaconBlockBodyBellatrix

* Rename SignedBeaconBlockMerge to SignedBeaconBlockBellatrix

* Rename CopyBeaconBlockMerge to CopyBeaconBlockMerge

* Rename NewBeaconBlockMerge to NewBeaconBlockBellatrix

* Rename BeaconBlockMerge to BeaconBlockBellatrix

* Rename some comments and strings in pkg proto: Merge -> Bellatrix

* Rename PbMergeBlock to PbBellatrixBlock

* Many renames of merge -> bellatrix in proto package

* Rename some Merge -> Bellatrix in beacon chain package

* More names

* Fix formating in config/params/config.go

* Rename Merge -> Bellatrix in proto/prysm/storage

* Several renames and corrections Merge -> Bellatrix

Co-authored-by: Potuz <potuz@potuz.net>

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Potuz <potuz@potuz.net>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-01-26 07:24:47 +00:00
Potuz
f0fd29d367 Is optimistic (#10124)
* add synced_tips structure

* create optimisticStore type

* Add ForkChoice.Optimistic()

Adds the logic implementing whether a synced block is optimistic or not.

* Terence's bug catch

* Reinforce warning

* Preston's review

* add comment on optimistic sync
2022-01-26 01:42:18 +00:00
terence tsao
22982e5221 Add on_tick from spec (#10130)
* Add `on_tick` from spec

* Update BUILD.bazel
2022-01-25 22:07:57 +00:00
terence tsao
5a505509c3 Use store in blockchain service (#10125)
* Use store in blockchain service

* handle nil cases
2022-01-25 18:47:39 +00:00
Radosław Kapka
588658a649 Extract conditional slot processing to new function (#10114)
* Extract conditional slot processing to new function

* rename function
2022-01-25 14:52:18 +00:00
terence tsao
ae31eed013 Add blockchain store for checkpoints (#10118)
* Add store with tests

* Rm genesis time

* Previous finalized

* Go fmt

* Update beacon-chain/blockchain/store/type.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-01-24 19:53:05 +00:00
Nishant Das
e26cde5e09 Make Balance Trie Computation Default (#10112)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-24 11:36:10 +00:00
Radosław Kapka
5cc201288d Native beacon state: v1 (#10086) 2022-01-24 11:24:38 +01:00
Nishant Das
b4fa626aff Use Batch Decompression For Signatures (#10105)
* use batch decompress method

* terence's review

* gosimple

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
2022-01-21 22:57:29 +00:00
Nishant Das
498e5efed1 Make Get Block Optimization Default (#10113)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
2022-01-21 22:22:44 +00:00
Nishant Das
f563d00e79 Filter Duplicates Messages For Sync Contributions (#10106)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-21 18:25:59 +00:00
Nishant Das
ff50448ca2 Make Batch Verification Of Gossip Default (#10111)
* make default

* add it back

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-21 17:26:42 +00:00
Preston Van Loon
26120e26cb Resolve issues with G304 (#10115) 2022-01-21 16:52:31 +00:00
terence tsao
8d610319bc Remove commented code (#10104)
* Update service.go

* go fmt

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-20 18:15:05 +00:00
terence tsao
288b38be8e Bellatrix p2p changes (#10072)
* Add `process_execution_payload` to `process_block`

* bring in bellatrix p2p changes

* lint

* fix testcase to use MaxChunkSize as config to change allowed size instead of config

* add left out config params

* Fix TestForkSchedule_CorrectNumberOfForks

* fixed Nishant's comments

* gofmt

* add switch case

Co-authored-by: Mohamed Zahoor <zahoor@zahoor.in>
Co-authored-by: Zahoor Mohamed <zahoor@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-01-20 22:12:15 +08:00
Nishant Das
af06bb9737 Do Not Copy State During Upgrade (#10102)
* no copy

* no copy

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-20 06:54:21 +00:00
Nishant Das
52f1190b17 Cleanup Up Resources Correctly (#10103)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-20 06:21:06 +00:00
Preston Van Loon
7df01feb58 Remove libfuzzer / afl based builds in favor of go 1.18 fuzz testing (#10065)
* Remove libfuzzer / afl based builds in favor of go 1.18 fuzz testing

* rm beacon-fuzz archive

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-20 04:51:02 +00:00
Nishant Das
49c73e65a1 make it better (#10101) 2022-01-20 10:37:27 +08:00
terence tsao
06c3306ac2 Update stategen for Bellatrix (#10089)
* Update stategen to for Bellatrix

* Update BUILD.bazel

* Update replay.go
2022-01-19 22:18:11 +00:00
Preston Van Loon
c3228cd5a7 e2e: Add web3signer component (#10088)
* Add initial web3signer binary

* Add support for web3signer component in e2e

* revert some changes

* Reference https://github.com/ConsenSys/web3signer/issues/485 in commentary

* gofmt

* Add notice about java 11 requirement

* Sec issue, revert to 0750 permissions

* remove unused param

* remove unused import

* use hexutil

* Fix yaml struct tags. See https://github.com/ConsenSys/web3signer/issues/485\#issuecomment-1015994840

* fmt

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-01-19 16:22:23 +00:00
james-prysm
8da8855ad5 Web3Signer: Sign Method Implementation (#10084)
* breaking up changes from cli pr

* reverting some changes, adding in changes from remote-web3signer

* adding raul's change

* adding fork info to signing calls

* fixing imports

* gaz

* fixing gofmt

* removing unneeded comment

* Update validator/client/aggregate.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update validator/client/sync_committee.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* addressing comments

* revert proto changes

* proto changes

* reserve

* reserve

* switching to passing slot from passing fork, using slot to find the fork

* removing unneeded check

* fixing missed unit test

* optional

* optional

* gaz

* improving some definitions with constants

* improving some definitions with constants

* rem opt

* rem

* gaz

* moving mocks to its own folder

* adding in bazel field to fix

* fixing type check error

* fixing build

* fixing strict imports

* fixing dependencies

* changing bazel build

* changing bazel build

* changing bazel build

* removing testing only dependency

* removing dependency on testing util package

* update bazel build

* Update checktags_test.go

* Update active_balance.go

* Update sync_committee_minimal.go

* Update sync_committee_mainnet.go

* Update active_balance_disabled.go

* Update committee.go

* Update committee_disabled.go

* Update sync_committee.pb.gw.go

* Update powchain.pb.gw.go

* Update proposer_indices.go

* Update proposer_indices_disabled.go

* Update sync_committee.go

* Update mainnet.go

* Update p2p_messages.pb.gw.go

* Update finalized_block_root_container.pb.gw.go

* Update beacon_block.pb.gw.go

* Update attestation.pb.gw.go

* Update secret_key_test.go

* Update beacon_state.pb.gw.go

* Update version.pb.gw.go

* Update sync_committee.pb.gw.go

* Update sync_committee_disabled.go

* Update mainnet_test.go

* Update minimal.go

* Update signature_test.go

* Update gocast.go

* Update cgo_symbolizer.go

* Update validator.pb.gw.go

* Update beacon_state.pb.gw.go

* Update signature.go

* Update public_key_test.go

* Update minimal_test.go

* Update checktags_test.go

* Update bls_benchmark_test.go

* Update public_key.go

* Update secret_key.go

* Update aliases.go

* Update init.go

* Update stub.go

* Update journald_linux.go

* Update attestation.pb.gw.go

* Update config_utils_develop.go

* Update stub.go

* Update stub.go

* Update beacon_block.pb.gw.go

* Update validator.pb.gw.go

* Update node.pb.gw.go

* Update config_utils_prod.go

* Update journald.go

* Update beacon_block.pb.gw.go

* Update beacon_chain.pb.gw.go

* Update beacon_state.pb.gw.go

* Update events.pb.gw.go

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* addressing comments from review

* updating length of comment

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update stub.go

revert changes

* Update validator/client/aggregate_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* addressing final comments

* fixing gofmt

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-01-18 14:31:58 -06:00
Mohamed Zahoor
4ca04dcc4b Add storage wrapper for BeaconState (#9963)
* adding proto wrapper for BeaconState

* remove BUILD.bazel file spelling mistake

* address terence review comments

* remove capitalization

Co-authored-by: Nishant Das <nishdas93@gmail.com>

* renamed as storage, use v1alpha1 ojects

* fix lint in the generated file

* store state as ssz bytes

* added a seperate version for managing storage

* goimports generated files to satisfy lint

* changed proto to oneof

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-01-18 15:21:56 +00:00
Radosław Kapka
68ef8ae99a Remove CanonicalStateChan (#10093)
(cherry picked from commit 69a2ffd7c9)
2022-01-18 14:24:47 +00:00
Leo Lara
64c02c405b Move ErrNilValidatorsInState from one in each state version to a common one (#10074)
To increase DRY and enable DRY in tests and other users of the Beacon Chain state package,
an error that was duplicated unnecessarily in each version of the state is moved to the root
Beacon Chain state package.
2022-01-18 08:19:20 +00:00
terence tsao
33d1ae0792 Eth2api: GetBlockSSZ supports bellatrix block (#10077)
* add rpc and tests

* rename
2022-01-17 02:30:09 +00:00
Preston Van Loon
182bd615ac refactoring: Deduplicate code (#10090)
* Deduplicate sync committee indices from state

* Deduplicate code in listblocks RPC endpoints, which only differ in response object

* Deduplicate test code in config/fieldparams

* Delete stale benchmark target. This is already included in go_default_test

* deduplicate test cases in TestIsSlashableValidator_OK and fix blst
BUILD file

* Deduplicate TestStore_IsFinalizedChildBlock

* Revert crypto/bls/blst/BUILD.bazel

* Deduplicate TestStore_SaveBlock_NoDuplicates

* Use a generic wrapper for beacon blocks to streamline test

* Deduplicate TestStore_BlocksCRUD

* Deduplicate TestStore_BlocksHandleZeroCase

* Deduplicate TestStore_BlocksBatchDelete

* deduplicate TestStore_BlocksHandleInvalidEndSlot

* Deduplicate TestStore_BlocksCRUD_NoCache

* Deduplicate common block test setup. Make TestStore_Blocks_FiltersCorrectly support multiple forks in test

* Deduplicate the rest of these tests

* lint

* Deprecation warning

* Add test for WrappedSignedBeaconBlock

* Test error path
2022-01-16 18:44:42 +00:00
terence tsao
3c54eb1cf6 Add propose block bellatrix (#10085)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-01-14 21:24:22 +00:00
Michael Neuder
9166f40a04 Eth1 connections (#10073)
* Adding Eth1 connections data message to service definition

* PoC implementation for ETH1 connections RPC

* naming revisions

* cleanup

* register powchain info fetcher

* regenerate protos

* gazelle Bazel

* fixing comments

Co-authored-by: Michael Neuder <michaelneuder@Michaels-MacBook-Pro.local>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-01-14 11:13:04 -05:00
Preston Van Loon
98990ba280 p2p: Update libp2p and libp2p-pubsub (#10082)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-14 14:06:20 +00:00
Radosław Kapka
9ace518539 Custom types for the native beacon state (#10080)
* custom-types package

* split types into distinct files
2022-01-14 13:25:41 +00:00
Chase Jeter
0ea4b02b8b Added analyzer for detecting recursive/nested mutex read locks. (#10066)
* Added analyzer for detecting recursive/nested mutex read locks.

* Added type assertion checks and removed unused 'iTypes' directory

* Clean up

* Bazel file fixes

* Cleaned up code and added comments.

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-01-14 17:12:21 +08:00
Radosław Kapka
d73794ce72 Make Radek and Nishant code owners of duplicated state code (#10079)
* Make Radek and Nishant code owners of the native state feature

* newline

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-01-13 21:16:42 +00:00
Preston Van Loon
2ad0ad5877 Spectests: Refactor ssz_static tests to deduplicate code and test custom HTR (#10081)
* Copy over refactoring efforts from PR #9402

* Remove unused type

* godoc commentary

* Better example err msg

* Better example err msg
2022-01-13 19:52:39 +00:00
Radosław Kapka
963acefe12 Split state package into state-proto and state-native (#10069)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: kasey <489222+kasey@users.noreply.github.com>
Co-authored-by: Dan Loewenherz <dloewenherz.adm@gmail.com>
Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
Co-authored-by: Fredrik Svantes <fredrik@ethereum.org>
Co-authored-by: Leo Lara <leolara@users.noreply.github.com>
2022-01-13 12:23:53 +01:00
Radosław Kapka
0ee85bcbab Do not reuse endpoint struct between API Middleware requests (#10076) 2022-01-12 18:58:32 +00:00
Preston Van Loon
5fb0bcfacd tools/deployContract: Remove specific k8s code (#10075) 2022-01-12 18:31:38 +00:00
james-prysm
f52a214cce Web3Signer: Object Mapper and Types (#10061)
* initial commit

* fixing some types and renaming file

* initial map function commit and moving to a v1 folder

* adding in remaining mapper functions and comments for types and initial commit on unit tests

* adding readme

* fixing unit tests and errors caught

* fixed a few missed properties and names

* updating error handling to be more descriptive

* Update validator/keymanager/remote-web3signer/client_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/v1/custom_mappers.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/v1/custom_mappers.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* fixing comments

* fixing comments

* adding more error checks

* Update validator/keymanager/remote-web3signer/client.go

* fmt and bazel build

* fixing unit tests

* lint needed

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-01-11 17:15:40 -05:00
terence tsao
88b94eae18 Export reverse byte order function (#10040)
* Reverse byte order

* Update BUILD.bazel

* Go imports

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-11 18:42:03 +00:00
Potuz
3d02addfe4 Update README.md (#10071)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-01-11 16:32:07 +00:00
Raul Jordan
53f7030871 Initialize Merkle Layers and Recompute Dirty Fields in BeaconState Proofs (#10032)
* handle trie recomputes

* recomp

* state trie

* args

* deep source

* amend test

* proofs test more fixes

* tests pass
2022-01-11 17:36:03 +08:00
terence tsao
84335b0084 Bellatrix spec tests (#10014)
* Add merge spectests

* fix build

* gazelle

* operation

* all tests should pass

* go fmt

* fix test

* fix dependency

* rm unused constants

* rename to bellatrix

* Gaz

* Rm unused files

* Use Bellatrix for test names

* Add more spec tests

* Gaz and fix test

* sync with dev

* Preston's feedback

* Go fmt

* sync with dev

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-10 18:27:08 +00:00
Leo Lara
b1c2454658 #10036 Replace codename Merge with Bellatrix (1st step) (#10044)
* Rename BeaconStateMerge to BeaconStateBellatrix

* Rename version.Merge to version.Bellatrix

* Rename ComputeFieldRootsWithHasherMerge to ComputeFieldRootsWithHasherBellatrix

* Rename test names to Bellatrix

* Rename comments and strings to Bellatrix

* Fix formatting in a few files

* Revert wrong renaming in test name

* Revert renaming to Bellatrix in mainnet_config.go

* Revert renaming of db key without migration

* Regenerate from proto changes

* Rename new use of already renamed symbols

* gofmt and goimports after regenerating protofiles

* revert weird imports

Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
2022-01-10 16:47:30 +00:00
Fredrik Svantes
29a8b6c08f Updating from buster to bullseye (#10025)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-01-10 15:48:43 +00:00
Radosław Kapka
3bce9df382 Allow comma-separated event topics (#10052) 2022-01-07 17:17:29 +00:00
terence tsao
8ee3019954 Can upgrade to Bellatrix in process slots (#10062)
* Add `process_execution_payload` to `process_block`

* Add upgrade bellatrix to process slots

* Add upgrade bellatrix to process slots

* gazelle
2022-01-07 16:47:33 +00:00
terence tsao
ef1227b14c Add process_execution_payload to process_block (#10060) 2022-01-06 19:36:39 +00:00
Preston Van Loon
80b7f691ae p2p: no alloc extract of gossip digest (#10055)
* Extract gossip topic without a full copy via strings.Split

* Add basic benchmark

* gofmt

* Add empty topic test case

* Remove nl

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-01-06 18:17:51 +00:00
terence tsao
c69bce5d84 Use fieldparams for BLS public key (#10042)
* Use fieldparams for pubkey length

* Fix validator tests

* fix more tests

* fix mock validator

* Fix typo

* bunch of typos

* Update bytes.go

* Update BUILD.bazel

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-06 17:33:08 +00:00
Fredrik Svantes
ad06230291 Updating eip3076_spec_tests and eth_networks URLs (#10057)
From old eth2-clients to new eth-clients

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-01-06 16:58:37 +00:00
terence tsao
a1a301ad5d process payload (#10054) 2022-01-06 08:22:38 -08:00
james-prysm
d28ae62d02 Web3signer keymanager: key manager implementation (#10016)
* initial commit for work on web3signer-keymanager

* WIP key manager client

* adding in more valid unit tests

* refactoring some of my code to work better with current implementations

* adding comments, some error situations and fixing unit tests

* updating go.mod

* gaz

* addressing sandbox errors on test

* fixing more sand box debug items

* missed yet another error case on my test

* reverting go.mod

* addressing comments

* missed removal of unit test

* removing some deepsource issues

* adjusting based on review comments

* addressing comments

* adding the missed types

* changing public keys to only pull once

* fixing commented code format

* gaz

* Update validator/keymanager/remote-web3signer/keymanager.go

updating comments

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/client.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* fixing conflicts

* addressing comments

* fixing deepsource issues

* more deep source issue fix

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-01-05 14:32:46 -06:00
terence tsao
d6a02833dc Add HTR for txs (#10030)
* Add htr for txs

* gaz

* minimal

* test pack
2022-01-05 18:36:39 +00:00
terence tsao
fade8aa7a9 Add validate payload helpers (#10028)
* Add merge complete helper

* Update BUILD.bazel

* minimal

* Add validate payload helpers

* gaz

* update tests

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-01-05 15:28:58 +00:00
terence tsao
8604832499 Use prysm require test lib (#10039)
* Use prysm require test lib

* Fix typo

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-05 07:06:51 +00:00
terence tsao
ceabee0198 Update Transactions and ExtraData non empty check (#10049) 2022-01-03 23:53:39 +01:00
terence tsao
ebedf481f8 Update bazel version to 4.2.2 (#10045) 2021-12-27 23:51:23 +00:00
terence tsao
9f27af43cb Spectest 1.1.8 (#10041)
* Update spec test to v1.1.8

* Update names

* Update network repo

* Update WORKSPACE

* Better names

* Update beacon-chain/core/time/slot_epoch.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-12-26 23:18:28 +00:00
terence tsao
09690590f0 Update spec test to v1.1.7 (#10033) 2021-12-23 21:33:41 +00:00
terence tsao
e3c8491177 Add is_execution_enabled helper (#10027) 2021-12-22 15:03:54 -08:00
terence tsao
bfc90e9abb Add is_merge_block helper (#10026)
* Add merge complete helper

* Update BUILD.bazel

* minimal

* Add is_merge_block helper

* rename to `IsMergeBlock`
2021-12-22 15:52:50 +00:00
Raul Jordan
d173a6e695 Merkle Proof Support for BeaconState Finalized Root and Sync Committees (#10029) 2021-12-19 17:02:42 -08:00
Potuz
7f857ae23a Fix sync commitee endpoint (#10015) 2021-12-18 17:04:30 +01:00
terence tsao
23e39d3d64 Fix state alias v3 (#10022) 2021-12-16 22:44:43 +00:00
terence tsao
ec65c442c4 Minor cleanups on monitor service (#10013)
* Minor cleanups on monitor service

* Update tests

* Update process_block_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-15 20:42:20 +00:00
terence tsao
b38b0186b8 Use signature length from fieldparams (#10019)
* Use bls sig length from fieldparams

* fmt

* fix tests

* fix tests

* fix tests

* Update tags_test.go

* fix tests

* Update BUILD.bazel
2021-12-15 15:14:30 -05:00
terence tsao
4bf935928b Use field params root length (#10012)
* Use config length 32

* Add root length to minimal

* Fix build

* Gaz
2021-12-14 18:42:05 +00:00
terence tsao
acc4ee8d9d Add beacon state upgrade to merge version helper function (#9981) 2021-12-14 01:18:27 -08:00
terence tsao
7b7ed87ad8 Add more merge processing (non substrantive) (#10000)
* Add more merge processing

* Update tests

* raul's feedback

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-12-13 18:04:37 +00:00
Nishant Das
5ab88da183 Use Static Sizes for State Fields (#10007) 2021-12-13 10:16:23 +01:00
Raul Jordan
d6338f6042 Update Web UI Version to v1.0.2 (#10009)
* latest web ui

* fmt

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-12-11 11:36:00 -06:00
Antonio Sanso
c84c6ab547 Add Missing spec tests in sss_static.go (#10003)
* Update ssz_static.go

* Update ssz_static.go

* Update testing/spectest/shared/phase0/ssz_static/ssz_static.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Add Missing spec tests in sss_static.go

* Update testing/spectest/shared/altair/ssz_static/ssz_static.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-12-10 15:18:57 -06:00
Radosław Kapka
52d8a1646f Standardize config overriding in tests across repo (#9998)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-10 04:18:47 +00:00
kasey
3c61cc7d8a allow checkpoint or genesis origin; refactoring (#9976)
* allow checkpoint or genesis origin; refactoring

some quick readability improvements and simplifying the logic enforcing
the startup ordering of the attestation processing routine

* address PR feedback

* gofmt

* Update beacon-chain/blockchain/receive_attestation.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Apply suggestions from code review

use log.WithError for aggregation friendliness

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

Co-authored-by: kasey <kasey@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-09 22:23:00 +00:00
Raul Jordan
37ca409cc1 Add More Fields to Remote Signer Sign Request (#10004)
* add in new fields

* sign request added

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-09 21:53:54 +00:00
Raul Jordan
b381ad49b5 Add Justifications for Gosec Ignored (#10005)
* pin gosec

* edit

* go back to master

* justifications

* Update crypto/bls/blst/signature.go

* proper format

* gosec
2021-12-09 19:40:48 +00:00
james-prysm
00c3a7dcaf Web3signer: Add an HTTP Client Wrapper to Interact with Remote Server (#9991)
* initial commit for web3signer code work in progress

* adding more functions for web3signer

* more improvements to unit tests and web3signer functions

* fixing unit test

* removing path construction

* fixing failing unit test

* adding more happy path unit tests

* fixing unit tests

* temp removing keymanagerfiles being wip

* removing some comments

* Update validator/keymanager/remote-web3signer/client.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/client.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* making errors lowercase

* addressing review comments

* missed resolving a conflict

* addressing deepsource issues

* bazel test and gazelle

* no lint

* deadcode

* addressing comments

* fixing comments

* small fix for readability

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-12-09 09:33:53 -06:00
james-prysm
d66edc9670 fix cors middleware (#9999)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-12-08 17:13:28 -05:00
Raul Jordan
4b249607da Return ERROR Status in ImportKeystores standard API endpoint if slashing protection fails (#9995)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-08 19:52:05 +00:00
Radosław Kapka
02483ba89c Add missing config cleanup in tests (#9996)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-08 19:24:16 +00:00
terence tsao
9629c354f1 Use merge configs for processing (#9982) 2021-12-08 11:02:53 -08:00
Raul Jordan
886332c8fa Add Safe Sub64 Method (#9993)
* add sub 64

* add sub

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-08 09:44:54 +00:00
Raul Jordan
424c8f6b46 API Middleware for Keymanager Standard API Endpoints (#9936)
* begin the middleware approach

* attempt middleware

* middleware works in tandem with web ui

* handle delete as well

* delete request

* DELETE working

* tool to perform imports

* functioning

* commentary

* build

* gaz

* smol test

* enable keymanager api use protonames

* edit

* one rule

* rem gw

* Fix custom compiler

(cherry picked from commit 3b1f65919e04ddf7e07c8f60cba1be883a736476)

* gen proto

* imports

* Update validator/node/node.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* remaining comments

* update item

* rpc

* add

* run gateway

* simplify

* rem flag

* deep source

Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-07 20:26:21 +00:00
terence tsao
cee3b626f3 make db merge compatible (#9987)
* make db merge compatible

* util merge.go per @rauljordan

* Go fmt

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-07 18:51:05 +00:00
Radosław Kapka
5569a68452 Code cleanup (#9992)
* Value assigned to a variable is never read before being overwritten

* The result of append is not used anywhere

* Suspicious assignment of range-loop vars detected

* Unused method receiver detected

* Revert "Auxiliary commit to revert individual files from 54edcb445484a2e5d79612e19af8e949b8861253"

This reverts commit bbd1e1beabf7b0c5cfc4f514dcc820062ad6c063.

* Method modifies receiver

* Fix test

* Duplicate imports detected

* Incorrectly formatted error string

* Types of function parameters can be combined

* One more "Unused method receiver detected"

* Unused parameter detected in function
2021-12-07 17:52:39 +00:00
Raul Jordan
1eff00fb33 Display Num Pruned Items in Slasher Only When Actually Prunes (#9989)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-07 14:38:19 +00:00
terence tsao
2bcda2e021 Add more merge related protobufs (#9986)
* Add merge protos

* Update proto/eth/v2/beacon_block.proto

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-12-07 08:08:14 +00:00
Raul Jordan
98fea2e94d Slasher Significant Optimizations (#9833)
* optimizations to slasher runtime

* remove unnecessary code

* test for epoch update

* commentary

* Gaz

* fmt

* amend test

* better logging

* better logs

* log

* div 0

* more logging

* no log

* use map instead

* passing

* comments

* passing

* for select loop wait for init

* sub

* srv

* debug

* fix panic

* gaz

* builds

* sim gen

* ineff

* commentary

* data

* log

* base

* try

* rem logs

* sim logs

* fix wait for sync event

* ev

* init

* init

* Update beacon-chain/slasher/service.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* comments

* elapsed

* Update testing/slasher/simulator/simulator.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* timeout

* inner cancel

* ctx err everywhere

* Add context aware to several potentially long running db operations

* Fix missing param after updating with develop

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-12-06 21:45:38 +00:00
Radosław Kapka
e53be1acbe Do not rely on peer order in TestDebugServer_ListPeers (#9988)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-12-06 19:29:48 +00:00
Nishant Das
c9f5f8dad5 Use Cached Finalized State When Pruning Deposits (#9985)
* use cached finalized state

* Minor grammar edits

* fmt

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-12-06 12:38:23 +00:00
terence tsao
03bf463d0f Remove unused validator proto imports (#9983) 2021-12-04 21:05:54 +01:00
terence tsao
afb49aaeb9 Add and use merge penalty params (#9878)
* Add and use merge penalty params

* add attester slashing test

* Update config_test.go

* Use DeterministicGenesisStateMerge
2021-12-03 22:00:34 +00:00
Potuz
1fd65ad7fd typos (#9979) 2021-12-03 20:18:43 +00:00
Potuz
b1df3b55b0 more typos (#9980) 2021-12-03 16:31:30 -03:00
Preston Van Loon
3767574c77 stateutil: Reduce allocations by specifying slice capacity eager (#9977)
* Reduce allocations by specifying slice capacity eager

* gofmt
2021-12-03 09:06:05 +00:00
Raul Jordan
d3c97da4e1 Ensure Slashing Protection Exports and Keymanager API Work According to Spec (#9938)
* password compliance

* delete keys tests

* changes to slashing protection exports

* export tests pass

* fix up failures

* gaz

* table driven tests for delete keystores

* comment

* rem deletion logic

* look ma, no db

* fix up tests

* ineff

* gaz

* broken test fix

* Update validator/keymanager/imported/delete.go

* rem
2021-12-02 09:58:49 -05:00
Raul Jordan
1d216a8737 Filter Errored Keys from Returned Slashing Protection History in Standard API (#9968)
* add err condition

* naming
2021-12-02 03:32:34 +00:00
Raul Jordan
790bf03123 Replace a Few IntFlags with Uint64Flags (#9959)
* use uints instead of ints

* fix method

* fix

* fix

* builds

* deepsource

* deep source
2021-12-01 23:34:53 +00:00
Preston Van Loon
ab60b1c7b2 Update go-ethereum to v1.10.13 (#9967)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-01 19:31:21 +00:00
Nishant Das
236a5c4167 Cleanup From Deepsource (#9961)
* ds cleanup

* fix

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-01 18:56:07 +00:00
Nishant Das
5e2229ce9d Update Libp2p to v0.15.1 (#9960)
* fix deps

* tidy it all

* fix build

* remove tls patch

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-01 18:09:34 +00:00
Potuz
6ffba5c769 Add v1alpha1_to_v2.go (#9966)
* Add v1alpha1_to_v2.go

* add tests

* gazelle

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-01 17:39:43 +00:00
Potuz
3e61763bd7 fix operation precedence (#9965) 2021-12-01 17:14:08 +00:00
terence tsao
23bdce2354 Fix grpc client connected... logging (#9956)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-12-01 14:45:39 +00:00
Nishant Das
d94bf32dcf Faster Doppelganger Check (#9964)
* faster check

* potuz's review

* potuz's review
2021-12-01 12:37:10 +00:00
Nishant Das
7cbef104b0 Remove Balances Timeout (#9957)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-12-01 04:03:26 +00:00
Potuz
cd6d0d9cf1 Monitor aggregated logs (#9943) 2021-12-01 03:35:55 +00:00
Potuz
afbe02697d Monitor service (#9933)
* Add a service for the monitor

* Do not block service start

* gaz

* move channel subscription outide go routine

* add service start test

* fix panic on node tests

* Radek's first pass

* Radek's take 2

* uncap error messages

* revert reversal

* Terence take 1

* gaz

* Missing locks found by Terence

* Track via bool not empty interface

* Add tests for every function

* fix allocation of slice

* Minor cleanups

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-11-30 19:27:03 -03:00
terence tsao
2c921ec628 Update spec tests to v1.1.6 (#9955)
* Update spec test to v1.1.6

* Update spec test to v1.1.6
2021-11-30 21:21:59 +00:00
terence tsao
0e72938914 Uncap error messages (#9952) 2021-11-30 07:41:07 -08:00
Potuz
71d55d1cff Check for syncstatus before performing a voluntary exit (#9951)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-11-30 10:40:59 +00:00
terence tsao
d8aa0f8827 Alter config filed name to devnet if it's not populated in file (#9949) 2021-11-29 19:27:26 -08:00
Nishant Das
37bc407b56 Refactor States To Allow for Single Cached Hasher (#9922)
* initial changes

* gaz

* unexport and add in godoc

* nocache

* fix edge case

* fix bad implementation

* fix build file

* add it in

* terence's review

* gaz

* fix build

* Apply suggestions from code review

remove assigned ctx

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-11-29 16:30:17 +00:00
Potuz
5983d0a397 Allow requests for next sync committee (#9945)
* Allow requests for next sync committee

* fix deepsource and variable rename

* Minor cleanup

* Potuz's comments

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-11-28 17:34:24 +00:00
terence tsao
85faecf2ca Add test utility merge state (#9944)
* Add test utility merge state

* gaz

* gaz
2021-11-26 15:53:25 +00:00
terence tsao
f42227aa04 Rest of the merge state implementation (#9939)
* Add rest of the state implementations

* Update BUILD.bazel

* Update state_trie_test.go

* fix test

* fix test

* Update beacon-chain/state/v3/state_trie.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

* Update beacon-chain/state/v3/state_trie.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

* add ctx

* go fmt

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2021-11-25 17:41:05 -03:00
terence tsao
c9d5b4ba0e Add merge beacon block wrappers (#9906) 2021-11-24 14:26:17 -08:00
Potuz
fed004686b Add verbosity to aggregation logs (#9937) 2021-11-24 11:35:45 -08:00
Raul Jordan
4ae7513835 Import Keystores Standard API Implementation (#9924)
* begin

* rem deleted code

* delete keystores all tests

* surface errors to user

* add in changes

* del

* tests

* slice

* begin import process

* add import keystores logic

* unit tests for import

* tests for all import keystores keymanager issues

* change proto

* pbs

* renaming works

* use proper request

* pb

* comment

* gaz

* fix up cli cmd

* test

* add gw

* precond

* tests

* radek comments
2021-11-24 10:40:49 -05:00
Nishant Das
1d53fd2fd3 revert change (#9931) 2021-11-24 07:09:15 -08:00
Potuz
a2c1185032 Monitor sync committee (#9923)
* Add sync committeee contributions to monitor

* gaz

* Raul's review

* Added lock around TrackedValidators

* add comment to trackedIndex

* add missing locks because of trackedIndex

* Terence fixes 2

* moved TrackedValidator to service from config

* Terence comment fix

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-11-24 09:56:34 +08:00
terence tsao
448d62d6e3 Add merge beacon chain objects and generate ssz.go (#9929) 2021-11-23 23:34:31 +00:00
terence tsao
4858de7875 Use prysmaticlabs/fastssz (#9928)
* Use prysmaticlabs/fastssz

* Generated code
2021-11-23 21:28:24 +00:00
terence tsao
cd1e3f2b3e Rename coinbase to fee recipient (#9918)
* Rename coinbase to fee recipient

* Fix imports

* Update field name

* Fee receipient

* Fix goimports

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-23 17:49:06 +00:00
Nishant Das
6f20d17d15 Rename To Signature Batch (#9926)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-11-23 17:57:06 +01:00
terence tsao
94fd99f5cd Add getters and setters for beacon state v3 (part 2) (#9916) 2021-11-22 15:56:23 -08:00
Potuz
d4a420ddfd Monitor blocks (#9910)
* Add proposer logging to validator monitor

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-11-22 18:20:31 -03:00
terence tsao
838b19e985 Add getters and setters for beacon state v3 (part 1) (#9915) 2021-11-22 09:37:55 -08:00
Potuz
788338a004 Stop packing deposits early if we reach max allowed (#9806)
* Stop packing deposits early if we reach max allowed

* Add logs to proposals without deposits

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer_deposits.go

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer_deposits.go

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer_deposits.go

* reinsert debug log

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-11-20 14:14:07 +00:00
kasey
39c33b82ad Switch to lazy state balance cache (#9822)
* quick lazy balance cache proof of concept

* WIP refactoring to use lazy cache

* updating tests to use functional opts

* updating the rest of the tests, all passing

* use mock stategen where possible

reduces the number of test cases that require db setup

* rename test opt method for clear link

* Update beacon-chain/blockchain/process_block.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* test assumption that zerohash is in db

* remove unused MockDB (mocking stategen instead)

* fix cache bug, switch to sync.Mutex

* improve test coverage for the state cache

* uncomment failing genesis test for discussion

* gofmt

* remove unused Service struct member

* cleanup unused func input

* combining type declaration in signature

* don't export the state cache constructor

* work around blockchain deps w/ new file

service_test brings in a ton of dependencies that make bazel rules
for blockchain complex, so just sticking these mocks in their own
file simplifies things.

* gofmt

* remove intentionally failing test

this test established that the zero root can't be used to look up the
state, resulting in a change in another PR to update stategen to use the
GenesisState db method instead when the zero root is detected.

* fixed error introduced by develop refresh

* fix import ordering

* appease deepsource

* remove unused function

* godoc comments on new requires/assert

* defensive constructor per terence's PR comment

* more differentiated balance cache metric names

Co-authored-by: kasey <kasey@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-19 15:59:26 +00:00
Potuz
905e0f4c1c Monitor metrics (#9921)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-11-19 15:34:28 +00:00
Nishant Das
0ade1f121d Add Balance Field Trie (#9793)
* save stuff

* fix in v1

* clean up more

* fix bugs

* add comments and clean up

* add flag + test

* add tests

* fmt

* radek's review

* gaz

* kasey's review

* gaz and new conditional

* improve naming
2021-11-19 20:01:15 +08:00
Raul Jordan
ee52f8dff3 Implement Validator Standard Key Manager API Delete Keystores (#9886)
* begin

* implement delete and filter export history

* rem deleted code

* delete keystores all tests

* gaz

* test

* double import fix

* test

* surface errors to user

* add in changes

* edit proto

* edit

* del

* tests

* gaz

* slice

* duplicate key found in request
2021-11-19 04:11:54 +00:00
Potuz
50159c2e48 Monitor attestations (#9901)
Log attestation performance on the validator monitor
2021-11-18 22:14:56 -03:00
Preston Van Loon
cae58bbbd8 Unskip v2 end to end check for prior release (#9920) 2021-11-18 23:17:19 +00:00
Raul Jordan
9b37418761 Warn Users In Case Slashing Protection Exports are Empty (#9919)
* export text

* Update cmd/validator/slashing-protection/export.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2021-11-18 20:49:19 +00:00
Raul Jordan
a78cdf86cc Warn Users if Slashing Protection History is Empty (#9909)
* warning in case not found

* better comment

* fatal
2021-11-17 15:39:43 +00:00
Nishant Das
1c4ea75a18 Prevent Reprocessing of a Block From Our Pending Queue (#9904)
* fix bugs

* test

* raul's review
2021-11-17 10:05:50 -05:00
terence tsao
6f4c80531c Add field roots for beacon state v3 (#9914)
* Add field roots for beacon state

* Update BUILD.bazel

* Adding an exception for state v3

* fix deadcode

Co-authored-by: nisdas <nishdas93@gmail.com>
2021-11-17 09:04:49 +00:00
terence tsao
e3246922eb Add merge state type definitions (#9908) 2021-11-16 08:36:13 -08:00
Nishant Das
5962363847 Add in Deposit Map To Cache (#9885)
* add in map + tests

* tes

* raul's review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-11-16 02:07:42 +00:00
Potuz
5e8cf9cd28 Monitor exits (#9899)
* Validator monitor process slashings

* Preston's comment

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* add test for tracked index

* Prestons requested changes

* Process voluntary exits in validator monitor

* Address Reviewers' comments

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-11-15 19:45:18 +00:00
Raul Jordan
b5ca09bce6 Add Network Flags to Slashing Protection Export Command (#9907) 2021-11-15 19:21:54 +00:00
terence tsao
720ee3f2a4 Add merge state protobuf (#9888)
* Add beacon state protobuf

* Update proto/prysm/v1alpha1/beacon_state.proto

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update proto/prysm/v1alpha1/beacon_state.proto

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Regenerate pbs

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-15 18:53:43 +00:00
Potuz
3d139d35f6 Validator monitor process slashings (#9898)
* Validator monitor process slashings

* Preston's comment

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* add test for tracked index

* Prestons requested changes

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-11-15 17:39:55 +00:00
terence tsao
ea38969af2 Register start-up state when interop mode (#9900)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-15 16:32:54 +00:00
terence tsao
f753ce81cc Add merge block protobuf (#9887)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-15 15:42:18 +00:00
Nishant Das
9d678b0c47 Clean Up Methods In Prysm (#9903)
* clean up

* go simple
2021-11-15 10:13:52 -05:00
Nishant Das
4a4a7e97df handle canceled contexts (#9893)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-11-13 16:03:27 +08:00
Nishant Das
652b1617ed Use Next Slot Cache In More Places (#9884)
* add changes

* terence's review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-11-12 23:46:06 +00:00
Nishant Das
42edc4f8dd Improve RNG Documentation (#9892)
* add comments

* comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-12 20:30:09 +00:00
terence tsao
70d5bc448f Add cli override setting for merge (#9891) 2021-11-12 12:01:14 -08:00
Preston Van Loon
d1159308c8 Update security.txt (#9896)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-12 15:55:45 +00:00
Nishant Das
e01298bd08 Remove Superflous Errors From Parameter Registration (#9894) 2021-11-12 15:28:21 +00:00
terence tsao
2bcb62db28 Remove unused import (#9890) 2021-11-11 22:54:48 +00:00
terence tsao
672fb72a7f Update spec tests to v1.1.5 (#9875)
* Update spec tests to v1.1.5

* Ignore `TERMINAL_BLOCK_HASH_ACTIVATION_EPOCH`

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-10 02:41:16 +00:00
terence tsao
7fbd5b06da time pkg: can upgrade to merge helper (#9879)
* Helper: can upgrade to merge

* Typo
2021-11-09 18:36:50 +00:00
Raul Jordan
0fb91437fc Group Slashing Protection History Packages Idiomatically (#9873)
* rename

* gaz

* gaz

* Gaz

* rename

* edit

* gaz

* gaz

* build

* fix

* build

* fix up

* fix

* gaz

* cli import export

* gaz

* flag

* rev

* comm

* package renames

* radek
2021-11-09 16:49:28 +00:00
Raul Jordan
6e731bdedd Implement List Keystores for Standard API (#9863)
* start api

* keystores list

* gaz

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-11-08 19:08:17 +00:00
Dan Loewenherz
40eb718ba2 Fix typos: repones -> response, attestion -> attestation (#9868)
* Fix typo: repones -> response

* Fix typo: attestion -> attestation
2021-11-07 11:02:01 -06:00
terence tsao
e1840f7523 Share finalized state at start up (#9843)
* Reuse finalized beacon state at startup

* Better logging for replay

* Update tests

* Fix lint

* Add `WithFinalizedStateAtStartup`

* Update service.go

* Remove unused fields

* Update service_test.go

* Update service_test.go

* Update service_test.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-06 13:45:16 +00:00
terence tsao
b07e1ba7a4 GenesisState when zero hash tests (#9866) 2021-11-06 06:57:54 +01:00
Raul Jordan
233171d17c [Service Config Revamp] - Sync Service With Functional Options (#9859)
* sync config refactor

* rem

* rem

* testing

* gaz

* next

* fuzz

* build

* fuzz

* rev

* log

* cfg
2021-11-05 19:08:58 +00:00
kasey
2b0e132201 fix #9851 using GenesisState when zero hash in stategen StateByRoot/StateByRootInitialSync (#9852)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-11-05 18:14:37 +00:00
Raul Jordan
ad9e5331f5 Update Prysm Web to Use v1.0.1 (#9858)
* update web

* site data update

* fmt
2021-11-04 16:12:32 -04:00
terence tsao
341a2f1ea3 Use math.MaxUint64 (#9857)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-04 18:48:09 +00:00
Raul Jordan
7974fe01cd [Service Revamp] - Powchain Service With Functional Options (#9856)
* begin powchain service refactor

* begin refactor

* powchain passes

* options pkg

* gaz

* rev

* rev

* comments

* move to right place

* bazel powchain

* fix test

* log

* contract addr

* happy path and comments

* gaz

* new service
2021-11-04 14:19:44 -04:00
Raul Jordan
ae56f643eb Rename Web UI Performance Endpoint to Summary (#9855)
* rename endpoint

* rename endpoint
2021-11-04 15:13:16 +00:00
terence tsao
2ea09b621e Simplify should update justified (#9837)
* Simplify should update justified

* Update tests

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-03 22:29:03 +00:00
terence tsao
40fedee137 Use prev epoch source naming correctly (#9840)
* Use prev epoch source correctly

* Update epoch_precompute_test.go

* Update type.go

* Update tests

* Gazelle

* Move version to read only

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-11-03 21:50:41 +00:00
kasey
7cdddcb015 Weak subjectivity verification refactor (#9832)
* weak subjectivity verification refactor

This separates weak subjectivity verification into a
distinct type which does not have a dependency on
blockchain.Service

* remove unused variable

* saving enqueued init blocks before ws verify

* remove TODO, handled in previous commit

* accept suggested comment change

start comment w/ name of function NewWeakSubjectivityVerifier

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* update log w/ Raul's suggested language

* explicit zero value for clarity

* add comments clarifying how we adhere to spec

* more clear TODO per Raul's feedback

* gofmt

Co-authored-by: kasey <kasey@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-11-03 15:07:46 +00:00
Raul Jordan
4440ac199f Empty Genesis Validators Root Check in Slashing Protection Export (#9849)
* test for empty genesis validators root

* precod

* fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-03 14:16:57 +00:00
Radosław Kapka
d78428c49e Return proper responses from KeyManagement service (#9846)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-02 15:46:11 +00:00
Raul Jordan
2e45fada34 Validate Password on RPC CreateWallet Request (#9848)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-02 15:20:43 +00:00
Raul Jordan
4c18d291f4 Rename Interop-Cold-Start Package to Deterministic-Genesis (#9841)
* rename interop-cold-start

* rev

* rev

* BUILD

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-02 14:55:36 +00:00
Preston Van Loon
dfe33b0770 stateutil: Remove duplicated MerkleizeTrieLeaves method (#9847) 2021-11-02 14:31:16 +00:00
Raul Jordan
63308239d9 Define Validator Key Management Standard API Schema (#9817)
* begin service for key management

* begin defining schema

* generate bindings

* rev

* add in custom compiler

* use custom plugin with option

* goimports

* fix up proto to take in multiple passwords

* keymanagent proto edit

* rev

* rev

* dev

* builds

* comment

* indent

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-01 17:26:15 +00:00
terence tsao
712cc18ee0 Use BeaconBlockIsNil helper more (#9834)
* Replace manual checks with helper

* Rename to `BeaconBlockIsNil`

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-01 14:13:05 +00:00
Nishant Das
3d318cffa2 Fix Individual Votes RPC Endpoint (#9831)
* fix

* fix tests

* add terence's review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-11-01 18:36:08 +08:00
terence tsao
026207fc16 Remove de-duplication condition for seen aggregates (#9830)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-10-29 21:53:49 +00:00
Raul Jordan
6b7b30ce47 Use GET Request for Slashing Protection Export (#9838) 2021-10-29 18:23:45 +00:00
Yash Bhutwala
105bb70b5e remove extra condition (#9836) 2021-10-29 09:29:33 -07:00
Potuz
9564ab1f7f Fix validator performance logs (#9828)
* Fix validator performance logs

* Add test

* Clean both phase 0 and altair calculations

* Add back total inclusion distance

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-10-28 16:56:27 +00:00
Raul Jordan
0b09e3e955 Refresh Web Backend JWT Secret Upon File Changes With FSNotify (#9810)
* refresh auth file

* refresh auth token from file changes

* gaz

* test for refresh

* rem token

* secret

* refresh

* remove wallet dir on test end
2021-10-28 10:24:39 -04:00
Radosław Kapka
bb319e02e8 Event support for contribution_and_proof and voluntary_exit (#9779)
* Event support for `contribution_and_prrof`

* event test

* fix panic in tests

* fix

* Revert "Auxiliary commit to revert individual files from dc8d01a15f0056c1fb48733219feab6461f71695"

This reverts commit f5f198564079781f80e1a045cefad7c27f89af25.

* remove receiver

* revive test

* move sending events to sync package

* remove receiver

* remove notification test

* build file

* notifier tests

* revert removal of exit event in API

* simplify exit test

* send notification in contribution API method

* test fix

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-10-28 08:56:22 +00:00
terence tsao
53c86429e4 Add altair test for RPC end point (#9829) 2021-10-27 21:00:09 -04:00
Raul Jordan
61172d5007 Add Missing Objects to Keymanager Protobuf (#9827)
* add missing objects

* update keymanager

* fix up missing sign calls

* gaz

* msg block root

* naming
2021-10-27 18:30:53 +00:00
Yash Bhutwala
1507719613 fix altair individual votes endpoint (#9825) 2021-10-27 15:15:51 +00:00
Raul Jordan
4c677e7b40 Remove JWT Expiration (#9813)
* remove expiry from claims

* fix tests

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-10-26 10:24:09 +00:00
Raul Jordan
28f50862cb Goimport All Items in Proto Folder In Bash Script (#9815)
* update goimports to format all, including gw.pb.go files

* update hack

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-10-26 10:07:49 +00:00
Nishant Das
2dfb0696f7 Fix Validator Exit in our V1 Method (#9819) 2021-10-26 10:14:11 +02:00
terence tsao
ae2c883aaf Generate secret key from big number test only (#9816)
* Generate secret key from big number test only

* Gazelle

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-10-25 19:14:19 +00:00
Raul Jordan
ad9ef9d803 Simplify Prysm Backend Password Requirements (#9814)
* simpler reqs

* gaz

* tidy mod
2021-10-25 13:17:47 -05:00
Kirill Fedoseev
b837f90b35 Fix GetDuties (#9811)
* Fix GetDuties

* Add regression test
2021-10-25 09:09:47 -05:00
Håvard Anda Estensen
7f3ec4221f Add errcheck and gosimple linters (#9729)
* Add errcheck linter

* Check unchecked error

* Add gosimple linter

* Remove type assertion to same type

* Omit nil check

len() for nil slices is defined as zero

* Revert "Remove type assertion to same type"

This reverts commit af69ca1ac8.

* Revert "Revert "Remove type assertion to same type""

This reverts commit 5fe8931504.

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-10-22 17:40:03 -05:00
Radosław Kapka
5b3375638a Improve description of datadir flag (#9809)
* Improve description of `datadir` flag

* improve doc
2021-10-22 12:53:13 +00:00
terence tsao
3c721418db Check participation flag offset length (#9784)
* Check participation flag offset length

* Fix test

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-10-21 15:01:09 +00:00
terence tsao
d7cad27cc3 Pool returns empty contribution slice instead of nil (#9808) 2021-10-20 14:45:57 -07:00
terence tsao
290b4273dd InitializePrecomputeValidators check overflows (#9807) 2021-10-20 13:06:43 -05:00
Raul Jordan
a797a7aaac Refactor Web Authentication to Use Auth Token (#9740)
* builds

* initialize auth token from scratch

* change auth

* add improvements

* moar auth fixes

* web auth changes running

* url encode

* tests

* gaz

* navigate auth

* separate line

* auth token

* 304

* persistent auth tokens and jwts

* fixed up test

* auth token path test and integration test

* auth token test

* auth token command

* gaz

* gaz

* Radek feedback

* fix up

* test

* Update validator/rpc/auth_token.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-10-20 16:37:05 +00:00
Raul Jordan
f7c34b0fd6 Validate Keystores Validator Client RPC Endpoint (#9799)
* validate endpoint

* validate keystores proto

* wallet validate

* validate keystores endpoint

* better err message

* added in gaz
2021-10-20 09:23:59 -05:00
Nishant Das
c6874e33f7 fix transitions (#9804) 2021-10-20 19:36:08 +08:00
Radosław Kapka
13ddc171eb Ignore validators without committee assignment when fetching attester duties (#9780)
* Ignore validators without committee assignment when fetching attester duties

* simplify

* fix tests

* slice capacity

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-10-19 17:41:12 +00:00
Radosław Kapka
acde184aa7 Fill out Version for SSZ block (#9801) 2021-10-19 16:45:38 +00:00
Radosław Kapka
5fd6474e56 Allow submitting sync committee subscriptions for next period (#9798)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-10-19 14:04:04 +00:00
Preston Van Loon
65db331eaf Remove bazel-go-ethereum fork, use upstream go-ethereum (#9725)
* Check in go-ethereum crypto/sepc256k1 package with proper build rules

* gaz

* Add karalabe/usb

* viz improvement

* Remove bazel-go-ethereum, use vendored libraries only

* move vendor stuff to third_party so that go mod wont be mad anymore

* fix geth e2e flags

* fix geth e2e flags

* remove old rules_foreign_cc toolchain

* Update cross compile docker image to support os x

* works for geth build

* remove copy of sepc256k1

* revert changes in tools/cross-toolchain

* gaz

* Update go-ethereum to 1.10.10

* Revert "revert changes in tools/cross-toolchain"

This reverts commit 2e8128f7c3.

* revert tags changes

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-10-19 00:00:22 +00:00
Radosław Kapka
7b8aedbfe4 Update Beacon API to v2.1.0 (#9797)
* `Eth-Consensus-Version` header

* rename unused receiver

* omit receiver name

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-10-18 18:55:18 +00:00
terence tsao
cf956c718d Update to Spectest 1.1.3 (#9786)
* Update transition spec test setup

* Update WORKSPACE

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-10-18 18:13:40 +00:00
Raul Jordan
975f0ea1af [Service Config Revamp] - Blockchain Initialization (#9783)
* viz

* unexport config

* builds

* viz

* viz

* register cfg

* fuzz

* blockchain opts

* deepsource

* rename flag opts
2021-10-18 17:48:05 +00:00
terence tsao
a80b1c252a Refactor rpc proposer into smaller files (#9796)
* Refactor rpc proposer into smaller files

* Update BUILD.bazel

* Fix test

* Update BUILD.bazel
2021-10-18 11:35:32 -05:00
Preston Van Loon
1f51e59bfd Bazel: minimal build config alias (#9794) 2021-10-18 14:22:57 +00:00
Nishant Das
bfcb113f78 Fix Doppelganger Protection (#9748)
* add fix

* fix tests

* Update beacon-chain/rpc/prysm/v1alpha1/validator/status.go

* fix tests

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-10-18 13:09:14 +00:00
terence tsao
20c7efda2c Process slashings return error on unknown state (#9778)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-10-18 12:40:47 +00:00
Radosław Kapka
f2990d8fdd Return errors for unknown block/state versions (#9781)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-10-18 12:18:49 +00:00
Nishant Das
d1f3050d20 Ignore G307 for Gosec (#9790) 2021-10-18 14:04:01 +02:00
Raul Jordan
4dbb5d6974 Add WebUI Security Headers (#9775) 2021-10-15 12:40:23 +02:00
Raul Jordan
59547aea66 Deregister Remote Slashing Protection Until Further Notice (#9774)
* avoid registering

* disable endpoints

* remote slasher protection register

* gaz

* fatal on external protection flag call

* radek comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-10-14 19:33:05 +00:00
Radosław Kapka
545424dd09 Fix epoch calculation when fetching sync committee duties (#9767)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-10-14 18:52:26 +00:00
Nishant Das
508b18f1bd Return Error For Batch Verifier (#9766)
* fix it

* make err check happy

* radek's review

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-10-13 23:52:22 +00:00
Radosław Kapka
e644e6b626 Remove special preparation of graffiti (#9770)
* add attestations to migration tests

* remove sepcial preparation of graffiti

* Revert "add attestations to migration tests"

This reverts commit adbe8cf4bf.
2021-10-13 15:32:15 +00:00
Radosław Kapka
280dc4ecf0 Register v1alpha2 endpoints in the gateway (#9768)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-10-12 15:13:52 +00:00
terence tsao
7a825a79ae E2e test: stricter participation check (#9718)
* Stricter participation check

* 0.99 is still better than 0.95...

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-10-12 11:32:36 +00:00
Nishant Das
b81f5fc7a5 Handle Invalid Number Of Leaves (#9761)
* fix bad trie

* handle edge case

* radek's review

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-10-12 10:36:57 +00:00
Nishant Das
06c084ff52 Log out Gossip Handling Errors (#9765) 2021-10-12 07:44:47 +00:00
Preston Van Loon
76e06438e9 Update bazel version to 4.x (#9763)
* Update to bazel 4.0.0

* bazel 4.2.1

* Regenerate crosstool configs

* restore manual tags

* restore manual tags
2021-10-12 06:17:24 +00:00
Nishant Das
f8f037b63d updateInterval (#9764) 2021-10-11 22:12:07 -05:00
terence tsao
f114a47b5b Improve "synced block.." log (#9760) 2021-10-09 15:27:34 +02:00
Nishant Das
65d2df4609 Add in Stronger Length Checks (#9758)
* add changes

* radek's review

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-10-08 17:41:36 +00:00
Radosław Kapka
63349d863b Do not return data for validators without sync duties (#9756)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-10-08 15:20:40 +00:00
Nishant Das
e2a00d3e2e Update to v1.1.2 version of the Spec (#9755) 2021-10-08 15:08:57 +02:00
terence tsao
271ee2ed32 Cleanup slot helpers into intended locations (#9752)
* Refactor slot time helpers to intended locations

* Gazelle

* Update beacon-chain/core/time/slot_epoch.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/core/time/slot_epoch_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Go fmt

* Update transition_fuzz_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-10-07 19:15:35 +00:00
Radosław Kapka
b5f0bd88b0 Add sync details to error messages (#9750) 2021-10-07 20:50:03 +02:00
terence tsao
e7085897ad Use BLS signature length config value (#9746)
* Use `BeaconConfig().BLSSignatureLength`

* Update BUILD.bazel

* Fix build

* Update BUILD.bazel
2021-10-07 10:37:53 -05:00
Raul Jordan
d9b98e9913 Health Endpoints Not Registered in Validator gRPC Gateway Fix (#9747) 2021-10-06 19:07:29 +00:00
terence tsao
a9f9026c78 Minor cleanups (#9743)
* Minor cleanups

* Delete slasher client files

* Revert "Delete slasher client files"

This reverts commit 0c995a1d4a.

* Update slasher_client.go
2021-10-06 13:23:40 -05:00
Nishant Das
b128d446f2 Bring Eth2FastAggregateVerify Inline With the Spec (#9742)
* fix

* tie it closer to the spec
2021-10-06 10:48:56 +08:00
Radosław Kapka
9aa50352b6 Allow fetching sync committee duties for current and next period's epochs (#9728) 2021-10-05 08:46:16 -07:00
Mohamed Zahoor
362dfa691a GetBlock() - Simple optimisations (#9541)
* GetBlock() optimizations

* addressed preston's comments

* fix fmt

* added benchmarks

* fix deepsource

* merge fix

* remove unnesesary check

* fixed terence's comments

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer.go

Co-authored-by: Nishant Das <nishdas93@gmail.com>

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-10-05 17:28:02 +08:00
1715 changed files with 102918 additions and 38287 deletions

View File

@@ -36,6 +36,9 @@ run --define blst_disabled=false
build:blst_disabled --define blst_disabled=true
build:blst_disabled --define gotags=blst_disabled
build:minimal --//proto:network=minimal
build:minimal --@io_bazel_rules_go//go/config:tags=minimal
# Release flags
build:release --compilation_mode=opt
build:release --config=llvm
@@ -59,20 +62,7 @@ build:llvm-asan --config=llvm
build:llvm-asan --config=asan
build:llvm-asan --linkopt -fuse-ld=ld.lld
build:fuzz --define=gotags=libfuzzer
build:fuzz --config=llvm-asan
build:fuzz --copt=-fsanitize=fuzzer-no-link
build:fuzz --linkopt=-fsanitize=fuzzer
build:fuzz --copt=-fno-omit-frame-pointer
build:fuzz --define=FUZZING_ENGINE=libfuzzer
build:fuzz --copt=-DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
build:fuzz --linkopt -Wl,--no-as-needed
build:fuzz --define=gc_goopts=-d=libfuzzer,checkptr
build:fuzz --run_under=//tools:fuzz_wrapper
build:fuzz --compilation_mode=opt
build:fuzz --define=blst_disabled=true
test:fuzz --local_test_jobs="HOST_CPUS*.5"
build:fuzz --@io_bazel_rules_go//go/config:tags=fuzz
# Build binary with cgo symbolizer for debugging / profiling.
build:cgo_symbolizer --config=llvm
@@ -227,3 +217,6 @@ build:remote --remote_local_fallback
# Ignore GoStdLib with remote caching
build --modify_execution_info='GoStdlib.*=+no-remote-cache'
# Set bazel gotag
build --define gotags=bazel

View File

@@ -1 +1 @@
3.7.0
5.0.0

View File

@@ -10,40 +10,37 @@
# Prysm specific remote-cache properties.
#build:remote-cache --disk_cache=
build:remote-cache --remote_download_toplevel
build:remote-cache --remote_download_minimal
build:remote-cache --remote_cache=grpc://bazel-remote-cache:9092
build:remote-cache --experimental_remote_downloader=grpc://bazel-remote-cache:9092
build:remote-cache --remote_local_fallback
build:remote-cache --experimental_remote_cache_async
build:remote-cache --experimental_remote_merkle_tree_cache
build:remote-cache --experimental_action_cache_store_output_metadata
build:remote-cache --experimental_remote_cache_compression
# Enforce stricter environment rules, which eliminates some non-hermetic
# behavior and therefore improves both the remote cache hit rate and the
# correctness and repeatability of the build.
build:remote-cache --incompatible_strict_action_env=true
build --experimental_use_hermetic_linux_sandbox
# Import workspace options.
import %workspace%/.bazelrc
startup --host_jvm_args=-Xmx2g --host_jvm_args=-Xms2g
query --repository_cache=/tmp/repositorycache
query --experimental_repository_cache_hardlinks
build --repository_cache=/tmp/repositorycache
build --experimental_repository_cache_hardlinks
startup --host_jvm_args=-Xmx4g --host_jvm_args=-Xms2g
build --experimental_strict_action_env
build --disk_cache=/tmp/bazelbuilds
build --experimental_multi_threaded_digest
build --sandbox_tmpfs_path=/tmp
build --verbose_failures
build --announce_rc
build --show_progress_rate_limit=5
build --curses=yes --color=no
build --curses=no --color=no
build --keep_going
build --test_output=errors
build --flaky_test_attempts=5
# Disabled race detection due to unstable test results under constrained environment build kite
# build --features=race
# Disable flaky test detection for fuzzing.
test:fuzz --flaky_test_attempts=1
# Better caching
build:nostamp --nostamp
build:nostamp --workspace_status_command=./hack/workspace_status_ci.sh

8
.github/CODEOWNERS vendored
View File

@@ -6,3 +6,11 @@
# Anyone on prylabs team can approve dependency updates.
deps.bzl @prysmaticlabs/core-team
# Radek and Nishant are responsible for changes that can affect the native state feature.
# See https://www.notion.so/prysmaticlabs/Native-Beacon-State-Redesign-6cc9744b4ec1439bb34fa829b36a35c1
/beacon-chain/state/fieldtrie/ @rkapka @nisdas
/beacon-chain/state/v1/ @rkapka @nisdas
/beacon-chain/state/v2/ @rkapka @nisdas
/beacon-chain/state/v3/ @rkapka @nisdas
/beacon-chain/state/state-native/ @rkapka @nisdas

View File

@@ -7,7 +7,7 @@ cd $GITHUB_WORKSPACE
cp go.mod go.mod.orig
cp go.sum go.sum.orig
go mod tidy
go mod tidy -compat=1.17
echo "Checking go.mod and go.sum:"
checks=0

View File

@@ -34,12 +34,17 @@ jobs:
- name: Gosec security scanner
uses: securego/gosec@master
with:
args: '-exclude-dir=crypto/bls/herumi ./...'
args: '-exclude=G307 -exclude-dir=crypto/bls/herumi ./...'
- name: Golangci-lint
uses: golangci/golangci-lint-action@v2
with:
args: --print-issued-lines --sort-results --no-config --timeout=10m --disable-all -E deadcode --skip-files=validator/web/site_data.go
args: --print-issued-lines --sort-results --no-config --timeout=10m --disable-all -E deadcode -E errcheck -E gosimple --skip-files=validator/web/site_data.go --skip-dirs=proto
- name: staticcheck
uses: dominikh/staticcheck-action@v1.1.0
with:
version: "2021.1.1"
build:
name: Build

4
.gitignore vendored
View File

@@ -30,9 +30,5 @@ password.txt
# Dist files
dist
# libfuzzer
oom-*
crash-*
# deepsource cli
bin

View File

@@ -5,21 +5,22 @@ Contact: mailto:security@prysmaticlabs.com
Encryption: openpgp4fpr:0AE0051D647BA3C1A917AF4072E33E4DF1A5036E
Encryption: openpgp4fpr:CD08DE68C60B82D3EE2A3F7D95452A701810FEDB
Encryption: openpgp4fpr:317D6E91058F8F3C2303BA7756313E44581297A6
Encryption: openpgp4fpr:79C59A585E3FD3AFFA00F5C22940A6479DA7C9EC
Preferred-Languages: en
Canonical: https://github.com/prysmaticlabs/prysm/tree/master/.well-known/security.txt
-----BEGIN PGP SIGNATURE-----
iQIzBAEBCgAdFiEECuAFHWR7o8GpF69AcuM+TfGlA24FAl++klgACgkQcuM+TfGl
A27rQw/6A29p1W20J0v+h218p8XWLSUpTIGLnZTxw6KqdyVXMzlsQK0YG4G2s2AB
0LKh7Ae/Di5E0U+Z4AjUW5nc5eaCxK36GMscH9Ah0rgJwNYxEJw7/2o8ZqVT/Ip2
+56rFihRqxFZfaCNKFVuZFaL9jKewV9FKYP38ID6/SnTcrOHiu2AoAlyZGmB03p+
iT57SPRHatygeY4xb/gwcfREFWEv+VHGyBTv8A+6ABZDxyurboCFMERHzFICrbmk
8UdHxxlWZDnHAbAUyAwpERC5znx6IHXQJwF8TMtu6XY6a6axT2XBOyJDF9/mZOz+
kdkz6loX5uxaQBGLtTv6Kqf1yUGANOZ16VhHvWwL209LmHmigIVQ+qSM6c79PsW/
vrsqdz3GBsiMC5Fq2vYgnbgzpfE8Atjn0y7E+j4R7IvwOAE/Ro/b++nqnc4YqhME
P/yTcfGftaCrdSNnQCXeoV9JxpFM5Xy8KV3eexvNKbcgA/9DtgxL5i+s5ZJkUT9A
+qJvoRrRyIym32ghkHgtFJKB3PLCdobeoOVRk6EnMo9zKSiSK2rZEJW8Ccbo515D
W9qUOn3GF7lNVuUFAU/YKEdmDp/AVaViZ7vH+8aq0LC0HBkZ8XlzWnWoArS8sMhw
fX0R9g/HMgrwNte/d0mwim5lJ2Plgv60Bh4grJqwZJeWbU0zi1U=
=uW+X
iQIzBAEBCgAdFiEECuAFHWR7o8GpF69AcuM+TfGlA24FAmGOfiYACgkQcuM+TfGl
A24YwRAAiQk3w6yzqSEggrOlNoNn04iu/rWZdn5ihkQgzACXy8XH2D1gdKLChE/X
7e5bUtgE2aCuHryQjwoKxqZakviBJFstVmHgF64rXv2zKhpqA30Mj4fI+T3zn8I+
+FpFV0TTsxNLDx+AcR1eQ1nSayO7ImUDIfOQNDDnSZZy42Bc+F+QIGKB3aH/8bpG
kT+bDTZrXvX+TE1gZTbAtZG8sH8g/zadoWEHIhfXUuYb0kTz+DRzAxoqU4j4Z4ee
1zSfFAgfJwxJP4kWD7s4xkE1sBbCgGBeD6cW/C2lbcfIei+XSizLpHW3jD9dNqh4
fLkmEspSa/LV/iXFq8nFzu/GLww4q+sQZDzzDKZyws54CrATinRitZMhzoIL0bTn
yFZVOGHosFAMEVZ36dl1Aw2+B2W6tr2CVr9c5zfV+kup5/KZH1EmT5nYY/zFwfg2
jYCFB5wmYeiyWZvuprgJXRArgVZLZaJxwWazlPVk4i/4vPvRgvfHqOwHCBe8DXy0
VHPhpewwb/ECYek1KoaNQflgR8iH2GMHkC5RjhGDAt1S0AQDtite5m4ZYt1kvO9E
k/znkv89dduhL9CKDvZvnI+DICwsTrf//4KJ8PM/qaPAJa4GvtiUU/eS/jKBivtv
OP5dZQtX6KPc9ewqqZgn622uHSezoBidgeTkdZsJ6tw2eIu0lsY=
=V7L0
-----END PGP SIGNATURE-----

View File

@@ -3,7 +3,7 @@ load("@com_github_atlassian_bazel_tools//gometalinter:def.bzl", "gometalinter")
load("@com_github_atlassian_bazel_tools//goimports:def.bzl", "goimports")
load("@io_kubernetes_build//defs:run_in_workspace.bzl", "workspace_binary")
load("@io_bazel_rules_go//go:def.bzl", "nogo")
load("@graknlabs_bazel_distribution//common:rules.bzl", "assemble_targz", "assemble_versioned")
load("@vaticle_bazel_distribution//common:rules.bzl", "assemble_targz", "assemble_versioned")
load("@bazel_skylib//rules:common_settings.bzl", "string_setting")
prefix = "github.com/prysmaticlabs/prysm"
@@ -16,6 +16,8 @@ exports_files([
# gazelle:map_kind go_library go_library @prysm//tools/go:def.bzl
# gazelle:map_kind go_test go_test @prysm//tools/go:def.bzl
# gazelle:map_kind go_repository go_repository @prysm//tools/go:def.bzl
# gazelle:build_tags bazel
# gazelle:exclude tools/analyzers/**/testdata/**
gazelle(
name = "gazelle",
prefix = prefix,
@@ -86,50 +88,52 @@ nogo(
config = "nogo_config.json",
visibility = ["//visibility:public"],
deps = [
"@org_golang_x_tools//go/analysis/passes/unsafeptr:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/unreachable:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/unmarshal:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/tests:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/structtag:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/stdmethods:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/shift:go_tool_library",
# "@org_golang_x_tools//go/analysis/passes/shadow:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/printf:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/pkgfact:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/nilness:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/nilfunc:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/loopclosure:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/httpresponse:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/findcall:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/deepequalerrors:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/ctrlflow:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/copylock:go_tool_library",
# "@org_golang_x_tools//go/analysis/passes/cgocall:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/buildtag:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/buildssa:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/bools:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/atomicalign:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/atomic:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/assign:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/inspect:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/asmdecl:go_tool_library",
"//tools/analyzers/maligned:go_tool_library",
"//tools/analyzers/cryptorand:go_tool_library",
"//tools/analyzers/errcheck:go_tool_library",
"//tools/analyzers/featureconfig:go_tool_library",
"//tools/analyzers/comparesame:go_tool_library",
"//tools/analyzers/shadowpredecl:go_tool_library",
"//tools/analyzers/nop:go_tool_library",
"//tools/analyzers/slicedirect:go_tool_library",
"//tools/analyzers/interfacechecker:go_tool_library",
"//tools/analyzers/ineffassign:go_tool_library",
"//tools/analyzers/properpermissions:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/unsafeptr:go_default_library",
"@org_golang_x_tools//go/analysis/passes/unreachable:go_default_library",
"@org_golang_x_tools//go/analysis/passes/unmarshal:go_default_library",
"@org_golang_x_tools//go/analysis/passes/tests:go_default_library",
"@org_golang_x_tools//go/analysis/passes/structtag:go_default_library",
"@org_golang_x_tools//go/analysis/passes/stdmethods:go_default_library",
"@org_golang_x_tools//go/analysis/passes/shift:go_default_library",
# "@org_golang_x_tools//go/analysis/passes/shadow:go_default_library",
"@org_golang_x_tools//go/analysis/passes/printf:go_default_library",
"@org_golang_x_tools//go/analysis/passes/pkgfact:go_default_library",
"@org_golang_x_tools//go/analysis/passes/nilness:go_default_library",
"@org_golang_x_tools//go/analysis/passes/nilfunc:go_default_library",
"@org_golang_x_tools//go/analysis/passes/loopclosure:go_default_library",
"@org_golang_x_tools//go/analysis/passes/httpresponse:go_default_library",
"@org_golang_x_tools//go/analysis/passes/findcall:go_default_library",
"@org_golang_x_tools//go/analysis/passes/deepequalerrors:go_default_library",
"@org_golang_x_tools//go/analysis/passes/ctrlflow:go_default_library",
"@org_golang_x_tools//go/analysis/passes/copylock:go_default_library",
# "@org_golang_x_tools//go/analysis/passes/cgocall:go_default_library",
"@org_golang_x_tools//go/analysis/passes/buildtag:go_default_library",
"@org_golang_x_tools//go/analysis/passes/buildssa:go_default_library",
"@org_golang_x_tools//go/analysis/passes/bools:go_default_library",
"@org_golang_x_tools//go/analysis/passes/atomicalign:go_default_library",
"@org_golang_x_tools//go/analysis/passes/atomic:go_default_library",
"@org_golang_x_tools//go/analysis/passes/assign:go_default_library",
"@org_golang_x_tools//go/analysis/passes/inspect:go_default_library",
"@org_golang_x_tools//go/analysis/passes/asmdecl:go_default_library",
"//tools/analyzers/maligned:go_default_library",
"//tools/analyzers/cryptorand:go_default_library",
"//tools/analyzers/errcheck:go_default_library",
"//tools/analyzers/featureconfig:go_default_library",
"//tools/analyzers/comparesame:go_default_library",
"//tools/analyzers/shadowpredecl:go_default_library",
"//tools/analyzers/nop:go_default_library",
"//tools/analyzers/slicedirect:go_default_library",
"//tools/analyzers/interfacechecker:go_default_library",
"//tools/analyzers/ineffassign:go_default_library",
"//tools/analyzers/properpermissions:go_default_library",
"//tools/analyzers/recursivelock:go_default_library",
"//tools/analyzers/uintcast:go_default_library",
] + select({
# nogo checks that fail with coverage enabled.
":coverage_enabled": [],
"//conditions:default": [
"@org_golang_x_tools//go/analysis/passes/lostcancel:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/composite:go_tool_library",
"@org_golang_x_tools//go/analysis/passes/lostcancel:go_default_library",
"@org_golang_x_tools//go/analysis/passes/composite:go_default_library",
],
}),
)
@@ -144,21 +148,6 @@ common_files = {
"//:README.md": "README.md",
}
toolchain(
name = "built_cmake_toolchain",
toolchain = "@rules_foreign_cc//tools/build_defs/native_tools:built_cmake",
toolchain_type = "@rules_foreign_cc//tools/build_defs:cmake_toolchain",
)
string_setting(
name = "gotags",
build_setting_default = "",
values = [
"",
"libfuzzer",
],
)
sh_binary(
name = "prysm_sh",
srcs = ["prysm.sh"],

View File

@@ -190,7 +190,7 @@ Anyone can become a part-time contributor and help out on implementing Ethereum
Core contributors are remote contractors of Prysmatic Labs, LLC. and are considered critical team members of our organization. Core devs have all of the responsibilities of part-time contributors plus the majority of the following:
- Stay up to date on the latest beacon chain sepcification
- Stay up to date on the latest beacon chain specification
- Monitor github issues and PRs to make sure owner, labels, descriptions are correct
- Formulate independent ideas, suggest new work to do, point out improvements to existing approaches
- Participate in code review, ensure code quality is excellent, and have ensure high code coverage

View File

@@ -2,7 +2,7 @@
[![Build status](https://badge.buildkite.com/b555891daf3614bae4284dcf365b2340cefc0089839526f096.svg?branch=master)](https://buildkite.com/prysmatic-labs/prysm)
[![Go Report Card](https://goreportcard.com/badge/github.com/prysmaticlabs/prysm)](https://goreportcard.com/report/github.com/prysmaticlabs/prysm)
[![ETH2.0_Spec_Version 1.0.0](https://img.shields.io/badge/ETH2.0%20Spec%20Version-v1.0.0-blue.svg)](https://github.com/ethereum/consensus-specs/tree/v1.0.0)
[![Consensus_Spec_Version 1.1.8](https://img.shields.io/badge/Consensus%20Spec%20Version-v1.1.8-blue.svg)](https://github.com/ethereum/consensus-specs/tree/v1.1.8)
[![Discord](https://user-images.githubusercontent.com/7288322/34471967-1df7808a-efbb-11e7-9088-ed0b04151291.png)](https://discord.gg/CTYGPUJ)
This is the core repository for Prysm, a [Golang](https://golang.org/) implementation of the [Ethereum Consensus](https://ethereum.org/en/eth2/) specification, developed by [Prysmatic Labs](https://prysmaticlabs.com). See the [Changelog](https://github.com/prysmaticlabs/prysm/releases) for details of the latest releases and upcoming breaking changes.

View File

@@ -60,10 +60,10 @@ bazel_skylib_workspace()
http_archive(
name = "bazel_gazelle",
sha256 = "62ca106be173579c0a167deb23358fdfe71ffa1e4cfdddf5582af26520f1c66f",
sha256 = "de69a09dc70417580aabf20a28619bb3ef60d038470c7cf8442fafcf627c21cb",
urls = [
"https://mirror.bazel.build/github.com/bazelbuild/bazel-gazelle/releases/download/v0.23.0/bazel-gazelle-v0.23.0.tar.gz",
"https://github.com/bazelbuild/bazel-gazelle/releases/download/v0.23.0/bazel-gazelle-v0.23.0.tar.gz",
"https://mirror.bazel.build/github.com/bazelbuild/bazel-gazelle/releases/download/v0.24.0/bazel-gazelle-v0.24.0.tar.gz",
"https://github.com/bazelbuild/bazel-gazelle/releases/download/v0.24.0/bazel-gazelle-v0.24.0.tar.gz",
],
)
@@ -85,16 +85,13 @@ http_archive(
name = "io_bazel_rules_go",
patch_args = ["-p1"],
patches = [
# Required until https://github.com/bazelbuild/rules_go/pull/2450 merges otherwise nilness
# nogo check fails for certain third_party dependencies.
"//third_party:io_bazel_rules_go.patch",
# Expose internals of go_test for custom build transitions.
"//third_party:io_bazel_rules_go_test.patch",
],
sha256 = "7c10271940c6bce577d51a075ae77728964db285dac0a46614a7934dc34303e6",
sha256 = "2b1641428dff9018f9e85c0384f03ec6c10660d935b750e3fa1492a281a53b0f",
urls = [
"https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.26.0/rules_go-v0.26.0.tar.gz",
"https://github.com/bazelbuild/rules_go/releases/download/v0.26.0/rules_go-v0.26.0.tar.gz",
"https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.29.0/rules_go-v0.29.0.zip",
"https://github.com/bazelbuild/rules_go/releases/download/v0.29.0/rules_go-v0.29.0.zip",
],
)
@@ -120,13 +117,6 @@ http_archive(
urls = ["https://github.com/fuzzitdev/fuzzit/releases/download/v2.4.76/fuzzit_Linux_x86_64.zip"],
)
git_repository(
name = "graknlabs_bazel_distribution",
commit = "962f3a7e56942430c0ec120c24f9e9f2a9c2ce1a",
remote = "https://github.com/graknlabs/bazel-distribution",
shallow_since = "1569509514 +0300",
)
load(
"@io_bazel_rules_docker//repositories:repositories.bzl",
container_repositories = "repositories",
@@ -186,7 +176,7 @@ load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_depe
go_rules_dependencies()
go_register_toolchains(
go_version = "1.16.4",
go_version = "1.17.6",
nogo = "@//:nogo",
)
@@ -222,10 +212,10 @@ filegroup(
)
""",
sha256 = "91434d5fd5e1c6eb7b0174fed2afe25e09bddf00e1e4c431db931b2cee4e7773",
url = "https://github.com/eth2-clients/slashing-protection-interchange-tests/archive/b8413ca42dc92308019d0d4db52c87e9e125c4e9.tar.gz",
url = "https://github.com/eth-clients/slashing-protection-interchange-tests/archive/b8413ca42dc92308019d0d4db52c87e9e125c4e9.tar.gz",
)
consensus_spec_version = "v1.1.0"
consensus_spec_version = "v1.1.10"
bls_test_version = "v0.1.1"
@@ -241,7 +231,7 @@ filegroup(
visibility = ["//visibility:public"],
)
""",
sha256 = "0f58ec1c8995ab2c0a3aada873e1a02f333c8659fdacaa1c76fbdce28b6177dc",
sha256 = "28043009cc2f6fc9804e73c8c1fc2cb27062f1591e6884f3015ae1dd7a276883",
url = "https://github.com/ethereum/consensus-spec-tests/releases/download/%s/general.tar.gz" % consensus_spec_version,
)
@@ -257,7 +247,7 @@ filegroup(
visibility = ["//visibility:public"],
)
""",
sha256 = "d2eeadef3f8885748f769a06eb916bea60fb00112a90cec3288ea78db5557df3",
sha256 = "bc1a283ca068f310f04d70c4f6a8eaa0b8f7e9318073a8bdc2ee233111b4e339",
url = "https://github.com/ethereum/consensus-spec-tests/releases/download/%s/minimal.tar.gz" % consensus_spec_version,
)
@@ -273,7 +263,7 @@ filegroup(
visibility = ["//visibility:public"],
)
""",
sha256 = "bbc345432f4eaa4babe2619e16fcf3f607113ede82d1bd0fff2633c1376419f7",
sha256 = "bbabb482c229ff9d4e2c7b77c992edb452f9d0af7c6d8dd4f922f06a7b101e81",
url = "https://github.com/ethereum/consensus-spec-tests/releases/download/%s/mainnet.tar.gz" % consensus_spec_version,
)
@@ -288,7 +278,7 @@ filegroup(
visibility = ["//visibility:public"],
)
""",
sha256 = "787a50c2a03bebab92f089333f598780ca7b4849b36a4897e6603ddd3e86ec24",
sha256 = "408a5524548ad3fcf387f65ac7ec52781d9ee899499720bb12451b48a15818d4",
strip_prefix = "consensus-specs-" + consensus_spec_version[1:],
url = "https://github.com/ethereum/consensus-specs/archive/refs/tags/%s.tar.gz" % consensus_spec_version,
)
@@ -319,9 +309,9 @@ filegroup(
visibility = ["//visibility:public"],
)
""",
sha256 = "9dc47bf6b14aed7fac8833e35ab83a69131b43fa5789b3256bf1ac3d4861aeb8",
strip_prefix = "eth2-networks-7fa1b868985ee24aad65567f9250cf7fa86f97b1",
url = "https://github.com/eth2-clients/eth2-networks/archive/7fa1b868985ee24aad65567f9250cf7fa86f97b1.tar.gz",
sha256 = "4e8a18b21d056c4032605621b1a6632198eabab57cb90c61e273f344c287f1b2",
strip_prefix = "eth2-networks-791a5369c5981e829698b17fbcdcdacbdaba97c8",
url = "https://github.com/eth-clients/eth2-networks/archive/791a5369c5981e829698b17fbcdcdacbdaba97c8.tar.gz",
)
http_archive(
@@ -341,16 +331,6 @@ git_repository(
# Group the sources of the library so that CMake rule have access to it
all_content = """filegroup(name = "all", srcs = glob(["**"]), visibility = ["//visibility:public"])"""
http_archive(
name = "sigp_beacon_fuzz_corpora",
build_file = "//third_party:beacon-fuzz/corpora.BUILD",
sha256 = "42993d0901a316afda45b4ba6d53c7c21f30c551dcec290a4ca131c24453d1ef",
strip_prefix = "beacon-fuzz-corpora-bac24ad78d45cc3664c0172241feac969c1ac29b",
urls = [
"https://github.com/sigp/beacon-fuzz-corpora/archive/bac24ad78d45cc3664c0172241feac969c1ac29b.tar.gz",
],
)
# External dependencies
http_archive(
@@ -362,9 +342,9 @@ filegroup(
visibility = ["//visibility:public"],
)
""",
sha256 = "54ce527b83d092da01127f2e3816f4d5cfbab69354caba8537f1ea55889b6d7c",
sha256 = "4797a7e594a5b1f4c1c8080701613f3ee451b01ec0861499ea7d9b60877a6b23",
urls = [
"https://github.com/prysmaticlabs/prysm-web-ui/releases/download/v1.0.0-beta.4/prysm-web-ui.tar.gz",
"https://github.com/prysmaticlabs/prysm-web-ui/releases/download/v1.0.3/prysm-web-ui.tar.gz",
],
)
@@ -377,6 +357,10 @@ load("@prysm//third_party/herumi:herumi.bzl", "bls_dependencies")
bls_dependencies()
load("@prysm//testing/endtoend:deps.bzl", "e2e_deps")
e2e_deps()
load(
"@io_bazel_rules_docker//go:image.bzl",
_go_image_repos = "repositories",
@@ -395,10 +379,6 @@ load(
_cc_image_repos()
load("@com_github_ethereum_go_ethereum//:deps.bzl", "geth_dependencies")
geth_dependencies()
load("@io_bazel_rules_go//extras:embed_data_deps.bzl", "go_embed_data_dependencies")
go_embed_data_dependencies()

View File

@@ -0,0 +1,54 @@
load("@prysm//tools/go:def.bzl", "go_library", "go_test")
go_library(
name = "go_default_library",
srcs = [
"checkpoint.go",
"client.go",
"doc.go",
"errors.go",
],
importpath = "github.com/prysmaticlabs/prysm/api/client/beacon",
visibility = ["//visibility:public"],
deps = [
"//beacon-chain/core/helpers:go_default_library",
"//beacon-chain/rpc/apimiddleware:go_default_library",
"//beacon-chain/state:go_default_library",
"//encoding/bytesutil:go_default_library",
"//encoding/ssz/detect:go_default_library",
"//io/file:go_default_library",
"//network/forks:go_default_library",
"//proto/prysm/v1alpha1:go_default_library",
"//proto/prysm/v1alpha1/block:go_default_library",
"//runtime/version:go_default_library",
"//time/slots:go_default_library",
"@com_github_ethereum_go_ethereum//common/hexutil:go_default_library",
"@com_github_pkg_errors//:go_default_library",
"@com_github_prysmaticlabs_eth2_types//:go_default_library",
"@com_github_sirupsen_logrus//:go_default_library",
"@org_golang_x_mod//semver:go_default_library",
],
)
go_test(
name = "go_default_test",
srcs = [
"checkpoint_test.go",
"client_test.go",
],
embed = [":go_default_library"],
deps = [
"//beacon-chain/state:go_default_library",
"//config/params:go_default_library",
"//encoding/ssz/detect:go_default_library",
"//network/forks:go_default_library",
"//proto/prysm/v1alpha1:go_default_library",
"//proto/prysm/v1alpha1/wrapper:go_default_library",
"//runtime/version:go_default_library",
"//testing/require:go_default_library",
"//testing/util:go_default_library",
"//time/slots:go_default_library",
"@com_github_pkg_errors//:go_default_library",
"@com_github_prysmaticlabs_eth2_types//:go_default_library",
],
)

View File

@@ -0,0 +1,262 @@
package beacon
import (
"context"
"fmt"
"path"
"github.com/pkg/errors"
types "github.com/prysmaticlabs/eth2-types"
"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
"github.com/prysmaticlabs/prysm/beacon-chain/state"
"github.com/prysmaticlabs/prysm/encoding/ssz/detect"
"github.com/prysmaticlabs/prysm/io/file"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/block"
"github.com/prysmaticlabs/prysm/runtime/version"
"github.com/prysmaticlabs/prysm/time/slots"
log "github.com/sirupsen/logrus"
"golang.org/x/mod/semver"
)
// OriginData represents the BeaconState and SignedBeaconBlock necessary to start an empty Beacon Node
// using Checkpoint Sync.
type OriginData struct {
wsd *WeakSubjectivityData
sb []byte
bb []byte
st state.BeaconState
b block.SignedBeaconBlock
cf *detect.VersionedUnmarshaler
}
// CheckpointString returns the standard string representation of a Checkpoint for the block root and epoch for the
// SignedBeaconBlock value found by DownloadOriginData.
// The format is a a hex-encoded block root, followed by the epoch of the block, separated by a colon. For example:
// "0x1c35540cac127315fabb6bf29181f2ae0de1a3fc909d2e76ba771e61312cc49a:74888"
func (od *OriginData) CheckpointString() string {
return fmt.Sprintf("%#x:%d", od.wsd.BlockRoot, od.wsd.Epoch)
}
// SaveBlock saves the downloaded block to a unique file in the given path.
// For readability and collision avoidance, the file name includes: type, config name, slot and root
func (od *OriginData) SaveBlock(dir string) (string, error) {
blockPath := path.Join(dir, fname("state", od.cf, od.st.Slot(), od.wsd.BlockRoot))
return blockPath, file.WriteFile(blockPath, od.sb)
}
// SaveState saves the downloaded state to a unique file in the given path.
// For readability and collision avoidance, the file name includes: type, config name, slot and root
func (od *OriginData) SaveState(dir string) (string, error) {
statePath := path.Join(dir, fname("state", od.cf, od.st.Slot(), od.wsd.StateRoot))
return statePath, file.WriteFile(statePath, od.sb)
}
// StateBytes returns the ssz-encoded bytes of the downloaded BeaconState value.
func (od *OriginData) StateBytes() []byte {
return od.sb
}
// BlockBytes returns the ssz-encoded bytes of the downloaded SignedBeaconBlock value.
func (od *OriginData) BlockBytes() []byte {
return od.bb
}
func fname(prefix string, cf *detect.VersionedUnmarshaler, slot types.Slot, root [32]byte) string {
return fmt.Sprintf("%s_%s_%s_%d-%#x.ssz", prefix, cf.Config.ConfigName, version.String(cf.Fork), slot, root)
}
// this method downloads the head state, which can be used to find the correct chain config
// and use prysm's helper methods to compute the latest weak subjectivity epoch.
func getWeakSubjectivityEpochFromHead(ctx context.Context, client *Client) (types.Epoch, error) {
headBytes, err := client.GetState(ctx, IdHead)
if err != nil {
return 0, err
}
cf, err := detect.FromState(headBytes)
if err != nil {
return 0, errors.Wrap(err, "error detecting chain config for beacon state")
}
log.Printf("detected supported config in remote head state, name=%s, fork=%s", cf.Config.ConfigName, version.String(cf.Fork))
headState, err := cf.UnmarshalBeaconState(headBytes)
if err != nil {
return 0, errors.Wrap(err, "error unmarshaling state to correct version")
}
epoch, err := helpers.LatestWeakSubjectivityEpoch(ctx, headState, cf.Config)
if err != nil {
return 0, errors.Wrap(err, "error computing the weak subjectivity epoch from head state")
}
log.Printf("(computed client-side) weak subjectivity epoch = %d", epoch)
return epoch, nil
}
const (
prysmMinimumVersion = "v2.0.7"
prysmImplementationName = "Prysm"
)
// ErrUnsupportedPrysmCheckpointVersion indicates remote beacon node can't be used for checkpoint retrieval.
var ErrUnsupportedPrysmCheckpointVersion = errors.New("node does not meet minimum version requirements for checkpoint retrieval")
// for older endpoints or clients that do not support the weak_subjectivity api method
// we gather the necessary data for a checkpoint sync by:
// - inspecting the remote server's head state and computing the weak subjectivity epoch locally
// - requesting the state at the first slot of the epoch
// - using hash_tree_root(state.latest_block_header) to compute the block the state integrates
// - requesting that block by its root
func downloadBackwardsCompatible(ctx context.Context, client *Client) (*OriginData, error) {
log.Print("falling back to generic checkpoint derivation, weak_subjectivity API not supported by server")
nv, err := client.GetNodeVersion(ctx)
if err != nil {
return nil, errors.Wrap(err, "unable to proceed with fallback method without confirming node version")
}
if nv.implementation == prysmImplementationName && semver.Compare(nv.semver, prysmMinimumVersion) < 0 {
return nil, errors.Wrapf(ErrUnsupportedPrysmCheckpointVersion, "%s < minimum (%s)", nv.semver, prysmMinimumVersion)
}
epoch, err := getWeakSubjectivityEpochFromHead(ctx, client)
if err != nil {
return nil, errors.Wrap(err, "error computing weak subjectivity epoch via head state inspection")
}
// use first slot of the epoch for the state slot
slot, err := slots.EpochStart(epoch)
if err != nil {
return nil, errors.Wrapf(err, "error computing first slot of epoch=%d", epoch)
}
log.Printf("requesting checkpoint state at slot %d", slot)
// get the state at the first slot of the epoch
stateBytes, err := client.GetState(ctx, IdFromSlot(slot))
if err != nil {
return nil, errors.Wrapf(err, "failed to request state by slot from api, slot=%d", slot)
}
// ConfigFork is used to unmarshal the BeaconState so we can read the block root in latest_block_header
cf, err := detect.FromState(stateBytes)
if err != nil {
return nil, errors.Wrap(err, "error detecting chain config for beacon state")
}
log.Printf("detected supported config in checkpoint state, name=%s, fork=%s", cf.Config.ConfigName, version.String(cf.Fork))
st, err := cf.UnmarshalBeaconState(stateBytes)
if err != nil {
return nil, errors.Wrap(err, "error using detected config fork to unmarshal state bytes")
}
// compute state and block roots
stateRoot, err := st.HashTreeRoot(ctx)
if err != nil {
return nil, errors.Wrap(err, "error computing hash_tree_root of state")
}
header := st.LatestBlockHeader()
header.StateRoot = stateRoot[:]
computedBlockRoot, err := header.HashTreeRoot()
if err != nil {
return nil, errors.Wrap(err, "error while computing block root using state data")
}
blockBytes, err := client.GetBlock(ctx, IdFromRoot(computedBlockRoot))
if err != nil {
return nil, errors.Wrapf(err, "error requesting block by root = %d", computedBlockRoot)
}
block, err := cf.UnmarshalBeaconBlock(blockBytes)
if err != nil {
return nil, errors.Wrap(err, "unable to unmarshal block to a supported type using the detected fork schedule")
}
blockRoot, err := block.Block().HashTreeRoot()
if err != nil {
return nil, errors.Wrap(err, "error computing hash_tree_root for block obtained via root")
}
log.Printf("BeaconState slot=%d, Block slot=%d", st.Slot(), block.Block().Slot())
log.Printf("BeaconState htr=%#xd, Block state_root=%#x", stateRoot, block.Block().StateRoot())
log.Printf("BeaconBlock root computed from state=%#x, Block htr=%#x", computedBlockRoot, blockRoot)
return &OriginData{
wsd: &WeakSubjectivityData{
BlockRoot: blockRoot,
StateRoot: stateRoot,
Epoch: epoch,
},
st: st,
sb: stateBytes,
b: block,
bb: blockBytes,
cf: cf,
}, nil
}
// DownloadOriginData attempts to use the proposed weak_subjectivity beacon node api
// to obtain the weak_subjectivity metadata (epoch, block_root, state_root) needed to sync
// a beacon node from the canonical weak subjectivity checkpoint. As this is a proposed API
// that will only be supported by prysm at first, in the event of a 404 we fallback to using a
// different technique where we first download the head state which can be used to compute the
// weak subjectivity epoch on the client side.
func DownloadOriginData(ctx context.Context, client *Client) (*OriginData, error) {
ws, err := client.GetWeakSubjectivity(ctx)
if err != nil {
// a 404/405 is expected if querying an endpoint that doesn't support the weak subjectivity checkpoint api
if !errors.Is(err, ErrNotOK) {
return nil, errors.Wrap(err, "unexpected API response for prysm-only weak subjectivity checkpoint API")
}
// fall back to vanilla Beacon Node API method
return downloadBackwardsCompatible(ctx, client)
}
log.Printf("server weak subjectivity checkpoint response - epoch=%d, block_root=%#x, state_root=%#x", ws.Epoch, ws.BlockRoot, ws.StateRoot)
// use first slot of the epoch for the block slot
slot, err := slots.EpochStart(ws.Epoch)
if err != nil {
return nil, errors.Wrapf(err, "error computing first slot of epoch=%d", ws.Epoch)
}
log.Printf("requesting checkpoint state at slot %d", slot)
stateBytes, err := client.GetState(ctx, IdFromSlot(slot))
if err != nil {
return nil, errors.Wrapf(err, "failed to request state by slot from api, slot=%d", slot)
}
cf, err := detect.FromState(stateBytes)
if err != nil {
return nil, errors.Wrap(err, "error detecting chain config for beacon state")
}
log.Printf("detected supported config in checkpoint state, name=%s, fork=%s", cf.Config.ConfigName, version.String(cf.Fork))
state, err := cf.UnmarshalBeaconState(stateBytes)
if err != nil {
return nil, errors.Wrap(err, "error using detected config fork to unmarshal state bytes")
}
stateRoot, err := state.HashTreeRoot(ctx)
if err != nil {
return nil, errors.Wrapf(err, "failed to compute htr for state at slot=%d", slot)
}
blockRoot, err := state.LatestBlockHeader().HashTreeRoot()
if err != nil {
return nil, errors.Wrap(err, "error computing hash_tree_root of latest_block_header")
}
blockBytes, err := client.GetBlock(ctx, IdFromRoot(ws.BlockRoot))
if err != nil {
return nil, errors.Wrapf(err, "error requesting block by slot = %d", slot)
}
block, err := cf.UnmarshalBeaconBlock(blockBytes)
if err != nil {
return nil, errors.Wrap(err, "unable to unmarshal block to a supported type using the detected fork schedule")
}
realBlockRoot, err := block.Block().HashTreeRoot()
if err != nil {
return nil, errors.Wrap(err, "error computing hash_tree_root of retrieved block")
}
log.Printf("BeaconState slot=%d, Block slot=%d", state.Slot(), block.Block().Slot())
log.Printf("BeaconState htr=%#xd, Block state_root=%#x", stateRoot, block.Block().StateRoot())
log.Printf("BeaconState latest_block_header htr=%#xd, block htr=%#x", blockRoot, realBlockRoot)
return &OriginData{
wsd: ws,
st: state,
b: block,
sb: stateBytes,
bb: blockBytes,
cf: cf,
}, nil
}

View File

@@ -0,0 +1,407 @@
package beacon
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"testing"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/wrapper"
"github.com/prysmaticlabs/prysm/beacon-chain/state"
"github.com/prysmaticlabs/prysm/network/forks"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/testing/util"
"github.com/prysmaticlabs/prysm/time/slots"
types "github.com/prysmaticlabs/eth2-types"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/encoding/ssz/detect"
"github.com/prysmaticlabs/prysm/runtime/version"
"github.com/pkg/errors"
"github.com/prysmaticlabs/prysm/testing/require"
)
type testRT struct {
rt func(*http.Request) (*http.Response, error)
}
func (rt *testRT) RoundTrip(req *http.Request) (*http.Response, error) {
if rt.rt != nil {
return rt.rt(req)
}
return nil, errors.New("RoundTripper not implemented")
}
var _ http.RoundTripper = &testRT{}
func marshalToEnvelope(val interface{}) ([]byte, error) {
raw, err := json.Marshal(val)
if err != nil {
return nil, errors.Wrap(err, "error marshaling value to place in data envelope")
}
env := struct {
Data json.RawMessage `json:"data"`
}{
Data: raw,
}
return json.Marshal(env)
}
func TestMarshalToEnvelope(t *testing.T) {
d := struct {
Version string `json:"version"`
}{
Version: "Prysm/v2.0.5 (linux amd64)",
}
encoded, err := marshalToEnvelope(d)
require.NoError(t, err)
expected := `{"data":{"version":"Prysm/v2.0.5 (linux amd64)"}}`
require.Equal(t, expected, string(encoded))
}
func TestFallbackVersionCheck(t *testing.T) {
c := &Client{
hc: &http.Client{},
host: "localhost:3500",
scheme: "http",
}
c.hc.Transport = &testRT{rt: func(req *http.Request) (*http.Response, error) {
res := &http.Response{Request: req}
switch req.URL.Path {
case getNodeVersionPath:
res.StatusCode = http.StatusOK
b := bytes.NewBuffer(nil)
d := struct {
Version string `json:"version"`
}{
Version: "Prysm/v2.0.5 (linux amd64)",
}
encoded, err := marshalToEnvelope(d)
require.NoError(t, err)
b.Write(encoded)
res.Body = io.NopCloser(b)
case getWeakSubjectivityPath:
res.StatusCode = http.StatusNotFound
}
return res, nil
}}
ctx := context.Background()
_, err := DownloadOriginData(ctx, c)
require.ErrorIs(t, err, ErrUnsupportedPrysmCheckpointVersion)
}
func TestFname(t *testing.T) {
vu := &detect.VersionedUnmarshaler{
Config: params.MainnetConfig(),
Fork: version.Phase0,
}
slot := types.Slot(23)
prefix := "block"
var root [32]byte
copy(root[:], []byte{0x23, 0x23, 0x23})
expected := "block_mainnet_phase0_23-0x2323230000000000000000000000000000000000000000000000000000000000.ssz"
actual := fname(prefix, vu, slot, root)
require.Equal(t, expected, actual)
vu.Config = params.MinimalSpecConfig()
vu.Fork = version.Altair
slot = 17
prefix = "state"
copy(root[29:], []byte{0x17, 0x17, 0x17})
expected = "state_minimal_altair_17-0x2323230000000000000000000000000000000000000000000000000000171717.ssz"
actual = fname(prefix, vu, slot, root)
require.Equal(t, expected, actual)
}
func TestDownloadOriginData(t *testing.T) {
ctx := context.Background()
cfg := params.MainnetConfig()
epoch := cfg.AltairForkEpoch - 1
// set up checkpoint state, using the epoch that will be computed as the ws checkpoint state based on the head state
wSlot, err := slots.EpochStart(epoch)
require.NoError(t, err)
wst, err := util.NewBeaconState()
require.NoError(t, err)
fork, err := forkForEpoch(cfg, epoch)
require.NoError(t, wst.SetFork(fork))
// set up checkpoint block
b, err := wrapper.WrappedSignedBeaconBlock(util.NewBeaconBlock())
require.NoError(t, wrapper.SetBlockParentRoot(b, cfg.ZeroHash))
require.NoError(t, wrapper.SetBlockSlot(b, wSlot))
require.NoError(t, wrapper.SetProposerIndex(b, 0))
// set up state header pointing at checkpoint block - this is how the block is downloaded by root
header, err := b.Header()
require.NoError(t, err)
require.NoError(t, wst.SetLatestBlockHeader(header.Header))
// order of operations can be confusing here:
// - when computing the state root, make sure block header is complete, EXCEPT the state root should be zero-value
// - before computing the block root (to match the request route), the block should include the state root
// *computed from the state with a header that does not have a state root set yet*
wRoot, err := wst.HashTreeRoot(ctx)
require.NoError(t, err)
require.NoError(t, wrapper.SetBlockStateRoot(b, wRoot))
serBlock, err := b.MarshalSSZ()
require.NoError(t, err)
bRoot, err := b.Block().HashTreeRoot()
require.NoError(t, err)
wsSerialized, err := wst.MarshalSSZ()
require.NoError(t, err)
expectedWSD := WeakSubjectivityData{
BlockRoot: bRoot,
StateRoot: wRoot,
Epoch: epoch,
}
hc := &http.Client{
Transport: &testRT{rt: func(req *http.Request) (*http.Response, error) {
res := &http.Response{Request: req}
switch req.URL.Path {
case getWeakSubjectivityPath:
res.StatusCode = http.StatusOK
cp := struct {
Epoch string `json:"epoch"`
Root string `json:"root"`
}{
Epoch: fmt.Sprintf("%d", slots.ToEpoch(b.Block().Slot())),
Root: fmt.Sprintf("%#x", bRoot),
}
wsr := struct {
Checkpoint interface{} `json:"ws_checkpoint"`
StateRoot string `json:"state_root"`
}{
Checkpoint: cp,
StateRoot: fmt.Sprintf("%#x", wRoot),
}
rb, err := marshalToEnvelope(wsr)
require.NoError(t, err)
res.Body = io.NopCloser(bytes.NewBuffer(rb))
case renderGetStatePath(IdFromSlot(wSlot)):
res.StatusCode = http.StatusOK
res.Body = io.NopCloser(bytes.NewBuffer(wsSerialized))
case renderGetBlockPath(IdFromRoot(bRoot)):
res.StatusCode = http.StatusOK
res.Body = io.NopCloser(bytes.NewBuffer(serBlock))
}
return res, nil
}},
}
c := &Client{
hc: hc,
host: "localhost:3500",
scheme: "http",
}
od, err := DownloadOriginData(ctx, c)
require.NoError(t, err)
require.Equal(t, expectedWSD.Epoch, od.wsd.Epoch)
require.Equal(t, expectedWSD.StateRoot, od.wsd.StateRoot)
require.Equal(t, expectedWSD.BlockRoot, od.wsd.BlockRoot)
require.DeepEqual(t, wsSerialized, od.sb)
require.DeepEqual(t, serBlock, od.bb)
require.DeepEqual(t, wst.Fork().CurrentVersion, od.cf.Version[:])
require.DeepEqual(t, version.Phase0, od.cf.Fork)
}
// runs downloadBackwardsCompatible directly
// and via DownloadOriginData with a round tripper that triggers the backwards compatible code path
func TestDownloadBackwardsCompatibleCombined(t *testing.T) {
ctx := context.Background()
cfg := params.MainnetConfig()
st, expectedEpoch := defaultTestHeadState(t, cfg)
serialized, err := st.MarshalSSZ()
require.NoError(t, err)
// set up checkpoint state, using the epoch that will be computed as the ws checkpoint state based on the head state
wSlot, err := slots.EpochStart(expectedEpoch)
require.NoError(t, err)
wst, err := util.NewBeaconState()
require.NoError(t, err)
fork, err := forkForEpoch(cfg, cfg.GenesisEpoch)
require.NoError(t, wst.SetFork(fork))
// set up checkpoint block
b, err := wrapper.WrappedSignedBeaconBlock(util.NewBeaconBlock())
require.NoError(t, wrapper.SetBlockParentRoot(b, cfg.ZeroHash))
require.NoError(t, wrapper.SetBlockSlot(b, wSlot))
require.NoError(t, wrapper.SetProposerIndex(b, 0))
// set up state header pointing at checkpoint block - this is how the block is downloaded by root
header, err := b.Header()
require.NoError(t, err)
require.NoError(t, wst.SetLatestBlockHeader(header.Header))
// order of operations can be confusing here:
// - when computing the state root, make sure block header is complete, EXCEPT the state root should be zero-value
// - before computing the block root (to match the request route), the block should include the state root
// *computed from the state with a header that does not have a state root set yet*
wRoot, err := wst.HashTreeRoot(ctx)
require.NoError(t, err)
require.NoError(t, wrapper.SetBlockStateRoot(b, wRoot))
serBlock, err := b.MarshalSSZ()
require.NoError(t, err)
bRoot, err := b.Block().HashTreeRoot()
require.NoError(t, err)
wsSerialized, err := wst.MarshalSSZ()
require.NoError(t, err)
hc := &http.Client{
Transport: &testRT{rt: func(req *http.Request) (*http.Response, error) {
res := &http.Response{Request: req}
switch req.URL.Path {
case getNodeVersionPath:
res.StatusCode = http.StatusOK
b := bytes.NewBuffer(nil)
d := struct {
Version string `json:"version"`
}{
Version: "Lighthouse/v0.1.5 (Linux x86_64)",
}
encoded, err := marshalToEnvelope(d)
require.NoError(t, err)
b.Write(encoded)
res.Body = io.NopCloser(b)
case getWeakSubjectivityPath:
res.StatusCode = http.StatusNotFound
case renderGetStatePath(IdHead):
res.StatusCode = http.StatusOK
res.Body = io.NopCloser(bytes.NewBuffer(serialized))
case renderGetStatePath(IdFromSlot(wSlot)):
res.StatusCode = http.StatusOK
res.Body = io.NopCloser(bytes.NewBuffer(wsSerialized))
case renderGetBlockPath(IdFromRoot(bRoot)):
res.StatusCode = http.StatusOK
res.Body = io.NopCloser(bytes.NewBuffer(serBlock))
}
return res, nil
}},
}
c := &Client{
hc: hc,
host: "localhost:3500",
scheme: "http",
}
odPub, err := DownloadOriginData(ctx, c)
require.NoError(t, err)
odPriv, err := downloadBackwardsCompatible(ctx, c)
require.NoError(t, err)
require.DeepEqual(t, odPriv.wsd, odPub.wsd)
require.DeepEqual(t, odPriv.sb, odPub.sb)
require.DeepEqual(t, odPriv.bb, odPub.bb)
require.DeepEqual(t, odPriv.cf.Fork, odPub.cf.Fork)
require.DeepEqual(t, odPriv.cf.Version, odPub.cf.Version)
}
func TestGetWeakSubjectivityEpochFromHead(t *testing.T) {
st, expectedEpoch := defaultTestHeadState(t, params.MainnetConfig())
serialized, err := st.MarshalSSZ()
require.NoError(t, err)
hc := &http.Client{
Transport: &testRT{rt: func(req *http.Request) (*http.Response, error) {
res := &http.Response{Request: req}
switch req.URL.Path {
case renderGetStatePath(IdHead):
res.StatusCode = http.StatusOK
res.Body = io.NopCloser(bytes.NewBuffer(serialized))
}
return res, nil
}},
}
c := &Client{
hc: hc,
host: "localhost:3500",
scheme: "http",
}
actualEpoch, err := getWeakSubjectivityEpochFromHead(context.Background(), c)
require.NoError(t, err)
require.Equal(t, expectedEpoch, actualEpoch)
}
func forkForEpoch(cfg *params.BeaconChainConfig, epoch types.Epoch) (*ethpb.Fork, error) {
os := forks.NewOrderedSchedule(cfg)
currentVersion, err := os.VersionForEpoch(epoch)
if err != nil {
return nil, err
}
prevVersion, err := os.Previous(currentVersion)
if err != nil {
if !errors.Is(err, forks.ErrNoPreviousVersion) {
return nil, err
}
// use same version for both in the case of genesis
prevVersion = currentVersion
}
forkEpoch := cfg.ForkVersionSchedule[currentVersion]
return &ethpb.Fork{
PreviousVersion: prevVersion[:],
CurrentVersion: currentVersion[:],
Epoch: forkEpoch,
}, nil
}
func defaultTestHeadState(t *testing.T, cfg *params.BeaconChainConfig) (state.BeaconState, types.Epoch) {
st, err := util.NewBeaconStateAltair()
require.NoError(t, err)
fork, err := forkForEpoch(cfg, cfg.AltairForkEpoch)
require.NoError(t, err)
require.NoError(t, st.SetFork(fork))
slot, err := slots.EpochStart(cfg.AltairForkEpoch)
require.NoError(t, err)
require.NoError(t, st.SetSlot(slot))
var validatorCount, avgBalance uint64 = 100, 35
require.NoError(t, populateValidators(cfg, st, validatorCount, avgBalance))
require.NoError(t, st.SetFinalizedCheckpoint(&ethpb.Checkpoint{
Epoch: fork.Epoch - 10,
Root: make([]byte, 32),
}))
// to see the math for this, look at helpers.LatestWeakSubjectivityEpoch
// and for the values use mainnet config values, the validatorCount and avgBalance above, and altair fork epoch
expectedEpoch := slots.ToEpoch(st.Slot()) - 224
return st, expectedEpoch
}
// TODO(10429): refactor beacon state options in testing/util to take a state.BeaconState so this can become an option
func populateValidators(cfg *params.BeaconChainConfig, st state.BeaconState, valCount, avgBalance uint64) error {
validators := make([]*ethpb.Validator, valCount)
balances := make([]uint64, len(validators))
for i := uint64(0); i < valCount; i++ {
validators[i] = &ethpb.Validator{
PublicKey: make([]byte, cfg.BLSPubkeyLength),
WithdrawalCredentials: make([]byte, 32),
EffectiveBalance: avgBalance * 1e9,
ExitEpoch: cfg.FarFutureEpoch,
}
balances[i] = validators[i].EffectiveBalance
}
if err := st.SetValidators(validators); err != nil {
return err
}
if err := st.SetBalances(balances); err != nil {
return err
}
return nil
}

437
api/client/beacon/client.go Normal file
View File

@@ -0,0 +1,437 @@
package beacon
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"io/ioutil"
"net"
"net/http"
"net/url"
"path"
"regexp"
"sort"
"strconv"
"text/template"
"time"
"github.com/prysmaticlabs/prysm/network/forks"
"github.com/ethereum/go-ethereum/common/hexutil"
"github.com/pkg/errors"
types "github.com/prysmaticlabs/eth2-types"
"github.com/prysmaticlabs/prysm/beacon-chain/rpc/apimiddleware"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
log "github.com/sirupsen/logrus"
)
const (
getSignedBlockPath = "/eth/v2/beacon/blocks"
getBlockRootPath = "/eth/v1/beacon/blocks/{{.Id}}/root"
getForkForStatePath = "/eth/v1/beacon/states/{{.Id}}/fork"
getWeakSubjectivityPath = "/eth/v1/beacon/weak_subjectivity"
getForkSchedulePath = "/eth/v1/config/fork_schedule"
getStatePath = "/eth/v2/debug/beacon/states"
getNodeVersionPath = "/eth/v1/node/version"
)
// StateOrBlockId represents the block_id / state_id parameters that several of the Eth Beacon API methods accept.
// StateOrBlockId constants are defined for named identifiers, and helper methods are provided
// for slot and root identifiers. Example text from the Eth Beacon Node API documentation:
//
// "Block identifier can be one of: "head" (canonical head in node's view), "genesis", "finalized",
// <slot>, <hex encoded blockRoot with 0x prefix>."
type StateOrBlockId string
const (
IdFinalized StateOrBlockId = "finalized"
IdGenesis StateOrBlockId = "genesis"
IdHead StateOrBlockId = "head"
IdJustified StateOrBlockId = "justified"
)
// IdFromRoot encodes a block root in the format expected by the API in places where a root can be used to identify
// a BeaconState or SignedBeaconBlock.
func IdFromRoot(r [32]byte) StateOrBlockId {
return StateOrBlockId(fmt.Sprintf("%#x", r))
}
// IdFromRoot encodes a Slot in the format expected by the API in places where a slot can be used to identify
// a BeaconState or SignedBeaconBlock.
func IdFromSlot(s types.Slot) StateOrBlockId {
return StateOrBlockId(strconv.FormatUint(uint64(s), 10))
}
// idTemplate is used to create template functions that can interpolate StateOrBlockId values.
func idTemplate(ts string) func(StateOrBlockId) string {
t := template.Must(template.New("").Parse(ts))
f := func(id StateOrBlockId) string {
b := bytes.NewBuffer(nil)
err := t.Execute(b, struct{ Id string }{Id: string(id)})
if err != nil {
panic(fmt.Sprintf("invalid idTemplate: %s", ts))
}
return b.String()
}
// run the template to ensure that it is valid
// this should happen load time (using package scoped vars) to ensure runtime errors aren't possible
_ = f(IdGenesis)
return f
}
// ClientOpt is a functional option for the Client type (http.Client wrapper)
type ClientOpt func(*Client)
// WithTimeout sets the .Timeout attribute of the wrapped http.Client.
func WithTimeout(timeout time.Duration) ClientOpt {
return func(c *Client) {
c.hc.Timeout = timeout
}
}
// Client provides a collection of helper methods for calling the Eth Beacon Node API endpoints.
type Client struct {
hc *http.Client
host string
scheme string
}
// NewClient constructs a new client with the provided options (ex WithTimeout).
// `host` is the base host + port used to construct request urls. This value can be
// a URL string, or NewClient will assume an http endpoint if just `host:port` is used.
func NewClient(host string, opts ...ClientOpt) (*Client, error) {
host, err := validHostname(host)
if err != nil {
return nil, err
}
c := &Client{
hc: &http.Client{},
scheme: "http",
host: host,
}
for _, o := range opts {
o(c)
}
return c, nil
}
func validHostname(h string) (string, error) {
// try to parse as url (being permissive)
u, err := url.Parse(h)
if err == nil && u.Host != "" {
return u.Host, nil
}
// try to parse as host:port
host, port, err := net.SplitHostPort(h)
if err != nil {
return "", err
}
return fmt.Sprintf("%s:%s", host, port), nil
}
func (c *Client) urlForPath(methodPath string) *url.URL {
u := &url.URL{
Scheme: c.scheme,
Host: c.host,
}
u.Path = path.Join(u.Path, methodPath)
return u
}
type reqOption func(*http.Request)
func withSSZEncoding() reqOption {
return func(req *http.Request) {
req.Header.Set("Accept", "application/octet-stream")
}
}
// get is a generic, opinionated GET function to reduce boilerplate amongst the getters in this package.
func (c *Client) get(ctx context.Context, path string, opts ...reqOption) ([]byte, error) {
u := c.urlForPath(path)
log.Printf("requesting %s", u.String())
req, err := http.NewRequestWithContext(ctx, http.MethodGet, u.String(), nil)
if err != nil {
return nil, err
}
for _, o := range opts {
o(req)
}
r, err := c.hc.Do(req)
if err != nil {
return nil, err
}
defer func() {
err = r.Body.Close()
}()
if r.StatusCode != http.StatusOK {
return nil, non200Err(r)
}
b, err := io.ReadAll(r.Body)
if err != nil {
return nil, errors.Wrap(err, "error reading http response body from GetBlock")
}
return b, nil
}
func renderGetBlockPath(id StateOrBlockId) string {
return path.Join(getSignedBlockPath, string(id))
}
// GetBlock retrieves the SignedBeaconBlock for the given block id.
// Block identifier can be one of: "head" (canonical head in node's view), "genesis", "finalized",
// <slot>, <hex encoded blockRoot with 0x prefix>. Variables of type StateOrBlockId are exported by this package
// for the named identifiers.
// The return value contains the ssz-encoded bytes.
func (c *Client) GetBlock(ctx context.Context, blockId StateOrBlockId) ([]byte, error) {
blockPath := renderGetBlockPath(blockId)
b, err := c.get(ctx, blockPath, withSSZEncoding())
if err != nil {
return nil, errors.Wrapf(err, "error requesting state by id = %s", blockId)
}
return b, nil
}
var getBlockRootTpl = idTemplate(getBlockRootPath)
// GetBlockRoot retrieves the hash_tree_root of the BeaconBlock for the given block id.
// Block identifier can be one of: "head" (canonical head in node's view), "genesis", "finalized",
// <slot>, <hex encoded blockRoot with 0x prefix>. Variables of type StateOrBlockId are exported by this package
// for the named identifiers.
func (c *Client) GetBlockRoot(ctx context.Context, blockId StateOrBlockId) ([32]byte, error) {
rootPath := getBlockRootTpl(blockId)
b, err := c.get(ctx, rootPath)
if err != nil {
return [32]byte{}, errors.Wrapf(err, "error requesting block root by id = %s", blockId)
}
jsonr := &struct{ Data struct{ Root string } }{}
err = json.Unmarshal(b, jsonr)
if err != nil {
return [32]byte{}, errors.Wrap(err, "error decoding json data from get block root response")
}
rs, err := hexutil.Decode(jsonr.Data.Root)
if err != nil {
return [32]byte{}, errors.Wrap(err, fmt.Sprintf("error decoding hex-encoded value %s", jsonr.Data.Root))
}
return bytesutil.ToBytes32(rs), nil
}
var getForkTpl = idTemplate(getForkForStatePath)
// GetFork queries the Beacon Node API for the Fork from the state identified by stateId.
// Block identifier can be one of: "head" (canonical head in node's view), "genesis", "finalized",
// <slot>, <hex encoded blockRoot with 0x prefix>. Variables of type StateOrBlockId are exported by this package
// for the named identifiers.
func (c *Client) GetFork(ctx context.Context, stateId StateOrBlockId) (*ethpb.Fork, error) {
body, err := c.get(ctx, getForkTpl(stateId))
if err != nil {
return nil, errors.Wrapf(err, "error requesting fork by state id = %s", stateId)
}
fr := &forkResponse{}
dataWrapper := &struct{ Data *forkResponse }{Data: fr}
err = json.Unmarshal(body, dataWrapper)
if err != nil {
return nil, errors.Wrap(err, "error decoding json response in GetFork")
}
return fr.Fork()
}
// GetForkSchedule retrieve all forks, past present and future, of which this node is aware.
func (c *Client) GetForkSchedule(ctx context.Context) (forks.OrderedSchedule, error) {
body, err := c.get(ctx, getForkSchedulePath)
if err != nil {
return nil, errors.Wrap(err, "error requesting fork schedule")
}
fsr := &forkScheduleResponse{}
err = json.Unmarshal(body, fsr)
if err != nil {
return nil, err
}
ofs, err := fsr.OrderedForkSchedule()
if err != nil {
return nil, errors.Wrap(err, fmt.Sprintf("problem unmarshaling %s response", getForkSchedulePath))
}
return ofs, nil
}
type NodeVersion struct {
implementation string
semver string
systemInfo string
}
var versionRE = regexp.MustCompile(`^(\w+)\/(v\d+\.\d+\.\d+) \((.*)\)$`)
func parseNodeVersion(v string) (*NodeVersion, error) {
groups := versionRE.FindStringSubmatch(v)
if len(groups) != 4 {
return nil, errors.Wrapf(ErrInvalidNodeVersion, "could not be parsed: %s", v)
}
return &NodeVersion{
implementation: groups[1],
semver: groups[2],
systemInfo: groups[3],
}, nil
}
// GetNodeVersion requests that the beacon node identify information about its implementation in a format
// similar to a HTTP User-Agent field. ex: Lighthouse/v0.1.5 (Linux x86_64)
func (c *Client) GetNodeVersion(ctx context.Context) (*NodeVersion, error) {
b, err := c.get(ctx, getNodeVersionPath)
if err != nil {
return nil, errors.Wrap(err, "error requesting node version")
}
d := struct {
Data struct {
Version string `json:"version"`
} `json:"data"`
}{}
err = json.Unmarshal(b, &d)
if err != nil {
return nil, errors.Wrapf(err, "error unmarshaling response body: %s", string(b))
}
return parseNodeVersion(d.Data.Version)
}
func renderGetStatePath(id StateOrBlockId) string {
return path.Join(getStatePath, string(id))
}
// GetState retrieves the BeaconState for the given state id.
// State identifier can be one of: "head" (canonical head in node's view), "genesis", "finalized",
// <slot>, <hex encoded stateRoot with 0x prefix>. Variables of type StateOrBlockId are exported by this package
// for the named identifiers.
// The return value contains the ssz-encoded bytes.
func (c *Client) GetState(ctx context.Context, stateId StateOrBlockId) ([]byte, error) {
statePath := path.Join(getStatePath, string(stateId))
b, err := c.get(ctx, statePath, withSSZEncoding())
if err != nil {
return nil, errors.Wrapf(err, "error requesting state by id = %s", stateId)
}
return b, nil
}
// GetWeakSubjectivity calls a proposed API endpoint that is unique to prysm
// This api method does the following:
// - computes weak subjectivity epoch
// - finds the highest non-skipped block preceding the epoch
// - returns the htr of the found block and returns this + the value of state_root from the block
func (c *Client) GetWeakSubjectivity(ctx context.Context) (*WeakSubjectivityData, error) {
body, err := c.get(ctx, getWeakSubjectivityPath)
if err != nil {
return nil, err
}
v := &apimiddleware.WeakSubjectivityResponse{}
err = json.Unmarshal(body, v)
if err != nil {
return nil, err
}
epoch, err := strconv.ParseUint(v.Data.Checkpoint.Epoch, 10, 64)
if err != nil {
return nil, err
}
blockRoot, err := hexutil.Decode(v.Data.Checkpoint.Root)
if err != nil {
return nil, err
}
stateRoot, err := hexutil.Decode(v.Data.StateRoot)
if err != nil {
return nil, err
}
return &WeakSubjectivityData{
Epoch: types.Epoch(epoch),
BlockRoot: bytesutil.ToBytes32(blockRoot),
StateRoot: bytesutil.ToBytes32(stateRoot),
}, nil
}
// WeakSubjectivityData represents the state root, block root and epoch of the BeaconState + SignedBeaconBlock
// that falls at the beginning of the current weak subjectivity period. These values can be used to construct
// a weak subjectivity checkpoint, or to download a BeaconState+SignedBeaconBlock pair that can be used to bootstrap
// a new Beacon Node using Checkpoint Sync.
type WeakSubjectivityData struct {
BlockRoot [32]byte
StateRoot [32]byte
Epoch types.Epoch
}
func non200Err(response *http.Response) error {
bodyBytes, err := ioutil.ReadAll(response.Body)
var body string
if err != nil {
body = "(Unable to read response body.)"
} else {
body = "response body:\n" + string(bodyBytes)
}
msg := fmt.Sprintf("code=%d, url=%s, body=%s", response.StatusCode, response.Request.URL, body)
switch response.StatusCode {
case 404:
return errors.Wrap(ErrNotFound, msg)
default:
return errors.Wrap(ErrNotOK, msg)
}
}
type forkResponse struct {
PreviousVersion string `json:"previous_version"`
CurrentVersion string `json:"current_version"`
Epoch string `json:"epoch"`
}
func (f *forkResponse) Fork() (*ethpb.Fork, error) {
epoch, err := strconv.ParseUint(f.Epoch, 10, 64)
if err != nil {
return nil, err
}
cSlice, err := hexutil.Decode(f.CurrentVersion)
if err != nil {
return nil, err
}
if len(cSlice) != 4 {
return nil, fmt.Errorf("got %d byte version for CurrentVersion, expected 4 bytes. hex=%s", len(cSlice), f.CurrentVersion)
}
pSlice, err := hexutil.Decode(f.PreviousVersion)
if err != nil {
return nil, err
}
if len(pSlice) != 4 {
return nil, fmt.Errorf("got %d byte version, expected 4 bytes. version hex=%s", len(pSlice), f.PreviousVersion)
}
return &ethpb.Fork{
CurrentVersion: cSlice,
PreviousVersion: pSlice,
Epoch: types.Epoch(epoch),
}, nil
}
type forkScheduleResponse struct {
Data []forkResponse
}
func (fsr *forkScheduleResponse) OrderedForkSchedule() (forks.OrderedSchedule, error) {
ofs := make(forks.OrderedSchedule, 0)
for _, d := range fsr.Data {
epoch, err := strconv.Atoi(d.Epoch)
if err != nil {
return nil, err
}
vSlice, err := hexutil.Decode(d.CurrentVersion)
if err != nil {
return nil, err
}
if len(vSlice) != 4 {
return nil, fmt.Errorf("got %d byte version, expected 4 bytes. version hex=%s", len(vSlice), d.CurrentVersion)
}
version := bytesutil.ToBytes4(vSlice)
ofs = append(ofs, forks.ForkScheduleEntry{
Version: version,
Epoch: types.Epoch(uint64(epoch)),
})
}
sort.Sort(ofs)
return ofs, nil
}

View File

@@ -0,0 +1,58 @@
package beacon
import (
"testing"
"github.com/prysmaticlabs/prysm/testing/require"
)
func TestParseNodeVersion(t *testing.T) {
cases := []struct {
name string
v string
err error
nv *NodeVersion
}{
{
name: "empty string",
v: "",
err: ErrInvalidNodeVersion,
},
{
name: "Prysm as the version string",
v: "Prysm",
err: ErrInvalidNodeVersion,
},
{
name: "semver only",
v: "v2.0.6",
err: ErrInvalidNodeVersion,
},
{
name: "implementation and semver only",
v: "Prysm/v2.0.6",
err: ErrInvalidNodeVersion,
},
{
name: "complete version",
v: "Prysm/v2.0.6 (linux amd64)",
nv: &NodeVersion{
implementation: "Prysm",
semver: "v2.0.6",
systemInfo: "linux amd64",
},
},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
nv, err := parseNodeVersion(c.v)
if c.err != nil {
require.ErrorIs(t, err, c.err)
} else {
require.NoError(t, err)
require.DeepEqual(t, c.nv, nv)
}
})
}
}

6
api/client/beacon/doc.go Normal file
View File

@@ -0,0 +1,6 @@
/*
Package beacon provides a client for interacting with the standard Eth Beacon Node API.
Interactive swagger documentation for the API is available here: https://ethereum.github.io/beacon-APIs/
*/
package beacon

View File

@@ -0,0 +1,13 @@
package beacon
import "github.com/pkg/errors"
// ErrNotOK is used to indicate when an HTTP request to the Beacon Node API failed with any non-2xx response code.
// More specific errors may be returned, but an error in reaction to a non-2xx response will always wrap ErrNotOK.
var ErrNotOK = errors.New("did not receive 2xx response from API")
// ErrNotFound specifically means that a '404 - NOT FOUND' response was received from the API.
var ErrNotFound = errors.Wrap(ErrNotOK, "recv 404 NotFound response from API")
// ErrInvalidNodeVersion indicates that the /eth/v1/node/version api response format was not recognized.
var ErrInvalidNodeVersion = errors.New("invalid node version response")

View File

@@ -5,6 +5,7 @@ go_library(
srcs = [
"gateway.go",
"log.go",
"options.go",
],
importpath = "github.com/prysmaticlabs/prysm/api/gateway",
visibility = [

View File

@@ -3,6 +3,7 @@ package apimiddleware
import (
"net/http"
"reflect"
"time"
"github.com/gorilla/mux"
)
@@ -14,6 +15,8 @@ import (
type ApiProxyMiddleware struct {
GatewayAddress string
EndpointCreator EndpointFactory
Timeout time.Duration
router *mux.Router
}
// EndpointFactory is responsible for creating new instances of Endpoint values.
@@ -29,6 +32,8 @@ type Endpoint struct {
GetResponse interface{} // The struct corresponding to the JSON structure used in a GET response.
PostRequest interface{} // The struct corresponding to the JSON structure used in a POST request.
PostResponse interface{} // The struct corresponding to the JSON structure used in a POST response.
DeleteRequest interface{} // The struct corresponding to the JSON structure used in a DELETE request.
DeleteResponse interface{} // The struct corresponding to the JSON structure used in a DELETE response.
RequestURLLiterals []string // Names of URL parameters that should not be base64-encoded.
RequestQueryParams []QueryParam // Query parameters of the request.
Err ErrorJson // The struct corresponding to the error that should be returned in case of a request failure.
@@ -74,16 +79,23 @@ type fieldProcessor struct {
// Run starts the proxy, registering all proxy endpoints.
func (m *ApiProxyMiddleware) Run(gatewayRouter *mux.Router) {
for _, path := range m.EndpointCreator.Paths() {
m.handleApiPath(gatewayRouter, path, m.EndpointCreator)
gatewayRouter.HandleFunc(path, m.WithMiddleware(path))
}
m.router = gatewayRouter
}
func (m *ApiProxyMiddleware) handleApiPath(gatewayRouter *mux.Router, path string, endpointFactory EndpointFactory) {
gatewayRouter.HandleFunc(path, func(w http.ResponseWriter, req *http.Request) {
endpoint, err := endpointFactory.Create(path)
// ServeHTTP for the proxy middleware.
func (m *ApiProxyMiddleware) ServeHTTP(w http.ResponseWriter, req *http.Request) {
m.router.ServeHTTP(w, req)
}
// WithMiddleware wraps the given endpoint handler with the middleware logic.
func (m *ApiProxyMiddleware) WithMiddleware(path string) http.HandlerFunc {
return func(w http.ResponseWriter, req *http.Request) {
endpoint, err := m.EndpointCreator.Create(path)
if err != nil {
errJson := InternalServerErrorWithMessage(err, "could not create endpoint")
WriteError(w, errJson, nil)
log.WithError(err).Errorf("Could not create endpoint for path: %s", path)
return
}
for _, handler := range endpoint.CustomHandlers {
@@ -93,16 +105,14 @@ func (m *ApiProxyMiddleware) handleApiPath(gatewayRouter *mux.Router, path strin
}
if req.Method == "POST" {
if errJson := deserializeRequestBodyIntoContainerWrapped(endpoint, req, w); errJson != nil {
if errJson := handlePostRequestForEndpoint(endpoint, w, req); errJson != nil {
WriteError(w, errJson, nil)
return
}
}
if errJson := ProcessRequestContainerFields(endpoint.PostRequest); errJson != nil {
WriteError(w, errJson, nil)
return
}
if errJson := SetRequestBodyToRequestContainer(endpoint.PostRequest, req); errJson != nil {
if req.Method == "DELETE" {
if errJson := handleDeleteRequestForEndpoint(endpoint, req); errJson != nil {
WriteError(w, errJson, nil)
return
}
@@ -112,7 +122,7 @@ func (m *ApiProxyMiddleware) handleApiPath(gatewayRouter *mux.Router, path strin
WriteError(w, errJson, nil)
return
}
grpcResp, errJson := ProxyRequest(req)
grpcResp, errJson := m.ProxyRequest(req)
if errJson != nil {
WriteError(w, errJson, nil)
return
@@ -125,18 +135,20 @@ func (m *ApiProxyMiddleware) handleApiPath(gatewayRouter *mux.Router, path strin
var respJson []byte
if !GrpcResponseIsEmpty(grpcRespBody) {
if errJson := DeserializeGrpcResponseBodyIntoErrorJson(endpoint.Err, grpcRespBody); errJson != nil {
respHasError, errJson := HandleGrpcResponseError(endpoint.Err, grpcResp, grpcRespBody, w)
if errJson != nil {
WriteError(w, errJson, nil)
return
}
if endpoint.Err.Msg() != "" {
HandleGrpcResponseError(endpoint.Err, grpcResp, w)
if respHasError {
return
}
var resp interface{}
if req.Method == "GET" {
resp = endpoint.GetResponse
} else if req.Method == "DELETE" {
resp = endpoint.DeleteResponse
} else {
resp = endpoint.PostResponse
}
@@ -164,7 +176,27 @@ func (m *ApiProxyMiddleware) handleApiPath(gatewayRouter *mux.Router, path strin
WriteError(w, errJson, nil)
return
}
})
}
}
func handlePostRequestForEndpoint(endpoint *Endpoint, w http.ResponseWriter, req *http.Request) ErrorJson {
if errJson := deserializeRequestBodyIntoContainerWrapped(endpoint, req, w); errJson != nil {
return errJson
}
if errJson := ProcessRequestContainerFields(endpoint.PostRequest); errJson != nil {
return errJson
}
return SetRequestBodyToRequestContainer(endpoint.PostRequest, req)
}
func handleDeleteRequestForEndpoint(endpoint *Endpoint, req *http.Request) ErrorJson {
if errJson := DeserializeRequestBodyIntoContainer(req.Body, endpoint.DeleteRequest); errJson != nil {
return errJson
}
if errJson := ProcessRequestContainerFields(endpoint.DeleteRequest); errJson != nil {
return errJson
}
return SetRequestBodyToRequestContainer(endpoint.DeleteRequest, req)
}
func deserializeRequestBodyIntoContainerWrapped(endpoint *Endpoint, req *http.Request, w http.ResponseWriter) ErrorJson {

View File

@@ -5,10 +5,10 @@ import (
"encoding/json"
"io"
"io/ioutil"
"net"
"net/http"
"strconv"
"strings"
"time"
"github.com/pkg/errors"
"github.com/prysmaticlabs/prysm/api/grpc"
@@ -75,11 +75,14 @@ func (m *ApiProxyMiddleware) PrepareRequestForProxying(endpoint Endpoint, req *h
}
// ProxyRequest proxies the request to grpc-gateway.
func ProxyRequest(req *http.Request) (*http.Response, ErrorJson) {
func (m *ApiProxyMiddleware) ProxyRequest(req *http.Request) (*http.Response, ErrorJson) {
// We do not use http.DefaultClient because it does not have any timeout.
netClient := &http.Client{Timeout: time.Minute * 2}
netClient := &http.Client{Timeout: m.Timeout}
grpcResp, err := netClient.Do(req)
if err != nil {
if err, ok := err.(net.Error); ok && err.Timeout() {
return nil, TimeoutError()
}
return nil, InternalServerErrorWithMessage(err, "could not proxy request")
}
if grpcResp == nil {
@@ -97,26 +100,30 @@ func ReadGrpcResponseBody(r io.Reader) ([]byte, ErrorJson) {
return body, nil
}
// DeserializeGrpcResponseBodyIntoErrorJson deserializes the body from the grpc-gateway's response into an error struct.
// The struct can be later examined to check if the request resulted in an error.
func DeserializeGrpcResponseBodyIntoErrorJson(errJson ErrorJson, body []byte) ErrorJson {
if err := json.Unmarshal(body, errJson); err != nil {
return InternalServerErrorWithMessage(err, "could not unmarshal error")
}
return nil
}
// HandleGrpcResponseError acts on an error that resulted from a grpc-gateway's response.
func HandleGrpcResponseError(errJson ErrorJson, resp *http.Response, w http.ResponseWriter) {
// Something went wrong, but the request completed, meaning we can write headers and the error message.
for h, vs := range resp.Header {
for _, v := range vs {
w.Header().Set(h, v)
func HandleGrpcResponseError(errJson ErrorJson, resp *http.Response, respBody []byte, w http.ResponseWriter) (bool, ErrorJson) {
responseHasError := false
if err := json.Unmarshal(respBody, errJson); err != nil {
return false, InternalServerErrorWithMessage(err, "could not unmarshal error")
}
if errJson.Msg() != "" {
responseHasError = true
// Something went wrong, but the request completed, meaning we can write headers and the error message.
for h, vs := range resp.Header {
for _, v := range vs {
w.Header().Set(h, v)
}
}
// Handle gRPC timeout.
if resp.StatusCode == http.StatusGatewayTimeout {
WriteError(w, TimeoutError(), resp.Header)
} else {
// Set code to HTTP code because unmarshalled body contained gRPC code.
errJson.SetCode(resp.StatusCode)
WriteError(w, errJson, resp.Header)
}
}
// Set code to HTTP code because unmarshalled body contained gRPC code.
errJson.SetCode(resp.StatusCode)
WriteError(w, errJson, resp.Header)
return responseHasError, nil
}
// GrpcResponseIsEmpty determines whether the grpc-gateway's response body contains no data.
@@ -201,9 +208,11 @@ func WriteMiddlewareResponseHeadersAndBody(grpcResp *http.Response, responseJson
// WriteError writes the error by manipulating headers and the body of the final response.
func WriteError(w http.ResponseWriter, errJson ErrorJson, responseHeader http.Header) {
// Include custom error in the error JSON.
hasCustomError := false
if responseHeader != nil {
customError, ok := responseHeader["Grpc-Metadata-"+grpc.CustomErrorMetadataKey]
if ok {
hasCustomError = true
// Assume header has only one value and read the 0 index.
if err := json.Unmarshal([]byte(customError[0]), errJson); err != nil {
log.WithError(err).Error("Could not unmarshal custom error message")
@@ -212,10 +221,29 @@ func WriteError(w http.ResponseWriter, errJson ErrorJson, responseHeader http.He
}
}
j, err := json.Marshal(errJson)
if err != nil {
log.WithError(err).Error("Could not marshal error message")
return
var j []byte
if hasCustomError {
var err error
j, err = json.Marshal(errJson)
if err != nil {
log.WithError(err).Error("Could not marshal error message")
return
}
} else {
var err error
// We marshal the response body into a DefaultErrorJson if the custom error is not present.
// This is because the ErrorJson argument is the endpoint's error definition, which may contain custom fields.
// In such a scenario marhaling the endpoint's error would populate the resulting JSON
// with these fields even if they are not present in the gRPC header.
d := &DefaultErrorJson{
Message: errJson.Msg(),
Code: errJson.StatusCode(),
}
j, err = json.Marshal(d)
if err != nil {
log.WithError(err).Error("Could not marshal error message")
return
}
}
w.Header().Set("Content-Length", strconv.Itoa(len(j)))

View File

@@ -63,6 +63,11 @@ func (e *testErrorJson) SetCode(code int) {
e.Code = code
}
// SetMsg sets the error's underlying message.
func (e *testErrorJson) SetMsg(msg string) {
e.Message = msg
}
func TestDeserializeRequestBodyIntoContainer(t *testing.T) {
t.Run("ok", func(t *testing.T) {
var bodyJson bytes.Buffer
@@ -156,29 +161,6 @@ func TestReadGrpcResponseBody(t *testing.T) {
assert.Equal(t, "foo", string(body))
}
func TestDeserializeGrpcResponseBodyIntoErrorJson(t *testing.T) {
t.Run("ok", func(t *testing.T) {
e := &testErrorJson{
Message: "foo",
Code: 500,
}
body, err := json.Marshal(e)
require.NoError(t, err)
eToDeserialize := &testErrorJson{}
errJson := DeserializeGrpcResponseBodyIntoErrorJson(eToDeserialize, body)
require.Equal(t, true, errJson == nil)
assert.Equal(t, "foo", eToDeserialize.Msg())
assert.Equal(t, 500, eToDeserialize.StatusCode())
})
t.Run("error", func(t *testing.T) {
errJson := DeserializeGrpcResponseBodyIntoErrorJson(nil, nil)
require.NotNil(t, errJson)
assert.Equal(t, true, strings.Contains(errJson.Msg(), "could not unmarshal error"))
})
}
func TestHandleGrpcResponseError(t *testing.T) {
response := &http.Response{
StatusCode: 400,
@@ -190,10 +172,14 @@ func TestHandleGrpcResponseError(t *testing.T) {
writer := httptest.NewRecorder()
errJson := &testErrorJson{
Message: "foo",
Code: 500,
Code: 400,
}
b, err := json.Marshal(errJson)
require.NoError(t, err)
HandleGrpcResponseError(errJson, response, writer)
hasError, e := HandleGrpcResponseError(errJson, response, b, writer)
require.Equal(t, true, e == nil)
assert.Equal(t, true, hasError)
v, ok := writer.Header()["Foo"]
require.Equal(t, true, ok, "header not found")
require.Equal(t, 1, len(v), "wrong number of header values")

View File

@@ -15,6 +15,7 @@ type ErrorJson interface {
StatusCode() int
SetCode(code int)
Msg() string
SetMsg(msg string)
}
// DefaultErrorJson is a JSON representation of a simple error value, containing only a message and an error code.
@@ -40,6 +41,13 @@ func InternalServerError(err error) *DefaultErrorJson {
}
}
func TimeoutError() *DefaultErrorJson {
return &DefaultErrorJson{
Message: "Request timeout",
Code: http.StatusRequestTimeout,
}
}
// StatusCode returns the error's underlying error code.
func (e *DefaultErrorJson) StatusCode() int {
return e.Code
@@ -54,3 +62,8 @@ func (e *DefaultErrorJson) Msg() string {
func (e *DefaultErrorJson) SetCode(code int) {
e.Code = code
}
// SetMsg sets the error's underlying message.
func (e *DefaultErrorJson) SetMsg(msg string) {
e.Message = msg
}

View File

@@ -34,74 +34,52 @@ type PbMux struct {
type PbHandlerRegistration func(context.Context, *gwruntime.ServeMux, *grpc.ClientConn) error
// MuxHandler is a function that implements the mux handler functionality.
type MuxHandler func(http.Handler, http.ResponseWriter, *http.Request)
type MuxHandler func(
apiMiddlewareHandler *apimiddleware.ApiProxyMiddleware,
h http.HandlerFunc,
w http.ResponseWriter,
req *http.Request,
)
// Config parameters for setting up the gateway service.
type config struct {
maxCallRecvMsgSize uint64
remoteCert string
gatewayAddr string
remoteAddr string
allowedOrigins []string
apiMiddlewareEndpointFactory apimiddleware.EndpointFactory
muxHandler MuxHandler
pbHandlers []*PbMux
router *mux.Router
timeout time.Duration
}
// Gateway is the gRPC gateway to serve HTTP JSON traffic as a proxy and forward it to the gRPC server.
type Gateway struct {
conn *grpc.ClientConn
pbHandlers []*PbMux
muxHandler MuxHandler
maxCallRecvMsgSize uint64
router *mux.Router
server *http.Server
cancel context.CancelFunc
remoteCert string
gatewayAddr string
apiMiddlewareEndpointFactory apimiddleware.EndpointFactory
ctx context.Context
startFailure error
remoteAddr string
allowedOrigins []string
cfg *config
conn *grpc.ClientConn
server *http.Server
cancel context.CancelFunc
proxy *apimiddleware.ApiProxyMiddleware
ctx context.Context
startFailure error
}
// New returns a new instance of the Gateway.
func New(
ctx context.Context,
pbHandlers []*PbMux,
muxHandler MuxHandler,
remoteAddr,
gatewayAddress string,
) *Gateway {
func New(ctx context.Context, opts ...Option) (*Gateway, error) {
g := &Gateway{
pbHandlers: pbHandlers,
muxHandler: muxHandler,
router: mux.NewRouter(),
gatewayAddr: gatewayAddress,
ctx: ctx,
remoteAddr: remoteAddr,
allowedOrigins: []string{},
ctx: ctx,
cfg: &config{
router: mux.NewRouter(),
},
}
return g
}
// WithRouter allows adding a custom mux router to the gateway.
func (g *Gateway) WithRouter(r *mux.Router) *Gateway {
g.router = r
return g
}
// WithAllowedOrigins allows adding a set of allowed origins to the gateway.
func (g *Gateway) WithAllowedOrigins(origins []string) *Gateway {
g.allowedOrigins = origins
return g
}
// WithRemoteCert allows adding a custom certificate to the gateway,
func (g *Gateway) WithRemoteCert(cert string) *Gateway {
g.remoteCert = cert
return g
}
// WithMaxCallRecvMsgSize allows specifying the maximum allowed gRPC message size.
func (g *Gateway) WithMaxCallRecvMsgSize(size uint64) *Gateway {
g.maxCallRecvMsgSize = size
return g
}
// WithApiMiddleware allows adding API Middleware proxy to the gateway.
func (g *Gateway) WithApiMiddleware(endpointFactory apimiddleware.EndpointFactory) *Gateway {
g.apiMiddlewareEndpointFactory = endpointFactory
return g
for _, opt := range opts {
if err := opt(g); err != nil {
return nil, err
}
}
return g, nil
}
// Start the gateway service.
@@ -109,7 +87,7 @@ func (g *Gateway) Start() {
ctx, cancel := context.WithCancel(g.ctx)
g.cancel = cancel
conn, err := g.dial(ctx, "tcp", g.remoteAddr)
conn, err := g.dial(ctx, "tcp", g.cfg.remoteAddr)
if err != nil {
log.WithError(err).Error("Failed to connect to gRPC server")
g.startFailure = err
@@ -117,7 +95,7 @@ func (g *Gateway) Start() {
}
g.conn = conn
for _, h := range g.pbHandlers {
for _, h := range g.cfg.pbHandlers {
for _, r := range h.Registrations {
if err := r(ctx, h.Mux, g.conn); err != nil {
log.WithError(err).Error("Failed to register handler")
@@ -126,29 +104,29 @@ func (g *Gateway) Start() {
}
}
for _, p := range h.Patterns {
g.router.PathPrefix(p).Handler(h.Mux)
g.cfg.router.PathPrefix(p).Handler(h.Mux)
}
}
corsMux := g.corsMiddleware(g.router)
corsMux := g.corsMiddleware(g.cfg.router)
if g.muxHandler != nil {
g.router.PathPrefix("/").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
g.muxHandler(corsMux, w, r)
if g.cfg.apiMiddlewareEndpointFactory != nil && !g.cfg.apiMiddlewareEndpointFactory.IsNil() {
g.registerApiMiddleware()
}
if g.cfg.muxHandler != nil {
g.cfg.router.PathPrefix("/").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
g.cfg.muxHandler(g.proxy, corsMux.ServeHTTP, w, r)
})
}
if g.apiMiddlewareEndpointFactory != nil && !g.apiMiddlewareEndpointFactory.IsNil() {
go g.registerApiMiddleware()
}
g.server = &http.Server{
Addr: g.gatewayAddr,
Handler: g.router,
Addr: g.cfg.gatewayAddr,
Handler: corsMux,
}
go func() {
log.WithField("address", g.gatewayAddr).Info("Starting gRPC gateway")
log.WithField("address", g.cfg.gatewayAddr).Info("Starting gRPC gateway")
if err := g.server.ListenAndServe(); err != http.ErrServerClosed {
log.WithError(err).Error("Failed to start gRPC gateway")
g.startFailure = err
@@ -162,11 +140,9 @@ func (g *Gateway) Status() error {
if g.startFailure != nil {
return g.startFailure
}
if s := g.conn.GetState(); s != connectivity.Ready {
return fmt.Errorf("grpc server is %s", s)
}
return nil
}
@@ -183,18 +159,16 @@ func (g *Gateway) Stop() error {
}
}
}
if g.cancel != nil {
g.cancel()
}
return nil
}
func (g *Gateway) corsMiddleware(h http.Handler) http.Handler {
c := cors.New(cors.Options{
AllowedOrigins: g.allowedOrigins,
AllowedMethods: []string{http.MethodPost, http.MethodGet, http.MethodOptions},
AllowedOrigins: g.cfg.allowedOrigins,
AllowedMethods: []string{http.MethodPost, http.MethodGet, http.MethodDelete, http.MethodOptions},
AllowCredentials: true,
MaxAge: 600,
AllowedHeaders: []string{"*"},
@@ -236,8 +210,8 @@ func (g *Gateway) dial(ctx context.Context, network, addr string) (*grpc.ClientC
// "addr" must be a valid TCP address with a port number.
func (g *Gateway) dialTCP(ctx context.Context, addr string) (*grpc.ClientConn, error) {
security := grpc.WithInsecure()
if len(g.remoteCert) > 0 {
creds, err := credentials.NewClientTLSFromFile(g.remoteCert, "")
if len(g.cfg.remoteCert) > 0 {
creds, err := credentials.NewClientTLSFromFile(g.cfg.remoteCert, "")
if err != nil {
return nil, err
}
@@ -245,7 +219,7 @@ func (g *Gateway) dialTCP(ctx context.Context, addr string) (*grpc.ClientConn, e
}
opts := []grpc.DialOption{
security,
grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(int(g.maxCallRecvMsgSize))),
grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(int(g.cfg.maxCallRecvMsgSize))),
}
return grpc.DialContext(ctx, addr, opts...)
@@ -266,16 +240,17 @@ func (g *Gateway) dialUnix(ctx context.Context, addr string) (*grpc.ClientConn,
opts := []grpc.DialOption{
grpc.WithInsecure(),
grpc.WithContextDialer(f),
grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(int(g.maxCallRecvMsgSize))),
grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(int(g.cfg.maxCallRecvMsgSize))),
}
return grpc.DialContext(ctx, addr, opts...)
}
func (g *Gateway) registerApiMiddleware() {
proxy := &apimiddleware.ApiProxyMiddleware{
GatewayAddress: g.gatewayAddr,
EndpointCreator: g.apiMiddlewareEndpointFactory,
g.proxy = &apimiddleware.ApiProxyMiddleware{
GatewayAddress: g.cfg.gatewayAddr,
EndpointCreator: g.cfg.apiMiddlewareEndpointFactory,
Timeout: g.cfg.timeout,
}
log.Info("Starting API middleware")
proxy.Run(g.router)
g.proxy.Run(g.cfg.router)
}

View File

@@ -40,26 +40,30 @@ func TestGateway_Customized(t *testing.T) {
size := uint64(100)
endpointFactory := &mockEndpointFactory{}
g := New(
context.Background(),
[]*PbMux{},
func(handler http.Handler, writer http.ResponseWriter, request *http.Request) {
opts := []Option{
WithRouter(r),
WithRemoteCert(cert),
WithAllowedOrigins(origins),
WithMaxCallRecvMsgSize(size),
WithApiMiddleware(endpointFactory),
WithMuxHandler(func(
_ *apimiddleware.ApiProxyMiddleware,
_ http.HandlerFunc,
_ http.ResponseWriter,
_ *http.Request,
) {
}),
}
},
"",
"",
).WithRouter(r).
WithRemoteCert(cert).
WithAllowedOrigins(origins).
WithMaxCallRecvMsgSize(size).
WithApiMiddleware(endpointFactory)
g, err := New(context.Background(), opts...)
require.NoError(t, err)
assert.Equal(t, r, g.router)
assert.Equal(t, cert, g.remoteCert)
require.Equal(t, 1, len(g.allowedOrigins))
assert.Equal(t, origins[0], g.allowedOrigins[0])
assert.Equal(t, size, g.maxCallRecvMsgSize)
assert.Equal(t, endpointFactory, g.apiMiddlewareEndpointFactory)
assert.Equal(t, r, g.cfg.router)
assert.Equal(t, cert, g.cfg.remoteCert)
require.Equal(t, 1, len(g.cfg.allowedOrigins))
assert.Equal(t, origins[0], g.cfg.allowedOrigins[0])
assert.Equal(t, size, g.cfg.maxCallRecvMsgSize)
assert.Equal(t, endpointFactory, g.cfg.apiMiddlewareEndpointFactory)
}
func TestGateway_StartStop(t *testing.T) {
@@ -75,23 +79,27 @@ func TestGateway_StartStop(t *testing.T) {
selfAddress := fmt.Sprintf("%s:%d", rpcHost, ctx.Int(flags.RPCPort.Name))
gatewayAddress := fmt.Sprintf("%s:%d", gatewayHost, gatewayPort)
g := New(
ctx.Context,
[]*PbMux{},
func(handler http.Handler, writer http.ResponseWriter, request *http.Request) {
opts := []Option{
WithGatewayAddr(gatewayAddress),
WithRemoteAddr(selfAddress),
WithMuxHandler(func(
_ *apimiddleware.ApiProxyMiddleware,
_ http.HandlerFunc,
_ http.ResponseWriter,
_ *http.Request,
) {
}),
}
},
selfAddress,
gatewayAddress,
)
g, err := New(context.Background(), opts...)
require.NoError(t, err)
g.Start()
go func() {
require.LogsContain(t, hook, "Starting gRPC gateway")
require.LogsDoNotContain(t, hook, "Starting API middleware")
}()
err := g.Stop()
err = g.Stop()
require.NoError(t, err)
}
@@ -106,15 +114,15 @@ func TestGateway_NilHandler_NotFoundHandlerRegistered(t *testing.T) {
selfAddress := fmt.Sprintf("%s:%d", rpcHost, ctx.Int(flags.RPCPort.Name))
gatewayAddress := fmt.Sprintf("%s:%d", gatewayHost, gatewayPort)
g := New(
ctx.Context,
[]*PbMux{},
/* muxHandler */ nil,
selfAddress,
gatewayAddress,
)
opts := []Option{
WithGatewayAddr(gatewayAddress),
WithRemoteAddr(selfAddress),
}
g, err := New(context.Background(), opts...)
require.NoError(t, err)
writer := httptest.NewRecorder()
g.router.ServeHTTP(writer, &http.Request{Method: "GET", Host: "localhost", URL: &url.URL{Path: "/foo"}})
g.cfg.router.ServeHTTP(writer, &http.Request{Method: "GET", Host: "localhost", URL: &url.URL{Path: "/foo"}})
assert.Equal(t, http.StatusNotFound, writer.Code)
}

93
api/gateway/options.go Normal file
View File

@@ -0,0 +1,93 @@
package gateway
import (
"time"
"github.com/gorilla/mux"
gwruntime "github.com/grpc-ecosystem/grpc-gateway/v2/runtime"
"github.com/prysmaticlabs/prysm/api/gateway/apimiddleware"
)
type Option func(g *Gateway) error
func (g *Gateway) SetRouter(r *mux.Router) *Gateway {
g.cfg.router = r
return g
}
func WithPbHandlers(handlers []*PbMux) Option {
return func(g *Gateway) error {
g.cfg.pbHandlers = handlers
return nil
}
}
func WithMuxHandler(m MuxHandler) Option {
return func(g *Gateway) error {
g.cfg.muxHandler = m
return nil
}
}
func WithGatewayAddr(addr string) Option {
return func(g *Gateway) error {
g.cfg.gatewayAddr = addr
return nil
}
}
func WithRemoteAddr(addr string) Option {
return func(g *Gateway) error {
g.cfg.remoteAddr = addr
return nil
}
}
// WithRouter allows adding a custom mux router to the gateway.
func WithRouter(r *mux.Router) Option {
return func(g *Gateway) error {
g.cfg.router = r
return nil
}
}
// WithAllowedOrigins allows adding a set of allowed origins to the gateway.
func WithAllowedOrigins(origins []string) Option {
return func(g *Gateway) error {
g.cfg.allowedOrigins = origins
return nil
}
}
// WithRemoteCert allows adding a custom certificate to the gateway,
func WithRemoteCert(cert string) Option {
return func(g *Gateway) error {
g.cfg.remoteCert = cert
return nil
}
}
// WithMaxCallRecvMsgSize allows specifying the maximum allowed gRPC message size.
func WithMaxCallRecvMsgSize(size uint64) Option {
return func(g *Gateway) error {
g.cfg.maxCallRecvMsgSize = size
return nil
}
}
// WithApiMiddleware allows adding an API middleware proxy to the gateway.
func WithApiMiddleware(endpointFactory apimiddleware.EndpointFactory) Option {
return func(g *Gateway) error {
g.cfg.apiMiddlewareEndpointFactory = endpointFactory
return nil
}
}
// WithTimeout allows changing the timeout value for API calls.
func WithTimeout(seconds uint64) Option {
return func(g *Gateway) error {
g.cfg.timeout = time.Second * time.Duration(seconds)
gwruntime.DefaultContextTimeout = time.Second * time.Duration(seconds)
return nil
}
}

View File

@@ -110,9 +110,8 @@ func TestLockUnlock_CleansUnused(t *testing.T) {
lock := NewMultilock("dog", "cat", "owl")
lock.Lock()
assert.Equal(t, 3, len(locks.list))
defer lock.Unlock()
lock.Unlock()
<-time.After(100 * time.Millisecond)
wg.Done()
}()
wg.Wait()

View File

@@ -4,12 +4,16 @@ go_library(
name = "go_default_library",
srcs = [
"chain_info.go",
"error.go",
"head.go",
"head_sync_committee_info.go",
"info.go",
"init_sync_process_block.go",
"log.go",
"metrics.go",
"new_slot.go",
"optimistic_sync.go",
"options.go",
"pow_block.go",
"process_attestation.go",
"process_attestation_helpers.go",
"process_block.go",
@@ -17,20 +21,24 @@ go_library(
"receive_attestation.go",
"receive_block.go",
"service.go",
"state_balance_cache.go",
"weak_subjectivity_checks.go",
],
importpath = "github.com/prysmaticlabs/prysm/beacon-chain/blockchain",
visibility = [
"//beacon-chain:__subpackages__",
"//testing/fuzz:__pkg__",
"//cmd/beacon-chain:__subpackages__",
"//testing/slasher/simulator:__pkg__",
"//testing/spectest:__subpackages__",
],
deps = [
"//async:go_default_library",
"//async/event:go_default_library",
"//beacon-chain/blockchain/store:go_default_library",
"//beacon-chain/cache:go_default_library",
"//beacon-chain/cache/depositcache:go_default_library",
"//beacon-chain/core/altair:go_default_library",
"//beacon-chain/core/blocks:go_default_library",
"//beacon-chain/core/epoch/precompute:go_default_library",
"//beacon-chain/core/feed:go_default_library",
"//beacon-chain/core/feed/state:go_default_library",
@@ -41,20 +49,25 @@ go_library(
"//beacon-chain/db:go_default_library",
"//beacon-chain/db/filters:go_default_library",
"//beacon-chain/forkchoice:go_default_library",
"//beacon-chain/forkchoice/doubly-linked-tree:go_default_library",
"//beacon-chain/forkchoice/protoarray:go_default_library",
"//beacon-chain/forkchoice/types:go_default_library",
"//beacon-chain/operations/attestations:go_default_library",
"//beacon-chain/operations/slashings:go_default_library",
"//beacon-chain/operations/voluntaryexits:go_default_library",
"//beacon-chain/p2p:go_default_library",
"//beacon-chain/powchain:go_default_library",
"//beacon-chain/powchain/engine-api-client/v1:go_default_library",
"//beacon-chain/state:go_default_library",
"//beacon-chain/state/stategen:go_default_library",
"//cmd/beacon-chain/flags:go_default_library",
"//config/features:go_default_library",
"//config/fieldparams:go_default_library",
"//config/params:go_default_library",
"//crypto/bls:go_default_library",
"//encoding/bytesutil:go_default_library",
"//monitoring/tracing:go_default_library",
"//proto/engine/v1:go_default_library",
"//proto/eth/v1:go_default_library",
"//proto/prysm/v1alpha1:go_default_library",
"//proto/prysm/v1alpha1/attestation:go_default_library",
@@ -62,7 +75,9 @@ go_library(
"//runtime/version:go_default_library",
"//time:go_default_library",
"//time/slots:go_default_library",
"@com_github_emicklei_dot//:go_default_library",
"@com_github_ethereum_go_ethereum//common:go_default_library",
"@com_github_ethereum_go_ethereum//common/hexutil:go_default_library",
"@com_github_holiman_uint256//:go_default_library",
"@com_github_pkg_errors//:go_default_library",
"@com_github_prometheus_client_golang//prometheus:go_default_library",
"@com_github_prometheus_client_golang//prometheus/promauto:go_default_library",
@@ -89,10 +104,13 @@ go_test(
"checktags_test.go",
"head_sync_committee_info_test.go",
"head_test.go",
"info_test.go",
"init_test.go",
"log_test.go",
"metrics_test.go",
"mock_engine_test.go",
"mock_test.go",
"optimistic_sync_test.go",
"pow_block_test.go",
"process_attestation_test.go",
"process_block_test.go",
"receive_attestation_test.go",
@@ -115,6 +133,7 @@ go_test(
"//beacon-chain/powchain:go_default_library",
"//beacon-chain/state/stateutil:go_default_library",
"//beacon-chain/state/v1:go_default_library",
"//config/fieldparams:go_default_library",
"//config/params:go_default_library",
"//encoding/bytesutil:go_default_library",
"//proto/prysm/v1alpha1:go_default_library",
@@ -139,6 +158,7 @@ go_test(
"chain_info_norace_test.go",
"checktags_test.go",
"init_test.go",
"mock_test.go",
"receive_block_test.go",
"service_norace_test.go",
],

View File

@@ -6,12 +6,16 @@ import (
types "github.com/prysmaticlabs/eth2-types"
"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
"github.com/prysmaticlabs/prysm/beacon-chain/forkchoice"
doublylinkedtree "github.com/prysmaticlabs/prysm/beacon-chain/forkchoice/doubly-linked-tree"
"github.com/prysmaticlabs/prysm/beacon-chain/forkchoice/protoarray"
"github.com/prysmaticlabs/prysm/beacon-chain/state"
fieldparams "github.com/prysmaticlabs/prysm/config/fieldparams"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/block"
"github.com/prysmaticlabs/prysm/time/slots"
"go.opencensus.io/trace"
)
@@ -35,7 +39,7 @@ type TimeFetcher interface {
// GenesisFetcher retrieves the Ethereum consensus data related to its genesis.
type GenesisFetcher interface {
GenesisValidatorRoot() [32]byte
GenesisValidatorsRoot() [32]byte
}
// HeadFetcher defines a common interface for methods in blockchain service which
@@ -47,14 +51,16 @@ type HeadFetcher interface {
HeadState(ctx context.Context) (state.BeaconState, error)
HeadValidatorsIndices(ctx context.Context, epoch types.Epoch) ([]types.ValidatorIndex, error)
HeadSeed(ctx context.Context, epoch types.Epoch) ([32]byte, error)
HeadGenesisValidatorRoot() [32]byte
HeadGenesisValidatorsRoot() [32]byte
HeadETH1Data() *ethpb.Eth1Data
HeadPublicKeyToValidatorIndex(ctx context.Context, pubKey [48]byte) (types.ValidatorIndex, bool)
HeadValidatorIndexToPublicKey(ctx context.Context, index types.ValidatorIndex) ([48]byte, error)
ProtoArrayStore() *protoarray.Store
HeadPublicKeyToValidatorIndex(pubKey [fieldparams.BLSPubkeyLength]byte) (types.ValidatorIndex, bool)
HeadValidatorIndexToPublicKey(ctx context.Context, index types.ValidatorIndex) ([fieldparams.BLSPubkeyLength]byte, error)
ChainHeads() ([][32]byte, []types.Slot)
IsOptimistic(ctx context.Context) (bool, error)
IsOptimisticForRoot(ctx context.Context, root [32]byte) (bool, error)
HeadSyncCommitteeFetcher
HeadDomainFetcher
ForkChoicer() forkchoice.ForkChoicer
}
// ForkFetcher retrieves the current fork information of the Ethereum beacon chain.
@@ -76,31 +82,45 @@ type FinalizationFetcher interface {
PreviousJustifiedCheckpt() *ethpb.Checkpoint
}
// FinalizedCheckpt returns the latest finalized checkpoint from head state.
// FinalizedCheckpt returns the latest finalized checkpoint from chain store.
func (s *Service) FinalizedCheckpt() *ethpb.Checkpoint {
if s.finalizedCheckpt == nil {
cp := s.store.FinalizedCheckpt()
if cp == nil {
return &ethpb.Checkpoint{Root: params.BeaconConfig().ZeroHash[:]}
}
return ethpb.CopyCheckpoint(s.finalizedCheckpt)
return ethpb.CopyCheckpoint(cp)
}
// CurrentJustifiedCheckpt returns the current justified checkpoint from head state.
// CurrentJustifiedCheckpt returns the current justified checkpoint from chain store.
func (s *Service) CurrentJustifiedCheckpt() *ethpb.Checkpoint {
if s.justifiedCheckpt == nil {
cp := s.store.JustifiedCheckpt()
if cp == nil {
return &ethpb.Checkpoint{Root: params.BeaconConfig().ZeroHash[:]}
}
return ethpb.CopyCheckpoint(s.justifiedCheckpt)
return ethpb.CopyCheckpoint(cp)
}
// PreviousJustifiedCheckpt returns the previous justified checkpoint from head state.
// PreviousJustifiedCheckpt returns the previous justified checkpoint from chain store.
func (s *Service) PreviousJustifiedCheckpt() *ethpb.Checkpoint {
if s.prevJustifiedCheckpt == nil {
cp := s.store.PrevJustifiedCheckpt()
if cp == nil {
return &ethpb.Checkpoint{Root: params.BeaconConfig().ZeroHash[:]}
}
return ethpb.CopyCheckpoint(s.prevJustifiedCheckpt)
return ethpb.CopyCheckpoint(cp)
}
// BestJustifiedCheckpt returns the best justified checkpoint from store.
func (s *Service) BestJustifiedCheckpt() *ethpb.Checkpoint {
cp := s.store.BestJustifiedCheckpt()
// If there is no best justified checkpoint, return the checkpoint with root as zeros to be used for genesis cases.
if cp == nil {
return &ethpb.Checkpoint{Root: params.BeaconConfig().ZeroHash[:]}
}
return ethpb.CopyCheckpoint(cp)
}
// HeadSlot returns the slot of the head of the chain.
@@ -197,8 +217,8 @@ func (s *Service) HeadSeed(ctx context.Context, epoch types.Epoch) ([32]byte, er
return helpers.Seed(s.headState(ctx), epoch, params.BeaconConfig().DomainBeaconAttester)
}
// HeadGenesisValidatorRoot returns genesis validator root of the head state.
func (s *Service) HeadGenesisValidatorRoot() [32]byte {
// HeadGenesisValidatorsRoot returns genesis validators root of the head state.
func (s *Service) HeadGenesisValidatorsRoot() [32]byte {
s.headLock.RLock()
defer s.headLock.RUnlock()
@@ -206,7 +226,7 @@ func (s *Service) HeadGenesisValidatorRoot() [32]byte {
return [32]byte{}
}
return s.headGenesisValidatorRoot()
return s.headGenesisValidatorsRoot()
}
// HeadETH1Data returns the eth1data of the current head state.
@@ -220,26 +240,21 @@ func (s *Service) HeadETH1Data() *ethpb.Eth1Data {
return s.head.state.Eth1Data()
}
// ProtoArrayStore returns the proto array store object.
func (s *Service) ProtoArrayStore() *protoarray.Store {
return s.cfg.ForkChoiceStore.Store()
}
// GenesisTime returns the genesis time of beacon chain.
func (s *Service) GenesisTime() time.Time {
return s.genesisTime
}
// GenesisValidatorRoot returns the genesis validator
// GenesisValidatorsRoot returns the genesis validator
// root of the chain.
func (s *Service) GenesisValidatorRoot() [32]byte {
func (s *Service) GenesisValidatorsRoot() [32]byte {
s.headLock.RLock()
defer s.headLock.RUnlock()
if !s.hasHeadState() {
return [32]byte{}
}
return bytesutil.ToBytes32(s.head.state.GenesisValidatorRoot())
return bytesutil.ToBytes32(s.head.state.GenesisValidatorsRoot())
}
// CurrentFork retrieves the latest fork information of the beacon chain.
@@ -270,45 +285,92 @@ func (s *Service) IsCanonical(ctx context.Context, blockRoot [32]byte) (bool, er
// ChainHeads returns all possible chain heads (leaves of fork choice tree).
// Heads roots and heads slots are returned.
func (s *Service) ChainHeads() ([][32]byte, []types.Slot) {
nodes := s.ProtoArrayStore().Nodes()
// Deliberate choice to not preallocate space for below.
// Heads cant be more than 2-3 in the worst case where pre-allocation will be 64 to begin with.
headsRoots := make([][32]byte, 0)
headsSlots := make([]types.Slot, 0)
nonExistentNode := ^uint64(0)
for _, node := range nodes {
// Possible heads have no children.
if node.BestDescendant() == nonExistentNode && node.BestChild() == nonExistentNode {
headsRoots = append(headsRoots, node.Root())
headsSlots = append(headsSlots, node.Slot())
}
}
return headsRoots, headsSlots
return s.cfg.ForkChoiceStore.Tips()
}
// HeadPublicKeyToValidatorIndex returns the validator index of the `pubkey` in current head state.
func (s *Service) HeadPublicKeyToValidatorIndex(ctx context.Context, pubKey [48]byte) (types.ValidatorIndex, bool) {
func (s *Service) HeadPublicKeyToValidatorIndex(pubKey [fieldparams.BLSPubkeyLength]byte) (types.ValidatorIndex, bool) {
s.headLock.RLock()
defer s.headLock.RUnlock()
if !s.hasHeadState() {
return 0, false
}
return s.headState(ctx).ValidatorIndexByPubkey(pubKey)
return s.headValidatorIndexAtPubkey(pubKey)
}
// HeadValidatorIndexToPublicKey returns the pubkey of the validator `index` in current head state.
func (s *Service) HeadValidatorIndexToPublicKey(_ context.Context, index types.ValidatorIndex) ([48]byte, error) {
func (s *Service) HeadValidatorIndexToPublicKey(_ context.Context, index types.ValidatorIndex) ([fieldparams.BLSPubkeyLength]byte, error) {
s.headLock.RLock()
defer s.headLock.RUnlock()
if !s.hasHeadState() {
return [48]byte{}, nil
return [fieldparams.BLSPubkeyLength]byte{}, nil
}
v, err := s.headValidatorAtIndex(index)
if err != nil {
return [48]byte{}, err
return [fieldparams.BLSPubkeyLength]byte{}, err
}
return v.PublicKey(), nil
}
// ForkChoicer returns the forkchoice interface
func (s *Service) ForkChoicer() forkchoice.ForkChoicer {
return s.cfg.ForkChoiceStore
}
// IsOptimistic returns true if the current head is optimistic.
func (s *Service) IsOptimistic(ctx context.Context) (bool, error) {
s.headLock.RLock()
defer s.headLock.RUnlock()
if slots.ToEpoch(s.CurrentSlot()) < params.BeaconConfig().BellatrixForkEpoch {
return false, nil
}
return s.IsOptimisticForRoot(ctx, s.head.root)
}
// IsOptimisticForRoot takes the root and slot as aguments instead of the current head
// and returns true if it is optimistic.
func (s *Service) IsOptimisticForRoot(ctx context.Context, root [32]byte) (bool, error) {
optimistic, err := s.cfg.ForkChoiceStore.IsOptimistic(ctx, root)
if err == nil {
return optimistic, nil
}
if err != protoarray.ErrUnknownNodeRoot && err != doublylinkedtree.ErrNilNode {
return false, err
}
ss, err := s.cfg.BeaconDB.StateSummary(ctx, root)
if err != nil {
return false, err
}
if ss == nil {
return false, errInvalidNilSummary
}
validatedCheckpoint, err := s.cfg.BeaconDB.LastValidatedCheckpoint(ctx)
if err != nil {
return false, err
}
if slots.ToEpoch(ss.Slot) > validatedCheckpoint.Epoch {
return true, nil
}
if slots.ToEpoch(ss.Slot)+1 < validatedCheckpoint.Epoch {
return false, nil
}
lastValidated, err := s.cfg.BeaconDB.StateSummary(ctx, bytesutil.ToBytes32(validatedCheckpoint.Root))
if err != nil {
return false, err
}
return ss.Slot > lastValidated.Slot, nil
}
// SetGenesisTime sets the genesis time of beacon chain.
func (s *Service) SetGenesisTime(t time.Time) {
s.genesisTime = t
}
// ForkChoiceStore returns the fork choice store in the service
func (s *Service) ForkChoiceStore() forkchoice.ForkChoicer {
return s.cfg.ForkChoiceStore
}

View File

@@ -14,48 +14,62 @@ import (
func TestHeadSlot_DataRace(t *testing.T) {
beaconDB := testDB.SetupDB(t)
s := &Service{
cfg: &Config{BeaconDB: beaconDB},
cfg: &config{BeaconDB: beaconDB},
}
wait := make(chan struct{})
go func() {
defer close(wait)
require.NoError(t, s.saveHead(context.Background(), [32]byte{}))
}()
s.HeadSlot()
<-wait
}
func TestHeadRoot_DataRace(t *testing.T) {
beaconDB := testDB.SetupDB(t)
s := &Service{
cfg: &Config{BeaconDB: beaconDB, StateGen: stategen.New(beaconDB)},
cfg: &config{BeaconDB: beaconDB, StateGen: stategen.New(beaconDB)},
head: &head{root: [32]byte{'A'}},
}
wait := make(chan struct{})
go func() {
defer close(wait)
require.NoError(t, s.saveHead(context.Background(), [32]byte{}))
}()
_, err := s.HeadRoot(context.Background())
require.NoError(t, err)
<-wait
}
func TestHeadBlock_DataRace(t *testing.T) {
beaconDB := testDB.SetupDB(t)
wsb, err := wrapper.WrappedSignedBeaconBlock(&ethpb.SignedBeaconBlock{})
require.NoError(t, err)
s := &Service{
cfg: &Config{BeaconDB: beaconDB, StateGen: stategen.New(beaconDB)},
head: &head{block: wrapper.WrappedPhase0SignedBeaconBlock(&ethpb.SignedBeaconBlock{})},
cfg: &config{BeaconDB: beaconDB, StateGen: stategen.New(beaconDB)},
head: &head{block: wsb},
}
wait := make(chan struct{})
go func() {
defer close(wait)
require.NoError(t, s.saveHead(context.Background(), [32]byte{}))
}()
_, err := s.HeadBlock(context.Background())
_, err = s.HeadBlock(context.Background())
require.NoError(t, err)
<-wait
}
func TestHeadState_DataRace(t *testing.T) {
beaconDB := testDB.SetupDB(t)
s := &Service{
cfg: &Config{BeaconDB: beaconDB, StateGen: stategen.New(beaconDB)},
cfg: &config{BeaconDB: beaconDB, StateGen: stategen.New(beaconDB)},
}
wait := make(chan struct{})
go func() {
defer close(wait)
require.NoError(t, s.saveHead(context.Background(), [32]byte{}))
}()
_, err := s.HeadState(context.Background())
require.NoError(t, err)
<-wait
}

View File

@@ -8,8 +8,10 @@ import (
types "github.com/prysmaticlabs/eth2-types"
"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
testDB "github.com/prysmaticlabs/prysm/beacon-chain/db/testing"
doublylinkedtree "github.com/prysmaticlabs/prysm/beacon-chain/forkchoice/doubly-linked-tree"
"github.com/prysmaticlabs/prysm/beacon-chain/forkchoice/protoarray"
v1 "github.com/prysmaticlabs/prysm/beacon-chain/state/v1"
fieldparams "github.com/prysmaticlabs/prysm/config/fieldparams"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
@@ -39,12 +41,18 @@ func TestHeadRoot_Nil(t *testing.T) {
assert.DeepEqual(t, params.BeaconConfig().ZeroHash[:], headRoot, "Incorrect pre chain start value")
}
func TestService_ForkChoiceStore(t *testing.T) {
c := &Service{cfg: &config{ForkChoiceStore: doublylinkedtree.New(0, 0)}}
p := c.ForkChoiceStore()
require.Equal(t, 0, int(p.FinalizedEpoch()))
}
func TestFinalizedCheckpt_CanRetrieve(t *testing.T) {
beaconDB := testDB.SetupDB(t)
cp := &ethpb.Checkpoint{Epoch: 5, Root: bytesutil.PadTo([]byte("foo"), 32)}
c := setupBeaconChain(t, beaconDB)
c.finalizedCheckpt = cp
c.store.SetFinalizedCheckpt(cp)
assert.Equal(t, cp.Epoch, c.FinalizedCheckpt().Epoch, "Unexpected finalized epoch")
}
@@ -55,9 +63,9 @@ func TestFinalizedCheckpt_GenesisRootOk(t *testing.T) {
genesisRoot := [32]byte{'A'}
cp := &ethpb.Checkpoint{Root: genesisRoot[:]}
c := setupBeaconChain(t, beaconDB)
c.finalizedCheckpt = cp
c.genesisRoot = genesisRoot
assert.DeepEqual(t, c.genesisRoot[:], c.FinalizedCheckpt().Root)
c.store.SetFinalizedCheckpt(cp)
c.originBlockRoot = genesisRoot
assert.DeepEqual(t, c.originBlockRoot[:], c.FinalizedCheckpt().Root)
}
func TestCurrentJustifiedCheckpt_CanRetrieve(t *testing.T) {
@@ -66,7 +74,7 @@ func TestCurrentJustifiedCheckpt_CanRetrieve(t *testing.T) {
c := setupBeaconChain(t, beaconDB)
assert.Equal(t, params.BeaconConfig().ZeroHash, bytesutil.ToBytes32(c.CurrentJustifiedCheckpt().Root), "Unexpected justified epoch")
cp := &ethpb.Checkpoint{Epoch: 6, Root: bytesutil.PadTo([]byte("foo"), 32)}
c.justifiedCheckpt = cp
c.store.SetJustifiedCheckpt(cp)
assert.Equal(t, cp.Epoch, c.CurrentJustifiedCheckpt().Epoch, "Unexpected justified epoch")
}
@@ -76,9 +84,9 @@ func TestJustifiedCheckpt_GenesisRootOk(t *testing.T) {
c := setupBeaconChain(t, beaconDB)
genesisRoot := [32]byte{'B'}
cp := &ethpb.Checkpoint{Root: genesisRoot[:]}
c.justifiedCheckpt = cp
c.genesisRoot = genesisRoot
assert.DeepEqual(t, c.genesisRoot[:], c.CurrentJustifiedCheckpt().Root)
c.store.SetJustifiedCheckpt(cp)
c.originBlockRoot = genesisRoot
assert.DeepEqual(t, c.originBlockRoot[:], c.CurrentJustifiedCheckpt().Root)
}
func TestPreviousJustifiedCheckpt_CanRetrieve(t *testing.T) {
@@ -87,7 +95,7 @@ func TestPreviousJustifiedCheckpt_CanRetrieve(t *testing.T) {
cp := &ethpb.Checkpoint{Epoch: 7, Root: bytesutil.PadTo([]byte("foo"), 32)}
c := setupBeaconChain(t, beaconDB)
assert.Equal(t, params.BeaconConfig().ZeroHash, bytesutil.ToBytes32(c.CurrentJustifiedCheckpt().Root), "Unexpected justified epoch")
c.prevJustifiedCheckpt = cp
c.store.SetPrevJustifiedCheckpt(cp)
assert.Equal(t, cp.Epoch, c.PreviousJustifiedCheckpt().Epoch, "Unexpected previous justified epoch")
}
@@ -97,9 +105,9 @@ func TestPrevJustifiedCheckpt_GenesisRootOk(t *testing.T) {
genesisRoot := [32]byte{'C'}
cp := &ethpb.Checkpoint{Root: genesisRoot[:]}
c := setupBeaconChain(t, beaconDB)
c.prevJustifiedCheckpt = cp
c.genesisRoot = genesisRoot
assert.DeepEqual(t, c.genesisRoot[:], c.PreviousJustifiedCheckpt().Root)
c.store.SetPrevJustifiedCheckpt(cp)
c.originBlockRoot = genesisRoot
assert.DeepEqual(t, c.originBlockRoot[:], c.PreviousJustifiedCheckpt().Root)
}
func TestHeadSlot_CanRetrieve(t *testing.T) {
@@ -120,12 +128,14 @@ func TestHeadRoot_CanRetrieve(t *testing.T) {
func TestHeadRoot_UseDB(t *testing.T) {
beaconDB := testDB.SetupDB(t)
c := &Service{cfg: &Config{BeaconDB: beaconDB}}
c := &Service{cfg: &config{BeaconDB: beaconDB}}
c.head = &head{root: params.BeaconConfig().ZeroHash}
b := util.NewBeaconBlock()
br, err := b.Block.HashTreeRoot()
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(context.Background(), wrapper.WrappedPhase0SignedBeaconBlock(b)))
wsb, err := wrapper.WrappedSignedBeaconBlock(b)
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(context.Background(), wsb))
require.NoError(t, beaconDB.SaveStateSummary(context.Background(), &ethpb.StateSummary{Root: br[:]}))
require.NoError(t, beaconDB.SaveHeadBlockRoot(context.Background(), br))
r, err := c.HeadRoot(context.Background())
@@ -138,8 +148,10 @@ func TestHeadBlock_CanRetrieve(t *testing.T) {
b.Block.Slot = 1
s, err := v1.InitializeFromProto(&ethpb.BeaconState{})
require.NoError(t, err)
wsb, err := wrapper.WrappedSignedBeaconBlock(b)
require.NoError(t, err)
c := &Service{}
c.head = &head{block: wrapper.WrappedPhase0SignedBeaconBlock(b), state: s}
c.head = &head{block: wsb, state: s}
recevied, err := c.HeadBlock(context.Background())
require.NoError(t, err)
@@ -184,15 +196,15 @@ func TestCurrentFork_NilHeadSTate(t *testing.T) {
}
}
func TestGenesisValidatorRoot_CanRetrieve(t *testing.T) {
func TestGenesisValidatorsRoot_CanRetrieve(t *testing.T) {
// Should not panic if head state is nil.
c := &Service{}
assert.Equal(t, [32]byte{}, c.GenesisValidatorRoot(), "Did not get correct genesis validator root")
assert.Equal(t, [32]byte{}, c.GenesisValidatorsRoot(), "Did not get correct genesis validators root")
s, err := v1.InitializeFromProto(&ethpb.BeaconState{GenesisValidatorsRoot: []byte{'a'}})
require.NoError(t, err)
c.head = &head{state: s}
assert.Equal(t, [32]byte{'a'}, c.GenesisValidatorRoot(), "Did not get correct genesis validator root")
assert.Equal(t, [32]byte{'a'}, c.GenesisValidatorsRoot(), "Did not get correct genesis validators root")
}
func TestHeadETH1Data_Nil(t *testing.T) {
@@ -221,7 +233,9 @@ func TestIsCanonical_Ok(t *testing.T) {
blk.Block.Slot = 0
root, err := blk.Block.HashTreeRoot()
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(blk)))
wsb, err := wrapper.WrappedSignedBeaconBlock(blk)
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, wsb))
require.NoError(t, beaconDB.SaveGenesisBlockRoot(ctx, root))
can, err := c.IsCanonical(ctx, root)
require.NoError(t, err)
@@ -264,51 +278,64 @@ func TestService_HeadSeed(t *testing.T) {
require.DeepEqual(t, seed, root)
}
func TestService_HeadGenesisValidatorRoot(t *testing.T) {
func TestService_HeadGenesisValidatorsRoot(t *testing.T) {
s, _ := util.DeterministicGenesisState(t, 1)
c := &Service{}
c.head = &head{}
root := c.HeadGenesisValidatorRoot()
root := c.HeadGenesisValidatorsRoot()
require.Equal(t, [32]byte{}, root)
c.head = &head{state: s}
root = c.HeadGenesisValidatorRoot()
require.DeepEqual(t, root[:], s.GenesisValidatorRoot())
root = c.HeadGenesisValidatorsRoot()
require.DeepEqual(t, root[:], s.GenesisValidatorsRoot())
}
func TestService_ProtoArrayStore(t *testing.T) {
c := &Service{cfg: &Config{ForkChoiceStore: protoarray.New(0, 0, [32]byte{})}}
p := c.ProtoArrayStore()
require.Equal(t, 0, int(p.FinalizedEpoch()))
}
func TestService_ChainHeads(t *testing.T) {
func TestService_ChainHeads_ProtoArray(t *testing.T) {
ctx := context.Background()
c := &Service{cfg: &Config{ForkChoiceStore: protoarray.New(0, 0, [32]byte{})}}
require.NoError(t, c.cfg.ForkChoiceStore.ProcessBlock(ctx, 100, [32]byte{'a'}, [32]byte{}, [32]byte{}, 0, 0))
require.NoError(t, c.cfg.ForkChoiceStore.ProcessBlock(ctx, 101, [32]byte{'b'}, [32]byte{'a'}, [32]byte{}, 0, 0))
require.NoError(t, c.cfg.ForkChoiceStore.ProcessBlock(ctx, 102, [32]byte{'c'}, [32]byte{'b'}, [32]byte{}, 0, 0))
require.NoError(t, c.cfg.ForkChoiceStore.ProcessBlock(ctx, 103, [32]byte{'d'}, [32]byte{}, [32]byte{}, 0, 0))
require.NoError(t, c.cfg.ForkChoiceStore.ProcessBlock(ctx, 104, [32]byte{'e'}, [32]byte{'b'}, [32]byte{}, 0, 0))
c := &Service{cfg: &config{ForkChoiceStore: protoarray.New(0, 0,
params.BeaconConfig().ZeroHash)}}
require.NoError(t, c.cfg.ForkChoiceStore.InsertOptimisticBlock(ctx, 100, [32]byte{'a'}, [32]byte{}, params.BeaconConfig().ZeroHash, 0, 0))
require.NoError(t, c.cfg.ForkChoiceStore.InsertOptimisticBlock(ctx, 101, [32]byte{'b'}, [32]byte{'a'}, params.BeaconConfig().ZeroHash, 0, 0))
require.NoError(t, c.cfg.ForkChoiceStore.InsertOptimisticBlock(ctx, 102, [32]byte{'c'}, [32]byte{'b'}, params.BeaconConfig().ZeroHash, 0, 0))
require.NoError(t, c.cfg.ForkChoiceStore.InsertOptimisticBlock(ctx, 103, [32]byte{'d'}, [32]byte{}, params.BeaconConfig().ZeroHash, 0, 0))
require.NoError(t, c.cfg.ForkChoiceStore.InsertOptimisticBlock(ctx, 104, [32]byte{'e'}, [32]byte{'b'}, params.BeaconConfig().ZeroHash, 0, 0))
roots, slots := c.ChainHeads()
require.DeepEqual(t, [][32]byte{{'c'}, {'d'}, {'e'}}, roots)
require.DeepEqual(t, []types.Slot{102, 103, 104}, slots)
}
func TestService_ChainHeads_DoublyLinkedTree(t *testing.T) {
ctx := context.Background()
c := &Service{cfg: &config{ForkChoiceStore: doublylinkedtree.New(0, 0)}}
require.NoError(t, c.cfg.ForkChoiceStore.InsertOptimisticBlock(ctx, 100, [32]byte{'a'}, [32]byte{}, params.BeaconConfig().ZeroHash, 0, 0))
require.NoError(t, c.cfg.ForkChoiceStore.InsertOptimisticBlock(ctx, 101, [32]byte{'b'}, [32]byte{'a'}, params.BeaconConfig().ZeroHash, 0, 0))
require.NoError(t, c.cfg.ForkChoiceStore.InsertOptimisticBlock(ctx, 102, [32]byte{'c'}, [32]byte{'b'}, params.BeaconConfig().ZeroHash, 0, 0))
require.NoError(t, c.cfg.ForkChoiceStore.InsertOptimisticBlock(ctx, 103, [32]byte{'d'}, [32]byte{}, params.BeaconConfig().ZeroHash, 0, 0))
require.NoError(t, c.cfg.ForkChoiceStore.InsertOptimisticBlock(ctx, 104, [32]byte{'e'}, [32]byte{'b'}, params.BeaconConfig().ZeroHash, 0, 0))
roots, slots := c.ChainHeads()
require.Equal(t, 3, len(roots))
rootMap := map[[32]byte]types.Slot{[32]byte{'c'}: 102, [32]byte{'d'}: 103, [32]byte{'e'}: 104}
for i, root := range roots {
slot, ok := rootMap[root]
require.Equal(t, true, ok)
require.Equal(t, slot, slots[i])
}
}
func TestService_HeadPublicKeyToValidatorIndex(t *testing.T) {
s, _ := util.DeterministicGenesisState(t, 10)
c := &Service{}
c.head = &head{state: s}
_, e := c.HeadPublicKeyToValidatorIndex(context.Background(), [48]byte{})
_, e := c.HeadPublicKeyToValidatorIndex([fieldparams.BLSPubkeyLength]byte{})
require.Equal(t, false, e)
v, err := s.ValidatorAtIndex(0)
require.NoError(t, err)
i, e := c.HeadPublicKeyToValidatorIndex(context.Background(), bytesutil.ToBytes48(v.PublicKey))
i, e := c.HeadPublicKeyToValidatorIndex(bytesutil.ToBytes48(v.PublicKey))
require.Equal(t, true, e)
require.Equal(t, types.ValidatorIndex(0), i)
}
@@ -317,12 +344,12 @@ func TestService_HeadPublicKeyToValidatorIndexNil(t *testing.T) {
c := &Service{}
c.head = nil
idx, e := c.HeadPublicKeyToValidatorIndex(context.Background(), [48]byte{})
idx, e := c.HeadPublicKeyToValidatorIndex([fieldparams.BLSPubkeyLength]byte{})
require.Equal(t, false, e)
require.Equal(t, types.ValidatorIndex(0), idx)
c.head = &head{state: nil}
i, e := c.HeadPublicKeyToValidatorIndex(context.Background(), [48]byte{})
i, e := c.HeadPublicKeyToValidatorIndex([fieldparams.BLSPubkeyLength]byte{})
require.Equal(t, false, e)
require.Equal(t, types.ValidatorIndex(0), i)
}
@@ -347,10 +374,160 @@ func TestService_HeadValidatorIndexToPublicKeyNil(t *testing.T) {
p, err := c.HeadValidatorIndexToPublicKey(context.Background(), 0)
require.NoError(t, err)
require.Equal(t, [48]byte{}, p)
require.Equal(t, [fieldparams.BLSPubkeyLength]byte{}, p)
c.head = &head{state: nil}
p, err = c.HeadValidatorIndexToPublicKey(context.Background(), 0)
require.NoError(t, err)
require.Equal(t, [48]byte{}, p)
require.Equal(t, [fieldparams.BLSPubkeyLength]byte{}, p)
}
func TestService_IsOptimistic_ProtoArray(t *testing.T) {
params.SetupTestConfigCleanup(t)
cfg := params.BeaconConfig()
cfg.BellatrixForkEpoch = 0
params.OverrideBeaconConfig(cfg)
ctx := context.Background()
c := &Service{cfg: &config{ForkChoiceStore: protoarray.New(0, 0, [32]byte{})}, head: &head{slot: 101, root: [32]byte{'b'}}}
require.NoError(t, c.cfg.ForkChoiceStore.InsertOptimisticBlock(ctx, 100, [32]byte{'a'}, [32]byte{}, params.BeaconConfig().ZeroHash, 0, 0))
require.NoError(t, c.cfg.ForkChoiceStore.InsertOptimisticBlock(ctx, 101, [32]byte{'b'}, [32]byte{'a'}, params.BeaconConfig().ZeroHash, 0, 0))
opt, err := c.IsOptimistic(ctx)
require.NoError(t, err)
require.Equal(t, true, opt)
}
func TestService_IsOptimistic_DoublyLinkedTree(t *testing.T) {
params.SetupTestConfigCleanup(t)
cfg := params.BeaconConfig()
cfg.BellatrixForkEpoch = 0
params.OverrideBeaconConfig(cfg)
ctx := context.Background()
c := &Service{cfg: &config{ForkChoiceStore: doublylinkedtree.New(0, 0)}, head: &head{slot: 101, root: [32]byte{'b'}}}
require.NoError(t, c.cfg.ForkChoiceStore.InsertOptimisticBlock(ctx, 100, [32]byte{'a'}, [32]byte{}, params.BeaconConfig().ZeroHash, 0, 0))
require.NoError(t, c.cfg.ForkChoiceStore.InsertOptimisticBlock(ctx, 101, [32]byte{'b'}, [32]byte{'a'}, params.BeaconConfig().ZeroHash, 0, 0))
opt, err := c.IsOptimistic(ctx)
require.NoError(t, err)
require.Equal(t, true, opt)
}
func TestService_IsOptimisticBeforeBellatrix(t *testing.T) {
ctx := context.Background()
c := &Service{genesisTime: time.Now()}
opt, err := c.IsOptimistic(ctx)
require.NoError(t, err)
require.Equal(t, false, opt)
}
func TestService_IsOptimisticForRoot_ProtoArray(t *testing.T) {
ctx := context.Background()
c := &Service{cfg: &config{ForkChoiceStore: protoarray.New(0, 0, [32]byte{})}, head: &head{slot: 101, root: [32]byte{'b'}}}
require.NoError(t, c.cfg.ForkChoiceStore.InsertOptimisticBlock(ctx, 100, [32]byte{'a'}, [32]byte{}, params.BeaconConfig().ZeroHash, 0, 0))
require.NoError(t, c.cfg.ForkChoiceStore.InsertOptimisticBlock(ctx, 101, [32]byte{'b'}, [32]byte{'a'}, params.BeaconConfig().ZeroHash, 0, 0))
opt, err := c.IsOptimisticForRoot(ctx, [32]byte{'a'})
require.NoError(t, err)
require.Equal(t, true, opt)
}
func TestService_IsOptimisticForRoot_DoublyLinkedTree(t *testing.T) {
ctx := context.Background()
c := &Service{cfg: &config{ForkChoiceStore: doublylinkedtree.New(0, 0)}, head: &head{slot: 101, root: [32]byte{'b'}}}
require.NoError(t, c.cfg.ForkChoiceStore.InsertOptimisticBlock(ctx, 100, [32]byte{'a'}, [32]byte{}, params.BeaconConfig().ZeroHash, 0, 0))
require.NoError(t, c.cfg.ForkChoiceStore.InsertOptimisticBlock(ctx, 101, [32]byte{'b'}, [32]byte{'a'}, params.BeaconConfig().ZeroHash, 0, 0))
opt, err := c.IsOptimisticForRoot(ctx, [32]byte{'a'})
require.NoError(t, err)
require.Equal(t, true, opt)
}
func TestService_IsOptimisticForRoot_DB_ProtoArray(t *testing.T) {
beaconDB := testDB.SetupDB(t)
ctx := context.Background()
c := &Service{cfg: &config{BeaconDB: beaconDB, ForkChoiceStore: protoarray.New(0, 0, [32]byte{})}, head: &head{slot: 101, root: [32]byte{'b'}}}
c.head = &head{root: params.BeaconConfig().ZeroHash}
b := util.NewBeaconBlock()
b.Block.Slot = 10
br, err := b.Block.HashTreeRoot()
require.NoError(t, err)
wsb, err := wrapper.WrappedSignedBeaconBlock(b)
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(context.Background(), wsb))
require.NoError(t, beaconDB.SaveStateSummary(context.Background(), &ethpb.StateSummary{Root: br[:], Slot: 10}))
optimisticBlock := util.NewBeaconBlock()
optimisticBlock.Block.Slot = 11
optimisticRoot, err := optimisticBlock.Block.HashTreeRoot()
require.NoError(t, err)
wsb, err = wrapper.WrappedSignedBeaconBlock(optimisticBlock)
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(context.Background(), wsb))
validatedBlock := util.NewBeaconBlock()
validatedBlock.Block.Slot = 9
validatedRoot, err := validatedBlock.Block.HashTreeRoot()
require.NoError(t, err)
wsb, err = wrapper.WrappedSignedBeaconBlock(validatedBlock)
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(context.Background(), wsb))
validatedCheckpoint := &ethpb.Checkpoint{Root: br[:]}
require.NoError(t, beaconDB.SaveLastValidatedCheckpoint(ctx, validatedCheckpoint))
require.NoError(t, beaconDB.SaveStateSummary(context.Background(), &ethpb.StateSummary{Root: optimisticRoot[:], Slot: 11}))
optimistic, err := c.IsOptimisticForRoot(ctx, optimisticRoot)
require.NoError(t, err)
require.Equal(t, true, optimistic)
require.NoError(t, beaconDB.SaveStateSummary(context.Background(), &ethpb.StateSummary{Root: validatedRoot[:], Slot: 9}))
validated, err := c.IsOptimisticForRoot(ctx, validatedRoot)
require.NoError(t, err)
require.Equal(t, false, validated)
}
func TestService_IsOptimisticForRoot__DB_DoublyLinkedTree(t *testing.T) {
beaconDB := testDB.SetupDB(t)
ctx := context.Background()
c := &Service{cfg: &config{BeaconDB: beaconDB, ForkChoiceStore: doublylinkedtree.New(0, 0)}, head: &head{slot: 101, root: [32]byte{'b'}}}
c.head = &head{root: params.BeaconConfig().ZeroHash}
b := util.NewBeaconBlock()
b.Block.Slot = 10
br, err := b.Block.HashTreeRoot()
require.NoError(t, err)
wsb, err := wrapper.WrappedSignedBeaconBlock(b)
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(context.Background(), wsb))
require.NoError(t, beaconDB.SaveStateSummary(context.Background(), &ethpb.StateSummary{Root: br[:], Slot: 10}))
optimisticBlock := util.NewBeaconBlock()
optimisticBlock.Block.Slot = 11
optimisticRoot, err := optimisticBlock.Block.HashTreeRoot()
require.NoError(t, err)
wsb, err = wrapper.WrappedSignedBeaconBlock(optimisticBlock)
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(context.Background(), wsb))
validatedBlock := util.NewBeaconBlock()
validatedBlock.Block.Slot = 9
validatedRoot, err := validatedBlock.Block.HashTreeRoot()
require.NoError(t, err)
wsb, err = wrapper.WrappedSignedBeaconBlock(validatedBlock)
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(context.Background(), wsb))
validatedCheckpoint := &ethpb.Checkpoint{Root: br[:]}
require.NoError(t, beaconDB.SaveLastValidatedCheckpoint(ctx, validatedCheckpoint))
require.NoError(t, beaconDB.SaveStateSummary(context.Background(), &ethpb.StateSummary{Root: optimisticRoot[:], Slot: 11}))
optimistic, err := c.IsOptimisticForRoot(ctx, optimisticRoot)
require.NoError(t, err)
require.Equal(t, true, optimistic)
require.NoError(t, beaconDB.SaveStateSummary(context.Background(), &ethpb.StateSummary{Root: validatedRoot[:], Slot: 9}))
validated, err := c.IsOptimisticForRoot(ctx, validatedRoot)
require.NoError(t, err)
require.Equal(t, false, validated)
}

View File

@@ -1,3 +1,4 @@
//go:build !develop
// +build !develop
package blockchain

View File

@@ -0,0 +1,19 @@
package blockchain
import "github.com/pkg/errors"
var (
// errNilJustifiedInStore is returned when a nil justified checkpt is returned from store.
errNilJustifiedInStore = errors.New("nil justified checkpoint returned from store")
// errNilBestJustifiedInStore is returned when a nil justified checkpt is returned from store.
errNilBestJustifiedInStore = errors.New("nil best justified checkpoint returned from store")
// errNilFinalizedInStore is returned when a nil finalized checkpt is returned from store.
errNilFinalizedInStore = errors.New("nil finalized checkpoint returned from store")
// errInvalidNilSummary is returned when a nil summary is returned from the DB.
errInvalidNilSummary = errors.New("nil summary returned from the DB")
// errNilParentInDB is returned when a nil parent block is returned from the DB.
errNilParentInDB = errors.New("nil parent block in DB")
// errWrongBlockCound is returned when the wrong number of blocks or
// block roots is used
errWrongBlockCount = errors.New("wrong number of blocks or block roots")
)

View File

@@ -10,10 +10,11 @@ import (
"github.com/prysmaticlabs/prysm/beacon-chain/core/feed"
statefeed "github.com/prysmaticlabs/prysm/beacon-chain/core/feed/state"
"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
"github.com/prysmaticlabs/prysm/beacon-chain/core/time"
doublylinkedtree "github.com/prysmaticlabs/prysm/beacon-chain/forkchoice/doubly-linked-tree"
"github.com/prysmaticlabs/prysm/beacon-chain/forkchoice/protoarray"
"github.com/prysmaticlabs/prysm/beacon-chain/state"
"github.com/prysmaticlabs/prysm/config/features"
fieldparams "github.com/prysmaticlabs/prysm/config/fieldparams"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
ethpbv1 "github.com/prysmaticlabs/prysm/proto/eth/v1"
@@ -23,6 +24,21 @@ import (
"go.opencensus.io/trace"
)
// UpdateHeadWithBalances updates the beacon state head after getting justified balanced from cache.
func (s *Service) UpdateHeadWithBalances(ctx context.Context) error {
cp := s.store.JustifiedCheckpt()
if cp == nil {
return errors.New("no justified checkpoint")
}
balances, err := s.justifiedBalances.get(ctx, bytesutil.ToBytes32(cp.Root))
if err != nil {
msg := fmt.Sprintf("could not read balances for state w/ justified checkpoint %#x", cp.Root)
return errors.Wrap(err, msg)
}
return s.updateHead(ctx, balances)
}
// This defines the current chain service's view of head.
type head struct {
slot types.Slot // current head slot.
@@ -37,24 +53,20 @@ func (s *Service) updateHead(ctx context.Context, balances []uint64) error {
ctx, span := trace.StartSpan(ctx, "blockChain.updateHead")
defer span.End()
// To get the proper head update, a node first checks its best justified
// can become justified. This is designed to prevent bounce attack and
// ensure head gets its best justified info.
if s.bestJustifiedCheckpt.Epoch > s.justifiedCheckpt.Epoch {
s.justifiedCheckpt = s.bestJustifiedCheckpt
if err := s.cacheJustifiedStateBalances(ctx, bytesutil.ToBytes32(s.justifiedCheckpt.Root)); err != nil {
return err
}
}
// Get head from the fork choice service.
f := s.finalizedCheckpt
j := s.justifiedCheckpt
// To get head before the first justified epoch, the fork choice will start with genesis root
f := s.store.FinalizedCheckpt()
if f == nil {
return errNilFinalizedInStore
}
j := s.store.JustifiedCheckpt()
if j == nil {
return errNilJustifiedInStore
}
// To get head before the first justified epoch, the fork choice will start with origin root
// instead of zero hashes.
headStartRoot := bytesutil.ToBytes32(j.Root)
if headStartRoot == params.BeaconConfig().ZeroHash {
headStartRoot = s.genesisRoot
headStartRoot = s.originBlockRoot
}
// In order to process head, fork choice store requires justified info.
@@ -66,7 +78,11 @@ func (s *Service) updateHead(ctx context.Context, balances []uint64) error {
if err != nil {
return err
}
s.cfg.ForkChoiceStore = protoarray.New(j.Epoch, f.Epoch, bytesutil.ToBytes32(f.Root))
if features.Get().EnableForkChoiceDoublyLinkedTree {
s.cfg.ForkChoiceStore = doublylinkedtree.New(j.Epoch, f.Epoch)
} else {
s.cfg.ForkChoiceStore = protoarray.New(j.Epoch, f.Epoch, bytesutil.ToBytes32(f.Root))
}
if err := s.insertBlockToForkChoiceStore(ctx, jb.Block(), headStartRoot, f, j); err != nil {
return err
}
@@ -107,8 +123,8 @@ func (s *Service) saveHead(ctx context.Context, headRoot [32]byte) error {
if err != nil {
return err
}
if newHeadBlock == nil || newHeadBlock.IsNil() || newHeadBlock.Block().IsNil() {
return errors.New("cannot save nil head block")
if err := helpers.BeaconBlockIsNil(newHeadBlock); err != nil {
return err
}
// Get the new head state from cached state or DB.
@@ -132,16 +148,21 @@ func (s *Service) saveHead(ctx context.Context, headRoot [32]byte) error {
"oldSlot": fmt.Sprintf("%d", headSlot),
}).Debug("Chain reorg occurred")
absoluteSlotDifference := slots.AbsoluteValueSlotDifference(newHeadSlot, headSlot)
isOptimistic, err := s.IsOptimistic(ctx)
if err != nil {
return errors.Wrap(err, "could not check if node is optimistically synced")
}
s.cfg.StateNotifier.StateFeed().Send(&feed.Event{
Type: statefeed.Reorg,
Data: &ethpbv1.EventChainReorg{
Slot: newHeadSlot,
Depth: absoluteSlotDifference,
OldHeadBlock: oldHeadRoot[:],
NewHeadBlock: headRoot[:],
OldHeadState: oldStateRoot,
NewHeadState: newStateRoot,
Epoch: slots.ToEpoch(newHeadSlot),
Slot: newHeadSlot,
Depth: absoluteSlotDifference,
OldHeadBlock: oldHeadRoot[:],
NewHeadBlock: headRoot[:],
OldHeadState: oldStateRoot,
NewHeadState: newStateRoot,
Epoch: slots.ToEpoch(newHeadSlot),
ExecutionOptimistic: isOptimistic,
},
})
@@ -163,7 +184,7 @@ func (s *Service) saveHead(ctx context.Context, headRoot [32]byte) error {
// Forward an event capturing a new chain head over a common event feed
// done in a goroutine to avoid blocking the critical runtime main routine.
go func() {
if err := s.notifyNewHeadEvent(newHeadSlot, newHeadState, newStateRoot, headRoot[:]); err != nil {
if err := s.notifyNewHeadEvent(ctx, newHeadSlot, newHeadState, newStateRoot, headRoot[:]); err != nil {
log.WithError(err).Error("Could not notify event feed of new chain head")
}
}()
@@ -175,7 +196,7 @@ func (s *Service) saveHead(ctx context.Context, headRoot [32]byte) error {
// root in DB. With the inception of initial-sync-cache-state flag, it uses finalized
// check point as anchors to resume sync therefore head is no longer needed to be saved on per slot basis.
func (s *Service) saveHeadNoDB(ctx context.Context, b block.SignedBeaconBlock, r [32]byte, hs state.BeaconState) error {
if err := helpers.VerifyNilBeaconBlock(b); err != nil {
if err := helpers.BeaconBlockIsNil(b); err != nil {
return err
}
cachedHeadRoot, err := s.HeadRoot(ctx)
@@ -248,16 +269,16 @@ func (s *Service) headBlock() block.SignedBeaconBlock {
// It does a full copy on head state for immutability.
// This is a lock free version.
func (s *Service) headState(ctx context.Context) state.BeaconState {
ctx, span := trace.StartSpan(ctx, "blockChain.headState")
_, span := trace.StartSpan(ctx, "blockChain.headState")
defer span.End()
return s.head.state.Copy()
}
// This returns the genesis validator root of the head state.
// This returns the genesis validators root of the head state.
// This is a lock free version.
func (s *Service) headGenesisValidatorRoot() [32]byte {
return bytesutil.ToBytes32(s.head.state.GenesisValidatorRoot())
func (s *Service) headGenesisValidatorsRoot() [32]byte {
return bytesutil.ToBytes32(s.head.state.GenesisValidatorsRoot())
}
// This returns the validator referenced by the provided index in
@@ -267,73 +288,30 @@ func (s *Service) headValidatorAtIndex(index types.ValidatorIndex) (state.ReadOn
return s.head.state.ValidatorAtIndexReadOnly(index)
}
// This returns the validator index referenced by the provided pubkey in
// the head state.
// This is a lock free version.
func (s *Service) headValidatorIndexAtPubkey(pubKey [fieldparams.BLSPubkeyLength]byte) (types.ValidatorIndex, bool) {
return s.head.state.ValidatorIndexByPubkey(pubKey)
}
// Returns true if head state exists.
// This is the lock free version.
func (s *Service) hasHeadState() bool {
return s.head != nil && s.head.state != nil
}
// This caches justified state balances to be used for fork choice.
func (s *Service) cacheJustifiedStateBalances(ctx context.Context, justifiedRoot [32]byte) error {
if err := s.cfg.BeaconDB.SaveBlocks(ctx, s.getInitSyncBlocks()); err != nil {
return err
}
s.clearInitSyncBlocks()
var justifiedState state.BeaconState
var err error
if justifiedRoot == s.genesisRoot {
justifiedState, err = s.cfg.BeaconDB.GenesisState(ctx)
if err != nil {
return err
}
} else {
justifiedState, err = s.cfg.StateGen.StateByRoot(ctx, justifiedRoot)
if err != nil {
return err
}
}
if justifiedState == nil || justifiedState.IsNil() {
return errors.New("justified state can't be nil")
}
epoch := time.CurrentEpoch(justifiedState)
justifiedBalances := make([]uint64, justifiedState.NumValidators())
if err := justifiedState.ReadFromEveryValidator(func(idx int, val state.ReadOnlyValidator) error {
if helpers.IsActiveValidatorUsingTrie(val, epoch) {
justifiedBalances[idx] = val.EffectiveBalance()
} else {
justifiedBalances[idx] = 0
}
return nil
}); err != nil {
return err
}
s.justifiedBalancesLock.Lock()
defer s.justifiedBalancesLock.Unlock()
s.justifiedBalances = justifiedBalances
return nil
}
func (s *Service) getJustifiedBalances() []uint64 {
s.justifiedBalancesLock.RLock()
defer s.justifiedBalancesLock.RUnlock()
return s.justifiedBalances
}
// Notifies a common event feed of a new chain head event. Called right after a new
// chain head is determined, set, and saved to disk.
func (s *Service) notifyNewHeadEvent(
ctx context.Context,
newHeadSlot types.Slot,
newHeadState state.BeaconState,
newHeadStateRoot,
newHeadRoot []byte,
) error {
previousDutyDependentRoot := s.genesisRoot[:]
currentDutyDependentRoot := s.genesisRoot[:]
previousDutyDependentRoot := s.originBlockRoot[:]
currentDutyDependentRoot := s.originBlockRoot[:]
var previousDutyEpoch types.Epoch
currentDutyEpoch := slots.ToEpoch(newHeadSlot)
@@ -360,6 +338,10 @@ func (s *Service) notifyNewHeadEvent(
return errors.Wrap(err, "could not get duty dependent root")
}
}
isOptimistic, err := s.IsOptimistic(ctx)
if err != nil {
return errors.Wrap(err, "could not check if node is optimistically synced")
}
s.cfg.StateNotifier.StateFeed().Send(&feed.Event{
Type: statefeed.NewHead,
Data: &ethpbv1.EventHead{
@@ -369,6 +351,7 @@ func (s *Service) notifyNewHeadEvent(
EpochTransition: slots.IsEpochStart(newHeadSlot),
PreviousDutyDependentRoot: previousDutyDependentRoot,
CurrentDutyDependentRoot: currentDutyDependentRoot,
ExecutionOptimistic: isOptimistic,
},
})
return nil

View File

@@ -130,7 +130,7 @@ func (s *Service) domainWithHeadState(ctx context.Context, slot types.Slot, doma
if err != nil {
return nil, err
}
return signing.Domain(headState.Fork(), slots.ToEpoch(headState.Slot()), domain, headState.GenesisValidatorRoot())
return signing.Domain(headState.Fork(), slots.ToEpoch(headState.Slot()), domain, headState.GenesisValidatorsRoot())
}
// returns the head state that is advanced up to `slot`. It utilizes the cache `syncCommitteeHeadState` by retrieving using `slot` as key.
@@ -157,11 +157,9 @@ func (s *Service) getSyncCommitteeHeadState(ctx context.Context, slot types.Slot
if headState == nil || headState.IsNil() {
return nil, errors.New("nil state")
}
if slot > headState.Slot() {
headState, err = transition.ProcessSlots(ctx, headState, slot)
if err != nil {
return nil, err
}
headState, err = transition.ProcessSlotsIfPossible(ctx, headState, slot)
if err != nil {
return nil, err
}
syncHeadStateMiss.Inc()
err = syncCommitteeHeadStateCache.Put(slot, headState)

View File

@@ -18,7 +18,7 @@ import (
func TestService_headSyncCommitteeFetcher_Errors(t *testing.T) {
beaconDB := dbtest.SetupDB(t)
c := &Service{
cfg: &Config{
cfg: &config{
StateGen: stategen.New(beaconDB),
},
}
@@ -36,7 +36,7 @@ func TestService_headSyncCommitteeFetcher_Errors(t *testing.T) {
func TestService_HeadDomainFetcher_Errors(t *testing.T) {
beaconDB := dbtest.SetupDB(t)
c := &Service{
cfg: &Config{
cfg: &config{
StateGen: stategen.New(beaconDB),
},
}
@@ -122,7 +122,7 @@ func TestService_HeadSyncCommitteeDomain(t *testing.T) {
c := &Service{}
c.head = &head{state: s}
wanted, err := signing.Domain(s.Fork(), slots.ToEpoch(s.Slot()), params.BeaconConfig().DomainSyncCommittee, s.GenesisValidatorRoot())
wanted, err := signing.Domain(s.Fork(), slots.ToEpoch(s.Slot()), params.BeaconConfig().DomainSyncCommittee, s.GenesisValidatorsRoot())
require.NoError(t, err)
d, err := c.HeadSyncCommitteeDomain(context.Background(), 0)
@@ -136,7 +136,7 @@ func TestService_HeadSyncContributionProofDomain(t *testing.T) {
c := &Service{}
c.head = &head{state: s}
wanted, err := signing.Domain(s.Fork(), slots.ToEpoch(s.Slot()), params.BeaconConfig().DomainContributionAndProof, s.GenesisValidatorRoot())
wanted, err := signing.Domain(s.Fork(), slots.ToEpoch(s.Slot()), params.BeaconConfig().DomainContributionAndProof, s.GenesisValidatorsRoot())
require.NoError(t, err)
d, err := c.HeadSyncContributionProofDomain(context.Background(), 0)
@@ -150,7 +150,7 @@ func TestService_HeadSyncSelectionProofDomain(t *testing.T) {
c := &Service{}
c.head = &head{state: s}
wanted, err := signing.Domain(s.Fork(), slots.ToEpoch(s.Slot()), params.BeaconConfig().DomainSyncCommitteeSelectionProof, s.GenesisValidatorRoot())
wanted, err := signing.Domain(s.Fork(), slots.ToEpoch(s.Slot()), params.BeaconConfig().DomainSyncCommitteeSelectionProof, s.GenesisValidatorsRoot())
require.NoError(t, err)
d, err := c.HeadSyncSelectionProofDomain(context.Background(), 0)

View File

@@ -39,9 +39,10 @@ func TestSaveHead_Different(t *testing.T) {
service := setupBeaconChain(t, beaconDB)
util.NewBeaconBlock()
oldBlock := wrapper.WrappedPhase0SignedBeaconBlock(
oldBlock, err := wrapper.WrappedSignedBeaconBlock(
util.NewBeaconBlock(),
)
require.NoError(t, err)
require.NoError(t, service.cfg.BeaconDB.SaveBlock(context.Background(), oldBlock))
oldRoot, err := oldBlock.Block().HashTreeRoot()
require.NoError(t, err)
@@ -55,7 +56,9 @@ func TestSaveHead_Different(t *testing.T) {
newHeadSignedBlock.Block.Slot = 1
newHeadBlock := newHeadSignedBlock.Block
require.NoError(t, service.cfg.BeaconDB.SaveBlock(context.Background(), wrapper.WrappedPhase0SignedBeaconBlock(newHeadSignedBlock)))
wsb, err := wrapper.WrappedSignedBeaconBlock(newHeadSignedBlock)
require.NoError(t, err)
require.NoError(t, service.cfg.BeaconDB.SaveBlock(context.Background(), wsb))
newRoot, err := newHeadBlock.HashTreeRoot()
require.NoError(t, err)
headState, err := util.NewBeaconState()
@@ -80,9 +83,10 @@ func TestSaveHead_Different_Reorg(t *testing.T) {
beaconDB := testDB.SetupDB(t)
service := setupBeaconChain(t, beaconDB)
oldBlock := wrapper.WrappedPhase0SignedBeaconBlock(
oldBlock, err := wrapper.WrappedSignedBeaconBlock(
util.NewBeaconBlock(),
)
require.NoError(t, err)
require.NoError(t, service.cfg.BeaconDB.SaveBlock(context.Background(), oldBlock))
oldRoot, err := oldBlock.Block().HashTreeRoot()
require.NoError(t, err)
@@ -98,7 +102,9 @@ func TestSaveHead_Different_Reorg(t *testing.T) {
newHeadSignedBlock.Block.ParentRoot = reorgChainParent[:]
newHeadBlock := newHeadSignedBlock.Block
require.NoError(t, service.cfg.BeaconDB.SaveBlock(context.Background(), wrapper.WrappedPhase0SignedBeaconBlock(newHeadSignedBlock)))
wsb, err := wrapper.WrappedSignedBeaconBlock(newHeadSignedBlock)
require.NoError(t, err)
require.NoError(t, service.cfg.BeaconDB.SaveBlock(context.Background(), wsb))
newRoot, err := newHeadBlock.HashTreeRoot()
require.NoError(t, err)
headState, err := util.NewBeaconState()
@@ -123,13 +129,15 @@ func TestSaveHead_Different_Reorg(t *testing.T) {
func TestCacheJustifiedStateBalances_CanCache(t *testing.T) {
beaconDB := testDB.SetupDB(t)
service := setupBeaconChain(t, beaconDB)
ctx := context.Background()
state, _ := util.DeterministicGenesisState(t, 100)
r := [32]byte{'a'}
require.NoError(t, service.cfg.BeaconDB.SaveStateSummary(context.Background(), &ethpb.StateSummary{Root: r[:]}))
require.NoError(t, service.cfg.BeaconDB.SaveState(context.Background(), state, r))
require.NoError(t, service.cacheJustifiedStateBalances(context.Background(), r))
require.DeepEqual(t, service.getJustifiedBalances(), state.Balances(), "Incorrect justified balances")
balances, err := service.justifiedBalances.get(ctx, r)
require.NoError(t, err)
require.DeepEqual(t, balances, state.Balances(), "Incorrect justified balances")
}
func TestUpdateHead_MissingJustifiedRoot(t *testing.T) {
@@ -137,13 +145,15 @@ func TestUpdateHead_MissingJustifiedRoot(t *testing.T) {
service := setupBeaconChain(t, beaconDB)
b := util.NewBeaconBlock()
require.NoError(t, service.cfg.BeaconDB.SaveBlock(context.Background(), wrapper.WrappedPhase0SignedBeaconBlock(b)))
wsb, err := wrapper.WrappedSignedBeaconBlock(b)
require.NoError(t, err)
require.NoError(t, service.cfg.BeaconDB.SaveBlock(context.Background(), wsb))
r, err := b.Block.HashTreeRoot()
require.NoError(t, err)
service.justifiedCheckpt = &ethpb.Checkpoint{Root: r[:]}
service.finalizedCheckpt = &ethpb.Checkpoint{}
service.bestJustifiedCheckpt = &ethpb.Checkpoint{}
service.store.SetJustifiedCheckpt(&ethpb.Checkpoint{Root: r[:]})
service.store.SetFinalizedCheckpt(&ethpb.Checkpoint{})
service.store.SetBestJustifiedCheckpt(&ethpb.Checkpoint{})
require.NoError(t, service.updateHead(context.Background(), []uint64{}))
}
@@ -153,14 +163,14 @@ func Test_notifyNewHeadEvent(t *testing.T) {
bState, _ := util.DeterministicGenesisState(t, 10)
notifier := &mock.MockStateNotifier{RecordEvents: true}
srv := &Service{
cfg: &Config{
cfg: &config{
StateNotifier: notifier,
},
genesisRoot: [32]byte{1},
originBlockRoot: [32]byte{1},
}
newHeadStateRoot := [32]byte{2}
newHeadRoot := [32]byte{3}
err := srv.notifyNewHeadEvent(1, bState, newHeadStateRoot[:], newHeadRoot[:])
err := srv.notifyNewHeadEvent(context.Background(), 1, bState, newHeadStateRoot[:], newHeadRoot[:])
require.NoError(t, err)
events := notifier.ReceivedEvents()
require.Equal(t, 1, len(events))
@@ -172,8 +182,8 @@ func Test_notifyNewHeadEvent(t *testing.T) {
Block: newHeadRoot[:],
State: newHeadStateRoot[:],
EpochTransition: false,
PreviousDutyDependentRoot: srv.genesisRoot[:],
CurrentDutyDependentRoot: srv.genesisRoot[:],
PreviousDutyDependentRoot: srv.originBlockRoot[:],
CurrentDutyDependentRoot: srv.originBlockRoot[:],
}
require.DeepSSZEqual(t, wanted, eventHead)
})
@@ -182,10 +192,10 @@ func Test_notifyNewHeadEvent(t *testing.T) {
notifier := &mock.MockStateNotifier{RecordEvents: true}
genesisRoot := [32]byte{1}
srv := &Service{
cfg: &Config{
cfg: &config{
StateNotifier: notifier,
},
genesisRoot: genesisRoot,
originBlockRoot: genesisRoot,
}
epoch1Start, err := slots.EpochStart(1)
require.NoError(t, err)
@@ -195,7 +205,7 @@ func Test_notifyNewHeadEvent(t *testing.T) {
newHeadStateRoot := [32]byte{2}
newHeadRoot := [32]byte{3}
err = srv.notifyNewHeadEvent(epoch2Start, bState, newHeadStateRoot[:], newHeadRoot[:])
err = srv.notifyNewHeadEvent(context.Background(), epoch2Start, bState, newHeadStateRoot[:], newHeadRoot[:])
require.NoError(t, err)
events := notifier.ReceivedEvents()
require.Equal(t, 1, len(events))
@@ -231,7 +241,9 @@ func TestSaveOrphanedAtts(t *testing.T) {
service := setupBeaconChain(t, beaconDB)
service.genesisTime = time.Now()
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(b)))
wsb, err := wrapper.WrappedSignedBeaconBlock(b)
require.NoError(t, err)
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wsb))
require.NoError(t, service.saveOrphanedAtts(ctx, r))
require.Equal(t, len(b.Block.Body.Attestations), service.cfg.AttPool.AggregatedAttestationCount())
@@ -257,7 +269,9 @@ func TestSaveOrphanedAtts_CanFilter(t *testing.T) {
service := setupBeaconChain(t, beaconDB)
service.genesisTime = time.Now().Add(time.Duration(-1*int64(params.BeaconConfig().SlotsPerEpoch+1)*int64(params.BeaconConfig().SecondsPerSlot)) * time.Second)
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(b)))
wsb, err := wrapper.WrappedSignedBeaconBlock(b)
require.NoError(t, err)
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wsb))
require.NoError(t, service.saveOrphanedAtts(ctx, r))
require.Equal(t, 0, service.cfg.AttPool.AggregatedAttestationCount())

View File

@@ -1,99 +0,0 @@
package blockchain
import (
"encoding/hex"
"fmt"
"net/http"
"github.com/emicklei/dot"
"github.com/prysmaticlabs/prysm/config/params"
)
const template = `<html>
<head>
<script src="//cdnjs.cloudflare.com/ajax/libs/viz.js/2.1.2/viz.js"></script>
<script src="//cdnjs.cloudflare.com/ajax/libs/viz.js/2.1.2/full.render.js"></script>
<body>
<script type="application/javascript">
var graph = ` + "`%s`;" + `
var viz = new Viz();
viz.renderSVGElement(graph) // reading the graph.
.then(function(element) {
document.body.appendChild(element); // appends to document.
})
.catch(error => {
// Create a new Viz instance (@see Caveats page for more info)
viz = new Viz();
// Possibly display the error
console.error(error);
});
</script>
</head>
</body>
</html>`
// TreeHandler is a handler to serve /tree page in metrics.
func (s *Service) TreeHandler(w http.ResponseWriter, r *http.Request) {
headState, err := s.HeadState(r.Context())
if err != nil {
log.WithError(err).Error("Could not get head state")
return
}
if headState == nil || headState.IsNil() {
if _, err := w.Write([]byte("Unavailable during initial syncing")); err != nil {
log.WithError(err).Error("Failed to render p2p info page")
}
}
nodes := s.cfg.ForkChoiceStore.Nodes()
graph := dot.NewGraph(dot.Directed)
graph.Attr("rankdir", "RL")
graph.Attr("labeljust", "l")
dotNodes := make([]*dot.Node, len(nodes))
avgBalance := uint64(averageBalance(headState.Balances()))
for i := len(nodes) - 1; i >= 0; i-- {
// Construct label for each node.
slot := fmt.Sprintf("%d", nodes[i].Slot())
weight := fmt.Sprintf("%d", nodes[i].Weight()/1e9) // Convert unit Gwei to unit ETH.
votes := fmt.Sprintf("%d", nodes[i].Weight()/1e9/avgBalance)
index := fmt.Sprintf("%d", i)
g := nodes[i].Graffiti()
graffiti := hex.EncodeToString(g[:8])
label := "slot: " + slot + "\n votes: " + votes + "\n weight: " + weight + "\n graffiti: " + graffiti
var dotN dot.Node
if nodes[i].Parent() != ^uint64(0) {
dotN = graph.Node(index).Box().Attr("label", label)
}
if nodes[i].Slot() == s.HeadSlot() &&
nodes[i].BestDescendant() == ^uint64(0) &&
nodes[i].Parent() != ^uint64(0) {
dotN = dotN.Attr("color", "green")
}
dotNodes[i] = &dotN
}
for i := len(nodes) - 1; i >= 0; i-- {
if nodes[i].Parent() != ^uint64(0) && nodes[i].Parent() < uint64(len(dotNodes)) {
graph.Edge(*dotNodes[i], *dotNodes[nodes[i].Parent()])
}
}
w.WriteHeader(http.StatusOK)
w.Header().Set("Content-Type", "text/html")
if _, err := fmt.Fprintf(w, template, graph.String()); err != nil {
log.WithError(err).Error("Failed to render p2p info page")
}
}
func averageBalance(balances []uint64) float64 {
total := uint64(0)
for i := 0; i < len(balances); i++ {
total += balances[i]
}
return float64(total) / float64(len(balances)) / float64(params.BeaconConfig().GweiPerEth)
}

View File

@@ -1,49 +0,0 @@
package blockchain
import (
"context"
"net/http"
"net/http/httptest"
"testing"
testDB "github.com/prysmaticlabs/prysm/beacon-chain/db/testing"
"github.com/prysmaticlabs/prysm/beacon-chain/forkchoice/protoarray"
"github.com/prysmaticlabs/prysm/beacon-chain/state/stategen"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/wrapper"
"github.com/prysmaticlabs/prysm/testing/assert"
"github.com/prysmaticlabs/prysm/testing/require"
"github.com/prysmaticlabs/prysm/testing/util"
)
func TestService_TreeHandler(t *testing.T) {
req, err := http.NewRequest("GET", "/tree", nil)
require.NoError(t, err)
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
headState, err := util.NewBeaconState()
require.NoError(t, err)
require.NoError(t, headState.SetBalances([]uint64{params.BeaconConfig().GweiPerEth}))
cfg := &Config{
BeaconDB: beaconDB,
ForkChoiceStore: protoarray.New(
0, // justifiedEpoch
0, // finalizedEpoch
[32]byte{'a'},
),
StateGen: stategen.New(beaconDB),
}
s, err := NewService(ctx, cfg)
require.NoError(t, err)
require.NoError(t, s.cfg.ForkChoiceStore.ProcessBlock(ctx, 0, [32]byte{'a'}, [32]byte{'g'}, [32]byte{'c'}, 0, 0))
require.NoError(t, s.cfg.ForkChoiceStore.ProcessBlock(ctx, 1, [32]byte{'b'}, [32]byte{'a'}, [32]byte{'c'}, 0, 0))
s.setHead([32]byte{'a'}, wrapper.WrappedPhase0SignedBeaconBlock(util.NewBeaconBlock()), headState)
rr := httptest.NewRecorder()
handler := http.HandlerFunc(s.TreeHandler)
handler.ServeHTTP(rr, req)
assert.Equal(t, http.StatusOK, rr.Code)
}

View File

@@ -6,6 +6,7 @@ import (
"time"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/block"
"github.com/prysmaticlabs/prysm/runtime/version"
@@ -17,7 +18,7 @@ import (
var log = logrus.WithField("prefix", "blockchain")
// logs state transition related data every slot.
func logStateTransitionData(b block.BeaconBlock) {
func logStateTransitionData(b block.BeaconBlock) error {
log := log.WithField("slot", b.Slot())
if len(b.Body().Attestations()) > 0 {
log = log.WithField("attestations", len(b.Body().Attestations()))
@@ -34,13 +35,23 @@ func logStateTransitionData(b block.BeaconBlock) {
if len(b.Body().VoluntaryExits()) > 0 {
log = log.WithField("voluntaryExits", len(b.Body().VoluntaryExits()))
}
if b.Version() == version.Altair {
if b.Version() == version.Altair || b.Version() == version.Bellatrix {
agg, err := b.Body().SyncAggregate()
if err == nil {
log = log.WithField("syncBitsCount", agg.SyncCommitteeBits.Count())
if err != nil {
return err
}
log = log.WithField("syncBitsCount", agg.SyncCommitteeBits.Count())
}
if b.Version() == version.Bellatrix {
p, err := b.Body().ExecutionPayload()
if err != nil {
return err
}
log = log.WithField("payloadHash", fmt.Sprintf("%#x", bytesutil.Trunc(p.BlockHash)))
log = log.WithField("txCount", len(p.Transactions))
}
log.Info("Finished applying state transition")
return nil
}
func logBlockSyncStatus(block block.BeaconBlock, blockRoot [32]byte, finalized *ethpb.Checkpoint, receivedTime time.Time, genesisTime uint64) error {
@@ -55,6 +66,8 @@ func logBlockSyncStatus(block block.BeaconBlock, blockRoot [32]byte, finalized *
"epoch": slots.ToEpoch(block.Slot()),
"finalizedEpoch": finalized.Epoch,
"finalizedRoot": fmt.Sprintf("0x%s...", hex.EncodeToString(finalized.Root)[:8]),
"parentRoot": fmt.Sprintf("0x%s...", hex.EncodeToString(block.ParentRoot())[:8]),
"version": version.String(block.Version()),
}).Info("Synced new block")
log.WithFields(logrus.Fields{
"slot": block.Slot,

View File

@@ -3,6 +3,7 @@ package blockchain
import (
"testing"
enginev1 "github.com/prysmaticlabs/prysm/proto/engine/v1"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/block"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/wrapper"
@@ -11,6 +12,17 @@ import (
)
func Test_logStateTransitionData(t *testing.T) {
payloadBlk := &ethpb.BeaconBlockBellatrix{
Body: &ethpb.BeaconBlockBodyBellatrix{
SyncAggregate: &ethpb.SyncAggregate{},
ExecutionPayload: &enginev1.ExecutionPayload{
BlockHash: []byte{1, 2, 3},
Transactions: [][]byte{{}, {}},
},
},
}
wrappedPayloadBlk, err := wrapper.WrappedBeaconBlock(payloadBlk)
require.NoError(t, err)
tests := []struct {
name string
b block.BeaconBlock
@@ -55,11 +67,15 @@ func Test_logStateTransitionData(t *testing.T) {
VoluntaryExits: []*ethpb.SignedVoluntaryExit{{}}}}),
want: "\"Finished applying state transition\" attestations=1 attesterSlashings=1 deposits=1 prefix=blockchain proposerSlashings=1 slot=0 voluntaryExits=1",
},
{name: "has payload",
b: wrappedPayloadBlk,
want: "\"Finished applying state transition\" payloadHash=0x010203 prefix=blockchain slot=0 syncBitsCount=0 txCount=2",
},
}
for _, tt := range tests {
hook := logTest.NewGlobal()
t.Run(tt.name, func(t *testing.T) {
logStateTransitionData(tt.b)
require.NoError(t, logStateTransitionData(tt.b))
require.LogsContain(t, hook, tt.want)
})
}

View File

@@ -130,6 +130,14 @@ var (
Name: "sync_head_state_hit",
Help: "The number of sync head state requests that are present in the cache.",
})
stateBalanceCacheHit = promauto.NewCounter(prometheus.CounterOpts{
Name: "state_balance_cache_hit",
Help: "Count the number of state balance cache hits.",
})
stateBalanceCacheMiss = promauto.NewCounter(prometheus.CounterOpts{
Name: "state_balance_cache_miss",
Help: "Count the number of state balance cache hits.",
})
)
// reportSlotMetrics reports slot related metrics.
@@ -245,7 +253,7 @@ func reportEpochMetrics(ctx context.Context, postState, headState state.BeaconSt
if err != nil {
return err
}
case version.Altair:
case version.Altair, version.Bellatrix:
v, b, err = altair.InitializePrecomputeValidators(ctx, headState)
if err != nil {
return err

View File

@@ -0,0 +1,49 @@
package blockchain
import (
"context"
"github.com/ethereum/go-ethereum/common"
"github.com/pkg/errors"
enginev1 "github.com/prysmaticlabs/prysm/proto/engine/v1"
)
type mockEngineService struct {
newPayloadError error
forkchoiceError error
blks map[[32]byte]*enginev1.ExecutionBlock
}
func (m *mockEngineService) NewPayload(context.Context, *enginev1.ExecutionPayload) ([]byte, error) {
return nil, m.newPayloadError
}
func (m *mockEngineService) ForkchoiceUpdated(context.Context, *enginev1.ForkchoiceState, *enginev1.PayloadAttributes) (*enginev1.PayloadIDBytes, []byte, error) {
return nil, nil, m.forkchoiceError
}
func (*mockEngineService) GetPayloadV1(
_ context.Context, _ enginev1.PayloadIDBytes,
) *enginev1.ExecutionPayload {
return nil
}
func (*mockEngineService) GetPayload(context.Context, [8]byte) (*enginev1.ExecutionPayload, error) {
return nil, nil
}
func (*mockEngineService) ExchangeTransitionConfiguration(context.Context, *enginev1.TransitionConfiguration) error {
return nil
}
func (*mockEngineService) LatestExecutionBlock(context.Context) (*enginev1.ExecutionBlock, error) {
return nil, nil
}
func (m *mockEngineService) ExecutionBlockByHash(_ context.Context, hash common.Hash) (*enginev1.ExecutionBlock, error) {
blk, ok := m.blks[common.BytesToHash(hash.Bytes())]
if !ok {
return nil, errors.New("block not found")
}
return blk, nil
}

View File

@@ -0,0 +1,50 @@
package blockchain
import (
"context"
"errors"
"testing"
testDB "github.com/prysmaticlabs/prysm/beacon-chain/db/testing"
"github.com/prysmaticlabs/prysm/beacon-chain/forkchoice/protoarray"
"github.com/prysmaticlabs/prysm/beacon-chain/state"
"github.com/prysmaticlabs/prysm/beacon-chain/state/stategen"
)
func testServiceOptsWithDB(t *testing.T) []Option {
beaconDB := testDB.SetupDB(t)
fcs := protoarray.New(0, 0, [32]byte{'a'})
return []Option{
WithDatabase(beaconDB),
WithStateGen(stategen.New(beaconDB)),
WithForkChoiceStore(fcs),
}
}
// warning: only use these opts when you are certain there are no db calls
// in your code path. this is a lightweight way to satisfy the stategen/beacondb
// initialization requirements w/o the overhead of db init.
func testServiceOptsNoDB() []Option {
return []Option{
withStateBalanceCache(satisfactoryStateBalanceCache()),
}
}
type mockStateByRooter struct {
state state.BeaconState
err error
}
var _ stateByRooter = &mockStateByRooter{}
func (m mockStateByRooter) StateByRoot(_ context.Context, _ [32]byte) (state.BeaconState, error) {
return m.state, m.err
}
// returns an instance of the state balance cache that can be used
// to satisfy the requirement for one in NewService, but which will
// always return an error if used.
func satisfactoryStateBalanceCache() *stateBalanceCache {
err := errors.New("satisfactoryStateBalanceCache doesn't perform real caching")
return &stateBalanceCache{stateGen: mockStateByRooter{err: err}}
}

View File

@@ -0,0 +1,72 @@
package blockchain
import (
"bytes"
"context"
"github.com/pkg/errors"
types "github.com/prysmaticlabs/eth2-types"
"github.com/prysmaticlabs/prysm/time/slots"
)
// NewSlot mimics the implementation of `on_tick` in fork choice consensus spec.
// It resets the proposer boost root in fork choice, and it updates store's justified checkpoint
// if a better checkpoint on the store's finalized checkpoint chain.
// This should only be called at the start of every slot interval.
//
// Spec pseudocode definition:
// # Reset store.proposer_boost_root if this is a new slot
// if current_slot > previous_slot:
// store.proposer_boost_root = Root()
//
// # Not a new epoch, return
// if not (current_slot > previous_slot and compute_slots_since_epoch_start(current_slot) == 0):
// return
//
// # Update store.justified_checkpoint if a better checkpoint on the store.finalized_checkpoint chain
// if store.best_justified_checkpoint.epoch > store.justified_checkpoint.epoch:
// finalized_slot = compute_start_slot_at_epoch(store.finalized_checkpoint.epoch)
// ancestor_at_finalized_slot = get_ancestor(store, store.best_justified_checkpoint.root, finalized_slot)
// if ancestor_at_finalized_slot == store.finalized_checkpoint.root:
// store.justified_checkpoint = store.best_justified_checkpoint
func (s *Service) NewSlot(ctx context.Context, slot types.Slot) error {
// Reset proposer boost root in fork choice.
if err := s.cfg.ForkChoiceStore.ResetBoostedProposerRoot(ctx); err != nil {
return errors.Wrap(err, "could not reset boosted proposer root in fork choice")
}
// Return if it's not a new epoch.
if !slots.IsEpochStart(slot) {
return nil
}
// Update store.justified_checkpoint if a better checkpoint on the store.finalized_checkpoint chain
bj := s.store.BestJustifiedCheckpt()
if bj == nil {
return errNilBestJustifiedInStore
}
j := s.store.JustifiedCheckpt()
if j == nil {
return errNilJustifiedInStore
}
f := s.store.FinalizedCheckpt()
if f == nil {
return errNilFinalizedInStore
}
if bj.Epoch > j.Epoch {
finalizedSlot, err := slots.EpochStart(f.Epoch)
if err != nil {
return err
}
r, err := s.ancestor(ctx, bj.Root, finalizedSlot)
if err != nil {
return err
}
if bytes.Equal(r, f.Root) {
s.store.SetJustifiedCheckpt(bj)
}
}
return nil
}

View File

@@ -0,0 +1,100 @@
package blockchain
import (
"context"
"testing"
types "github.com/prysmaticlabs/eth2-types"
"github.com/prysmaticlabs/prysm/beacon-chain/blockchain/store"
testDB "github.com/prysmaticlabs/prysm/beacon-chain/db/testing"
"github.com/prysmaticlabs/prysm/beacon-chain/forkchoice/protoarray"
"github.com/prysmaticlabs/prysm/beacon-chain/state/stategen"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/testing/require"
)
func TestService_newSlot(t *testing.T) {
beaconDB := testDB.SetupDB(t)
fcs := protoarray.New(0, 0, [32]byte{})
opts := []Option{
WithDatabase(beaconDB),
WithStateGen(stategen.New(beaconDB)),
WithForkChoiceStore(fcs),
}
ctx := context.Background()
require.NoError(t, fcs.InsertOptimisticBlock(ctx, 0, [32]byte{}, [32]byte{}, [32]byte{}, 0, 0)) // genesis
require.NoError(t, fcs.InsertOptimisticBlock(ctx, 32, [32]byte{'a'}, [32]byte{}, [32]byte{}, 0, 0)) // finalized
require.NoError(t, fcs.InsertOptimisticBlock(ctx, 64, [32]byte{'b'}, [32]byte{'a'}, [32]byte{}, 0, 0)) // justified
require.NoError(t, fcs.InsertOptimisticBlock(ctx, 96, [32]byte{'c'}, [32]byte{'a'}, [32]byte{}, 0, 0)) // best justified
require.NoError(t, fcs.InsertOptimisticBlock(ctx, 97, [32]byte{'d'}, [32]byte{}, [32]byte{}, 0, 0)) // bad
type args struct {
slot types.Slot
finalized *ethpb.Checkpoint
justified *ethpb.Checkpoint
bestJustified *ethpb.Checkpoint
shouldEqual bool
}
tests := []struct {
name string
args args
}{
{
name: "Not epoch boundary. No change",
args: args{
slot: params.BeaconConfig().SlotsPerEpoch + 1,
finalized: &ethpb.Checkpoint{Epoch: 1, Root: bytesutil.PadTo([]byte{'a'}, 32)},
justified: &ethpb.Checkpoint{Epoch: 2, Root: bytesutil.PadTo([]byte{'b'}, 32)},
bestJustified: &ethpb.Checkpoint{Epoch: 3, Root: bytesutil.PadTo([]byte{'c'}, 32)},
shouldEqual: false,
},
},
{
name: "Justified higher than best justified. No change",
args: args{
slot: params.BeaconConfig().SlotsPerEpoch,
finalized: &ethpb.Checkpoint{Epoch: 1, Root: bytesutil.PadTo([]byte{'a'}, 32)},
justified: &ethpb.Checkpoint{Epoch: 3, Root: bytesutil.PadTo([]byte{'b'}, 32)},
bestJustified: &ethpb.Checkpoint{Epoch: 2, Root: bytesutil.PadTo([]byte{'c'}, 32)},
shouldEqual: false,
},
},
{
name: "Best justified not on the same chain as finalized. No change",
args: args{
slot: params.BeaconConfig().SlotsPerEpoch,
finalized: &ethpb.Checkpoint{Epoch: 1, Root: bytesutil.PadTo([]byte{'a'}, 32)},
justified: &ethpb.Checkpoint{Epoch: 2, Root: bytesutil.PadTo([]byte{'b'}, 32)},
bestJustified: &ethpb.Checkpoint{Epoch: 3, Root: bytesutil.PadTo([]byte{'d'}, 32)},
shouldEqual: false,
},
},
{
name: "Best justified on the same chain as finalized. Yes change",
args: args{
slot: params.BeaconConfig().SlotsPerEpoch,
finalized: &ethpb.Checkpoint{Epoch: 1, Root: bytesutil.PadTo([]byte{'a'}, 32)},
justified: &ethpb.Checkpoint{Epoch: 2, Root: bytesutil.PadTo([]byte{'b'}, 32)},
bestJustified: &ethpb.Checkpoint{Epoch: 3, Root: bytesutil.PadTo([]byte{'c'}, 32)},
shouldEqual: true,
},
},
}
for _, test := range tests {
service, err := NewService(ctx, opts...)
require.NoError(t, err)
store := store.New(test.args.justified, test.args.finalized)
store.SetBestJustifiedCheckpt(test.args.bestJustified)
service.store = store
require.NoError(t, service.NewSlot(ctx, test.args.slot))
if test.args.shouldEqual {
require.DeepSSZEqual(t, service.store.BestJustifiedCheckpt(), service.store.JustifiedCheckpt())
} else {
require.DeepNotSSZEqual(t, service.store.BestJustifiedCheckpt(), service.store.JustifiedCheckpt())
}
}
}

View File

@@ -0,0 +1,189 @@
package blockchain
import (
"context"
"fmt"
"github.com/pkg/errors"
"github.com/prysmaticlabs/prysm/beacon-chain/core/blocks"
"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
v1 "github.com/prysmaticlabs/prysm/beacon-chain/powchain/engine-api-client/v1"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
enginev1 "github.com/prysmaticlabs/prysm/proto/engine/v1"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/block"
"github.com/sirupsen/logrus"
"go.opencensus.io/trace"
)
// notifyForkchoiceUpdate signals execution engine the fork choice updates. Execution engine should:
// 1. Re-organizes the execution payload chain and corresponding state to make head_block_hash the head.
// 2. Applies finality to the execution state: it irreversibly persists the chain of all execution payloads and corresponding state, up to and including finalized_block_hash.
func (s *Service) notifyForkchoiceUpdate(ctx context.Context, headBlk block.BeaconBlock, headRoot [32]byte, finalizedRoot [32]byte) (*enginev1.PayloadIDBytes, error) {
ctx, span := trace.StartSpan(ctx, "blockChain.notifyForkchoiceUpdate")
defer span.End()
if headBlk == nil || headBlk.IsNil() || headBlk.Body().IsNil() {
return nil, errors.New("nil head block")
}
// Must not call fork choice updated until the transition conditions are met on the Pow network.
isExecutionBlk, err := blocks.IsExecutionBlock(headBlk.Body())
if err != nil {
return nil, errors.Wrap(err, "could not determine if block is execution block")
}
if !isExecutionBlk {
return nil, nil
}
headPayload, err := headBlk.Body().ExecutionPayload()
if err != nil {
return nil, errors.Wrap(err, "could not get execution payload")
}
finalizedBlock, err := s.cfg.BeaconDB.Block(ctx, s.ensureRootNotZeros(finalizedRoot))
if err != nil {
return nil, errors.Wrap(err, "could not get finalized block")
}
var finalizedHash []byte
if blocks.IsPreBellatrixVersion(finalizedBlock.Block().Version()) {
finalizedHash = params.BeaconConfig().ZeroHash[:]
} else {
payload, err := finalizedBlock.Block().Body().ExecutionPayload()
if err != nil {
return nil, errors.Wrap(err, "could not get finalized block execution payload")
}
finalizedHash = payload.BlockHash
}
fcs := &enginev1.ForkchoiceState{
HeadBlockHash: headPayload.BlockHash,
SafeBlockHash: headPayload.BlockHash,
FinalizedBlockHash: finalizedHash,
}
// payload attribute is only required when requesting payload, here we are just updating fork choice, so it is nil.
payloadID, _, err := s.cfg.ExecutionEngineCaller.ForkchoiceUpdated(ctx, fcs, nil /*payload attribute*/)
if err != nil {
switch err {
case v1.ErrAcceptedSyncingPayloadStatus:
log.WithFields(logrus.Fields{
"headSlot": headBlk.Slot(),
"headHash": fmt.Sprintf("%#x", bytesutil.Trunc(headPayload.BlockHash)),
"finalizedHash": fmt.Sprintf("%#x", bytesutil.Trunc(finalizedHash)),
}).Info("Called fork choice updated with optimistic block")
return payloadID, nil
default:
return nil, errors.Wrap(err, "could not notify forkchoice update from execution engine")
}
}
if err := s.cfg.ForkChoiceStore.SetOptimisticToValid(ctx, headRoot); err != nil {
return nil, errors.Wrap(err, "could not set block to valid")
}
return payloadID, nil
}
// notifyForkchoiceUpdate signals execution engine on a new payload
func (s *Service) notifyNewPayload(ctx context.Context, preStateVersion, postStateVersion int,
preStateHeader, postStateHeader *ethpb.ExecutionPayloadHeader, blk block.SignedBeaconBlock, root [32]byte) error {
ctx, span := trace.StartSpan(ctx, "blockChain.notifyNewPayload")
defer span.End()
// Execution payload is only supported in Bellatrix and beyond. Pre
// merge blocks are never optimistic
if blocks.IsPreBellatrixVersion(postStateVersion) {
return s.cfg.ForkChoiceStore.SetOptimisticToValid(ctx, root)
}
if err := helpers.BeaconBlockIsNil(blk); err != nil {
return err
}
body := blk.Block().Body()
enabled, err := blocks.IsExecutionEnabledUsingHeader(postStateHeader, body)
if err != nil {
return errors.Wrap(err, "could not determine if execution is enabled")
}
if !enabled {
return s.cfg.ForkChoiceStore.SetOptimisticToValid(ctx, root)
}
payload, err := body.ExecutionPayload()
if err != nil {
return errors.Wrap(err, "could not get execution payload")
}
_, err = s.cfg.ExecutionEngineCaller.NewPayload(ctx, payload)
if err != nil {
switch err {
case v1.ErrAcceptedSyncingPayloadStatus:
log.WithFields(logrus.Fields{
"slot": blk.Block().Slot(),
"blockHash": fmt.Sprintf("%#x", bytesutil.Trunc(payload.BlockHash)),
}).Info("Called new payload with optimistic block")
return nil
default:
return errors.Wrap(err, "could not validate execution payload from execution engine")
}
}
if err := s.cfg.ForkChoiceStore.SetOptimisticToValid(ctx, root); err != nil {
return errors.Wrap(err, "could not set optimistic status")
}
// During the transition event, the transition block should be verified for sanity.
if blocks.IsPreBellatrixVersion(preStateVersion) {
// Handle case where pre-state is Altair but block contains payload.
// To reach here, the block must have contained a valid payload.
return s.validateMergeBlock(ctx, blk)
}
atTransition, err := blocks.IsMergeTransitionBlockUsingPreStatePayloadHeader(preStateHeader, body)
if err != nil {
return errors.Wrap(err, "could not check if merge block is terminal")
}
if !atTransition {
return nil
}
return s.validateMergeBlock(ctx, blk)
}
// optimisticCandidateBlock returns true if this block can be optimistically synced.
//
// Spec pseudocode definition:
// def is_optimistic_candidate_block(opt_store: OptimisticStore, current_slot: Slot, block: BeaconBlock) -> bool:
// if is_execution_block(opt_store.blocks[block.parent_root]):
// return True
//
// justified_root = opt_store.block_states[opt_store.head_block_root].current_justified_checkpoint.root
// if is_execution_block(opt_store.blocks[justified_root]):
// return True
//
// if block.slot + SAFE_SLOTS_TO_IMPORT_OPTIMISTICALLY <= current_slot:
// return True
//
// return False
func (s *Service) optimisticCandidateBlock(ctx context.Context, blk block.BeaconBlock) (bool, error) {
if blk.Slot()+params.BeaconConfig().SafeSlotsToImportOptimistically <= s.CurrentSlot() {
return true, nil
}
parent, err := s.cfg.BeaconDB.Block(ctx, bytesutil.ToBytes32(blk.ParentRoot()))
if err != nil {
return false, err
}
if parent == nil {
return false, errNilParentInDB
}
parentIsExecutionBlock, err := blocks.IsExecutionBlock(parent.Block().Body())
if err != nil {
return false, err
}
if parentIsExecutionBlock {
return true, nil
}
j := s.store.JustifiedCheckpt()
if j == nil {
return false, errNilJustifiedInStore
}
jBlock, err := s.cfg.BeaconDB.Block(ctx, bytesutil.ToBytes32(j.Root))
if err != nil {
return false, err
}
return blocks.IsExecutionBlock(jBlock.Block().Body())
}

View File

@@ -0,0 +1,692 @@
package blockchain
import (
"context"
"testing"
"time"
"github.com/prysmaticlabs/prysm/beacon-chain/core/blocks"
testDB "github.com/prysmaticlabs/prysm/beacon-chain/db/testing"
"github.com/prysmaticlabs/prysm/beacon-chain/forkchoice/protoarray"
engine "github.com/prysmaticlabs/prysm/beacon-chain/powchain/engine-api-client/v1"
"github.com/prysmaticlabs/prysm/beacon-chain/state"
"github.com/prysmaticlabs/prysm/beacon-chain/state/stategen"
fieldparams "github.com/prysmaticlabs/prysm/config/fieldparams"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
v1 "github.com/prysmaticlabs/prysm/proto/engine/v1"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/block"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/wrapper"
"github.com/prysmaticlabs/prysm/runtime/version"
"github.com/prysmaticlabs/prysm/testing/assert"
"github.com/prysmaticlabs/prysm/testing/require"
"github.com/prysmaticlabs/prysm/testing/util"
"github.com/prysmaticlabs/prysm/time/slots"
)
func Test_NotifyForkchoiceUpdate(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
altairBlk, err := wrapper.WrappedSignedBeaconBlock(util.NewBeaconBlockAltair())
require.NoError(t, err)
altairBlkRoot, err := altairBlk.Block().HashTreeRoot()
require.NoError(t, err)
bellatrixBlk, err := wrapper.WrappedSignedBeaconBlock(util.NewBeaconBlockBellatrix())
require.NoError(t, err)
bellatrixBlkRoot, err := bellatrixBlk.Block().HashTreeRoot()
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, altairBlk))
require.NoError(t, beaconDB.SaveBlock(ctx, bellatrixBlk))
fcs := protoarray.New(0, 0, [32]byte{'a'})
opts := []Option{
WithDatabase(beaconDB),
WithStateGen(stategen.New(beaconDB)),
WithForkChoiceStore(fcs),
}
service, err := NewService(ctx, opts...)
require.NoError(t, err)
require.NoError(t, fcs.InsertOptimisticBlock(ctx, 0, [32]byte{}, [32]byte{}, params.BeaconConfig().ZeroHash, 0, 0))
tests := []struct {
name string
blk block.BeaconBlock
finalizedRoot [32]byte
newForkchoiceErr error
errString string
}{
{
name: "nil block",
errString: "nil head block",
},
{
name: "phase0 block",
blk: func() block.BeaconBlock {
b, err := wrapper.WrappedBeaconBlock(&ethpb.BeaconBlock{Body: &ethpb.BeaconBlockBody{}})
require.NoError(t, err)
return b
}(),
},
{
name: "altair block",
blk: func() block.BeaconBlock {
b, err := wrapper.WrappedBeaconBlock(&ethpb.BeaconBlockAltair{Body: &ethpb.BeaconBlockBodyAltair{}})
require.NoError(t, err)
return b
}(),
},
{
name: "not execution block",
blk: func() block.BeaconBlock {
b, err := wrapper.WrappedBeaconBlock(&ethpb.BeaconBlockBellatrix{
Body: &ethpb.BeaconBlockBodyBellatrix{
ExecutionPayload: &v1.ExecutionPayload{
ParentHash: make([]byte, fieldparams.RootLength),
FeeRecipient: make([]byte, fieldparams.FeeRecipientLength),
StateRoot: make([]byte, fieldparams.RootLength),
ReceiptsRoot: make([]byte, fieldparams.RootLength),
LogsBloom: make([]byte, fieldparams.LogsBloomLength),
PrevRandao: make([]byte, fieldparams.RootLength),
BaseFeePerGas: make([]byte, fieldparams.RootLength),
BlockHash: make([]byte, fieldparams.RootLength),
},
},
})
require.NoError(t, err)
return b
}(),
},
{
name: "happy case: finalized root is altair block",
blk: func() block.BeaconBlock {
b, err := wrapper.WrappedBeaconBlock(&ethpb.BeaconBlockBellatrix{
Body: &ethpb.BeaconBlockBodyBellatrix{
ExecutionPayload: &v1.ExecutionPayload{},
},
})
require.NoError(t, err)
return b
}(),
finalizedRoot: altairBlkRoot,
},
{
name: "happy case: finalized root is bellatrix block",
blk: func() block.BeaconBlock {
b, err := wrapper.WrappedBeaconBlock(&ethpb.BeaconBlockBellatrix{
Body: &ethpb.BeaconBlockBodyBellatrix{
ExecutionPayload: &v1.ExecutionPayload{},
},
})
require.NoError(t, err)
return b
}(),
finalizedRoot: bellatrixBlkRoot,
},
{
name: "forkchoice updated with optimistic block",
blk: func() block.BeaconBlock {
b, err := wrapper.WrappedBeaconBlock(&ethpb.BeaconBlockBellatrix{
Body: &ethpb.BeaconBlockBodyBellatrix{
ExecutionPayload: &v1.ExecutionPayload{},
},
})
require.NoError(t, err)
return b
}(),
newForkchoiceErr: engine.ErrAcceptedSyncingPayloadStatus,
finalizedRoot: bellatrixBlkRoot,
},
{
name: "forkchoice updated with invalid block",
blk: func() block.BeaconBlock {
b, err := wrapper.WrappedBeaconBlock(&ethpb.BeaconBlockBellatrix{
Body: &ethpb.BeaconBlockBodyBellatrix{
ExecutionPayload: &v1.ExecutionPayload{},
},
})
require.NoError(t, err)
return b
}(),
newForkchoiceErr: engine.ErrInvalidPayloadStatus,
finalizedRoot: bellatrixBlkRoot,
errString: "could not notify forkchoice update from execution engine: payload status is INVALID",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
engine := &mockEngineService{forkchoiceError: tt.newForkchoiceErr}
service.cfg.ExecutionEngineCaller = engine
_, err := service.notifyForkchoiceUpdate(ctx, tt.blk, service.headRoot(), tt.finalizedRoot)
if tt.errString != "" {
require.ErrorContains(t, tt.errString, err)
} else {
require.NoError(t, err)
}
})
}
}
func Test_NotifyNewPayload(t *testing.T) {
cfg := params.BeaconConfig()
cfg.TerminalTotalDifficulty = "2"
params.OverrideBeaconConfig(cfg)
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
fcs := protoarray.New(0, 0, [32]byte{'a'})
opts := []Option{
WithDatabase(beaconDB),
WithStateGen(stategen.New(beaconDB)),
WithForkChoiceStore(fcs),
}
phase0State, _ := util.DeterministicGenesisState(t, 1)
altairState, _ := util.DeterministicGenesisStateAltair(t, 1)
bellatrixState, _ := util.DeterministicGenesisStateBellatrix(t, 2)
blk := &ethpb.SignedBeaconBlockBellatrix{
Block: &ethpb.BeaconBlockBellatrix{
Body: &ethpb.BeaconBlockBodyBellatrix{
ExecutionPayload: &v1.ExecutionPayload{},
},
},
}
a := &ethpb.SignedBeaconBlockAltair{
Block: &ethpb.BeaconBlockAltair{
Body: &ethpb.BeaconBlockBodyAltair{},
},
}
altairBlk, err := wrapper.WrappedSignedBeaconBlock(a)
require.NoError(t, err)
bellatrixBlk, err := wrapper.WrappedSignedBeaconBlock(blk)
require.NoError(t, err)
service, err := NewService(ctx, opts...)
require.NoError(t, err)
tests := []struct {
name string
preState state.BeaconState
postState state.BeaconState
blk block.SignedBeaconBlock
newPayloadErr error
errString string
}{
{
name: "phase 0 post state",
postState: phase0State,
preState: phase0State,
},
{
name: "altair post state",
postState: altairState,
preState: altairState,
},
{
name: "nil beacon block",
postState: bellatrixState,
preState: bellatrixState,
errString: "signed beacon block can't be nil",
},
{
name: "new payload with optimistic block",
postState: bellatrixState,
preState: bellatrixState,
blk: bellatrixBlk,
newPayloadErr: engine.ErrAcceptedSyncingPayloadStatus,
},
{
name: "new payload with invalid block",
postState: bellatrixState,
preState: bellatrixState,
blk: bellatrixBlk,
newPayloadErr: engine.ErrInvalidPayloadStatus,
errString: "could not validate execution payload from execution engine: payload status is INVALID",
},
{
name: "altair pre state, altair block",
postState: bellatrixState,
preState: altairState,
blk: altairBlk,
},
{
name: "altair pre state, happy case",
postState: bellatrixState,
preState: altairState,
blk: func() block.SignedBeaconBlock {
blk := &ethpb.SignedBeaconBlockBellatrix{
Block: &ethpb.BeaconBlockBellatrix{
Body: &ethpb.BeaconBlockBodyBellatrix{
ExecutionPayload: &v1.ExecutionPayload{
ParentHash: bytesutil.PadTo([]byte{'a'}, fieldparams.RootLength),
},
},
},
}
b, err := wrapper.WrappedSignedBeaconBlock(blk)
require.NoError(t, err)
return b
}(),
},
{
name: "could not get merge block",
postState: bellatrixState,
preState: bellatrixState,
blk: bellatrixBlk,
errString: "could not get merge block parent hash and total difficulty",
},
{
name: "not at merge transition",
postState: bellatrixState,
preState: bellatrixState,
blk: func() block.SignedBeaconBlock {
blk := &ethpb.SignedBeaconBlockBellatrix{
Block: &ethpb.BeaconBlockBellatrix{
Body: &ethpb.BeaconBlockBodyBellatrix{
ExecutionPayload: &v1.ExecutionPayload{
ParentHash: make([]byte, fieldparams.RootLength),
FeeRecipient: make([]byte, fieldparams.FeeRecipientLength),
StateRoot: make([]byte, fieldparams.RootLength),
ReceiptsRoot: make([]byte, fieldparams.RootLength),
LogsBloom: make([]byte, fieldparams.LogsBloomLength),
PrevRandao: make([]byte, fieldparams.RootLength),
BaseFeePerGas: make([]byte, fieldparams.RootLength),
BlockHash: make([]byte, fieldparams.RootLength),
},
},
},
}
b, err := wrapper.WrappedSignedBeaconBlock(blk)
require.NoError(t, err)
return b
}(),
},
{
name: "could not get merge block",
postState: bellatrixState,
preState: bellatrixState,
blk: bellatrixBlk,
errString: "could not get merge block parent hash and total difficulty",
},
{
name: "happy case",
postState: bellatrixState,
preState: bellatrixState,
blk: func() block.SignedBeaconBlock {
blk := &ethpb.SignedBeaconBlockBellatrix{
Block: &ethpb.BeaconBlockBellatrix{
Body: &ethpb.BeaconBlockBodyBellatrix{
ExecutionPayload: &v1.ExecutionPayload{
ParentHash: bytesutil.PadTo([]byte{'a'}, fieldparams.RootLength),
},
},
},
}
b, err := wrapper.WrappedSignedBeaconBlock(blk)
require.NoError(t, err)
return b
}(),
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
engine := &mockEngineService{newPayloadError: tt.newPayloadErr, blks: map[[32]byte]*v1.ExecutionBlock{}}
engine.blks[[32]byte{'a'}] = &v1.ExecutionBlock{
ParentHash: bytesutil.PadTo([]byte{'b'}, fieldparams.RootLength),
TotalDifficulty: "0x2",
}
engine.blks[[32]byte{'b'}] = &v1.ExecutionBlock{
ParentHash: bytesutil.PadTo([]byte{'3'}, fieldparams.RootLength),
TotalDifficulty: "0x1",
}
service.cfg.ExecutionEngineCaller = engine
var payload *ethpb.ExecutionPayloadHeader
if tt.preState.Version() == version.Bellatrix {
payload, err = tt.preState.LatestExecutionPayloadHeader()
require.NoError(t, err)
}
root := [32]byte{'a'}
require.NoError(t, service.cfg.ForkChoiceStore.InsertOptimisticBlock(ctx, 0, root, root, params.BeaconConfig().ZeroHash, 0, 0))
postVersion, postHeader, err := getStateVersionAndPayload(tt.postState)
require.NoError(t, err)
err = service.notifyNewPayload(ctx, tt.preState.Version(), postVersion, payload, postHeader, tt.blk, root)
if tt.errString != "" {
require.ErrorContains(t, tt.errString, err)
} else {
require.NoError(t, err)
}
})
}
}
func Test_NotifyNewPayload_SetOptimisticToValid(t *testing.T) {
cfg := params.BeaconConfig()
cfg.TerminalTotalDifficulty = "2"
params.OverrideBeaconConfig(cfg)
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
fcs := protoarray.New(0, 0, [32]byte{'a'})
opts := []Option{
WithDatabase(beaconDB),
WithStateGen(stategen.New(beaconDB)),
WithForkChoiceStore(fcs),
}
bellatrixState, _ := util.DeterministicGenesisStateBellatrix(t, 2)
blk := &ethpb.SignedBeaconBlockBellatrix{
Block: &ethpb.BeaconBlockBellatrix{
Body: &ethpb.BeaconBlockBodyBellatrix{
ExecutionPayload: &v1.ExecutionPayload{
ParentHash: bytesutil.PadTo([]byte{'a'}, fieldparams.RootLength),
},
},
},
}
bellatrixBlk, err := wrapper.WrappedSignedBeaconBlock(blk)
require.NoError(t, err)
service, err := NewService(ctx, opts...)
require.NoError(t, err)
engine := &mockEngineService{blks: map[[32]byte]*v1.ExecutionBlock{}}
engine.blks[[32]byte{'a'}] = &v1.ExecutionBlock{
ParentHash: bytesutil.PadTo([]byte{'b'}, fieldparams.RootLength),
TotalDifficulty: "0x2",
}
engine.blks[[32]byte{'b'}] = &v1.ExecutionBlock{
ParentHash: bytesutil.PadTo([]byte{'3'}, fieldparams.RootLength),
TotalDifficulty: "0x1",
}
service.cfg.ExecutionEngineCaller = engine
payload, err := bellatrixState.LatestExecutionPayloadHeader()
require.NoError(t, err)
root := [32]byte{'c'}
require.NoError(t, service.cfg.ForkChoiceStore.InsertOptimisticBlock(ctx, 1, root, [32]byte{'a'}, params.BeaconConfig().ZeroHash, 0, 0))
postVersion, postHeader, err := getStateVersionAndPayload(bellatrixState)
require.NoError(t, err)
err = service.notifyNewPayload(ctx, bellatrixState.Version(), postVersion, payload, postHeader, bellatrixBlk, root)
require.NoError(t, err)
optimistic, err := service.IsOptimisticForRoot(ctx, root)
require.NoError(t, err)
require.Equal(t, false, optimistic)
}
func Test_IsOptimisticCandidateBlock(t *testing.T) {
params.SetupTestConfigCleanup(t)
params.OverrideBeaconConfig(params.MainnetConfig())
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
fcs := protoarray.New(0, 0, [32]byte{'a'})
opts := []Option{
WithDatabase(beaconDB),
WithStateGen(stategen.New(beaconDB)),
WithForkChoiceStore(fcs),
}
service, err := NewService(ctx, opts...)
require.NoError(t, err)
params.BeaconConfig().SafeSlotsToImportOptimistically = 128
service.genesisTime = time.Now().Add(-time.Second * 12 * 2 * 128)
parentBlk := util.NewBeaconBlockBellatrix()
wrappedParentBlock, err := wrapper.WrappedSignedBeaconBlock(parentBlk)
require.NoError(t, err)
parentRoot, err := wrappedParentBlock.Block().HashTreeRoot()
require.NoError(t, err)
tests := []struct {
name string
blk block.BeaconBlock
justified block.SignedBeaconBlock
want bool
}{
{
name: "deep block",
blk: func(tt *testing.T) block.BeaconBlock {
blk := util.NewBeaconBlockBellatrix()
blk.Block.Slot = 1
blk.Block.ParentRoot = parentRoot[:]
wr, err := wrapper.WrappedBellatrixBeaconBlock(blk.Block)
require.NoError(tt, err)
return wr
}(t),
justified: func(tt *testing.T) block.SignedBeaconBlock {
blk := util.NewBeaconBlockBellatrix()
blk.Block.Slot = 32
blk.Block.ParentRoot = parentRoot[:]
wr, err := wrapper.WrappedSignedBeaconBlock(blk)
require.NoError(tt, err)
return wr
}(t),
want: true,
},
{
name: "shallow block, Altair justified chkpt",
blk: func(tt *testing.T) block.BeaconBlock {
blk := util.NewBeaconBlockAltair()
blk.Block.Slot = 200
blk.Block.ParentRoot = parentRoot[:]
wr, err := wrapper.WrappedAltairBeaconBlock(blk.Block)
require.NoError(tt, err)
return wr
}(t),
justified: func(tt *testing.T) block.SignedBeaconBlock {
blk := util.NewBeaconBlockAltair()
blk.Block.Slot = 32
blk.Block.ParentRoot = parentRoot[:]
wr, err := wrapper.WrappedSignedBeaconBlock(blk)
require.NoError(tt, err)
return wr
}(t),
want: false,
},
{
name: "shallow block, Bellatrix justified chkpt without execution",
blk: func(tt *testing.T) block.BeaconBlock {
blk := util.NewBeaconBlockBellatrix()
blk.Block.Slot = 200
blk.Block.ParentRoot = parentRoot[:]
wr, err := wrapper.WrappedBellatrixBeaconBlock(blk.Block)
require.NoError(tt, err)
return wr
}(t),
justified: func(tt *testing.T) block.SignedBeaconBlock {
blk := util.NewBeaconBlockBellatrix()
blk.Block.Slot = 32
blk.Block.ParentRoot = parentRoot[:]
wr, err := wrapper.WrappedSignedBeaconBlock(blk)
require.NoError(tt, err)
return wr
}(t),
want: false,
},
{
name: "shallow block, execution enabled justified chkpt",
blk: func(tt *testing.T) block.BeaconBlock {
blk := util.NewBeaconBlockBellatrix()
blk.Block.Slot = 200
blk.Block.ParentRoot = parentRoot[:]
wr, err := wrapper.WrappedBellatrixBeaconBlock(blk.Block)
require.NoError(tt, err)
return wr
}(t),
justified: func(tt *testing.T) block.SignedBeaconBlock {
blk := util.NewBeaconBlockBellatrix()
blk.Block.Slot = 32
blk.Block.ParentRoot = parentRoot[:]
blk.Block.Body.ExecutionPayload.ParentHash = bytesutil.PadTo([]byte{'a'}, fieldparams.RootLength)
blk.Block.Body.ExecutionPayload.FeeRecipient = bytesutil.PadTo([]byte{'a'}, fieldparams.FeeRecipientLength)
blk.Block.Body.ExecutionPayload.StateRoot = bytesutil.PadTo([]byte{'a'}, fieldparams.RootLength)
blk.Block.Body.ExecutionPayload.ReceiptsRoot = bytesutil.PadTo([]byte{'a'}, fieldparams.RootLength)
blk.Block.Body.ExecutionPayload.LogsBloom = bytesutil.PadTo([]byte{'a'}, fieldparams.LogsBloomLength)
blk.Block.Body.ExecutionPayload.PrevRandao = bytesutil.PadTo([]byte{'a'}, fieldparams.RootLength)
blk.Block.Body.ExecutionPayload.BaseFeePerGas = bytesutil.PadTo([]byte{'a'}, fieldparams.RootLength)
blk.Block.Body.ExecutionPayload.BlockHash = bytesutil.PadTo([]byte{'a'}, fieldparams.RootLength)
wr, err := wrapper.WrappedSignedBeaconBlock(blk)
require.NoError(tt, err)
return wr
}(t),
want: true,
},
}
for _, tt := range tests {
jroot, err := tt.justified.Block().HashTreeRoot()
require.NoError(t, err)
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, tt.justified))
service.store.SetJustifiedCheckpt(
&ethpb.Checkpoint{
Root: jroot[:],
Epoch: slots.ToEpoch(tt.justified.Block().Slot()),
})
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wrappedParentBlock))
candidate, err := service.optimisticCandidateBlock(ctx, tt.blk)
require.NoError(t, err)
require.Equal(t, tt.want, candidate, tt.name)
}
}
func Test_IsOptimisticShallowExecutionParent(t *testing.T) {
params.SetupTestConfigCleanup(t)
params.OverrideBeaconConfig(params.MainnetConfig())
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
opts := []Option{
WithDatabase(beaconDB),
WithStateGen(stategen.New(beaconDB)),
}
service, err := NewService(ctx, opts...)
require.NoError(t, err)
params.BeaconConfig().SafeSlotsToImportOptimistically = 128
service.genesisTime = time.Now().Add(-time.Second * 12 * 2 * 128)
payload := &v1.ExecutionPayload{
ParentHash: make([]byte, 32),
FeeRecipient: make([]byte, 20),
StateRoot: make([]byte, 32),
ReceiptsRoot: make([]byte, 32),
LogsBloom: make([]byte, 256),
PrevRandao: make([]byte, 32),
BaseFeePerGas: bytesutil.PadTo([]byte{1, 2, 3, 4}, fieldparams.RootLength),
BlockHash: make([]byte, 32),
BlockNumber: 100,
}
body := &ethpb.BeaconBlockBodyBellatrix{ExecutionPayload: payload}
block := &ethpb.BeaconBlockBellatrix{Body: body, Slot: 200}
rawSigned := &ethpb.SignedBeaconBlockBellatrix{Block: block}
blk := util.HydrateSignedBeaconBlockBellatrix(rawSigned)
wr, err := wrapper.WrappedSignedBeaconBlock(blk)
require.NoError(t, err)
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wr))
blkRoot, err := wr.Block().HashTreeRoot()
require.NoError(t, err)
childBlock := util.NewBeaconBlockBellatrix()
childBlock.Block.ParentRoot = blkRoot[:]
// shallow block
childBlock.Block.Slot = 201
wrappedChild, err := wrapper.WrappedSignedBeaconBlock(childBlock)
require.NoError(t, err)
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wrappedChild))
candidate, err := service.optimisticCandidateBlock(ctx, wrappedChild.Block())
require.NoError(t, err)
require.Equal(t, true, candidate)
}
func Test_UpdateLastValidatedCheckpoint(t *testing.T) {
params.SetupTestConfigCleanup(t)
params.OverrideBeaconConfig(params.MainnetConfig())
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
stateGen := stategen.New(beaconDB)
fcs := protoarray.New(0, 0, [32]byte{})
opts := []Option{
WithDatabase(beaconDB),
WithStateGen(stateGen),
WithForkChoiceStore(fcs),
}
service, err := NewService(ctx, opts...)
require.NoError(t, err)
genesisStateRoot := [32]byte{}
genesisBlk := blocks.NewGenesisBlock(genesisStateRoot[:])
wr, err := wrapper.WrappedSignedBeaconBlock(genesisBlk)
require.NoError(t, err)
assert.NoError(t, beaconDB.SaveBlock(ctx, wr))
genesisRoot, err := genesisBlk.Block.HashTreeRoot()
require.NoError(t, err)
assert.NoError(t, beaconDB.SaveGenesisBlockRoot(ctx, genesisRoot))
require.NoError(t, fcs.InsertOptimisticBlock(ctx, 0, genesisRoot, params.BeaconConfig().ZeroHash,
params.BeaconConfig().ZeroHash, 0, 0))
genesisSummary := &ethpb.StateSummary{
Root: genesisStateRoot[:],
Slot: 0,
}
require.NoError(t, beaconDB.SaveStateSummary(ctx, genesisSummary))
// Get last validated checkpoint
origCheckpoint, err := service.cfg.BeaconDB.LastValidatedCheckpoint(ctx)
require.NoError(t, err)
require.NoError(t, beaconDB.SaveLastValidatedCheckpoint(ctx, origCheckpoint))
// Optimistic finalized checkpoint
blk := util.NewBeaconBlock()
blk.Block.Slot = 320
blk.Block.ParentRoot = genesisRoot[:]
wr, err = wrapper.WrappedSignedBeaconBlock(blk)
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, wr))
opRoot, err := blk.Block.HashTreeRoot()
require.NoError(t, err)
opCheckpoint := &ethpb.Checkpoint{
Root: opRoot[:],
Epoch: 10,
}
opStateSummary := &ethpb.StateSummary{
Root: opRoot[:],
Slot: 320,
}
require.NoError(t, beaconDB.SaveStateSummary(ctx, opStateSummary))
require.NoError(t, fcs.InsertOptimisticBlock(ctx, 320, opRoot, genesisRoot,
params.BeaconConfig().ZeroHash, 10, 10))
assert.NoError(t, beaconDB.SaveGenesisBlockRoot(ctx, opRoot))
require.NoError(t, service.updateFinalized(ctx, opCheckpoint))
cp, err := service.cfg.BeaconDB.LastValidatedCheckpoint(ctx)
require.NoError(t, err)
require.DeepEqual(t, origCheckpoint.Root, cp.Root)
require.Equal(t, origCheckpoint.Epoch, cp.Epoch)
// Validated finalized checkpoint
blk = util.NewBeaconBlock()
blk.Block.Slot = 640
blk.Block.ParentRoot = opRoot[:]
wr, err = wrapper.WrappedSignedBeaconBlock(blk)
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, wr))
validRoot, err := blk.Block.HashTreeRoot()
require.NoError(t, err)
validCheckpoint := &ethpb.Checkpoint{
Root: validRoot[:],
Epoch: 20,
}
validSummary := &ethpb.StateSummary{
Root: validRoot[:],
Slot: 640,
}
require.NoError(t, beaconDB.SaveStateSummary(ctx, validSummary))
require.NoError(t, fcs.InsertOptimisticBlock(ctx, 640, validRoot, params.BeaconConfig().ZeroHash,
params.BeaconConfig().ZeroHash, 20, 20))
require.NoError(t, fcs.SetOptimisticToValid(ctx, validRoot))
assert.NoError(t, beaconDB.SaveGenesisBlockRoot(ctx, validRoot))
require.NoError(t, service.updateFinalized(ctx, validCheckpoint))
cp, err = service.cfg.BeaconDB.LastValidatedCheckpoint(ctx)
require.NoError(t, err)
optimistic, err := service.IsOptimisticForRoot(ctx, validRoot)
require.NoError(t, err)
require.Equal(t, false, optimistic)
require.DeepEqual(t, validCheckpoint.Root, cp.Root)
require.Equal(t, validCheckpoint.Epoch, cp.Epoch)
}

View File

@@ -0,0 +1,155 @@
package blockchain
import (
"github.com/prysmaticlabs/prysm/async/event"
"github.com/prysmaticlabs/prysm/beacon-chain/cache/depositcache"
statefeed "github.com/prysmaticlabs/prysm/beacon-chain/core/feed/state"
"github.com/prysmaticlabs/prysm/beacon-chain/db"
"github.com/prysmaticlabs/prysm/beacon-chain/forkchoice"
"github.com/prysmaticlabs/prysm/beacon-chain/operations/attestations"
"github.com/prysmaticlabs/prysm/beacon-chain/operations/slashings"
"github.com/prysmaticlabs/prysm/beacon-chain/operations/voluntaryexits"
"github.com/prysmaticlabs/prysm/beacon-chain/p2p"
"github.com/prysmaticlabs/prysm/beacon-chain/powchain"
enginev1 "github.com/prysmaticlabs/prysm/beacon-chain/powchain/engine-api-client/v1"
"github.com/prysmaticlabs/prysm/beacon-chain/state"
"github.com/prysmaticlabs/prysm/beacon-chain/state/stategen"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
)
type Option func(s *Service) error
// WithMaxGoroutines to control resource use of the blockchain service.
func WithMaxGoroutines(x int) Option {
return func(s *Service) error {
s.cfg.MaxRoutines = x
return nil
}
}
// WithWeakSubjectivityCheckpoint for checkpoint sync.
func WithWeakSubjectivityCheckpoint(c *ethpb.Checkpoint) Option {
return func(s *Service) error {
s.cfg.WeakSubjectivityCheckpt = c
return nil
}
}
// WithDatabase for head access.
func WithDatabase(beaconDB db.HeadAccessDatabase) Option {
return func(s *Service) error {
s.cfg.BeaconDB = beaconDB
return nil
}
}
// WithChainStartFetcher to retrieve information about genesis.
func WithChainStartFetcher(f powchain.ChainStartFetcher) Option {
return func(s *Service) error {
s.cfg.ChainStartFetcher = f
return nil
}
}
// WithExecutionEngineCaller to call execution engine.
func WithExecutionEngineCaller(c enginev1.Caller) Option {
return func(s *Service) error {
s.cfg.ExecutionEngineCaller = c
return nil
}
}
// WithDepositCache for deposit lifecycle after chain inclusion.
func WithDepositCache(c *depositcache.DepositCache) Option {
return func(s *Service) error {
s.cfg.DepositCache = c
return nil
}
}
// WithAttestationPool for attestation lifecycle after chain inclusion.
func WithAttestationPool(p attestations.Pool) Option {
return func(s *Service) error {
s.cfg.AttPool = p
return nil
}
}
// WithExitPool for exits lifecycle after chain inclusion.
func WithExitPool(p voluntaryexits.PoolManager) Option {
return func(s *Service) error {
s.cfg.ExitPool = p
return nil
}
}
// WithSlashingPool for slashings lifecycle after chain inclusion.
func WithSlashingPool(p slashings.PoolManager) Option {
return func(s *Service) error {
s.cfg.SlashingPool = p
return nil
}
}
// WithP2PBroadcaster to broadcast messages after appropriate processing.
func WithP2PBroadcaster(p p2p.Broadcaster) Option {
return func(s *Service) error {
s.cfg.P2p = p
return nil
}
}
// WithStateNotifier to notify an event feed of state processing.
func WithStateNotifier(n statefeed.Notifier) Option {
return func(s *Service) error {
s.cfg.StateNotifier = n
return nil
}
}
// WithForkChoiceStore to update an optimized fork-choice representation.
func WithForkChoiceStore(f forkchoice.ForkChoicer) Option {
return func(s *Service) error {
s.cfg.ForkChoiceStore = f
return nil
}
}
// WithAttestationService for dealing with attestation lifecycles.
func WithAttestationService(srv *attestations.Service) Option {
return func(s *Service) error {
s.cfg.AttService = srv
return nil
}
}
// WithStateGen for managing state regeneration and replay.
func WithStateGen(g *stategen.State) Option {
return func(s *Service) error {
s.cfg.StateGen = g
return nil
}
}
// WithSlasherAttestationsFeed to forward attestations into slasher if enabled.
func WithSlasherAttestationsFeed(f *event.Feed) Option {
return func(s *Service) error {
s.cfg.SlasherAttestationsFeed = f
return nil
}
}
func withStateBalanceCache(c *stateBalanceCache) Option {
return func(s *Service) error {
s.justifiedBalances = c
return nil
}
}
// WithFinalizedStateAtStartUp to store finalized state at start up.
func WithFinalizedStateAtStartUp(st state.BeaconState) Option {
return func(s *Service) error {
s.cfg.FinalizedStateAtStartUp = st
return nil
}
}

View File

@@ -0,0 +1,141 @@
package blockchain
import (
"bytes"
"context"
"fmt"
"math/big"
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/common/hexutil"
"github.com/holiman/uint256"
"github.com/pkg/errors"
types "github.com/prysmaticlabs/eth2-types"
"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
enginev1 "github.com/prysmaticlabs/prysm/proto/engine/v1"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/block"
"github.com/prysmaticlabs/prysm/time/slots"
"github.com/sirupsen/logrus"
)
// validateMergeBlock validates terminal block hash in the event of manual overrides before checking for total difficulty.
//
// def validate_merge_block(block: BeaconBlock) -> None:
// if TERMINAL_BLOCK_HASH != Hash32():
// # If `TERMINAL_BLOCK_HASH` is used as an override, the activation epoch must be reached.
// assert compute_epoch_at_slot(block.slot) >= TERMINAL_BLOCK_HASH_ACTIVATION_EPOCH
// assert block.body.execution_payload.parent_hash == TERMINAL_BLOCK_HASH
// return
//
// pow_block = get_pow_block(block.body.execution_payload.parent_hash)
// # Check if `pow_block` is available
// assert pow_block is not None
// pow_parent = get_pow_block(pow_block.parent_hash)
// # Check if `pow_parent` is available
// assert pow_parent is not None
// # Check if `pow_block` is a valid terminal PoW block
// assert is_valid_terminal_pow_block(pow_block, pow_parent)
func (s *Service) validateMergeBlock(ctx context.Context, b block.SignedBeaconBlock) error {
if err := helpers.BeaconBlockIsNil(b); err != nil {
return err
}
payload, err := b.Block().Body().ExecutionPayload()
if err != nil {
return err
}
if payload == nil {
return errors.New("nil execution payload")
}
if err := validateTerminalBlockHash(b.Block().Slot(), payload); err != nil {
return errors.Wrap(err, "could not validate terminal block hash")
}
mergeBlockParentHash, mergeBlockTD, err := s.getBlkParentHashAndTD(ctx, payload.ParentHash)
if err != nil {
return errors.Wrap(err, "could not get merge block parent hash and total difficulty")
}
_, mergeBlockParentTD, err := s.getBlkParentHashAndTD(ctx, mergeBlockParentHash)
if err != nil {
return errors.Wrap(err, "could not get merge parent block total difficulty")
}
valid, err := validateTerminalBlockDifficulties(mergeBlockTD, mergeBlockParentTD)
if err != nil {
return err
}
if !valid {
return fmt.Errorf("invalid TTD, configTTD: %s, currentTTD: %s, parentTTD: %s",
params.BeaconConfig().TerminalTotalDifficulty, mergeBlockTD, mergeBlockParentTD)
}
log.WithFields(logrus.Fields{
"slot": b.Block().Slot(),
"mergeBlockHash": common.BytesToHash(payload.ParentHash).String(),
"mergeBlockParentHash": common.BytesToHash(mergeBlockParentHash).String(),
"terminalTotalDifficulty": params.BeaconConfig().TerminalTotalDifficulty,
"mergeBlockTotalDifficulty": mergeBlockTD,
"mergeBlockParentTotalDifficulty": mergeBlockParentTD,
}).Info("Validated terminal block")
return nil
}
// getBlkParentHashAndTD retrieves the parent hash and total difficulty of the given block.
func (s *Service) getBlkParentHashAndTD(ctx context.Context, blkHash []byte) ([]byte, *uint256.Int, error) {
blk, err := s.cfg.ExecutionEngineCaller.ExecutionBlockByHash(ctx, common.BytesToHash(blkHash))
if err != nil {
return nil, nil, errors.Wrap(err, "could not get pow block")
}
if blk == nil {
return nil, nil, errors.New("pow block is nil")
}
blkTDBig, err := hexutil.DecodeBig(blk.TotalDifficulty)
if err != nil {
return nil, nil, errors.Wrap(err, "could not decode merge block total difficulty")
}
blkTDUint256, overflows := uint256.FromBig(blkTDBig)
if overflows {
return nil, nil, errors.New("total difficulty overflows")
}
return blk.ParentHash, blkTDUint256, nil
}
// validateTerminalBlockHash validates if the merge block is a valid terminal PoW block.
// spec code:
// if TERMINAL_BLOCK_HASH != Hash32():
// # If `TERMINAL_BLOCK_HASH` is used as an override, the activation epoch must be reached.
// assert compute_epoch_at_slot(block.slot) >= TERMINAL_BLOCK_HASH_ACTIVATION_EPOCH
// assert block.body.execution_payload.parent_hash == TERMINAL_BLOCK_HASH
// return
func validateTerminalBlockHash(blkSlot types.Slot, payload *enginev1.ExecutionPayload) error {
if bytesutil.ToBytes32(params.BeaconConfig().TerminalBlockHash.Bytes()) == [32]byte{} {
return nil
}
if params.BeaconConfig().TerminalBlockHashActivationEpoch > slots.ToEpoch(blkSlot) {
return errors.New("terminal block hash activation epoch not reached")
}
if !bytes.Equal(payload.ParentHash, params.BeaconConfig().TerminalBlockHash.Bytes()) {
return errors.New("parent hash does not match terminal block hash")
}
return nil
}
// validateTerminalBlockDifficulties validates terminal pow block by comparing own total difficulty with parent's total difficulty.
//
// def is_valid_terminal_pow_block(block: PowBlock, parent: PowBlock) -> bool:
// is_total_difficulty_reached = block.total_difficulty >= TERMINAL_TOTAL_DIFFICULTY
// is_parent_total_difficulty_valid = parent.total_difficulty < TERMINAL_TOTAL_DIFFICULTY
// return is_total_difficulty_reached and is_parent_total_difficulty_valid
func validateTerminalBlockDifficulties(currentDifficulty *uint256.Int, parentDifficulty *uint256.Int) (bool, error) {
b, ok := new(big.Int).SetString(params.BeaconConfig().TerminalTotalDifficulty, 10)
if !ok {
return false, errors.New("failed to parse terminal total difficulty")
}
ttd, of := uint256.FromBig(b)
if of {
return false, errors.New("overflow terminal total difficulty")
}
totalDifficultyReached := currentDifficulty.Cmp(ttd) >= 0
parentTotalDifficultyValid := ttd.Cmp(parentDifficulty) > 0
return totalDifficultyReached && parentTotalDifficultyValid, nil
}

View File

@@ -0,0 +1,210 @@
package blockchain
import (
"context"
"fmt"
"math/big"
"testing"
"github.com/holiman/uint256"
testDB "github.com/prysmaticlabs/prysm/beacon-chain/db/testing"
"github.com/prysmaticlabs/prysm/beacon-chain/forkchoice/protoarray"
"github.com/prysmaticlabs/prysm/beacon-chain/state/stategen"
fieldparams "github.com/prysmaticlabs/prysm/config/fieldparams"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
enginev1 "github.com/prysmaticlabs/prysm/proto/engine/v1"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/wrapper"
"github.com/prysmaticlabs/prysm/testing/require"
)
func Test_validTerminalPowBlock(t *testing.T) {
tests := []struct {
name string
currentDifficulty *uint256.Int
parentDifficulty *uint256.Int
ttd uint64
want bool
}{
{
name: "current > ttd, parent > ttd",
currentDifficulty: uint256.NewInt(2),
parentDifficulty: uint256.NewInt(2),
ttd: 1,
want: false,
},
{
name: "current < ttd, parent < ttd",
currentDifficulty: uint256.NewInt(2),
parentDifficulty: uint256.NewInt(2),
ttd: 3,
want: false,
},
{
name: "current == ttd, parent == ttd",
currentDifficulty: uint256.NewInt(2),
parentDifficulty: uint256.NewInt(2),
ttd: 2,
want: false,
},
{
name: "current > ttd, parent == ttd",
currentDifficulty: uint256.NewInt(2),
parentDifficulty: uint256.NewInt(1),
ttd: 1,
want: false,
},
{
name: "current == ttd, parent < ttd",
currentDifficulty: uint256.NewInt(2),
parentDifficulty: uint256.NewInt(1),
ttd: 2,
want: true,
},
{
name: "current > ttd, parent < ttd",
currentDifficulty: uint256.NewInt(3),
parentDifficulty: uint256.NewInt(1),
ttd: 2,
want: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
cfg := params.BeaconConfig()
cfg.TerminalTotalDifficulty = fmt.Sprint(tt.ttd)
params.OverrideBeaconConfig(cfg)
got, err := validateTerminalBlockDifficulties(tt.currentDifficulty, tt.parentDifficulty)
require.NoError(t, err)
if got != tt.want {
t.Errorf("validateTerminalBlockDifficulties() = %v, want %v", got, tt.want)
}
})
}
}
func Test_validTerminalPowBlockSpecConfig(t *testing.T) {
cfg := params.BeaconConfig()
cfg.TerminalTotalDifficulty = "115792089237316195423570985008687907853269984665640564039457584007913129638912"
params.OverrideBeaconConfig(cfg)
i, _ := new(big.Int).SetString("115792089237316195423570985008687907853269984665640564039457584007913129638912", 10)
current, of := uint256.FromBig(i)
require.Equal(t, of, false)
i, _ = new(big.Int).SetString("115792089237316195423570985008687907853269984665640564039457584007913129638911", 10)
parent, of := uint256.FromBig(i)
require.Equal(t, of, false)
got, err := validateTerminalBlockDifficulties(current, parent)
require.NoError(t, err)
require.Equal(t, true, got)
}
func Test_validateMergeBlock(t *testing.T) {
cfg := params.BeaconConfig()
cfg.TerminalTotalDifficulty = "2"
params.OverrideBeaconConfig(cfg)
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
fcs := protoarray.New(0, 0, [32]byte{'a'})
opts := []Option{
WithDatabase(beaconDB),
WithStateGen(stategen.New(beaconDB)),
WithForkChoiceStore(fcs),
}
service, err := NewService(ctx, opts...)
require.NoError(t, err)
engine := &mockEngineService{blks: map[[32]byte]*enginev1.ExecutionBlock{}}
service.cfg.ExecutionEngineCaller = engine
engine.blks[[32]byte{'a'}] = &enginev1.ExecutionBlock{
ParentHash: bytesutil.PadTo([]byte{'b'}, fieldparams.RootLength),
TotalDifficulty: "0x2",
}
engine.blks[[32]byte{'b'}] = &enginev1.ExecutionBlock{
ParentHash: bytesutil.PadTo([]byte{'3'}, fieldparams.RootLength),
TotalDifficulty: "0x1",
}
blk := &ethpb.SignedBeaconBlockBellatrix{
Block: &ethpb.BeaconBlockBellatrix{
Slot: 1,
Body: &ethpb.BeaconBlockBodyBellatrix{
ExecutionPayload: &enginev1.ExecutionPayload{
ParentHash: bytesutil.PadTo([]byte{'a'}, fieldparams.RootLength),
},
},
},
}
b, err := wrapper.WrappedSignedBeaconBlock(blk)
require.NoError(t, err)
require.NoError(t, service.validateMergeBlock(ctx, b))
cfg.TerminalTotalDifficulty = "1"
params.OverrideBeaconConfig(cfg)
require.ErrorContains(t, "invalid TTD, configTTD: 1, currentTTD: 2, parentTTD: 1", service.validateMergeBlock(ctx, b))
}
func Test_getBlkParentHashAndTD(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
fcs := protoarray.New(0, 0, [32]byte{'a'})
opts := []Option{
WithDatabase(beaconDB),
WithStateGen(stategen.New(beaconDB)),
WithForkChoiceStore(fcs),
}
service, err := NewService(ctx, opts...)
require.NoError(t, err)
engine := &mockEngineService{blks: map[[32]byte]*enginev1.ExecutionBlock{}}
service.cfg.ExecutionEngineCaller = engine
h := [32]byte{'a'}
p := [32]byte{'b'}
td := "0x1"
engine.blks[h] = &enginev1.ExecutionBlock{
ParentHash: p[:],
TotalDifficulty: td,
}
parentHash, totalDifficulty, err := service.getBlkParentHashAndTD(ctx, h[:])
require.NoError(t, err)
require.Equal(t, p, bytesutil.ToBytes32(parentHash))
require.Equal(t, td, totalDifficulty.String())
_, _, err = service.getBlkParentHashAndTD(ctx, []byte{'c'})
require.ErrorContains(t, "could not get pow block: block not found", err)
engine.blks[h] = nil
_, _, err = service.getBlkParentHashAndTD(ctx, h[:])
require.ErrorContains(t, "pow block is nil", err)
engine.blks[h] = &enginev1.ExecutionBlock{
ParentHash: p[:],
TotalDifficulty: "1",
}
_, _, err = service.getBlkParentHashAndTD(ctx, h[:])
require.ErrorContains(t, "could not decode merge block total difficulty: hex string without 0x prefix", err)
engine.blks[h] = &enginev1.ExecutionBlock{
ParentHash: p[:],
TotalDifficulty: "0XFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF",
}
_, _, err = service.getBlkParentHashAndTD(ctx, h[:])
require.ErrorContains(t, "could not decode merge block total difficulty: hex number > 256 bits", err)
}
func Test_validateTerminalBlockHash(t *testing.T) {
require.NoError(t, validateTerminalBlockHash(1, &enginev1.ExecutionPayload{}))
cfg := params.BeaconConfig()
cfg.TerminalBlockHash = [32]byte{0x01}
params.OverrideBeaconConfig(cfg)
require.ErrorContains(t, "terminal block hash activation epoch not reached", validateTerminalBlockHash(1, &enginev1.ExecutionPayload{}))
cfg.TerminalBlockHashActivationEpoch = 0
params.OverrideBeaconConfig(cfg)
require.ErrorContains(t, "parent hash does not match terminal block hash", validateTerminalBlockHash(1, &enginev1.ExecutionPayload{}))
require.NoError(t, validateTerminalBlockHash(1, &enginev1.ExecutionPayload{ParentHash: cfg.TerminalBlockHash.Bytes()}))
}

View File

@@ -2,6 +2,7 @@ package blockchain
import (
"context"
"time"
"github.com/pkg/errors"
"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
@@ -9,12 +10,11 @@ import (
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/attestation"
"github.com/prysmaticlabs/prysm/time"
"github.com/prysmaticlabs/prysm/time/slots"
"go.opencensus.io/trace"
)
// onAttestation is called whenever an attestation is received, verifies the attestation is valid and saves
// OnAttestation is called whenever an attestation is received, verifies the attestation is valid and saves
// it to the DB. As a stateless function, this does not hold nor delay attestation based on the spec descriptions.
// The delay is handled by the caller in `processAttestations`.
//
@@ -36,7 +36,7 @@ import (
//
// # Update latest messages for attesting indices
// update_latest_messages(store, indexed_attestation.attesting_indices, attestation)
func (s *Service) onAttestation(ctx context.Context, a *ethpb.Attestation) error {
func (s *Service) OnAttestation(ctx context.Context, a *ethpb.Attestation) error {
ctx, span := trace.StartSpan(ctx, "blockChain.onAttestation")
defer span.End()
@@ -59,10 +59,10 @@ func (s *Service) onAttestation(ctx context.Context, a *ethpb.Attestation) error
return err
}
genesisTime := baseState.GenesisTime()
genesisTime := uint64(s.genesisTime.Unix())
// Verify attestation target is from current epoch or previous epoch.
if err := s.verifyAttTargetEpoch(ctx, genesisTime, uint64(time.Now().Unix()), tgt); err != nil {
if err := verifyAttTargetEpoch(ctx, genesisTime, uint64(time.Now().Unix()), tgt); err != nil {
return err
}

View File

@@ -42,11 +42,9 @@ func (s *Service) getAttPreState(ctx context.Context, c *ethpb.Checkpoint) (stat
if err != nil {
return nil, err
}
if epochStartSlot > baseState.Slot() {
baseState, err = transition.ProcessSlots(ctx, baseState, epochStartSlot)
if err != nil {
return nil, errors.Wrapf(err, "could not process slots up to epoch %d", c.Epoch)
}
baseState, err = transition.ProcessSlotsIfPossible(ctx, baseState, epochStartSlot)
if err != nil {
return nil, errors.Wrapf(err, "could not process slots up to epoch %d", c.Epoch)
}
// Sharing the same state across caches is perfectly fine here, the fetching
@@ -61,7 +59,7 @@ func (s *Service) getAttPreState(ctx context.Context, c *ethpb.Checkpoint) (stat
}
// verifyAttTargetEpoch validates attestation is from the current or previous epoch.
func (s *Service) verifyAttTargetEpoch(_ context.Context, genesisTime, nowTime uint64, c *ethpb.Checkpoint) error {
func verifyAttTargetEpoch(_ context.Context, genesisTime, nowTime uint64, c *ethpb.Checkpoint) error {
currentSlot := types.Slot((nowTime - genesisTime) / params.BeaconConfig().SecondsPerSlot)
currentEpoch := slots.ToEpoch(currentSlot)
var prevEpoch types.Epoch
@@ -87,7 +85,7 @@ func (s *Service) verifyBeaconBlock(ctx context.Context, data *ethpb.Attestation
if (b == nil || b.IsNil()) && s.hasInitSyncBlock(r) {
b = s.getInitSyncBlock(r)
}
if err := helpers.VerifyNilBeaconBlock(b); err != nil {
if err := helpers.BeaconBlockIsNil(b); err != nil {
return err
}
if b.Block().Slot() > data.Slot {

View File

@@ -3,12 +3,15 @@ package blockchain
import (
"context"
"testing"
"time"
types "github.com/prysmaticlabs/eth2-types"
"github.com/prysmaticlabs/prysm/beacon-chain/core/transition"
testDB "github.com/prysmaticlabs/prysm/beacon-chain/db/testing"
doublylinkedtree "github.com/prysmaticlabs/prysm/beacon-chain/forkchoice/doubly-linked-tree"
"github.com/prysmaticlabs/prysm/beacon-chain/forkchoice/protoarray"
"github.com/prysmaticlabs/prysm/beacon-chain/state/stategen"
fieldparams "github.com/prysmaticlabs/prysm/config/fieldparams"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
@@ -16,20 +19,19 @@ import (
"github.com/prysmaticlabs/prysm/testing/assert"
"github.com/prysmaticlabs/prysm/testing/require"
"github.com/prysmaticlabs/prysm/testing/util"
"github.com/prysmaticlabs/prysm/time"
"github.com/prysmaticlabs/prysm/time/slots"
)
func TestStore_OnAttestation_ErrorConditions(t *testing.T) {
func TestStore_OnAttestation_ErrorConditions_ProtoArray(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
cfg := &Config{
BeaconDB: beaconDB,
ForkChoiceStore: protoarray.New(0, 0, [32]byte{}),
StateGen: stategen.New(beaconDB),
opts := []Option{
WithDatabase(beaconDB),
WithForkChoiceStore(protoarray.New(0, 0, [32]byte{})),
WithStateGen(stategen.New(beaconDB)),
}
service, err := NewService(ctx, cfg)
service, err := NewService(ctx, opts...)
require.NoError(t, err)
_, err = blockTree1(t, beaconDB, []byte{'g'})
@@ -37,13 +39,17 @@ func TestStore_OnAttestation_ErrorConditions(t *testing.T) {
BlkWithOutState := util.NewBeaconBlock()
BlkWithOutState.Block.Slot = 0
require.NoError(t, beaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(BlkWithOutState)))
wsb, err := wrapper.WrappedSignedBeaconBlock(BlkWithOutState)
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, wsb))
BlkWithOutStateRoot, err := BlkWithOutState.Block.HashTreeRoot()
require.NoError(t, err)
BlkWithStateBadAtt := util.NewBeaconBlock()
BlkWithStateBadAtt.Block.Slot = 1
require.NoError(t, beaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(BlkWithStateBadAtt)))
wsb, err = wrapper.WrappedSignedBeaconBlock(BlkWithStateBadAtt)
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, wsb))
BlkWithStateBadAttRoot, err := BlkWithStateBadAtt.Block.HashTreeRoot()
require.NoError(t, err)
@@ -54,7 +60,9 @@ func TestStore_OnAttestation_ErrorConditions(t *testing.T) {
BlkWithValidState := util.NewBeaconBlock()
BlkWithValidState.Block.Slot = 2
require.NoError(t, beaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(BlkWithValidState)))
wsb, err = wrapper.WrappedSignedBeaconBlock(BlkWithValidState)
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, wsb))
BlkWithValidStateRoot, err := BlkWithValidState.Block.HashTreeRoot()
require.NoError(t, err)
@@ -103,9 +111,9 @@ func TestStore_OnAttestation_ErrorConditions(t *testing.T) {
name: "process nil field (a.Target) in attestation",
a: &ethpb.Attestation{
Data: &ethpb.AttestationData{
BeaconBlockRoot: make([]byte, 32),
BeaconBlockRoot: make([]byte, fieldparams.RootLength),
Target: nil,
Source: &ethpb.Checkpoint{Root: make([]byte, 32)},
Source: &ethpb.Checkpoint{Root: make([]byte, fieldparams.RootLength)},
},
AggregationBits: make([]byte, 1),
Signature: make([]byte, 96),
@@ -116,7 +124,7 @@ func TestStore_OnAttestation_ErrorConditions(t *testing.T) {
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := service.onAttestation(ctx, tt.a)
err := service.OnAttestation(ctx, tt.a)
if tt.wantedErr != "" {
assert.ErrorContains(t, tt.wantedErr, err)
} else {
@@ -126,19 +134,132 @@ func TestStore_OnAttestation_ErrorConditions(t *testing.T) {
}
}
func TestStore_OnAttestation_Ok(t *testing.T) {
func TestStore_OnAttestation_ErrorConditions_DoublyLinkedTree(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
cfg := &Config{
BeaconDB: beaconDB,
ForkChoiceStore: protoarray.New(0, 0, [32]byte{}),
StateGen: stategen.New(beaconDB),
opts := []Option{
WithDatabase(beaconDB),
WithForkChoiceStore(doublylinkedtree.New(0, 0)),
WithStateGen(stategen.New(beaconDB)),
}
service, err := NewService(ctx, cfg)
service, err := NewService(ctx, opts...)
require.NoError(t, err)
_, err = blockTree1(t, beaconDB, []byte{'g'})
require.NoError(t, err)
BlkWithOutState := util.NewBeaconBlock()
BlkWithOutState.Block.Slot = 0
wsb, err := wrapper.WrappedSignedBeaconBlock(BlkWithOutState)
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, wsb))
BlkWithOutStateRoot, err := BlkWithOutState.Block.HashTreeRoot()
require.NoError(t, err)
BlkWithStateBadAtt := util.NewBeaconBlock()
BlkWithStateBadAtt.Block.Slot = 1
wsb, err = wrapper.WrappedSignedBeaconBlock(BlkWithStateBadAtt)
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, wsb))
BlkWithStateBadAttRoot, err := BlkWithStateBadAtt.Block.HashTreeRoot()
require.NoError(t, err)
s, err := util.NewBeaconState()
require.NoError(t, err)
require.NoError(t, s.SetSlot(100*params.BeaconConfig().SlotsPerEpoch))
require.NoError(t, service.cfg.BeaconDB.SaveState(ctx, s, BlkWithStateBadAttRoot))
BlkWithValidState := util.NewBeaconBlock()
BlkWithValidState.Block.Slot = 2
wsb, err = wrapper.WrappedSignedBeaconBlock(BlkWithValidState)
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, wsb))
BlkWithValidStateRoot, err := BlkWithValidState.Block.HashTreeRoot()
require.NoError(t, err)
s, err = util.NewBeaconState()
require.NoError(t, err)
err = s.SetFork(&ethpb.Fork{
Epoch: 0,
CurrentVersion: params.BeaconConfig().GenesisForkVersion,
PreviousVersion: params.BeaconConfig().GenesisForkVersion,
})
require.NoError(t, err)
require.NoError(t, service.cfg.BeaconDB.SaveState(ctx, s, BlkWithValidStateRoot))
tests := []struct {
name string
a *ethpb.Attestation
wantedErr string
}{
{
name: "attestation's data slot not aligned with target vote",
a: util.HydrateAttestation(&ethpb.Attestation{Data: &ethpb.AttestationData{Slot: params.BeaconConfig().SlotsPerEpoch, Target: &ethpb.Checkpoint{Root: make([]byte, 32)}}}),
wantedErr: "slot 32 does not match target epoch 0",
},
{
name: "no pre state for attestations's target block",
a: util.HydrateAttestation(&ethpb.Attestation{Data: &ethpb.AttestationData{Target: &ethpb.Checkpoint{Root: BlkWithOutStateRoot[:]}}}),
wantedErr: "could not get pre state for epoch 0",
},
{
name: "process attestation doesn't match current epoch",
a: util.HydrateAttestation(&ethpb.Attestation{Data: &ethpb.AttestationData{Slot: 100 * params.BeaconConfig().SlotsPerEpoch, Target: &ethpb.Checkpoint{Epoch: 100,
Root: BlkWithStateBadAttRoot[:]}}}),
wantedErr: "target epoch 100 does not match current epoch",
},
{
name: "process nil attestation",
a: nil,
wantedErr: "attestation can't be nil",
},
{
name: "process nil field (a.Data) in attestation",
a: &ethpb.Attestation{},
wantedErr: "attestation's data can't be nil",
},
{
name: "process nil field (a.Target) in attestation",
a: &ethpb.Attestation{
Data: &ethpb.AttestationData{
BeaconBlockRoot: make([]byte, fieldparams.RootLength),
Target: nil,
Source: &ethpb.Checkpoint{Root: make([]byte, fieldparams.RootLength)},
},
AggregationBits: make([]byte, 1),
Signature: make([]byte, 96),
},
wantedErr: "attestation's target can't be nil",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := service.OnAttestation(ctx, tt.a)
if tt.wantedErr != "" {
assert.ErrorContains(t, tt.wantedErr, err)
} else {
assert.NoError(t, err)
}
})
}
}
func TestStore_OnAttestation_Ok_ProtoArray(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
fcs := protoarray.New(0, 0, [32]byte{'a'})
opts := []Option{
WithDatabase(beaconDB),
WithStateGen(stategen.New(beaconDB)),
WithForkChoiceStore(fcs),
}
service, err := NewService(ctx, opts...)
require.NoError(t, err)
genesisState, pks := util.DeterministicGenesisState(t, 64)
require.NoError(t, genesisState.SetGenesisTime(uint64(time.Now().Unix())-params.BeaconConfig().SecondsPerSlot))
service.SetGenesisTime(time.Unix(time.Now().Unix()-int64(params.BeaconConfig().SecondsPerSlot), 0))
require.NoError(t, service.saveGenesisData(ctx, genesisState))
att, err := util.GenerateAttestations(genesisState, pks, 1, 0, false)
require.NoError(t, err)
@@ -147,28 +268,54 @@ func TestStore_OnAttestation_Ok(t *testing.T) {
copied, err = transition.ProcessSlots(ctx, copied, 1)
require.NoError(t, err)
require.NoError(t, service.cfg.BeaconDB.SaveState(ctx, copied, tRoot))
require.NoError(t, service.cfg.ForkChoiceStore.ProcessBlock(ctx, 0, tRoot, tRoot, tRoot, 1, 1))
require.NoError(t, service.onAttestation(ctx, att[0]))
require.NoError(t, service.cfg.ForkChoiceStore.InsertOptimisticBlock(ctx, 0, tRoot, tRoot, params.BeaconConfig().ZeroHash, 1, 1))
require.NoError(t, service.OnAttestation(ctx, att[0]))
}
func TestStore_OnAttestation_Ok_DoublyLinkedTree(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
fcs := doublylinkedtree.New(0, 0)
opts := []Option{
WithDatabase(beaconDB),
WithStateGen(stategen.New(beaconDB)),
WithForkChoiceStore(fcs),
}
service, err := NewService(ctx, opts...)
require.NoError(t, err)
genesisState, pks := util.DeterministicGenesisState(t, 64)
service.SetGenesisTime(time.Unix(time.Now().Unix()-int64(params.BeaconConfig().SecondsPerSlot), 0))
require.NoError(t, service.saveGenesisData(ctx, genesisState))
att, err := util.GenerateAttestations(genesisState, pks, 1, 0, false)
require.NoError(t, err)
tRoot := bytesutil.ToBytes32(att[0].Data.Target.Root)
copied := genesisState.Copy()
copied, err = transition.ProcessSlots(ctx, copied, 1)
require.NoError(t, err)
require.NoError(t, service.cfg.BeaconDB.SaveState(ctx, copied, tRoot))
require.NoError(t, service.cfg.ForkChoiceStore.InsertOptimisticBlock(ctx, 0, tRoot, tRoot, params.BeaconConfig().ZeroHash, 1, 1))
require.NoError(t, service.OnAttestation(ctx, att[0]))
}
func TestStore_SaveCheckpointState(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
cfg := &Config{
BeaconDB: beaconDB,
StateGen: stategen.New(beaconDB),
opts := []Option{
WithDatabase(beaconDB),
WithStateGen(stategen.New(beaconDB)),
}
service, err := NewService(ctx, cfg)
service, err := NewService(ctx, opts...)
require.NoError(t, err)
s, err := util.NewBeaconState()
require.NoError(t, err)
err = s.SetFinalizedCheckpoint(&ethpb.Checkpoint{Root: bytesutil.PadTo([]byte{'A'}, 32)})
err = s.SetFinalizedCheckpoint(&ethpb.Checkpoint{Root: bytesutil.PadTo([]byte{'A'}, fieldparams.RootLength)})
require.NoError(t, err)
val := &ethpb.Validator{
PublicKey: bytesutil.PadTo([]byte("foo"), 48),
WithdrawalCredentials: bytesutil.PadTo([]byte("bar"), 32),
WithdrawalCredentials: bytesutil.PadTo([]byte("bar"), fieldparams.RootLength),
}
err = s.SetValidators([]*ethpb.Validator{val})
require.NoError(t, err)
@@ -177,23 +324,23 @@ func TestStore_SaveCheckpointState(t *testing.T) {
r := [32]byte{'g'}
require.NoError(t, service.cfg.BeaconDB.SaveState(ctx, s, r))
service.justifiedCheckpt = &ethpb.Checkpoint{Root: r[:]}
service.bestJustifiedCheckpt = &ethpb.Checkpoint{Root: r[:]}
service.finalizedCheckpt = &ethpb.Checkpoint{Root: r[:]}
service.prevFinalizedCheckpt = &ethpb.Checkpoint{Root: r[:]}
service.store.SetJustifiedCheckpt(&ethpb.Checkpoint{Root: r[:]})
service.store.SetBestJustifiedCheckpt(&ethpb.Checkpoint{Root: r[:]})
service.store.SetFinalizedCheckpt(&ethpb.Checkpoint{Root: r[:]})
service.store.SetPrevFinalizedCheckpt(&ethpb.Checkpoint{Root: r[:]})
r = bytesutil.ToBytes32([]byte{'A'})
cp1 := &ethpb.Checkpoint{Epoch: 1, Root: bytesutil.PadTo([]byte{'A'}, 32)}
cp1 := &ethpb.Checkpoint{Epoch: 1, Root: bytesutil.PadTo([]byte{'A'}, fieldparams.RootLength)}
require.NoError(t, service.cfg.BeaconDB.SaveState(ctx, s, bytesutil.ToBytes32([]byte{'A'})))
require.NoError(t, service.cfg.BeaconDB.SaveStateSummary(ctx, &ethpb.StateSummary{Root: bytesutil.PadTo([]byte{'A'}, 32)}))
require.NoError(t, service.cfg.BeaconDB.SaveStateSummary(ctx, &ethpb.StateSummary{Root: bytesutil.PadTo([]byte{'A'}, fieldparams.RootLength)}))
s1, err := service.getAttPreState(ctx, cp1)
require.NoError(t, err)
assert.Equal(t, 1*params.BeaconConfig().SlotsPerEpoch, s1.Slot(), "Unexpected state slot")
cp2 := &ethpb.Checkpoint{Epoch: 2, Root: bytesutil.PadTo([]byte{'B'}, 32)}
cp2 := &ethpb.Checkpoint{Epoch: 2, Root: bytesutil.PadTo([]byte{'B'}, fieldparams.RootLength)}
require.NoError(t, service.cfg.BeaconDB.SaveState(ctx, s, bytesutil.ToBytes32([]byte{'B'})))
require.NoError(t, service.cfg.BeaconDB.SaveStateSummary(ctx, &ethpb.StateSummary{Root: bytesutil.PadTo([]byte{'B'}, 32)}))
require.NoError(t, service.cfg.BeaconDB.SaveStateSummary(ctx, &ethpb.StateSummary{Root: bytesutil.PadTo([]byte{'B'}, fieldparams.RootLength)}))
s2, err := service.getAttPreState(ctx, cp2)
require.NoError(t, err)
assert.Equal(t, 2*params.BeaconConfig().SlotsPerEpoch, s2.Slot(), "Unexpected state slot")
@@ -211,13 +358,13 @@ func TestStore_SaveCheckpointState(t *testing.T) {
assert.Equal(t, 2*params.BeaconConfig().SlotsPerEpoch, s2.Slot(), "Unexpected state slot")
require.NoError(t, s.SetSlot(params.BeaconConfig().SlotsPerEpoch+1))
service.justifiedCheckpt = &ethpb.Checkpoint{Root: r[:]}
service.bestJustifiedCheckpt = &ethpb.Checkpoint{Root: r[:]}
service.finalizedCheckpt = &ethpb.Checkpoint{Root: r[:]}
service.prevFinalizedCheckpt = &ethpb.Checkpoint{Root: r[:]}
cp3 := &ethpb.Checkpoint{Epoch: 1, Root: bytesutil.PadTo([]byte{'C'}, 32)}
service.store.SetJustifiedCheckpt(&ethpb.Checkpoint{Root: r[:]})
service.store.SetBestJustifiedCheckpt(&ethpb.Checkpoint{Root: r[:]})
service.store.SetFinalizedCheckpt(&ethpb.Checkpoint{Root: r[:]})
service.store.SetPrevFinalizedCheckpt(&ethpb.Checkpoint{Root: r[:]})
cp3 := &ethpb.Checkpoint{Epoch: 1, Root: bytesutil.PadTo([]byte{'C'}, fieldparams.RootLength)}
require.NoError(t, service.cfg.BeaconDB.SaveState(ctx, s, bytesutil.ToBytes32([]byte{'C'})))
require.NoError(t, service.cfg.BeaconDB.SaveStateSummary(ctx, &ethpb.StateSummary{Root: bytesutil.PadTo([]byte{'C'}, 32)}))
require.NoError(t, service.cfg.BeaconDB.SaveStateSummary(ctx, &ethpb.StateSummary{Root: bytesutil.PadTo([]byte{'C'}, fieldparams.RootLength)}))
s3, err := service.getAttPreState(ctx, cp3)
require.NoError(t, err)
assert.Equal(t, s.Slot(), s3.Slot(), "Unexpected state slot")
@@ -227,16 +374,16 @@ func TestStore_UpdateCheckpointState(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
cfg := &Config{
BeaconDB: beaconDB,
StateGen: stategen.New(beaconDB),
opts := []Option{
WithDatabase(beaconDB),
WithStateGen(stategen.New(beaconDB)),
}
service, err := NewService(ctx, cfg)
service, err := NewService(ctx, opts...)
require.NoError(t, err)
epoch := types.Epoch(1)
baseState, _ := util.DeterministicGenesisState(t, 1)
checkpoint := &ethpb.Checkpoint{Epoch: epoch, Root: bytesutil.PadTo([]byte("hi"), 32)}
checkpoint := &ethpb.Checkpoint{Epoch: epoch, Root: bytesutil.PadTo([]byte("hi"), fieldparams.RootLength)}
require.NoError(t, service.cfg.BeaconDB.SaveState(ctx, baseState, bytesutil.ToBytes32(checkpoint.Root)))
returned, err := service.getAttPreState(ctx, checkpoint)
require.NoError(t, err)
@@ -247,7 +394,7 @@ func TestStore_UpdateCheckpointState(t *testing.T) {
assert.Equal(t, returned.Slot(), cached.Slot(), "State should have been cached")
epoch = 2
newCheckpoint := &ethpb.Checkpoint{Epoch: epoch, Root: bytesutil.PadTo([]byte("bye"), 32)}
newCheckpoint := &ethpb.Checkpoint{Epoch: epoch, Root: bytesutil.PadTo([]byte("bye"), fieldparams.RootLength)}
require.NoError(t, service.cfg.BeaconDB.SaveState(ctx, baseState, bytesutil.ToBytes32(newCheckpoint.Root)))
returned, err = service.getAttPreState(ctx, newCheckpoint)
require.NoError(t, err)
@@ -264,47 +411,30 @@ func TestStore_UpdateCheckpointState(t *testing.T) {
func TestAttEpoch_MatchPrevEpoch(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
cfg := &Config{BeaconDB: beaconDB}
service, err := NewService(ctx, cfg)
require.NoError(t, err)
nowTime := uint64(params.BeaconConfig().SlotsPerEpoch) * params.BeaconConfig().SecondsPerSlot
require.NoError(t, service.verifyAttTargetEpoch(ctx, 0, nowTime, &ethpb.Checkpoint{Root: make([]byte, 32)}))
require.NoError(t, verifyAttTargetEpoch(ctx, 0, nowTime, &ethpb.Checkpoint{Root: make([]byte, fieldparams.RootLength)}))
}
func TestAttEpoch_MatchCurrentEpoch(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
cfg := &Config{BeaconDB: beaconDB}
service, err := NewService(ctx, cfg)
require.NoError(t, err)
nowTime := uint64(params.BeaconConfig().SlotsPerEpoch) * params.BeaconConfig().SecondsPerSlot
require.NoError(t, service.verifyAttTargetEpoch(ctx, 0, nowTime, &ethpb.Checkpoint{Epoch: 1}))
require.NoError(t, verifyAttTargetEpoch(ctx, 0, nowTime, &ethpb.Checkpoint{Epoch: 1}))
}
func TestAttEpoch_NotMatch(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
cfg := &Config{BeaconDB: beaconDB}
service, err := NewService(ctx, cfg)
require.NoError(t, err)
nowTime := 2 * uint64(params.BeaconConfig().SlotsPerEpoch) * params.BeaconConfig().SecondsPerSlot
err = service.verifyAttTargetEpoch(ctx, 0, nowTime, &ethpb.Checkpoint{Root: make([]byte, 32)})
err := verifyAttTargetEpoch(ctx, 0, nowTime, &ethpb.Checkpoint{Root: make([]byte, fieldparams.RootLength)})
assert.ErrorContains(t, "target epoch 0 does not match current epoch 2 or prev epoch 1", err)
}
func TestVerifyBeaconBlock_NoBlock(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
cfg := &Config{BeaconDB: beaconDB}
service, err := NewService(ctx, cfg)
opts := testServiceOptsWithDB(t)
service, err := NewService(ctx, opts...)
require.NoError(t, err)
d := util.HydrateAttestationData(&ethpb.AttestationData{})
@@ -313,15 +443,16 @@ func TestVerifyBeaconBlock_NoBlock(t *testing.T) {
func TestVerifyBeaconBlock_futureBlock(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
cfg := &Config{BeaconDB: beaconDB}
service, err := NewService(ctx, cfg)
opts := testServiceOptsWithDB(t)
service, err := NewService(ctx, opts...)
require.NoError(t, err)
b := util.NewBeaconBlock()
b.Block.Slot = 2
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(b)))
wsb, err := wrapper.WrappedSignedBeaconBlock(b)
require.NoError(t, err)
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wsb))
r, err := b.Block.HashTreeRoot()
require.NoError(t, err)
d := &ethpb.AttestationData{Slot: 1, BeaconBlockRoot: r[:]}
@@ -331,15 +462,16 @@ func TestVerifyBeaconBlock_futureBlock(t *testing.T) {
func TestVerifyBeaconBlock_OK(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
cfg := &Config{BeaconDB: beaconDB}
service, err := NewService(ctx, cfg)
opts := testServiceOptsWithDB(t)
service, err := NewService(ctx, opts...)
require.NoError(t, err)
b := util.NewBeaconBlock()
b.Block.Slot = 2
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(b)))
wsb, err := wrapper.WrappedSignedBeaconBlock(b)
require.NoError(t, err)
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wsb))
r, err := b.Block.HashTreeRoot()
require.NoError(t, err)
d := &ethpb.AttestationData{Slot: 2, BeaconBlockRoot: r[:]}
@@ -347,26 +479,69 @@ func TestVerifyBeaconBlock_OK(t *testing.T) {
assert.NoError(t, service.verifyBeaconBlock(ctx, d), "Did not receive the wanted error")
}
func TestVerifyFinalizedConsistency_InconsistentRoot(t *testing.T) {
func TestVerifyFinalizedConsistency_InconsistentRoot_ProtoArray(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
cfg := &Config{BeaconDB: beaconDB, ForkChoiceStore: protoarray.New(0, 0, [32]byte{})}
service, err := NewService(ctx, cfg)
fcs := protoarray.New(0, 0, [32]byte{'a'})
opts := []Option{
WithDatabase(beaconDB),
WithStateGen(stategen.New(beaconDB)),
WithForkChoiceStore(fcs),
}
service, err := NewService(ctx, opts...)
require.NoError(t, err)
b32 := util.NewBeaconBlock()
b32.Block.Slot = 32
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(b32)))
wsb, err := wrapper.WrappedSignedBeaconBlock(b32)
require.NoError(t, err)
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wsb))
r32, err := b32.Block.HashTreeRoot()
require.NoError(t, err)
service.finalizedCheckpt = &ethpb.Checkpoint{Epoch: 1}
service.store.SetFinalizedCheckpt(&ethpb.Checkpoint{Epoch: 1})
b33 := util.NewBeaconBlock()
b33.Block.Slot = 33
b33.Block.ParentRoot = r32[:]
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(b33)))
wsb, err = wrapper.WrappedSignedBeaconBlock(b33)
require.NoError(t, err)
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wsb))
r33, err := b33.Block.HashTreeRoot()
require.NoError(t, err)
err = service.VerifyFinalizedConsistency(context.Background(), r33[:])
require.ErrorContains(t, "Root and finalized store are not consistent", err)
}
func TestVerifyFinalizedConsistency_InconsistentRoot_DoublyLinkedTree(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
fcs := doublylinkedtree.New(0, 0)
opts := []Option{
WithDatabase(beaconDB),
WithStateGen(stategen.New(beaconDB)),
WithForkChoiceStore(fcs),
}
service, err := NewService(ctx, opts...)
require.NoError(t, err)
b32 := util.NewBeaconBlock()
b32.Block.Slot = 32
wsb, err := wrapper.WrappedSignedBeaconBlock(b32)
require.NoError(t, err)
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wsb))
r32, err := b32.Block.HashTreeRoot()
require.NoError(t, err)
service.store.SetFinalizedCheckpt(&ethpb.Checkpoint{Epoch: 1})
b33 := util.NewBeaconBlock()
b33.Block.Slot = 33
b33.Block.ParentRoot = r32[:]
wsb, err = wrapper.WrappedSignedBeaconBlock(b33)
require.NoError(t, err)
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wsb))
r33, err := b33.Block.HashTreeRoot()
require.NoError(t, err)
@@ -376,24 +551,27 @@ func TestVerifyFinalizedConsistency_InconsistentRoot(t *testing.T) {
func TestVerifyFinalizedConsistency_OK(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
cfg := &Config{BeaconDB: beaconDB, ForkChoiceStore: protoarray.New(0, 0, [32]byte{})}
service, err := NewService(ctx, cfg)
opts := testServiceOptsWithDB(t)
service, err := NewService(ctx, opts...)
require.NoError(t, err)
b32 := util.NewBeaconBlock()
b32.Block.Slot = 32
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(b32)))
wsb, err := wrapper.WrappedSignedBeaconBlock(b32)
require.NoError(t, err)
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wsb))
r32, err := b32.Block.HashTreeRoot()
require.NoError(t, err)
service.finalizedCheckpt = &ethpb.Checkpoint{Epoch: 1, Root: r32[:]}
service.store.SetFinalizedCheckpt(&ethpb.Checkpoint{Root: r32[:], Epoch: 1})
b33 := util.NewBeaconBlock()
b33.Block.Slot = 33
b33.Block.ParentRoot = r32[:]
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(b33)))
wsb, err = wrapper.WrappedSignedBeaconBlock(b33)
require.NoError(t, err)
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wsb))
r33, err := b33.Block.HashTreeRoot()
require.NoError(t, err)
@@ -403,10 +581,9 @@ func TestVerifyFinalizedConsistency_OK(t *testing.T) {
func TestVerifyFinalizedConsistency_IsCanonical(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
cfg := &Config{BeaconDB: beaconDB, ForkChoiceStore: protoarray.New(0, 0, [32]byte{})}
service, err := NewService(ctx, cfg)
opts := testServiceOptsWithDB(t)
service, err := NewService(ctx, opts...)
require.NoError(t, err)
b32 := util.NewBeaconBlock()
@@ -414,7 +591,7 @@ func TestVerifyFinalizedConsistency_IsCanonical(t *testing.T) {
r32, err := b32.Block.HashTreeRoot()
require.NoError(t, err)
service.finalizedCheckpt = &ethpb.Checkpoint{Epoch: 1, Root: r32[:]}
service.store.SetFinalizedCheckpt(&ethpb.Checkpoint{Root: r32[:], Epoch: 1})
b33 := util.NewBeaconBlock()
b33.Block.Slot = 33
@@ -422,8 +599,8 @@ func TestVerifyFinalizedConsistency_IsCanonical(t *testing.T) {
r33, err := b33.Block.HashTreeRoot()
require.NoError(t, err)
require.NoError(t, service.cfg.ForkChoiceStore.ProcessBlock(ctx, b32.Block.Slot, r32, [32]byte{}, [32]byte{}, 0, 0))
require.NoError(t, service.cfg.ForkChoiceStore.ProcessBlock(ctx, b33.Block.Slot, r33, r32, [32]byte{}, 0, 0))
require.NoError(t, service.cfg.ForkChoiceStore.InsertOptimisticBlock(ctx, b32.Block.Slot, r32, [32]byte{}, params.BeaconConfig().ZeroHash, 0, 0))
require.NoError(t, service.cfg.ForkChoiceStore.InsertOptimisticBlock(ctx, b33.Block.Slot, r33, r32, params.BeaconConfig().ZeroHash, 0, 0))
_, err = service.cfg.ForkChoiceStore.Head(ctx, 0, r32, []uint64{}, 0)
require.NoError(t, err)

View File

@@ -6,11 +6,13 @@ import (
"time"
"github.com/pkg/errors"
"github.com/prysmaticlabs/prysm/beacon-chain/core/blocks"
"github.com/prysmaticlabs/prysm/beacon-chain/core/feed"
statefeed "github.com/prysmaticlabs/prysm/beacon-chain/core/feed/state"
"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
coreTime "github.com/prysmaticlabs/prysm/beacon-chain/core/time"
"github.com/prysmaticlabs/prysm/beacon-chain/core/transition"
forkchoicetypes "github.com/prysmaticlabs/prysm/beacon-chain/forkchoice/types"
"github.com/prysmaticlabs/prysm/beacon-chain/state"
"github.com/prysmaticlabs/prysm/config/features"
"github.com/prysmaticlabs/prysm/config/params"
@@ -21,6 +23,7 @@ import (
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/attestation"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/block"
"github.com/prysmaticlabs/prysm/runtime/version"
"github.com/prysmaticlabs/prysm/time/slots"
"go.opencensus.io/trace"
)
@@ -88,9 +91,8 @@ var initialSyncBlockCacheSize = uint64(2 * params.BeaconConfig().SlotsPerEpoch)
func (s *Service) onBlock(ctx context.Context, signed block.SignedBeaconBlock, blockRoot [32]byte) error {
ctx, span := trace.StartSpan(ctx, "blockChain.onBlock")
defer span.End()
if signed == nil || signed.IsNil() || signed.Block().IsNil() {
return errors.New("nil block")
if err := helpers.BeaconBlockIsNil(signed); err != nil {
return err
}
b := signed.Block()
@@ -99,14 +101,36 @@ func (s *Service) onBlock(ctx context.Context, signed block.SignedBeaconBlock, b
return err
}
preStateVersion, preStateHeader, err := getStateVersionAndPayload(preState)
if err != nil {
return err
}
postState, err := transition.ExecuteStateTransition(ctx, preState, signed)
if err != nil {
return err
}
if err := s.savePostStateInfo(ctx, blockRoot, signed, postState, false /* reg sync */); err != nil {
return err
}
postStateVersion, postStateHeader, err := getStateVersionAndPayload(postState)
if err != nil {
return err
}
if err := s.notifyNewPayload(ctx, preStateVersion, postStateVersion, preStateHeader, postStateHeader, signed, blockRoot); err != nil {
return errors.Wrap(err, "could not verify new payload")
}
// We add a proposer score boost to fork choice for the block root if applicable, right after
// running a successful state transition for the block.
secondsIntoSlot := uint64(time.Since(s.genesisTime).Seconds()) % params.BeaconConfig().SecondsPerSlot
if err := s.cfg.ForkChoiceStore.BoostProposerRoot(ctx, &forkchoicetypes.ProposerBoostRootArgs{
BlockRoot: blockRoot,
BlockSlot: signed.Block().Slot(),
CurrentSlot: slots.SinceGenesis(s.genesisTime),
SecondsIntoSlot: secondsIntoSlot,
}); err != nil {
return err
}
// If slasher is configured, forward the attestations in the block via
// an event feed for processing.
@@ -136,25 +160,40 @@ func (s *Service) onBlock(ctx context.Context, signed block.SignedBeaconBlock, b
}
// Update justified check point.
currJustifiedEpoch := s.justifiedCheckpt.Epoch
justified := s.store.JustifiedCheckpt()
if justified == nil {
return errNilJustifiedInStore
}
currJustifiedEpoch := justified.Epoch
if postState.CurrentJustifiedCheckpoint().Epoch > currJustifiedEpoch {
if err := s.updateJustified(ctx, postState); err != nil {
return err
}
}
newFinalized := postState.FinalizedCheckpointEpoch() > s.finalizedCheckpt.Epoch
finalized := s.store.FinalizedCheckpt()
if finalized == nil {
return errNilFinalizedInStore
}
newFinalized := postState.FinalizedCheckpointEpoch() > finalized.Epoch
if newFinalized {
if err := s.finalizedImpliesNewJustified(ctx, postState); err != nil {
return errors.Wrap(err, "could not save new justified")
}
s.prevFinalizedCheckpt = s.finalizedCheckpt
s.finalizedCheckpt = postState.FinalizedCheckpoint()
s.store.SetPrevFinalizedCheckpt(finalized)
s.store.SetFinalizedCheckpt(postState.FinalizedCheckpoint())
s.store.SetPrevJustifiedCheckpt(justified)
s.store.SetJustifiedCheckpt(postState.CurrentJustifiedCheckpoint())
}
if err := s.updateHead(ctx, s.getJustifiedBalances()); err != nil {
balances, err := s.justifiedBalances.get(ctx, bytesutil.ToBytes32(justified.Root))
if err != nil {
msg := fmt.Sprintf("could not read balances for state w/ justified checkpoint %#x", justified.Root)
return errors.Wrap(err, msg)
}
if err := s.updateHead(ctx, balances); err != nil {
log.WithError(err).Warn("Could not update head")
}
if _, err := s.notifyForkchoiceUpdate(ctx, s.headBlock().Block(), s.headRoot(), bytesutil.ToBytes32(finalized.Root)); err != nil {
return err
}
if err := s.pruneCanonicalAttsFromPool(ctx, blockRoot, signed); err != nil {
return err
@@ -172,18 +211,16 @@ func (s *Service) onBlock(ctx context.Context, signed block.SignedBeaconBlock, b
})
// Updating next slot state cache can happen in the background. It shouldn't block rest of the process.
if features.Get().EnableNextSlotStateCache {
go func() {
// Use a custom deadline here, since this method runs asynchronously.
// We ignore the parent method's context and instead create a new one
// with a custom deadline, therefore using the background context instead.
slotCtx, cancel := context.WithTimeout(context.Background(), slotDeadline)
defer cancel()
if err := transition.UpdateNextSlotCache(slotCtx, blockRoot[:], postState); err != nil {
log.WithError(err).Debug("could not update next slot state cache")
}
}()
}
go func() {
// Use a custom deadline here, since this method runs asynchronously.
// We ignore the parent method's context and instead create a new one
// with a custom deadline, therefore using the background context instead.
slotCtx, cancel := context.WithTimeout(context.Background(), slotDeadline)
defer cancel()
if err := transition.UpdateNextSlotCache(slotCtx, blockRoot[:], postState); err != nil {
log.WithError(err).Debug("could not update next slot state cache")
}
}()
// Save justified check point to db.
if postState.CurrentJustifiedCheckpoint().Epoch > currJustifiedEpoch {
@@ -201,14 +238,19 @@ func (s *Service) onBlock(ctx context.Context, signed block.SignedBeaconBlock, b
if err := s.cfg.ForkChoiceStore.Prune(ctx, fRoot); err != nil {
return errors.Wrap(err, "could not prune proto array fork choice nodes")
}
isOptimistic, err := s.cfg.ForkChoiceStore.IsOptimistic(ctx, fRoot)
if err != nil {
return errors.Wrap(err, "could not check if node is optimistically synced")
}
go func() {
// Send an event regarding the new finalized checkpoint over a common event feed.
s.cfg.StateNotifier.StateFeed().Send(&feed.Event{
Type: statefeed.FinalizedCheckpoint,
Data: &ethpbv1.EventFinalizedCheckpoint{
Epoch: postState.FinalizedCheckpoint().Epoch,
Block: postState.FinalizedCheckpoint().Root,
State: signed.Block().StateRoot(),
Epoch: postState.FinalizedCheckpoint().Epoch,
Block: postState.FinalizedCheckpoint().Root,
State: signed.Block().StateRoot(),
ExecutionOptimistic: isOptimistic,
},
})
@@ -229,6 +271,24 @@ func (s *Service) onBlock(ctx context.Context, signed block.SignedBeaconBlock, b
return s.handleEpochBoundary(ctx, postState)
}
func getStateVersionAndPayload(st state.BeaconState) (int, *ethpb.ExecutionPayloadHeader, error) {
if st == nil {
return 0, nil, errors.New("nil state")
}
var preStateHeader *ethpb.ExecutionPayloadHeader
var err error
preStateVersion := st.Version()
switch preStateVersion {
case version.Phase0, version.Altair:
default:
preStateHeader, err = st.LatestExecutionPayloadHeader()
if err != nil {
return 0, nil, err
}
}
return preStateVersion, preStateHeader, nil
}
func (s *Service) onBlockBatch(ctx context.Context, blks []block.SignedBeaconBlock,
blockRoots [][32]byte) ([]*ethpb.Checkpoint, []*ethpb.Checkpoint, error) {
ctx, span := trace.StartSpan(ctx, "blockChain.onBlockBatch")
@@ -237,8 +297,13 @@ func (s *Service) onBlockBatch(ctx context.Context, blks []block.SignedBeaconBlo
if len(blks) == 0 || len(blockRoots) == 0 {
return nil, nil, errors.New("no blocks provided")
}
if blks[0] == nil || blks[0].IsNil() || blks[0].Block().IsNil() {
return nil, nil, errors.New("nil block")
if len(blks) != len(blockRoots) {
return nil, nil, errWrongBlockCount
}
if err := helpers.BeaconBlockIsNil(blks[0]); err != nil {
return nil, nil, err
}
b := blks[0].Block()
@@ -256,14 +321,29 @@ func (s *Service) onBlockBatch(ctx context.Context, blks []block.SignedBeaconBlo
jCheckpoints := make([]*ethpb.Checkpoint, len(blks))
fCheckpoints := make([]*ethpb.Checkpoint, len(blks))
sigSet := &bls.SignatureSet{
sigSet := &bls.SignatureBatch{
Signatures: [][]byte{},
PublicKeys: []bls.PublicKey{},
Messages: [][32]byte{},
}
var set *bls.SignatureSet
type versionAndHeader struct {
version int
header *ethpb.ExecutionPayloadHeader
}
preVersionAndHeaders := make([]*versionAndHeader, len(blks))
postVersionAndHeaders := make([]*versionAndHeader, len(blks))
var set *bls.SignatureBatch
boundaries := make(map[[32]byte]state.BeaconState)
for i, b := range blks {
v, h, err := getStateVersionAndPayload(preState)
if err != nil {
return nil, nil, err
}
preVersionAndHeaders[i] = &versionAndHeader{
version: v,
header: h,
}
set, preState, err = transition.ExecuteStateTransitionNoVerifyAnySig(ctx, preState, b)
if err != nil {
return nil, nil, err
@@ -271,12 +351,18 @@ func (s *Service) onBlockBatch(ctx context.Context, blks []block.SignedBeaconBlo
// Save potential boundary states.
if slots.IsEpochStart(preState.Slot()) {
boundaries[blockRoots[i]] = preState.Copy()
if err := s.handleEpochBoundary(ctx, preState); err != nil {
return nil, nil, errors.Wrap(err, "could not handle epoch boundary state")
}
}
jCheckpoints[i] = preState.CurrentJustifiedCheckpoint()
fCheckpoints[i] = preState.FinalizedCheckpoint()
v, h, err = getStateVersionAndPayload(preState)
if err != nil {
return nil, nil, err
}
postVersionAndHeaders[i] = &versionAndHeader{
version: v,
header: h,
}
sigSet.Join(set)
}
verify, err := sigSet.Verify()
@@ -286,6 +372,26 @@ func (s *Service) onBlockBatch(ctx context.Context, blks []block.SignedBeaconBlo
if !verify {
return nil, nil, errors.New("batch block signature verification failed")
}
// blocks have been verified, add them to forkchoice and call the engine
for i, b := range blks {
s.saveInitSyncBlock(blockRoots[i], b)
if err := s.insertBlockToForkChoiceStore(ctx, b.Block(), blockRoots[i], fCheckpoints[i], jCheckpoints[i]); err != nil {
return nil, nil, err
}
if err := s.notifyNewPayload(ctx,
preVersionAndHeaders[i].version,
postVersionAndHeaders[i].version,
preVersionAndHeaders[i].header,
postVersionAndHeaders[i].header, b, blockRoots[i]); err != nil {
return nil, nil, err
}
if _, err := s.notifyForkchoiceUpdate(ctx, b.Block(), blockRoots[i], bytesutil.ToBytes32(fCheckpoints[i].Root)); err != nil {
return nil, nil, err
}
}
for r, st := range boundaries {
if err := s.cfg.StateGen.SaveState(ctx, r, st); err != nil {
return nil, nil, err
@@ -307,12 +413,7 @@ func (s *Service) onBlockBatch(ctx context.Context, blks []block.SignedBeaconBlo
// their state summaries and split them off to relative hot/cold storage.
func (s *Service) handleBlockAfterBatchVerify(ctx context.Context, signed block.SignedBeaconBlock,
blockRoot [32]byte, fCheckpoint, jCheckpoint *ethpb.Checkpoint) error {
b := signed.Block()
s.saveInitSyncBlock(blockRoot, signed)
if err := s.insertBlockToForkChoiceStore(ctx, b, blockRoot, fCheckpoint, jCheckpoint); err != nil {
return err
}
if err := s.cfg.BeaconDB.SaveStateSummary(ctx, &ethpb.StateSummary{
Slot: signed.Block().Slot(),
Root: blockRoot[:],
@@ -328,19 +429,27 @@ func (s *Service) handleBlockAfterBatchVerify(ctx context.Context, signed block.
s.clearInitSyncBlocks()
}
if jCheckpoint.Epoch > s.justifiedCheckpt.Epoch {
justified := s.store.JustifiedCheckpt()
if justified == nil {
return errNilJustifiedInStore
}
if jCheckpoint.Epoch > justified.Epoch {
if err := s.updateJustifiedInitSync(ctx, jCheckpoint); err != nil {
return err
}
}
finalized := s.store.FinalizedCheckpt()
if finalized == nil {
return errNilFinalizedInStore
}
// Update finalized check point. Prune the block cache and helper caches on every new finalized epoch.
if fCheckpoint.Epoch > s.finalizedCheckpt.Epoch {
if fCheckpoint.Epoch > finalized.Epoch {
if err := s.updateFinalized(ctx, fCheckpoint); err != nil {
return err
}
s.prevFinalizedCheckpt = s.finalizedCheckpt
s.finalizedCheckpt = fCheckpoint
s.store.SetPrevFinalizedCheckpt(finalized)
s.store.SetFinalizedCheckpt(fCheckpoint)
}
return nil
}
@@ -419,13 +528,27 @@ func (s *Service) insertBlockToForkChoiceStore(ctx context.Context, blk block.Be
return err
}
// Feed in block to fork choice store.
if err := s.cfg.ForkChoiceStore.ProcessBlock(ctx,
blk.Slot(), root, bytesutil.ToBytes32(blk.ParentRoot()), bytesutil.ToBytes32(blk.Body().Graffiti()),
jCheckpoint.Epoch,
fCheckpoint.Epoch); err != nil {
return errors.Wrap(err, "could not process block for proto array fork choice")
payloadHash, err := getBlockPayloadHash(blk)
if err != nil {
return err
}
return nil
return s.cfg.ForkChoiceStore.InsertOptimisticBlock(ctx,
blk.Slot(), root, bytesutil.ToBytes32(blk.ParentRoot()), payloadHash,
jCheckpoint.Epoch,
fCheckpoint.Epoch)
}
func getBlockPayloadHash(blk block.BeaconBlock) ([32]byte, error) {
payloadHash := [32]byte{}
if blocks.IsPreBellatrixVersion(blk.Version()) {
return payloadHash, nil
}
payload, err := blk.Body().ExecutionPayload()
if err != nil {
return payloadHash, err
}
return bytesutil.ToBytes32(payload.BlockHash), nil
}
// This saves post state info to DB or cache. This also saves post state info to fork choice store.

View File

@@ -13,7 +13,6 @@ import (
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
"github.com/prysmaticlabs/prysm/monitoring/tracing"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/attestation"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/block"
"github.com/prysmaticlabs/prysm/time/slots"
"go.opencensus.io/trace"
@@ -45,7 +44,7 @@ func (s *Service) getBlockPreState(ctx context.Context, b block.BeaconBlock) (st
}
// Verify block slot time is not from the future.
if err := slots.VerifyTime(preState.GenesisTime(), b.Slot(), params.BeaconNetworkConfig().MaximumGossipClockDisparity); err != nil {
if err := slots.VerifyTime(uint64(s.genesisTime.Unix()), b.Slot(), params.BeaconNetworkConfig().MaximumGossipClockDisparity); err != nil {
return nil, err
}
@@ -92,7 +91,11 @@ func (s *Service) verifyBlkPreState(ctx context.Context, b block.BeaconBlock) er
func (s *Service) VerifyBlkDescendant(ctx context.Context, root [32]byte) error {
ctx, span := trace.StartSpan(ctx, "blockChain.VerifyBlkDescendant")
defer span.End()
fRoot := s.ensureRootNotZeros(bytesutil.ToBytes32(s.finalizedCheckpt.Root))
finalized := s.store.FinalizedCheckpt()
if finalized == nil {
return errNilFinalizedInStore
}
fRoot := s.ensureRootNotZeros(bytesutil.ToBytes32(finalized.Root))
finalizedBlkSigned, err := s.cfg.BeaconDB.Block(ctx, fRoot)
if err != nil {
return err
@@ -110,7 +113,7 @@ func (s *Service) VerifyBlkDescendant(ctx context.Context, root [32]byte) error
}
if !bytes.Equal(bFinalizedRoot, fRoot[:]) {
err := fmt.Errorf("block %#x is not a descendent of the current finalized block slot %d, %#x != %#x",
err := fmt.Errorf("block %#x is not a descendant of the current finalized block slot %d, %#x != %#x",
bytesutil.Trunc(root[:]), finalizedBlk.Slot(), bytesutil.Trunc(bFinalizedRoot),
bytesutil.Trunc(fRoot[:]))
tracing.AnnotateError(span, err)
@@ -122,7 +125,11 @@ func (s *Service) VerifyBlkDescendant(ctx context.Context, root [32]byte) error
// verifyBlkFinalizedSlot validates input block is not less than or equal
// to current finalized slot.
func (s *Service) verifyBlkFinalizedSlot(b block.BeaconBlock) error {
finalizedSlot, err := slots.EpochStart(s.finalizedCheckpt.Epoch)
finalized := s.store.FinalizedCheckpt()
if finalized == nil {
return errNilFinalizedInStore
}
finalizedSlot, err := slots.EpochStart(finalized.Epoch)
if err != nil {
return err
}
@@ -135,7 +142,24 @@ func (s *Service) verifyBlkFinalizedSlot(b block.BeaconBlock) error {
// shouldUpdateCurrentJustified prevents bouncing attack, by only update conflicting justified
// checkpoints in the fork choice if in the early slots of the epoch.
// Otherwise, delay incorporation of new justified checkpoint until next epoch boundary.
// See https://ethresear.ch/t/prevention-of-bouncing-attack-on-ffg/6114 for more detailed analysis and discussion.
//
// Spec code:
// def should_update_justified_checkpoint(store: Store, new_justified_checkpoint: Checkpoint) -> bool:
// """
// To address the bouncing attack, only update conflicting justified
// checkpoints in the fork choice if in the early slots of the epoch.
// Otherwise, delay incorporation of new justified checkpoint until next epoch boundary.
//
// See https://ethresear.ch/t/prevention-of-bouncing-attack-on-ffg/6114 for more detailed analysis and discussion.
// """
// if compute_slots_since_epoch_start(get_current_slot(store)) < SAFE_SLOTS_TO_UPDATE_JUSTIFIED:
// return True
//
// justified_slot = compute_start_slot_at_epoch(store.justified_checkpoint.epoch)
// if not get_ancestor(store, new_justified_checkpoint.root, justified_slot) == store.justified_checkpoint.root:
// return False
//
// return True
func (s *Service) shouldUpdateCurrentJustified(ctx context.Context, newJustifiedCheckpt *ethpb.Checkpoint) (bool, error) {
ctx, span := trace.StartSpan(ctx, "blockChain.shouldUpdateCurrentJustified")
defer span.End()
@@ -143,51 +167,20 @@ func (s *Service) shouldUpdateCurrentJustified(ctx context.Context, newJustified
if slots.SinceEpochStarts(s.CurrentSlot()) < params.BeaconConfig().SafeSlotsToUpdateJustified {
return true, nil
}
var newJustifiedBlockSigned block.SignedBeaconBlock
justified := s.store.JustifiedCheckpt()
jSlot, err := slots.EpochStart(justified.Epoch)
if err != nil {
return false, err
}
justifiedRoot := s.ensureRootNotZeros(bytesutil.ToBytes32(newJustifiedCheckpt.Root))
var err error
if s.hasInitSyncBlock(justifiedRoot) {
newJustifiedBlockSigned = s.getInitSyncBlock(justifiedRoot)
} else {
newJustifiedBlockSigned, err = s.cfg.BeaconDB.Block(ctx, justifiedRoot)
if err != nil {
return false, err
}
}
if newJustifiedBlockSigned == nil || newJustifiedBlockSigned.IsNil() || newJustifiedBlockSigned.Block().IsNil() {
return false, errors.New("nil new justified block")
}
newJustifiedBlock := newJustifiedBlockSigned.Block()
jSlot, err := slots.EpochStart(s.justifiedCheckpt.Epoch)
b, err := s.ancestor(ctx, justifiedRoot[:], jSlot)
if err != nil {
return false, err
}
if newJustifiedBlock.Slot() <= jSlot {
if !bytes.Equal(b, justified.Root) {
return false, nil
}
var justifiedBlockSigned block.SignedBeaconBlock
cachedJustifiedRoot := s.ensureRootNotZeros(bytesutil.ToBytes32(s.justifiedCheckpt.Root))
if s.hasInitSyncBlock(cachedJustifiedRoot) {
justifiedBlockSigned = s.getInitSyncBlock(cachedJustifiedRoot)
} else {
justifiedBlockSigned, err = s.cfg.BeaconDB.Block(ctx, cachedJustifiedRoot)
if err != nil {
return false, err
}
}
if justifiedBlockSigned == nil || justifiedBlockSigned.IsNil() || justifiedBlockSigned.Block().IsNil() {
return false, errors.New("nil justified block")
}
justifiedBlock := justifiedBlockSigned.Block()
b, err := s.ancestor(ctx, justifiedRoot[:], justifiedBlock.Slot())
if err != nil {
return false, err
}
if !bytes.Equal(b, s.justifiedCheckpt.Root) {
return false, nil
}
return true, nil
}
@@ -196,8 +189,12 @@ func (s *Service) updateJustified(ctx context.Context, state state.ReadOnlyBeaco
defer span.End()
cpt := state.CurrentJustifiedCheckpoint()
if cpt.Epoch > s.bestJustifiedCheckpt.Epoch {
s.bestJustifiedCheckpt = cpt
bestJustified := s.store.BestJustifiedCheckpt()
if bestJustified == nil {
return errNilBestJustifiedInStore
}
if cpt.Epoch > bestJustified.Epoch {
s.store.SetBestJustifiedCheckpt(cpt)
}
canUpdate, err := s.shouldUpdateCurrentJustified(ctx, cpt)
if err != nil {
@@ -205,11 +202,12 @@ func (s *Service) updateJustified(ctx context.Context, state state.ReadOnlyBeaco
}
if canUpdate {
s.prevJustifiedCheckpt = s.justifiedCheckpt
s.justifiedCheckpt = cpt
if err := s.cacheJustifiedStateBalances(ctx, bytesutil.ToBytes32(s.justifiedCheckpt.Root)); err != nil {
return err
justified := s.store.JustifiedCheckpt()
if justified == nil {
return errNilJustifiedInStore
}
s.store.SetPrevJustifiedCheckpt(justified)
s.store.SetJustifiedCheckpt(cpt)
}
return nil
@@ -219,13 +217,18 @@ func (s *Service) updateJustified(ctx context.Context, state state.ReadOnlyBeaco
// caches justified checkpoint balances for fork choice and save justified checkpoint in DB.
// This method does not have defense against fork choice bouncing attack, which is why it's only recommend to be used during initial syncing.
func (s *Service) updateJustifiedInitSync(ctx context.Context, cp *ethpb.Checkpoint) error {
s.prevJustifiedCheckpt = s.justifiedCheckpt
s.justifiedCheckpt = cp
if err := s.cacheJustifiedStateBalances(ctx, bytesutil.ToBytes32(s.justifiedCheckpt.Root)); err != nil {
justified := s.store.JustifiedCheckpt()
if justified == nil {
return errNilJustifiedInStore
}
s.store.SetPrevJustifiedCheckpt(justified)
if err := s.cfg.BeaconDB.SaveJustifiedCheckpoint(ctx, cp); err != nil {
return err
}
s.store.SetJustifiedCheckpt(cp)
return s.cfg.BeaconDB.SaveJustifiedCheckpoint(ctx, cp)
return nil
}
func (s *Service) updateFinalized(ctx context.Context, cp *ethpb.Checkpoint) error {
@@ -244,10 +247,19 @@ func (s *Service) updateFinalized(ctx context.Context, cp *ethpb.Checkpoint) err
}
fRoot := bytesutil.ToBytes32(cp.Root)
optimistic, err := s.cfg.ForkChoiceStore.IsOptimistic(ctx, fRoot)
if err != nil {
return err
}
if !optimistic {
err = s.cfg.BeaconDB.SaveLastValidatedCheckpoint(ctx, cp)
if err != nil {
return err
}
}
if err := s.cfg.StateGen.MigrateToCold(ctx, fRoot); err != nil {
return errors.Wrap(err, "could not migrate to cold")
}
return nil
}
@@ -324,49 +336,6 @@ func (s *Service) ancestorByDB(ctx context.Context, r [32]byte, slot types.Slot)
return s.ancestorByDB(ctx, bytesutil.ToBytes32(b.ParentRoot()), slot)
}
// This updates justified check point in store, if the new justified is later than stored justified or
// the store's justified is not in chain with finalized check point.
//
// Spec definition:
// # Potentially update justified if different from store
// if store.justified_checkpoint != state.current_justified_checkpoint:
// # Update justified if new justified is later than store justified
// if state.current_justified_checkpoint.epoch > store.justified_checkpoint.epoch:
// store.justified_checkpoint = state.current_justified_checkpoint
// return
// # Update justified if store justified is not in chain with finalized checkpoint
// finalized_slot = compute_start_slot_at_epoch(store.finalized_checkpoint.epoch)
// ancestor_at_finalized_slot = get_ancestor(store, store.justified_checkpoint.root, finalized_slot)
// if ancestor_at_finalized_slot != store.finalized_checkpoint.root:
// store.justified_checkpoint = state.current_justified_checkpoint
func (s *Service) finalizedImpliesNewJustified(ctx context.Context, state state.BeaconState) error {
// Update justified if it's different than the one cached in the store.
if !attestation.CheckPointIsEqual(s.justifiedCheckpt, state.CurrentJustifiedCheckpoint()) {
if state.CurrentJustifiedCheckpoint().Epoch > s.justifiedCheckpt.Epoch {
s.justifiedCheckpt = state.CurrentJustifiedCheckpoint()
return s.cacheJustifiedStateBalances(ctx, bytesutil.ToBytes32(s.justifiedCheckpt.Root))
}
// Update justified if store justified is not in chain with finalized check point.
finalizedSlot, err := slots.EpochStart(s.finalizedCheckpt.Epoch)
if err != nil {
return err
}
justifiedRoot := s.ensureRootNotZeros(bytesutil.ToBytes32(s.justifiedCheckpt.Root))
anc, err := s.ancestor(ctx, justifiedRoot[:], finalizedSlot)
if err != nil {
return err
}
if !bytes.Equal(anc, s.finalizedCheckpt.Root) {
s.justifiedCheckpt = state.CurrentJustifiedCheckpoint()
if err := s.cacheJustifiedStateBalances(ctx, bytesutil.ToBytes32(s.justifiedCheckpt.Root)); err != nil {
return err
}
}
}
return nil
}
// This retrieves missing blocks from DB (ie. the blocks that couldn't be received over sync) and inserts them to fork choice store.
// This is useful for block tree visualizer and additional vote accounting.
func (s *Service) fillInForkChoiceMissingBlocks(ctx context.Context, blk block.BeaconBlock,
@@ -377,7 +346,11 @@ func (s *Service) fillInForkChoiceMissingBlocks(ctx context.Context, blk block.B
parentRoot := bytesutil.ToBytes32(blk.ParentRoot())
slot := blk.Slot()
// Fork choice only matters from last finalized slot.
fSlot, err := slots.EpochStart(s.finalizedCheckpt.Epoch)
finalized := s.store.FinalizedCheckpt()
if finalized == nil {
return errNilFinalizedInStore
}
fSlot, err := slots.EpochStart(finalized.Epoch)
if err != nil {
return err
}
@@ -402,14 +375,17 @@ func (s *Service) fillInForkChoiceMissingBlocks(ctx context.Context, blk block.B
for i := len(pendingNodes) - 1; i >= 0; i-- {
b := pendingNodes[i]
r := pendingRoots[i]
if err := s.cfg.ForkChoiceStore.ProcessBlock(ctx,
b.Slot(), r, bytesutil.ToBytes32(b.ParentRoot()), bytesutil.ToBytes32(b.Body().Graffiti()),
payloadHash, err := getBlockPayloadHash(blk)
if err != nil {
return err
}
if err := s.cfg.ForkChoiceStore.InsertOptimisticBlock(ctx,
b.Slot(), r, bytesutil.ToBytes32(b.ParentRoot()), payloadHash,
jCheckpoint.Epoch,
fCheckpoint.Epoch); err != nil {
return errors.Wrap(err, "could not process block for proto array fork choice")
}
}
return nil
}
@@ -456,7 +432,7 @@ func (s *Service) deletePoolAtts(atts []*ethpb.Attestation) error {
// fork choice justification routine.
func (s *Service) ensureRootNotZeros(root [32]byte) [32]byte {
if root == params.BeaconConfig().ZeroHash {
return s.genesisRoot
return s.originBlockRoot
}
return root
}

File diff suppressed because it is too large Load Diff

View File

@@ -7,6 +7,7 @@ import (
"time"
"github.com/pkg/errors"
"github.com/prysmaticlabs/prysm/async/event"
"github.com/prysmaticlabs/prysm/beacon-chain/core/feed"
"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
"github.com/prysmaticlabs/prysm/beacon-chain/state"
@@ -41,7 +42,7 @@ func (s *Service) ReceiveAttestationNoPubsub(ctx context.Context, att *ethpb.Att
ctx, span := trace.StartSpan(ctx, "beacon-chain.blockchain.ReceiveAttestationNoPubsub")
defer span.End()
if err := s.onAttestation(ctx, att); err != nil {
if err := s.OnAttestation(ctx, att); err != nil {
return errors.Wrap(err, "could not process attestation")
}
@@ -87,6 +88,9 @@ func (s *Service) VerifyFinalizedConsistency(ctx context.Context, root []byte) e
}
f := s.FinalizedCheckpt()
if f == nil {
return errNilFinalizedInStore
}
ss, err := slots.EpochStart(f.Epoch)
if err != nil {
return err
@@ -103,38 +107,87 @@ func (s *Service) VerifyFinalizedConsistency(ctx context.Context, root []byte) e
}
// This routine processes fork choice attestations from the pool to account for validator votes and fork choice.
func (s *Service) processAttestationsRoutine(subscribedToStateEvents chan<- struct{}) {
func (s *Service) spawnProcessAttestationsRoutine(stateFeed *event.Feed) {
// Wait for state to be initialized.
stateChannel := make(chan *feed.Event, 1)
stateSub := s.cfg.StateNotifier.StateFeed().Subscribe(stateChannel)
subscribedToStateEvents <- struct{}{}
<-stateChannel
stateSub.Unsubscribe()
if s.genesisTime.IsZero() {
log.Warn("ProcessAttestations routine waiting for genesis time")
for s.genesisTime.IsZero() {
time.Sleep(1 * time.Second)
}
log.Warn("Genesis time received, now available to process attestations")
}
st := slots.NewSlotTicker(s.genesisTime, params.BeaconConfig().SecondsPerSlot)
for {
stateSub := stateFeed.Subscribe(stateChannel)
go func() {
select {
case <-s.ctx.Done():
stateSub.Unsubscribe()
return
case <-st.C():
// Continue when there's no fork choice attestation, there's nothing to process and update head.
// This covers the condition when the node is still initial syncing to the head of the chain.
if s.cfg.AttPool.ForkchoiceAttestationCount() == 0 {
continue
case <-stateChannel:
stateSub.Unsubscribe()
break
}
if s.genesisTime.IsZero() {
log.Warn("ProcessAttestations routine waiting for genesis time")
for s.genesisTime.IsZero() {
if err := s.ctx.Err(); err != nil {
log.WithError(err).Error("Giving up waiting for genesis time")
return
}
time.Sleep(1 * time.Second)
}
s.processAttestations(s.ctx)
if err := s.updateHead(s.ctx, s.getJustifiedBalances()); err != nil {
log.Warnf("Resolving fork due to new attestation: %v", err)
log.Warn("Genesis time received, now available to process attestations")
}
st := slots.NewSlotTicker(s.genesisTime, params.BeaconConfig().SecondsPerSlot)
for {
select {
case <-s.ctx.Done():
return
case <-st.C():
if err := s.NewSlot(s.ctx, s.CurrentSlot()); err != nil {
log.WithError(err).Error("Could not process new slot")
return
}
// Continue when there's no fork choice attestation, there's nothing to process and update head.
// This covers the condition when the node is still initial syncing to the head of the chain.
if s.cfg.AttPool.ForkchoiceAttestationCount() == 0 {
continue
}
s.processAttestations(s.ctx)
justified := s.store.JustifiedCheckpt()
if justified == nil {
log.WithError(errNilJustifiedInStore).Error("Could not get justified checkpoint")
continue
}
balances, err := s.justifiedBalances.get(s.ctx, bytesutil.ToBytes32(justified.Root))
if err != nil {
log.WithError(err).Errorf("Unable to get justified balances for root %v", justified.Root)
continue
}
prevHead := s.headRoot()
if err := s.updateHead(s.ctx, balances); err != nil {
log.WithError(err).Warn("Resolving fork due to new attestation")
}
s.notifyEngineIfChangedHead(prevHead)
}
}
}()
}
// This calls notify Forkchoice Update in the event that the head has changed
func (s *Service) notifyEngineIfChangedHead(prevHead [32]byte) {
if s.headRoot() == prevHead {
return
}
finalized := s.store.FinalizedCheckpt()
if finalized == nil {
log.WithError(errNilFinalizedInStore).Error("could not get finalized checkpoint")
return
}
_, err := s.notifyForkchoiceUpdate(s.ctx,
s.headBlock().Block(),
s.headRoot(),
bytesutil.ToBytes32(finalized.Root),
)
if err != nil {
log.WithError(err).Error("could not notify forkchoice update")
}
}

View File

@@ -9,9 +9,7 @@ import (
"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
"github.com/prysmaticlabs/prysm/beacon-chain/core/transition"
testDB "github.com/prysmaticlabs/prysm/beacon-chain/db/testing"
"github.com/prysmaticlabs/prysm/beacon-chain/forkchoice/protoarray"
"github.com/prysmaticlabs/prysm/beacon-chain/operations/attestations"
"github.com/prysmaticlabs/prysm/beacon-chain/state/stategen"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
@@ -42,21 +40,24 @@ func TestAttestationCheckPtState_FarFutureSlot(t *testing.T) {
func TestVerifyLMDFFGConsistent_NotOK(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
opts := testServiceOptsWithDB(t)
cfg := &Config{BeaconDB: beaconDB, ForkChoiceStore: protoarray.New(0, 0, [32]byte{})}
service, err := NewService(ctx, cfg)
service, err := NewService(ctx, opts...)
require.NoError(t, err)
b32 := util.NewBeaconBlock()
b32.Block.Slot = 32
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(b32)))
wsb, err := wrapper.WrappedSignedBeaconBlock(b32)
require.NoError(t, err)
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wsb))
r32, err := b32.Block.HashTreeRoot()
require.NoError(t, err)
b33 := util.NewBeaconBlock()
b33.Block.Slot = 33
b33.Block.ParentRoot = r32[:]
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(b33)))
wsb, err = wrapper.WrappedSignedBeaconBlock(b33)
require.NoError(t, err)
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wsb))
r33, err := b33.Block.HashTreeRoot()
require.NoError(t, err)
@@ -70,21 +71,24 @@ func TestVerifyLMDFFGConsistent_NotOK(t *testing.T) {
func TestVerifyLMDFFGConsistent_OK(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
cfg := &Config{BeaconDB: beaconDB, ForkChoiceStore: protoarray.New(0, 0, [32]byte{})}
service, err := NewService(ctx, cfg)
opts := testServiceOptsWithDB(t)
service, err := NewService(ctx, opts...)
require.NoError(t, err)
b32 := util.NewBeaconBlock()
b32.Block.Slot = 32
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(b32)))
wsb, err := wrapper.WrappedSignedBeaconBlock(b32)
require.NoError(t, err)
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wsb))
r32, err := b32.Block.HashTreeRoot()
require.NoError(t, err)
b33 := util.NewBeaconBlock()
b33.Block.Slot = 33
b33.Block.ParentRoot = r32[:]
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(b33)))
wsb, err = wrapper.WrappedSignedBeaconBlock(b33)
require.NoError(t, err)
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wsb))
r33, err := b33.Block.HashTreeRoot()
require.NoError(t, err)
@@ -99,17 +103,12 @@ func TestVerifyLMDFFGConsistent_OK(t *testing.T) {
func TestProcessAttestations_Ok(t *testing.T) {
hook := logTest.NewGlobal()
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
opts := testServiceOptsWithDB(t)
opts = append(opts, WithAttestationPool(attestations.NewPool()))
cfg := &Config{
BeaconDB: beaconDB,
ForkChoiceStore: protoarray.New(0, 0, [32]byte{}),
StateGen: stategen.New(beaconDB),
AttPool: attestations.NewPool(),
}
service, err := NewService(ctx, cfg)
service.genesisTime = prysmTime.Now().Add(-1 * time.Duration(params.BeaconConfig().SecondsPerSlot) * time.Second)
service, err := NewService(ctx, opts...)
require.NoError(t, err)
service.genesisTime = prysmTime.Now().Add(-1 * time.Duration(params.BeaconConfig().SecondsPerSlot) * time.Second)
genesisState, pks := util.DeterministicGenesisState(t, 64)
require.NoError(t, genesisState.SetGenesisTime(uint64(prysmTime.Now().Unix())-params.BeaconConfig().SecondsPerSlot))
require.NoError(t, service.saveGenesisData(ctx, genesisState))
@@ -120,9 +119,45 @@ func TestProcessAttestations_Ok(t *testing.T) {
copied, err = transition.ProcessSlots(ctx, copied, 1)
require.NoError(t, err)
require.NoError(t, service.cfg.BeaconDB.SaveState(ctx, copied, tRoot))
require.NoError(t, service.cfg.ForkChoiceStore.ProcessBlock(ctx, 0, tRoot, tRoot, tRoot, 1, 1))
require.NoError(t, service.cfg.ForkChoiceStore.InsertOptimisticBlock(ctx, 0, tRoot, tRoot, params.BeaconConfig().ZeroHash, 1, 1))
require.NoError(t, service.cfg.AttPool.SaveForkchoiceAttestations(atts))
service.processAttestations(ctx)
require.Equal(t, 0, len(service.cfg.AttPool.ForkchoiceAttestations()))
require.LogsDoNotContain(t, hook, "Could not process attestation for fork choice")
}
func TestNotifyEngineIfChangedHead(t *testing.T) {
hook := logTest.NewGlobal()
ctx := context.Background()
opts := testServiceOptsWithDB(t)
service, err := NewService(ctx, opts...)
require.NoError(t, err)
service.notifyEngineIfChangedHead(service.headRoot())
hookErr := "could not notify forkchoice update"
finalizedErr := "could not get finalized checkpoint"
require.LogsDoNotContain(t, hook, finalizedErr)
require.LogsDoNotContain(t, hook, hookErr)
service.notifyEngineIfChangedHead([32]byte{'a'})
require.LogsContain(t, hook, finalizedErr)
hook.Reset()
b := util.NewBeaconBlock()
b.Block.Slot = 1
wsb, err := wrapper.WrappedSignedBeaconBlock(b)
require.NoError(t, err)
require.NoError(t, service.cfg.BeaconDB.SaveBlock(ctx, wsb))
r, err := b.Block.HashTreeRoot()
require.NoError(t, err)
finalized := &ethpb.Checkpoint{Root: r[:], Epoch: 0}
service.head = &head{
slot: 1,
root: r,
block: wsb,
}
service.store.SetFinalizedCheckpt(finalized)
service.notifyEngineIfChangedHead([32]byte{'b'})
require.LogsDoNotContain(t, hook, finalizedErr)
require.LogsDoNotContain(t, hook, hookErr)
}

View File

@@ -53,14 +53,20 @@ func (s *Service) ReceiveBlock(ctx context.Context, block block.SignedBeaconBloc
}
// Reports on block and fork choice metrics.
reportSlotMetrics(blockCopy.Block().Slot(), s.HeadSlot(), s.CurrentSlot(), s.finalizedCheckpt)
finalized := s.store.FinalizedCheckpt()
if finalized == nil {
return errNilFinalizedInStore
}
reportSlotMetrics(blockCopy.Block().Slot(), s.HeadSlot(), s.CurrentSlot(), finalized)
// Log block sync status.
if err := logBlockSyncStatus(blockCopy.Block(), blockRoot, s.finalizedCheckpt, receivedTime, uint64(s.genesisTime.Unix())); err != nil {
if err := logBlockSyncStatus(blockCopy.Block(), blockRoot, finalized, receivedTime, uint64(s.genesisTime.Unix())); err != nil {
return err
}
// Log state transition data.
logStateTransitionData(blockCopy.Block())
if err := logStateTransitionData(blockCopy.Block()); err != nil {
return err
}
return nil
}
@@ -98,10 +104,21 @@ func (s *Service) ReceiveBlockBatch(ctx context.Context, blocks []block.SignedBe
})
// Reports on blockCopy and fork choice metrics.
reportSlotMetrics(blockCopy.Block().Slot(), s.HeadSlot(), s.CurrentSlot(), s.finalizedCheckpt)
finalized := s.store.FinalizedCheckpt()
if finalized == nil {
return errNilFinalizedInStore
}
reportSlotMetrics(blockCopy.Block().Slot(), s.HeadSlot(), s.CurrentSlot(), finalized)
}
if err := s.VerifyWeakSubjectivityRoot(s.ctx); err != nil {
if err := s.cfg.BeaconDB.SaveBlocks(ctx, s.getInitSyncBlocks()); err != nil {
return err
}
finalized := s.store.FinalizedCheckpt()
if finalized == nil {
return errNilFinalizedInStore
}
if err := s.wsVerifier.VerifyWeakSubjectivity(s.ctx, finalized.Epoch); err != nil {
// log.Fatalf will prevent defer from being called
span.End()
// Exit run time if the node failed to verify weak subjectivity checkpoint.
@@ -145,8 +162,12 @@ func (s *Service) checkSaveHotStateDB(ctx context.Context) error {
currentEpoch := slots.ToEpoch(s.CurrentSlot())
// Prevent `sinceFinality` going underflow.
var sinceFinality types.Epoch
if currentEpoch > s.finalizedCheckpt.Epoch {
sinceFinality = currentEpoch - s.finalizedCheckpt.Epoch
finalized := s.store.FinalizedCheckpt()
if finalized == nil {
return errNilFinalizedInStore
}
if currentEpoch > finalized.Epoch {
sinceFinality = currentEpoch - finalized.Epoch
}
if sinceFinality >= epochsSinceFinalitySaveHotStateDB {

View File

@@ -33,6 +33,7 @@ func TestService_ReceiveBlock(t *testing.T) {
assert.NoError(t, err)
return blk
}
params.SetupTestConfigCleanup(t)
bc := params.BeaconConfig()
bc.ShardCommitteePeriod = 0 // Required for voluntary exits test in reasonable time.
params.OverrideBeaconConfig(bc)
@@ -124,29 +125,28 @@ func TestService_ReceiveBlock(t *testing.T) {
genesisBlockRoot := bytesutil.ToBytes32(nil)
require.NoError(t, beaconDB.SaveState(ctx, genesis, genesisBlockRoot))
cfg := &Config{
BeaconDB: beaconDB,
ForkChoiceStore: protoarray.New(
0, // justifiedEpoch
0, // finalizedEpoch
genesisBlockRoot,
),
AttPool: attestations.NewPool(),
ExitPool: voluntaryexits.NewPool(),
StateNotifier: &blockchainTesting.MockStateNotifier{RecordEvents: true},
StateGen: stategen.New(beaconDB),
opts := []Option{
WithDatabase(beaconDB),
WithForkChoiceStore(protoarray.New(0, 0, genesisBlockRoot)),
WithAttestationPool(attestations.NewPool()),
WithExitPool(voluntaryexits.NewPool()),
WithStateNotifier(&blockchainTesting.MockStateNotifier{RecordEvents: true}),
WithStateGen(stategen.New(beaconDB)),
WithFinalizedStateAtStartUp(genesis),
}
s, err := NewService(ctx, cfg)
s, err := NewService(ctx, opts...)
require.NoError(t, err)
require.NoError(t, s.saveGenesisData(ctx, genesis))
gBlk, err := s.cfg.BeaconDB.GenesisBlock(ctx)
require.NoError(t, err)
gRoot, err := gBlk.Block().HashTreeRoot()
require.NoError(t, err)
s.finalizedCheckpt = &ethpb.Checkpoint{Root: gRoot[:]}
s.store.SetFinalizedCheckpt(&ethpb.Checkpoint{Root: gRoot[:]})
root, err := tt.args.block.Block.HashTreeRoot()
require.NoError(t, err)
err = s.ReceiveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(tt.args.block), root)
wsb, err := wrapper.WrappedSignedBeaconBlock(tt.args.block)
require.NoError(t, err)
err = s.ReceiveBlock(ctx, wsb, root)
if tt.wantedErr != "" {
assert.ErrorContains(t, tt.wantedErr, err)
} else {
@@ -165,32 +165,31 @@ func TestService_ReceiveBlockUpdateHead(t *testing.T) {
beaconDB := testDB.SetupDB(t)
genesisBlockRoot := bytesutil.ToBytes32(nil)
require.NoError(t, beaconDB.SaveState(ctx, genesis, genesisBlockRoot))
cfg := &Config{
BeaconDB: beaconDB,
ForkChoiceStore: protoarray.New(
0, // justifiedEpoch
0, // finalizedEpoch
genesisBlockRoot,
),
AttPool: attestations.NewPool(),
ExitPool: voluntaryexits.NewPool(),
StateNotifier: &blockchainTesting.MockStateNotifier{RecordEvents: true},
StateGen: stategen.New(beaconDB),
opts := []Option{
WithDatabase(beaconDB),
WithForkChoiceStore(protoarray.New(0, 0, genesisBlockRoot)),
WithAttestationPool(attestations.NewPool()),
WithExitPool(voluntaryexits.NewPool()),
WithStateNotifier(&blockchainTesting.MockStateNotifier{RecordEvents: true}),
WithStateGen(stategen.New(beaconDB)),
}
s, err := NewService(ctx, cfg)
s, err := NewService(ctx, opts...)
require.NoError(t, err)
require.NoError(t, s.saveGenesisData(ctx, genesis))
gBlk, err := s.cfg.BeaconDB.GenesisBlock(ctx)
require.NoError(t, err)
gRoot, err := gBlk.Block().HashTreeRoot()
require.NoError(t, err)
s.finalizedCheckpt = &ethpb.Checkpoint{Root: gRoot[:]}
s.store.SetFinalizedCheckpt(&ethpb.Checkpoint{Root: gRoot[:]})
root, err := b.Block.HashTreeRoot()
require.NoError(t, err)
wg := sync.WaitGroup{}
wg.Add(1)
go func() {
require.NoError(t, s.ReceiveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(b), root))
wsb, err := wrapper.WrappedSignedBeaconBlock(b)
require.NoError(t, err)
require.NoError(t, s.ReceiveBlock(ctx, wsb, root))
wg.Done()
}()
wg.Wait()
@@ -198,7 +197,7 @@ func TestService_ReceiveBlockUpdateHead(t *testing.T) {
t.Errorf("Received %d state notifications, expected at least 1", recvd)
}
// Verify fork choice has processed the block. (Genesis block and the new block)
assert.Equal(t, 2, len(s.cfg.ForkChoiceStore.Nodes()))
assert.Equal(t, 2, s.cfg.ForkChoiceStore.NodeCount())
}
func TestService_ReceiveBlockBatch(t *testing.T) {
@@ -248,17 +247,13 @@ func TestService_ReceiveBlockBatch(t *testing.T) {
beaconDB := testDB.SetupDB(t)
genesisBlockRoot, err := genesis.HashTreeRoot(ctx)
require.NoError(t, err)
cfg := &Config{
BeaconDB: beaconDB,
ForkChoiceStore: protoarray.New(
0, // justifiedEpoch
0, // finalizedEpoch
genesisBlockRoot,
),
StateNotifier: &blockchainTesting.MockStateNotifier{RecordEvents: true},
StateGen: stategen.New(beaconDB),
opts := []Option{
WithDatabase(beaconDB),
WithForkChoiceStore(protoarray.New(0, 0, genesisBlockRoot)),
WithStateNotifier(&blockchainTesting.MockStateNotifier{RecordEvents: true}),
WithStateGen(stategen.New(beaconDB)),
}
s, err := NewService(ctx, cfg)
s, err := NewService(ctx, opts...)
require.NoError(t, err)
err = s.saveGenesisData(ctx, genesis)
require.NoError(t, err)
@@ -267,10 +262,12 @@ func TestService_ReceiveBlockBatch(t *testing.T) {
gRoot, err := gBlk.Block().HashTreeRoot()
require.NoError(t, err)
s.finalizedCheckpt = &ethpb.Checkpoint{Root: gRoot[:]}
s.store.SetFinalizedCheckpt(&ethpb.Checkpoint{Root: gRoot[:]})
root, err := tt.args.block.Block.HashTreeRoot()
require.NoError(t, err)
blks := []block.SignedBeaconBlock{wrapper.WrappedPhase0SignedBeaconBlock(tt.args.block)}
wsb, err := wrapper.WrappedSignedBeaconBlock(tt.args.block)
require.NoError(t, err)
blks := []block.SignedBeaconBlock{wsb}
roots := [][32]byte{root}
err = s.ReceiveBlockBatch(ctx, blks, roots)
if tt.wantedErr != "" {
@@ -284,37 +281,41 @@ func TestService_ReceiveBlockBatch(t *testing.T) {
}
func TestService_HasInitSyncBlock(t *testing.T) {
s, err := NewService(context.Background(), &Config{StateNotifier: &blockchainTesting.MockStateNotifier{}})
opts := testServiceOptsNoDB()
opts = append(opts, WithStateNotifier(&blockchainTesting.MockStateNotifier{}))
s, err := NewService(context.Background(), opts...)
require.NoError(t, err)
r := [32]byte{'a'}
if s.HasInitSyncBlock(r) {
t.Error("Should not have block")
}
s.saveInitSyncBlock(r, wrapper.WrappedPhase0SignedBeaconBlock(util.NewBeaconBlock()))
wsb, err := wrapper.WrappedSignedBeaconBlock(util.NewBeaconBlock())
require.NoError(t, err)
s.saveInitSyncBlock(r, wsb)
if !s.HasInitSyncBlock(r) {
t.Error("Should have block")
}
}
func TestCheckSaveHotStateDB_Enabling(t *testing.T) {
beaconDB := testDB.SetupDB(t)
opts := testServiceOptsWithDB(t)
hook := logTest.NewGlobal()
s, err := NewService(context.Background(), &Config{StateGen: stategen.New(beaconDB)})
s, err := NewService(context.Background(), opts...)
require.NoError(t, err)
st := params.BeaconConfig().SlotsPerEpoch.Mul(uint64(epochsSinceFinalitySaveHotStateDB))
s.genesisTime = time.Now().Add(time.Duration(-1*int64(st)*int64(params.BeaconConfig().SecondsPerSlot)) * time.Second)
s.finalizedCheckpt = &ethpb.Checkpoint{}
s.store.SetFinalizedCheckpt(&ethpb.Checkpoint{})
require.NoError(t, s.checkSaveHotStateDB(context.Background()))
assert.LogsContain(t, hook, "Entering mode to save hot states in DB")
}
func TestCheckSaveHotStateDB_Disabling(t *testing.T) {
beaconDB := testDB.SetupDB(t)
hook := logTest.NewGlobal()
s, err := NewService(context.Background(), &Config{StateGen: stategen.New(beaconDB)})
opts := testServiceOptsWithDB(t)
s, err := NewService(context.Background(), opts...)
require.NoError(t, err)
s.finalizedCheckpt = &ethpb.Checkpoint{}
s.store.SetFinalizedCheckpt(&ethpb.Checkpoint{})
require.NoError(t, s.checkSaveHotStateDB(context.Background()))
s.genesisTime = time.Now()
@@ -323,11 +324,11 @@ func TestCheckSaveHotStateDB_Disabling(t *testing.T) {
}
func TestCheckSaveHotStateDB_Overflow(t *testing.T) {
beaconDB := testDB.SetupDB(t)
hook := logTest.NewGlobal()
s, err := NewService(context.Background(), &Config{StateGen: stategen.New(beaconDB)})
opts := testServiceOptsWithDB(t)
s, err := NewService(context.Background(), opts...)
require.NoError(t, err)
s.finalizedCheckpt = &ethpb.Checkpoint{Epoch: 10000000}
s.store.SetFinalizedCheckpt(&ethpb.Checkpoint{Epoch: 10000000})
s.genesisTime = time.Now()
require.NoError(t, s.checkSaveHotStateDB(context.Background()))

View File

@@ -3,6 +3,7 @@
package blockchain
import (
"bytes"
"context"
"fmt"
"runtime"
@@ -12,6 +13,7 @@ import (
"github.com/pkg/errors"
types "github.com/prysmaticlabs/eth2-types"
"github.com/prysmaticlabs/prysm/async/event"
"github.com/prysmaticlabs/prysm/beacon-chain/blockchain/store"
"github.com/prysmaticlabs/prysm/beacon-chain/cache"
"github.com/prysmaticlabs/prysm/beacon-chain/cache/depositcache"
"github.com/prysmaticlabs/prysm/beacon-chain/core/feed"
@@ -20,19 +22,23 @@ import (
"github.com/prysmaticlabs/prysm/beacon-chain/core/transition"
"github.com/prysmaticlabs/prysm/beacon-chain/db"
f "github.com/prysmaticlabs/prysm/beacon-chain/forkchoice"
doublylinkedtree "github.com/prysmaticlabs/prysm/beacon-chain/forkchoice/doubly-linked-tree"
"github.com/prysmaticlabs/prysm/beacon-chain/forkchoice/protoarray"
"github.com/prysmaticlabs/prysm/beacon-chain/operations/attestations"
"github.com/prysmaticlabs/prysm/beacon-chain/operations/slashings"
"github.com/prysmaticlabs/prysm/beacon-chain/operations/voluntaryexits"
"github.com/prysmaticlabs/prysm/beacon-chain/p2p"
"github.com/prysmaticlabs/prysm/beacon-chain/powchain"
enginev1 "github.com/prysmaticlabs/prysm/beacon-chain/powchain/engine-api-client/v1"
"github.com/prysmaticlabs/prysm/beacon-chain/state"
"github.com/prysmaticlabs/prysm/beacon-chain/state/stategen"
"github.com/prysmaticlabs/prysm/cmd/beacon-chain/flags"
"github.com/prysmaticlabs/prysm/config/features"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/block"
prysmTime "github.com/prysmaticlabs/prysm/time"
"github.com/prysmaticlabs/prysm/time/slots"
"github.com/sirupsen/logrus"
"go.opencensus.io/trace"
@@ -45,30 +51,26 @@ const headSyncMinEpochsAfterCheckpoint = 128
// Service represents a service that handles the internal
// logic of managing the full PoS beacon chain.
type Service struct {
cfg *Config
ctx context.Context
cancel context.CancelFunc
genesisTime time.Time
head *head
headLock sync.RWMutex
genesisRoot [32]byte
justifiedCheckpt *ethpb.Checkpoint
prevJustifiedCheckpt *ethpb.Checkpoint
bestJustifiedCheckpt *ethpb.Checkpoint
finalizedCheckpt *ethpb.Checkpoint
prevFinalizedCheckpt *ethpb.Checkpoint
cfg *config
ctx context.Context
cancel context.CancelFunc
genesisTime time.Time
head *head
headLock sync.RWMutex
// originBlockRoot is the genesis root, or weak subjectivity checkpoint root, depending on how the node is initialized
originBlockRoot [32]byte
nextEpochBoundarySlot types.Slot
boundaryRoots [][32]byte
checkpointStateCache *cache.CheckpointStateCache
initSyncBlocks map[[32]byte]block.SignedBeaconBlock
initSyncBlocksLock sync.RWMutex
justifiedBalances []uint64
justifiedBalancesLock sync.RWMutex
wsVerified bool
justifiedBalances *stateBalanceCache
wsVerifier *WeakSubjectivityVerifier
store *store.Store
}
// Config options for the service.
type Config struct {
// config options for the service.
type config struct {
BeaconBlockBuf int
ChainStartFetcher powchain.ChainStartFetcher
BeaconDB db.HeadAccessDatabase
@@ -84,165 +86,309 @@ type Config struct {
StateGen *stategen.State
SlasherAttestationsFeed *event.Feed
WeakSubjectivityCheckpt *ethpb.Checkpoint
BlockFetcher powchain.POWBlockFetcher
FinalizedStateAtStartUp state.BeaconState
ExecutionEngineCaller enginev1.Caller
}
// NewService instantiates a new block service instance that will
// be registered into a running beacon node.
func NewService(ctx context.Context, cfg *Config) (*Service, error) {
func NewService(ctx context.Context, opts ...Option) (*Service, error) {
ctx, cancel := context.WithCancel(ctx)
return &Service{
cfg: cfg,
srv := &Service{
ctx: ctx,
cancel: cancel,
boundaryRoots: [][32]byte{},
checkpointStateCache: cache.NewCheckpointStateCache(),
initSyncBlocks: make(map[[32]byte]block.SignedBeaconBlock),
justifiedBalances: make([]uint64, 0),
}, nil
cfg: &config{},
store: &store.Store{},
}
for _, opt := range opts {
if err := opt(srv); err != nil {
return nil, err
}
}
var err error
if srv.justifiedBalances == nil {
srv.justifiedBalances, err = newStateBalanceCache(srv.cfg.StateGen)
if err != nil {
return nil, err
}
}
srv.wsVerifier, err = NewWeakSubjectivityVerifier(srv.cfg.WeakSubjectivityCheckpt, srv.cfg.BeaconDB)
if err != nil {
return nil, err
}
return srv, nil
}
// Start a blockchain service's main event loop.
func (s *Service) Start() {
// For running initial sync with state cache, in an event of restart, we use
// last finalized check point as start point to sync instead of head
// state. This is because we no longer save state every slot during sync.
cp, err := s.cfg.BeaconDB.FinalizedCheckpoint(s.ctx)
if err != nil {
log.Fatalf("Could not fetch finalized cp: %v", err)
}
saved := s.cfg.FinalizedStateAtStartUp
r := bytesutil.ToBytes32(cp.Root)
// Before the first finalized epoch, in the current epoch,
// the finalized root is defined as zero hashes instead of genesis root hash.
// We want to use genesis root to retrieve for state.
if r == params.BeaconConfig().ZeroHash {
genesisBlock, err := s.cfg.BeaconDB.GenesisBlock(s.ctx)
if err != nil {
log.Fatalf("Could not fetch finalized cp: %v", err)
if saved != nil && !saved.IsNil() {
if err := s.StartFromSavedState(saved); err != nil {
log.Fatal(err)
}
if genesisBlock != nil && !genesisBlock.IsNil() {
r, err = genesisBlock.Block().HashTreeRoot()
if err != nil {
log.Fatalf("Could not tree hash genesis block: %v", err)
}
}
}
beaconState, err := s.cfg.StateGen.StateByRoot(s.ctx, r)
if err != nil {
log.Fatalf("Could not fetch beacon state by root: %v", err)
}
// Make sure that attestation processor is subscribed and ready for state initializing event.
attestationProcessorSubscribed := make(chan struct{}, 1)
// If the chain has already been initialized, simply start the block processing routine.
if beaconState != nil && !beaconState.IsNil() {
log.Info("Blockchain data already exists in DB, initializing...")
s.genesisTime = time.Unix(int64(beaconState.GenesisTime()), 0)
s.cfg.AttService.SetGenesisTime(beaconState.GenesisTime())
if err := s.initializeChainInfo(s.ctx); err != nil {
log.Fatalf("Could not set up chain info: %v", err)
}
// We start a counter to genesis, if needed.
gState, err := s.cfg.BeaconDB.GenesisState(s.ctx)
if err != nil {
log.Fatalf("Could not retrieve genesis state: %v", err)
}
gRoot, err := gState.HashTreeRoot(s.ctx)
if err != nil {
log.Fatalf("Could not hash tree root genesis state: %v", err)
}
go slots.CountdownToGenesis(s.ctx, s.genesisTime, uint64(gState.NumValidators()), gRoot)
justifiedCheckpoint, err := s.cfg.BeaconDB.JustifiedCheckpoint(s.ctx)
if err != nil {
log.Fatalf("Could not get justified checkpoint: %v", err)
}
finalizedCheckpoint, err := s.cfg.BeaconDB.FinalizedCheckpoint(s.ctx)
if err != nil {
log.Fatalf("Could not get finalized checkpoint: %v", err)
}
// Resume fork choice.
s.justifiedCheckpt = ethpb.CopyCheckpoint(justifiedCheckpoint)
if err := s.cacheJustifiedStateBalances(s.ctx, s.ensureRootNotZeros(bytesutil.ToBytes32(s.justifiedCheckpt.Root))); err != nil {
log.Fatalf("Could not cache justified state balances: %v", err)
}
s.prevJustifiedCheckpt = ethpb.CopyCheckpoint(justifiedCheckpoint)
s.bestJustifiedCheckpt = ethpb.CopyCheckpoint(justifiedCheckpoint)
s.finalizedCheckpt = ethpb.CopyCheckpoint(finalizedCheckpoint)
s.prevFinalizedCheckpt = ethpb.CopyCheckpoint(finalizedCheckpoint)
s.resumeForkChoice(justifiedCheckpoint, finalizedCheckpoint)
ss, err := slots.EpochStart(s.finalizedCheckpt.Epoch)
if err != nil {
log.Fatalf("Could not get start slot of finalized epoch: %v", err)
}
h := s.headBlock().Block()
if h.Slot() > ss {
log.WithFields(logrus.Fields{
"startSlot": ss,
"endSlot": h.Slot(),
}).Info("Loading blocks to fork choice store, this may take a while.")
if err := s.fillInForkChoiceMissingBlocks(s.ctx, h, s.finalizedCheckpt, s.justifiedCheckpt); err != nil {
log.Fatalf("Could not fill in fork choice store missing blocks: %v", err)
}
}
if err := s.VerifyWeakSubjectivityRoot(s.ctx); err != nil {
// Exit run time if the node failed to verify weak subjectivity checkpoint.
log.Fatalf("Could not verify weak subjectivity checkpoint: %v", err)
}
s.cfg.StateNotifier.StateFeed().Send(&feed.Event{
Type: statefeed.Initialized,
Data: &statefeed.InitializedData{
StartTime: s.genesisTime,
GenesisValidatorsRoot: beaconState.GenesisValidatorRoot(),
},
})
} else {
log.Info("Waiting to reach the validator deposit threshold to start the beacon chain...")
if s.cfg.ChainStartFetcher == nil {
log.Fatal("Not configured web3Service for POW chain")
return // return need for TestStartUninitializedChainWithoutConfigPOWChain.
if err := s.startFromPOWChain(); err != nil {
log.Fatal(err)
}
go func() {
stateChannel := make(chan *feed.Event, 1)
stateSub := s.cfg.StateNotifier.StateFeed().Subscribe(stateChannel)
defer stateSub.Unsubscribe()
<-attestationProcessorSubscribed
for {
select {
case event := <-stateChannel:
if event.Type == statefeed.ChainStarted {
data, ok := event.Data.(*statefeed.ChainStartedData)
if !ok {
log.Error("event data is not type *statefeed.ChainStartedData")
return
}
log.WithField("starttime", data.StartTime).Debug("Received chain start event")
s.processChainStartTime(s.ctx, data.StartTime)
return
}
case <-s.ctx.Done():
log.Debug("Context closed, exiting goroutine")
return
case err := <-stateSub.Err():
log.WithError(err).Error("Subscription to state notifier failed")
return
}
}
}()
}
go s.processAttestationsRoutine(attestationProcessorSubscribed)
s.spawnProcessAttestationsRoutine(s.cfg.StateNotifier.StateFeed())
}
// processChainStartTime initializes a series of deposits from the ChainStart deposits in the eth1
// Stop the blockchain service's main event loop and associated goroutines.
func (s *Service) Stop() error {
defer s.cancel()
if s.cfg.StateGen != nil && s.head != nil && s.head.state != nil {
if err := s.cfg.StateGen.ForceCheckpoint(s.ctx, s.head.state.FinalizedCheckpoint().Root); err != nil {
return err
}
}
// Save initial sync cached blocks to the DB before stop.
return s.cfg.BeaconDB.SaveBlocks(s.ctx, s.getInitSyncBlocks())
}
// Status always returns nil unless there is an error condition that causes
// this service to be unhealthy.
func (s *Service) Status() error {
if s.originBlockRoot == params.BeaconConfig().ZeroHash {
return errors.New("genesis state has not been created")
}
if runtime.NumGoroutine() > s.cfg.MaxRoutines {
return fmt.Errorf("too many goroutines %d", runtime.NumGoroutine())
}
return nil
}
func (s *Service) StartFromSavedState(saved state.BeaconState) error {
log.Info("Blockchain data already exists in DB, initializing...")
s.genesisTime = time.Unix(int64(saved.GenesisTime()), 0) // lint:ignore uintcast -- Genesis time will not exceed int64 in your lifetime.
s.cfg.AttService.SetGenesisTime(saved.GenesisTime())
originRoot, err := s.originRootFromSavedState(s.ctx)
if err != nil {
return err
}
s.originBlockRoot = originRoot
if err := s.initializeHeadFromDB(s.ctx); err != nil {
return errors.Wrap(err, "could not set up chain info")
}
spawnCountdownIfPreGenesis(s.ctx, s.genesisTime, s.cfg.BeaconDB)
justified, err := s.cfg.BeaconDB.JustifiedCheckpoint(s.ctx)
if err != nil {
return errors.Wrap(err, "could not get justified checkpoint")
}
finalized, err := s.cfg.BeaconDB.FinalizedCheckpoint(s.ctx)
if err != nil {
return errors.Wrap(err, "could not get finalized checkpoint")
}
s.store = store.New(justified, finalized)
var store f.ForkChoicer
fRoot := bytesutil.ToBytes32(finalized.Root)
if features.Get().EnableForkChoiceDoublyLinkedTree {
store = doublylinkedtree.New(justified.Epoch, finalized.Epoch)
} else {
store = protoarray.New(justified.Epoch, finalized.Epoch, fRoot)
}
s.cfg.ForkChoiceStore = store
fb, err := s.cfg.BeaconDB.Block(s.ctx, s.ensureRootNotZeros(fRoot))
if err != nil {
return errors.Wrap(err, "could not get finalized checkpoint block")
}
if fb == nil {
return errNilFinalizedInStore
}
payloadHash, err := getBlockPayloadHash(fb.Block())
if err != nil {
return errors.Wrap(err, "could not get execution payload hash")
}
fSlot := fb.Block().Slot()
if err := store.InsertOptimisticBlock(s.ctx, fSlot, fRoot, params.BeaconConfig().ZeroHash,
payloadHash, justified.Epoch, finalized.Epoch); err != nil {
return errors.Wrap(err, "could not insert finalized block to forkchoice")
}
lastValidatedCheckpoint, err := s.cfg.BeaconDB.LastValidatedCheckpoint(s.ctx)
if err != nil {
return errors.Wrap(err, "could not get last validated checkpoint")
}
if bytes.Equal(finalized.Root, lastValidatedCheckpoint.Root) {
if err := store.SetOptimisticToValid(s.ctx, fRoot); err != nil {
return errors.Wrap(err, "could not set finalized block as validated")
}
}
h := s.headBlock().Block()
if h.Slot() > fSlot {
log.WithFields(logrus.Fields{
"startSlot": fSlot,
"endSlot": h.Slot(),
}).Info("Loading blocks to fork choice store, this may take a while.")
if err := s.fillInForkChoiceMissingBlocks(s.ctx, h, finalized, justified); err != nil {
return errors.Wrap(err, "could not fill in fork choice store missing blocks")
}
}
// not attempting to save initial sync blocks here, because there shouldn't be any until
// after the statefeed.Initialized event is fired (below)
if err := s.wsVerifier.VerifyWeakSubjectivity(s.ctx, finalized.Epoch); err != nil {
// Exit run time if the node failed to verify weak subjectivity checkpoint.
return errors.Wrap(err, "could not verify initial checkpoint provided for chain sync")
}
s.cfg.StateNotifier.StateFeed().Send(&feed.Event{
Type: statefeed.Initialized,
Data: &statefeed.InitializedData{
StartTime: s.genesisTime,
GenesisValidatorsRoot: saved.GenesisValidatorsRoot(),
},
})
return nil
}
func (s *Service) originRootFromSavedState(ctx context.Context) ([32]byte, error) {
// first check if we have started from checkpoint sync and have a root
originRoot, err := s.cfg.BeaconDB.OriginCheckpointBlockRoot(ctx)
if err == nil {
return originRoot, nil
}
if !errors.Is(err, db.ErrNotFound) {
return originRoot, errors.Wrap(err, "could not retrieve checkpoint sync chain origin data from db")
}
// we got here because OriginCheckpointBlockRoot gave us an ErrNotFound. this means the node was started from a genesis state,
// so we should have a value for GenesisBlock
genesisBlock, err := s.cfg.BeaconDB.GenesisBlock(ctx)
if err != nil {
return originRoot, errors.Wrap(err, "could not get genesis block from db")
}
if err := helpers.BeaconBlockIsNil(genesisBlock); err != nil {
return originRoot, err
}
genesisBlkRoot, err := genesisBlock.Block().HashTreeRoot()
if err != nil {
return genesisBlkRoot, errors.Wrap(err, "could not get signing root of genesis block")
}
return genesisBlkRoot, nil
}
// initializeHeadFromDB uses the finalized checkpoint and head block found in the database to set the current head
// note that this may block until stategen replays blocks between the finalized and head blocks
// if the head sync flag was specified and the gap between the finalized and head blocks is at least 128 epochs long
func (s *Service) initializeHeadFromDB(ctx context.Context) error {
finalized, err := s.cfg.BeaconDB.FinalizedCheckpoint(ctx)
if err != nil {
return errors.Wrap(err, "could not get finalized checkpoint from db")
}
if finalized == nil {
// This should never happen. At chain start, the finalized checkpoint
// would be the genesis state and block.
return errors.New("no finalized epoch in the database")
}
finalizedRoot := s.ensureRootNotZeros(bytesutil.ToBytes32(finalized.Root))
var finalizedState state.BeaconState
finalizedState, err = s.cfg.StateGen.Resume(ctx, s.cfg.FinalizedStateAtStartUp)
if err != nil {
return errors.Wrap(err, "could not get finalized state from db")
}
if flags.Get().HeadSync {
headBlock, err := s.cfg.BeaconDB.HeadBlock(ctx)
if err != nil {
return errors.Wrap(err, "could not retrieve head block")
}
headEpoch := slots.ToEpoch(headBlock.Block().Slot())
var epochsSinceFinality types.Epoch
if headEpoch > finalized.Epoch {
epochsSinceFinality = headEpoch - finalized.Epoch
}
// Head sync when node is far enough beyond known finalized epoch,
// this becomes really useful during long period of non-finality.
if epochsSinceFinality >= headSyncMinEpochsAfterCheckpoint {
headRoot, err := headBlock.Block().HashTreeRoot()
if err != nil {
return errors.Wrap(err, "could not hash head block")
}
finalizedState, err := s.cfg.StateGen.Resume(ctx, s.cfg.FinalizedStateAtStartUp)
if err != nil {
return errors.Wrap(err, "could not get finalized state from db")
}
log.Infof("Regenerating state from the last checkpoint at slot %d to current head slot of %d."+
"This process may take a while, please wait.", finalizedState.Slot(), headBlock.Block().Slot())
headState, err := s.cfg.StateGen.StateByRoot(ctx, headRoot)
if err != nil {
return errors.Wrap(err, "could not retrieve head state")
}
s.setHead(headRoot, headBlock, headState)
return nil
} else {
log.Warnf("Finalized checkpoint at slot %d is too close to the current head slot, "+
"resetting head from the checkpoint ('--%s' flag is ignored).",
finalizedState.Slot(), flags.HeadSync.Name)
}
}
finalizedBlock, err := s.cfg.BeaconDB.Block(ctx, finalizedRoot)
if err != nil {
return errors.Wrap(err, "could not get finalized block from db")
}
if finalizedState == nil || finalizedState.IsNil() || finalizedBlock == nil || finalizedBlock.IsNil() {
return errors.New("finalized state and block can't be nil")
}
s.setHead(finalizedRoot, finalizedBlock, finalizedState)
return nil
}
func (s *Service) startFromPOWChain() error {
log.Info("Waiting to reach the validator deposit threshold to start the beacon chain...")
if s.cfg.ChainStartFetcher == nil {
return errors.New("not configured web3Service for POW chain")
}
go func() {
stateChannel := make(chan *feed.Event, 1)
stateSub := s.cfg.StateNotifier.StateFeed().Subscribe(stateChannel)
defer stateSub.Unsubscribe()
for {
select {
case event := <-stateChannel:
if event.Type == statefeed.ChainStarted {
data, ok := event.Data.(*statefeed.ChainStartedData)
if !ok {
log.Error("event data is not type *statefeed.ChainStartedData")
return
}
log.WithField("starttime", data.StartTime).Debug("Received chain start event")
s.onPowchainStart(s.ctx, data.StartTime)
return
}
case <-s.ctx.Done():
log.Debug("Context closed, exiting goroutine")
return
case err := <-stateSub.Err():
log.WithError(err).Error("Subscription to state notifier failed")
return
}
}
}()
return nil
}
// onPowchainStart initializes a series of deposits from the ChainStart deposits in the eth1
// deposit contract, initializes the beacon chain's state, and kicks off the beacon chain.
func (s *Service) processChainStartTime(ctx context.Context, genesisTime time.Time) {
func (s *Service) onPowchainStart(ctx context.Context, genesisTime time.Time) {
preGenesisState := s.cfg.ChainStartFetcher.PreGenesisState()
initializedState, err := s.initializeBeaconChain(ctx, genesisTime, preGenesisState, s.cfg.ChainStartFetcher.ChainStartEth1Data())
if err != nil {
@@ -261,7 +407,7 @@ func (s *Service) processChainStartTime(ctx context.Context, genesisTime time.Ti
Type: statefeed.Initialized,
Data: &statefeed.InitializedData{
StartTime: genesisTime,
GenesisValidatorsRoot: initializedState.GenesisValidatorRoot(),
GenesisValidatorsRoot: initializedState.GenesisValidatorsRoot(),
},
})
}
@@ -306,32 +452,6 @@ func (s *Service) initializeBeaconChain(
return genesisState, nil
}
// Stop the blockchain service's main event loop and associated goroutines.
func (s *Service) Stop() error {
defer s.cancel()
if s.cfg.StateGen != nil && s.head != nil && s.head.state != nil {
if err := s.cfg.StateGen.ForceCheckpoint(s.ctx, s.head.state.FinalizedCheckpoint().Root); err != nil {
return err
}
}
// Save initial sync cached blocks to the DB before stop.
return s.cfg.BeaconDB.SaveBlocks(s.ctx, s.getInitSyncBlocks())
}
// Status always returns nil unless there is an error condition that causes
// this service to be unhealthy.
func (s *Service) Status() error {
if s.genesisRoot == params.BeaconConfig().ZeroHash {
return errors.New("genesis state has not been created")
}
if runtime.NumGoroutine() > s.cfg.MaxRoutines {
return fmt.Errorf("too many goroutines %d", runtime.NumGoroutine())
}
return nil
}
// This gets called when beacon chain is first initialized to save genesis data (state, block, and more) in db.
func (s *Service) saveGenesisData(ctx context.Context, genesisState state.BeaconState) error {
if err := s.cfg.BeaconDB.SaveGenesisData(ctx, genesisState); err != nil {
@@ -346,123 +466,34 @@ func (s *Service) saveGenesisData(ctx context.Context, genesisState state.Beacon
return errors.Wrap(err, "could not get genesis block root")
}
s.genesisRoot = genesisBlkRoot
s.originBlockRoot = genesisBlkRoot
s.cfg.StateGen.SaveFinalizedState(0 /*slot*/, genesisBlkRoot, genesisState)
// Finalized checkpoint at genesis is a zero hash.
genesisCheckpoint := genesisState.FinalizedCheckpoint()
s.store = store.New(genesisCheckpoint, genesisCheckpoint)
s.justifiedCheckpt = ethpb.CopyCheckpoint(genesisCheckpoint)
if err := s.cacheJustifiedStateBalances(ctx, genesisBlkRoot); err != nil {
payloadHash, err := getBlockPayloadHash(genesisBlk.Block())
if err != nil {
return err
}
s.prevJustifiedCheckpt = ethpb.CopyCheckpoint(genesisCheckpoint)
s.bestJustifiedCheckpt = ethpb.CopyCheckpoint(genesisCheckpoint)
s.finalizedCheckpt = ethpb.CopyCheckpoint(genesisCheckpoint)
s.prevFinalizedCheckpt = ethpb.CopyCheckpoint(genesisCheckpoint)
if err := s.cfg.ForkChoiceStore.ProcessBlock(ctx,
if err := s.cfg.ForkChoiceStore.InsertOptimisticBlock(ctx,
genesisBlk.Block().Slot(),
genesisBlkRoot,
params.BeaconConfig().ZeroHash,
[32]byte{},
payloadHash,
genesisCheckpoint.Epoch,
genesisCheckpoint.Epoch); err != nil {
log.Fatalf("Could not process genesis block for fork choice: %v", err)
}
if err := s.cfg.ForkChoiceStore.SetOptimisticToValid(ctx, genesisBlkRoot); err != nil {
log.Fatalf("Could not set optimistic status of genesis block to false: %v", err)
}
s.setHead(genesisBlkRoot, genesisBlk, genesisState)
return nil
}
// This gets called to initialize chain info variables using the finalized checkpoint stored in DB
func (s *Service) initializeChainInfo(ctx context.Context) error {
genesisBlock, err := s.cfg.BeaconDB.GenesisBlock(ctx)
if err != nil {
return errors.Wrap(err, "could not get genesis block from db")
}
if genesisBlock == nil || genesisBlock.IsNil() {
return errors.New("no genesis block in db")
}
genesisBlkRoot, err := genesisBlock.Block().HashTreeRoot()
if err != nil {
return errors.Wrap(err, "could not get signing root of genesis block")
}
s.genesisRoot = genesisBlkRoot
finalized, err := s.cfg.BeaconDB.FinalizedCheckpoint(ctx)
if err != nil {
return errors.Wrap(err, "could not get finalized checkpoint from db")
}
if finalized == nil {
// This should never happen. At chain start, the finalized checkpoint
// would be the genesis state and block.
return errors.New("no finalized epoch in the database")
}
finalizedRoot := s.ensureRootNotZeros(bytesutil.ToBytes32(finalized.Root))
var finalizedState state.BeaconState
finalizedState, err = s.cfg.StateGen.Resume(ctx)
if err != nil {
return errors.Wrap(err, "could not get finalized state from db")
}
if flags.Get().HeadSync {
headBlock, err := s.cfg.BeaconDB.HeadBlock(ctx)
if err != nil {
return errors.Wrap(err, "could not retrieve head block")
}
headEpoch := slots.ToEpoch(headBlock.Block().Slot())
var epochsSinceFinality types.Epoch
if headEpoch > finalized.Epoch {
epochsSinceFinality = headEpoch - finalized.Epoch
}
// Head sync when node is far enough beyond known finalized epoch,
// this becomes really useful during long period of non-finality.
if epochsSinceFinality >= headSyncMinEpochsAfterCheckpoint {
headRoot, err := headBlock.Block().HashTreeRoot()
if err != nil {
return errors.Wrap(err, "could not hash head block")
}
finalizedState, err := s.cfg.StateGen.Resume(ctx)
if err != nil {
return errors.Wrap(err, "could not get finalized state from db")
}
log.Infof("Regenerating state from the last checkpoint at slot %d to current head slot of %d."+
"This process may take a while, please wait.", finalizedState.Slot(), headBlock.Block().Slot())
headState, err := s.cfg.StateGen.StateByRoot(ctx, headRoot)
if err != nil {
return errors.Wrap(err, "could not retrieve head state")
}
s.setHead(headRoot, headBlock, headState)
return nil
} else {
log.Warnf("Finalized checkpoint at slot %d is too close to the current head slot, "+
"resetting head from the checkpoint ('--%s' flag is ignored).",
finalizedState.Slot(), flags.HeadSync.Name)
}
}
finalizedBlock, err := s.cfg.BeaconDB.Block(ctx, finalizedRoot)
if err != nil {
return errors.Wrap(err, "could not get finalized block from db")
}
if finalizedState == nil || finalizedState.IsNil() || finalizedBlock == nil || finalizedBlock.IsNil() {
return errors.New("finalized state and block can't be nil")
}
s.setHead(finalizedRoot, finalizedBlock, finalizedState)
return nil
}
// This is called when a client starts from non-genesis slot. This passes last justified and finalized
// information to fork choice service to initializes fork choice store.
func (s *Service) resumeForkChoice(justifiedCheckpoint, finalizedCheckpoint *ethpb.Checkpoint) {
store := protoarray.New(justifiedCheckpoint.Epoch, finalizedCheckpoint.Epoch, bytesutil.ToBytes32(finalizedCheckpoint.Root))
s.cfg.ForkChoiceStore = store
}
// This returns true if block has been processed before. Two ways to verify the block has been processed:
// 1.) Check fork choice store.
// 2.) Check DB.
@@ -474,3 +505,20 @@ func (s *Service) hasBlock(ctx context.Context, root [32]byte) bool {
return s.cfg.BeaconDB.HasBlock(ctx, root)
}
func spawnCountdownIfPreGenesis(ctx context.Context, genesisTime time.Time, db db.HeadAccessDatabase) {
currentTime := prysmTime.Now()
if currentTime.After(genesisTime) {
return
}
gState, err := db.GenesisState(ctx)
if err != nil {
log.Fatalf("Could not retrieve genesis state: %v", err)
}
gRoot, err := gState.HashTreeRoot(ctx)
if err != nil {
log.Fatalf("Could not hash tree root genesis state: %v", err)
}
go slots.CountdownToGenesis(ctx, genesisTime, uint64(gState.NumValidators()), gRoot)
}

View File

@@ -18,7 +18,7 @@ func init() {
func TestChainService_SaveHead_DataRace(t *testing.T) {
beaconDB := testDB.SetupDB(t)
s := &Service{
cfg: &Config{BeaconDB: beaconDB},
cfg: &config{BeaconDB: beaconDB},
}
go func() {
require.NoError(t, s.saveHead(context.Background(), [32]byte{}))

View File

@@ -9,6 +9,8 @@ import (
"github.com/ethereum/go-ethereum/common"
"github.com/prysmaticlabs/prysm/async/event"
"github.com/prysmaticlabs/prysm/beacon-chain/blockchain/store"
mock "github.com/prysmaticlabs/prysm/beacon-chain/blockchain/testing"
"github.com/prysmaticlabs/prysm/beacon-chain/cache/depositcache"
b "github.com/prysmaticlabs/prysm/beacon-chain/core/blocks"
"github.com/prysmaticlabs/prysm/beacon-chain/core/feed"
@@ -17,6 +19,7 @@ import (
"github.com/prysmaticlabs/prysm/beacon-chain/core/transition"
"github.com/prysmaticlabs/prysm/beacon-chain/db"
testDB "github.com/prysmaticlabs/prysm/beacon-chain/db/testing"
doublylinkedtree "github.com/prysmaticlabs/prysm/beacon-chain/forkchoice/doubly-linked-tree"
"github.com/prysmaticlabs/prysm/beacon-chain/forkchoice/protoarray"
"github.com/prysmaticlabs/prysm/beacon-chain/operations/attestations"
"github.com/prysmaticlabs/prysm/beacon-chain/p2p"
@@ -94,11 +97,12 @@ func setupBeaconChain(t *testing.T, beaconDB db.Database) *Service {
DepositContainers: []*ethpb.DepositContainer{},
})
require.NoError(t, err)
web3Service, err = powchain.NewService(ctx, &powchain.Web3ServiceConfig{
BeaconDB: beaconDB,
HttpEndpoints: []string{endpoint},
DepositContract: common.Address{},
})
web3Service, err = powchain.NewService(
ctx,
powchain.WithDatabase(beaconDB),
powchain.WithHttpEndpoints([]string{endpoint}),
powchain.WithDepositContractAddress(common.Address{}),
)
require.NoError(t, err, "Unable to set up web3 service")
attService, err := attestations.NewService(ctx, &attestations.Config{Pool: attestations.NewPool()})
@@ -107,23 +111,23 @@ func setupBeaconChain(t *testing.T, beaconDB db.Database) *Service {
depositCache, err := depositcache.New()
require.NoError(t, err)
cfg := &Config{
BeaconBlockBuf: 0,
BeaconDB: beaconDB,
DepositCache: depositCache,
ChainStartFetcher: web3Service,
P2p: &mockBroadcaster{},
StateNotifier: &mockBeaconNode{},
AttPool: attestations.NewPool(),
StateGen: stategen.New(beaconDB),
ForkChoiceStore: protoarray.New(0, 0, params.BeaconConfig().ZeroHash),
AttService: attService,
stateGen := stategen.New(beaconDB)
// Safe a state in stategen to purposes of testing a service stop / shutdown.
require.NoError(t, stateGen.SaveState(ctx, bytesutil.ToBytes32(bState.FinalizedCheckpoint().Root), bState))
opts := []Option{
WithDatabase(beaconDB),
WithDepositCache(depositCache),
WithChainStartFetcher(web3Service),
WithAttestationPool(attestations.NewPool()),
WithP2PBroadcaster(&mockBroadcaster{}),
WithStateNotifier(&mockBeaconNode{}),
WithForkChoiceStore(protoarray.New(0, 0, params.BeaconConfig().ZeroHash)),
WithAttestationService(attService),
WithStateGen(stateGen),
}
// Safe a state in stategen to purposes of testing a service stop / shutdown.
require.NoError(t, cfg.StateGen.SaveState(ctx, bytesutil.ToBytes32(bState.FinalizedCheckpoint().Root), bState))
chainService, err := NewService(ctx, cfg)
chainService, err := NewService(ctx, opts...)
require.NoError(t, err, "Unable to setup chain service")
chainService.genesisTime = time.Unix(1, 0) // non-zero time
@@ -140,7 +144,9 @@ func TestChainStartStop_Initialized(t *testing.T) {
genesisBlk := util.NewBeaconBlock()
blkRoot, err := genesisBlk.Block.HashTreeRoot()
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(genesisBlk)))
wsb, err := wrapper.WrappedSignedBeaconBlock(genesisBlk)
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, wsb))
s, err := util.NewBeaconState()
require.NoError(t, err)
require.NoError(t, s.SetSlot(1))
@@ -149,7 +155,12 @@ func TestChainStartStop_Initialized(t *testing.T) {
require.NoError(t, beaconDB.SaveGenesisBlockRoot(ctx, blkRoot))
require.NoError(t, beaconDB.SaveJustifiedCheckpoint(ctx, &ethpb.Checkpoint{Root: blkRoot[:]}))
require.NoError(t, beaconDB.SaveFinalizedCheckpoint(ctx, &ethpb.Checkpoint{Root: blkRoot[:]}))
ss := &ethpb.StateSummary{
Slot: 1,
Root: blkRoot[:],
}
require.NoError(t, beaconDB.SaveStateSummary(ctx, ss))
chainService.cfg.FinalizedStateAtStartUp = s
// Test the start function.
chainService.Start()
@@ -170,13 +181,17 @@ func TestChainStartStop_GenesisZeroHashes(t *testing.T) {
genesisBlk := util.NewBeaconBlock()
blkRoot, err := genesisBlk.Block.HashTreeRoot()
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(genesisBlk)))
wsb, err := wrapper.WrappedSignedBeaconBlock(genesisBlk)
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, wsb))
s, err := util.NewBeaconState()
require.NoError(t, err)
require.NoError(t, beaconDB.SaveState(ctx, s, blkRoot))
require.NoError(t, beaconDB.SaveGenesisBlockRoot(ctx, blkRoot))
require.NoError(t, beaconDB.SaveBlock(ctx, wsb))
require.NoError(t, beaconDB.SaveJustifiedCheckpoint(ctx, &ethpb.Checkpoint{Root: params.BeaconConfig().ZeroHash[:]}))
require.NoError(t, beaconDB.SaveFinalizedCheckpoint(ctx, &ethpb.Checkpoint{Root: blkRoot[:]}))
chainService.cfg.FinalizedStateAtStartUp = s
// Test the start function.
chainService.Start()
@@ -239,7 +254,9 @@ func TestChainService_CorrectGenesisRoots(t *testing.T) {
genesisBlk := util.NewBeaconBlock()
blkRoot, err := genesisBlk.Block.HashTreeRoot()
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(genesisBlk)))
wsb, err := wrapper.WrappedSignedBeaconBlock(genesisBlk)
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, wsb))
s, err := util.NewBeaconState()
require.NoError(t, err)
require.NoError(t, s.SetSlot(0))
@@ -247,12 +264,12 @@ func TestChainService_CorrectGenesisRoots(t *testing.T) {
require.NoError(t, beaconDB.SaveHeadBlockRoot(ctx, blkRoot))
require.NoError(t, beaconDB.SaveGenesisBlockRoot(ctx, blkRoot))
require.NoError(t, beaconDB.SaveFinalizedCheckpoint(ctx, &ethpb.Checkpoint{Root: blkRoot[:]}))
chainService.cfg.FinalizedStateAtStartUp = s
// Test the start function.
chainService.Start()
require.DeepEqual(t, blkRoot[:], chainService.finalizedCheckpt.Root, "Finalize Checkpoint root is incorrect")
require.DeepEqual(t, params.BeaconConfig().ZeroHash[:], chainService.justifiedCheckpt.Root, "Justified Checkpoint root is incorrect")
require.DeepEqual(t, blkRoot[:], chainService.store.FinalizedCheckpt().Root, "Finalize Checkpoint root is incorrect")
require.DeepEqual(t, params.BeaconConfig().ZeroHash[:], chainService.store.JustifiedCheckpt().Root, "Justified Checkpoint root is incorrect")
require.NoError(t, chainService.Stop(), "Unable to stop chain service")
@@ -266,7 +283,9 @@ func TestChainService_InitializeChainInfo(t *testing.T) {
genesisRoot, err := genesis.Block.HashTreeRoot()
require.NoError(t, err)
require.NoError(t, beaconDB.SaveGenesisBlockRoot(ctx, genesisRoot))
require.NoError(t, beaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(genesis)))
wsb, err := wrapper.WrappedSignedBeaconBlock(genesis)
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, wsb))
finalizedSlot := params.BeaconConfig().SlotsPerEpoch*2 + 1
headBlock := util.NewBeaconBlock()
@@ -275,15 +294,22 @@ func TestChainService_InitializeChainInfo(t *testing.T) {
headState, err := util.NewBeaconState()
require.NoError(t, err)
require.NoError(t, headState.SetSlot(finalizedSlot))
require.NoError(t, headState.SetGenesisValidatorRoot(params.BeaconConfig().ZeroHash[:]))
require.NoError(t, headState.SetGenesisValidatorsRoot(params.BeaconConfig().ZeroHash[:]))
headRoot, err := headBlock.Block.HashTreeRoot()
require.NoError(t, err)
require.NoError(t, beaconDB.SaveState(ctx, headState, headRoot))
require.NoError(t, beaconDB.SaveState(ctx, headState, genesisRoot))
require.NoError(t, beaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(headBlock)))
wsb, err = wrapper.WrappedSignedBeaconBlock(headBlock)
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, wsb))
require.NoError(t, beaconDB.SaveFinalizedCheckpoint(ctx, &ethpb.Checkpoint{Epoch: slots.ToEpoch(finalizedSlot), Root: headRoot[:]}))
c := &Service{cfg: &Config{BeaconDB: beaconDB, StateGen: stategen.New(beaconDB)}}
require.NoError(t, c.initializeChainInfo(ctx))
attSrv, err := attestations.NewService(ctx, &attestations.Config{})
require.NoError(t, err)
stateGen := stategen.New(beaconDB)
c, err := NewService(ctx, WithDatabase(beaconDB), WithStateGen(stateGen), WithAttestationService(attSrv), WithStateNotifier(&mock.MockStateNotifier{}), WithFinalizedStateAtStartUp(headState))
require.NoError(t, err)
require.NoError(t, stateGen.SaveState(ctx, headRoot, headState))
require.NoError(t, c.StartFromSavedState(headState))
headBlk, err := c.HeadBlock(ctx)
require.NoError(t, err)
assert.DeepEqual(t, headBlock, headBlk.Proto(), "Head block incorrect")
@@ -296,7 +322,7 @@ func TestChainService_InitializeChainInfo(t *testing.T) {
if !bytes.Equal(headRoot[:], r) {
t.Error("head slot incorrect")
}
assert.Equal(t, genesisRoot, c.genesisRoot, "Genesis block root incorrect")
assert.Equal(t, genesisRoot, c.originBlockRoot, "Genesis block root incorrect")
}
func TestChainService_InitializeChainInfo_SetHeadAtGenesis(t *testing.T) {
@@ -307,7 +333,9 @@ func TestChainService_InitializeChainInfo_SetHeadAtGenesis(t *testing.T) {
genesisRoot, err := genesis.Block.HashTreeRoot()
require.NoError(t, err)
require.NoError(t, beaconDB.SaveGenesisBlockRoot(ctx, genesisRoot))
require.NoError(t, beaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(genesis)))
wsb, err := wrapper.WrappedSignedBeaconBlock(genesis)
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, wsb))
finalizedSlot := params.BeaconConfig().SlotsPerEpoch*2 + 1
headBlock := util.NewBeaconBlock()
@@ -316,19 +344,32 @@ func TestChainService_InitializeChainInfo_SetHeadAtGenesis(t *testing.T) {
headState, err := util.NewBeaconState()
require.NoError(t, err)
require.NoError(t, headState.SetSlot(finalizedSlot))
require.NoError(t, headState.SetGenesisValidatorRoot(params.BeaconConfig().ZeroHash[:]))
require.NoError(t, headState.SetGenesisValidatorsRoot(params.BeaconConfig().ZeroHash[:]))
headRoot, err := headBlock.Block.HashTreeRoot()
require.NoError(t, err)
require.NoError(t, beaconDB.SaveState(ctx, headState, headRoot))
require.NoError(t, beaconDB.SaveState(ctx, headState, genesisRoot))
require.NoError(t, beaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(headBlock)))
c := &Service{cfg: &Config{BeaconDB: beaconDB, StateGen: stategen.New(beaconDB)}}
require.NoError(t, c.initializeChainInfo(ctx))
wsb, err = wrapper.WrappedSignedBeaconBlock(headBlock)
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, wsb))
attSrv, err := attestations.NewService(ctx, &attestations.Config{})
require.NoError(t, err)
ss := &ethpb.StateSummary{
Slot: finalizedSlot,
Root: headRoot[:],
}
require.NoError(t, beaconDB.SaveStateSummary(ctx, ss))
require.NoError(t, beaconDB.SaveFinalizedCheckpoint(ctx, &ethpb.Checkpoint{Root: headRoot[:], Epoch: slots.ToEpoch(finalizedSlot)}))
stateGen := stategen.New(beaconDB)
c, err := NewService(ctx, WithDatabase(beaconDB), WithStateGen(stateGen), WithAttestationService(attSrv), WithStateNotifier(&mock.MockStateNotifier{}), WithFinalizedStateAtStartUp(headState))
require.NoError(t, err)
require.NoError(t, c.StartFromSavedState(headState))
s, err := c.HeadState(ctx)
require.NoError(t, err)
assert.DeepSSZEqual(t, headState.InnerStateUnsafe(), s.InnerStateUnsafe(), "Head state incorrect")
assert.Equal(t, genesisRoot, c.genesisRoot, "Genesis block root incorrect")
assert.DeepEqual(t, genesis, c.head.block.Proto())
assert.Equal(t, genesisRoot, c.originBlockRoot, "Genesis block root incorrect")
assert.DeepEqual(t, headBlock, c.head.block.Proto())
}
func TestChainService_InitializeChainInfo_HeadSync(t *testing.T) {
@@ -349,14 +390,18 @@ func TestChainService_InitializeChainInfo_HeadSync(t *testing.T) {
genesisRoot, err := genesisBlock.Block.HashTreeRoot()
require.NoError(t, err)
require.NoError(t, beaconDB.SaveGenesisBlockRoot(ctx, genesisRoot))
require.NoError(t, beaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(genesisBlock)))
wsb, err := wrapper.WrappedSignedBeaconBlock(genesisBlock)
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, wsb))
finalizedBlock := util.NewBeaconBlock()
finalizedBlock.Block.Slot = finalizedSlot
finalizedBlock.Block.ParentRoot = genesisRoot[:]
finalizedRoot, err := finalizedBlock.Block.HashTreeRoot()
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(finalizedBlock)))
wsb, err = wrapper.WrappedSignedBeaconBlock(finalizedBlock)
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, wsb))
// Set head slot close to the finalization point, no head sync is triggered.
headBlock := util.NewBeaconBlock()
@@ -364,28 +409,32 @@ func TestChainService_InitializeChainInfo_HeadSync(t *testing.T) {
headBlock.Block.ParentRoot = finalizedRoot[:]
headRoot, err := headBlock.Block.HashTreeRoot()
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(headBlock)))
wsb, err = wrapper.WrappedSignedBeaconBlock(headBlock)
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, wsb))
headState, err := util.NewBeaconState()
require.NoError(t, err)
require.NoError(t, headState.SetSlot(headBlock.Block.Slot))
require.NoError(t, headState.SetGenesisValidatorRoot(params.BeaconConfig().ZeroHash[:]))
require.NoError(t, beaconDB.SaveState(ctx, headState, genesisRoot))
require.NoError(t, beaconDB.SaveState(ctx, headState, finalizedRoot))
require.NoError(t, headState.SetGenesisValidatorsRoot(params.BeaconConfig().ZeroHash[:]))
require.NoError(t, beaconDB.SaveState(ctx, headState, headRoot))
require.NoError(t, beaconDB.SaveState(ctx, headState, finalizedRoot))
require.NoError(t, beaconDB.SaveHeadBlockRoot(ctx, headRoot))
require.NoError(t, beaconDB.SaveFinalizedCheckpoint(ctx, &ethpb.Checkpoint{
Epoch: slots.ToEpoch(finalizedBlock.Block.Slot),
Root: finalizedRoot[:],
}))
c := &Service{cfg: &Config{BeaconDB: beaconDB, StateGen: stategen.New(beaconDB)}}
require.NoError(t, c.initializeChainInfo(ctx))
attSrv, err := attestations.NewService(ctx, &attestations.Config{})
require.NoError(t, err)
stateGen := stategen.New(beaconDB)
c, err := NewService(ctx, WithDatabase(beaconDB), WithStateGen(stateGen), WithAttestationService(attSrv), WithStateNotifier(&mock.MockStateNotifier{}), WithFinalizedStateAtStartUp(headState))
require.NoError(t, err)
require.NoError(t, c.StartFromSavedState(headState))
s, err := c.HeadState(ctx)
require.NoError(t, err)
assert.DeepSSZEqual(t, headState.InnerStateUnsafe(), s.InnerStateUnsafe(), "Head state incorrect")
assert.Equal(t, genesisRoot, c.genesisRoot, "Genesis block root incorrect")
assert.Equal(t, genesisRoot, c.originBlockRoot, "Genesis block root incorrect")
// Since head sync is not triggered, chain is initialized to the last finalization checkpoint.
assert.DeepEqual(t, finalizedBlock, c.head.block.Proto())
assert.LogsContain(t, hook, "resetting head from the checkpoint ('--head-sync' flag is ignored)")
@@ -397,16 +446,18 @@ func TestChainService_InitializeChainInfo_HeadSync(t *testing.T) {
headBlock.Block.ParentRoot = finalizedRoot[:]
headRoot, err = headBlock.Block.HashTreeRoot()
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(headBlock)))
wsb, err = wrapper.WrappedSignedBeaconBlock(headBlock)
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, wsb))
require.NoError(t, beaconDB.SaveState(ctx, headState, headRoot))
require.NoError(t, beaconDB.SaveHeadBlockRoot(ctx, headRoot))
hook.Reset()
require.NoError(t, c.initializeChainInfo(ctx))
require.NoError(t, c.initializeHeadFromDB(ctx))
s, err = c.HeadState(ctx)
require.NoError(t, err)
assert.DeepSSZEqual(t, headState.InnerStateUnsafe(), s.InnerStateUnsafe(), "Head state incorrect")
assert.Equal(t, genesisRoot, c.genesisRoot, "Genesis block root incorrect")
assert.Equal(t, genesisRoot, c.originBlockRoot, "Genesis block root incorrect")
// Head slot is far beyond the latest finalized checkpoint, head sync is triggered.
assert.DeepEqual(t, headBlock, c.head.block.Proto())
assert.LogsContain(t, hook, "Regenerating state from the last checkpoint at slot 225")
@@ -417,7 +468,7 @@ func TestChainService_SaveHeadNoDB(t *testing.T) {
beaconDB := testDB.SetupDB(t)
ctx := context.Background()
s := &Service{
cfg: &Config{BeaconDB: beaconDB, StateGen: stategen.New(beaconDB)},
cfg: &config{BeaconDB: beaconDB, StateGen: stategen.New(beaconDB)},
}
blk := util.NewBeaconBlock()
blk.Block.Slot = 1
@@ -426,7 +477,9 @@ func TestChainService_SaveHeadNoDB(t *testing.T) {
newState, err := util.NewBeaconState()
require.NoError(t, err)
require.NoError(t, s.cfg.StateGen.SaveState(ctx, r, newState))
require.NoError(t, s.saveHeadNoDB(ctx, wrapper.WrappedPhase0SignedBeaconBlock(blk), r, newState))
wsb, err := wrapper.WrappedSignedBeaconBlock(blk)
require.NoError(t, err)
require.NoError(t, s.saveHeadNoDB(ctx, wsb, r, newState))
newB, err := s.cfg.BeaconDB.HeadBlock(ctx)
require.NoError(t, err)
@@ -435,19 +488,43 @@ func TestChainService_SaveHeadNoDB(t *testing.T) {
}
}
func TestHasBlock_ForkChoiceAndDB(t *testing.T) {
func TestHasBlock_ForkChoiceAndDB_ProtoArray(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
s := &Service{
cfg: &Config{ForkChoiceStore: protoarray.New(0, 0, [32]byte{}), BeaconDB: beaconDB},
finalizedCheckpt: &ethpb.Checkpoint{Root: make([]byte, 32)},
cfg: &config{ForkChoiceStore: protoarray.New(0, 0, [32]byte{}), BeaconDB: beaconDB},
store: &store.Store{},
}
s.store.SetFinalizedCheckpt(&ethpb.Checkpoint{Epoch: 0, Root: params.BeaconConfig().ZeroHash[:]})
block := util.NewBeaconBlock()
r, err := block.Block.HashTreeRoot()
require.NoError(t, err)
beaconState, err := util.NewBeaconState()
require.NoError(t, err)
require.NoError(t, s.insertBlockAndAttestationsToForkChoiceStore(ctx, wrapper.WrappedPhase0SignedBeaconBlock(block).Block(), r, beaconState))
wsb, err := wrapper.WrappedSignedBeaconBlock(block)
require.NoError(t, err)
require.NoError(t, s.insertBlockAndAttestationsToForkChoiceStore(ctx, wsb.Block(), r, beaconState))
assert.Equal(t, false, s.hasBlock(ctx, [32]byte{}), "Should not have block")
assert.Equal(t, true, s.hasBlock(ctx, r), "Should have block")
}
func TestHasBlock_ForkChoiceAndDB_DoublyLinkedTree(t *testing.T) {
ctx := context.Background()
beaconDB := testDB.SetupDB(t)
s := &Service{
cfg: &config{ForkChoiceStore: doublylinkedtree.New(0, 0), BeaconDB: beaconDB},
store: &store.Store{},
}
s.store.SetFinalizedCheckpt(&ethpb.Checkpoint{Epoch: 0, Root: params.BeaconConfig().ZeroHash[:]})
block := util.NewBeaconBlock()
r, err := block.Block.HashTreeRoot()
require.NoError(t, err)
beaconState, err := util.NewBeaconState()
require.NoError(t, err)
wsb, err := wrapper.WrappedSignedBeaconBlock(block)
require.NoError(t, err)
require.NoError(t, s.insertBlockAndAttestationsToForkChoiceStore(ctx, wsb.Block(), r, beaconState))
assert.Equal(t, false, s.hasBlock(ctx, [32]byte{}), "Should not have block")
assert.Equal(t, true, s.hasBlock(ctx, r), "Should have block")
@@ -457,7 +534,7 @@ func TestServiceStop_SaveCachedBlocks(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
beaconDB := testDB.SetupDB(t)
s := &Service{
cfg: &Config{BeaconDB: beaconDB, StateGen: stategen.New(beaconDB)},
cfg: &config{BeaconDB: beaconDB, StateGen: stategen.New(beaconDB)},
ctx: ctx,
cancel: cancel,
initSyncBlocks: make(map[[32]byte]block.SignedBeaconBlock),
@@ -465,7 +542,9 @@ func TestServiceStop_SaveCachedBlocks(t *testing.T) {
block := util.NewBeaconBlock()
r, err := block.Block.HashTreeRoot()
require.NoError(t, err)
s.saveInitSyncBlock(r, wrapper.WrappedPhase0SignedBeaconBlock(block))
wsb, err := wrapper.WrappedSignedBeaconBlock(block)
require.NoError(t, err)
s.saveInitSyncBlock(r, wsb)
require.NoError(t, s.Stop())
require.Equal(t, true, s.cfg.BeaconDB.HasBlock(ctx, r))
}
@@ -476,7 +555,7 @@ func TestProcessChainStartTime_ReceivedFeed(t *testing.T) {
stateChannel := make(chan *feed.Event, 1)
stateSub := service.cfg.StateNotifier.StateFeed().Subscribe(stateChannel)
defer stateSub.Unsubscribe()
service.processChainStartTime(context.Background(), time.Now())
service.onPowchainStart(context.Background(), time.Now())
stateEvent := <-stateChannel
require.Equal(t, int(stateEvent.Type), statefeed.Initialized)
@@ -488,10 +567,12 @@ func BenchmarkHasBlockDB(b *testing.B) {
beaconDB := testDB.SetupDB(b)
ctx := context.Background()
s := &Service{
cfg: &Config{BeaconDB: beaconDB},
cfg: &config{BeaconDB: beaconDB},
}
block := util.NewBeaconBlock()
require.NoError(b, s.cfg.BeaconDB.SaveBlock(ctx, wrapper.WrappedPhase0SignedBeaconBlock(block)))
wsb, err := wrapper.WrappedSignedBeaconBlock(block)
require.NoError(b, err)
require.NoError(b, s.cfg.BeaconDB.SaveBlock(ctx, wsb))
r, err := block.Block.HashTreeRoot()
require.NoError(b, err)
@@ -501,20 +582,46 @@ func BenchmarkHasBlockDB(b *testing.B) {
}
}
func BenchmarkHasBlockForkChoiceStore(b *testing.B) {
func BenchmarkHasBlockForkChoiceStore_ProtoArray(b *testing.B) {
ctx := context.Background()
beaconDB := testDB.SetupDB(b)
s := &Service{
cfg: &Config{ForkChoiceStore: protoarray.New(0, 0, [32]byte{}), BeaconDB: beaconDB},
finalizedCheckpt: &ethpb.Checkpoint{Root: make([]byte, 32)},
cfg: &config{ForkChoiceStore: protoarray.New(0, 0, [32]byte{}), BeaconDB: beaconDB},
store: &store.Store{},
}
block := &ethpb.SignedBeaconBlock{Block: &ethpb.BeaconBlock{Body: &ethpb.BeaconBlockBody{}}}
r, err := block.Block.HashTreeRoot()
s.store.SetFinalizedCheckpt(&ethpb.Checkpoint{Epoch: 0, Root: params.BeaconConfig().ZeroHash[:]})
blk := &ethpb.SignedBeaconBlock{Block: &ethpb.BeaconBlock{Body: &ethpb.BeaconBlockBody{}}}
r, err := blk.Block.HashTreeRoot()
require.NoError(b, err)
bs := &ethpb.BeaconState{FinalizedCheckpoint: &ethpb.Checkpoint{Root: make([]byte, 32)}, CurrentJustifiedCheckpoint: &ethpb.Checkpoint{Root: make([]byte, 32)}}
beaconState, err := v1.InitializeFromProto(bs)
require.NoError(b, err)
require.NoError(b, s.insertBlockAndAttestationsToForkChoiceStore(ctx, wrapper.WrappedPhase0SignedBeaconBlock(block).Block(), r, beaconState))
wsb, err := wrapper.WrappedSignedBeaconBlock(blk)
require.NoError(b, err)
require.NoError(b, s.insertBlockAndAttestationsToForkChoiceStore(ctx, wsb.Block(), r, beaconState))
b.ResetTimer()
for i := 0; i < b.N; i++ {
require.Equal(b, true, s.cfg.ForkChoiceStore.HasNode(r), "Block is not in fork choice store")
}
}
func BenchmarkHasBlockForkChoiceStore_DoublyLinkedTree(b *testing.B) {
ctx := context.Background()
beaconDB := testDB.SetupDB(b)
s := &Service{
cfg: &config{ForkChoiceStore: doublylinkedtree.New(0, 0), BeaconDB: beaconDB},
store: &store.Store{},
}
s.store.SetFinalizedCheckpt(&ethpb.Checkpoint{Epoch: 0, Root: params.BeaconConfig().ZeroHash[:]})
block := &ethpb.SignedBeaconBlock{Block: &ethpb.BeaconBlock{Body: &ethpb.BeaconBlockBody{}}}
r, err := block.Block.HashTreeRoot()
require.NoError(b, err)
bs := &ethpb.BeaconState{FinalizedCheckpoint: &ethpb.Checkpoint{Root: make([]byte, 32)}, CurrentJustifiedCheckpoint: &ethpb.Checkpoint{Root: make([]byte, 32)}}
beaconState, err := v1.InitializeFromProto(bs)
require.NoError(b, err)
wsb, err := wrapper.WrappedSignedBeaconBlock(block)
require.NoError(b, err)
require.NoError(b, s.insertBlockAndAttestationsToForkChoiceStore(ctx, wsb.Block(), r, beaconState))
b.ResetTimer()
for i := 0; i < b.N; i++ {

View File

@@ -0,0 +1,83 @@
package blockchain
import (
"context"
"errors"
"sync"
"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
"github.com/prysmaticlabs/prysm/beacon-chain/core/time"
"github.com/prysmaticlabs/prysm/beacon-chain/state"
"github.com/prysmaticlabs/prysm/beacon-chain/state/stategen"
)
var errNilStateFromStategen = errors.New("justified state can't be nil")
type stateBalanceCache struct {
sync.Mutex
balances []uint64
root [32]byte
stateGen stateByRooter
}
type stateByRooter interface {
StateByRoot(context.Context, [32]byte) (state.BeaconState, error)
}
// newStateBalanceCache exists to remind us that stateBalanceCache needs a stagegen
// to avoid nil pointer bugs when updating the cache in the read path (get())
func newStateBalanceCache(sg *stategen.State) (*stateBalanceCache, error) {
if sg == nil {
return nil, errors.New("can't initialize state balance cache without stategen")
}
return &stateBalanceCache{stateGen: sg}, nil
}
// update is called by get() when the requested root doesn't match
// the previously read value. This cache assumes we only want to cache one
// set of balances for a single root (the current justified root).
//
// warning: this is not thread-safe on its own, relies on get() for locking
func (c *stateBalanceCache) update(ctx context.Context, justifiedRoot [32]byte) ([]uint64, error) {
stateBalanceCacheMiss.Inc()
justifiedState, err := c.stateGen.StateByRoot(ctx, justifiedRoot)
if err != nil {
return nil, err
}
if justifiedState == nil || justifiedState.IsNil() {
return nil, errNilStateFromStategen
}
epoch := time.CurrentEpoch(justifiedState)
justifiedBalances := make([]uint64, justifiedState.NumValidators())
var balanceAccumulator = func(idx int, val state.ReadOnlyValidator) error {
if helpers.IsActiveValidatorUsingTrie(val, epoch) {
justifiedBalances[idx] = val.EffectiveBalance()
} else {
justifiedBalances[idx] = 0
}
return nil
}
if err := justifiedState.ReadFromEveryValidator(balanceAccumulator); err != nil {
return nil, err
}
c.balances = justifiedBalances
c.root = justifiedRoot
return c.balances, nil
}
// getBalances takes an explicit justifiedRoot so it can invalidate the singleton cache key
// when the justified root changes, and takes a context so that the long-running stategen
// read path can connect to the upstream cancellation/timeout chain.
func (c *stateBalanceCache) get(ctx context.Context, justifiedRoot [32]byte) ([]uint64, error) {
c.Lock()
defer c.Unlock()
if justifiedRoot != [32]byte{} && justifiedRoot == c.root {
stateBalanceCacheHit.Inc()
return c.balances, nil
}
return c.update(ctx, justifiedRoot)
}

View File

@@ -0,0 +1,236 @@
package blockchain
import (
"context"
"encoding/binary"
"errors"
"testing"
types "github.com/prysmaticlabs/eth2-types"
"github.com/prysmaticlabs/prysm/beacon-chain/state"
v2 "github.com/prysmaticlabs/prysm/beacon-chain/state/v2"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/testing/require"
"github.com/prysmaticlabs/prysm/time/slots"
)
type mockStateByRoot struct {
state state.BeaconState
err error
}
func (m *mockStateByRoot) StateByRoot(context.Context, [32]byte) (state.BeaconState, error) {
return m.state, m.err
}
type testStateOpt func(*ethpb.BeaconStateAltair)
func testStateWithValidators(v []*ethpb.Validator) testStateOpt {
return func(a *ethpb.BeaconStateAltair) {
a.Validators = v
}
}
func testStateWithSlot(slot types.Slot) testStateOpt {
return func(a *ethpb.BeaconStateAltair) {
a.Slot = slot
}
}
func testStateFixture(opts ...testStateOpt) state.BeaconState {
a := &ethpb.BeaconStateAltair{}
for _, o := range opts {
o(a)
}
s, _ := v2.InitializeFromProtoUnsafe(a)
return s
}
func generateTestValidators(count int, opts ...func(*ethpb.Validator)) []*ethpb.Validator {
vs := make([]*ethpb.Validator, count)
var i uint32 = 0
for ; i < uint32(count); i++ {
pk := make([]byte, 48)
binary.LittleEndian.PutUint32(pk, i)
v := &ethpb.Validator{PublicKey: pk}
for _, o := range opts {
o(v)
}
vs[i] = v
}
return vs
}
func oddValidatorsExpired(currentSlot types.Slot) func(*ethpb.Validator) {
return func(v *ethpb.Validator) {
pki := binary.LittleEndian.Uint64(v.PublicKey)
if pki%2 == 0 {
v.ExitEpoch = types.Epoch(int(slots.ToEpoch(currentSlot)) + 1)
} else {
v.ExitEpoch = types.Epoch(int(slots.ToEpoch(currentSlot)) - 1)
}
}
}
func oddValidatorsQueued(currentSlot types.Slot) func(*ethpb.Validator) {
return func(v *ethpb.Validator) {
v.ExitEpoch = types.Epoch(int(slots.ToEpoch(currentSlot)) + 1)
pki := binary.LittleEndian.Uint64(v.PublicKey)
if pki%2 == 0 {
v.ActivationEpoch = types.Epoch(int(slots.ToEpoch(currentSlot)) - 1)
} else {
v.ActivationEpoch = types.Epoch(int(slots.ToEpoch(currentSlot)) + 1)
}
}
}
func allValidatorsValid(currentSlot types.Slot) func(*ethpb.Validator) {
return func(v *ethpb.Validator) {
v.ActivationEpoch = types.Epoch(int(slots.ToEpoch(currentSlot)) - 1)
v.ExitEpoch = types.Epoch(int(slots.ToEpoch(currentSlot)) + 1)
}
}
func balanceIsKeyTimes2(v *ethpb.Validator) {
pki := binary.LittleEndian.Uint64(v.PublicKey)
v.EffectiveBalance = uint64(pki) * 2
}
func testHalfExpiredValidators() ([]*ethpb.Validator, []uint64) {
balances := []uint64{0, 0, 4, 0, 8, 0, 12, 0, 16, 0}
return generateTestValidators(10,
oddValidatorsExpired(types.Slot(99)),
balanceIsKeyTimes2), balances
}
func testHalfQueuedValidators() ([]*ethpb.Validator, []uint64) {
balances := []uint64{0, 0, 4, 0, 8, 0, 12, 0, 16, 0}
return generateTestValidators(10,
oddValidatorsQueued(types.Slot(99)),
balanceIsKeyTimes2), balances
}
func testAllValidValidators() ([]*ethpb.Validator, []uint64) {
balances := []uint64{0, 2, 4, 6, 8, 10, 12, 14, 16, 18}
return generateTestValidators(10,
allValidatorsValid(types.Slot(99)),
balanceIsKeyTimes2), balances
}
func TestStateBalanceCache(t *testing.T) {
type sbcTestCase struct {
err error
root [32]byte
sbc *stateBalanceCache
balances []uint64
name string
}
sentinelCacheMiss := errors.New("Cache missed, as expected!")
sentinelBalances := []uint64{1, 2, 3, 4, 5}
halfExpiredValidators, halfExpiredBalances := testHalfExpiredValidators()
halfQueuedValidators, halfQueuedBalances := testHalfQueuedValidators()
allValidValidators, allValidBalances := testAllValidValidators()
cases := []sbcTestCase{
{
root: bytesutil.ToBytes32([]byte{'A'}),
balances: sentinelBalances,
sbc: &stateBalanceCache{
stateGen: &mockStateByRooter{
err: sentinelCacheMiss,
},
root: bytesutil.ToBytes32([]byte{'A'}),
balances: sentinelBalances,
},
name: "cache hit",
},
// this works by using a staterooter that returns a known error
// so really we're testing the miss by making sure stategen got called
// this also tells us stategen errors are propagated
{
sbc: &stateBalanceCache{
stateGen: &mockStateByRooter{
err: sentinelCacheMiss,
},
root: bytesutil.ToBytes32([]byte{'B'}),
},
err: sentinelCacheMiss,
root: bytesutil.ToBytes32([]byte{'A'}),
name: "cache miss",
},
{
sbc: &stateBalanceCache{
stateGen: &mockStateByRooter{},
root: bytesutil.ToBytes32([]byte{'B'}),
},
err: errNilStateFromStategen,
root: bytesutil.ToBytes32([]byte{'A'}),
name: "error for nil state upon cache miss",
},
{
sbc: &stateBalanceCache{
stateGen: &mockStateByRooter{
state: testStateFixture(
testStateWithSlot(99),
testStateWithValidators(halfExpiredValidators)),
},
},
balances: halfExpiredBalances,
root: bytesutil.ToBytes32([]byte{'A'}),
name: "test filtering by exit epoch",
},
{
sbc: &stateBalanceCache{
stateGen: &mockStateByRooter{
state: testStateFixture(
testStateWithSlot(99),
testStateWithValidators(halfQueuedValidators)),
},
},
balances: halfQueuedBalances,
root: bytesutil.ToBytes32([]byte{'A'}),
name: "test filtering by activation epoch",
},
{
sbc: &stateBalanceCache{
stateGen: &mockStateByRooter{
state: testStateFixture(
testStateWithSlot(99),
testStateWithValidators(allValidValidators)),
},
},
balances: allValidBalances,
root: bytesutil.ToBytes32([]byte{'A'}),
name: "happy path",
},
{
sbc: &stateBalanceCache{
stateGen: &mockStateByRooter{
state: testStateFixture(
testStateWithSlot(99),
testStateWithValidators(allValidValidators)),
},
},
balances: allValidBalances,
root: [32]byte{},
name: "zero root",
},
}
ctx := context.Background()
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
cache := c.sbc
cacheRootStart := cache.root
b, err := cache.get(ctx, c.root)
require.ErrorIs(t, err, c.err)
require.DeepEqual(t, c.balances, b)
if c.err != nil {
// if there was an error somewhere, the root should not have changed (unless it already matched)
require.Equal(t, cacheRootStart, cache.root)
} else {
// when successful, the cache should always end with a root matching the request
require.Equal(t, c.root, cache.root)
}
})
}
}

View File

@@ -0,0 +1,27 @@
load("@prysm//tools/go:def.bzl", "go_library", "go_test")
go_library(
name = "go_default_library",
srcs = [
"doc.go",
"new.go",
"setter_getter.go",
"type.go",
],
importpath = "github.com/prysmaticlabs/prysm/beacon-chain/blockchain/store",
visibility = ["//beacon-chain:__subpackages__"],
deps = ["//proto/prysm/v1alpha1:go_default_library"],
)
go_test(
name = "go_default_test",
srcs = [
"new_test.go",
"setter_getter_test.go",
],
embed = [":go_default_library"],
deps = [
"//proto/prysm/v1alpha1:go_default_library",
"//testing/require:go_default_library",
],
)

View File

@@ -0,0 +1,4 @@
// Package store implements the store object defined in the phase0 fork choice spec.
// It serves as a helpful middleware layer in between blockchain pkg and fork choice protoarray pkg.
// All the getters and setters are concurrent thread safe
package store

View File

@@ -0,0 +1,15 @@
package store
import (
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
)
func New(justifiedCheckpt *ethpb.Checkpoint, finalizedCheckpt *ethpb.Checkpoint) *Store {
return &Store{
justifiedCheckpt: justifiedCheckpt,
prevJustifiedCheckpt: justifiedCheckpt,
bestJustifiedCheckpt: justifiedCheckpt,
finalizedCheckpt: finalizedCheckpt,
prevFinalizedCheckpt: finalizedCheckpt,
}
}

View File

@@ -0,0 +1,25 @@
package store
import (
"testing"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/testing/require"
)
func TestNew(t *testing.T) {
j := &ethpb.Checkpoint{
Epoch: 0,
Root: []byte("hi"),
}
f := &ethpb.Checkpoint{
Epoch: 0,
Root: []byte("hello"),
}
s := New(j, f)
require.DeepSSZEqual(t, s.JustifiedCheckpt(), j)
require.DeepSSZEqual(t, s.BestJustifiedCheckpt(), j)
require.DeepSSZEqual(t, s.PrevJustifiedCheckpt(), j)
require.DeepSSZEqual(t, s.FinalizedCheckpt(), f)
require.DeepSSZEqual(t, s.PrevFinalizedCheckpt(), f)
}

View File

@@ -0,0 +1,73 @@
package store
import ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
// PrevJustifiedCheckpt returns the previous justified checkpoint in the Store.
func (s *Store) PrevJustifiedCheckpt() *ethpb.Checkpoint {
s.RLock()
defer s.RUnlock()
return s.prevJustifiedCheckpt
}
// BestJustifiedCheckpt returns the best justified checkpoint in the Store.
func (s *Store) BestJustifiedCheckpt() *ethpb.Checkpoint {
s.RLock()
defer s.RUnlock()
return s.bestJustifiedCheckpt
}
// JustifiedCheckpt returns the justified checkpoint in the Store.
func (s *Store) JustifiedCheckpt() *ethpb.Checkpoint {
s.RLock()
defer s.RUnlock()
return s.justifiedCheckpt
}
// PrevFinalizedCheckpt returns the previous finalized checkpoint in the Store.
func (s *Store) PrevFinalizedCheckpt() *ethpb.Checkpoint {
s.RLock()
defer s.RUnlock()
return s.prevFinalizedCheckpt
}
// FinalizedCheckpt returns the finalized checkpoint in the Store.
func (s *Store) FinalizedCheckpt() *ethpb.Checkpoint {
s.RLock()
defer s.RUnlock()
return s.finalizedCheckpt
}
// SetPrevJustifiedCheckpt sets the previous justified checkpoint in the Store.
func (s *Store) SetPrevJustifiedCheckpt(cp *ethpb.Checkpoint) {
s.Lock()
defer s.Unlock()
s.prevJustifiedCheckpt = cp
}
// SetBestJustifiedCheckpt sets the best justified checkpoint in the Store.
func (s *Store) SetBestJustifiedCheckpt(cp *ethpb.Checkpoint) {
s.Lock()
defer s.Unlock()
s.bestJustifiedCheckpt = cp
}
// SetJustifiedCheckpt sets the justified checkpoint in the Store.
func (s *Store) SetJustifiedCheckpt(cp *ethpb.Checkpoint) {
s.Lock()
defer s.Unlock()
s.justifiedCheckpt = cp
}
// SetFinalizedCheckpt sets the finalized checkpoint in the Store.
func (s *Store) SetFinalizedCheckpt(cp *ethpb.Checkpoint) {
s.Lock()
defer s.Unlock()
s.finalizedCheckpt = cp
}
// SetPrevFinalizedCheckpt sets the previous finalized checkpoint in the Store.
func (s *Store) SetPrevFinalizedCheckpt(cp *ethpb.Checkpoint) {
s.Lock()
defer s.Unlock()
s.prevFinalizedCheckpt = cp
}

View File

@@ -0,0 +1,53 @@
package store
import (
"testing"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/testing/require"
)
func Test_store_PrevJustifiedCheckpt(t *testing.T) {
s := &Store{}
var cp *ethpb.Checkpoint
require.Equal(t, cp, s.PrevJustifiedCheckpt())
cp = &ethpb.Checkpoint{Epoch: 1, Root: []byte{'a'}}
s.SetPrevJustifiedCheckpt(cp)
require.Equal(t, cp, s.PrevJustifiedCheckpt())
}
func Test_store_BestJustifiedCheckpt(t *testing.T) {
s := &Store{}
var cp *ethpb.Checkpoint
require.Equal(t, cp, s.BestJustifiedCheckpt())
cp = &ethpb.Checkpoint{Epoch: 1, Root: []byte{'a'}}
s.SetBestJustifiedCheckpt(cp)
require.Equal(t, cp, s.BestJustifiedCheckpt())
}
func Test_store_JustifiedCheckpt(t *testing.T) {
s := &Store{}
var cp *ethpb.Checkpoint
require.Equal(t, cp, s.JustifiedCheckpt())
cp = &ethpb.Checkpoint{Epoch: 1, Root: []byte{'a'}}
s.SetJustifiedCheckpt(cp)
require.Equal(t, cp, s.JustifiedCheckpt())
}
func Test_store_FinalizedCheckpt(t *testing.T) {
s := &Store{}
var cp *ethpb.Checkpoint
require.Equal(t, cp, s.FinalizedCheckpt())
cp = &ethpb.Checkpoint{Epoch: 1, Root: []byte{'a'}}
s.SetFinalizedCheckpt(cp)
require.Equal(t, cp, s.FinalizedCheckpt())
}
func Test_store_PrevFinalizedCheckpt(t *testing.T) {
s := &Store{}
var cp *ethpb.Checkpoint
require.Equal(t, cp, s.PrevFinalizedCheckpt())
cp = &ethpb.Checkpoint{Epoch: 1, Root: []byte{'a'}}
s.SetPrevFinalizedCheckpt(cp)
require.Equal(t, cp, s.PrevFinalizedCheckpt())
}

View File

@@ -0,0 +1,28 @@
package store
import (
"sync"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
)
// Store is defined in the fork choice consensus spec for tracking current time and various versions of checkpoints.
//
// Spec code:
// class Store(object):
// time: uint64
// genesis_time: uint64
// justified_checkpoint: Checkpoint
// finalized_checkpoint: Checkpoint
// best_justified_checkpoint: Checkpoint
// proposerBoostRoot: Root
type Store struct {
justifiedCheckpt *ethpb.Checkpoint
finalizedCheckpt *ethpb.Checkpoint
bestJustifiedCheckpt *ethpb.Checkpoint
sync.RWMutex
// These are not part of the consensus spec, but we do use them to return gRPC API requests.
// TODO(10094): Consider removing in v3.
prevFinalizedCheckpt *ethpb.Checkpoint
prevJustifiedCheckpt *ethpb.Checkpoint
}

View File

@@ -8,7 +8,6 @@ go_library(
visibility = [
"//beacon-chain:__subpackages__",
"//testing:__subpackages__",
"//testing/fuzz:__pkg__",
],
deps = [
"//async/event:go_default_library",
@@ -19,9 +18,9 @@ go_library(
"//beacon-chain/core/feed/state:go_default_library",
"//beacon-chain/core/helpers:go_default_library",
"//beacon-chain/db:go_default_library",
"//beacon-chain/forkchoice/protoarray:go_default_library",
"//beacon-chain/forkchoice:go_default_library",
"//beacon-chain/state:go_default_library",
"//beacon-chain/state/v1:go_default_library",
"//config/fieldparams:go_default_library",
"//config/params:go_default_library",
"//encoding/bytesutil:go_default_library",
"//proto/prysm/v1alpha1:go_default_library",

View File

@@ -18,9 +18,9 @@ import (
statefeed "github.com/prysmaticlabs/prysm/beacon-chain/core/feed/state"
"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
"github.com/prysmaticlabs/prysm/beacon-chain/db"
"github.com/prysmaticlabs/prysm/beacon-chain/forkchoice/protoarray"
"github.com/prysmaticlabs/prysm/beacon-chain/forkchoice"
"github.com/prysmaticlabs/prysm/beacon-chain/state"
v1 "github.com/prysmaticlabs/prysm/beacon-chain/state/v1"
fieldparams "github.com/prysmaticlabs/prysm/config/fieldparams"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
@@ -28,36 +28,44 @@ import (
"github.com/sirupsen/logrus"
)
var ErrNilState = errors.New("nil state")
// ChainService defines the mock interface for testing
type ChainService struct {
State state.BeaconState
Root []byte
Block block.SignedBeaconBlock
Optimistic bool
ValidAttestation bool
ValidatorsRoot [32]byte
PublicKey [fieldparams.BLSPubkeyLength]byte
FinalizedCheckPoint *ethpb.Checkpoint
CurrentJustifiedCheckPoint *ethpb.Checkpoint
PreviousJustifiedCheckPoint *ethpb.Checkpoint
BlocksReceived []block.SignedBeaconBlock
Slot *types.Slot // Pointer because 0 is a useful value, so checking against it can be incorrect.
Balance *precompute.Balance
Genesis time.Time
ValidatorsRoot [32]byte
CanonicalRoots map[[32]byte]bool
Fork *ethpb.Fork
ETH1Data *ethpb.Eth1Data
InitSyncBlockRoots map[[32]byte]bool
DB db.Database
State state.BeaconState
Block block.SignedBeaconBlock
VerifyBlkDescendantErr error
stateNotifier statefeed.Notifier
BlocksReceived []block.SignedBeaconBlock
SyncCommitteeIndices []types.CommitteeIndex
blockNotifier blockfeed.Notifier
opNotifier opfeed.Notifier
ValidAttestation bool
ForkChoiceStore *protoarray.Store
VerifyBlkDescendantErr error
Slot *types.Slot // Pointer because 0 is a useful value, so checking against it can be incorrect.
SyncCommitteeIndices []types.CommitteeIndex
Root []byte
SyncCommitteeDomain []byte
SyncSelectionProofDomain []byte
SyncContributionProofDomain []byte
PublicKey [48]byte
SyncCommitteePubkeys [][]byte
InitSyncBlockRoots map[[32]byte]bool
Genesis time.Time
ForkChoiceStore forkchoice.ForkChoicer
}
// ForkChoicer mocks the same method in the chain service
func (s *ChainService) ForkChoicer() forkchoice.ForkChoicer {
return s.ForkChoiceStore
}
// StateNotifier mocks the same method in the chain service.
@@ -158,7 +166,7 @@ func (mon *MockOperationNotifier) OperationFeed() *event.Feed {
// ReceiveBlockInitialSync mocks ReceiveBlockInitialSync method in chain service.
func (s *ChainService) ReceiveBlockInitialSync(ctx context.Context, block block.SignedBeaconBlock, _ [32]byte) error {
if s.State == nil {
s.State = &v1.BeaconState{}
return ErrNilState
}
if !bytes.Equal(s.Root, block.Block().ParentRoot()) {
return errors.Errorf("wanted %#x but got %#x", s.Root, block.Block().ParentRoot())
@@ -185,7 +193,7 @@ func (s *ChainService) ReceiveBlockInitialSync(ctx context.Context, block block.
// ReceiveBlockBatch processes blocks in batches from initial-sync.
func (s *ChainService) ReceiveBlockBatch(ctx context.Context, blks []block.SignedBeaconBlock, _ [][32]byte) error {
if s.State == nil {
s.State = &v1.BeaconState{}
return ErrNilState
}
for _, block := range blks {
if !bytes.Equal(s.Root, block.Block().ParentRoot()) {
@@ -214,7 +222,7 @@ func (s *ChainService) ReceiveBlockBatch(ctx context.Context, blks []block.Signe
// ReceiveBlock mocks ReceiveBlock method in chain service.
func (s *ChainService) ReceiveBlock(ctx context.Context, block block.SignedBeaconBlock, _ [32]byte) error {
if s.State == nil {
s.State = &v1.BeaconState{}
return ErrNilState
}
if !bytes.Equal(s.Root, block.Block().ParentRoot()) {
return errors.Errorf("wanted %#x but got %#x", s.Root, block.Block().ParentRoot())
@@ -285,12 +293,12 @@ func (s *ChainService) PreviousJustifiedCheckpt() *ethpb.Checkpoint {
}
// ReceiveAttestation mocks ReceiveAttestation method in chain service.
func (s *ChainService) ReceiveAttestation(_ context.Context, _ *ethpb.Attestation) error {
func (_ *ChainService) ReceiveAttestation(_ context.Context, _ *ethpb.Attestation) error {
return nil
}
// ReceiveAttestationNoPubsub mocks ReceiveAttestationNoPubsub method in chain service.
func (s *ChainService) ReceiveAttestationNoPubsub(context.Context, *ethpb.Attestation) error {
func (_ *ChainService) ReceiveAttestationNoPubsub(context.Context, *ethpb.Attestation) error {
return nil
}
@@ -317,18 +325,13 @@ func (s *ChainService) HeadETH1Data() *ethpb.Eth1Data {
return s.ETH1Data
}
// ProtoArrayStore mocks the same method in the chain service.
func (s *ChainService) ProtoArrayStore() *protoarray.Store {
return s.ForkChoiceStore
}
// GenesisTime mocks the same method in the chain service.
func (s *ChainService) GenesisTime() time.Time {
return s.Genesis
}
// GenesisValidatorRoot mocks the same method in the chain service.
func (s *ChainService) GenesisValidatorRoot() [32]byte {
// GenesisValidatorsRoot mocks the same method in the chain service.
func (s *ChainService) GenesisValidatorsRoot() [32]byte {
return s.ValidatorsRoot
}
@@ -368,8 +371,8 @@ func (s *ChainService) HasInitSyncBlock(rt [32]byte) bool {
return s.InitSyncBlockRoots[rt]
}
// HeadGenesisValidatorRoot mocks HeadGenesisValidatorRoot method in chain service.
func (s *ChainService) HeadGenesisValidatorRoot() [32]byte {
// HeadGenesisValidatorsRoot mocks HeadGenesisValidatorsRoot method in chain service.
func (_ *ChainService) HeadGenesisValidatorsRoot() [32]byte {
return [32]byte{}
}
@@ -379,7 +382,7 @@ func (s *ChainService) VerifyBlkDescendant(_ context.Context, _ [32]byte) error
}
// VerifyLmdFfgConsistency mocks VerifyLmdFfgConsistency and always returns nil.
func (s *ChainService) VerifyLmdFfgConsistency(_ context.Context, a *ethpb.Attestation) error {
func (_ *ChainService) VerifyLmdFfgConsistency(_ context.Context, a *ethpb.Attestation) error {
if !bytes.Equal(a.Data.BeaconBlockRoot, a.Data.Target.Root) {
return errors.New("LMD and FFG miss matched")
}
@@ -395,7 +398,7 @@ func (s *ChainService) VerifyFinalizedConsistency(_ context.Context, r []byte) e
}
// ChainHeads mocks ChainHeads and always return nil.
func (s *ChainService) ChainHeads() ([][32]byte, []types.Slot) {
func (_ *ChainService) ChainHeads() ([][32]byte, []types.Slot) {
return [][32]byte{
bytesutil.ToBytes32(bytesutil.PadTo([]byte("foo"), 32)),
bytesutil.ToBytes32(bytesutil.PadTo([]byte("bar"), 32)),
@@ -404,12 +407,12 @@ func (s *ChainService) ChainHeads() ([][32]byte, []types.Slot) {
}
// HeadPublicKeyToValidatorIndex mocks HeadPublicKeyToValidatorIndex and always return 0 and true.
func (s *ChainService) HeadPublicKeyToValidatorIndex(_ context.Context, _ [48]byte) (types.ValidatorIndex, bool) {
func (_ *ChainService) HeadPublicKeyToValidatorIndex(_ [fieldparams.BLSPubkeyLength]byte) (types.ValidatorIndex, bool) {
return 0, true
}
// HeadValidatorIndexToPublicKey mocks HeadValidatorIndexToPublicKey and always return empty and nil.
func (s *ChainService) HeadValidatorIndexToPublicKey(_ context.Context, _ types.ValidatorIndex) ([48]byte, error) {
func (s *ChainService) HeadValidatorIndexToPublicKey(_ context.Context, _ types.ValidatorIndex) ([fieldparams.BLSPubkeyLength]byte, error) {
return s.PublicKey, nil
}
@@ -437,3 +440,13 @@ func (s *ChainService) HeadSyncSelectionProofDomain(_ context.Context, _ types.S
func (s *ChainService) HeadSyncContributionProofDomain(_ context.Context, _ types.Slot) ([]byte, error) {
return s.SyncContributionProofDomain, nil
}
// IsOptimistic mocks the same method in the chain service.
func (s *ChainService) IsOptimistic(_ context.Context) (bool, error) {
return s.Optimistic, nil
}
// IsOptimisticForRoot mocks the same method in the chain service.
func (s *ChainService) IsOptimisticForRoot(_ context.Context, _ [32]byte) (bool, error) {
return s.Optimistic, nil
}

View File

@@ -4,57 +4,97 @@ import (
"context"
"fmt"
"github.com/pkg/errors"
types "github.com/prysmaticlabs/eth2-types"
"github.com/prysmaticlabs/prysm/beacon-chain/db/filters"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/time/slots"
)
// VerifyWeakSubjectivityRoot verifies the weak subjectivity root in the service struct.
var errWSBlockNotFound = errors.New("weak subjectivity root not found in db")
var errWSBlockNotFoundInEpoch = errors.New("weak subjectivity root not found in db within epoch")
type weakSubjectivityDB interface {
HasBlock(ctx context.Context, blockRoot [32]byte) bool
BlockRoots(ctx context.Context, f *filters.QueryFilter) ([][32]byte, error)
}
type WeakSubjectivityVerifier struct {
enabled bool
verified bool
root [32]byte
epoch types.Epoch
slot types.Slot
db weakSubjectivityDB
}
// NewWeakSubjectivityVerifier validates a checkpoint, and if valid, uses it to initialize a weak subjectivity verifier
func NewWeakSubjectivityVerifier(wsc *ethpb.Checkpoint, db weakSubjectivityDB) (*WeakSubjectivityVerifier, error) {
// TODO(7342): Weak subjectivity checks are currently optional. When we require the flag to be specified
// per 7342, a nil checkpoint, zero-root or zero-epoch should all fail validation
// and return an error instead of creating a WeakSubjectivityVerifier that permits any chain history.
if wsc == nil || len(wsc.Root) == 0 || wsc.Epoch == 0 {
log.Warn("No valid weak subjectivity checkpoint specified, running without weak subjectivity verification")
return &WeakSubjectivityVerifier{
enabled: false,
}, nil
}
startSlot, err := slots.EpochStart(wsc.Epoch)
if err != nil {
return nil, err
}
return &WeakSubjectivityVerifier{
enabled: true,
verified: false,
root: bytesutil.ToBytes32(wsc.Root),
epoch: wsc.Epoch,
db: db,
slot: startSlot,
}, nil
}
// VerifyWeakSubjectivity verifies the weak subjectivity root in the service struct.
// Reference design: https://github.com/ethereum/consensus-specs/blob/master/specs/phase0/weak-subjectivity.md#weak-subjectivity-sync-procedure
func (s *Service) VerifyWeakSubjectivityRoot(ctx context.Context) error {
// TODO(7342): Remove the following to fully use weak subjectivity in production.
if s.cfg.WeakSubjectivityCheckpt == nil || len(s.cfg.WeakSubjectivityCheckpt.Root) == 0 || s.cfg.WeakSubjectivityCheckpt.Epoch == 0 {
func (v *WeakSubjectivityVerifier) VerifyWeakSubjectivity(ctx context.Context, finalizedEpoch types.Epoch) error {
if v.verified || !v.enabled {
return nil
}
// Do nothing if the weak subjectivity has previously been verified,
// or weak subjectivity epoch is higher than last finalized epoch.
if s.wsVerified {
return nil
}
if s.cfg.WeakSubjectivityCheckpt.Epoch > s.finalizedCheckpt.Epoch {
// Two conditions are described in the specs:
// IF epoch_number > store.finalized_checkpoint.epoch,
// then ASSERT during block sync that block with root block_root
// is in the sync path at epoch epoch_number. Emit descriptive critical error if this assert fails,
// then exit client process.
// we do not handle this case ^, because we can only blocks that have been processed / are currently
// in line for finalization, we don't have the ability to look ahead. so we only satisfy the following:
// IF epoch_number <= store.finalized_checkpoint.epoch,
// then ASSERT that the block in the canonical chain at epoch epoch_number has root block_root.
// Emit descriptive critical error if this assert fails, then exit client process.
if v.epoch > finalizedEpoch {
return nil
}
log.Infof("Performing weak subjectivity check for root %#x in epoch %d", v.root, v.epoch)
r := bytesutil.ToBytes32(s.cfg.WeakSubjectivityCheckpt.Root)
log.Infof("Performing weak subjectivity check for root %#x in epoch %d", r, s.cfg.WeakSubjectivityCheckpt.Epoch)
// Save initial sync cached blocks to DB.
if err := s.cfg.BeaconDB.SaveBlocks(ctx, s.getInitSyncBlocks()); err != nil {
return err
}
// A node should have the weak subjectivity block in the DB.
if !s.cfg.BeaconDB.HasBlock(ctx, r) {
return fmt.Errorf("node does not have root in DB: %#x", r)
}
startSlot, err := slots.EpochStart(s.cfg.WeakSubjectivityCheckpt.Epoch)
if err != nil {
return err
if !v.db.HasBlock(ctx, v.root) {
return errors.Wrap(errWSBlockNotFound, fmt.Sprintf("missing root %#x", v.root))
}
endSlot := v.slot + params.BeaconConfig().SlotsPerEpoch
filter := filters.NewFilter().SetStartSlot(v.slot).SetEndSlot(endSlot)
// A node should have the weak subjectivity block corresponds to the correct epoch in the DB.
filter := filters.NewFilter().SetStartSlot(startSlot).SetEndSlot(startSlot + params.BeaconConfig().SlotsPerEpoch)
roots, err := s.cfg.BeaconDB.BlockRoots(ctx, filter)
log.Infof("Searching block roots for weak subjectivity root=%#x, between slots %d-%d", v.root, v.slot, endSlot)
roots, err := v.db.BlockRoots(ctx, filter)
if err != nil {
return err
return errors.Wrap(err, "error while retrieving block roots to verify weak subjectivity")
}
for _, root := range roots {
if r == root {
log.Info("Weak subjectivity check has passed")
s.wsVerified = true
if v.root == root {
log.Info("Weak subjectivity check has passed!!")
v.verified = true
return nil
}
}
return fmt.Errorf("node does not have root in db corresponding to epoch: %#x %d", r, s.cfg.WeakSubjectivityCheckpt.Epoch)
return errors.Wrap(errWSBlockNotFoundInEpoch, fmt.Sprintf("root=%#x, epoch=%d", v.root, v.epoch))
}

View File

@@ -4,78 +4,87 @@ import (
"context"
"testing"
"github.com/pkg/errors"
types "github.com/prysmaticlabs/eth2-types"
"github.com/prysmaticlabs/prysm/beacon-chain/blockchain/store"
testDB "github.com/prysmaticlabs/prysm/beacon-chain/db/testing"
fieldparams "github.com/prysmaticlabs/prysm/config/fieldparams"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1/wrapper"
"github.com/prysmaticlabs/prysm/testing/require"
"github.com/prysmaticlabs/prysm/testing/util"
"github.com/prysmaticlabs/prysm/time/slots"
)
func TestService_VerifyWeakSubjectivityRoot(t *testing.T) {
beaconDB := testDB.SetupDB(t)
b := util.NewBeaconBlock()
b.Block.Slot = 32
require.NoError(t, beaconDB.SaveBlock(context.Background(), wrapper.WrappedPhase0SignedBeaconBlock(b)))
b.Block.Slot = 1792480
wsb, err := wrapper.WrappedSignedBeaconBlock(b)
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(context.Background(), wsb))
r, err := b.Block.HashTreeRoot()
require.NoError(t, err)
blockEpoch := slots.ToEpoch(b.Block.Slot)
tests := []struct {
wsVerified bool
wantErr bool
disabled bool
wantErr error
checkpt *ethpb.Checkpoint
finalizedEpoch types.Epoch
errString string
name string
}{
{
name: "nil root and epoch",
wantErr: false,
},
{
name: "already verified",
checkpt: &ethpb.Checkpoint{Epoch: 2},
finalizedEpoch: 2,
wsVerified: true,
wantErr: false,
name: "nil root and epoch",
disabled: true,
},
{
name: "not yet to verify, ws epoch higher than finalized epoch",
checkpt: &ethpb.Checkpoint{Epoch: 2},
finalizedEpoch: 1,
wantErr: false,
checkpt: &ethpb.Checkpoint{Root: bytesutil.PadTo([]byte{'a'}, 32), Epoch: blockEpoch},
finalizedEpoch: blockEpoch - 1,
},
{
name: "can't find the block in DB",
checkpt: &ethpb.Checkpoint{Root: bytesutil.PadTo([]byte{'a'}, 32), Epoch: 1},
finalizedEpoch: 3,
wantErr: true,
errString: "node does not have root in DB",
checkpt: &ethpb.Checkpoint{Root: bytesutil.PadTo([]byte{'a'}, fieldparams.RootLength), Epoch: 1},
finalizedEpoch: blockEpoch + 1,
wantErr: errWSBlockNotFound,
},
{
name: "can't find the block corresponds to ws epoch in DB",
checkpt: &ethpb.Checkpoint{Root: r[:], Epoch: 2}, // Root belongs in epoch 1.
finalizedEpoch: 3,
wantErr: true,
errString: "node does not have root in db corresponding to epoch",
checkpt: &ethpb.Checkpoint{Root: r[:], Epoch: blockEpoch - 2}, // Root belongs in epoch 1.
finalizedEpoch: blockEpoch - 1,
wantErr: errWSBlockNotFoundInEpoch,
},
{
name: "can verify and pass",
checkpt: &ethpb.Checkpoint{Root: r[:], Epoch: 1},
finalizedEpoch: 3,
wantErr: false,
checkpt: &ethpb.Checkpoint{Root: r[:], Epoch: blockEpoch},
finalizedEpoch: blockEpoch + 1,
},
{
name: "equal epoch",
checkpt: &ethpb.Checkpoint{Root: r[:], Epoch: blockEpoch},
finalizedEpoch: blockEpoch,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
wv, err := NewWeakSubjectivityVerifier(tt.checkpt, beaconDB)
require.Equal(t, !tt.disabled, wv.enabled)
require.NoError(t, err)
s := &Service{
cfg: &Config{BeaconDB: beaconDB, WeakSubjectivityCheckpt: tt.checkpt},
wsVerified: tt.wsVerified,
finalizedCheckpt: &ethpb.Checkpoint{Epoch: tt.finalizedEpoch},
cfg: &config{BeaconDB: beaconDB, WeakSubjectivityCheckpt: tt.checkpt},
store: &store.Store{},
wsVerifier: wv,
}
if err := s.VerifyWeakSubjectivityRoot(context.Background()); (err != nil) != tt.wantErr {
require.ErrorContains(t, tt.errString, err)
s.store.SetFinalizedCheckpt(&ethpb.Checkpoint{Epoch: tt.finalizedEpoch})
err = s.wsVerifier.VerifyWeakSubjectivity(context.Background(), s.store.FinalizedCheckpt().Epoch)
if tt.wantErr == nil {
require.NoError(t, err)
} else {
require.Equal(t, true, errors.Is(err, tt.wantErr))
}
})
}

View File

@@ -1,44 +1,35 @@
load("@prysm//tools/go:def.bzl", "go_library", "go_test")
# gazelle:exclude committee_disabled.go
# gazelle:exclude proposer_indices_disabled.go
go_library(
name = "go_default_library",
srcs = [
"active_balance.go",
"active_balance_disabled.go", # keep
"attestation_data.go",
"checkpoint_state.go",
"committee.go",
"committee_disabled.go", # keep
"committees.go",
"common.go",
"doc.go",
"error.go",
"proposer_indices.go",
"proposer_indices_disabled.go", # keep
"proposer_indices_type.go",
"skip_slot_cache.go",
"subnet_ids.go",
"sync_committee.go",
"sync_committee_disabled.go", # keep
"sync_committee_head_state.go",
"sync_subnet_ids.go",
] + select({
"//testing/fuzz:fuzzing_enabled": [
"active_balance_disabled.go",
"committee_disabled.go",
"proposer_indices_disabled.go",
"sync_committee_disabled.go",
],
"//conditions:default": [
"active_balance.go",
"committee.go",
"proposer_indices.go",
"sync_committee.go",
],
}),
],
importpath = "github.com/prysmaticlabs/prysm/beacon-chain/cache",
visibility = [
"//beacon-chain:__subpackages__",
"//testing/fuzz:__pkg__",
"//tools:__subpackages__",
],
deps = [
"//beacon-chain/state:go_default_library",
"//beacon-chain/state/v2:go_default_library",
"//cache/lru:go_default_library",
"//config/features:go_default_library",
"//config/params:go_default_library",
@@ -48,6 +39,7 @@ go_library(
"//encoding/bytesutil:go_default_library",
"//math:go_default_library",
"//proto/prysm/v1alpha1:go_default_library",
"//runtime/version:go_default_library",
"@com_github_hashicorp_golang_lru//:go_default_library",
"@com_github_patrickmn_go_cache//:go_default_library",
"@com_github_pkg_errors//:go_default_library",
@@ -81,7 +73,9 @@ go_test(
"//beacon-chain/state:go_default_library",
"//beacon-chain/state/v1:go_default_library",
"//beacon-chain/state/v2:go_default_library",
"//beacon-chain/state/v3:go_default_library",
"//config/features:go_default_library",
"//config/fieldparams:go_default_library",
"//config/params:go_default_library",
"//encoding/bytesutil:go_default_library",
"//proto/prysm/v1alpha1:go_default_library",

View File

@@ -1,4 +1,5 @@
// +build !libfuzzer
//go:build !fuzz
// +build !fuzz
package cache
@@ -17,10 +18,12 @@ import (
"github.com/prysmaticlabs/prysm/config/params"
)
var (
const (
// maxBalanceCacheSize defines the max number of active balances can cache.
maxBalanceCacheSize = uint64(4)
maxBalanceCacheSize = int(4)
)
var (
// BalanceCacheMiss tracks the number of balance requests that aren't present in the cache.
balanceCacheMiss = promauto.NewCounter(prometheus.CounterOpts{
Name: "total_effective_balance_cache_miss",
@@ -42,7 +45,7 @@ type BalanceCache struct {
// NewEffectiveBalanceCache creates a new effective balance cache for storing/accessing total balance by epoch.
func NewEffectiveBalanceCache() *BalanceCache {
return &BalanceCache{
cache: lruwrpr.New(int(maxBalanceCacheSize)),
cache: lruwrpr.New(maxBalanceCacheSize),
}
}

View File

@@ -1,4 +1,5 @@
// +build libfuzzer
//go:build fuzz
// +build fuzz
package cache

View File

@@ -1,4 +1,5 @@
// +build !libfuzzer
//go:build !fuzz
// +build !fuzz
package cache
@@ -19,11 +20,13 @@ import (
mathutil "github.com/prysmaticlabs/prysm/math"
)
var (
const (
// maxCommitteesCacheSize defines the max number of shuffled committees on per randao basis can cache.
// Due to reorgs and long finality, it's good to keep the old cache around for quickly switch over.
maxCommitteesCacheSize = uint64(32)
maxCommitteesCacheSize = int(32)
)
var (
// CommitteeCacheMiss tracks the number of committee requests that aren't present in the cache.
CommitteeCacheMiss = promauto.NewCounter(prometheus.CounterOpts{
Name: "committee_cache_miss",
@@ -55,7 +58,7 @@ func committeeKeyFn(obj interface{}) (string, error) {
// NewCommitteesCache creates a new committee cache for storing/accessing shuffled indices of a committee.
func NewCommitteesCache() *CommitteeCache {
return &CommitteeCache{
CommitteeCache: lruwrpr.New(int(maxCommitteesCacheSize)),
CommitteeCache: lruwrpr.New(maxCommitteesCacheSize),
inProgress: make(map[string]bool),
}
}

View File

@@ -1,4 +1,5 @@
// +build libfuzzer
//go:build fuzz
// +build fuzz
// This file is used in fuzzer builds to bypass global committee caches.
package cache

View File

@@ -33,7 +33,7 @@ func TestCommitteeCache_FuzzCommitteesByEpoch(t *testing.T) {
require.NoError(t, err)
}
assert.Equal(t, maxCommitteesCacheSize, uint64(len(cache.CommitteeCache.Keys())), "Incorrect key size")
assert.Equal(t, maxCommitteesCacheSize, len(cache.CommitteeCache.Keys()), "Incorrect key size")
}
func TestCommitteeCache_FuzzActiveIndices(t *testing.T) {
@@ -50,5 +50,5 @@ func TestCommitteeCache_FuzzActiveIndices(t *testing.T) {
assert.DeepEqual(t, c.SortedIndices, indices)
}
assert.Equal(t, maxCommitteesCacheSize, uint64(len(cache.CommitteeCache.Keys())), "Incorrect key size")
assert.Equal(t, maxCommitteesCacheSize, len(cache.CommitteeCache.Keys()), "Incorrect key size")
}

View File

@@ -102,7 +102,7 @@ func TestCommitteeCache_CanRotate(t *testing.T) {
}
k := cache.CommitteeCache.Keys()
assert.Equal(t, maxCommitteesCacheSize, uint64(len(k)))
assert.Equal(t, maxCommitteesCacheSize, len(k))
sort.Slice(k, func(i, j int) bool {
return k[i].(string) < k[j].(string)

View File

@@ -8,8 +8,12 @@ go_library(
"pending_deposits.go",
],
importpath = "github.com/prysmaticlabs/prysm/beacon-chain/cache/depositcache",
visibility = ["//beacon-chain:__subpackages__"],
visibility = [
"//beacon-chain:__subpackages__",
"//testing/spectest:__subpackages__",
],
deps = [
"//config/fieldparams:go_default_library",
"//config/params:go_default_library",
"//container/trie:go_default_library",
"//crypto/hash:go_default_library",

View File

@@ -5,7 +5,6 @@
package depositcache
import (
"bytes"
"context"
"encoding/hex"
"math/big"
@@ -15,10 +14,10 @@ import (
"github.com/pkg/errors"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promauto"
fieldparams "github.com/prysmaticlabs/prysm/config/fieldparams"
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/container/trie"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
dbpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/sirupsen/logrus"
"go.opencensus.io/trace"
@@ -51,9 +50,10 @@ type FinalizedDeposits struct {
// stores all the deposit related data that is required by the beacon-node.
type DepositCache struct {
// Beacon chain deposits in memory.
pendingDeposits []*dbpb.DepositContainer
deposits []*dbpb.DepositContainer
pendingDeposits []*ethpb.DepositContainer
deposits []*ethpb.DepositContainer
finalizedDeposits *FinalizedDeposits
depositsByKey map[[fieldparams.BLSPubkeyLength]byte][]*ethpb.DepositContainer
depositsLock sync.RWMutex
}
@@ -67,8 +67,9 @@ func New() (*DepositCache, error) {
// finalizedDeposits.MerkleTrieIndex is initialized to -1 because it represents the index of the last trie item.
// Inserting the first item into the trie will set the value of the index to 0.
return &DepositCache{
pendingDeposits: []*dbpb.DepositContainer{},
deposits: []*dbpb.DepositContainer{},
pendingDeposits: []*ethpb.DepositContainer{},
deposits: []*ethpb.DepositContainer{},
depositsByKey: map[[fieldparams.BLSPubkeyLength]byte][]*ethpb.DepositContainer{},
finalizedDeposits: &FinalizedDeposits{Deposits: finalizedDepositsTrie, MerkleTrieIndex: -1},
}, nil
}
@@ -76,7 +77,7 @@ func New() (*DepositCache, error) {
// InsertDeposit into the database. If deposit or block number are nil
// then this method does nothing.
func (dc *DepositCache) InsertDeposit(ctx context.Context, d *ethpb.Deposit, blockNum uint64, index int64, depositRoot [32]byte) error {
ctx, span := trace.StartSpan(ctx, "DepositsCache.InsertDeposit")
_, span := trace.StartSpan(ctx, "DepositsCache.InsertDeposit")
defer span.End()
if d == nil {
log.WithFields(logrus.Fields{
@@ -95,29 +96,41 @@ func (dc *DepositCache) InsertDeposit(ctx context.Context, d *ethpb.Deposit, blo
}
// Keep the slice sorted on insertion in order to avoid costly sorting on retrieval.
heightIdx := sort.Search(len(dc.deposits), func(i int) bool { return dc.deposits[i].Index >= index })
depCtr := &ethpb.DepositContainer{Deposit: d, Eth1BlockHeight: blockNum, DepositRoot: depositRoot[:], Index: index}
newDeposits := append(
[]*dbpb.DepositContainer{{Deposit: d, Eth1BlockHeight: blockNum, DepositRoot: depositRoot[:], Index: index}},
[]*ethpb.DepositContainer{depCtr},
dc.deposits[heightIdx:]...)
dc.deposits = append(dc.deposits[:heightIdx], newDeposits...)
// Append the deposit to our map, in the event no deposits
// exist for the pubkey , it is simply added to the map.
pubkey := bytesutil.ToBytes48(d.Data.PublicKey)
dc.depositsByKey[pubkey] = append(dc.depositsByKey[pubkey], depCtr)
historicalDepositsCount.Inc()
return nil
}
// InsertDepositContainers inserts a set of deposit containers into our deposit cache.
func (dc *DepositCache) InsertDepositContainers(ctx context.Context, ctrs []*dbpb.DepositContainer) {
ctx, span := trace.StartSpan(ctx, "DepositsCache.InsertDepositContainers")
func (dc *DepositCache) InsertDepositContainers(ctx context.Context, ctrs []*ethpb.DepositContainer) {
_, span := trace.StartSpan(ctx, "DepositsCache.InsertDepositContainers")
defer span.End()
dc.depositsLock.Lock()
defer dc.depositsLock.Unlock()
sort.SliceStable(ctrs, func(i int, j int) bool { return ctrs[i].Index < ctrs[j].Index })
dc.deposits = ctrs
for _, c := range ctrs {
// Use a new value, as the reference
// of c changes in the next iteration.
newPtr := c
pKey := bytesutil.ToBytes48(newPtr.Deposit.Data.PublicKey)
dc.depositsByKey[pKey] = append(dc.depositsByKey[pKey], newPtr)
}
historicalDepositsCount.Add(float64(len(ctrs)))
}
// InsertFinalizedDeposits inserts deposits up to eth1DepositIndex (inclusive) into the finalized deposits cache.
func (dc *DepositCache) InsertFinalizedDeposits(ctx context.Context, eth1DepositIndex int64) {
ctx, span := trace.StartSpan(ctx, "DepositsCache.InsertFinalizedDeposits")
_, span := trace.StartSpan(ctx, "DepositsCache.InsertFinalizedDeposits")
defer span.End()
dc.depositsLock.Lock()
defer dc.depositsLock.Unlock()
@@ -136,7 +149,10 @@ func (dc *DepositCache) InsertFinalizedDeposits(ctx context.Context, eth1Deposit
log.WithError(err).Error("Could not hash deposit data. Finalized deposit cache not updated.")
return
}
depositTrie.Insert(depHash[:], insertIndex)
if err = depositTrie.Insert(depHash[:], insertIndex); err != nil {
log.WithError(err).Error("Could not insert deposit hash")
return
}
insertIndex++
}
@@ -147,8 +163,8 @@ func (dc *DepositCache) InsertFinalizedDeposits(ctx context.Context, eth1Deposit
}
// AllDepositContainers returns all historical deposit containers.
func (dc *DepositCache) AllDepositContainers(ctx context.Context) []*dbpb.DepositContainer {
ctx, span := trace.StartSpan(ctx, "DepositsCache.AllDepositContainers")
func (dc *DepositCache) AllDepositContainers(ctx context.Context) []*ethpb.DepositContainer {
_, span := trace.StartSpan(ctx, "DepositsCache.AllDepositContainers")
defer span.End()
dc.depositsLock.RLock()
defer dc.depositsLock.RUnlock()
@@ -159,11 +175,15 @@ func (dc *DepositCache) AllDepositContainers(ctx context.Context) []*dbpb.Deposi
// AllDeposits returns a list of historical deposits until the given block number
// (inclusive). If no block is specified then this method returns all historical deposits.
func (dc *DepositCache) AllDeposits(ctx context.Context, untilBlk *big.Int) []*ethpb.Deposit {
ctx, span := trace.StartSpan(ctx, "DepositsCache.AllDeposits")
_, span := trace.StartSpan(ctx, "DepositsCache.AllDeposits")
defer span.End()
dc.depositsLock.RLock()
defer dc.depositsLock.RUnlock()
return dc.allDeposits(untilBlk)
}
func (dc *DepositCache) allDeposits(untilBlk *big.Int) []*ethpb.Deposit {
var deposits []*ethpb.Deposit
for _, ctnr := range dc.deposits {
if untilBlk == nil || untilBlk.Uint64() >= ctnr.Eth1BlockHeight {
@@ -176,7 +196,7 @@ func (dc *DepositCache) AllDeposits(ctx context.Context, untilBlk *big.Int) []*e
// DepositsNumberAndRootAtHeight returns number of deposits made up to blockheight and the
// root that corresponds to the latest deposit at that blockheight.
func (dc *DepositCache) DepositsNumberAndRootAtHeight(ctx context.Context, blockHeight *big.Int) (uint64, [32]byte) {
ctx, span := trace.StartSpan(ctx, "DepositsCache.DepositsNumberAndRootAtHeight")
_, span := trace.StartSpan(ctx, "DepositsCache.DepositsNumberAndRootAtHeight")
defer span.End()
dc.depositsLock.RLock()
defer dc.depositsLock.RUnlock()
@@ -192,26 +212,28 @@ func (dc *DepositCache) DepositsNumberAndRootAtHeight(ctx context.Context, block
// DepositByPubkey looks through historical deposits and finds one which contains
// a certain public key within its deposit data.
func (dc *DepositCache) DepositByPubkey(ctx context.Context, pubKey []byte) (*ethpb.Deposit, *big.Int) {
ctx, span := trace.StartSpan(ctx, "DepositsCache.DepositByPubkey")
_, span := trace.StartSpan(ctx, "DepositsCache.DepositByPubkey")
defer span.End()
dc.depositsLock.RLock()
defer dc.depositsLock.RUnlock()
var deposit *ethpb.Deposit
var blockNum *big.Int
for _, ctnr := range dc.deposits {
if bytes.Equal(ctnr.Deposit.Data.PublicKey, pubKey) {
deposit = ctnr.Deposit
blockNum = big.NewInt(int64(ctnr.Eth1BlockHeight))
break
}
deps, ok := dc.depositsByKey[bytesutil.ToBytes48(pubKey)]
if !ok || len(deps) == 0 {
return deposit, blockNum
}
// We always return the first deposit if a particular
// validator key has multiple deposits assigned to
// it.
deposit = deps[0].Deposit
blockNum = big.NewInt(int64(deps[0].Eth1BlockHeight))
return deposit, blockNum
}
// FinalizedDeposits returns the finalized deposits trie.
func (dc *DepositCache) FinalizedDeposits(ctx context.Context) *FinalizedDeposits {
ctx, span := trace.StartSpan(ctx, "DepositsCache.FinalizedDeposits")
_, span := trace.StartSpan(ctx, "DepositsCache.FinalizedDeposits")
defer span.End()
dc.depositsLock.RLock()
defer dc.depositsLock.RUnlock()
@@ -231,7 +253,7 @@ func (dc *DepositCache) NonFinalizedDeposits(ctx context.Context, untilBlk *big.
defer dc.depositsLock.RUnlock()
if dc.finalizedDeposits == nil {
return dc.AllDeposits(ctx, untilBlk)
return dc.allDeposits(untilBlk)
}
lastFinalizedDepositIndex := dc.finalizedDeposits.MerkleTrieIndex

View File

@@ -10,7 +10,6 @@ import (
"github.com/prysmaticlabs/prysm/config/params"
"github.com/prysmaticlabs/prysm/container/trie"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
dbpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/testing/assert"
"github.com/prysmaticlabs/prysm/testing/require"
@@ -44,31 +43,31 @@ func TestInsertDeposit_MaintainsSortedOrderByIndex(t *testing.T) {
}{
{
blkNum: 0,
deposit: &ethpb.Deposit{},
deposit: &ethpb.Deposit{Data: &ethpb.Deposit_Data{PublicKey: []byte{'A'}}},
index: 0,
expectedErr: "",
},
{
blkNum: 0,
deposit: &ethpb.Deposit{},
deposit: &ethpb.Deposit{Data: &ethpb.Deposit_Data{PublicKey: []byte{'B'}}},
index: 3,
expectedErr: "wanted deposit with index 1 to be inserted but received 3",
},
{
blkNum: 0,
deposit: &ethpb.Deposit{},
deposit: &ethpb.Deposit{Data: &ethpb.Deposit_Data{PublicKey: []byte{'C'}}},
index: 1,
expectedErr: "",
},
{
blkNum: 0,
deposit: &ethpb.Deposit{},
deposit: &ethpb.Deposit{Data: &ethpb.Deposit_Data{PublicKey: []byte{'D'}}},
index: 4,
expectedErr: "wanted deposit with index 2 to be inserted but received 4",
},
{
blkNum: 0,
deposit: &ethpb.Deposit{},
deposit: &ethpb.Deposit{Data: &ethpb.Deposit_Data{PublicKey: []byte{'E'}}},
index: 2,
expectedErr: "",
},
@@ -93,7 +92,7 @@ func TestAllDeposits_ReturnsAllDeposits(t *testing.T) {
dc, err := New()
require.NoError(t, err)
deposits := []*dbpb.DepositContainer{
deposits := []*ethpb.DepositContainer{
{
Eth1BlockHeight: 10,
Deposit: &ethpb.Deposit{},
@@ -133,7 +132,7 @@ func TestAllDeposits_FiltersDepositUpToAndIncludingBlockNumber(t *testing.T) {
dc, err := New()
require.NoError(t, err)
deposits := []*dbpb.DepositContainer{
deposits := []*ethpb.DepositContainer{
{
Eth1BlockHeight: 10,
Deposit: &ethpb.Deposit{},
@@ -174,7 +173,7 @@ func TestDepositsNumberAndRootAtHeight(t *testing.T) {
t.Run("requesting_last_item_works", func(t *testing.T) {
dc, err := New()
require.NoError(t, err)
dc.deposits = []*dbpb.DepositContainer{
dc.deposits = []*ethpb.DepositContainer{
{
Eth1BlockHeight: 10,
Index: 0,
@@ -205,7 +204,7 @@ func TestDepositsNumberAndRootAtHeight(t *testing.T) {
dc, err := New()
require.NoError(t, err)
dc.deposits = []*dbpb.DepositContainer{
dc.deposits = []*ethpb.DepositContainer{
{
Eth1BlockHeight: 10,
Index: 0,
@@ -221,7 +220,7 @@ func TestDepositsNumberAndRootAtHeight(t *testing.T) {
dc, err := New()
require.NoError(t, err)
dc.deposits = []*dbpb.DepositContainer{
dc.deposits = []*ethpb.DepositContainer{
{
Eth1BlockHeight: 8,
Index: 0,
@@ -247,7 +246,7 @@ func TestDepositsNumberAndRootAtHeight(t *testing.T) {
dc, err := New()
require.NoError(t, err)
dc.deposits = []*dbpb.DepositContainer{
dc.deposits = []*ethpb.DepositContainer{
{
Eth1BlockHeight: 8,
Index: 0,
@@ -263,7 +262,7 @@ func TestDepositsNumberAndRootAtHeight(t *testing.T) {
dc, err := New()
require.NoError(t, err)
dc.deposits = []*dbpb.DepositContainer{
dc.deposits = []*ethpb.DepositContainer{
{
Eth1BlockHeight: 8,
Index: 0,
@@ -279,7 +278,7 @@ func TestDepositsNumberAndRootAtHeight(t *testing.T) {
dc, err := New()
require.NoError(t, err)
dc.deposits = []*dbpb.DepositContainer{
dc.deposits = []*ethpb.DepositContainer{
{
Eth1BlockHeight: 8,
Index: 0,
@@ -316,8 +315,7 @@ func TestDepositsNumberAndRootAtHeight(t *testing.T) {
func TestDepositByPubkey_ReturnsFirstMatchingDeposit(t *testing.T) {
dc, err := New()
require.NoError(t, err)
dc.deposits = []*dbpb.DepositContainer{
ctrs := []*ethpb.DepositContainer{
{
Eth1BlockHeight: 9,
Deposit: &ethpb.Deposit{
@@ -359,6 +357,7 @@ func TestDepositByPubkey_ReturnsFirstMatchingDeposit(t *testing.T) {
},
},
}
dc.InsertDepositContainers(context.Background(), ctrs)
pk1 := bytesutil.PadTo([]byte("pk1"), 48)
dep, blkNum := dc.DepositByPubkey(context.Background(), pk1)
@@ -374,7 +373,7 @@ func TestFinalizedDeposits_DepositsCachedCorrectly(t *testing.T) {
dc, err := New()
require.NoError(t, err)
finalizedDeposits := []*dbpb.DepositContainer{
finalizedDeposits := []*ethpb.DepositContainer{
{
Deposit: &ethpb.Deposit{
Data: &ethpb.Deposit_Data{
@@ -406,7 +405,7 @@ func TestFinalizedDeposits_DepositsCachedCorrectly(t *testing.T) {
Index: 2,
},
}
dc.deposits = append(finalizedDeposits, &dbpb.DepositContainer{
dc.deposits = append(finalizedDeposits, &ethpb.DepositContainer{
Deposit: &ethpb.Deposit{
Data: &ethpb.Deposit_Data{
PublicKey: bytesutil.PadTo([]byte{3}, 48),
@@ -438,7 +437,7 @@ func TestFinalizedDeposits_UtilizesPreviouslyCachedDeposits(t *testing.T) {
dc, err := New()
require.NoError(t, err)
oldFinalizedDeposits := []*dbpb.DepositContainer{
oldFinalizedDeposits := []*ethpb.DepositContainer{
{
Deposit: &ethpb.Deposit{
Data: &ethpb.Deposit_Data{
@@ -460,7 +459,7 @@ func TestFinalizedDeposits_UtilizesPreviouslyCachedDeposits(t *testing.T) {
Index: 1,
},
}
newFinalizedDeposit := dbpb.DepositContainer{
newFinalizedDeposit := ethpb.DepositContainer{
Deposit: &ethpb.Deposit{
Data: &ethpb.Deposit_Data{
PublicKey: bytesutil.PadTo([]byte{2}, 48),
@@ -473,7 +472,7 @@ func TestFinalizedDeposits_UtilizesPreviouslyCachedDeposits(t *testing.T) {
dc.deposits = oldFinalizedDeposits
dc.InsertFinalizedDeposits(context.Background(), 1)
// Artificially exclude old deposits so that they can only be retrieved from previously finalized deposits.
dc.deposits = []*dbpb.DepositContainer{&newFinalizedDeposit}
dc.deposits = []*ethpb.DepositContainer{&newFinalizedDeposit}
dc.InsertFinalizedDeposits(context.Background(), 2)
@@ -506,7 +505,7 @@ func TestNonFinalizedDeposits_ReturnsAllNonFinalizedDeposits(t *testing.T) {
dc, err := New()
require.NoError(t, err)
finalizedDeposits := []*dbpb.DepositContainer{
finalizedDeposits := []*ethpb.DepositContainer{
{
Eth1BlockHeight: 10,
Deposit: &ethpb.Deposit{
@@ -531,7 +530,7 @@ func TestNonFinalizedDeposits_ReturnsAllNonFinalizedDeposits(t *testing.T) {
},
}
dc.deposits = append(finalizedDeposits,
&dbpb.DepositContainer{
&ethpb.DepositContainer{
Eth1BlockHeight: 10,
Deposit: &ethpb.Deposit{
Data: &ethpb.Deposit_Data{
@@ -542,7 +541,7 @@ func TestNonFinalizedDeposits_ReturnsAllNonFinalizedDeposits(t *testing.T) {
},
Index: 2,
},
&dbpb.DepositContainer{
&ethpb.DepositContainer{
Eth1BlockHeight: 11,
Deposit: &ethpb.Deposit{
Data: &ethpb.Deposit_Data{
@@ -563,7 +562,7 @@ func TestNonFinalizedDeposits_ReturnsNonFinalizedDepositsUpToBlockNumber(t *test
dc, err := New()
require.NoError(t, err)
finalizedDeposits := []*dbpb.DepositContainer{
finalizedDeposits := []*ethpb.DepositContainer{
{
Eth1BlockHeight: 10,
Deposit: &ethpb.Deposit{
@@ -588,7 +587,7 @@ func TestNonFinalizedDeposits_ReturnsNonFinalizedDepositsUpToBlockNumber(t *test
},
}
dc.deposits = append(finalizedDeposits,
&dbpb.DepositContainer{
&ethpb.DepositContainer{
Eth1BlockHeight: 10,
Deposit: &ethpb.Deposit{
Data: &ethpb.Deposit_Data{
@@ -599,7 +598,7 @@ func TestNonFinalizedDeposits_ReturnsNonFinalizedDepositsUpToBlockNumber(t *test
},
Index: 2,
},
&dbpb.DepositContainer{
&ethpb.DepositContainer{
Eth1BlockHeight: 11,
Deposit: &ethpb.Deposit{
Data: &ethpb.Deposit_Data{
@@ -626,24 +625,28 @@ func TestPruneProofs_Ok(t *testing.T) {
index int64
}{
{
blkNum: 0,
deposit: &ethpb.Deposit{Proof: makeDepositProof()},
index: 0,
blkNum: 0,
deposit: &ethpb.Deposit{Proof: makeDepositProof(),
Data: &ethpb.Deposit_Data{PublicKey: bytesutil.PadTo([]byte("pk0"), 48)}},
index: 0,
},
{
blkNum: 0,
deposit: &ethpb.Deposit{Proof: makeDepositProof()},
index: 1,
blkNum: 0,
deposit: &ethpb.Deposit{Proof: makeDepositProof(),
Data: &ethpb.Deposit_Data{PublicKey: bytesutil.PadTo([]byte("pk1"), 48)}},
index: 1,
},
{
blkNum: 0,
deposit: &ethpb.Deposit{Proof: makeDepositProof()},
index: 2,
blkNum: 0,
deposit: &ethpb.Deposit{Proof: makeDepositProof(),
Data: &ethpb.Deposit_Data{PublicKey: bytesutil.PadTo([]byte("pk2"), 48)}},
index: 2,
},
{
blkNum: 0,
deposit: &ethpb.Deposit{Proof: makeDepositProof()},
index: 3,
blkNum: 0,
deposit: &ethpb.Deposit{Proof: makeDepositProof(),
Data: &ethpb.Deposit_Data{PublicKey: bytesutil.PadTo([]byte("pk3"), 48)}},
index: 3,
},
}
@@ -669,24 +672,26 @@ func TestPruneProofs_SomeAlreadyPruned(t *testing.T) {
index int64
}{
{
blkNum: 0,
deposit: &ethpb.Deposit{Proof: nil},
index: 0,
blkNum: 0,
deposit: &ethpb.Deposit{Proof: nil, Data: &ethpb.Deposit_Data{
PublicKey: bytesutil.PadTo([]byte("pk0"), 48)}},
index: 0,
},
{
blkNum: 0,
deposit: &ethpb.Deposit{Proof: nil, Data: &ethpb.Deposit_Data{
PublicKey: bytesutil.PadTo([]byte("pk1"), 48)}}, index: 1,
},
{
blkNum: 0,
deposit: &ethpb.Deposit{Proof: nil},
index: 1,
},
{
blkNum: 0,
deposit: &ethpb.Deposit{Proof: makeDepositProof()},
deposit: &ethpb.Deposit{Proof: makeDepositProof(), Data: &ethpb.Deposit_Data{PublicKey: bytesutil.PadTo([]byte("pk2"), 48)}},
index: 2,
},
{
blkNum: 0,
deposit: &ethpb.Deposit{Proof: makeDepositProof()},
index: 3,
blkNum: 0,
deposit: &ethpb.Deposit{Proof: makeDepositProof(),
Data: &ethpb.Deposit_Data{PublicKey: bytesutil.PadTo([]byte("pk3"), 48)}},
index: 3,
},
}
@@ -709,24 +714,28 @@ func TestPruneProofs_PruneAllWhenDepositIndexTooBig(t *testing.T) {
index int64
}{
{
blkNum: 0,
deposit: &ethpb.Deposit{Proof: makeDepositProof()},
index: 0,
blkNum: 0,
deposit: &ethpb.Deposit{Proof: makeDepositProof(),
Data: &ethpb.Deposit_Data{PublicKey: bytesutil.PadTo([]byte("pk0"), 48)}},
index: 0,
},
{
blkNum: 0,
deposit: &ethpb.Deposit{Proof: makeDepositProof()},
index: 1,
blkNum: 0,
deposit: &ethpb.Deposit{Proof: makeDepositProof(),
Data: &ethpb.Deposit_Data{PublicKey: bytesutil.PadTo([]byte("pk1"), 48)}},
index: 1,
},
{
blkNum: 0,
deposit: &ethpb.Deposit{Proof: makeDepositProof()},
index: 2,
blkNum: 0,
deposit: &ethpb.Deposit{Proof: makeDepositProof(),
Data: &ethpb.Deposit_Data{PublicKey: bytesutil.PadTo([]byte("pk2"), 48)}},
index: 2,
},
{
blkNum: 0,
deposit: &ethpb.Deposit{Proof: makeDepositProof()},
index: 3,
blkNum: 0,
deposit: &ethpb.Deposit{Proof: makeDepositProof(),
Data: &ethpb.Deposit_Data{PublicKey: bytesutil.PadTo([]byte("pk3"), 48)}},
index: 3,
},
}
@@ -752,24 +761,28 @@ func TestPruneProofs_CorrectlyHandleLastIndex(t *testing.T) {
index int64
}{
{
blkNum: 0,
deposit: &ethpb.Deposit{Proof: makeDepositProof()},
index: 0,
blkNum: 0,
deposit: &ethpb.Deposit{Proof: makeDepositProof(),
Data: &ethpb.Deposit_Data{PublicKey: bytesutil.PadTo([]byte("pk0"), 48)}},
index: 0,
},
{
blkNum: 0,
deposit: &ethpb.Deposit{Proof: makeDepositProof()},
index: 1,
blkNum: 0,
deposit: &ethpb.Deposit{Proof: makeDepositProof(),
Data: &ethpb.Deposit_Data{PublicKey: bytesutil.PadTo([]byte("pk1"), 48)}},
index: 1,
},
{
blkNum: 0,
deposit: &ethpb.Deposit{Proof: makeDepositProof()},
index: 2,
blkNum: 0,
deposit: &ethpb.Deposit{Proof: makeDepositProof(),
Data: &ethpb.Deposit_Data{PublicKey: bytesutil.PadTo([]byte("pk2"), 48)}},
index: 2,
},
{
blkNum: 0,
deposit: &ethpb.Deposit{Proof: makeDepositProof()},
index: 3,
blkNum: 0,
deposit: &ethpb.Deposit{Proof: makeDepositProof(),
Data: &ethpb.Deposit_Data{PublicKey: bytesutil.PadTo([]byte("pk3"), 48)}},
index: 3,
},
}
@@ -785,6 +798,36 @@ func TestPruneProofs_CorrectlyHandleLastIndex(t *testing.T) {
assert.DeepEqual(t, [][]byte(nil), dc.deposits[3].Deposit.Proof)
}
func TestDepositMap_WorksCorrectly(t *testing.T) {
dc, err := New()
require.NoError(t, err)
pk0 := bytesutil.PadTo([]byte("pk0"), 48)
dep, _ := dc.DepositByPubkey(context.Background(), pk0)
var nilDep *ethpb.Deposit
assert.DeepEqual(t, nilDep, dep)
dep = &ethpb.Deposit{Proof: makeDepositProof(), Data: &ethpb.Deposit_Data{PublicKey: pk0, Amount: 1000}}
assert.NoError(t, dc.InsertDeposit(context.Background(), dep, 1000, 0, [32]byte{}))
dep, _ = dc.DepositByPubkey(context.Background(), pk0)
assert.NotEqual(t, nilDep, dep)
assert.Equal(t, uint64(1000), dep.Data.Amount)
dep = &ethpb.Deposit{Proof: makeDepositProof(), Data: &ethpb.Deposit_Data{PublicKey: pk0, Amount: 10000}}
assert.NoError(t, dc.InsertDeposit(context.Background(), dep, 1000, 1, [32]byte{}))
// Make sure we have the same deposit returned over here.
dep, _ = dc.DepositByPubkey(context.Background(), pk0)
assert.NotEqual(t, nilDep, dep)
assert.Equal(t, uint64(1000), dep.Data.Amount)
// Make sure another key doesn't work.
pk1 := bytesutil.PadTo([]byte("pk1"), 48)
dep, _ = dc.DepositByPubkey(context.Background(), pk1)
assert.DeepEqual(t, nilDep, dep)
}
func makeDepositProof() [][]byte {
proof := make([][]byte, int(params.BeaconConfig().DepositContractTreeDepth)+1)
for i := range proof {

View File

@@ -8,7 +8,6 @@ import (
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promauto"
"github.com/prysmaticlabs/prysm/crypto/hash"
dbpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/sirupsen/logrus"
"go.opencensus.io/trace"
@@ -24,13 +23,13 @@ var (
// PendingDepositsFetcher specifically outlines a struct that can retrieve deposits
// which have not yet been included in the chain.
type PendingDepositsFetcher interface {
PendingContainers(ctx context.Context, untilBlk *big.Int) []*dbpb.DepositContainer
PendingContainers(ctx context.Context, untilBlk *big.Int) []*ethpb.DepositContainer
}
// InsertPendingDeposit into the database. If deposit or block number are nil
// then this method does nothing.
func (dc *DepositCache) InsertPendingDeposit(ctx context.Context, d *ethpb.Deposit, blockNum uint64, index int64, depositRoot [32]byte) {
ctx, span := trace.StartSpan(ctx, "DepositsCache.InsertPendingDeposit")
_, span := trace.StartSpan(ctx, "DepositsCache.InsertPendingDeposit")
defer span.End()
if d == nil {
log.WithFields(logrus.Fields{
@@ -42,7 +41,7 @@ func (dc *DepositCache) InsertPendingDeposit(ctx context.Context, d *ethpb.Depos
dc.depositsLock.Lock()
defer dc.depositsLock.Unlock()
dc.pendingDeposits = append(dc.pendingDeposits,
&dbpb.DepositContainer{Deposit: d, Eth1BlockHeight: blockNum, Index: index, DepositRoot: depositRoot[:]})
&ethpb.DepositContainer{Deposit: d, Eth1BlockHeight: blockNum, Index: index, DepositRoot: depositRoot[:]})
pendingDepositsCount.Inc()
span.AddAttributes(trace.Int64Attribute("count", int64(len(dc.pendingDeposits))))
}
@@ -66,13 +65,13 @@ func (dc *DepositCache) PendingDeposits(ctx context.Context, untilBlk *big.Int)
// PendingContainers returns a list of deposit containers until the given block number
// (inclusive).
func (dc *DepositCache) PendingContainers(ctx context.Context, untilBlk *big.Int) []*dbpb.DepositContainer {
ctx, span := trace.StartSpan(ctx, "DepositsCache.PendingDeposits")
func (dc *DepositCache) PendingContainers(ctx context.Context, untilBlk *big.Int) []*ethpb.DepositContainer {
_, span := trace.StartSpan(ctx, "DepositsCache.PendingDeposits")
defer span.End()
dc.depositsLock.RLock()
defer dc.depositsLock.RUnlock()
var depositCntrs []*dbpb.DepositContainer
var depositCntrs []*ethpb.DepositContainer
for _, ctnr := range dc.pendingDeposits {
if untilBlk == nil || untilBlk.Uint64() >= ctnr.Eth1BlockHeight {
depositCntrs = append(depositCntrs, ctnr)
@@ -91,7 +90,7 @@ func (dc *DepositCache) PendingContainers(ctx context.Context, untilBlk *big.Int
// RemovePendingDeposit from the database. The deposit is indexed by the
// Index. This method does nothing if deposit ptr is nil.
func (dc *DepositCache) RemovePendingDeposit(ctx context.Context, d *ethpb.Deposit) {
ctx, span := trace.StartSpan(ctx, "DepositsCache.RemovePendingDeposit")
_, span := trace.StartSpan(ctx, "DepositsCache.RemovePendingDeposit")
defer span.End()
if d == nil {
@@ -129,7 +128,7 @@ func (dc *DepositCache) RemovePendingDeposit(ctx context.Context, d *ethpb.Depos
// PrunePendingDeposits removes any deposit which is older than the given deposit merkle tree index.
func (dc *DepositCache) PrunePendingDeposits(ctx context.Context, merkleTreeIndex int64) {
ctx, span := trace.StartSpan(ctx, "DepositsCache.PrunePendingDeposits")
_, span := trace.StartSpan(ctx, "DepositsCache.PrunePendingDeposits")
defer span.End()
if merkleTreeIndex == 0 {
@@ -140,7 +139,7 @@ func (dc *DepositCache) PrunePendingDeposits(ctx context.Context, merkleTreeInde
dc.depositsLock.Lock()
defer dc.depositsLock.Unlock()
var cleanDeposits []*dbpb.DepositContainer
var cleanDeposits []*ethpb.DepositContainer
for _, dp := range dc.pendingDeposits {
if dp.Index >= merkleTreeIndex {
cleanDeposits = append(cleanDeposits, dp)

View File

@@ -6,7 +6,6 @@ import (
"testing"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
dbpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/testing/assert"
"google.golang.org/protobuf/proto"
@@ -42,7 +41,7 @@ func TestRemovePendingDeposit_OK(t *testing.T) {
}
depToRemove := &ethpb.Deposit{Proof: proof1, Data: data}
otherDep := &ethpb.Deposit{Proof: proof2, Data: data}
db.pendingDeposits = []*dbpb.DepositContainer{
db.pendingDeposits = []*ethpb.DepositContainer{
{Deposit: depToRemove, Index: 1},
{Deposit: otherDep, Index: 5},
}
@@ -55,7 +54,7 @@ func TestRemovePendingDeposit_OK(t *testing.T) {
func TestRemovePendingDeposit_IgnoresNilDeposit(t *testing.T) {
dc := DepositCache{}
dc.pendingDeposits = []*dbpb.DepositContainer{{Deposit: &ethpb.Deposit{}}}
dc.pendingDeposits = []*ethpb.DepositContainer{{Deposit: &ethpb.Deposit{}}}
dc.RemovePendingDeposit(context.Background(), nil /*deposit*/)
assert.Equal(t, 1, len(dc.pendingDeposits), "Deposit unexpectedly removed")
}
@@ -79,7 +78,7 @@ func TestPendingDeposit_RoundTrip(t *testing.T) {
func TestPendingDeposits_OK(t *testing.T) {
dc := DepositCache{}
dc.pendingDeposits = []*dbpb.DepositContainer{
dc.pendingDeposits = []*ethpb.DepositContainer{
{Eth1BlockHeight: 2, Deposit: &ethpb.Deposit{Proof: [][]byte{[]byte("A")}}},
{Eth1BlockHeight: 4, Deposit: &ethpb.Deposit{Proof: [][]byte{[]byte("B")}}},
{Eth1BlockHeight: 6, Deposit: &ethpb.Deposit{Proof: [][]byte{[]byte("c")}}},
@@ -99,7 +98,7 @@ func TestPendingDeposits_OK(t *testing.T) {
func TestPrunePendingDeposits_ZeroMerkleIndex(t *testing.T) {
dc := DepositCache{}
dc.pendingDeposits = []*dbpb.DepositContainer{
dc.pendingDeposits = []*ethpb.DepositContainer{
{Eth1BlockHeight: 2, Index: 2},
{Eth1BlockHeight: 4, Index: 4},
{Eth1BlockHeight: 6, Index: 6},
@@ -109,7 +108,7 @@ func TestPrunePendingDeposits_ZeroMerkleIndex(t *testing.T) {
}
dc.PrunePendingDeposits(context.Background(), 0)
expected := []*dbpb.DepositContainer{
expected := []*ethpb.DepositContainer{
{Eth1BlockHeight: 2, Index: 2},
{Eth1BlockHeight: 4, Index: 4},
{Eth1BlockHeight: 6, Index: 6},
@@ -123,7 +122,7 @@ func TestPrunePendingDeposits_ZeroMerkleIndex(t *testing.T) {
func TestPrunePendingDeposits_OK(t *testing.T) {
dc := DepositCache{}
dc.pendingDeposits = []*dbpb.DepositContainer{
dc.pendingDeposits = []*ethpb.DepositContainer{
{Eth1BlockHeight: 2, Index: 2},
{Eth1BlockHeight: 4, Index: 4},
{Eth1BlockHeight: 6, Index: 6},
@@ -133,7 +132,7 @@ func TestPrunePendingDeposits_OK(t *testing.T) {
}
dc.PrunePendingDeposits(context.Background(), 6)
expected := []*dbpb.DepositContainer{
expected := []*ethpb.DepositContainer{
{Eth1BlockHeight: 6, Index: 6},
{Eth1BlockHeight: 8, Index: 8},
{Eth1BlockHeight: 10, Index: 10},
@@ -142,7 +141,7 @@ func TestPrunePendingDeposits_OK(t *testing.T) {
assert.DeepEqual(t, expected, dc.pendingDeposits)
dc.pendingDeposits = []*dbpb.DepositContainer{
dc.pendingDeposits = []*ethpb.DepositContainer{
{Eth1BlockHeight: 2, Index: 2},
{Eth1BlockHeight: 4, Index: 4},
{Eth1BlockHeight: 6, Index: 6},
@@ -152,7 +151,7 @@ func TestPrunePendingDeposits_OK(t *testing.T) {
}
dc.PrunePendingDeposits(context.Background(), 10)
expected = []*dbpb.DepositContainer{
expected = []*ethpb.DepositContainer{
{Eth1BlockHeight: 10, Index: 10},
{Eth1BlockHeight: 12, Index: 12},
}

View File

@@ -1,4 +1,5 @@
// +build !libfuzzer
//go:build !fuzz
// +build !fuzz
package cache

View File

@@ -1,4 +1,5 @@
// +build libfuzzer
//go:build fuzz
// +build fuzz
// This file is used in fuzzer builds to bypass proposer indices caches.
package cache

View File

@@ -120,26 +120,22 @@ func (c *SkipSlotCache) MarkInProgress(r [32]byte) error {
// MarkNotInProgress will release the lock on a given request. This should be
// called after put.
func (c *SkipSlotCache) MarkNotInProgress(r [32]byte) error {
func (c *SkipSlotCache) MarkNotInProgress(r [32]byte) {
if c.disabled {
return nil
return
}
c.lock.Lock()
defer c.lock.Unlock()
delete(c.inProgress, r)
return nil
}
// Put the response in the cache.
func (c *SkipSlotCache) Put(_ context.Context, r [32]byte, state state.BeaconState) error {
func (c *SkipSlotCache) Put(_ context.Context, r [32]byte, state state.BeaconState) {
if c.disabled {
return nil
return
}
// Copy state so cached value is not mutated.
c.cache.Add(r, state.Copy())
return nil
}

View File

@@ -28,8 +28,8 @@ func TestSkipSlotCache_RoundTrip(t *testing.T) {
})
require.NoError(t, err)
require.NoError(t, c.Put(ctx, r, s))
require.NoError(t, c.MarkNotInProgress(r))
c.Put(ctx, r, s)
c.MarkNotInProgress(r)
res, err := c.Get(ctx, r)
require.NoError(t, err)

View File

@@ -27,7 +27,7 @@ var SubnetIDs = newSubnetIDs()
func newSubnetIDs() *subnetIDs {
// Given a node can calculate committee assignments of current epoch and next epoch.
// Max size is set to 2 epoch length.
cacheSize := int(params.BeaconConfig().SlotsPerEpoch.Mul(params.BeaconConfig().MaxCommitteesPerSlot * 2))
cacheSize := int(params.BeaconConfig().SlotsPerEpoch.Mul(params.BeaconConfig().MaxCommitteesPerSlot * 2)) // lint:ignore uintcast -- constant values that would panic on startup if negative.
attesterCache := lruwrpr.New(cacheSize)
aggregatorCache := lruwrpr.New(cacheSize)
epochDuration := time.Duration(params.BeaconConfig().SlotsPerEpoch.Mul(params.BeaconConfig().SecondsPerSlot))

View File

@@ -4,6 +4,7 @@ import (
"testing"
types "github.com/prysmaticlabs/eth2-types"
fieldparams "github.com/prysmaticlabs/prysm/config/fieldparams"
"github.com/prysmaticlabs/prysm/testing/assert"
"github.com/prysmaticlabs/prysm/testing/require"
)
@@ -46,12 +47,12 @@ func TestSubnetIDsCache_PersistentCommitteeRoundtrip(t *testing.T) {
c := newSubnetIDs()
for i := 0; i < 20; i++ {
pubkey := [48]byte{byte(i)}
pubkey := [fieldparams.BLSPubkeyLength]byte{byte(i)}
c.AddPersistentCommittee(pubkey[:], []uint64{uint64(i)}, 0)
}
for i := uint64(0); i < 20; i++ {
pubkey := [48]byte{byte(i)}
pubkey := [fieldparams.BLSPubkeyLength]byte{byte(i)}
idxs, ok, _ := c.GetPersistentSubnets(pubkey[:])
if !ok {

View File

@@ -1,4 +1,5 @@
// +build !libfuzzer
//go:build !fuzz
// +build !fuzz
package cache

View File

@@ -1,4 +1,5 @@
// +build libfuzzer
//go:build fuzz
// +build fuzz
package cache

View File

@@ -6,8 +6,8 @@ import (
lru "github.com/hashicorp/golang-lru"
types "github.com/prysmaticlabs/eth2-types"
"github.com/prysmaticlabs/prysm/beacon-chain/state"
stateAltair "github.com/prysmaticlabs/prysm/beacon-chain/state/v2"
lruwrpr "github.com/prysmaticlabs/prysm/cache/lru"
"github.com/prysmaticlabs/prysm/runtime/version"
)
// SyncCommitteeHeadStateCache for the latest head state requested by a sync committee participant.
@@ -31,10 +31,11 @@ func (c *SyncCommitteeHeadStateCache) Put(slot types.Slot, st state.BeaconState)
if st == nil || st.IsNil() {
return ErrNilValueProvided
}
_, ok := st.(*stateAltair.BeaconState)
if !ok {
if st.Version() == version.Phase0 {
return ErrIncorrectType
}
c.cache.Add(slot, st)
return nil
}
@@ -47,9 +48,14 @@ func (c *SyncCommitteeHeadStateCache) Get(slot types.Slot) (state.BeaconState, e
if !exists {
return nil, ErrNotFound
}
st, ok := val.(*stateAltair.BeaconState)
st, ok := val.(state.BeaconState)
if !ok {
return nil, ErrIncorrectType
}
switch st.Version() {
case version.Altair, version.Bellatrix:
default:
return nil, ErrIncorrectType
}
return st, nil
}

View File

@@ -7,6 +7,7 @@ import (
"github.com/prysmaticlabs/prysm/beacon-chain/state"
v1 "github.com/prysmaticlabs/prysm/beacon-chain/state/v1"
v2 "github.com/prysmaticlabs/prysm/beacon-chain/state/v2"
v3 "github.com/prysmaticlabs/prysm/beacon-chain/state/v3"
"github.com/prysmaticlabs/prysm/config/params"
ethpb "github.com/prysmaticlabs/prysm/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/testing/require"
@@ -27,6 +28,13 @@ func TestSyncCommitteeHeadState(t *testing.T) {
},
})
require.NoError(t, err)
bellatrixState, err := v3.InitializeFromProto(&ethpb.BeaconStateBellatrix{
Fork: &ethpb.Fork{
PreviousVersion: params.BeaconConfig().GenesisForkVersion,
CurrentVersion: params.BeaconConfig().GenesisForkVersion,
},
})
require.NoError(t, err)
type put struct {
slot types.Slot
state state.BeaconState
@@ -82,6 +90,24 @@ func TestSyncCommitteeHeadState(t *testing.T) {
},
want: beaconState,
},
{
name: "not found when non-existent key in non-empty cache (bellatrix state)",
key: types.Slot(2),
put: &put{
slot: types.Slot(1),
state: bellatrixState,
},
wantErr: true,
},
{
name: "found with key (bellatrix state)",
key: types.Slot(100),
put: &put{
slot: types.Slot(100),
state: bellatrixState,
},
want: bellatrixState,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {

Some files were not shown because too many files have changed in this diff Show More