Commit Graph

129 Commits

Author SHA1 Message Date
kasey
918129cf36 Replace statefeed Initialize (#12285)
* refactor initialization to blocking startup method

* require genesisSetter in blockchain, fix tests

* work-around gazelle weirdness

* fix dep gazelle ignores

* only call SetGenesis once

* fix typo

* validator test setup and fix to return right error

* move waitForChainStart to Start

* wire up sync Service.genesisWaiter

* fix p2p genesisWaiter plumbing

* remove extra clock type, integrate into genesis

and rename

* use time.Now when no Nower is specified

* remove unused ClockSetter

* simplify rpc context checking

* fix typo

* use clock everywhere in sync; [32]byte val root

* don't use DeepEqual to compare [32]byte and []byte

* don't use clock in init sync, not wired up yet

* use clock waiter in blockchain as well

* use cancelable contexts in tests with goroutines

* missed a reference to WithClockSetter

* Update beacon-chain/startup/genesis.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/blockchain/service_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* more clear docs

* doc for NewClock

* move clock typedef to more logical file name

* adding documentation

* gaz

* fixes for capella

* reducing test raciness

* fix races in committee cache tests

* lint

* add tests on Duration slot math helper

* startup package test coverage

* fix bad merge

* set non-zero genesis time in tests that call Start

* happy deepsource, happy me-epsource

* replace Synced event with channel

* remove unused error

* remove accidental wip commit

* gaz!

* remove unused event constants

* remove sync statefeed subscription to fix deadlock

* remove state notifier

* fix build

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
2023-05-03 04:34:01 +00:00
Potuz
b56f3e70df log correctly the head changed when it actually changes (#12324)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-04-27 17:13:58 +00:00
Preston Van Loon
3894d60282 blockchain: add more spans to analyze block production timing (#12332)
* blockchain: add more spans to analyze block production timing

* There is a reason that we do not use the incoming context for forkchoice updates
2023-04-25 15:45:08 +00:00
Nishant Das
5cbbd26df4 Fallback in Update Head On Error (#12199)
* return if head call fails

* potuz's feedback

* potuz's feedback
2023-03-28 23:06:44 +08:00
Potuz
797cc360c7 Isolate forkchoice to blockchain package (#12174)
* Isolate Forkchoice to blockchain package

* Nishant's feedback

* Terence's feedback

* Nishant saving the day
2023-03-22 09:12:54 +08:00
Potuz
3536669a70 lock forkchoice on calls to Ancestor (#12162)
Co-authored-by: terencechain <terence@prysmaticlabs.com>
2023-03-20 15:51:42 -03:00
terencechain
d17996f8b0 Update to V4 🚀 (#12134)
* Update V3 from V4

* Fix build v3 -> v4

* Update ssz

* Update beacon_chain.pb.go

* Fix formatter import

* Update update-mockgen.sh comment to v4

* Fix conflicts. Pass build and tests

* Fix test
2023-03-17 18:52:56 +00:00
Potuz
93514de00f Proposer head v5 (#12075)
* Use ShouldOverrideFCU in regular sync

* fix build

* fix tests

* add feature flag gate in updatehead

* fix rpc tests

* fix grcp tests

* deepsource

* add locks and reuse isNewProposer

* flip flag

* Fix ticker for late blocks

* implement get_proposer_head in GetBeaconBlock

* fix unit tests
2023-03-04 20:19:23 -03:00
Potuz
0a87210514 Forkchoice external locks v2 (#12036)
* write locks

* fix forkchoice tests

* blockchain locks

* lock on IsOptimistic

* use forkchoice instead of chaininfo within savehead

* Use forkchoice HasNode to check if a block is consistent with finality

* interface fix

* Use forkchoice HasNode to check if a block is consistent with finality

* interface fix

* fix tests

* remove VerifyFinalizedBlkDescendant

* don't write lock wrappers

* fix validateBeaconBlock

* Terence's review and more missing locks

* add lock for InForkChoice

* lock head on fillMissingBlockPayload

* fix lock on IsOptimisticForRoot

* fix lock in fillMissingBlockPayloadId

* extra comments

* lock proposerBoost on spectests

* nishant's review

---------

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2023-03-02 09:10:52 -03:00
Potuz
ec13d52f03 Remove VerifyFinalizedBlkDescendant (#12046)
* Use forkchoice HasNode to check if a block is consistent with finality

* interface fix

* fix tests

* remove VerifyFinalizedBlkDescendant

* fix validateBeaconBlock
2023-02-25 13:39:13 -03:00
Potuz
ae76240f83 Use forkchoice HasNode to check if a block is consistent with finality (#12040)
* Use forkchoice HasNode to check if a block is consistent with finality

* interface fix

* fix tests
2023-02-23 21:21:34 +00:00
Potuz
c7f0a94b19 Remove state balance cache (#11964)
* Remove state balance cache

* remove unused metrics

* add missing balance update

* add comment about locks

* update balance only when updataing the justified checkpoint on unrealization

* update checkpoint balances at launch

* fix unit tests

* fix remaining test

* review

* fix build file

* fix test

---------

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-02-15 14:19:46 +00:00
terencechain
c070283bf1 Call FCU with attribute on non head block (#11919)
* Call FCU with attribute on non head block

* Fix condition

* Filer save head

* Refactor

* Rm

* Add tests

* Fix race test

* Potuz feedback

* Reorder r != currentHeadRoot

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-01-31 19:17:16 +00:00
Potuz
2817f8e8d6 update head should go even without attestations (#11503)
* update head should go even without attestations

* add unit tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-09-30 13:29:18 +00:00
terencechain
e3df25f443 Fix attestations update head error logging (#11514) 2022-09-29 12:16:42 -07:00
terencechain
e1c1d0f864 Don't return out of routine when update head errors out (#11470) 2022-09-20 17:31:59 +00:00
Potuz
7a8d780869 do not return on error during on_tick (#11463) 2022-09-19 08:10:11 -03:00
terencechain
c638e114db Add new metrics (#11374)
* Better batch block warning

* New metrics

* Revert "Better batch block warning"

This reverts commit e21fcfcebe.

* More metrics

* Add activation and exit queues

* Gaz
2022-08-31 18:05:50 -04:00
Raul Jordan
d077483577 Add V3 Suffix to All Prysm Packages (#11083)
* v3 import renamings

* tidy

* fmt

* rev

* Update beacon-chain/core/epoch/precompute/reward_penalty_test.go

* Update beacon-chain/core/helpers/validators_test.go

* Update beacon-chain/db/alias.go

* Update beacon-chain/db/alias.go

* Update beacon-chain/db/alias.go

* Update beacon-chain/db/iface/BUILD.bazel

* Update beacon-chain/db/kv/kv.go

* Update beacon-chain/db/kv/state.go

* Update beacon-chain/rpc/prysm/v1alpha1/validator/attester_test.go

* Update beacon-chain/rpc/prysm/v1alpha1/validator/attester_test.go

* Update beacon-chain/sync/initial-sync/service.go

* fix deps

* fix bad replacements

* fix bad replacements

* change back

* gohashtree version

* fix deps

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-08-16 12:20:13 +00:00
terencechain
7fd2c08be3 Propagate FCU invalid error (#10997)
* Wrap fcu error

* Wrap fcu error

* Wrap error better

* More test

* Add else

* Potuz feedback

* Propagate the correct root for fcu

* Always return true for invalid

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-09 20:51:03 +00:00
Potuz
f8b4d8c57a Deprecate store in blockchain pkg (#10903)
* Deprecate store WIP

* fix spectests build

* mock right interface

* sync tests build

* more tests builds

* blockchain tests

- TestFinalizedCheckpt_GenesisRootOk
- TestCurrentJustifiedCheckpt_CanRetrieve
- TestJustifiedCheckpt_GenesisRootOk
- TestHeadRoot_CanRetrieve
- TestHeadRoot_UseDB
- TestService_ProcessAttestationsAndUpdateHead
- TestService_VerifyWeakSubjectivityRoot
- TestVerifyFinalizedConsistency_InconsistentRoot_ProtoArray
- TestVerifyFinalizedConsistency_InconsistentRoot_DoublyLinkedTree
- TestVerifyFinalizedConsistency_Ok
- TestStore_OnBlock_ProtoArray
- TestStore_OnBlock_DoublyLinkedTree
- TestStore_OnBlockBatch_ProtoArray
- TestStore_OnBlockBatch_DoublyLinkedTree
- TestStore_OnBlockBatch_NotifyNewPayload
- TestCachedPreState_CanGetFromStateSummary_ProtoArray
- TestCachedPreState_CanGetFromStateSummary_DoublyLinkedTree

* more blockchain tests

- TestStore_OnBlockBatch_PruneOK_Protoarray
- TestFillForkChoiceMissingBlocks_CanSave_ProtoArray
- TestFillForkChoiceMissingBlocks_CanSave_DoublyLinkedTree
- TestFillForkChoiceMissingBlocks_RootsMatch_Protoarray
- TestFillForkChoiceMissingBlocks_RootsMatch_DoublyLinkedTree
- TestFillForkChoiceMissingBlocks_FilterFinalized_ProtoArray
- TestFillForkChoiceMissingBlocks_FilterFinalized_DoublyLinkedTree
- TestVerifyBlkDescendant
- Test_verifyBlkFinalizedSlot_invalidBlock
- TestChainService_SaveHeadNoDB

* update best justified from genesis

* deal with nil head on saveHead

* initialize prev justified checkpoint

* update finalization correctly

* update finalization logic

* update finalization logic

* track the wall clock on forkchoice spectests

* export copies of checkpoints from blockchain package

* do not use forkchoice's head on HeadRoot

* Remove debug remain

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* terence's review

* add forkchoice types deps

* wtf

* debugging

* init-sync: update justified and finalized checkpoints on db

* call updateFinalized instead of only DB

* remove debug symbols

* safe copy headroot

Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-06-25 03:57:52 +00:00
Potuz
e439f4aff6 Forkchoice duplicate store (#10840)
* forkchoice tests

* blockchain tests

* no fatal errors

* use ZeroHash for genesis

* deal with zerohashes at genesis

* avoid nil best justified checkpoint on new store

* unit tests

* Debug log

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* Terence's review

* update capitalization

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* update capitalization

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* update capitalization

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* update capitalization

Co-authored-by: terencechain <terence@prysmaticlabs.com>

Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-06-16 18:21:40 +00:00
David
adabd1fa4f service.head race condition fix (#10741)
* added various read mutex locks for service.head

* added RLocks around all calls to s.headRoot()

* added RLocks around all calls to s.headBlock()

* reduce lock surface-> Stop(),handleEpochBoundary()

* refactor Stop() to +performance, -lock_surface

* Apply suggestions from code review

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* fixed indentation

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-05-27 14:24:43 +00:00
terencechain
370cf1a6c8 Chain info: Return err if checkpoint is nil (#10729)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-20 18:41:33 +00:00
Potuz
61033ebea1 handle failure to update head (#10651)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-12 16:36:46 +00:00
terencechain
4ad1c4df01 Cache and use justified and finalized payload block hash (#10657)
* Cache and use justified and finalized payload block hash

* Fix tests

* Use real byte

* Fix conflicts

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-10 21:20:28 +00:00
terencechain
bbdf19cfd0 Process atts and update head before proposing (#10653)
* Process atts and updeate head

* Fix ctx

* New test and old tests

* Update validator_test.go

* Update validator_test.go

* Update service.go

* Rename to UpdateHead

* Update receive_attestation.go

* Update receive_attestation.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-09 23:12:50 +00:00
terencechain
dda48c452d Move head change attestation log (#10636) 2022-05-05 15:09:58 +00:00
Potuz
025d053fa4 Change log when head changes (#10633) 2022-05-05 11:40:21 +00:00
terencechain
7af89a82c9 Use justified hash as safe block hash (#10627)
* Use justified hash as safe block hash

* Remove debug log

* Feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-05 00:36:09 +00:00
terencechain
0faf7ac01f Debug log when new head is not in db (#10591) 2022-04-29 14:26:39 -07:00
terencechain
b7a82d0fd1 Can retrieve cached initial sync block and db block (#10568)
* Save cached initial sync blocks before getting head block

* Add better abstraction to get block

* Move unlock read to a better location

* Feedbacks

* Add head changed logging

* Harder hasBlock requirement

* Update beacon-chain/blockchain/service.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update receive_attestation.go

* Don't process head if the block is unknown

* Use a helper method

* Fix test

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-04-29 07:56:31 +00:00
Radosław Kapka
2060f876b1 Code cleanup in blockchain module (#10566)
* Code cleanup in `blockchain` module

* revert comment fix

* remove test

* forkchoice test fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-04-28 11:34:25 +00:00
Potuz
56187edb98 Use forkchoice first when checking canonical status (#10516) 2022-04-12 21:34:13 -03:00
terence tsao
0411b7eceb Payload ID caching (#10481) 2022-04-06 16:36:52 -07:00
terence tsao
bdab34fd01 Use correct head for notifyForkchoiceUpdate (#10485)
* Use correct head for notifyForkchoiceUpdate

* Fix existing tests

* Update receive_attestation_test.go
2022-04-06 19:05:53 +08:00
Potuz
ce15823f8d Save head after FCU (#10466)
* Update head after FCU

* rename function

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-04-01 00:08:01 +00:00
Potuz
94f80bd208 Call FCU on update head after process attestation (#10394)
* Call FCU on update head after process attestation

* add test

* fix test

* terence review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-03-21 01:50:08 +00:00
terence tsao
8a01d412f5 Implement fork choice spec tests (#10159)
* starting

* Reimplement store

* begin on proposer boost

* implement fork choice proposer boost algorithm

* boosting

* gaz

* add mutexes and previous root

* comment on compute proposer boost

* safe

* rem todo

* reset and add tests

* unit test for proposer boost score

* boost works

* Can process block

* Basic test case passing

* ex ante

* test

* propoer test

* More progresses

* More fixes

* rm unused pieces

* Refactor, add phase 0

* locks

* vanilla ex-ante attack

* test similar to spec test

* works works works

* boost test working for num votes > proposer boost weight

* commentary fixes

* rem unused

* comments

* Proposer boost use store time

* Reset proposer root

* debugging

* passing

* Rm unused visibility imports

* Move update head to better place

* Fix deepsrc complains

* Fix more complains

* Raul's feedback

* Use correct byte lengths

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-01-31 19:03:48 +00:00
terence tsao
22982e5221 Add on_tick from spec (#10130)
* Add `on_tick` from spec

* Update BUILD.bazel
2022-01-25 22:07:57 +00:00
terence tsao
5a505509c3 Use store in blockchain service (#10125)
* Use store in blockchain service

* handle nil cases
2022-01-25 18:47:39 +00:00
kasey
3c61cc7d8a allow checkpoint or genesis origin; refactoring (#9976)
* allow checkpoint or genesis origin; refactoring

some quick readability improvements and simplifying the logic enforcing
the startup ordering of the attestation processing routine

* address PR feedback

* gofmt

* Update beacon-chain/blockchain/receive_attestation.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Apply suggestions from code review

use log.WithError for aggregation friendliness

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

Co-authored-by: kasey <kasey@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-09 22:23:00 +00:00
kasey
39c33b82ad Switch to lazy state balance cache (#9822)
* quick lazy balance cache proof of concept

* WIP refactoring to use lazy cache

* updating tests to use functional opts

* updating the rest of the tests, all passing

* use mock stategen where possible

reduces the number of test cases that require db setup

* rename test opt method for clear link

* Update beacon-chain/blockchain/process_block.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* test assumption that zerohash is in db

* remove unused MockDB (mocking stategen instead)

* fix cache bug, switch to sync.Mutex

* improve test coverage for the state cache

* uncomment failing genesis test for discussion

* gofmt

* remove unused Service struct member

* cleanup unused func input

* combining type declaration in signature

* don't export the state cache constructor

* work around blockchain deps w/ new file

service_test brings in a ton of dependencies that make bazel rules
for blockchain complex, so just sticking these mocks in their own
file simplifies things.

* gofmt

* remove intentionally failing test

this test established that the zero root can't be used to look up the
state, resulting in a change in another PR to update stategen to use the
GenesisState db method instead when the zero root is detected.

* fixed error introduced by develop refresh

* fix import ordering

* appease deepsource

* remove unused function

* godoc comments on new requires/assert

* defensive constructor per terence's PR comment

* more differentiated balance cache metric names

Co-authored-by: kasey <kasey@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-19 15:59:26 +00:00
terence tsao
c21e43e4c5 Refactor: move functions beacon-chain/core/time -> time/slots (#9719)
* Move necessary functions beacon-chain/core/time -> time/slots

* Fix fuzz

* Fix build

* Update slot_epoch.go
2021-10-01 15:17:57 -05:00
terence tsao
8219af46e4 Move slot epoch from core to time pkg (#9714)
* Move slot epoch from core to time pkg

* Fix fuzz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-30 19:00:14 +00:00
Raul Jordan
6d79f61fda Process Slashings for Newly Optimized Slasher (#9687)
* add process slashings

* verify sig

* add method

* add test

* add in process slashings functionality

* target state for aggregate

* comment

* Radek comments
2021-09-28 10:27:40 -05:00
Raul Jordan
29513c804c Create Encoding Bytesutil (#9658)
* bytesutil

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-23 15:23:37 +00:00
Raul Jordan
f3d6dbcc1e Move Shared/Params Into Config/Params (#9642)
* config params into pkg

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-21 19:59:25 +00:00
terence tsao
3e71997290 Create time Package for Shared/timeutil, mclock and slotutil (#9594)
* add time pkg

* Go fmt
2021-09-15 00:09:04 +00:00
Preston Van Loon
4beb352e6f Move Slot and Epoch functions from helpers to core pkg (#9519)
* Move Slot and Epoch functions from helpers to core

* limited viz

* goimports

* fix fuzz build

* fix fuzz build

* fix

* fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-03 20:10:31 +00:00