**What type of PR is this?**
Other
**What does this PR do? Why is it needed?**
This pull request removes `NUMBER_OF_COLUMNS` and
`MAX_CELLS_IN_EXTENDED_MATRIX` configuration.
**Other notes for review**
Please read commit by commit, with commit messages.
**Acknowledgements**
- [x] I have read
[CONTRIBUTING.md](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md).
- [x] I have included a uniquely named [changelog fragment
file](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md#maintaining-changelogmd).
- [x] I have added a description to this PR with sufficient context for
reviewers to understand this PR.
* PeerDAS: Implement sync
* Fix Potuz's comment.
* Fix Potuz's comment.
* Fix Potuz's comment.
* Fix Potuz's comment.
* Fix Potuz's comment.
* Implement `TestFetchDataColumnSidecarsFromPeers`.
* Implement `TestSelectPeers`.
* Fix James' comment.
* Fix flakiness in `TestSelectPeers`.
* Revert "Fix Potuz's comment."
This reverts commit c45230b455.
* Revert "Fix James' comment."
This reverts commit a3f919205a.
* `selectPeers`: Avoid map with key but empty value.
* Fix Potuz's comment.
* Add DataColumnStorage and SubscribeAllDataSubnets flag.
* getBlobsV2: retry if reconstruction isnt successful
* test: engine client and sync package, metrics
* lint: fmt and log capitalisation
* lint: return error when it is not nil
* config: make retry interval configurable
* sidecar: recover function and different context for retrying
* lint: remove unused field
* beacon: default retry interval
* reconstruct: load once, correctly deliver the result to all waiting goroutines
* reconstruct: simplify multi goroutine case and avoid race condition
* engine: remove isDataAlreadyAvailable function
* sync: no goroutine, getblobsv2 in absence of block as well, wrap error
* exec: hardcode retry interval
* da: non blocking checks
* sync: remove unwanted checks
* execution: fix test
* execution: retry atomicity test
* da: updated IsDataAvailable
* sync: remove unwanted tests
* bazel: bazel run //:gazelle -- fix
* blockchain: fix CustodyGroupCount return
* lint: formatting
* lint: lint and use unused metrics
* execution: retry logic inside ReconstructDataColumnSidecars itself
* lint: format
* execution: ensure the retry actually happens when it needs to
* execution: ensure single responsibility, execution should not do DA check
* sync: don't call ReconstructDataColumnSidecars if not required
* blockchain: move IsDataAvailable interface to blockchain package
* execution: make reconstructSingleflight part of the service struct
* blockchain: cleaner DA check
* lint: formatting and remove confusing comment
* sync: fix lint, test and add extra test for when data is actually not available
* sync: new appropriate mock service
* execution: edge case - delete activeRetries on success
* execution: use service context instead of function's for retry
* blockchain: get variable samplesPerSlot only when required
* remove redundant function and fix name
* fix test
* fix more tests
* put samplesPerSlot at appropriate place
* tidy up IsDataAvailable
* correct bad merge
* fix bad merge
* remove redundant flag option
* refactor to deduplicate sidecar construction code
* - Add godocs
- Rename some functions to be closer to the spec
- Add err in return of commitments
* Replace mutating public method (but only internally used) `Populate` but private not mutating method `extract`.
* Implement a unique `processDataColumnSidecarsFromExecution` instead 2 separate functions from block and from sidecar.
* `ReceiveBlock`: Wrap errors.
* Remove useless tests.
* `ConstructionPopulator`: Add tests.
* Fix tests
* Move functions to be consistent with blobs.
* `fetchCellsAndProofsFromExecution`: Avoid useless flattening.
* `processDataColumnSidecarsFromExecution`: Stop using DB cache.
---------
Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
* create lc cache to track branches
* save lc stuff
* remove finalized data from LC cache on finalization
* read lc stuff
* edit tests
* changelog
* linter
* address commments
* address commments 2
* address commments 3
* address commments 4
* lint
* address commments 5 x_x
* set beacon lcStore to mimick registrable services
* clean up the error propagation
* pass the state to saveLCBootstrap since it's not saved in db yet
* Swap the wrong arguments in a call
I saw that the names of the passed arguments and the ones of the
function parameters don't match, so I suspect that it's a bug.
* Add changelog
* Add validation for the fillInForkChocieMissingBlocks checkpoints.
* Add test for checkpoint epoch validation in fillInForkChoiceMissingBlocks.
* Use a sentinel error rather than error string
---------
Co-authored-by: kasey <489222+kasey@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@pvl.dev>
By default when starting a node, we load the finalized checkpoint from
db and set it as head. When the chain has not been finalizing for a
while and the user does not start from the latest head, it may still be
benefitial to start from the latest justified checkpoint that has to be
a descendant of the finalized one.
* initialize genesis data asap at node start
* add genesis validation tests with embedded state verification
* Add test for hardcoded mainnet genesis validator root and time from init() function
* Add test for UnmarshalState in encoding/ssz/detect/configfork.go
* Add tests for genesis.Initialize
* Move genesis/embedded to genesis/internal/embedded
* Gazelle / BUILD fix
* James feedback
* Fix lint
* Revert lock
---------
Co-authored-by: Kasey <kasey@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@pvl.dev>
* Convert genesis times from seconds to time.Time
* Fixing failed forkchoice tests in a new commit so it doesn't get worse
Fixing failed spectest tests in a new commit so it doesn't get worse
Fixing forkchoice tests, then spectests
* Fixing forkchoice tests, then spectests. Now asking for help...
* Fix TestForkChoice_GetProposerHead
* Fix broken build
* Resolve TODO(preston) items
* Changelog fragment
* Resolve TODO(preston) items again
* Resolve lint issues
* Use consistant field names for sinceSlotStart (no spaces)
* Manu's feedback
* Renamed StartTime -> UnsafeStartTime, marked as deprecated because it doesn't handle overflow scenarios.
Renamed SlotTime -> StartTime
Renamed SlotAt -> At
Handled the error in cases where StartTime was used.
@james-prysm feedback
* Revert beacon-chain/blockchain/receive_block_test.go from 1b7844de
* Fixing issues after rebase
* Accepted suggestions from @potuz
* Remove CanonicalHeadSlot from merge conflicts
---------
Co-authored-by: potuz <potuz@prysmaticlabs.com>
* always init service through NewService
* move head state cache to service struct
* changelog
---------
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
* Migrate Prysm repo to Offchain Labs organization ahead of Pectra upgrade v6
* Replace prysmaticlabs with OffchainLabs on general markdowns
* Update mock
* Gazelle and add mock.go to excluded generated mock file
* Return the genesis block root from last validated checkpoint if zero
When starting a node we load the last validated checkpoint. On tests or a new node this checkpoint can have the zero blockroot (it returns the finalized checkpoint). This PR ensures that it returns the genesis block root instead.
It can't affect runnning code since the root is only used at startup in `setup_forkchoice`. But it may affect tests because now `OptimisticForRoot` will error out if there is no genesis block root set on db.
* Terence review
* fix test
* Decompose Electra block attestations
* comments
* changelog <3
* remove redundant comparison
* typo in comment
* fix changelog section name
* review from James and Sammy
* continue pruning on error
* only prune when committees are cached
* fix bad assignments in test
* Add metrics for pruned proofs & pending deposits
* Add PruneAllProofs & PruneAllPendingDeposits
* Add simple unit tests
* Add DepositPruner interface
* Add pruning logic at post finalization task
* Move pruner logic into new file(deposit_pruner.go)
Rationale:
As deposit_fetcher.go contains all pruning logics, it would be better to separate its interest into fetcher/inserter/pruner.
* Gofmt
* Add reference link for deprecating eth1 polling
* Add changelog
* Apply reviews from nisdas and james
* add pre and post deposit request tests
* nishant's comment
---------
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: james-prysm <james@prysmaticlabs.com>
* organize blob directories by period and epoch
* changelog
* remove Indices and replace with Summary
* old PR feedback
* log to advise about the speed of blob migration
* rename level->layer (hoping term is more clear)
* assert path in tests for increased legibility
* lint
* lint
* remove test covering a newly impossible error
* improve feedback from flag validation failure
* Try to clean dangling dirs epoch->flat migration
* lint
* Preston feedback
* try all layouts and short-circuit if base not found
---------
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
* move flag settings out of individual tests
* add tests with better or worst old update
* remove whitespace
* add zero fields for default update at deneb block in electra
---------
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
