* init
* reverting some functions
* rolling back a change and fixing linting
* wip
* wip
* fixing test
* breaking up proofs and cells for cleaner code
* fixing test and type
* fixing safe conversion
* fixing test
* fixing more tests
* fixing even more tests
* fix the 0 indices option
* adding a test for coverage
* small test update
* changelog
* radek's suggestions
* Update beacon-chain/core/peerdas/validator.go
Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
* addressing comments on kzg package
* addressing suggestions for reconstruction
* more manu feedback items
* removing unneeded files
* removing unneeded setter
---------
Co-authored-by: james-prysm <jhe@offchainlabs.com>
Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
* Reverted all config.FuluForkEpoch = config.FarFutureEpoch from 8b6f187b15
* Fix tests by referencing electra epoch / slot values in requests and test setup
* Changelog fragment
* wip
* fixing tests
* adding script to update workspace for eth clients
* updating test sepc to 1.6.0 and fixing broadcaster test
* fix specrefs
* more ethspecify fixes
* still trying to fix ethspecify
* fixing attestation tests
* fixing sha for consensus specs
* removing script for now until i have something more standard
* fixing more p2p tests
* fixing discovery tests
* attempting to fix discovery test flakeyness
* attempting to fix port binding issue
* more attempts to fix flakey tests
* Revert "more attempts to fix flakey tests"
This reverts commit 25e8183703.
* Revert "attempting to fix port binding issue"
This reverts commit 583df8000d.
* Revert "attempting to fix discovery test flakeyness"
This reverts commit 3c76525870.
* Revert "fixing discovery tests"
This reverts commit 8c701bf3b9.
* Revert "fixing more p2p tests"
This reverts commit 140d5db203.
* Revert "fixing attestation tests"
This reverts commit 26ded244cb.
* fixing attestation tests
* fixing more p2p tests
* fixing discovery tests
* attempting to fix discovery test flakeyness
* attempting to fix port binding issue
* more attempts to fix flakey tests
* changelog
* fixing import
* adding some missing dependencies, but TestService_BroadcastAttestationWithDiscoveryAttempts is still failing
* attempting to fix test
* reverting test as it migrated to other pr
* reverting test
* fixing test from merge
* Fix `TestService_BroadcastAttestationWithDiscoveryAttempts`.
* Fix again `TestService_Start_OnlyStartsOnce`.
* fixing TestListenForNewNodes
* removing manual set of fulu epoch
* missed a few
* fixing subnet test
* Update beacon-chain/rpc/eth/config/handlers_test.go
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* removing a few more missed spots of reverting fulu epoch setting
* updating test name based on feedback
* fixing rest apis, they actually need the setting of the epoch due to the guard
---------
Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* Remove Beacon API endpoints that were deprecated in Electra
* changelog <3
* build fix
* remove more stuff
* fix post-submit e2e and remove structs
* list endpoints in the changelog
---------
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
* Move ssz_query objects into testing folder (ensuring test objects only used in test environment)
* Add containers for response
* Export sszInfo
* Add QueryBeaconState/Block
* Add comments and few refactor
* Fix merge conflict issues
* Return 500 when calculate offset fails
* Add test for QueryBeaconState
* Add test for QueryBeaconBlock
* Changelog :)
* Rename `QuerySSZRequest` to `SSZQueryRequest`
* Fix middleware hooks for RPC to accept JSON from client and return SSZ
* Convert to `SSZObject` directly from proto
* Move marshalling/calculating hash tree root part after `CalculateOffsetAndLength`
* Make nogo happy
* Add informing comment for using proto unsafe conversion
---------
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Fix `/eth/v1/beacon/blob_sidecars/` beacon API is the fulu fork epoch is set to the far future epoch.
* Fix Terence's comment.
* adding a test
---------
Co-authored-by: james-prysm <james@prysmaticlabs.com>
* Do not verify block data when calculating rewards
* remove `Get` from function names
* changelog <3
* do not verify sync committee sig in handler
* Revert "remove `Get` from function names"
This reverts commit 770a89d990.
* typo fix
---------
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
* Avoid unnecessary calls to ExitInformation()
ExitInformation runs a loop over the whole validator set. This is needed
in case that there are slashings or exits to be processed in a block (we
could be caching or avoid this entirely post-Electra though). This PR
removes these calls on normal state transition to this function. h/t to
@terencechain for finding out this bug.
In addition, on processing withdrawal requests and registry updates, we
kept recomputing the exit information at the same time that the state is
updated and the function that updates the state already takes care of
tracking and updating the right exit information. So this PR removes the
calls to compute this exit information on a loop. Notice that this bug
has been present even before we had a function `ExitInformation()` so I
will document here to help the reviewer
Our previous behavior is to do this in a loop
```
st, err = validators.InitiateValidatorExit(ctx, st, vIdx, validators.ExitInformation(st))
```
This is a bit problematic since `ExitInformation` loops over the whole validator set to compute the exit information (and the total active balance) and then the function `InitiateValidatorExit` actually recomputes the total active balance looping again over the whole validator set and overwriting the pointer returned by `ExitInformation`.
On the other hand, the funciton `InitiateValidatorExit` does mutate the state `st` itself. So each call to `ExitInformation(st)` may actually return a different pointer.
The function ExitInformation computes as follows
```
err := s.ReadFromEveryValidator(func(idx int, val state.ReadOnlyValidator) error {
e := val.ExitEpoch()
if e != farFutureEpoch {
if e > exitInfo.HighestExitEpoch {
exitInfo.HighestExitEpoch = e
exitInfo.Churn = 1
} else if e == exitInfo.HighestExitEpoch {
exitInfo.Churn++
}
```
So it simply increases the churn for each validator that has epoch equal to the highest exit epoch.
The function `InitiateValidatorExit` mutates this pointer in the following way
if the state is post-electra, it disregards completely this pointer and computes the highest exit epoch and updates churn inconditionally, so the pointer `exitInfo.HighestExitEpoch` will always have the right value and is not even neded to be computed before. We could even avoid the fist loop even. If the state is pre-Electra then the function itself updates correctly the exit info for the next iteration.
* Only care about exits pre-Electra
* Update beacon-chain/core/transition/transition_no_verify_sig.go
Co-authored-by: terence <terence@prysmaticlabs.com>
* Radek's review
---------
Co-authored-by: terence <terence@prysmaticlabs.com>
* attempting to improve duties v2
* removing go routine
* changelog
* unnessesary variable
* fixing test
* small optimization existing early on CommitteeAssignments function
* fixing small bug
* fixes performance issues with duties v2
* fixed changelog
* gofmt
* `Broadcasted data column sidecar` log: Add `blobCount`.
* `broadcastAndReceiveDataColumns`: Broadcast and receive data columns in parallel.
* `ProposeBeaconBlock`: First broadcast/receive block, and then sidecars.
* `broadcastReceiveBlock`: Add log.
* Add changelog
* Fix deadlock-option 1.
* Fix deadlock-option 2.
* Take notifier out of the critical section
* only compute common info once, for all sidecars
---------
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
* Fix misleading log msg on shutdown
gRPCServer.GracefulStop blocks until it has been shutdown. Logging
"Initiated graceful stop" after it has been completed is misleading.
Names are added to the message to discern services. Also, a minimum test
is added mainly to verify the change made with this commit.
* Add changelog fragment file
* Capitalize log messages
* Update endtoend test for fixed log messages
---------
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* PeerDAS: Implement sync
* Fix Potuz's comment.
* Fix Potuz's comment.
* Fix Potuz's comment.
* Fix Potuz's comment.
* Fix Potuz's comment.
* Implement `TestFetchDataColumnSidecarsFromPeers`.
* Implement `TestSelectPeers`.
* Fix James' comment.
* Fix flakiness in `TestSelectPeers`.
* Revert "Fix Potuz's comment."
This reverts commit c45230b455.
* Revert "Fix James' comment."
This reverts commit a3f919205a.
* `selectPeers`: Avoid map with key but empty value.
* Fix Potuz's comment.
* Add DataColumnStorage and SubscribeAllDataSubnets flag.
* getBlobsV2: retry if reconstruction isnt successful
* test: engine client and sync package, metrics
* lint: fmt and log capitalisation
* lint: return error when it is not nil
* config: make retry interval configurable
* sidecar: recover function and different context for retrying
* lint: remove unused field
* beacon: default retry interval
* reconstruct: load once, correctly deliver the result to all waiting goroutines
* reconstruct: simplify multi goroutine case and avoid race condition
* engine: remove isDataAlreadyAvailable function
* sync: no goroutine, getblobsv2 in absence of block as well, wrap error
* exec: hardcode retry interval
* da: non blocking checks
* sync: remove unwanted checks
* execution: fix test
* execution: retry atomicity test
* da: updated IsDataAvailable
* sync: remove unwanted tests
* bazel: bazel run //:gazelle -- fix
* blockchain: fix CustodyGroupCount return
* lint: formatting
* lint: lint and use unused metrics
* execution: retry logic inside ReconstructDataColumnSidecars itself
* lint: format
* execution: ensure the retry actually happens when it needs to
* execution: ensure single responsibility, execution should not do DA check
* sync: don't call ReconstructDataColumnSidecars if not required
* blockchain: move IsDataAvailable interface to blockchain package
* execution: make reconstructSingleflight part of the service struct
* blockchain: cleaner DA check
* lint: formatting and remove confusing comment
* sync: fix lint, test and add extra test for when data is actually not available
* sync: new appropriate mock service
* execution: edge case - delete activeRetries on success
* execution: use service context instead of function's for retry
* blockchain: get variable samplesPerSlot only when required
* remove redundant function and fix name
* fix test
* fix more tests
* put samplesPerSlot at appropriate place
* tidy up IsDataAvailable
* correct bad merge
* fix bad merge
* remove redundant flag option
* refactor to deduplicate sidecar construction code
* - Add godocs
- Rename some functions to be closer to the spec
- Add err in return of commitments
* Replace mutating public method (but only internally used) `Populate` but private not mutating method `extract`.
* Implement a unique `processDataColumnSidecarsFromExecution` instead 2 separate functions from block and from sidecar.
* `ReceiveBlock`: Wrap errors.
* Remove useless tests.
* `ConstructionPopulator`: Add tests.
* Fix tests
* Move functions to be consistent with blobs.
* `fetchCellsAndProofsFromExecution`: Avoid useless flattening.
* `processDataColumnSidecarsFromExecution`: Stop using DB cache.
---------
Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
* create lc cache to track branches
* save lc stuff
* remove finalized data from LC cache on finalization
* read lc stuff
* edit tests
* changelog
* linter
* address commments
* address commments 2
* address commments 3
* address commments 4
* lint
* address commments 5 x_x
* set beacon lcStore to mimick registrable services
* clean up the error propagation
* pass the state to saveLCBootstrap since it's not saved in db yet
* Calculate max epoch and churn for slashing once
* calculate once for proposer and attester slashings
* changelog <3
* introduce struct
* check if err is nil in ProcessVoluntaryExits
* rename exitData to exitInfo and return from functions
* cleanup + tests
* cleanup after rebase
* Potuz's review
* pre-calculate total active balance
* remove `slashValidatorFunc` closure
* Avoid a second validator loop
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* remove balance parameter from slashing functions
---------
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: potuz <potuz@prysmaticlabs.com>
* Fix next epoch proposer duties
* Do not update state's slot when computing the proposer
Also do not call Fulu's proposer lookahead if the requested epoch is not
current or next.
* retract Terence's test
* Fix tests
* removing epoch check to pass spec test
* reverting rollback and fixing test setup
---------
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: james-prysm <james@prysmaticlabs.com>
* propose block changes from peerdas branch
* breaking out broadcast code into its own helper, changing fulu broadcast for rest api to properly send datasidecars
* renamed validate blobsidecars to validate blobs, and added check for max blobs
* gofmt
* adding in batch verification for blobs"
* changelog
* adding kzg tests, moving new kzg functions to validation.go
* linting and other small fixes
* fixing linting issues and adding some proposer tests
* missing dependencies
* fixing test
* fixing more tests
* gaz
* removed return on broadcast data columns
* more cleanup and unit test adjustments
* missed removal of unneeded field
* adding data column receiver initialization
* Update beacon-chain/rpc/eth/beacon/handlers.go
Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
* partial review feedback from manu
* gaz
* reverting some code to peerdas as I don't believe the broadcast code needs to be reused
* missed removal of build dependency
* fixing tests and adding another test based on manu's suggestion
* fixing linting
* Update beacon-chain/rpc/eth/beacon/handlers.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update beacon-chain/blockchain/kzg/validation.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* radek's review changes
* adding missed test
---------
Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* `startBaseServices`: Warm data column storage cache.
* `TestFindPeers_NodeDeduplication`: Use `t.context`.
* `BUILD.bazel`: Moge `# gazelle.ignore` at the top of the file.
Rationale: This directive is applied to the whole file, regardless its position in the file.
* Improve `TestConstructGenericBeaconBlock`: Courtesy of Terence
* Add `TestDataColumnStoragePath_FlagSpecified`.
* `appFlags`: Move `flags.SubscribeAllDataSubnets` (cosmetic).
* `appFlags`: Add `storage.DataColumnStoragePathFlag`.
* Add changelog.
