* `DataColumnStorageSummary`: Implement `HasAtLeastOneIndex`.
* `DataColumnStorage.Get`: Exit early if the root is found but no corresponding columns.
* `custodyColumnsFromPeers`: Simplify.
* Remove duplicate `uint64MapToSortedSlice` function.
* `DataColumnStorageSummary`: Add `Stored`.
* Refactor reconstruction related code.
* Force duties update on received blocks.
- Change the context on UpdateDuties to be passed by the calling
function.
- Change the context passed to UpdateDuties to not be dependent on a
slot context.
- Change the deadlines to be forced to be an entire epoch.
- Force duties to be initialized when receiving a HeadEvent if they
aren't already.
- Adds a read lock on the event handling
* review
* Add deadlines at start and healthyagain
* cancel once
* `finalizedState` ==> `FinalizedState`.
We'll need it in an other package later.
* `setTargetValidatorsCustodyRequirement`: Use finalized state instead of head state.
* Fix James's comment.
* Use otelgrpc for tracing grpc server and client.
* Changelog fragment
* gofmt
* Use context in prometheus service
* Remove async start of prometheus service
* Use random port to reduce the probability of concurrent tests using the same port
* Remove comment
* fix lint error
---------
Co-authored-by: Bastin <bastin.m@proton.me>
* `reconstructAndBroadcastDataColumnSidecars`: Improve logging.
* `ReconstructDataColumnSidecars`: Add comments and return early if needed.
* `reconstructAndBroadcastDataColumnSidecars`: Return early if not blobs are retrieved from the EL.
* `filterPeerWhichCustodyAtLeastOneDataColumn`: Remove unneded log field.
* Fix Terence's comment.
* Seperate subscribe data columns from attestation and sync committee subnets
* Fix test
* Rename to subscribe-data-subnets
* Update to subscribe-all-data-subnets
* `--subscribe-all-data-subnets`: Add `.` at the end of help, since it seems to be the consensus.
* `ConfigureGlobalFlags`: Fix log.
---------
Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
* include block in attr event and use stategen
* use no-copy state cache for proposal in same epoch
* only advance to the start of epoch
---------
Co-authored-by: Kasey <kasey@users.noreply.github.com>
* Add equivocation detection logic; broadcast slashing immediately on equivocation
* nit: comments
* move equivocation detection to validateBeaconBlockPubSub
* include broadcasting logic within the helper function
* fix lint
* Add unit tests for equivocation detection
* remove comment that are not required
* Add changelog file
* Add descriptive comment for detectAndBroadcastEquivocation
* use head block instead of block cache for equivocation detection
* add more equivocation unit tests; update a mock to include HeadState error
* update the order of the checks
* move slashing before state fetch; update Tests
* update changelog
* use verifyProposerSlashing to verify and reject block; remove verifySlashableBlock; update tests
* Update changelog
* nit: cleaner error check
* nit: clean up
* revert code logic; update string check; add a unit test
* improve errors; merge tests
* Update a unit test
* fix lint
---------
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Implement all needed KZG wrappers for peerDAS in the `kzg` package.
This way, If we need to change the KZG backend, the only package to
modify is the `kzg` package.
* `.bazelrc`: Add `build --compilation_mode=opt`
* Remove --compilation_mode=opt, use supranational blst headers.
* Fix Terence's comment.
* Fix Terence`s comments.
---------
Co-authored-by: Preston Van Loon <preston@pvl.dev>
* separate block/blob peer scoring
* Preston's test coverage feedback
* test to ensure we don't combine distinct errors
---------
Co-authored-by: Kasey <kasey@users.noreply.github.com>
* `data_column.go`: Factorize declarations (no functional changes).
* Verification for data columns: Do not recompute again if already done.
* `SaveDataColumns`: Delete because unused.
* `MissingDataColumns`: Use `DataColumnStorageSummarizer` instead of `DataColumnStorage`
* `TestFetchDataColumnsFromPeers`: Move trusted setup load out of the loop for optimization.
* `TestFetchDataColumnsFromPeers`: Use fulu block instead of deneb block.
* `fetchDataColumnsFromPeers`: Use functions already implemented in the `sync` package instead of duplicated them here.
* `fetchDataColumnsFromPeers` ==> `fetchMissingDataColumnsFromPeers`.
* Data columns initial sync: simplify
* Requests data columns by range: Move from initial sync to sync package.
Since it will eventually be used by the backfill package, and
the backfill packages does not depend on the initial sync package.
* adding in guard and tests
* adding in review from slack
* reverting error wrapping
* fixing tests
* fixing more tests
* removing defer cleanups
* Update beacon-chain/rpc/eth/validator/handlers_test.go
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* Update beacon-chain/rpc/eth/beacon/handlers_pool.go
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* fixing test and adding some coverage for validator_test.go for function
* gaz
* adding happy path test
* fixing buildkite removing test helper dependency
* fixing tests
* gaz
---------
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* Add support for electra fork epoch
* Fix tests
* fixing tests without needing to override electra fork config
* reverting electra fork override on blob test too, replacing with custom timefetcher
* reverting old changelog
* Fix genesis time
* Move mainnet test into mainnet_config_test.go
* Update spec test to v1.5.0-beta.5
---------
Co-authored-by: james-prysm <james@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@pvl.dev>
* Migrate Prysm repo to Offchain Labs organization ahead of Pectra upgrade v6
* Replace prysmaticlabs with OffchainLabs on general markdowns
* Update mock
* Gazelle and add mock.go to excluded generated mock file
* Reconstruct data columns from gossip source: Call `setSeenDataColumnIndex`.
* `reconstructAndBroadcastDataColumnSidecars`: Minor optimisation.
Avoid to range over all columns.
* Reconstructed data columns sidecars from EL: Avoid broadcasting already received data columns.
* add lcStore to Node
* changelog entry
* add atomic getters and setters for the store
* change store fields visibility to private
* refactor method names and add tests
* remove get from getters
* Trigger Consolidation
* Finally have it working
* Fix Build
* Revert Change
* Fix Context
* Finally have consolidations working
* Get Evaluator Working
* Get Withdrawals Working
* Changelog
* Finally Passes
* New line
* Try again
* fmt
* fmt
* Fix Test
* removing redundant loop
* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer_attestations_electra.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* removing unused import
* replacing with more used function
* resolving Unsafe cast from uint64 to int error
---------
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* cleanup
* fixing optimal sort order for struct
* reverting clictx change based on kasey's feedback
* adding in fix for old trace file flag
* more cleanup for clarity
* optimizing if statement check and fixing wallet open on web enable
* optimizing if statement check and fixing wallet open on web enable
* some more cleanup and bug fix on open wallet
* reverting debug.go changes will handle in a separate PR
* removing useless comment
* changing useWeb to enableAPI
* fixing tests and linting
* manu feedback and one optimization removing auth token check
* gaz
* wip
* fixing unit tests
* changing is aggregator function
* wip
* fully removing the use of committee from validator client, adding a wrapper type for duties
* fixing tests
* fixing linting
* fixing more tests
* changelog
* adding some more tests
* Update proto/prysm/v1alpha1/validator.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* radek's feedback
* removing accidently checked in
---------
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Extract the block fetcher's peer selection logic for data columns so it can be used in both by range and by root requests
* Refactor data column sidecar request to send requests to multiple peers instead of supernodes
* Remove comment
* Remove unused method
* Add tests for dmissiblePeersForDataColumns
* Extract data column fetching into standalone functions
* Remove AdmissibleCustodyGroupsPeers and replace the final call with requests to multiple peers
* Apply suggestions from code review
Co-authored-by: Manu NALEPA <nalepae@gmail.com>
* Wrap errors
* Use cached peedas.Info and properly convert custody groups to custody columns
* Rename filterPeersForRangeReq
* Preserve debugging descriptions when filtering out peers
* Remove unused functions.
* Initialize nested maps
* Fix comment
* First pass at retry logic for data column requests
* Select fresh peers for each retry
* Return an error if there are requested columns remaining
* Adjust errors
* Improve slightly the godoc.
* Improve wrapped error messages.
* `AdmissiblePeersForDataColumns`: Use value or `range`.
* Remove `convertCustodyGroupsToDataColumnsByPeer` since used only once.
* Minor fixes.
* Retry until we run out of peers
* Delete from the map of peers instead of filtering
* Remove unneeded break
* WIP: TestRequestDataColumnSidecars
* `RequestDataColumnSidecars`: Move the happy path in the for loop.
* Convert the peer ID to a node ID instead of using peer.EnodeID
* Extract AdmissiblePeersForDataColumns from a method into a function and use it (instead of a mock) in TestRequestDataColumnSidecars
* Track data column requests in tests to compare vs expectations
* Run gazelle
* Clean up test config changes so other tests don't break
* Clean up comments
* Minor changes.
* Add tests for peers that don't respond with all requested columns
* Respect MaxRequestDataColumnSidecars
---------
Co-authored-by: Manu NALEPA <nalepae@gmail.com>
Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
* Implement data column support for different storage layouts
* Fix errors
* Fix linting
* `slotFromFile`: First try to decode as a data column.
---------
Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
* Update ckzg version to include ComputeCells
* Minor fix
* Run `bazel run //:gazelle -- update-repos -from_file=go.mod -to_macro=deps.bzl%prysm_deps -prune=true`
---------
Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
* Add new message type to proto and generate .go files
* `proto/engine/v1`: Remove `execution_engine_eip7594.go` since this file does not exists.
Rerun ` hack/update-go-pbs.sh` and `hack/update-go-ssz.sh `.
---------
Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
* `TestBuildBwbSlices`: Add test case failing with the current implementation.
* Fix `buildBwbSlices` to comply with the new test case.
* `block_fetchers.go`: Improve logging and godoc.
* `DataColumnsRPCMinValidSlot`: Update to Fulu.
Previously, `buildBwbSlices` were built, and then only to big requests were batched in `buildDataColumnSidecarsByRangeRequests`.
In some edge cases, this lead to requesting data columns to peers for blocks with no blobs.
Splitting by batch directly in `buildBwbSlices` fixes the issue.
Rationale:
This log is the only one notifying the user a new fork happened.
A new fork is always a little bit stressful for a node operator.
Having at least one log indicating the client switched fork is something useful.
* `ColumnAlignsWithBlock`: Split lines.
* Data columns verifications: Batch
* Remove completely `DataColumnBatchVerifier`.
Only `DataColumnsVerifier` (with `s`) on columns remains.
It is the responsability of the function which receive the data column
(either by gossip, by range request or by root request) to verify the
data column wrt. corresponding checks.
* Fix Nishant's comment.
* `validateDataColumnsByRange`: `current` ==> `currentSlot`.
* `validateRequest`: Extract `remotePeer` variable.
* `dataColumnSidecarsByRangeRPCHandler`: Small non functional refactor.
* `streamDataColumnBatch`: Fix major bug.
Before this commit, the node was unable to respond with a data column index higher than the count of stored data columns.
For example, if there is 8 data columns stored for a given block, the node was
able to respond for data columns indices 1, 3, and 5, but not for 10, 16 or 127.
The issue was visible only for full nodes, since super nodes always store 128 data columns.
* Initial sync: Fetch data columns from all peers.
(Not only from supernodes.)
* Nishant's comment: Fix `lastSlot` and `endSlot` duplication.
* Address Nishant's comment.
* Fix Commitments Check
* `highestFinalizedEpoch`: Refactor (no functional change).
* `retrieveMissingDataColumnsFromPeers`: Fix logs.
* `VerifyDataColumnSidecarKZGProofs`: Optimise with capacity.
* Save data columns when initial syncing.
* `dataColumnSidecarsByRangeRPCHandler`: Add logs when a request enters.
* Improve logging.
* Improve logging.
* `peersWithDataColumns: Do not filter any more on peer head slot.
* Fix Nishant's comment.
---------
Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
* Improve logging.
* `retrieveMissingDataColumnsFromPeers`: Limit to `512` items per request.
* `retrieveMissingDataColumnsFromPeers`: Allow `nil` peers.
Before this commit:
If, when this funcion is called, we are not yet connected to enough peers, then `peers` will be possibly not be satisfaying,
and, if new peers are connected, we will never see them.
After this commit:
If `peers` is `nil`, then we regularly check for all connected peers.
If `peers` is not `nil`, then we use them.
* `BestFinalized`: Refactor (no functional change).
* `BestNonFinalized`: Refactor (no functional change).
* `beaconBlocksByRangeRPCHandler`: Remove useless log.
The same is already printed at the start of the function.
* `calculateHeadAndTargetEpochs`: Avoid `else`.
* `ConvertPeerIDToNodeID`: Improve error.
* Stop printing noisy "peer should be banned" logs.
* Initial sync: Request data columns from peers which:
- custody a superset of columns we need, and
- have a head slot >= our target slot.
* `requestDataColumnsFromPeers`: Shuffle peers before requesting.
Before this commit, we always requests peers in the same order,
until one responds something.
Without shuffling, we always requests data columns from the same
peer.
* `requestDataColumnsFromPeers`: If error from a peer, just log the error and skip the peer.
* Improve logging.
* Fix tests.
* `retrieveMissingDataColumnsFromPeers`: Improve logging.
* `dataColumnSidecarByRootRPCHandler`: Stop decreasing peer's score if asking for a column we do not custody.
* `dataColumnSidecarByRootRPCHandler`: If a data column is unavailable, stop waiting for it.
This behaviour was useful for peer sampling.
Now, just return the data column if we store it.
If we don't, skip.
* Dirty code comment.
* `retrieveMissingDataColumnsFromPeers`: Improve logs.
* `SendDataColumnsByRangeRequest`: Improve logs.
* `dataColumnSidecarsByRangeRPCHandler`: Improve logs.
* `columnErrBuilder`: Uses `Wrap` instead of `Join`.
Reason: `Join` makes a carriage return. The log is quite unreadable.
* `validateDataColumn`: Improve log.
* `areDataColumnsAvailable`: Improve log.
* `SendDataColumnSidecarByRoot` ==> `SendDataColumnSidecarsByRootRequest`.
* `handleDA`: Refactor error message.
* `sendRecentBeaconBlocksRequest` ==> `sendBeaconBlocksRequest`.
Reason: There is no notion at all of "recent" in the function.
If the caller decides to call this function only with "recent" blocks, that's fine.
However, the function itself will know nothing about the "recentness" of these blocks.
* `sendBatchRootRequest`: Improve comments.
* `sendBeaconBlocksRequest`: Avoid `else` usage and use map of bool instead of `struct{}`.
* `wrapAndReportValidation`: Remove `agent` from log.
Reason: This prevent the log to hold on one line, and it is not really useful to debug.
* `validateAggregateAndProof`: Add comments.
* `GetValidCustodyPeers`: Fix typo.
* `GetValidCustodyPeers` ==> `DataColumnsAdmissibleCustodyPeers`.
* `CustodyHandler` ==> `DataColumnsHandler`.
* `CustodyCountFromRemotePeer` ==> `DataColumnsCustodyCountFromRemotePeer`.
* Implement `DataColumnsAdmissibleSubnetSamplingPeers`.
* Use `SubnetSamplingSize` instead of `CustodySubnetCount` where needed.
* Revert "`wrapAndReportValidation`: Remove `agent` from log."
This reverts commit 55db351102.
* `scheduleReconstructedDataColumnsBroadcast`: Really minor refactor.
* `receivedDataColumnsFromRootLock` -> `dataColumnsFromRootLock`
* `reconstructDataColumns`: Stop looking into the DB to know if we have some columns.
Before this commit:
Each time we receive a column, we look into the filesystem for all columns we store.
==> For 128 columns, it looks for 1 + 2 + 3 + ... + 128 = 128(128+1)/2 = 8256 files look.
Also, as soon as a column is saved into the file system, then if, right after, we
look at the filesystem again, we assume the column will be available (strict consistency).
It happens not to be always true.
==> Sometimes, we can reconstruct and reseed columns more than once, because of this lack of filesystem strict consistency.
After this commit:
We use a (strictly consistent) cache to determine if we received a column or not.
==> No more consistency issue, and less stress for the filesystem.
* `dataColumnSidecarByRootRPCHandler`: Improve logging.
Before this commit, logged values assumed that all requested columns correspond to
the same block root, which is not always the case.
After this commit, we know which columns are requested for which root.
* Add a log when broadcasting a data column.
This is useful to debug "lost data columns" in devnet.
* Address Nishant's comment
* `sendBatchRootRequest`: Refactor and add comments.
* `sendBatchRootRequest`: Do send requests to peers that custodies a superset of our columns.
Before this commit, we sent "data columns by root requests" for data columns peers do not custody.
* Data columns: Use subnet sampling only.
(Instead of peer sampling.)
aaa
* `areDataColumnsAvailable`: Improve logs.
* `GetBeaconBlock`: Improve logs.
Rationale: A `begin` log should always be followed by a `success` log or a `failure` log.
* `CustodyCountFromRemotePeer`: Set happy path in the outer scope.
* `FindPeersWithSubnet`: Improve logging.
* `listenForNewNodes`: Avoid infinite loop in a small subnet.
* Address Nishant's comment.
* FIx Nishant's comment.
* `pingPeers`: Add log with new ENR when modified.
* `p2p Start`: Use idiomatic go error syntax.
* P2P `start`: Fix error message.
* Use not bootnodes at all if the `--chain-config-file` flag is used and no `--bootstrap-node` flag is used.
Before this commit, if the `--chain-config-file` flag is used and no `--bootstrap-node` flag is used, then bootnodes are (incorrectly) defaulted on `mainnet` ones.
* `validPeersExist`: Centralize logs.
* `AddConnectionHandler`: Improve logging.
"Peer connected" does not really reflect the fact that a new peer is actually connected. --> "New peer connection" is more clear.
Also, instead of writing `0`, `1`or `2` for direction, now it's writted "Unknown", "Inbound", "Outbound".
* Logging: Add 2 decimals for timestamt in text and JSON logs.
* Improve "no valid peers" logging.
* Improve "Some columns have no peers responsible for custody" logging.
* `pubsubSubscriptionRequestLimit`: Increase to be consistent with data columns.
* `sendPingRequest`: Improve logging.
* `FindPeersWithSubnet`: Regularly recheck in our current set of peers if we have enough peers for this topic.
Before this commit, new peers HAD to be found, even if current peers are eventually acceptable.
For very small network, it used to lead to infinite search.
* `subscribeDynamicWithSyncSubnets`: Use exactly the same subscription function initially and every slot.
* Make deepsource happier.
* Nishant's commend: Change peer disconnected log.
* NIshant's comment: Change `Too many incoming subscription` log from error to debug.
* `FindPeersWithSubnet`: Address Nishant's comment.
* `batchSize`: Address Nishant's comment.
* `pingPeers` ==> `pingPeersAndLogEnr`.
* Update beacon-chain/sync/subscriber.go
Co-authored-by: Nishant Das <nishdas93@gmail.com>
---------
Co-authored-by: Nishant Das <nishdas93@gmail.com>
* `broadcastAndReceiveDataColumns`: Use real `sidecar.ColumnIndex` instead of position in the slice.
And improve logging as well.
* `isDataColumnsAvailable`: Improve logging.
* `validateDataColumn`: Print `Accepted data column sidecar gossip` really at the end.
* Subscriber: Improve logging.
* `sendAndSaveDataColumnSidecars`: Use common used function for logging.
* `dataColumnSidecarByRootRPCHandler`: Logging - Pring `all` instead of all the columns for a super node.
* Verification: Improve logging.
* `DataColumnsWithholdCount`: Set as `uint64` instead `int`.
* `DataColumnFields`: Improve logging.
* Logging: Remove now useless private `columnFields`function.
* Avoid useless goroutines blocking for reconstruction.
* Update beacon-chain/sync/subscriber.go
Co-authored-by: Nishant Das <nishdas93@gmail.com>
* Address Nishant's comment.
* Improve logging.
---------
Co-authored-by: Nishant Das <nishdas93@gmail.com>
* Persist All Changes
* Fix All Tests
* Fix Build
* Fix Build
* Fix Build
* Fix Test Again
* Add missing verification
* Add Test Cases for Data Column Validation
* Fix comments for methods
* Fix comments for methods
* Fix Test
* Manu's Review
* Fix the obvious...
* Data columns sampling: Modify logging.
* `waitForChainStart`: Set it threadsafe - Do only wait once.
* Sampling: Wait for chain start before running the sampling.
Reason: `newDataColumnSampler1D` needs `s.ctxMap`.
`s.ctxMap` is only set when chain is started.
Previously `waitForChainStart` was only called in `s.registerHandlers`, it self called in a go-routine.
==> We had a race condition here: Sometimes `newDataColumnSampler1D` were called once `s.ctxMap` were set, sometimes not.
* Adresse Nishant's comments.
* Sampling: Improve logging.
* `waitForChainStart`: Remove `chainIsStarted` check.
* `SendDataColumnsByRangeRequest`: Add some new fields in logs.
* `BlobStorageSummary`: Implement `HasDataColumnIndex` and `AllDataColumnsAvailable`.
* Implement `fetchDataColumnsFromPeers`.
* `fetchBlobsFromPeer`: Return only one error.
* DeepSource: Pass heavy objects by pointers.
* `removeBlockFromQueue`: Remove redundant error checking.
* `fetchBlobsFromPeer`: Use same variable for `append`.
* Remove unused arguments.
* Combine types.
* `Persist`: Add documentation.
* Remove unused receiver
* Remove duplicated import.
* Stop using both pointer and value receiver at the same time.
* `verifyAndPopulateColumns`: Remove unused parameter
* Stop using mpty slice literal used to declare a variable.
* PeerDAS: Run reconstruction in parallel.
* `isDataAvailableDataColumns` --> `isDataColumnsAvailable`
* `isDataColumnsAvailable`: Return `nil` as soon as half of the columns are received.
* Make deepsource happy.
* Update ckzg4844 to latest version
* Run go mod tidy
* Remove unnecessary tests & run goimports
* Remove fieldparams from blockchain/kzg
* Add back blank line
* Avoid large copies
* Run gazelle
* Use trusted setup from the specs & fix issue with struct
* Run goimports
* Fix mistake in makeCellsAndProofs
---------
Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
* use recoverCellsAndKZGProofs
* make recoverAllCells and CellsToBlob private
* chore: all methods now return CellsAndProof struct
* chore: update code
* chore: move all ckzg related functionality into kzg package
* refactor code to match
* run: bazel run //:gazelle -- fix
* chore: add some docs and stop copying large objects when converting between types
* fixes
* manually add kzg.go dep to Build.Hazel
* move kzg methods to kzg.go
* chore: add RecoverCellsAndProofs method
* bazel run //:gazelle -- fix
* make Cells be flattened sequence of bytes
* chore: add test for flattening roundtrip
* chore: remove code that was doing the flattening outside of the kzg package
* fix merge
* fix
* remove now un-needed conversion
* use pointers for Cell parameters
* linter
* rename cell conversion methods (this only applies to old version of c-kzg)
* change recoverBlobs to recoverCellsAndProofs
* modify code to take in the cells and proofs for a particular blob instead of the blob itself
* add CellsAndProofs structure
* modify recoverCellsAndProofs to return `cellsAndProofs` structure
* modify `DataColumnSidecarsForReconstruct` to accept the `cellsAndKZGProofs` structure
* bazel run //:gazelle -- fix
* use kzg abstraction for kzg method
* move CellsAndProofs to kzg.go
* chore: move all ckzg related functionality into kzg package
* refactor code to match
* run: bazel run //:gazelle -- fix
* chore: add some docs and stop copying large objects when converting between types
* fixes
* manually add kzg.go dep to Build.Hazel
* move kzg methods to kzg.go
* chore: add RecoverCellsAndProofs method
* bazel run //:gazelle -- fix
* use BytesPerBlob constant
* chore: fix some deepsource issues
* one declaration for commans and blobs
* Wrap errors, add logs.
* `missingColumnRequest`: Fix blobs <-> data columns mix.
* `ColumnIndices`: Return `map[uint64]bool` instead of `[fieldparams.NumberOfColumns]bool`.
* `DataColumnSidecars`: `interfaces.SignedBeaconBlock` ==> `interfaces.ReadOnlySignedBeaconBlock`.
We don't need any of the non read-only methods.
* Fix comments.
* `handleUnblidedBlock` ==> `handleUnblindedBlock`.
* `SaveDataColumn`: Move log from debug to trace.
If we attempt to save an already existing data column sidecar,
a debug log was printed.
This case could be quite common now with the data column reconstruction enabled.
* `sampling_data_columns.go` --> `data_columns_sampling.go`.
* Reconstruct data columns.
* Support Data Columns For By Root Requests
* Revert Config Changes
* Fix Panic
* Fix Process Block
* Fix Flags
* Lint
* Support Checkpoint Sync
* Manu's Review
* Add Support For Columns in Remaining Methods
* Unmarshal Uncorrectly
* Add Data Structure for New Request Type
* Add Data Column By Range Handler
* Add Data Column Request Methods
* Add new validation for columns by range requests
* Fix Build
* Allow Prysm Node To Fetch Data Columns
* Allow Prysm Node To Fetch Data Columns And Sync
* Bug Fixes For Interop
* GoFmt
* Use different var
* Manu's Review
* Enable E2E And Add Fixes
* Register Same Topic For Data Columns
* Initialize Capacity Of Slice
* Fix Initialization of Data Column Receiver
* Remove Mix In From Merkle Proof
* E2E: Subscribe to all subnets.
* Remove Index Check
* Remaining Bug Fixes to Get It Working
* Change Evaluator to Allow Test to Finish
* Fix Build
* Add Data Column Verification
* Fix LoopVar Bug
* Do Not Allocate Memory
* Update beacon-chain/blockchain/process_block.go
Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
* Update beacon-chain/core/peerdas/helpers.go
Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
* Update beacon-chain/core/peerdas/helpers.go
Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
* Gofmt
* Fix It Again
* Fix Test Setup
* Fix Build
* Fix Trusted Setup panic
* Fix Trusted Setup panic
* Use New Test
---------
Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
* Add new DA check
* Exit early in the event no commitments exist.
* Gazelle
* Fix Mock Broadcaster
* Fix Test Setup
* Update beacon-chain/blockchain/process_block.go
Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
* Manu's Review
* Fix Build
---------
Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
* Remove capital letter from error messages.
* `[4]byte` => `[fieldparams.VersionLength]byte`.
* Prometheus: Remove extra `committee`.
They are probably due to a bad copy/paste.
Note: The name of the probe itself is remaining,
to ensure backward compatibility.
* Implement Proposer RPC for data columns.
* Fix TestProposer_ProposeBlock_OK test.
* Remove default peerDAS activation.
* `validateDataColumn`: Workaround to return a `VerifiedRODataColumn`
* Bump `c-kzg-4844` lib to the `das` branch.
* Implement `MerkleProofKZGCommitments`.
* Implement `das-core.md`.
* Use `peerdas.CustodyColumnSubnets` and `peerdas.CustodyColumns`.
* `CustodyColumnSubnets`: Include `i` in the for loop.
* Remove `computeSubscribedColumnSubnet`.
* Remove `peerdas.CustodyColumns` out of the for loop.
This release fixes two bugs related to the `payload_attributes` [event stream](https://ethereum.github.io/beacon-APIs/#/Events/eventstream). If you are using or planning to use this endpoint, upgrading to version 6.0.1 is mandatory.
Also, a reminder: like other Beacon API endpoints, when a node is syncing, it may return historical data as `finalized` or `head`. Until the node is fully synced to the head of the chain, you should avoid using this data, depending on your application's needs.
We currently recommend against using the `--enable-beacon-rest-api` flag on Mainnet. As you may have noticed, we put a deprecation notice on our gRPC code, in particular on gRPC-related flags. The reason for this is that we want to eventually remove gRPC and have REST HTTP as the standard way of communication between the validator client and the beacon node. That being said, the REST option is still unstable and thus marked as experimental in the flag's description (the flag is `--enable-beacon-rest-api`). Therefore we encourage everyone to keep using gRPC, which is currently the default. It is fine to test the REST option on testnets, but doing it on Mainnet can lead to missing attestations and even missing blocks.
**Updating to v6.0.0 or later is required for Pectra Mainnet!**
This patch release has a few important fixes from v6.0.0. Updating to this release is encouraged.
- Added immediate broadcasting of proposer slashings when equivocating blocks are detected during block processing. [[PR]](https://github.com/prysmaticlabs/prysm/pull/14693)
- Added 2 new errors: `HeadStateErr` and `ErrCouldNotVerifyBlockHeader`. [[PR]](https://github.com/prysmaticlabs/prysm/pull/14693)
- Implement pending consolidations Beacon API endpoint. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15219)
- PeerDAS: Add needed proto files and corresponding generated code. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15187)
- Add light client p2p validator and subscriber functions. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15214)
### Changed
- Refactored internal function `reValidateSubscriptions` to `pruneSubscriptions` in `beacon-chain/sync/subscriber.go` for improved clarity, addressing a TODO comment. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15160)
- Updated geth to v1.15.9. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15216)
- Removed the slot from `UpdateDuties`. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15223)
- fixed underflow with balances in leaking edge case with expected withdrawals. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15191)
- Fixes our generated ssz files to have the correct package name. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15199)
- Fixes our blob sidecar by root request lists for electra. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15209)
- Ensure that the `payload_attributes` event has a consistent view of the head state by passing the head block in the event and using stategen to retrieve the corresponding state. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15213)
- Pass parent context to update duties when dependent roots change. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15221)
- Process slots across epoch for payload attribute event. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15228)
- extend the payload attribute computation deadline to the beginning of the proposal slot. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15230)
This release introduces Mainnet support for the upcoming Electra + Prague (Pectra) fork. The fork is scheduled for mainnet epoch 364032 (May 7, 2025, 10:05:11 UTC). You MUST update Prysm Beacon Node, Prysm Validator Client, and your execution layer client to the Pectra ready release prior to the fork to stay on the correct chain.
Besides Pectra, we have more light client API support, cleanups, and a few bugfixes. Please review the changelog below and update your client as soon as practical before May 7.
This release is **mandatory** for all operators before May 7.
### Added
- Implemented validator identities Beacon API endpoint. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15086)
- Add SSZ support to light client updates by range API. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15082)
- Add light client ssz types to the spec test. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15097)
- Added the ability for execution requests to be tested in e2e with electra. [[PR]](https://github.com/prysmaticlabs/prysm/pull/14971)
- Add warning messages for gas limit ranges that might be problematic. Low gas limits (≤10% of default) may cause transactions to fail, while high gas limits (>150% of default) could lead to block propagation issues. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15078)
- Add light client store object to the beacon node object. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15120)
- prysmctl option in wrapper script to generate devnet ssz. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15145)
- Add support for Electra fork epoch. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15132)
### Changed
- The validator client will no longer use the full list of committee values but instead use the committee length and validator committee index. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15039)
- Remove the header `Content-Disposition` from the `httputil.WriteSSZ` function. No `filename` parameter is needed anymore. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15092)
- Sort attestations in proposer block by reward. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15093)
- More efficient query method for stategen to retrieve blocks between a given state and the replay target block. This avoids attempting to look up blocks that are not needed for head replay queries, which may be missing due to a previous rollback bug. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15063)
- removed old web3signer metrics in favor for a universal one. [[PR]](https://github.com/prysmaticlabs/prysm/pull/14920)
- Deprecated everything related with the gRPC API. [[PR]](https://github.com/prysmaticlabs/prysm/pull/14944)
- Migrate Prysm repo to Offchain Labs organization ahead of Pectra upgrade v6. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15140)
### Deprecated
- deprecates and removes usage of the `--trace` flag and`--cpuprofile` flag in favor of just using `--pprof`. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15083)
### Removed
- Remove /eth/v1/beacon/states/head/committees call when getting duties. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15039)
- Remove deprecated flags for the major release. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15165)
- Removed Beacon API endpoints which have been deprecated at the Deneb fork. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15166)
### Fixed
- The `--rpc` flag will now properly enable the keymanager APIs without web. The `--web` will enable both validator api endpoints and web. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15080)
- Use latest state to pack attestation. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15113)
- Clean up dangling block index entries for blocks that were previously deleted by incomplete cleanup code. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15040)
- Fixed to use io stream instead of stream read. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15089)
- When using a DV, send all aggregations for a slot and committee. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15110)
- Fixed a bug in consolidation request processing. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15122)
- Fix State Getter for pending withdrawal balance. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15123)
- Fixed a bug in checking for attestation lengths in our block. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15134)
- Fix Committee Index Check For Aggregates. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15146)
- Fix filtering by committee index post-Electra in `ListAttestationsV2`. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15148)
- Peers giving invalid data in range syncing are now downscored. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15149)
- Adding fork guard to attestation api endpoints so that it doesn't accidentally include wrong attestation types in the pool. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15161)
- fixed underflow with balances in leaking edge case with expected withdrawals. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15191)
- Attribute block and blob issues to correct peers during range syncing. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15173)
This release introduces support for the `Hoodi` testnet.
Release highlights:
- Ability to run the node on the `Hoodi` tesnet. See https://blog.ethereum.org/2025/03/18/hoodi-holesky for more information about `Hoodi`.
- A new feature that allows treat certain blocks as invalid. This is especially useful when the network is split, allowing the node to discontinue following unwanted forks.
Testnet operators are required to update to this release. Without this release you will be unable to run the node on the `Hoodi` testnet.
Mainnet operators are recommended to update to this release at their regular cadence.
### Added
- enable SSZ for builder API calls. [[PR]](https://github.com/prysmaticlabs/prysm/pull/14976)
- Add Hoodi testnet flag `--hoodi` to specify Hoodi testnet config and bootnodes. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15057)
- block_gossip topic support to the beacon api event stream. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15038)
- Added a static analyzer to discourage use of panic() in Prysm. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15075)
- Add a feature flag `--blacklist-roots` to allow the node to specify blocks that will be treated as invalid. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15030)
### Changed
- changed request object for `POST /eth/v1/beacon/states/head/validators` to omit the field if empty for satisfying other clients. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15031)
- Update spec test to v1.5.0-beta.3. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15050)
- Update Gossip and RPC message limits to comply with the spec. [[PR]](https://github.com/prysmaticlabs/prysm/pull/14799)
- Return 404 instead of 500 from API when when a blob for a requested index is not found. [[PR]](https://github.com/prysmaticlabs/prysm/pull/14845)
- Save Electra orphaned attestations into attestations pool's block attestations. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15060)
- Removed redundant string conversion in `BeaconDbStater.State` to improve code clarity and maintainability. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15081)
### Fixed
- Update seen unaggregated att cache to properly handle Electra attestations. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15034)
- cosmetic fix for calling `/prysm/validators/performance` when connecting to non prysm beacon nodes and removing the 404 error log. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15062)
- Tracked validator cache: Make sure no to loose the reference. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15077)
- Fixed proposing at genesis when starting post Bellatrix. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15084)
@@ -4,7 +4,7 @@ Note: The latest and most up-to-date documentation can be found on our [docs por
Excited by our work and want to get involved in building out our sharding releases? Or maybe you haven't learned as much about the Ethereum protocol but are a savvy developer?
You can explore our [Open Issues](https://github.com/prysmaticlabs/prysm/issues) in-the works for our different releases. Feel free to fork our repo and start creating PR’s after assigning yourself to an issue of interest. We are always chatting on [Discord](https://discord.gg/CTYGPUJ) drop us a line there if you want to get more involved or have any questions on our implementation!
You can explore our [Open Issues](https://github.com/OffchainLabs/prysm/issues) in-the works for our different releases. Feel free to fork our repo and start creating PR’s after assigning yourself to an issue of interest. We are always chatting on [Discord](https://discord.gg/CTYGPUJ) drop us a line there if you want to get more involved or have any questions on our implementation!
> [!IMPORTANT]
> Please, **do not send pull requests for trivial changes**, such as typos, these will be rejected. These types of pull requests incur a cost to reviewers and do not provide much value to the project. If you are unsure, please open an issue first to discuss the change.
@@ -15,15 +15,15 @@ You can explore our [Open Issues](https://github.com/prysmaticlabs/prysm/issues)
**2. Fork the Prysm repo.**
Sign in to your GitHub account or create a new account if you do not have one already. Then navigate your browser to https://github.com/prysmaticlabs/prysm/. In the upper right hand corner of the page, click “fork”. This will create a copy of the Prysm repo in your account.
Sign in to your GitHub account or create a new account if you do not have one already. Then navigate your browser to https://github.com/OffchainLabs/prysm/. In the upper right hand corner of the page, click “fork”. This will create a copy of the Prysm repo in your account.
$ git remote -v (you should see myrepo and prysm in the list of remotes)
```
**6. Find an issue to work on.**
Check out open issues at https://github.com/prysmaticlabs/prysm/issues and pick one. Leave a comment to let the development team know that you would like to work on it. Or examine the code for areas that can be improved and leave a comment to the development team to ask if they would like you to work on it.
Check out open issues at https://github.com/OffchainLabs/prysm/issues and pick one. Leave a comment to let the development team know that you would like to work on it. Or examine the code for areas that can be improved and leave a comment to the development team to ask if they would like you to work on it.
**7. Create a local branch with a name that clearly identifies what you will be working on.**
@@ -129,7 +129,7 @@ All PRs must must include a changelog fragment file in the `changelog` directory
**17. Create a pull request.**
Navigate your browser to https://github.com/prysmaticlabs/prysm and click on the new pull request button. In the “base” box on the left, leave the default selection “base develop”, the branch that you want your changes to be applied to. In the “compare” box on the right, select feature-in-progress-branch, the branch containing the changes you want to apply. You will then be asked to answer a few questions about your pull request. After you complete the questionnaire, the pull request will appear in the list of pull requests at https://github.com/prysmaticlabs/prysm/pulls. Ensure that you have added an entry to CHANGELOG.md if your PR is a user-facing change. See the [Maintaining CHANGELOG.md](#maintaining-changelogmd) section for more information.
Navigate your browser to https://github.com/OffchainLabs/prysm and click on the new pull request button. In the “base” box on the left, leave the default selection “base develop”, the branch that you want your changes to be applied to. In the “compare” box on the right, select feature-in-progress-branch, the branch containing the changes you want to apply. You will then be asked to answer a few questions about your pull request. After you complete the questionnaire, the pull request will appear in the list of pull requests at https://github.com/OffchainLabs/prysm/pulls. Ensure that you have added an entry to CHANGELOG.md if your PR is a user-facing change. See the [Maintaining CHANGELOG.md](#maintaining-changelogmd) section for more information.
This is the core repository for Prysm, a [Golang](https://golang.org/) implementation of the [Ethereum Consensus](https://ethereum.org/en/developers/docs/consensus-mechanisms/#proof-of-stake) [specification](https://github.com/ethereum/consensus-specs), developed by [Offchain Labs](https://www.offchainlabs.com). See the [Changelog](https://github.com/prysmaticlabs/prysm/releases) for details of the latest releases and upcoming breaking changes.
</div>
### Getting Started
---
A detailed set of installation and usage instructions as well as breakdowns of each individual component are available in the [official documentation portal](https://docs.prylabs.network). If you still have questions, feel free to stop by our [Discord](https://discord.gg/prysmaticlabs).
## 📖 Overview
### Staking on Mainnet
This is the core repository for Prysm, a [Golang](https://golang.org/) implementation of the [Ethereum Consensus](https://ethereum.org/en/developers/docs/consensus-mechanisms/#proof-of-stake) [specification](https://github.com/ethereum/consensus-specs), developed by [Offchain Labs](https://www.offchainlabs.com).
To participate in staking, you can join the [official eth2 launchpad](https://launchpad.ethereum.org). The launchpad is the only recommended way to become a validator on mainnet. You can explore validator rewards/penalties via Bitfly's block explorer: [beaconcha.in](https://beaconcha.in), and follow the latest blocks added to the chain on [beaconscan](https://beaconscan.com).
See the [Changelog](https://github.com/OffchainLabs/prysm/releases) for details of the latest releases and upcoming breaking changes.
---
## 🚀 Getting Started
A detailed set of installation and usage instructions as well as breakdowns of each individual component are available in the **[official documentation portal](https://docs.prylabs.network)**.
💬 **Need help?** Join our **[Discord Community](https://discord.gg/OffchainLabs)** for support.
---
## 🏆 Staking on Mainnet
To participate in staking, you can join the **[official Ethereum launchpad](https://launchpad.ethereum.org)**. The launchpad is the **only recommended** way to become a validator on mainnet.
🔍 Explore validator rewards/penalties:
- **[beaconcha.in](https://beaconcha.in)**
- **[beaconscan](https://beaconscan.com)**
---
## 🤝 Contributing
### 🔥 Branches
## Contributing
### Branches
Prysm maintains two permanent branches:
* [master](https://github.com/prysmaticlabs/prysm/tree/master): This points to the latest stable release. It is ideal for most users.
* [develop](https://github.com/prysmaticlabs/prysm/tree/develop): This is used for development, it contains the latest PRs. Developers should base their PRs on this branch.
-**[`master`](https://github.com/OffchainLabs/prysm/tree/master)** - This points to the latest stable release. It is ideal for most users.
-**[`develop`](https://github.com/OffchainLabs/prysm/tree/develop)** - This is used for development and contains the latest PRs. Developers should base their PRs on this branch.
### Guide
Want to get involved? Check out our [Contribution Guide](https://docs.prylabs.network/docs/contribute/contribution-guidelines/) to learn more!
### 🛠 Contribution Guide
## License
Want to get involved? Check out our **[Contribution Guide](https://docs.prylabs.network/docs/contribute/contribution-guidelines/)** to learn more!
[GNU General Public License v3.0](https://www.gnu.org/licenses/gpl-3.0.en.html)
[Releases](https://github.com/prysmaticlabs/prysm/releases/) contains all available releases. We recommend using the [most recently released version](https://github.com/prysmaticlabs/prysm/releases/latest).
[Releases](https://github.com/OffchainLabs/prysm/releases/) contains all available releases. We recommend using the [most recently released version](https://github.com/OffchainLabs/prysm/releases/latest).
## Reporting a Vulnerability
Please see our signed [security.txt](https://github.com/prysmaticlabs/prysm/blob/develop/.well-known/security.txt) for preferred encryption and reporting destination.
Please see our signed [security.txt](https://github.com/OffchainLabs/prysm/blob/develop/.well-known/security.txt) for preferred encryption and reporting destination.
**Please do not file a public ticket** mentioning the vulnerability, as doing so could increase the likelihood of the vulnerability being used before a fix has been created, released and installed on the network.
returnnil,fmt.Errorf("blob commitment count %d exceeds the maximum %d",len(bb.BlobKzgCommitments),params.BeaconConfig().MaxBlobsPerBlockByVersion(version.Electra))
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.