Seperate subscribe data columns from attestation and sync committee subnets

* Validate parent block exists before signature

* `ValidProposerSignature`: Add comment

---------

Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>

.github/workflows/changelog.yml

@@ -18,7 +18,7 @@ jobs:
         uses: dsaltares/fetch-gh-release-asset@aa2ab1243d6e0d5b405b973c89fa4d06a2d0fff7 # 1.1.2
         with:
           repo: OffchainLabs/unclog
-          version: "tags/v0.1.5"
+          version: "tags/v0.1.3"
           file: "unclog"
 
       - name: Get new changelog files

CHANGELOG.md

@@ -4,166 +4,6 @@ All notable changes to this project will be documented in this file.
 
 The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
 
-## [v6.0.3](https://github.com/prysmaticlabs/prysm/compare/v6.0.2...v6.0.3) - 2025-05-21
-
-This release has important bugfixes for users of the [Beacon API](https://ethereum.github.io/beacon-APIs/). These fixes include:
-- Fixed pending consolidations endpoint to return the correct response.
-- Fixed incorrect field name from pending partial withdrawals response.
-- Fixed attester slashing to return an empty array instead of nil/null. 
-- Fixed validator participation and active set changes endpoints to accept a `{state_id}` parameter. 
-
-Other improvements include:
-- Disabled deposit log processing routine for Electra and beyond.
-
-Operators are encouraged to update at their own convenience.
-
-### Added
-
-- ssz static spec tests for fulu. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15279)
-- finality and merkle proof spec tests for fulu. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15286)
-- sanity and rewards spec tests for fulu. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15285)
-
-### Changed
-
-- Added more tracing spans to various helpers related to GetDuties. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15271)
-- Disable log processing after deposit requests are activated. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15274)
-
-### Fixed
-
-- fixed wrong handler for get pending consolidations endpoint. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15290)
-- Fixed /eth/v2/beacon/pool/attester_slashings no slashings returns empty array instead of nil. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15291)
-- Fix Prysm endpoints `/prysm/v1/validators/{state_id}/participation` and `/prysm/v1/validators/{state_id}/active_set_changes` to properly handle `{state_id}`. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15245)
-
-## [v6.0.2](https://github.com/prysmaticlabs/prysm/compare/v6.0.1...v6.0.2) - 2025-05-12
-
-This is a patch release to fix a few important bugs. Most importantly, we have adjusted the index limit for field tries in the beacon state to better support Pectra states. This should alleviate memory issues that clients are seeing since Pectra mainnet fork.
-
-### Added
-
-- Enable light client gossip for optimistic and finality updates. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15220)
-- Implement peerDAS core functions. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15192)
-- Force duties start on received blocks. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15251)
-- Added additional tracing spans for the GetDuties routine. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15258)
-
-### Changed
-
-- Use otelgrpc for tracing grpc server and client. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15237)
-- Upgraded ristretto to v2.2.0, for RISC-V support. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15170)
-- Update spec to v1.5.0 compliance which changes minimal execution requests size. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15256)
-- Increase indices limit in field trie rebuilding. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15252)
-- Increase sepolia gas limit to 60M. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15253)
-
-### Fixed
-
-- Fixed wrong field name in pending partial withdrawals was returned on state json representation, described in https://github.com/ethereum/consensus-specs/blob/dev/specs/electra/beacon-chain.md#pendingpartialwithdrawal. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15254)
-- Fixed gocognit on propose block rest path. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15147)
-
-## [v6.0.1](https://github.com/prysmaticlabs/prysm/compare/v6.0.0...v6.0.1) - 2025-05-02
-
-This release fixes two bugs related to the `payload_attributes` [event stream](https://ethereum.github.io/beacon-APIs/#/Events/eventstream). If you are using or planning to use this endpoint, upgrading to version 6.0.1 is mandatory.  
-Also, a reminder: like other Beacon API endpoints, when a node is syncing, it may return historical data as `finalized` or `head`. Until the node is fully synced to the head of the chain, you should avoid using this data, depending on your application's needs.
-
-We currently recommend against using the `--enable-beacon-rest-api` flag on Mainnet. As you may have noticed, we put a deprecation notice on our gRPC code, in particular on gRPC-related flags. The reason for this is that we want to eventually remove gRPC and have REST HTTP as the standard way of communication between the validator client and the beacon node. That being said, the REST option is still unstable and thus marked as experimental in the flag's description (the flag is `--enable-beacon-rest-api`). Therefore we encourage everyone to keep using gRPC, which is currently the default. It is fine to test the REST option on testnets, but doing it on Mainnet can lead to missing attestations and even missing blocks.
-
-**Updating to v6.0.0 or later is required for Pectra Mainnet!**
-
-This patch release has a few important fixes from v6.0.0. Updating to this release is encouraged.
-
-### Added
-
-- `UpgradeToFulu` spectests. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15190)
-- PeerDAS related KZG wrappers. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15186)
-- Add light client p2p broadcaster functions. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15175)
-- Added immediate broadcasting of proposer slashings when equivocating blocks are detected during block processing. [[PR]](https://github.com/prysmaticlabs/prysm/pull/14693)
-- Added 2 new errors: `HeadStateErr` and `ErrCouldNotVerifyBlockHeader`. [[PR]](https://github.com/prysmaticlabs/prysm/pull/14693)
-- Implement pending consolidations Beacon API endpoint. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15219)
-- PeerDAS: Add needed proto files and corresponding generated code. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15187)
-- Add light client p2p validator and subscriber functions. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15214)
-
-### Changed
-
-- Refactored internal function `reValidateSubscriptions` to `pruneSubscriptions` in `beacon-chain/sync/subscriber.go` for improved clarity, addressing a TODO comment. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15160)
-- Updated geth to v1.15.9. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15216)
-- Removed the slot from `UpdateDuties`. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15223)
-- Update hoodie bootnodes. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15240)
-
-### Fixed
-
-- avoid nondeterministic default fork value when generate genesis. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15151)
-- `UpgradeToFulu`. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15190)
-- fixed underflow with balances in leaking edge case with expected withdrawals. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15191)
-- Fixes our generated ssz files to have the correct package name. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15199)
-- Fixes our blob sidecar by root request lists for electra. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15209)
-- Ensure that the `payload_attributes` event has a consistent view of the head state by passing the head block in the event and using stategen to retrieve the corresponding state. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15213)
-- Pass parent context to update duties when dependent roots change. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15221)
-- Process slots across epoch for payload attribute event. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15228)
-- extend the payload attribute computation deadline to the beginning of the proposal slot. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15230)
-
-### Security
-
-- Fix CVE-2025-22869. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15204)
-- Fix CVE-2025-22870. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15204)
-- Fix CVE-2025-22872. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15204)
-- Fix CVE-2025-30204. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15204)
-
-## [v6.0.0](https://github.com/prysmaticlabs/prysm/compare/v5.3.2...v6.0.0) - 2025-04-21
-
-This release introduces Mainnet support for the upcoming Electra + Prague (Pectra) fork. The fork is scheduled for mainnet epoch 364032 (May 7, 2025, 10:05:11 UTC). You MUST update Prysm Beacon Node, Prysm Validator Client, and your execution layer client to the Pectra ready release prior to the fork to stay on the correct chain.
-
-Besides Pectra, we have more light client API support, cleanups, and a few bugfixes. Please review the changelog below and update your client as soon as practical before May 7.
-
-This release is **mandatory** for all operators before May 7.
-
-### Added
-
-- Implemented validator identities Beacon API endpoint. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15086)
-- Add SSZ support to light client updates by range API. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15082)
-- Add light client ssz types to the spec test. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15097)
-- Added the ability for execution requests to be tested in e2e with electra. [[PR]](https://github.com/prysmaticlabs/prysm/pull/14971)
-- Add warning messages for gas limit ranges that might be problematic. Low gas limits (≤10% of default) may cause transactions to fail, while high gas limits (>150% of default) could lead to block propagation issues. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15078)
-- Add light client store object to the beacon node object. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15120)
-- prysmctl option in wrapper script to generate devnet ssz. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15145)
-- Add support for Electra fork epoch. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15132)
-
-### Changed
-
-- The validator client will no longer use the full list of committee values but instead use the committee length and validator committee index. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15039)
-- Remove the header `Content-Disposition` from the `httputil.WriteSSZ` function. No `filename` parameter is needed anymore. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15092)
-- Sort attestations in proposer block by reward. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15093)
-- More efficient query method for stategen to retrieve blocks between a given state and the replay target block. This avoids attempting to look up blocks that are not needed for head replay queries, which may be missing due to a previous rollback bug. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15063)
-- removed old web3signer metrics in favor for a universal one. [[PR]](https://github.com/prysmaticlabs/prysm/pull/14920)
-- Deprecated everything related with the gRPC API. [[PR]](https://github.com/prysmaticlabs/prysm/pull/14944)
-- Migrate Prysm repo to Offchain Labs organization ahead of Pectra upgrade v6. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15140)
-
-### Deprecated
-
-- deprecates and removes usage of the `--trace` flag and`--cpuprofile` flag in favor of just using `--pprof`. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15083)
-
-### Removed
-
-- Remove /eth/v1/beacon/states/head/committees call when getting duties. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15039)
-- Removed unused hack scripts. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15157)
-- Remove `disable-committee-aware-packing` flag. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15162)
-- Remove deprecated flags for the major release. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15165)
-- Removed Beacon API endpoints which have been deprecated at the Deneb fork. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15166)
-
-### Fixed
-
-- The `--rpc` flag will now properly enable the keymanager APIs without web. The `--web` will enable both validator api endpoints and web. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15080)
-- Use latest state to pack attestation. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15113)
-- Clean up dangling block index entries for blocks that were previously deleted by incomplete cleanup code. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15040)
-- Fixed to use io stream instead of stream read. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15089)
-- When using a DV, send all aggregations for a slot and committee. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15110)
-- Fixed a bug in consolidation request processing. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15122)
-- Fix State Getter for pending withdrawal balance. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15123)
-- Fixed a bug in checking for attestation lengths in our block. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15134)
-- Fix Committee Index Check For Aggregates. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15146)
-- Fix filtering by committee index post-Electra in `ListAttestationsV2`. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15148)
-- Peers giving invalid data in range syncing are now downscored. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15149)
-- Adding fork guard to attestation api endpoints so that it doesn't accidentally include wrong attestation types in the pool. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15161)
-- fixed underflow with balances in leaking edge case with expected withdrawals. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15191)
-- Attribute block and blob issues to correct peers during range syncing. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15173)
-
 ## [v5.3.2](https://github.com/prysmaticlabs/prysm/compare/v5.3.1...v5.3.2) - 2025-03-25
 
 This release introduces support for the `Hoodi` testnet.
@@ -3415,4 +3255,4 @@ There are no security updates in this release.
 
 # Older than v2.0.0
 
-For changelog history for releases older than v2.0.0, please refer to https://github.com/prysmaticlabs/prysm/releases
+For changelog history for releases older than v2.0.0, please refer to https://github.com/prysmaticlabs/prysm/releases

CONTRIBUTING.md

@@ -4,7 +4,7 @@ Note: The latest and most up-to-date documentation can be found on our [docs por
 
 Excited by our work and want to get involved in building out our sharding releases? Or maybe you haven't learned as much about the Ethereum protocol but are a savvy developer? 
 
-You can explore our [Open Issues](https://github.com/OffchainLabs/prysm/issues) in-the works for our different releases. Feel free to fork our repo and start creating PR’s after assigning yourself to an issue of interest. We are always chatting on [Discord](https://discord.gg/prysm) drop us a line there if you want to get more involved or have any questions on our implementation!
+You can explore our [Open Issues](https://github.com/OffchainLabs/prysm/issues) in-the works for our different releases. Feel free to fork our repo and start creating PR’s after assigning yourself to an issue of interest. We are always chatting on [Discord](https://discord.gg/CTYGPUJ) drop us a line there if you want to get more involved or have any questions on our implementation!
 
 > [!IMPORTANT] 
 > Please, **do not send pull requests for trivial changes**, such as typos, these will be rejected. These types of pull requests incur a cost to reviewers and do not provide much value to the project. If you are unsure, please open an issue first to discuss the change.

README.md

@@ -6,7 +6,7 @@
 [![Go Report Card](https://goreportcard.com/badge/github.com/OffchainLabs/prysm)](https://goreportcard.com/report/github.com/OffchainLabs/prysm)
 [![Consensus_Spec_Version 1.4.0](https://img.shields.io/badge/Consensus%20Spec%20Version-v1.4.0-blue.svg)](https://github.com/ethereum/consensus-specs/tree/v1.4.0)
 [![Execution_API_Version 1.0.0-beta.2](https://img.shields.io/badge/Execution%20API%20Version-v1.0.0.beta.2-blue.svg)](https://github.com/ethereum/execution-apis/tree/v1.0.0-beta.2/src/engine)
-[![Discord](https://user-images.githubusercontent.com/7288322/34471967-1df7808a-efbb-11e7-9088-ed0b04151291.png)](https://discord.gg/prysm)
+[![Discord](https://user-images.githubusercontent.com/7288322/34471967-1df7808a-efbb-11e7-9088-ed0b04151291.png)](https://discord.gg/OffchainLabs)
 [![GitPOAP Badge](https://public-api.gitpoap.io/v1/repo/OffchainLabs/prysm/badge)](https://www.gitpoap.io/gh/OffchainLabs/prysm)
 
 </div>
@@ -25,7 +25,7 @@ See the [Changelog](https://github.com/OffchainLabs/prysm/releases) for details
 
 A detailed set of installation and usage instructions as well as breakdowns of each individual component are available in the **[official documentation portal](https://docs.prylabs.network)**.
 
-💬 **Need help?** Join our **[Discord Community](https://discord.gg/prysm)** for support.
+💬 **Need help?** Join our **[Discord Community](https://discord.gg/OffchainLabs)** for support.
 
 ---
 

WORKSPACE

@@ -1,7 +1,7 @@
 workspace(name = "prysm")
 
-load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
 load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")
+load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
 
 http_archive(
     name = "rules_pkg",
@@ -16,8 +16,6 @@ load("@rules_pkg//:deps.bzl", "rules_pkg_dependencies")
 
 rules_pkg_dependencies()
 
-load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
-
 http_archive(
     name = "toolchains_protoc",
     sha256 = "abb1540f8a9e045422730670ebb2f25b41fa56ca5a7cf795175a110a0a68f4ad",
@@ -255,18 +253,56 @@ filegroup(
     url = "https://github.com/ethereum/EIPs/archive/5480440fe51742ed23342b68cf106cefd427e39d.tar.gz",
 )
 
-consensus_spec_version = "v1.6.0-alpha.0"
+consensus_spec_version = "v1.5.0-beta.5"
 
-load("@prysm//tools:download_spectests.bzl", "consensus_spec_tests")
+bls_test_version = "v0.1.1"
 
-consensus_spec_tests(
-    name = "consensus_spec_tests",
-    flavors = {
-        "general": "sha256-W7oKvoM0nAkyitykRxAw6kmCvjYC01IqiNJy0AmCnMM=",
-        "minimal": "sha256-ig7/zxomjv6buBWMom4IxAJh3lFJ9+JnY44E7c8ZNP8=",
-        "mainnet": "sha256-mjx+MkXtPhCNv4c4knLYLIkvIdpF7WTjx/ElvGPQzSo=",
-    },
-    version = consensus_spec_version,
+http_archive(
+    name = "consensus_spec_tests_general",
+    build_file_content = """
+filegroup(
+    name = "test_data",
+    srcs = glob([
+        "**/*.ssz_snappy",
+        "**/*.yaml",
+    ]),
+    visibility = ["//visibility:public"],
+)
+    """,
+    integrity = "sha256-H+Pt4z+HCVDnEBAv814yvsjR7f5l1IpumjFoTj2XnLE=",
+    url = "https://github.com/ethereum/consensus-spec-tests/releases/download/%s/general.tar.gz" % consensus_spec_version,
+)
+
+http_archive(
+    name = "consensus_spec_tests_minimal",
+    build_file_content = """
+filegroup(
+    name = "test_data",
+    srcs = glob([
+        "**/*.ssz_snappy",
+        "**/*.yaml",
+    ]),
+    visibility = ["//visibility:public"],
+)
+    """,
+    integrity = "sha256-Dqiwf5BG7yYyURGf+i87AIdArAyztvcgjoi2kSxrGvo=",
+    url = "https://github.com/ethereum/consensus-spec-tests/releases/download/%s/minimal.tar.gz" % consensus_spec_version,
+)
+
+http_archive(
+    name = "consensus_spec_tests_mainnet",
+    build_file_content = """
+filegroup(
+    name = "test_data",
+    srcs = glob([
+        "**/*.ssz_snappy",
+        "**/*.yaml",
+    ]),
+    visibility = ["//visibility:public"],
+)
+    """,
+    integrity = "sha256-xrmsFF243pzXHAjh1EQYKS9gtcwmtHK3wRZDSLlVVRk=",
+    url = "https://github.com/ethereum/consensus-spec-tests/releases/download/%s/mainnet.tar.gz" % consensus_spec_version,
 )
 
 http_archive(
@@ -280,13 +316,11 @@ filegroup(
     visibility = ["//visibility:public"],
 )
     """,
-    integrity = "sha256-u0RkIZIeGttb3sInR31mO64aBSwxALqO5SYIPlqEvPo=",
+    integrity = "sha256-c+gGapqifCvFtmtxfhOwieBDO2Syxp13GECWEpWM/Ho=",
     strip_prefix = "consensus-specs-" + consensus_spec_version[1:],
     url = "https://github.com/ethereum/consensus-specs/archive/refs/tags/%s.tar.gz" % consensus_spec_version,
 )
 
-bls_test_version = "v0.1.1"
-
 http_archive(
     name = "bls_spec_tests",
     build_file_content = """

api/client/builder/BUILD.bazel

@@ -13,6 +13,7 @@ go_library(
     deps = [
         "//api:go_default_library",
         "//api/client:go_default_library",
+        "//api/server:go_default_library",
         "//api/server/structs:go_default_library",
         "//config/fieldparams:go_default_library",
         "//config/params:go_default_library",
@@ -27,6 +28,7 @@ go_library(
         "//proto/engine/v1:go_default_library",
         "//proto/prysm/v1alpha1:go_default_library",
         "//runtime/version:go_default_library",
+        "@com_github_ethereum_go_ethereum//common:go_default_library",
         "@com_github_ethereum_go_ethereum//common/hexutil:go_default_library",
         "@com_github_pkg_errors//:go_default_library",
         "@com_github_prysmaticlabs_fastssz//:go_default_library",

api/client/builder/client.go

@@ -241,7 +241,7 @@ func (c *Client) GetHeader(ctx context.Context, slot primitives.Slot, parentHash
 		return nil, errors.Wrap(err, "error getting header from builder server")
 	}
 
-	bid, err := c.parseHeaderResponse(data, header, slot)
+	bid, err := c.parseHeaderResponse(data, header)
 	if err != nil {
 		return nil, errors.Wrapf(
 			err,
@@ -254,7 +254,7 @@ func (c *Client) GetHeader(ctx context.Context, slot primitives.Slot, parentHash
 	return bid, nil
 }
 
-func (c *Client) parseHeaderResponse(data []byte, header http.Header, slot primitives.Slot) (SignedBid, error) {
+func (c *Client) parseHeaderResponse(data []byte, header http.Header) (SignedBid, error) {
 	var versionHeader string
 	if c.sszEnabled || header.Get(api.VersionHeader) != "" {
 		versionHeader = header.Get(api.VersionHeader)
@@ -276,7 +276,7 @@ func (c *Client) parseHeaderResponse(data []byte, header http.Header, slot primi
 	}
 
 	if ver >= version.Electra {
-		return c.parseHeaderElectra(data, slot)
+		return c.parseHeaderElectra(data)
 	}
 	if ver >= version.Deneb {
 		return c.parseHeaderDeneb(data)
@@ -291,7 +291,7 @@ func (c *Client) parseHeaderResponse(data []byte, header http.Header, slot primi
 	return nil, fmt.Errorf("unsupported header version %s", versionHeader)
 }
 
-func (c *Client) parseHeaderElectra(data []byte, slot primitives.Slot) (SignedBid, error) {
+func (c *Client) parseHeaderElectra(data []byte) (SignedBid, error) {
 	if c.sszEnabled {
 		sb := &ethpb.SignedBuilderBidElectra{}
 		if err := sb.UnmarshalSSZ(data); err != nil {
@@ -303,7 +303,7 @@ func (c *Client) parseHeaderElectra(data []byte, slot primitives.Slot) (SignedBi
 	if err := json.Unmarshal(data, hr); err != nil {
 		return nil, errors.Wrap(err, "could not unmarshal ExecHeaderResponseElectra JSON")
 	}
-	p, err := hr.ToProto(slot)
+	p, err := hr.ToProto()
 	if err != nil {
 		return nil, errors.Wrap(err, "could not convert ExecHeaderResponseElectra to proto")
 	}

api/client/builder/client_test.go

@@ -532,7 +532,7 @@ func TestClient_GetHeader(t *testing.T) {
 				require.Equal(t, expectedPath, r.URL.Path)
 				epr := &ExecHeaderResponseElectra{}
 				require.NoError(t, json.Unmarshal([]byte(testExampleHeaderResponseElectra), epr))
-				pro, err := epr.ToProto(100)
+				pro, err := epr.ToProto()
 				require.NoError(t, err)
 				ssz, err := pro.MarshalSSZ()
 				require.NoError(t, err)
@@ -640,9 +640,9 @@ func TestSubmitBlindedBlock(t *testing.T) {
 				require.Equal(t, api.OctetStreamMediaType, r.Header.Get("Accept"))
 				epr := &ExecutionPayloadResponse{}
 				require.NoError(t, json.Unmarshal([]byte(testExampleExecutionPayload), epr))
-				ep := &structs.ExecutionPayload{}
+				ep := &ExecutionPayload{}
 				require.NoError(t, json.Unmarshal(epr.Data, ep))
-				pro, err := ep.ToConsensus()
+				pro, err := ep.ToProto()
 				require.NoError(t, err)
 				ssz, err := pro.MarshalSSZ()
 				require.NoError(t, err)
@@ -710,9 +710,9 @@ func TestSubmitBlindedBlock(t *testing.T) {
 				require.Equal(t, api.OctetStreamMediaType, r.Header.Get("Accept"))
 				epr := &ExecutionPayloadResponse{}
 				require.NoError(t, json.Unmarshal([]byte(testExampleExecutionPayloadCapella), epr))
-				ep := &structs.ExecutionPayloadCapella{}
+				ep := &ExecutionPayloadCapella{}
 				require.NoError(t, json.Unmarshal(epr.Data, ep))
-				pro, err := ep.ToConsensus()
+				pro, err := ep.ToProto()
 				require.NoError(t, err)
 				ssz, err := pro.MarshalSSZ()
 				require.NoError(t, err)

api/client/builder/types_test.go

@@ -328,72 +328,72 @@ func TestExecutionHeaderResponseUnmarshal(t *testing.T) {
 		},
 		{
 			expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-			actual:   hr.Data.Message.Header.ParentHash,
+			actual:   hexutil.Encode(hr.Data.Message.Header.ParentHash),
 			name:     "ExecHeaderResponse.ExecutionPayloadHeader.ParentHash",
 		},
 		{
 			expected: "0xabcf8e0d4e9587369b2301d0790347320302cc09",
-			actual:   hr.Data.Message.Header.FeeRecipient,
+			actual:   hexutil.Encode(hr.Data.Message.Header.FeeRecipient),
 			name:     "ExecHeaderResponse.ExecutionPayloadHeader.FeeRecipient",
 		},
 		{
 			expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-			actual:   hr.Data.Message.Header.StateRoot,
+			actual:   hexutil.Encode(hr.Data.Message.Header.StateRoot),
 			name:     "ExecHeaderResponse.ExecutionPayloadHeader.StateRoot",
 		},
 		{
 			expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-			actual:   hr.Data.Message.Header.ReceiptsRoot,
+			actual:   hexutil.Encode(hr.Data.Message.Header.ReceiptsRoot),
 			name:     "ExecHeaderResponse.ExecutionPayloadHeader.ReceiptsRoot",
 		},
 		{
 			expected: "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
-			actual:   hr.Data.Message.Header.LogsBloom,
+			actual:   hexutil.Encode(hr.Data.Message.Header.LogsBloom),
 			name:     "ExecHeaderResponse.ExecutionPayloadHeader.LogsBloom",
 		},
 		{
 			expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-			actual:   hr.Data.Message.Header.PrevRandao,
+			actual:   hexutil.Encode(hr.Data.Message.Header.PrevRandao),
 			name:     "ExecHeaderResponse.ExecutionPayloadHeader.PrevRandao",
 		},
 		{
 			expected: "1",
-			actual:   hr.Data.Message.Header.BlockNumber,
+			actual:   fmt.Sprintf("%d", hr.Data.Message.Header.BlockNumber),
 			name:     "ExecHeaderResponse.ExecutionPayloadHeader.BlockNumber",
 		},
 		{
 			expected: "1",
-			actual:   hr.Data.Message.Header.GasLimit,
+			actual:   fmt.Sprintf("%d", hr.Data.Message.Header.GasLimit),
 			name:     "ExecHeaderResponse.ExecutionPayloadHeader.GasLimit",
 		},
 		{
 			expected: "1",
-			actual:   hr.Data.Message.Header.GasUsed,
+			actual:   fmt.Sprintf("%d", hr.Data.Message.Header.GasUsed),
 			name:     "ExecHeaderResponse.ExecutionPayloadHeader.GasUsed",
 		},
 		{
 			expected: "1",
-			actual:   hr.Data.Message.Header.Timestamp,
+			actual:   fmt.Sprintf("%d", hr.Data.Message.Header.Timestamp),
 			name:     "ExecHeaderResponse.ExecutionPayloadHeader.Timestamp",
 		},
 		{
 			expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-			actual:   hr.Data.Message.Header.ExtraData,
+			actual:   hexutil.Encode(hr.Data.Message.Header.ExtraData),
 			name:     "ExecHeaderResponse.ExecutionPayloadHeader.ExtraData",
 		},
 		{
 			expected: "452312848583266388373324160190187140051835877600158453279131187530910662656",
-			actual:   hr.Data.Message.Header.BaseFeePerGas,
+			actual:   fmt.Sprintf("%d", hr.Data.Message.Header.BaseFeePerGas),
 			name:     "ExecHeaderResponse.ExecutionPayloadHeader.BaseFeePerGas",
 		},
 		{
 			expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-			actual:   hr.Data.Message.Header.BlockHash,
+			actual:   hexutil.Encode(hr.Data.Message.Header.BlockHash),
 			name:     "ExecHeaderResponse.ExecutionPayloadHeader.BlockHash",
 		},
 		{
 			expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-			actual:   hr.Data.Message.Header.TransactionsRoot,
+			actual:   hexutil.Encode(hr.Data.Message.Header.TransactionsRoot),
 			name:     "ExecHeaderResponse.ExecutionPayloadHeader.TransactionsRoot",
 		},
 	}
@@ -427,77 +427,77 @@ func TestExecutionHeaderResponseCapellaUnmarshal(t *testing.T) {
 		},
 		{
 			expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-			actual:   hr.Data.Message.Header.ParentHash,
+			actual:   hexutil.Encode(hr.Data.Message.Header.ParentHash),
 			name:     "ExecHeaderResponse.ExecutionPayloadHeader.ParentHash",
 		},
 		{
 			expected: "0xabcf8e0d4e9587369b2301d0790347320302cc09",
-			actual:   hr.Data.Message.Header.FeeRecipient,
+			actual:   hexutil.Encode(hr.Data.Message.Header.FeeRecipient),
 			name:     "ExecHeaderResponse.ExecutionPayloadHeader.FeeRecipient",
 		},
 		{
 			expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-			actual:   hr.Data.Message.Header.StateRoot,
+			actual:   hexutil.Encode(hr.Data.Message.Header.StateRoot),
 			name:     "ExecHeaderResponse.ExecutionPayloadHeader.StateRoot",
 		},
 		{
 			expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-			actual:   hr.Data.Message.Header.ReceiptsRoot,
+			actual:   hexutil.Encode(hr.Data.Message.Header.ReceiptsRoot),
 			name:     "ExecHeaderResponse.ExecutionPayloadHeader.ReceiptsRoot",
 		},
 		{
 			expected: "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
-			actual:   hr.Data.Message.Header.LogsBloom,
+			actual:   hexutil.Encode(hr.Data.Message.Header.LogsBloom),
 			name:     "ExecHeaderResponse.ExecutionPayloadHeader.LogsBloom",
 		},
 		{
 			expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-			actual:   hr.Data.Message.Header.PrevRandao,
+			actual:   hexutil.Encode(hr.Data.Message.Header.PrevRandao),
 			name:     "ExecHeaderResponse.ExecutionPayloadHeader.PrevRandao",
 		},
 		{
 			expected: "1",
-			actual:   hr.Data.Message.Header.BlockNumber,
+			actual:   fmt.Sprintf("%d", hr.Data.Message.Header.BlockNumber),
 			name:     "ExecHeaderResponse.ExecutionPayloadHeader.BlockNumber",
 		},
 		{
 			expected: "1",
-			actual:   hr.Data.Message.Header.GasLimit,
+			actual:   fmt.Sprintf("%d", hr.Data.Message.Header.GasLimit),
 			name:     "ExecHeaderResponse.ExecutionPayloadHeader.GasLimit",
 		},
 		{
 			expected: "1",
-			actual:   hr.Data.Message.Header.GasUsed,
+			actual:   fmt.Sprintf("%d", hr.Data.Message.Header.GasUsed),
 			name:     "ExecHeaderResponse.ExecutionPayloadHeader.GasUsed",
 		},
 		{
 			expected: "1",
-			actual:   hr.Data.Message.Header.Timestamp,
+			actual:   fmt.Sprintf("%d", hr.Data.Message.Header.Timestamp),
 			name:     "ExecHeaderResponse.ExecutionPayloadHeader.Timestamp",
 		},
 		{
 			expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-			actual:   hr.Data.Message.Header.ExtraData,
+			actual:   hexutil.Encode(hr.Data.Message.Header.ExtraData),
 			name:     "ExecHeaderResponse.ExecutionPayloadHeader.ExtraData",
 		},
 		{
 			expected: "452312848583266388373324160190187140051835877600158453279131187530910662656",
-			actual:   hr.Data.Message.Header.BaseFeePerGas,
+			actual:   fmt.Sprintf("%d", hr.Data.Message.Header.BaseFeePerGas),
 			name:     "ExecHeaderResponse.ExecutionPayloadHeader.BaseFeePerGas",
 		},
 		{
 			expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-			actual:   hr.Data.Message.Header.BlockHash,
+			actual:   hexutil.Encode(hr.Data.Message.Header.BlockHash),
 			name:     "ExecHeaderResponse.ExecutionPayloadHeader.BlockHash",
 		},
 		{
 			expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-			actual:   hr.Data.Message.Header.TransactionsRoot,
+			actual:   hexutil.Encode(hr.Data.Message.Header.TransactionsRoot),
 			name:     "ExecHeaderResponse.ExecutionPayloadHeader.TransactionsRoot",
 		},
 		{
 			expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-			actual:   hr.Data.Message.Header.WithdrawalsRoot,
+			actual:   hexutil.Encode(hr.Data.Message.Header.WithdrawalsRoot),
 			name:     "ExecHeaderResponse.ExecutionPayloadHeader.WithdrawalsRoot",
 		},
 	}
@@ -867,6 +867,88 @@ var testExampleExecutionPayloadDenebDifferentProofCount = fmt.Sprintf(`{
   }
 }`, hexutil.Encode(make([]byte, fieldparams.BlobLength)))
 
+func TestExecutionPayloadResponseUnmarshal(t *testing.T) {
+	epr := &ExecPayloadResponse{}
+	require.NoError(t, json.Unmarshal([]byte(testExampleExecutionPayload), epr))
+	cases := []struct {
+		expected string
+		actual   string
+		name     string
+	}{
+		{
+			expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
+			actual:   hexutil.Encode(epr.Data.ParentHash),
+			name:     "ExecPayloadResponse.ExecutionPayload.ParentHash",
+		},
+		{
+			expected: "0xabcf8e0d4e9587369b2301d0790347320302cc09",
+			actual:   hexutil.Encode(epr.Data.FeeRecipient),
+			name:     "ExecPayloadResponse.ExecutionPayload.FeeRecipient",
+		},
+		{
+			expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
+			actual:   hexutil.Encode(epr.Data.StateRoot),
+			name:     "ExecPayloadResponse.ExecutionPayload.StateRoot",
+		},
+		{
+			expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
+			actual:   hexutil.Encode(epr.Data.ReceiptsRoot),
+			name:     "ExecPayloadResponse.ExecutionPayload.ReceiptsRoot",
+		},
+		{
+			expected: "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
+			actual:   hexutil.Encode(epr.Data.LogsBloom),
+			name:     "ExecPayloadResponse.ExecutionPayload.LogsBloom",
+		},
+		{
+			expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
+			actual:   hexutil.Encode(epr.Data.PrevRandao),
+			name:     "ExecPayloadResponse.ExecutionPayload.PrevRandao",
+		},
+		{
+			expected: "1",
+			actual:   fmt.Sprintf("%d", epr.Data.BlockNumber),
+			name:     "ExecPayloadResponse.ExecutionPayload.BlockNumber",
+		},
+		{
+			expected: "1",
+			actual:   fmt.Sprintf("%d", epr.Data.GasLimit),
+			name:     "ExecPayloadResponse.ExecutionPayload.GasLimit",
+		},
+		{
+			expected: "1",
+			actual:   fmt.Sprintf("%d", epr.Data.GasUsed),
+			name:     "ExecPayloadResponse.ExecutionPayload.GasUsed",
+		},
+		{
+			expected: "1",
+			actual:   fmt.Sprintf("%d", epr.Data.Timestamp),
+			name:     "ExecPayloadResponse.ExecutionPayload.Timestamp",
+		},
+		{
+			expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
+			actual:   hexutil.Encode(epr.Data.ExtraData),
+			name:     "ExecPayloadResponse.ExecutionPayload.ExtraData",
+		},
+		{
+			expected: "452312848583266388373324160190187140051835877600158453279131187530910662656",
+			actual:   fmt.Sprintf("%d", epr.Data.BaseFeePerGas),
+			name:     "ExecPayloadResponse.ExecutionPayload.BaseFeePerGas",
+		},
+		{
+			expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
+			actual:   hexutil.Encode(epr.Data.BlockHash),
+			name:     "ExecPayloadResponse.ExecutionPayload.BlockHash",
+		},
+	}
+	for _, c := range cases {
+		require.Equal(t, c.expected, c.actual, fmt.Sprintf("unexpected value for field %s", c.name))
+	}
+	require.Equal(t, 1, len(epr.Data.Transactions))
+	txHash := "0x02f878831469668303f51d843b9ac9f9843b9aca0082520894c93269b73096998db66be0441e836d873535cb9c8894a19041886f000080c001a031cc29234036afbf9a1fb9476b463367cb1f957ac0b919b69bbc798436e604aaa018c4e9c3914eb27aadd0b91e10b18655739fcf8c1fc398763a9f1beecb8ddc86"
+	require.Equal(t, txHash, hexutil.Encode(epr.Data.Transactions[0]))
+}
+
 func TestExecutionPayloadResponseCapellaUnmarshal(t *testing.T) {
 	epr := &ExecPayloadResponseCapella{}
 	require.NoError(t, json.Unmarshal([]byte(testExampleExecutionPayloadCapella), epr))
@@ -877,67 +959,67 @@ func TestExecutionPayloadResponseCapellaUnmarshal(t *testing.T) {
 	}{
 		{
 			expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-			actual:   epr.Data.ParentHash,
+			actual:   hexutil.Encode(epr.Data.ParentHash),
 			name:     "ExecPayloadResponse.ExecutionPayload.ParentHash",
 		},
 		{
 			expected: "0xabcf8e0d4e9587369b2301d0790347320302cc09",
-			actual:   epr.Data.FeeRecipient,
+			actual:   hexutil.Encode(epr.Data.FeeRecipient),
 			name:     "ExecPayloadResponse.ExecutionPayload.FeeRecipient",
 		},
 		{
 			expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-			actual:   epr.Data.StateRoot,
+			actual:   hexutil.Encode(epr.Data.StateRoot),
 			name:     "ExecPayloadResponse.ExecutionPayload.StateRoot",
 		},
 		{
 			expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-			actual:   epr.Data.ReceiptsRoot,
+			actual:   hexutil.Encode(epr.Data.ReceiptsRoot),
 			name:     "ExecPayloadResponse.ExecutionPayload.ReceiptsRoot",
 		},
 		{
 			expected: "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
-			actual:   epr.Data.LogsBloom,
+			actual:   hexutil.Encode(epr.Data.LogsBloom),
 			name:     "ExecPayloadResponse.ExecutionPayload.LogsBloom",
 		},
 		{
 			expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-			actual:   epr.Data.PrevRandao,
+			actual:   hexutil.Encode(epr.Data.PrevRandao),
 			name:     "ExecPayloadResponse.ExecutionPayload.PrevRandao",
 		},
 		{
 			expected: "1",
-			actual:   epr.Data.BlockNumber,
+			actual:   fmt.Sprintf("%d", epr.Data.BlockNumber),
 			name:     "ExecPayloadResponse.ExecutionPayload.BlockNumber",
 		},
 		{
 			expected: "1",
-			actual:   epr.Data.GasLimit,
+			actual:   fmt.Sprintf("%d", epr.Data.GasLimit),
 			name:     "ExecPayloadResponse.ExecutionPayload.GasLimit",
 		},
 		{
 			expected: "1",
-			actual:   epr.Data.GasUsed,
+			actual:   fmt.Sprintf("%d", epr.Data.GasUsed),
 			name:     "ExecPayloadResponse.ExecutionPayload.GasUsed",
 		},
 		{
 			expected: "1",
-			actual:   epr.Data.Timestamp,
+			actual:   fmt.Sprintf("%d", epr.Data.Timestamp),
 			name:     "ExecPayloadResponse.ExecutionPayload.Timestamp",
 		},
 		{
 			expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-			actual:   epr.Data.ExtraData,
+			actual:   hexutil.Encode(epr.Data.ExtraData),
 			name:     "ExecPayloadResponse.ExecutionPayload.ExtraData",
 		},
 		{
 			expected: "452312848583266388373324160190187140051835877600158453279131187530910662656",
-			actual:   epr.Data.BaseFeePerGas,
+			actual:   fmt.Sprintf("%d", epr.Data.BaseFeePerGas),
 			name:     "ExecPayloadResponse.ExecutionPayload.BaseFeePerGas",
 		},
 		{
 			expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-			actual:   epr.Data.BlockHash,
+			actual:   hexutil.Encode(epr.Data.BlockHash),
 			name:     "ExecPayloadResponse.ExecutionPayload.BlockHash",
 		},
 	}
@@ -946,14 +1028,14 @@ func TestExecutionPayloadResponseCapellaUnmarshal(t *testing.T) {
 	}
 	require.Equal(t, 1, len(epr.Data.Transactions))
 	txHash := "0x02f878831469668303f51d843b9ac9f9843b9aca0082520894c93269b73096998db66be0441e836d873535cb9c8894a19041886f000080c001a031cc29234036afbf9a1fb9476b463367cb1f957ac0b919b69bbc798436e604aaa018c4e9c3914eb27aadd0b91e10b18655739fcf8c1fc398763a9f1beecb8ddc86"
-	require.Equal(t, txHash, epr.Data.Transactions[0])
+	require.Equal(t, txHash, hexutil.Encode(epr.Data.Transactions[0]))
 
 	require.Equal(t, 1, len(epr.Data.Withdrawals))
 	w := epr.Data.Withdrawals[0]
-	assert.Equal(t, "1", w.WithdrawalIndex)
-	assert.Equal(t, "1", w.ValidatorIndex)
-	assert.DeepEqual(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943", w.ExecutionAddress)
-	assert.Equal(t, "1", w.Amount)
+	assert.Equal(t, uint64(1), w.Index.Uint64())
+	assert.Equal(t, uint64(1), w.ValidatorIndex.Uint64())
+	assert.DeepEqual(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943", w.Address.String())
+	assert.Equal(t, uint64(1), w.Amount.Uint64())
 }
 
 func TestExecutionPayloadResponseDenebUnmarshal(t *testing.T) {
@@ -966,77 +1048,77 @@ func TestExecutionPayloadResponseDenebUnmarshal(t *testing.T) {
 	}{
 		{
 			expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-			actual:   epr.Data.ExecutionPayload.ParentHash,
+			actual:   hexutil.Encode(epr.Data.ExecutionPayload.ParentHash),
 			name:     "ExecPayloadResponse.ExecutionPayload.ParentHash",
 		},
 		{
 			expected: "0xabcf8e0d4e9587369b2301d0790347320302cc09",
-			actual:   epr.Data.ExecutionPayload.FeeRecipient,
+			actual:   hexutil.Encode(epr.Data.ExecutionPayload.FeeRecipient),
 			name:     "ExecPayloadResponse.ExecutionPayload.FeeRecipient",
 		},
 		{
 			expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-			actual:   epr.Data.ExecutionPayload.StateRoot,
+			actual:   hexutil.Encode(epr.Data.ExecutionPayload.StateRoot),
 			name:     "ExecPayloadResponse.ExecutionPayload.StateRoot",
 		},
 		{
 			expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-			actual:   epr.Data.ExecutionPayload.ReceiptsRoot,
+			actual:   hexutil.Encode(epr.Data.ExecutionPayload.ReceiptsRoot),
 			name:     "ExecPayloadResponse.ExecutionPayload.ReceiptsRoot",
 		},
 		{
 			expected: "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
-			actual:   epr.Data.ExecutionPayload.LogsBloom,
+			actual:   hexutil.Encode(epr.Data.ExecutionPayload.LogsBloom),
 			name:     "ExecPayloadResponse.ExecutionPayload.LogsBloom",
 		},
 		{
 			expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-			actual:   epr.Data.ExecutionPayload.PrevRandao,
+			actual:   hexutil.Encode(epr.Data.ExecutionPayload.PrevRandao),
 			name:     "ExecPayloadResponse.ExecutionPayload.PrevRandao",
 		},
 		{
 			expected: "1",
-			actual:   epr.Data.ExecutionPayload.BlockNumber,
+			actual:   fmt.Sprintf("%d", epr.Data.ExecutionPayload.BlockNumber),
 			name:     "ExecPayloadResponse.ExecutionPayload.BlockNumber",
 		},
 		{
 			expected: "1",
-			actual:   epr.Data.ExecutionPayload.GasLimit,
+			actual:   fmt.Sprintf("%d", epr.Data.ExecutionPayload.GasLimit),
 			name:     "ExecPayloadResponse.ExecutionPayload.GasLimit",
 		},
 		{
 			expected: "1",
-			actual:   epr.Data.ExecutionPayload.GasUsed,
+			actual:   fmt.Sprintf("%d", epr.Data.ExecutionPayload.GasUsed),
 			name:     "ExecPayloadResponse.ExecutionPayload.GasUsed",
 		},
 		{
 			expected: "1",
-			actual:   epr.Data.ExecutionPayload.Timestamp,
+			actual:   fmt.Sprintf("%d", epr.Data.ExecutionPayload.Timestamp),
 			name:     "ExecPayloadResponse.ExecutionPayload.Timestamp",
 		},
 		{
 			expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-			actual:   epr.Data.ExecutionPayload.ExtraData,
+			actual:   hexutil.Encode(epr.Data.ExecutionPayload.ExtraData),
 			name:     "ExecPayloadResponse.ExecutionPayload.ExtraData",
 		},
 		{
 			expected: "452312848583266388373324160190187140051835877600158453279131187530910662656",
-			actual:   epr.Data.ExecutionPayload.BaseFeePerGas,
+			actual:   fmt.Sprintf("%d", epr.Data.ExecutionPayload.BaseFeePerGas),
 			name:     "ExecPayloadResponse.ExecutionPayload.BaseFeePerGas",
 		},
 		{
 			expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-			actual:   epr.Data.ExecutionPayload.BlockHash,
+			actual:   hexutil.Encode(epr.Data.ExecutionPayload.BlockHash),
 			name:     "ExecPayloadResponse.ExecutionPayload.BlockHash",
 		},
 		{
 			expected: "2",
-			actual:   epr.Data.ExecutionPayload.BlobGasUsed,
+			actual:   fmt.Sprintf("%d", epr.Data.ExecutionPayload.BlobGasUsed),
 			name:     "ExecPayloadResponse.ExecutionPayload.BlobGasUsed",
 		},
 		{
 			expected: "3",
-			actual:   epr.Data.ExecutionPayload.ExcessBlobGas,
+			actual:   fmt.Sprintf("%d", epr.Data.ExecutionPayload.ExcessBlobGas),
 			name:     "ExecPayloadResponse.ExecutionPayload.ExcessBlobGas",
 		},
 	}
@@ -1045,16 +1127,64 @@ func TestExecutionPayloadResponseDenebUnmarshal(t *testing.T) {
 	}
 	require.Equal(t, 1, len(epr.Data.ExecutionPayload.Transactions))
 	txHash := "0x02f878831469668303f51d843b9ac9f9843b9aca0082520894c93269b73096998db66be0441e836d873535cb9c8894a19041886f000080c001a031cc29234036afbf9a1fb9476b463367cb1f957ac0b919b69bbc798436e604aaa018c4e9c3914eb27aadd0b91e10b18655739fcf8c1fc398763a9f1beecb8ddc86"
-	require.Equal(t, txHash, epr.Data.ExecutionPayload.Transactions[0])
+	require.Equal(t, txHash, hexutil.Encode(epr.Data.ExecutionPayload.Transactions[0]))
 
 	require.Equal(t, 1, len(epr.Data.ExecutionPayload.Withdrawals))
 	w := epr.Data.ExecutionPayload.Withdrawals[0]
-	assert.Equal(t, "1", w.WithdrawalIndex)
-	assert.Equal(t, "1", w.ValidatorIndex)
-	assert.DeepEqual(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943", w.ExecutionAddress)
-	assert.Equal(t, "1", w.Amount)
-	assert.Equal(t, "2", epr.Data.ExecutionPayload.BlobGasUsed)
-	assert.Equal(t, "3", epr.Data.ExecutionPayload.ExcessBlobGas)
+	assert.Equal(t, uint64(1), w.Index.Uint64())
+	assert.Equal(t, uint64(1), w.ValidatorIndex.Uint64())
+	assert.DeepEqual(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943", w.Address.String())
+	assert.Equal(t, uint64(1), w.Amount.Uint64())
+	assert.Equal(t, uint64(2), uint64(epr.Data.ExecutionPayload.BlobGasUsed))
+	assert.Equal(t, uint64(3), uint64(epr.Data.ExecutionPayload.ExcessBlobGas))
+}
+
+func TestExecutionPayloadResponseToProto(t *testing.T) {
+	hr := &ExecPayloadResponse{}
+	require.NoError(t, json.Unmarshal([]byte(testExampleExecutionPayload), hr))
+	p, err := hr.ToProto()
+	require.NoError(t, err)
+
+	parentHash, err := hexutil.Decode("0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2")
+	require.NoError(t, err)
+	feeRecipient, err := hexutil.Decode("0xabcf8e0d4e9587369b2301d0790347320302cc09")
+	require.NoError(t, err)
+	stateRoot, err := hexutil.Decode("0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2")
+	require.NoError(t, err)
+	receiptsRoot, err := hexutil.Decode("0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2")
+	require.NoError(t, err)
+	logsBloom, err := hexutil.Decode("0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000")
+	require.NoError(t, err)
+	prevRandao, err := hexutil.Decode("0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2")
+	require.NoError(t, err)
+	extraData, err := hexutil.Decode("0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2")
+	require.NoError(t, err)
+	blockHash, err := hexutil.Decode("0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2")
+	require.NoError(t, err)
+
+	tx, err := hexutil.Decode("0x02f878831469668303f51d843b9ac9f9843b9aca0082520894c93269b73096998db66be0441e836d873535cb9c8894a19041886f000080c001a031cc29234036afbf9a1fb9476b463367cb1f957ac0b919b69bbc798436e604aaa018c4e9c3914eb27aadd0b91e10b18655739fcf8c1fc398763a9f1beecb8ddc86")
+	require.NoError(t, err)
+	txList := [][]byte{tx}
+
+	bfpg, err := stringToUint256("452312848583266388373324160190187140051835877600158453279131187530910662656")
+	require.NoError(t, err)
+	expected := &v1.ExecutionPayload{
+		ParentHash:    parentHash,
+		FeeRecipient:  feeRecipient,
+		StateRoot:     stateRoot,
+		ReceiptsRoot:  receiptsRoot,
+		LogsBloom:     logsBloom,
+		PrevRandao:    prevRandao,
+		BlockNumber:   1,
+		GasLimit:      1,
+		GasUsed:       1,
+		Timestamp:     1,
+		ExtraData:     extraData,
+		BaseFeePerGas: bfpg.SSZBytes(),
+		BlockHash:     blockHash,
+		Transactions:  txList,
+	}
+	require.DeepEqual(t, expected, p)
 }
 
 func TestExecutionPayloadResponseCapellaToProto(t *testing.T) {
@@ -1222,6 +1352,16 @@ func pbEth1Data() *eth.Eth1Data {
 	}
 }
 
+func TestEth1DataMarshal(t *testing.T) {
+	ed := &Eth1Data{
+		Eth1Data: pbEth1Data(),
+	}
+	b, err := json.Marshal(ed)
+	require.NoError(t, err)
+	expected := `{"deposit_root":"0x0000000000000000000000000000000000000000000000000000000000000000","deposit_count":"23","block_hash":"0x0000000000000000000000000000000000000000000000000000000000000000"}`
+	require.Equal(t, expected, string(b))
+}
+
 func pbSyncAggregate() *eth.SyncAggregate {
 	return &eth.SyncAggregate{
 		SyncCommitteeSignature: make([]byte, 48),
@@ -1229,6 +1369,14 @@ func pbSyncAggregate() *eth.SyncAggregate {
 	}
 }
 
+func TestSyncAggregate_MarshalJSON(t *testing.T) {
+	sa := &SyncAggregate{pbSyncAggregate()}
+	b, err := json.Marshal(sa)
+	require.NoError(t, err)
+	expected := `{"sync_committee_bits":"0x01","sync_committee_signature":"0x000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"}`
+	require.Equal(t, expected, string(b))
+}
+
 func pbDeposit(t *testing.T) *eth.Deposit {
 	return &eth.Deposit{
 		Proof: [][]byte{ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2")},
@@ -1241,6 +1389,16 @@ func pbDeposit(t *testing.T) *eth.Deposit {
 	}
 }
 
+func TestDeposit_MarshalJSON(t *testing.T) {
+	d := &Deposit{
+		Deposit: pbDeposit(t),
+	}
+	b, err := json.Marshal(d)
+	require.NoError(t, err)
+	expected := `{"proof":["0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"],"data":{"pubkey":"0x93247f2209abcacf57b75a51dafae777f9dd38bc7053d1af526f220a7489a6d3a2753e5f3e8b1cfe39b56f43611df74a","withdrawal_credentials":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","amount":"1","signature":"0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"}}`
+	require.Equal(t, expected, string(b))
+}
+
 func pbSignedVoluntaryExit(t *testing.T) *eth.SignedVoluntaryExit {
 	return &eth.SignedVoluntaryExit{
 		Exit: &eth.VoluntaryExit{
@@ -1251,6 +1409,16 @@ func pbSignedVoluntaryExit(t *testing.T) *eth.SignedVoluntaryExit {
 	}
 }
 
+func TestVoluntaryExit(t *testing.T) {
+	ve := &SignedVoluntaryExit{
+		SignedVoluntaryExit: pbSignedVoluntaryExit(t),
+	}
+	b, err := json.Marshal(ve)
+	require.NoError(t, err)
+	expected := `{"message":{"epoch":"1","validator_index":"1"},"signature":"0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"}`
+	require.Equal(t, expected, string(b))
+}
+
 func pbAttestation(t *testing.T) *eth.Attestation {
 	return &eth.Attestation{
 		AggregationBits: bitfield.Bitlist{0x01},
@@ -1271,6 +1439,16 @@ func pbAttestation(t *testing.T) *eth.Attestation {
 	}
 }
 
+func TestAttestationMarshal(t *testing.T) {
+	a := &Attestation{
+		Attestation: pbAttestation(t),
+	}
+	b, err := json.Marshal(a)
+	require.NoError(t, err)
+	expected := `{"aggregation_bits":"0x01","data":{"slot":"1","index":"1","beacon_block_root":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","source":{"epoch":"1","root":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"},"target":{"epoch":"1","root":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"}},"signature":"0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"}`
+	require.Equal(t, expected, string(b))
+}
+
 func pbAttesterSlashing(t *testing.T) *eth.AttesterSlashing {
 	return &eth.AttesterSlashing{
 		Attestation_1: &eth.IndexedAttestation{
@@ -1311,7 +1489,9 @@ func pbAttesterSlashing(t *testing.T) *eth.AttesterSlashing {
 }
 
 func TestAttesterSlashing_MarshalJSON(t *testing.T) {
-	as := structs.AttesterSlashingFromConsensus(pbAttesterSlashing(t))
+	as := &AttesterSlashing{
+		AttesterSlashing: pbAttesterSlashing(t),
+	}
 	b, err := json.Marshal(as)
 	require.NoError(t, err)
 	expected := `{"attestation_1":{"attesting_indices":["1"],"data":{"slot":"1","index":"1","beacon_block_root":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","source":{"epoch":"1","root":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"},"target":{"epoch":"1","root":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"}},"signature":"0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"},"attestation_2":{"attesting_indices":["1"],"data":{"slot":"1","index":"1","beacon_block_root":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","source":{"epoch":"1","root":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"},"target":{"epoch":"1","root":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"}},"signature":"0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"}}`
@@ -1419,8 +1599,9 @@ func pbExecutionPayloadHeaderDeneb(t *testing.T) *v1.ExecutionPayloadHeaderDeneb
 }
 
 func TestExecutionPayloadHeader_MarshalJSON(t *testing.T) {
-	h, err := structs.ExecutionPayloadHeaderFromConsensus(pbExecutionPayloadHeader(t))
-	require.NoError(t, err)
+	h := &ExecutionPayloadHeader{
+		ExecutionPayloadHeader: pbExecutionPayloadHeader(t),
+	}
 	b, err := json.Marshal(h)
 	require.NoError(t, err)
 	expected := `{"parent_hash":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","fee_recipient":"0xabcf8e0d4e9587369b2301d0790347320302cc09","state_root":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","receipts_root":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","logs_bloom":"0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","prev_randao":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","block_number":"1","gas_limit":"1","gas_used":"1","timestamp":"1","extra_data":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","base_fee_per_gas":"452312848583266388373324160190187140051835877600158453279131187530910662656","block_hash":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","transactions_root":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"}`
@@ -1428,9 +1609,9 @@ func TestExecutionPayloadHeader_MarshalJSON(t *testing.T) {
 }
 
 func TestExecutionPayloadHeaderCapella_MarshalJSON(t *testing.T) {
-	h, err := structs.ExecutionPayloadHeaderCapellaFromConsensus(pbExecutionPayloadHeaderCapella(t))
-	require.NoError(t, err)
-	require.NoError(t, err)
+	h := &ExecutionPayloadHeaderCapella{
+		ExecutionPayloadHeaderCapella: pbExecutionPayloadHeaderCapella(t),
+	}
 	b, err := json.Marshal(h)
 	require.NoError(t, err)
 	expected := `{"parent_hash":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","fee_recipient":"0xabcf8e0d4e9587369b2301d0790347320302cc09","state_root":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","receipts_root":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","logs_bloom":"0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","prev_randao":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","block_number":"1","gas_limit":"1","gas_used":"1","timestamp":"1","extra_data":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","base_fee_per_gas":"452312848583266388373324160190187140051835877600158453279131187530910662656","block_hash":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","transactions_root":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","withdrawals_root":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"}`
@@ -1438,8 +1619,9 @@ func TestExecutionPayloadHeaderCapella_MarshalJSON(t *testing.T) {
 }
 
 func TestExecutionPayloadHeaderDeneb_MarshalJSON(t *testing.T) {
-	h, err := structs.ExecutionPayloadHeaderDenebFromConsensus(pbExecutionPayloadHeaderDeneb(t))
-	require.NoError(t, err)
+	h := &ExecutionPayloadHeaderDeneb{
+		ExecutionPayloadHeaderDeneb: pbExecutionPayloadHeaderDeneb(t),
+	}
 	b, err := json.Marshal(h)
 	require.NoError(t, err)
 	expected := `{"parent_hash":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","fee_recipient":"0xabcf8e0d4e9587369b2301d0790347320302cc09","state_root":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","receipts_root":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","logs_bloom":"0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","prev_randao":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","block_number":"1","gas_limit":"1","gas_used":"1","timestamp":"1","extra_data":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","base_fee_per_gas":"452312848583266388373324160190187140051835877600158453279131187530910662656","block_hash":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","transactions_root":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","withdrawals_root":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","blob_gas_used":"1","excess_blob_gas":"2"}`
@@ -1668,13 +1850,12 @@ func TestRoundTripUint256(t *testing.T) {
 func TestRoundTripProtoUint256(t *testing.T) {
 	h := pbExecutionPayloadHeader(t)
 	h.BaseFeePerGas = []byte{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31}
-	hm, err := structs.ExecutionPayloadHeaderFromConsensus(h)
-	require.NoError(t, err)
+	hm := &ExecutionPayloadHeader{ExecutionPayloadHeader: h}
 	m, err := json.Marshal(hm)
 	require.NoError(t, err)
-	hu := &structs.ExecutionPayloadHeader{}
+	hu := &ExecutionPayloadHeader{}
 	require.NoError(t, json.Unmarshal(m, hu))
-	hp, err := hu.ToConsensus()
+	hp, err := hu.ToProto()
 	require.NoError(t, err)
 	require.DeepEqual(t, h.BaseFeePerGas, hp.BaseFeePerGas)
 }
@@ -1682,7 +1863,7 @@ func TestRoundTripProtoUint256(t *testing.T) {
 func TestExecutionPayloadHeaderRoundtrip(t *testing.T) {
 	expected, err := os.ReadFile("testdata/execution-payload.json")
 	require.NoError(t, err)
-	hu := &structs.ExecutionPayloadHeader{}
+	hu := &ExecutionPayloadHeader{}
 	require.NoError(t, json.Unmarshal(expected, hu))
 	m, err := json.Marshal(hu)
 	require.NoError(t, err)
@@ -1692,7 +1873,7 @@ func TestExecutionPayloadHeaderRoundtrip(t *testing.T) {
 func TestExecutionPayloadHeaderCapellaRoundtrip(t *testing.T) {
 	expected, err := os.ReadFile("testdata/execution-payload-capella.json")
 	require.NoError(t, err)
-	hu := &structs.ExecutionPayloadHeaderCapella{}
+	hu := &ExecutionPayloadHeaderCapella{}
 	require.NoError(t, json.Unmarshal(expected, hu))
 	m, err := json.Marshal(hu)
 	require.NoError(t, err)
@@ -1813,9 +1994,11 @@ func TestEmptyResponseBody(t *testing.T) {
 			epr := &ExecutionPayloadResponse{}
 			require.NoError(t, json.Unmarshal(encoded, epr))
 			pp, err := epr.ParsePayload()
+			require.NoError(t, err)
+			pb, err := pp.PayloadProto()
 			if err == nil {
 				require.NoError(t, err)
-				require.Equal(t, false, pp == nil)
+				require.Equal(t, false, pb == nil)
 			} else {
 				require.ErrorIs(t, err, consensusblocks.ErrNilObject)
 			}

api/client/event/event_stream.go

@@ -14,10 +14,22 @@ import (
 )
 
 const (
-	EventHead = "head"
-
-	EventError           = "error"
-	EventConnectionError = "connection_error"
+	EventHead                        = "head"
+	EventBlock                       = "block"
+	EventAttestation                 = "attestation"
+	EventVoluntaryExit               = "voluntary_exit"
+	EventBlsToExecutionChange        = "bls_to_execution_change"
+	EventProposerSlashing            = "proposer_slashing"
+	EventAttesterSlashing            = "attester_slashing"
+	EventFinalizedCheckpoint         = "finalized_checkpoint"
+	EventChainReorg                  = "chain_reorg"
+	EventContributionAndProof        = "contribution_and_proof"
+	EventLightClientFinalityUpdate   = "light_client_finality_update"
+	EventLightClientOptimisticUpdate = "light_client_optimistic_update"
+	EventPayloadAttributes           = "payload_attributes"
+	EventBlobSidecar                 = "blob_sidecar"
+	EventError                       = "error"
+	EventConnectionError             = "connection_error"
 )
 
 var (

api/client/event/event_stream_test.go

@@ -83,7 +83,8 @@ func TestEventStream(t *testing.T) {
 func TestEventStreamRequestError(t *testing.T) {
 	topics := []string{"head"}
 	eventsChannel := make(chan *Event, 1)
-	ctx := t.Context()
+	ctx, cancel := context.WithCancel(context.Background())
+	defer cancel()
 
 	// use valid url that will result in failed request with nil body
 	stream, err := NewEventStream(ctx, http.DefaultClient, "http://badhost:1234", topics)

api/server/structs/BUILD.bazel

@@ -31,7 +31,6 @@ go_library(
         "//beacon-chain/state:go_default_library",
         "//config/fieldparams:go_default_library",
         "//config/params:go_default_library",
-        "//consensus-types/blocks:go_default_library",
         "//consensus-types/interfaces:go_default_library",
         "//consensus-types/primitives:go_default_library",
         "//consensus-types/validator:go_default_library",
@@ -45,7 +44,6 @@ go_library(
         "@com_github_ethereum_go_ethereum//common:go_default_library",
         "@com_github_ethereum_go_ethereum//common/hexutil:go_default_library",
         "@com_github_pkg_errors//:go_default_library",
-        "@org_golang_google_protobuf//proto:go_default_library",
     ],
 )
 
@@ -59,7 +57,6 @@ go_test(
     deps = [
         "//proto/engine/v1:go_default_library",
         "//proto/prysm/v1alpha1:go_default_library",
-        "//testing/assert:go_default_library",
         "//testing/require:go_default_library",
         "@com_github_ethereum_go_ethereum//common:go_default_library",
         "@com_github_ethereum_go_ethereum//common/hexutil:go_default_library",

api/server/structs/conversions_block_execution.go

@@ -7,15 +7,12 @@ import (
 	"github.com/OffchainLabs/prysm/v6/api/server"
 	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
 	"github.com/OffchainLabs/prysm/v6/config/params"
-	consensusblocks "github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
 	"github.com/OffchainLabs/prysm/v6/container/slice"
 	"github.com/OffchainLabs/prysm/v6/encoding/bytesutil"
 	enginev1 "github.com/OffchainLabs/prysm/v6/proto/engine/v1"
 	"github.com/ethereum/go-ethereum/common"
 	"github.com/ethereum/go-ethereum/common/hexutil"
-	"github.com/pkg/errors"
-	"google.golang.org/protobuf/proto"
 )
 
 // ----------------------------------------------------------------------------
@@ -135,13 +132,6 @@ func (e *ExecutionPayload) ToConsensus() (*enginev1.ExecutionPayload, error) {
 	}, nil
 }
 
-func (r *ExecutionPayload) PayloadProto() (proto.Message, error) {
-	if r == nil {
-		return nil, errors.Wrap(consensusblocks.ErrNilObject, "nil execution payload")
-	}
-	return r.ToConsensus()
-}
-
 func ExecutionPayloadHeaderFromConsensus(payload *enginev1.ExecutionPayloadHeader) (*ExecutionPayloadHeader, error) {
 	baseFeePerGas, err := sszBytesToUint256String(payload.BaseFeePerGas)
 	if err != nil {
@@ -393,13 +383,6 @@ func (e *ExecutionPayloadCapella) ToConsensus() (*enginev1.ExecutionPayloadCapel
 	}, nil
 }
 
-func (p *ExecutionPayloadCapella) PayloadProto() (proto.Message, error) {
-	if p == nil {
-		return nil, errors.Wrap(consensusblocks.ErrNilObject, "nil capella execution payload")
-	}
-	return p.ToConsensus()
-}
-
 func ExecutionPayloadHeaderCapellaFromConsensus(payload *enginev1.ExecutionPayloadHeaderCapella) (*ExecutionPayloadHeaderCapella, error) {
 	baseFeePerGas, err := sszBytesToUint256String(payload.BaseFeePerGas)
 	if err != nil {

api/server/structs/conversions_state.go

@@ -26,7 +26,10 @@ func BeaconStateFromConsensus(st beaconState.BeaconState) (*BeaconState, error)
 	for i, r := range srcSr {
 		sr[i] = hexutil.Encode(r)
 	}
-	srcHr := st.HistoricalRoots()
+	srcHr, err := st.HistoricalRoots()
+	if err != nil {
+		return nil, err
+	}
 	hr := make([]string, len(srcHr))
 	for i, r := range srcHr {
 		hr[i] = hexutil.Encode(r)
@@ -113,7 +116,10 @@ func BeaconStateAltairFromConsensus(st beaconState.BeaconState) (*BeaconStateAlt
 	for i, r := range srcSr {
 		sr[i] = hexutil.Encode(r)
 	}
-	srcHr := st.HistoricalRoots()
+	srcHr, err := st.HistoricalRoots()
+	if err != nil {
+		return nil, err
+	}
 	hr := make([]string, len(srcHr))
 	for i, r := range srcHr {
 		hr[i] = hexutil.Encode(r)
@@ -219,7 +225,10 @@ func BeaconStateBellatrixFromConsensus(st beaconState.BeaconState) (*BeaconState
 	for i, r := range srcSr {
 		sr[i] = hexutil.Encode(r)
 	}
-	srcHr := st.HistoricalRoots()
+	srcHr, err := st.HistoricalRoots()
+	if err != nil {
+		return nil, err
+	}
 	hr := make([]string, len(srcHr))
 	for i, r := range srcHr {
 		hr[i] = hexutil.Encode(r)
@@ -338,7 +347,10 @@ func BeaconStateCapellaFromConsensus(st beaconState.BeaconState) (*BeaconStateCa
 	for i, r := range srcSr {
 		sr[i] = hexutil.Encode(r)
 	}
-	srcHr := st.HistoricalRoots()
+	srcHr, err := st.HistoricalRoots()
+	if err != nil {
+		return nil, err
+	}
 	hr := make([]string, len(srcHr))
 	for i, r := range srcHr {
 		hr[i] = hexutil.Encode(r)
@@ -476,7 +488,10 @@ func BeaconStateDenebFromConsensus(st beaconState.BeaconState) (*BeaconStateDene
 	for i, r := range srcSr {
 		sr[i] = hexutil.Encode(r)
 	}
-	srcHr := st.HistoricalRoots()
+	srcHr, err := st.HistoricalRoots()
+	if err != nil {
+		return nil, err
+	}
 	hr := make([]string, len(srcHr))
 	for i, r := range srcHr {
 		hr[i] = hexutil.Encode(r)
@@ -614,7 +629,10 @@ func BeaconStateElectraFromConsensus(st beaconState.BeaconState) (*BeaconStateEl
 	for i, r := range srcSr {
 		sr[i] = hexutil.Encode(r)
 	}
-	srcHr := st.HistoricalRoots()
+	srcHr, err := st.HistoricalRoots()
+	if err != nil {
+		return nil, err
+	}
 	hr := make([]string, len(srcHr))
 	for i, r := range srcHr {
 		hr[i] = hexutil.Encode(r)
@@ -797,7 +815,10 @@ func BeaconStateFuluFromConsensus(st beaconState.BeaconState) (*BeaconStateFulu,
 	for i, r := range srcSr {
 		sr[i] = hexutil.Encode(r)
 	}
-	srcHr := st.HistoricalRoots()
+	srcHr, err := st.HistoricalRoots()
+	if err != nil {
+		return nil, err
+	}
 	hr := make([]string, len(srcHr))
 	for i, r := range srcHr {
 		hr[i] = hexutil.Encode(r)

api/server/structs/conversions_test.go

@@ -4,9 +4,7 @@ import (
 	"testing"
 
 	eth "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
-	"github.com/OffchainLabs/prysm/v6/testing/assert"
 	"github.com/OffchainLabs/prysm/v6/testing/require"
-	"github.com/ethereum/go-ethereum/common/hexutil"
 )
 
 func TestDepositSnapshotFromConsensus(t *testing.T) {
@@ -104,266 +102,12 @@ func TestProposerSlashing_ToConsensus(t *testing.T) {
 	require.ErrorContains(t, errNilValue.Error(), err)
 }
 
-func TestProposerSlashing_FromConsensus(t *testing.T) {
-	input := []*eth.ProposerSlashing{
-		{
-			Header_1: &eth.SignedBeaconBlockHeader{
-				Header: &eth.BeaconBlockHeader{
-					Slot:          1,
-					ProposerIndex: 2,
-					ParentRoot:    []byte{3},
-					StateRoot:     []byte{4},
-					BodyRoot:      []byte{5},
-				},
-				Signature: []byte{6},
-			},
-			Header_2: &eth.SignedBeaconBlockHeader{
-				Header: &eth.BeaconBlockHeader{
-					Slot:          7,
-					ProposerIndex: 8,
-					ParentRoot:    []byte{9},
-					StateRoot:     []byte{10},
-					BodyRoot:      []byte{11},
-				},
-				Signature: []byte{12},
-			},
-		},
-		{
-			Header_1: &eth.SignedBeaconBlockHeader{
-				Header: &eth.BeaconBlockHeader{
-					Slot:          13,
-					ProposerIndex: 14,
-					ParentRoot:    []byte{15},
-					StateRoot:     []byte{16},
-					BodyRoot:      []byte{17},
-				},
-				Signature: []byte{18},
-			},
-			Header_2: &eth.SignedBeaconBlockHeader{
-				Header: &eth.BeaconBlockHeader{
-					Slot:          19,
-					ProposerIndex: 20,
-					ParentRoot:    []byte{21},
-					StateRoot:     []byte{22},
-					BodyRoot:      []byte{23},
-				},
-				Signature: []byte{24},
-			},
-		},
-	}
-
-	expectedResult := []*ProposerSlashing{
-		{
-			SignedHeader1: &SignedBeaconBlockHeader{
-				Message: &BeaconBlockHeader{
-					Slot:          "1",
-					ProposerIndex: "2",
-					ParentRoot:    hexutil.Encode([]byte{3}),
-					StateRoot:     hexutil.Encode([]byte{4}),
-					BodyRoot:      hexutil.Encode([]byte{5}),
-				},
-				Signature: hexutil.Encode([]byte{6}),
-			},
-			SignedHeader2: &SignedBeaconBlockHeader{
-				Message: &BeaconBlockHeader{
-					Slot:          "7",
-					ProposerIndex: "8",
-					ParentRoot:    hexutil.Encode([]byte{9}),
-					StateRoot:     hexutil.Encode([]byte{10}),
-					BodyRoot:      hexutil.Encode([]byte{11}),
-				},
-				Signature: hexutil.Encode([]byte{12}),
-			},
-		},
-		{
-			SignedHeader1: &SignedBeaconBlockHeader{
-				Message: &BeaconBlockHeader{
-					Slot:          "13",
-					ProposerIndex: "14",
-					ParentRoot:    hexutil.Encode([]byte{15}),
-					StateRoot:     hexutil.Encode([]byte{16}),
-					BodyRoot:      hexutil.Encode([]byte{17}),
-				},
-				Signature: hexutil.Encode([]byte{18}),
-			},
-			SignedHeader2: &SignedBeaconBlockHeader{
-				Message: &BeaconBlockHeader{
-					Slot:          "19",
-					ProposerIndex: "20",
-					ParentRoot:    hexutil.Encode([]byte{21}),
-					StateRoot:     hexutil.Encode([]byte{22}),
-					BodyRoot:      hexutil.Encode([]byte{23}),
-				},
-				Signature: hexutil.Encode([]byte{24}),
-			},
-		},
-	}
-
-	result := ProposerSlashingsFromConsensus(input)
-	assert.DeepEqual(t, expectedResult, result)
-}
-
 func TestAttesterSlashing_ToConsensus(t *testing.T) {
 	a := &AttesterSlashing{Attestation1: nil, Attestation2: nil}
 	_, err := a.ToConsensus()
 	require.ErrorContains(t, errNilValue.Error(), err)
 }
 
-func TestAttesterSlashing_FromConsensus(t *testing.T) {
-	input := []*eth.AttesterSlashing{
-		{
-			Attestation_1: &eth.IndexedAttestation{
-				AttestingIndices: []uint64{1, 2},
-				Data: &eth.AttestationData{
-					Slot:            3,
-					CommitteeIndex:  4,
-					BeaconBlockRoot: []byte{5},
-					Source: &eth.Checkpoint{
-						Epoch: 6,
-						Root:  []byte{7},
-					},
-					Target: &eth.Checkpoint{
-						Epoch: 8,
-						Root:  []byte{9},
-					},
-				},
-				Signature: []byte{10},
-			},
-			Attestation_2: &eth.IndexedAttestation{
-				AttestingIndices: []uint64{11, 12},
-				Data: &eth.AttestationData{
-					Slot:            13,
-					CommitteeIndex:  14,
-					BeaconBlockRoot: []byte{15},
-					Source: &eth.Checkpoint{
-						Epoch: 16,
-						Root:  []byte{17},
-					},
-					Target: &eth.Checkpoint{
-						Epoch: 18,
-						Root:  []byte{19},
-					},
-				},
-				Signature: []byte{20},
-			},
-		},
-		{
-			Attestation_1: &eth.IndexedAttestation{
-				AttestingIndices: []uint64{21, 22},
-				Data: &eth.AttestationData{
-					Slot:            23,
-					CommitteeIndex:  24,
-					BeaconBlockRoot: []byte{25},
-					Source: &eth.Checkpoint{
-						Epoch: 26,
-						Root:  []byte{27},
-					},
-					Target: &eth.Checkpoint{
-						Epoch: 28,
-						Root:  []byte{29},
-					},
-				},
-				Signature: []byte{30},
-			},
-			Attestation_2: &eth.IndexedAttestation{
-				AttestingIndices: []uint64{31, 32},
-				Data: &eth.AttestationData{
-					Slot:            33,
-					CommitteeIndex:  34,
-					BeaconBlockRoot: []byte{35},
-					Source: &eth.Checkpoint{
-						Epoch: 36,
-						Root:  []byte{37},
-					},
-					Target: &eth.Checkpoint{
-						Epoch: 38,
-						Root:  []byte{39},
-					},
-				},
-				Signature: []byte{40},
-			},
-		},
-	}
-
-	expectedResult := []*AttesterSlashing{
-		{
-			Attestation1: &IndexedAttestation{
-				AttestingIndices: []string{"1", "2"},
-				Data: &AttestationData{
-					Slot:            "3",
-					CommitteeIndex:  "4",
-					BeaconBlockRoot: hexutil.Encode([]byte{5}),
-					Source: &Checkpoint{
-						Epoch: "6",
-						Root:  hexutil.Encode([]byte{7}),
-					},
-					Target: &Checkpoint{
-						Epoch: "8",
-						Root:  hexutil.Encode([]byte{9}),
-					},
-				},
-				Signature: hexutil.Encode([]byte{10}),
-			},
-			Attestation2: &IndexedAttestation{
-				AttestingIndices: []string{"11", "12"},
-				Data: &AttestationData{
-					Slot:            "13",
-					CommitteeIndex:  "14",
-					BeaconBlockRoot: hexutil.Encode([]byte{15}),
-					Source: &Checkpoint{
-						Epoch: "16",
-						Root:  hexutil.Encode([]byte{17}),
-					},
-					Target: &Checkpoint{
-						Epoch: "18",
-						Root:  hexutil.Encode([]byte{19}),
-					},
-				},
-				Signature: hexutil.Encode([]byte{20}),
-			},
-		},
-		{
-			Attestation1: &IndexedAttestation{
-				AttestingIndices: []string{"21", "22"},
-				Data: &AttestationData{
-					Slot:            "23",
-					CommitteeIndex:  "24",
-					BeaconBlockRoot: hexutil.Encode([]byte{25}),
-					Source: &Checkpoint{
-						Epoch: "26",
-						Root:  hexutil.Encode([]byte{27}),
-					},
-					Target: &Checkpoint{
-						Epoch: "28",
-						Root:  hexutil.Encode([]byte{29}),
-					},
-				},
-				Signature: hexutil.Encode([]byte{30}),
-			},
-			Attestation2: &IndexedAttestation{
-				AttestingIndices: []string{"31", "32"},
-				Data: &AttestationData{
-					Slot:            "33",
-					CommitteeIndex:  "34",
-					BeaconBlockRoot: hexutil.Encode([]byte{35}),
-					Source: &Checkpoint{
-						Epoch: "36",
-						Root:  hexutil.Encode([]byte{37}),
-					},
-					Target: &Checkpoint{
-						Epoch: "38",
-						Root:  hexutil.Encode([]byte{39}),
-					},
-				},
-				Signature: hexutil.Encode([]byte{40}),
-			},
-		},
-	}
-
-	result := AttesterSlashingsFromConsensus(input)
-	assert.DeepEqual(t, expectedResult, result)
-}
-
 func TestIndexedAttestation_ToConsensus(t *testing.T) {
 	a := &IndexedAttestation{
 		AttestingIndices: []string{"1"},

api/server/structs/endpoints_beacon.go

@@ -263,13 +263,6 @@ type ChainHead struct {
 	OptimisticStatus           bool   `json:"optimistic_status"`
 }
 
-type GetPendingConsolidationsResponse struct {
-	Version             string                  `json:"version"`
-	ExecutionOptimistic bool                    `json:"execution_optimistic"`
-	Finalized           bool                    `json:"finalized"`
-	Data                []*PendingConsolidation `json:"data"`
-}
-
 type GetPendingDepositsResponse struct {
 	Version             string            `json:"version"`
 	ExecutionOptimistic bool              `json:"execution_optimistic"`

api/server/structs/other.go

@@ -253,7 +253,7 @@ type PendingDeposit struct {
 }
 
 type PendingPartialWithdrawal struct {
-	Index             string `json:"validator_index"`
+	Index             string `json:"index"`
 	Amount            string `json:"amount"`
 	WithdrawableEpoch string `json:"withdrawable_epoch"`
 }

beacon-chain/README.md

@@ -4,6 +4,6 @@ This is the main project folder for the beacon chain implementation of Ethereum
 
 You can also read our main [README](https://github.com/prysmaticlabs/prysm/blob/master/README.md) and join our active chat room on Discord.
 
-[![Discord](https://user-images.githubusercontent.com/7288322/34471967-1df7808a-efbb-11e7-9088-ed0b04151291.png)](https://discord.gg/prysm)
+[![Discord](https://user-images.githubusercontent.com/7288322/34471967-1df7808a-efbb-11e7-9088-ed0b04151291.png)](https://discord.gg/CTYGPUJ)
 
 Also, read the official beacon chain [specification](https://github.com/ethereum/consensus-specs/blob/master/specs/phase0/beacon-chain.md), this design spec serves as a source of truth for the beacon chain implementation we follow at Prysmatic Labs.

beacon-chain/blockchain/BUILD.bazel

@@ -25,6 +25,7 @@ go_library(
         "receive_attestation.go",
         "receive_blob.go",
         "receive_block.go",
+        "receive_data_column.go",
         "service.go",
         "setup_forchoice.go",
         "tracked_proposer.go",
@@ -50,6 +51,7 @@ go_library(
         "//beacon-chain/core/feed/state:go_default_library",
         "//beacon-chain/core/helpers:go_default_library",
         "//beacon-chain/core/light-client:go_default_library",
+        "//beacon-chain/core/peerdas:go_default_library",
         "//beacon-chain/core/signing:go_default_library",
         "//beacon-chain/core/time:go_default_library",
         "//beacon-chain/core/transition:go_default_library",
@@ -146,6 +148,7 @@ go_test(
         "//beacon-chain/core/feed/state:go_default_library",
         "//beacon-chain/core/helpers:go_default_library",
         "//beacon-chain/core/light-client:go_default_library",
+        "//beacon-chain/core/peerdas:go_default_library",
         "//beacon-chain/core/signing:go_default_library",
         "//beacon-chain/core/transition:go_default_library",
         "//beacon-chain/das:go_default_library",

beacon-chain/blockchain/error.go

@@ -40,10 +40,12 @@ var (
 	errNotGenesisRoot = errors.New("root is not the genesis block root")
 	// errBlacklistedBlock is returned when a block is blacklisted as invalid.
 	errBlacklistedRoot = verification.AsVerificationFailure(errors.New("block root is blacklisted"))
+	// errMaxBlobsExceeded is returned when the number of blobs in a block exceeds the maximum allowed.
+	errMaxBlobsExceeded = verification.AsVerificationFailure(errors.New("Expected commitments in block exceeds MAX_BLOBS_PER_BLOCK"))
+	// errMaxDataColumnsExceeded is returned when the number of data columns exceeds the maximum allowed.
+	errMaxDataColumnsExceeded = verification.AsVerificationFailure(errors.New("Expected data columns for node exceeds NUMBER_OF_COLUMNS"))
 )
 
-var errMaxBlobsExceeded = verification.AsVerificationFailure(errors.New("Expected commitments in block exceeds MAX_BLOBS_PER_BLOCK"))
-
 // An invalid block is the block that fails state transition based on the core protocol rules.
 // The beacon node shall not be accepting nor building blocks that branch off from an invalid block.
 // Some examples of invalid blocks are:

beacon-chain/blockchain/execution_engine.go

@@ -184,17 +184,13 @@ func (s *Service) notifyForkchoiceUpdate(ctx context.Context, arg *fcuConfig) (*
 	return payloadID, nil
 }
 
-func (s *Service) firePayloadAttributesEvent(f event.SubscriberSender, block interfaces.ReadOnlySignedBeaconBlock, root [32]byte, nextSlot primitives.Slot) {
-	// If we're syncing a block in the past and init-sync is still running, we shouldn't fire this event.
-	if !s.cfg.SyncChecker.Synced() {
-		return
-	}
+func firePayloadAttributesEvent(_ context.Context, f event.SubscriberSender, nextSlot primitives.Slot) {
 	// the fcu args have differing amounts of completeness based on the code path,
 	// and there is work we only want to do if a client is actually listening to the events beacon api endpoint.
 	// temporary solution: just fire a blank event and fill in the details in the api handler.
 	f.Send(&feed.Event{
 		Type: statefeed.PayloadAttributes,
-		Data: payloadattribute.EventData{HeadBlock: block, HeadRoot: root, ProposalSlot: nextSlot},
+		Data: payloadattribute.EventData{ProposalSlot: nextSlot},
 	})
 }
 
@@ -439,6 +435,9 @@ func (s *Service) removeInvalidBlockAndState(ctx context.Context, blkRoots [][32
 			// Blobs may not exist for some blocks, leading to deletion failures. Log such errors at debug level.
 			log.WithError(err).Debug("Could not remove blob from blob storage")
 		}
+		if err := s.dataColumnStorage.Remove(root); err != nil {
+			log.WithError(err).Debug("Could not remove data columns from data column storage")
+		}
 	}
 	return nil
 }

beacon-chain/blockchain/forkchoice_update_execution.go

@@ -102,7 +102,7 @@ func (s *Service) forkchoiceUpdateWithExecution(ctx context.Context, args *fcuCo
 		log.WithError(err).Error("could not save head")
 	}
 
-	go s.firePayloadAttributesEvent(s.cfg.StateNotifier.StateFeed(), args.headBlock, args.headRoot, s.CurrentSlot()+1)
+	go firePayloadAttributesEvent(ctx, s.cfg.StateNotifier.StateFeed(), s.CurrentSlot()+1)
 
 	// Only need to prune attestations from pool if the head has changed.
 	s.pruneAttsFromPool(s.ctx, args.headState, args.headBlock)

beacon-chain/blockchain/head.go

@@ -285,7 +285,10 @@ func (s *Service) headState(ctx context.Context) state.BeaconState {
 // This returns a read only version of the head state.
 // It does not perform a copy of the head state.
 // This is a lock free version.
-func (s *Service) headStateReadOnly(_ context.Context) state.ReadOnlyBeaconState {
+func (s *Service) headStateReadOnly(ctx context.Context) state.ReadOnlyBeaconState {
+	_, span := trace.StartSpan(ctx, "blockChain.headStateReadOnly")
+	defer span.End()
+
 	return s.head.state
 }
 

beacon-chain/blockchain/kzg/BUILD.bazel

@@ -7,7 +7,7 @@ go_library(
         "trusted_setup.go",
         "validation.go",
     ],
-    embedsrcs = ["trusted_setup_4096.json"],
+    embedsrcs = ["trusted_setup.json"],
     importpath = "github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg",
     visibility = ["//visibility:public"],
     deps = [
@@ -29,8 +29,9 @@ go_test(
     embed = [":go_default_library"],
     deps = [
         "//consensus-types/blocks:go_default_library",
-        "//crypto/random:go_default_library",
         "//testing/require:go_default_library",
+        "@com_github_consensys_gnark_crypto//ecc/bls12-381/fr:go_default_library",
         "@com_github_crate_crypto_go_kzg_4844//:go_default_library",
+        "@com_github_sirupsen_logrus//:go_default_library",
     ],
 )

beacon-chain/blockchain/kzg/kzg.go

@@ -39,8 +39,7 @@ type CellsAndProofs struct {
 
 // BlobToKZGCommitment computes a KZG commitment from a given blob.
 func BlobToKZGCommitment(blob *Blob) (Commitment, error) {
-	var kzgBlob kzg4844.Blob
-	copy(kzgBlob[:], blob[:])
+	kzgBlob := kzg4844.Blob(*blob)
 
 	commitment, err := kzg4844.BlobToCommitment(&kzgBlob)
 	if err != nil {
@@ -52,10 +51,9 @@ func BlobToKZGCommitment(blob *Blob) (Commitment, error) {
 
 // ComputeCells computes the (extended) cells from a given blob.
 func ComputeCells(blob *Blob) ([]Cell, error) {
-	var ckzgBlob ckzg4844.Blob
-	copy(ckzgBlob[:], blob[:])
+	ckzgBlob := (*ckzg4844.Blob)(blob)
 
-	ckzgCells, err := ckzg4844.ComputeCells(&ckzgBlob)
+	ckzgCells, err := ckzg4844.ComputeCells(ckzgBlob)
 	if err != nil {
 		return nil, errors.Wrap(err, "compute cells")
 	}
@@ -70,8 +68,7 @@ func ComputeCells(blob *Blob) ([]Cell, error) {
 
 // ComputeBlobKZGProof computes the blob KZG proof from a given blob and its commitment.
 func ComputeBlobKZGProof(blob *Blob, commitment Commitment) (Proof, error) {
-	var kzgBlob kzg4844.Blob
-	copy(kzgBlob[:], blob[:])
+	kzgBlob := kzg4844.Blob(*blob)
 
 	proof, err := kzg4844.ComputeBlobProof(&kzgBlob, kzg4844.Commitment(commitment))
 	if err != nil {
@@ -82,10 +79,9 @@ func ComputeBlobKZGProof(blob *Blob, commitment Commitment) (Proof, error) {
 
 // ComputeCellsAndKZGProofs computes the cells and cells KZG proofs from a given blob.
 func ComputeCellsAndKZGProofs(blob *Blob) (CellsAndProofs, error) {
-	var ckzgBlob ckzg4844.Blob
-	copy(ckzgBlob[:], blob[:])
+	ckzgBlob := (*ckzg4844.Blob)(blob)
 
-	ckzgCells, ckzgProofs, err := ckzg4844.ComputeCellsAndKZGProofs(&ckzgBlob)
+	ckzgCells, ckzgProofs, err := ckzg4844.ComputeCellsAndKZGProofs(ckzgBlob)
 	if err != nil {
 		return CellsAndProofs{}, err
 	}
@@ -93,7 +89,7 @@ func ComputeCellsAndKZGProofs(blob *Blob) (CellsAndProofs, error) {
 	return makeCellsAndProofs(ckzgCells[:], ckzgProofs[:])
 }
 
-// VerifyCellKZGProofBatch verifies the KZG proofs for a given slice of commitments, cells indices, cells and proofs.
+// VerifyBlobKZGProof verifies the KZG proofs for a given slice of commitments, cells indices, cells and proofs.
 // Note: It is way more efficient to call once this function with big slices than calling it multiple times with small slices.
 func VerifyCellKZGProofBatch(commitmentsBytes []Bytes48, cellIndices []uint64, cells []Cell, proofsBytes []Bytes48) (bool, error) {
 	// Convert `Cell` type to `ckzg4844.Cell`
@@ -128,9 +124,8 @@ func makeCellsAndProofs(ckzgCells []ckzg4844.Cell, ckzgProofs []ckzg4844.KZGProo
 		return CellsAndProofs{}, errors.New("different number of cells/proofs")
 	}
 
-	cells := make([]Cell, 0, len(ckzgCells))
-	proofs := make([]Proof, 0, len(ckzgProofs))
-
+	var cells []Cell
+	var proofs []Proof
 	for i := range ckzgCells {
 		cells = append(cells, Cell(ckzgCells[i]))
 		proofs = append(proofs, Proof(ckzgProofs[i]))

beacon-chain/blockchain/kzg/trusted_setup.go

@@ -11,8 +11,7 @@ import (
 )
 
 var (
-	// https://github.com/ethereum/consensus-specs/blob/dev/presets/mainnet/trusted_setups/trusted_setup_4096.json
-	//go:embed trusted_setup_4096.json
+	//go:embed trusted_setup.json
 	embeddedTrustedSetup []byte // 1.2Mb
 	kzgContext           *GoKZG.Context
 	kzgLoaded            bool
@@ -30,11 +29,9 @@ func Start() error {
 	if err != nil {
 		return errors.Wrap(err, "could not parse trusted setup JSON")
 	}
-
 	kzgContext, err = GoKZG.NewContext4096(&GoKZG.JSONTrustedSetup{
 		SetupG2:         trustedSetup.G2Monomial[:],
-		SetupG1Lagrange: trustedSetup.G1Lagrange,
-	})
+		SetupG1Lagrange: trustedSetup.G1Lagrange})
 	if err != nil {
 		return errors.Wrap(err, "could not initialize go-kzg context")
 	}

beacon-chain/blockchain/kzg/validation_test.go

@@ -1,12 +1,16 @@
 package kzg
 
 import (
+	"bytes"
+	"crypto/sha256"
+	"encoding/binary"
 	"testing"
 
 	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
-	"github.com/OffchainLabs/prysm/v6/crypto/random"
 	"github.com/OffchainLabs/prysm/v6/testing/require"
+	"github.com/consensys/gnark-crypto/ecc/bls12-381/fr"
 	GoKZG "github.com/crate-crypto/go-kzg-4844"
+	"github.com/sirupsen/logrus"
 )
 
 func GenerateCommitmentAndProof(blob GoKZG.Blob) (GoKZG.KZGCommitment, GoKZG.KZGProof, error) {
@@ -37,7 +41,7 @@ func TestBytesToAny(t *testing.T) {
 }
 
 func TestGenerateCommitmentAndProof(t *testing.T) {
-	blob := random.GetRandBlob(123)
+	blob := getRandBlob(123)
 	commitment, proof, err := GenerateCommitmentAndProof(blob)
 	require.NoError(t, err)
 	expectedCommitment := GoKZG.KZGCommitment{180, 218, 156, 194, 59, 20, 10, 189, 186, 254, 132, 93, 7, 127, 104, 172, 238, 240, 237, 70, 83, 89, 1, 152, 99, 0, 165, 65, 143, 62, 20, 215, 230, 14, 205, 95, 28, 245, 54, 25, 160, 16, 178, 31, 232, 207, 38, 85}
@@ -45,3 +49,36 @@ func TestGenerateCommitmentAndProof(t *testing.T) {
 	require.Equal(t, expectedCommitment, commitment)
 	require.Equal(t, expectedProof, proof)
 }
+
+func deterministicRandomness(seed int64) [32]byte {
+	// Converts an int64 to a byte slice
+	buf := new(bytes.Buffer)
+	err := binary.Write(buf, binary.BigEndian, seed)
+	if err != nil {
+		logrus.WithError(err).Error("Failed to write int64 to bytes buffer")
+		return [32]byte{}
+	}
+	bytes := buf.Bytes()
+
+	return sha256.Sum256(bytes)
+}
+
+// Returns a serialized random field element in big-endian
+func getRandFieldElement(seed int64) [32]byte {
+	bytes := deterministicRandomness(seed)
+	var r fr.Element
+	r.SetBytes(bytes[:])
+
+	return GoKZG.SerializeScalar(r)
+}
+
+// Returns a random blob using the passed seed as entropy
+func getRandBlob(seed int64) GoKZG.Blob {
+	var blob GoKZG.Blob
+	bytesPerBlob := GoKZG.ScalarsPerBlob * GoKZG.SerializedScalarSize
+	for i := 0; i < bytesPerBlob; i += GoKZG.SerializedScalarSize {
+		fieldElementBytes := getRandFieldElement(seed + int64(i))
+		copy(blob[i:i+GoKZG.SerializedScalarSize], fieldElementBytes[:])
+	}
+	return blob
+}

beacon-chain/blockchain/mock_test.go

@@ -20,7 +20,6 @@ func testServiceOptsWithDB(t *testing.T) []Option {
 		WithForkChoiceStore(fcs),
 		WithClockSynchronizer(cs),
 		WithStateNotifier(&mock.MockStateNotifier{RecordEvents: true}),
-		WithSyncChecker(&mock.MockSyncChecker{}),
 	}
 }
 

beacon-chain/blockchain/options.go

@@ -1,10 +1,13 @@
 package blockchain
 
 import (
+	"time"
+
 	"github.com/OffchainLabs/prysm/v6/async/event"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/cache"
 	statefeed "github.com/OffchainLabs/prysm/v6/beacon-chain/core/feed/state"
 	lightclient "github.com/OffchainLabs/prysm/v6/beacon-chain/core/light-client"
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/db"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/db/filesystem"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/execution"
@@ -127,9 +130,9 @@ func WithBLSToExecPool(p blstoexec.PoolManager) Option {
 }
 
 // WithP2PBroadcaster to broadcast messages after appropriate processing.
-func WithP2PBroadcaster(p p2p.Broadcaster) Option {
+func WithP2PBroadcaster(p p2p.Acceser) Option {
 	return func(s *Service) error {
-		s.cfg.P2p = p
+		s.cfg.P2P = p
 		return nil
 	}
 }
@@ -208,6 +211,14 @@ func WithBlobStorage(b *filesystem.BlobStorage) Option {
 	}
 }
 
+// WithDataColumnStorage sets the data column storage backend for the blockchain service.
+func WithDataColumnStorage(b *filesystem.DataColumnStorage) Option {
+	return func(s *Service) error {
+		s.dataColumnStorage = b
+		return nil
+	}
+}
+
 func WithSyncChecker(checker Checker) Option {
 	return func(s *Service) error {
 		s.cfg.SyncChecker = checker
@@ -215,6 +226,13 @@ func WithSyncChecker(checker Checker) Option {
 	}
 }
 
+func WithCustodyInfo(custodyInfo *peerdas.CustodyInfo) Option {
+	return func(s *Service) error {
+		s.cfg.CustodyInfo = custodyInfo
+		return nil
+	}
+}
+
 func WithSlasherEnabled(enabled bool) Option {
 	return func(s *Service) error {
 		s.slasherEnabled = enabled
@@ -222,6 +240,13 @@ func WithSlasherEnabled(enabled bool) Option {
 	}
 }
 
+func WithGenesisTime(genesisTime time.Time) Option {
+	return func(s *Service) error {
+		s.genesisTime = genesisTime
+		return nil
+	}
+}
+
 func WithLightClientStore(lcs *lightclient.Store) Option {
 	return func(s *Service) error {
 		s.lcStore = lcs

beacon-chain/blockchain/process_block.go

@@ -3,10 +3,12 @@ package blockchain
 import (
 	"context"
 	"fmt"
+	"slices"
 	"time"
 
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/blocks"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
 	coreTime "github.com/OffchainLabs/prysm/v6/beacon-chain/core/time"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/transition"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/das"
@@ -239,8 +241,9 @@ func (s *Service) onBlockBatch(ctx context.Context, blks []consensusblocks.ROBlo
 				return err
 			}
 		}
+
 		if err := avs.IsDataAvailable(ctx, s.CurrentSlot(), b); err != nil {
-			return errors.Wrapf(err, "could not validate blob data availability at slot %d", b.Block().Slot())
+			return errors.Wrapf(err, "could not validate sidecar availability at slot %d", b.Block().Slot())
 		}
 		args := &forkchoicetypes.BlockAndCheckpoints{Block: b,
 			JustifiedCheckpoint: jCheckpoints[i],
@@ -601,20 +604,206 @@ func missingIndices(bs *filesystem.BlobStorage, root [32]byte, expected [][]byte
 	return missing, nil
 }
 
-// isDataAvailable blocks until all BlobSidecars committed to in the block are available,
-// or an error or context cancellation occurs. A nil result means that the data availability check is successful.
-// The function will first check the database to see if all sidecars have been persisted. If any
-// sidecars are missing, it will then read from the blobNotifier channel for the given root until the channel is
-// closed, the context hits cancellation/timeout, or notifications have been received for all the missing sidecars.
-func (s *Service) isDataAvailable(ctx context.Context, root [32]byte, signed interfaces.ReadOnlySignedBeaconBlock) error {
-	if signed.Version() < version.Deneb {
-		return nil
+func missingDataColumns(bs *filesystem.DataColumnStorage, root [32]byte, expected map[uint64]bool) (map[uint64]bool, error) {
+	if len(expected) == 0 {
+		return nil, nil
 	}
 
-	block := signed.Block()
+	if len(expected) > int(params.BeaconConfig().NumberOfColumns) {
+		return nil, errMaxDataColumnsExceeded
+	}
+
+	// Get a summary of the data columns stored in the database.
+	summary := bs.Summary(root)
+
+	// Check all expected data columns against the summary.
+	missing := make(map[uint64]bool)
+	for column := range expected {
+		if !summary.HasIndex(column) {
+			missing[column] = true
+		}
+	}
+
+	return missing, nil
+}
+
+// isDataAvailable blocks until all sidecars committed to in the block are available,
+// or an error or context cancellation occurs. A nil result means that the data availability check is successful.
+// The function will first check the database to see if all sidecars have been persisted. If any
+// sidecars are missing, it will then read from the sidecar notifier channel for the given root until the channel is
+// closed, the context hits cancellation/timeout, or notifications have been received for all the missing sidecars.
+func (s *Service) isDataAvailable(ctx context.Context, root [32]byte, signedBlock interfaces.ReadOnlySignedBeaconBlock) error {
+	block := signedBlock.Block()
 	if block == nil {
 		return errors.New("invalid nil beacon block")
 	}
+
+	blockVersion := block.Version()
+	if blockVersion >= version.Fulu {
+		return s.areDataColumnsAvailable(ctx, root, block)
+	}
+
+	if blockVersion >= version.Deneb {
+		return s.areBlobsAvailable(ctx, root, block)
+	}
+
+	return nil
+}
+
+// areDataColumnsAvailable blocks until all data columns committed to in the block are available,
+// or an error or context cancellation occurs. A nil result means that the data availability check is successful.
+func (s *Service) areDataColumnsAvailable(ctx context.Context, root [fieldparams.RootLength]byte, block interfaces.ReadOnlyBeaconBlock) error {
+	// We are only required to check within MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS
+	blockSlot, currentSlot := block.Slot(), s.CurrentSlot()
+	blockEpoch, currentEpoch := slots.ToEpoch(blockSlot), slots.ToEpoch(currentSlot)
+
+	if !params.WithinDAPeriod(blockEpoch, currentEpoch) {
+		return nil
+	}
+
+	body := block.Body()
+	if body == nil {
+		return errors.New("invalid nil beacon block body")
+	}
+
+	kzgCommitments, err := body.BlobKzgCommitments()
+	if err != nil {
+		return errors.Wrap(err, "blob KZG commitments")
+	}
+
+	// If block has not commitments there is nothing to wait for.
+	if len(kzgCommitments) == 0 {
+		return nil
+	}
+
+	// All columns to sample need to be available for the block to be considered available.
+	// https://github.com/ethereum/consensus-specs/blob/v1.5.0-alpha.10/specs/fulu/das-core.md#custody-sampling
+	nodeID := s.cfg.P2P.NodeID()
+
+	// Prevent custody group count to change during the rest of the function.
+	s.cfg.CustodyInfo.Mut.RLock()
+	defer s.cfg.CustodyInfo.Mut.RUnlock()
+
+	// Get the custody group sampling size for the node.
+	custodyGroupSamplingSize := s.cfg.CustodyInfo.CustodyGroupSamplingSize(peerdas.Actual)
+	peerInfo, _, err := peerdas.Info(nodeID, custodyGroupSamplingSize)
+	if err != nil {
+		return errors.Wrap(err, "peer info")
+	}
+
+	// Subscribe to newly data columns stored in the database.
+	identsChan := make(chan filesystem.DataColumnsIdent)
+	subscription := s.dataColumnStorage.DataColumnFeed.Subscribe(identsChan)
+	defer subscription.Unsubscribe()
+
+	// Get the count of data columns we already have in the store.
+	summary := s.dataColumnStorage.Summary(root)
+	storedDataColumnsCount := summary.Count()
+
+	// As soon as we have more than half of the data columns, we can reconstruct the missing ones.
+	// We don't need to wait for the rest of the data columns to declare the block as available.
+	if peerdas.CanSelfReconstruct(storedDataColumnsCount) {
+		return nil
+	}
+
+	// Get a map of data column indices that are not currently available.
+	missingMap, err := missingDataColumns(s.dataColumnStorage, root, peerInfo.CustodyColumns)
+	if err != nil {
+		return errors.Wrap(err, "missing data columns")
+	}
+
+	// If there are no missing indices, all data column sidecars are available.
+	// This is the happy path.
+	if len(missingMap) == 0 {
+		return nil
+	}
+
+	// Log for DA checks that cross over into the next slot; helpful for debugging.
+	nextSlot := slots.BeginsAt(block.Slot()+1, s.genesisTime)
+
+	// Avoid logging if DA check is called after next slot start.
+	if nextSlot.After(time.Now()) {
+		timer := time.AfterFunc(time.Until(nextSlot), func() {
+			missingMapCount := uint64(len(missingMap))
+
+			if missingMapCount == 0 {
+				return
+			}
+
+			var (
+				expected interface{} = "all"
+				missing  interface{} = "all"
+			)
+
+			numberOfColumns := params.BeaconConfig().NumberOfColumns
+			colMapCount := uint64(len(peerInfo.CustodyColumns))
+
+			if colMapCount < numberOfColumns {
+				expected = uint64MapToSortedSlice(peerInfo.CustodyColumns)
+			}
+
+			if missingMapCount < numberOfColumns {
+				missing = uint64MapToSortedSlice(missingMap)
+			}
+
+			log.WithFields(logrus.Fields{
+				"slot":            block.Slot(),
+				"root":            fmt.Sprintf("%#x", root),
+				"columnsExpected": expected,
+				"columnsWaiting":  missing,
+			}).Warning("Data columns still missing at slot end")
+		})
+		defer timer.Stop()
+	}
+
+	for {
+		select {
+		case idents := <-identsChan:
+			if idents.Root != root {
+				// This is not the root we are looking for.
+				continue
+			}
+
+			for _, index := range idents.Indices {
+				// This is a data column we are expecting.
+				if _, ok := missingMap[index]; ok {
+					storedDataColumnsCount++
+				}
+
+				// As soon as we have more than half of the data columns, we can reconstruct the missing ones.
+				// We don't need to wait for the rest of the data columns to declare the block as available.
+				if peerdas.CanSelfReconstruct(storedDataColumnsCount) {
+					return nil
+				}
+
+				// Remove the index from the missing map.
+				delete(missingMap, index)
+
+				// Return if there is no more missing data columns.
+				if len(missingMap) == 0 {
+					return nil
+				}
+			}
+
+		case <-ctx.Done():
+			var missingIndices interface{} = "all"
+			numberOfColumns := params.BeaconConfig().NumberOfColumns
+			missingIndicesCount := uint64(len(missingMap))
+
+			if missingIndicesCount < numberOfColumns {
+				missingIndices = uint64MapToSortedSlice(missingMap)
+			}
+
+			return errors.Wrapf(ctx.Err(), "data column sidecars slot: %d, BlockRoot: %#x, missing %v", block.Slot(), root, missingIndices)
+		}
+	}
+}
+
+// areBlobsAvailable blocks until all BlobSidecars committed to in the block are available,
+// or an error or context cancellation occurs. A nil result means that the data availability check is successful.
+func (s *Service) areBlobsAvailable(ctx context.Context, root [fieldparams.RootLength]byte, block interfaces.ReadOnlyBeaconBlock) error {
+	blockSlot := block.Slot()
+
 	// We are only required to check within MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS
 	if !params.WithinDAPeriod(slots.ToEpoch(block.Slot()), slots.ToEpoch(s.CurrentSlot())) {
 		return nil
@@ -636,7 +825,7 @@ func (s *Service) isDataAvailable(ctx context.Context, root [32]byte, signed int
 	// get a map of BlobSidecar indices that are not currently available.
 	missing, err := missingIndices(s.blobStorage, root, kzgCommitments, block.Slot())
 	if err != nil {
-		return err
+		return errors.Wrap(err, "missing indices")
 	}
 	// If there are no missing indices, all BlobSidecars are available.
 	if len(missing) == 0 {
@@ -648,15 +837,20 @@ func (s *Service) isDataAvailable(ctx context.Context, root [32]byte, signed int
 	nc := s.blobNotifiers.forRoot(root, block.Slot())
 
 	// Log for DA checks that cross over into the next slot; helpful for debugging.
-	nextSlot := slots.BeginsAt(signed.Block().Slot()+1, s.genesisTime)
+	nextSlot := slots.BeginsAt(block.Slot()+1, s.genesisTime)
 	// Avoid logging if DA check is called after next slot start.
 	if nextSlot.After(time.Now()) {
 		nst := time.AfterFunc(time.Until(nextSlot), func() {
 			if len(missing) == 0 {
 				return
 			}
-			log.WithFields(daCheckLogFields(root, signed.Block().Slot(), expected, len(missing))).
-				Error("Still waiting for DA check at slot end.")
+
+			log.WithFields(logrus.Fields{
+				"slot":          blockSlot,
+				"root":          fmt.Sprintf("%#x", root),
+				"blobsExpected": expected,
+				"blobsWaiting":  len(missing),
+			}).Error("Still waiting for blobs DA check at slot end.")
 		})
 		defer nst.Stop()
 	}
@@ -678,13 +872,14 @@ func (s *Service) isDataAvailable(ctx context.Context, root [32]byte, signed int
 	}
 }
 
-func daCheckLogFields(root [32]byte, slot primitives.Slot, expected, missing int) logrus.Fields {
-	return logrus.Fields{
-		"slot":          slot,
-		"root":          fmt.Sprintf("%#x", root),
-		"blobsExpected": expected,
-		"blobsWaiting":  missing,
+// uint64MapToSortedSlice produces a sorted uint64 slice from a map.
+func uint64MapToSortedSlice(input map[uint64]bool) []uint64 {
+	output := make([]uint64, 0, len(input))
+	for idx := range input {
+		output = append(output, idx)
 	}
+	slices.Sort[[]uint64](output)
+	return output
 }
 
 // lateBlockTasks  is called 4 seconds into the slot and performs tasks
@@ -729,13 +924,9 @@ func (s *Service) lateBlockTasks(ctx context.Context) {
 	attribute := s.getPayloadAttribute(ctx, headState, s.CurrentSlot()+1, headRoot[:])
 	// return early if we are not proposing next slot
 	if attribute.IsEmpty() {
-		headBlock, err := s.headBlock()
-		if err != nil {
-			log.WithError(err).WithField("head_root", headRoot).Error("unable to retrieve head block to fire payload attributes event")
-		}
 		// notifyForkchoiceUpdate fires the payload attribute event. But in this case, we won't
 		// call notifyForkchoiceUpdate, so the event is fired here.
-		go s.firePayloadAttributesEvent(s.cfg.StateNotifier.StateFeed(), headBlock, headRoot, s.CurrentSlot()+1)
+		go firePayloadAttributesEvent(ctx, s.cfg.StateNotifier.StateFeed(), s.CurrentSlot()+1)
 		return
 	}
 
@@ -770,7 +961,7 @@ func (s *Service) waitForSync() error {
 	}
 }
 
-func (s *Service) handleInvalidExecutionError(ctx context.Context, err error, blockRoot [32]byte, parentRoot [32]byte) error {
+func (s *Service) handleInvalidExecutionError(ctx context.Context, err error, blockRoot, parentRoot [32]byte) error {
 	if IsInvalidBlock(err) && InvalidBlockLVH(err) != [32]byte{} {
 		return s.pruneInvalidBlock(ctx, blockRoot, parentRoot, InvalidBlockLVH(err))
 	}

beacon-chain/blockchain/process_block_helpers.go

@@ -103,29 +103,15 @@ func (s *Service) sendStateFeedOnBlock(cfg *postBlockProcessConfig) {
 		log.WithError(err).Debug("Could not check if block is optimistic")
 		optimistic = true
 	}
-	currEpoch := slots.ToEpoch(s.CurrentSlot())
-	currDependenRoot, err := s.cfg.ForkChoiceStore.DependentRoot(currEpoch)
-	if err != nil {
-		log.WithError(err).Debug("Could not get dependent root")
-	}
-	prevDependentRoot := [32]byte{}
-	if currEpoch > 0 {
-		prevDependentRoot, err = s.cfg.ForkChoiceStore.DependentRoot(currEpoch - 1)
-		if err != nil {
-			log.WithError(err).Debug("Could not get previous dependent root")
-		}
-	}
 	// Send notification of the processed block to the state feed.
 	s.cfg.StateNotifier.StateFeed().Send(&feed.Event{
 		Type: statefeed.BlockProcessed,
 		Data: &statefeed.BlockProcessedData{
-			Slot:              cfg.roblock.Block().Slot(),
-			BlockRoot:         cfg.roblock.Root(),
-			SignedBlock:       cfg.roblock,
-			CurrDependentRoot: currDependenRoot,
-			PrevDependentRoot: prevDependentRoot,
-			Verified:          true,
-			Optimistic:        optimistic,
+			Slot:        cfg.roblock.Block().Slot(),
+			BlockRoot:   cfg.roblock.Root(),
+			SignedBlock: cfg.roblock,
+			Verified:    true,
+			Optimistic:  optimistic,
 		},
 	})
 }
@@ -309,11 +295,6 @@ func (s *Service) processLightClientFinalityUpdate(
 		Type: statefeed.LightClientFinalityUpdate,
 		Data: newUpdate,
 	})
-
-	if err = s.cfg.P2p.BroadcastLightClientFinalityUpdate(ctx, newUpdate); err != nil {
-		return errors.Wrap(err, "could not broadcast light client finality update")
-	}
-
 	return nil
 }
 
@@ -363,10 +344,6 @@ func (s *Service) processLightClientOptimisticUpdate(ctx context.Context, signed
 		Data: newUpdate,
 	})
 
-	if err = s.cfg.P2p.BroadcastLightClientOptimisticUpdate(ctx, newUpdate); err != nil {
-		return errors.Wrap(err, "could not broadcast light client optimistic update")
-	}
-
 	return nil
 }
 

beacon-chain/blockchain/process_block_test.go

@@ -2,6 +2,7 @@ package blockchain
 
 import (
 	"context"
+	"crypto/rand"
 	"fmt"
 	"math/big"
 	"strconv"
@@ -13,6 +14,7 @@ import (
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/blocks"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
 	lightClient "github.com/OffchainLabs/prysm/v6/beacon-chain/core/light-client"
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/signing"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/transition"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/das"
@@ -24,8 +26,8 @@ import (
 	doublylinkedtree "github.com/OffchainLabs/prysm/v6/beacon-chain/forkchoice/doubly-linked-tree"
 	forkchoicetypes "github.com/OffchainLabs/prysm/v6/beacon-chain/forkchoice/types"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/operations/attestations/kv"
-	mockp2p "github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/testing"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/state"
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/verification"
 	"github.com/OffchainLabs/prysm/v6/config/features"
 	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
 	"github.com/OffchainLabs/prysm/v6/config/params"
@@ -3246,6 +3248,154 @@ func TestSaveLightClientBootstrap(t *testing.T) {
 	reset()
 }
 
+type testIsAvailableParams struct {
+	options                 []Option
+	blobKzgCommitmentsCount uint64
+	columnsToSave           []uint64
+}
+
+func testIsAvailableSetup(t *testing.T, params testIsAvailableParams) (context.Context, context.CancelFunc, *Service, [fieldparams.RootLength]byte, interfaces.SignedBeaconBlock) {
+	ctx, cancel := context.WithCancel(context.Background())
+	dataColumnStorage := filesystem.NewEphemeralDataColumnStorage(t)
+
+	options := append(params.options, WithDataColumnStorage(dataColumnStorage))
+	service, _ := minimalTestService(t, options...)
+
+	genesisState, secretKeys := util.DeterministicGenesisStateElectra(t, 32 /*validator count*/)
+
+	err := service.saveGenesisData(ctx, genesisState)
+	require.NoError(t, err)
+
+	conf := util.DefaultBlockGenConfig()
+	conf.NumBlobKzgCommitments = params.blobKzgCommitmentsCount
+
+	signedBeaconBlock, err := util.GenerateFullBlockFulu(genesisState, secretKeys, conf, 10 /*block slot*/)
+	require.NoError(t, err)
+
+	root, err := signedBeaconBlock.Block.HashTreeRoot()
+	require.NoError(t, err)
+
+	dataColumnsParams := make([]verification.DataColumnParams, 0, len(params.columnsToSave))
+	for _, i := range params.columnsToSave {
+		dataColumnParam := verification.DataColumnParams{ColumnIndex: i}
+		dataColumnsParams = append(dataColumnsParams, dataColumnParam)
+	}
+
+	dataColumnParamsByBlockRoot := verification.DataColumnsParamsByRoot{root: dataColumnsParams}
+	_, verifiedRODataColumns := verification.CreateTestVerifiedRoDataColumnSidecars(t, dataColumnParamsByBlockRoot)
+
+	err = dataColumnStorage.Save(verifiedRODataColumns)
+	require.NoError(t, err)
+
+	signed, err := consensusblocks.NewSignedBeaconBlock(signedBeaconBlock)
+	require.NoError(t, err)
+
+	return ctx, cancel, service, root, signed
+}
+
+func TestIsDataAvailable(t *testing.T) {
+	t.Run("Fulu - out of retention window", func(t *testing.T) {
+		params := testIsAvailableParams{options: []Option{WithGenesisTime(time.Unix(0, 0))}}
+		ctx, _, service, root, signed := testIsAvailableSetup(t, params)
+
+		err := service.isDataAvailable(ctx, root, signed)
+		require.NoError(t, err)
+	})
+
+	t.Run("Fulu - no commitment in blocks", func(t *testing.T) {
+		ctx, _, service, root, signed := testIsAvailableSetup(t, testIsAvailableParams{})
+
+		err := service.isDataAvailable(ctx, root, signed)
+		require.NoError(t, err)
+	})
+
+	t.Run("Fulu - more than half of the columns in custody", func(t *testing.T) {
+		halfNumberOfColumns := params.BeaconConfig().NumberOfColumns / 2
+		indices := make([]uint64, 0, halfNumberOfColumns)
+		for i := range halfNumberOfColumns {
+			indices = append(indices, i)
+		}
+
+		params := testIsAvailableParams{
+			options:                 []Option{WithCustodyInfo(&peerdas.CustodyInfo{})},
+			columnsToSave:           indices,
+			blobKzgCommitmentsCount: 3,
+		}
+
+		ctx, _, service, root, signed := testIsAvailableSetup(t, params)
+
+		err := service.isDataAvailable(ctx, root, signed)
+		require.NoError(t, err)
+	})
+
+	t.Run("Fulu - no missing data columns", func(t *testing.T) {
+		params := testIsAvailableParams{
+			options:                 []Option{WithCustodyInfo(&peerdas.CustodyInfo{})},
+			columnsToSave:           []uint64{1, 17, 19, 42, 75, 87, 102, 117, 119}, // 119 is not needed
+			blobKzgCommitmentsCount: 3,
+		}
+
+		ctx, _, service, root, signed := testIsAvailableSetup(t, params)
+
+		err := service.isDataAvailable(ctx, root, signed)
+		require.NoError(t, err)
+	})
+
+	t.Run("Fulu - some initially missing data columns (no reconstruction)", func(t *testing.T) {
+		var custodyInfo peerdas.CustodyInfo
+		custodyInfo.TargetGroupCount.SetValidatorsCustodyRequirement(128)
+		custodyInfo.ToAdvertiseGroupCount.Set(128)
+
+		testParams := testIsAvailableParams{
+			options:                 []Option{WithCustodyInfo(&custodyInfo)},
+			blobKzgCommitmentsCount: 3,
+		}
+
+		ctx, _, service, root, signed := testIsAvailableSetup(t, testParams)
+
+		// If needed, generate a random root that is different from the block root.
+		var randomRoot [fieldparams.RootLength]byte
+		for randomRoot == root {
+			randomRootSlice := make([]byte, fieldparams.RootLength)
+			_, err := rand.Read(randomRootSlice)
+			require.NoError(t, err)
+			copy(randomRoot[:], randomRootSlice)
+		}
+
+		// TODO: Achieve the same result without using time.AfterFunc.
+		time.AfterFunc(10*time.Millisecond, func() {
+			halfNumberOfColumns := params.BeaconConfig().NumberOfColumns / 2
+			indices := make([]uint64, 0, halfNumberOfColumns)
+			for i := range halfNumberOfColumns {
+				indices = append(indices, i)
+			}
+
+			withSomeRequiredColumns := filesystem.DataColumnsIdent{Root: root, Indices: indices}
+			service.dataColumnStorage.DataColumnFeed.Send(withSomeRequiredColumns)
+		})
+
+		err := service.isDataAvailable(ctx, root, signed)
+		require.NoError(t, err)
+	})
+
+	t.Run("Fulu - some columns are definitively missing", func(t *testing.T) {
+		params := testIsAvailableParams{
+			options:                 []Option{WithCustodyInfo(&peerdas.CustodyInfo{})},
+			blobKzgCommitmentsCount: 3,
+		}
+
+		ctx, cancel, service, root, signed := testIsAvailableSetup(t, params)
+
+		// TODO: Achieve the same result without using time.AfterFunc.
+		time.AfterFunc(10*time.Millisecond, func() {
+			cancel()
+		})
+
+		err := service.isDataAvailable(ctx, root, signed)
+		require.NotNil(t, err)
+	})
+}
+
 func setupLightClientTestRequirements(ctx context.Context, t *testing.T, s *Service, v int, options ...util.LightClientOption) (*util.TestLightClient, *postBlockProcessConfig) {
 	var l *util.TestLightClient
 	switch v {
@@ -3310,7 +3460,6 @@ func TestProcessLightClientOptimisticUpdate(t *testing.T) {
 	params.OverrideBeaconConfig(beaconCfg)
 
 	s, tr := minimalTestService(t)
-	s.cfg.P2p = &mockp2p.FakeP2P{}
 	ctx := tr.ctx
 
 	testCases := []struct {
@@ -3446,7 +3595,6 @@ func TestProcessLightClientFinalityUpdate(t *testing.T) {
 	params.OverrideBeaconConfig(beaconCfg)
 
 	s, tr := minimalTestService(t)
-	s.cfg.P2p = &mockp2p.FakeP2P{}
 	ctx := tr.ctx
 
 	testCases := []struct {

beacon-chain/blockchain/receive_block.go

@@ -52,6 +52,13 @@ type BlobReceiver interface {
 	ReceiveBlob(context.Context, blocks.VerifiedROBlob) error
 }
 
+// DataColumnReceiver interface defines the methods of chain service for receiving new
+// data columns
+type DataColumnReceiver interface {
+	ReceiveDataColumn(blocks.VerifiedRODataColumn) error
+	ReceiveDataColumns([]blocks.VerifiedRODataColumn) error
+}
+
 // SlashingReceiver interface defines the methods of chain service for receiving validated slashing over the wire.
 type SlashingReceiver interface {
 	ReceiveAttesterSlashing(ctx context.Context, slashing ethpb.AttSlashing)
@@ -74,6 +81,7 @@ func (s *Service) ReceiveBlock(ctx context.Context, block interfaces.ReadOnlySig
 		log.WithField("blockRoot", fmt.Sprintf("%#x", blockRoot)).Debug("Ignoring already synced block")
 		return nil
 	}
+
 	receivedTime := time.Now()
 	s.blockBeingSynced.set(blockRoot)
 	defer s.blockBeingSynced.unset(blockRoot)
@@ -82,6 +90,7 @@ func (s *Service) ReceiveBlock(ctx context.Context, block interfaces.ReadOnlySig
 	if err != nil {
 		return err
 	}
+
 	preState, err := s.getBlockPreState(ctx, blockCopy.Block())
 	if err != nil {
 		return errors.Wrap(err, "could not get block's prestate")
@@ -97,10 +106,12 @@ func (s *Service) ReceiveBlock(ctx context.Context, block interfaces.ReadOnlySig
 	if err != nil {
 		return err
 	}
+
 	daWaitedTime, err := s.handleDA(ctx, blockCopy, blockRoot, avs)
 	if err != nil {
 		return err
 	}
+
 	// Defragment the state before continuing block processing.
 	s.defragmentState(postState)
 
@@ -229,24 +240,32 @@ func (s *Service) handleDA(
 	block interfaces.SignedBeaconBlock,
 	blockRoot [32]byte,
 	avs das.AvailabilityStore,
-) (time.Duration, error) {
-	daStartTime := time.Now()
-	if avs != nil {
-		rob, err := blocks.NewROBlockWithRoot(block, blockRoot)
-		if err != nil {
-			return 0, err
+) (elapsed time.Duration, err error) {
+	defer func(start time.Time) {
+		elapsed := time.Since(start)
+
+		if err == nil {
+			dataAvailWaitedTime.Observe(float64(elapsed.Milliseconds()))
 		}
-		if err := avs.IsDataAvailable(ctx, s.CurrentSlot(), rob); err != nil {
-			return 0, errors.Wrap(err, "could not validate blob data availability (AvailabilityStore.IsDataAvailable)")
-		}
-	} else {
-		if err := s.isDataAvailable(ctx, blockRoot, block); err != nil {
-			return 0, errors.Wrap(err, "could not validate blob data availability")
+	}(time.Now())
+
+	if avs == nil {
+		if err = s.isDataAvailable(ctx, blockRoot, block); err != nil {
+			return
 		}
+
+		return
 	}
-	daWaitedTime := time.Since(daStartTime)
-	dataAvailWaitedTime.Observe(float64(daWaitedTime.Milliseconds()))
-	return daWaitedTime, nil
+
+	var rob blocks.ROBlock
+	rob, err = blocks.NewROBlockWithRoot(block, blockRoot)
+	if err != nil {
+		return
+	}
+
+	err = avs.IsDataAvailable(ctx, s.CurrentSlot(), rob)
+
+	return
 }
 
 func (s *Service) reportPostBlockProcessing(

beacon-chain/blockchain/receive_data_column.go

@@ -0,0 +1,25 @@
+package blockchain
+
+import (
+	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
+	"github.com/pkg/errors"
+)
+
+// ReceiveDataColumns receives a batch of data columns.
+func (s *Service) ReceiveDataColumns(dataColumnSidecars []blocks.VerifiedRODataColumn) error {
+	if err := s.dataColumnStorage.Save(dataColumnSidecars); err != nil {
+		return errors.Wrap(err, "save data column sidecars")
+	}
+
+	return nil
+}
+
+// ReceiveDataColumn receives a single data column.
+// (It is only a wrapper around ReceiveDataColumns.)
+func (s *Service) ReceiveDataColumn(dataColumnSidecar blocks.VerifiedRODataColumn) error {
+	if err := s.dataColumnStorage.Save([]blocks.VerifiedRODataColumn{dataColumnSidecar}); err != nil {
+		return errors.Wrap(err, "save data column sidecars")
+	}
+
+	return nil
+}

beacon-chain/blockchain/service.go

@@ -16,6 +16,7 @@ import (
 	statefeed "github.com/OffchainLabs/prysm/v6/beacon-chain/core/feed/state"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
 	lightClient "github.com/OffchainLabs/prysm/v6/beacon-chain/core/light-client"
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
 	coreTime "github.com/OffchainLabs/prysm/v6/beacon-chain/core/time"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/transition"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/db"
@@ -30,6 +31,7 @@ import (
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/startup"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/state"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/state/stategen"
+	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
 	"github.com/OffchainLabs/prysm/v6/config/params"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
@@ -64,6 +66,7 @@ type Service struct {
 	blobNotifiers        *blobNotifierMap
 	blockBeingSynced     *currentlySyncingBlock
 	blobStorage          *filesystem.BlobStorage
+	dataColumnStorage    *filesystem.DataColumnStorage
 	slasherEnabled       bool
 	lcStore              *lightClient.Store
 }
@@ -81,7 +84,7 @@ type config struct {
 	ExitPool                voluntaryexits.PoolManager
 	SlashingPool            slashings.PoolManager
 	BLSToExecPool           blstoexec.PoolManager
-	P2p                     p2p.Broadcaster
+	P2P                     p2p.Acceser
 	MaxRoutines             int
 	StateNotifier           statefeed.Notifier
 	ForkChoiceStore         f.ForkChoicer
@@ -93,6 +96,7 @@ type config struct {
 	FinalizedStateAtStartUp state.BeaconState
 	ExecutionEngineCaller   execution.EngineCaller
 	SyncChecker             Checker
+	CustodyInfo             *peerdas.CustodyInfo
 }
 
 // Checker is an interface used to determine if a node is in initial sync
@@ -106,22 +110,26 @@ var ErrMissingClockSetter = errors.New("blockchain Service initialized without a
 type blobNotifierMap struct {
 	sync.RWMutex
 	notifiers map[[32]byte]chan uint64
-	seenIndex map[[32]byte][]bool
+	// TODO: Separate blobs from data columns
+	// seenIndex map[[32]byte][]bool
+	seenIndex map[[32]byte][fieldparams.NumberOfColumns]bool
 }
 
 // notifyIndex notifies a blob by its index for a given root.
 // It uses internal maps to keep track of seen indices and notifier channels.
 func (bn *blobNotifierMap) notifyIndex(root [32]byte, idx uint64, slot primitives.Slot) {
-	maxBlobsPerBlock := params.BeaconConfig().MaxBlobsPerBlock(slot)
-	if idx >= uint64(maxBlobsPerBlock) {
-		return
-	}
+	// TODO: Separate blobs from data columns
+	// maxBlobsPerBlock := params.BeaconConfig().MaxBlobsPerBlock(slot)
+	// if idx >= uint64(maxBlobsPerBlock) {
+	// 	return
+	// }
 
 	bn.Lock()
 	seen := bn.seenIndex[root]
-	if seen == nil {
-		seen = make([]bool, maxBlobsPerBlock)
-	}
+	// TODO: Separate blobs from data columns
+	// if seen == nil {
+	// seen = make([]bool, maxBlobsPerBlock)
+	// }
 	if seen[idx] {
 		bn.Unlock()
 		return
@@ -132,7 +140,9 @@ func (bn *blobNotifierMap) notifyIndex(root [32]byte, idx uint64, slot primitive
 	// Retrieve or create the notifier channel for the given root.
 	c, ok := bn.notifiers[root]
 	if !ok {
-		c = make(chan uint64, maxBlobsPerBlock)
+		// TODO: Separate blobs from data columns
+		// c = make(chan uint64, maxBlobsPerBlock)
+		c = make(chan uint64, fieldparams.NumberOfColumns)
 		bn.notifiers[root] = c
 	}
 
@@ -142,12 +152,15 @@ func (bn *blobNotifierMap) notifyIndex(root [32]byte, idx uint64, slot primitive
 }
 
 func (bn *blobNotifierMap) forRoot(root [32]byte, slot primitives.Slot) chan uint64 {
-	maxBlobsPerBlock := params.BeaconConfig().MaxBlobsPerBlock(slot)
+	// TODO: Separate blobs from data columns
+	// maxBlobsPerBlock := params.BeaconConfig().MaxBlobsPerBlock(slot)
 	bn.Lock()
 	defer bn.Unlock()
 	c, ok := bn.notifiers[root]
 	if !ok {
-		c = make(chan uint64, maxBlobsPerBlock)
+		// TODO: Separate blobs from data columns
+		// c = make(chan uint64, maxBlobsPerBlock)
+		c = make(chan uint64, fieldparams.NumberOfColumns)
 		bn.notifiers[root] = c
 	}
 	return c
@@ -173,7 +186,9 @@ func NewService(ctx context.Context, opts ...Option) (*Service, error) {
 	ctx, cancel := context.WithCancel(ctx)
 	bn := &blobNotifierMap{
 		notifiers: make(map[[32]byte]chan uint64),
-		seenIndex: make(map[[32]byte][]bool),
+		// TODO: Separate blobs from data columns
+		// seenIndex: make(map[[32]byte][]bool),
+		seenIndex: make(map[[32]byte][fieldparams.NumberOfColumns]bool),
 	}
 	srv := &Service{
 		ctx:                  ctx,

beacon-chain/blockchain/service_test.go

@@ -97,7 +97,7 @@ func setupBeaconChain(t *testing.T, beaconDB db.Database) *Service {
 		WithAttestationPool(attestations.NewPool()),
 		WithSlashingPool(slashings.NewPool()),
 		WithExitPool(voluntaryexits.NewPool()),
-		WithP2PBroadcaster(&mockBroadcaster{}),
+		WithP2PBroadcaster(&mockAccesser{}),
 		WithStateNotifier(&mockBeaconNode{}),
 		WithForkChoiceStore(fc),
 		WithAttestationService(attService),
@@ -587,7 +587,9 @@ func (s *MockClockSetter) SetClock(g *startup.Clock) error {
 func TestNotifyIndex(t *testing.T) {
 	// Initialize a blobNotifierMap
 	bn := &blobNotifierMap{
-		seenIndex: make(map[[32]byte][]bool),
+		// TODO: Separate blobs from data columns
+		// seenIndex: make(map[[32]byte][]bool),
+		seenIndex: make(map[[32]byte][fieldparams.NumberOfColumns]bool),
 		notifiers: make(map[[32]byte]chan uint64),
 	}
 

beacon-chain/blockchain/setup_test.go

@@ -24,7 +24,6 @@ import (
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/startup"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/state/stategen"
 	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
-	"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
 	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
 	"github.com/OffchainLabs/prysm/v6/testing/require"
 	"google.golang.org/protobuf/proto"
@@ -74,17 +73,7 @@ func (mb *mockBroadcaster) BroadcastBlob(_ context.Context, _ uint64, _ *ethpb.B
 	return nil
 }
 
-func (mb *mockBroadcaster) BroadcastLightClientOptimisticUpdate(_ context.Context, _ interfaces.LightClientOptimisticUpdate) error {
-	mb.broadcastCalled = true
-	return nil
-}
-
-func (mb *mockBroadcaster) BroadcastLightClientFinalityUpdate(_ context.Context, _ interfaces.LightClientFinalityUpdate) error {
-	mb.broadcastCalled = true
-	return nil
-}
-
-func (mb *mockBroadcaster) BroadcastDataColumn(_ [fieldparams.RootLength]byte, _ uint64, _ *ethpb.DataColumnSidecar, _ ...chan<- bool) error {
+func (mb *mockBroadcaster) BroadcastDataColumn(_ context.Context, _ [fieldparams.RootLength]byte, _ uint64, _ *ethpb.DataColumnSidecar) error {
 	mb.broadcastCalled = true
 	return nil
 }
@@ -144,6 +133,7 @@ func minimalTestService(t *testing.T, opts ...Option) (*Service, *testServiceReq
 		WithDepositCache(dc),
 		WithTrackedValidatorsCache(cache.NewTrackedValidatorsCache()),
 		WithBlobStorage(filesystem.NewEphemeralBlobStorage(t)),
+		WithDataColumnStorage(filesystem.NewEphemeralDataColumnStorage(t)),
 		WithSyncChecker(mock.MockChecker{}),
 		WithExecutionEngineCaller(&mockExecution.EngineClient{}),
 		WithP2PBroadcaster(&mockAccesser{}),

beacon-chain/blockchain/testing/mock.go

@@ -53,7 +53,6 @@ type ChainService struct {
 	InitSyncBlockRoots          map[[32]byte]bool
 	DB                          db.Database
 	State                       state.BeaconState
-	HeadStateErr                error
 	Block                       interfaces.ReadOnlySignedBeaconBlock
 	VerifyBlkDescendantErr      error
 	stateNotifier               statefeed.Notifier
@@ -75,6 +74,7 @@ type ChainService struct {
 	BlockSlot                   primitives.Slot
 	SyncingRoot                 [32]byte
 	Blobs                       []blocks.VerifiedROBlob
+	DataColumns                 []blocks.VerifiedRODataColumn
 	TargetRoot                  [32]byte
 	MockHeadSlot                *primitives.Slot
 }
@@ -365,9 +365,6 @@ func (s *ChainService) HeadState(context.Context) (state.BeaconState, error) {
 
 // HeadStateReadOnly mocks HeadStateReadOnly method in chain service.
 func (s *ChainService) HeadStateReadOnly(context.Context) (state.ReadOnlyBeaconState, error) {
-	if s.HeadStateErr != nil {
-		return nil, s.HeadStateErr
-	}
 	return s.State, nil
 }
 
@@ -715,18 +712,18 @@ func (c *ChainService) ReceiveBlob(_ context.Context, b blocks.VerifiedROBlob) e
 	return nil
 }
 
+// ReceiveDataColumn implements the same method in chain service
+func (c *ChainService) ReceiveDataColumn(dc blocks.VerifiedRODataColumn) error {
+	c.DataColumns = append(c.DataColumns, dc)
+	return nil
+}
+
+// ReceiveDataColumns implements the same method in chain service
+func (*ChainService) ReceiveDataColumns(_ []blocks.VerifiedRODataColumn) error {
+	return nil
+}
+
 // TargetRootForEpoch mocks the same method in the chain service
 func (c *ChainService) TargetRootForEpoch(_ [32]byte, _ primitives.Epoch) ([32]byte, error) {
 	return c.TargetRoot, nil
 }
-
-// MockSyncChecker is a mock implementation of blockchain.Checker.
-// We can't make an assertion here that this is true because that would create a circular dependency.
-type MockSyncChecker struct {
-	synced bool
-}
-
-// Synced satisfies the blockchain.Checker interface.
-func (m *MockSyncChecker) Synced() bool {
-	return m.synced
-}

beacon-chain/cache/committee.go

@@ -14,7 +14,6 @@ import (
 	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
 	"github.com/OffchainLabs/prysm/v6/container/slice"
 	mathutil "github.com/OffchainLabs/prysm/v6/math"
-	"github.com/OffchainLabs/prysm/v6/monitoring/tracing/trace"
 	lru "github.com/hashicorp/golang-lru"
 	"github.com/prometheus/client_golang/prometheus"
 	"github.com/prometheus/client_golang/prometheus/promauto"
@@ -105,16 +104,11 @@ func (c *CommitteeCache) CompressCommitteeCache() {
 // Committee fetches the shuffled indices by slot and committee index. Every list of indices
 // represent one committee. Returns true if the list exists with slot and committee index. Otherwise returns false, nil.
 func (c *CommitteeCache) Committee(ctx context.Context, slot primitives.Slot, seed [32]byte, index primitives.CommitteeIndex) ([]primitives.ValidatorIndex, error) {
-	ctx, span := trace.StartSpan(ctx, "committeeCache.Committee")
-	defer span.End()
-	span.SetAttributes(trace.Int64Attribute("slot", int64(slot)), trace.Int64Attribute("index", int64(index))) // lint:ignore uintcast -- OK for tracing.
-
 	if err := c.checkInProgress(ctx, seed); err != nil {
 		return nil, err
 	}
 
 	obj, exists := c.CommitteeCache.Get(key(seed))
-	span.SetAttributes(trace.BoolAttribute("cache_hit", exists))
 	if exists {
 		CommitteeCacheHit.Inc()
 	} else {
@@ -163,14 +157,11 @@ func (c *CommitteeCache) AddCommitteeShuffledList(ctx context.Context, committee
 
 // ActiveIndices returns the active indices of a given seed stored in cache.
 func (c *CommitteeCache) ActiveIndices(ctx context.Context, seed [32]byte) ([]primitives.ValidatorIndex, error) {
-	ctx, span := trace.StartSpan(ctx, "committeeCache.ActiveIndices")
-	defer span.End()
-
 	if err := c.checkInProgress(ctx, seed); err != nil {
 		return nil, err
 	}
 	obj, exists := c.CommitteeCache.Get(key(seed))
-	span.SetAttributes(trace.BoolAttribute("cache_hit", exists))
+
 	if exists {
 		CommitteeCacheHit.Inc()
 	} else {

beacon-chain/core/altair/upgrade.go

@@ -67,6 +67,10 @@ func UpgradeToAltair(ctx context.Context, state state.BeaconState) (state.Beacon
 	epoch := time.CurrentEpoch(state)
 
 	numValidators := state.NumValidators()
+	hrs, err := state.HistoricalRoots()
+	if err != nil {
+		return nil, err
+	}
 	s := &ethpb.BeaconStateAltair{
 		GenesisTime:           state.GenesisTime(),
 		GenesisValidatorsRoot: state.GenesisValidatorsRoot(),
@@ -79,7 +83,7 @@ func UpgradeToAltair(ctx context.Context, state state.BeaconState) (state.Beacon
 		LatestBlockHeader:           state.LatestBlockHeader(),
 		BlockRoots:                  state.BlockRoots(),
 		StateRoots:                  state.StateRoots(),
-		HistoricalRoots:             state.HistoricalRoots(),
+		HistoricalRoots:             hrs,
 		Eth1Data:                    state.Eth1Data(),
 		Eth1DataVotes:               state.Eth1DataVotes(),
 		Eth1DepositIndex:            state.Eth1DepositIndex(),

beacon-chain/core/altair/upgrade_test.go

@@ -82,8 +82,10 @@ func TestUpgradeToAltair(t *testing.T) {
 	require.DeepSSZEqual(t, preForkState.LatestBlockHeader(), aState.LatestBlockHeader())
 	require.DeepSSZEqual(t, preForkState.BlockRoots(), aState.BlockRoots())
 	require.DeepSSZEqual(t, preForkState.StateRoots(), aState.StateRoots())
-	r1 := preForkState.HistoricalRoots()
-	r2 := aState.HistoricalRoots()
+	r1, err := preForkState.HistoricalRoots()
+	require.NoError(t, err)
+	r2, err := aState.HistoricalRoots()
+	require.NoError(t, err)
 	require.DeepSSZEqual(t, r1, r2)
 	require.DeepSSZEqual(t, preForkState.Eth1Data(), aState.Eth1Data())
 	require.DeepSSZEqual(t, preForkState.Eth1DataVotes(), aState.Eth1DataVotes())

beacon-chain/core/blocks/payload.go

@@ -12,6 +12,7 @@ import (
 	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
+	"github.com/OffchainLabs/prysm/v6/encoding/bytesutil"
 	"github.com/OffchainLabs/prysm/v6/runtime/version"
 	"github.com/OffchainLabs/prysm/v6/time/slots"
 	"github.com/pkg/errors"
@@ -229,13 +230,29 @@ func verifyBlobCommitmentCount(slot primitives.Slot, body interfaces.ReadOnlyBea
 	if body.Version() < version.Deneb {
 		return nil
 	}
+
 	kzgs, err := body.BlobKzgCommitments()
 	if err != nil {
 		return err
 	}
-	maxBlobsPerBlock := params.BeaconConfig().MaxBlobsPerBlock(slot)
-	if len(kzgs) > maxBlobsPerBlock {
-		return fmt.Errorf("too many kzg commitments in block: %d", len(kzgs))
+
+	commitmentCount, maxBlobsPerBlock := len(kzgs), params.BeaconConfig().MaxBlobsPerBlock(slot)
+	if commitmentCount > maxBlobsPerBlock {
+		return fmt.Errorf("too many kzg commitments in block: actual count %d - max allowed %d", commitmentCount, maxBlobsPerBlock)
 	}
+
 	return nil
 }
+
+// GetBlockPayloadHash returns the hash of the execution payload of the block
+func GetBlockPayloadHash(blk interfaces.ReadOnlyBeaconBlock) ([32]byte, error) {
+	var payloadHash [32]byte
+	if IsPreBellatrixVersion(blk.Version()) {
+		return payloadHash, nil
+	}
+	payload, err := blk.Body().Execution()
+	if err != nil {
+		return payloadHash, err
+	}
+	return bytesutil.ToBytes32(payload.BlockHash()), nil
+}

beacon-chain/core/blocks/payload_test.go

@@ -926,8 +926,10 @@ func TestVerifyBlobCommitmentCount(t *testing.T) {
 	require.NoError(t, err)
 	require.NoError(t, blocks.VerifyBlobCommitmentCount(rb.Slot(), rb.Body()))
 
-	b = &ethpb.BeaconBlockDeneb{Body: &ethpb.BeaconBlockBodyDeneb{BlobKzgCommitments: make([][]byte, params.BeaconConfig().MaxBlobsPerBlock(rb.Slot())+1)}}
+	maxCommitmentsPerBlock := params.BeaconConfig().MaxBlobsPerBlock(rb.Slot())
+
+	b = &ethpb.BeaconBlockDeneb{Body: &ethpb.BeaconBlockBodyDeneb{BlobKzgCommitments: make([][]byte, maxCommitmentsPerBlock+1)}}
 	rb, err = consensusblocks.NewBeaconBlock(b)
 	require.NoError(t, err)
-	require.ErrorContains(t, fmt.Sprintf("too many kzg commitments in block: %d", params.BeaconConfig().MaxBlobsPerBlock(rb.Slot())+1), blocks.VerifyBlobCommitmentCount(rb.Slot(), rb.Body()))
+	require.ErrorContains(t, fmt.Sprintf("too many kzg commitments in block: actual count %d - max allowed %d", maxCommitmentsPerBlock+1, maxCommitmentsPerBlock), blocks.VerifyBlobCommitmentCount(rb.Slot(), rb.Body()))
 }

beacon-chain/core/blocks/proposer_slashing.go

@@ -16,9 +16,6 @@ import (
 	"google.golang.org/protobuf/proto"
 )
 
-// ErrCouldNotVerifyBlockHeader is returned when a block header's signature cannot be verified.
-var ErrCouldNotVerifyBlockHeader = errors.New("could not verify beacon block header")
-
 type slashValidatorFunc func(
 	ctx context.Context,
 	st state.BeaconState,
@@ -117,7 +114,7 @@ func VerifyProposerSlashing(
 	for _, header := range headers {
 		if err := signing.ComputeDomainVerifySigningRoot(beaconState, pIdx, slots.ToEpoch(hSlot),
 			header.Header, params.BeaconConfig().DomainBeaconProposer, header.Signature); err != nil {
-			return errors.Wrap(ErrCouldNotVerifyBlockHeader, err.Error())
+			return errors.Wrap(err, "could not verify beacon block header")
 		}
 	}
 	return nil

beacon-chain/core/blocks/signature.go

@@ -96,6 +96,24 @@ func VerifyBlockHeaderSignature(beaconState state.BeaconState, header *ethpb.Sig
 	return signing.VerifyBlockHeaderSigningRoot(header.Header, proposerPubKey, header.Signature, domain)
 }
 
+func VerifyBlockHeaderSignatureUsingCurrentFork(beaconState state.BeaconState, header *ethpb.SignedBeaconBlockHeader) error {
+	currentEpoch := slots.ToEpoch(header.Header.Slot)
+	fork, err := forks.Fork(currentEpoch)
+	if err != nil {
+		return err
+	}
+	domain, err := signing.Domain(fork, currentEpoch, params.BeaconConfig().DomainBeaconProposer, beaconState.GenesisValidatorsRoot())
+	if err != nil {
+		return err
+	}
+	proposer, err := beaconState.ValidatorAtIndex(header.Header.ProposerIndex)
+	if err != nil {
+		return err
+	}
+	proposerPubKey := proposer.PublicKey
+	return signing.VerifyBlockHeaderSigningRoot(header.Header, proposerPubKey, header.Signature, domain)
+}
+
 // VerifyBlockSignatureUsingCurrentFork verifies the proposer signature of a beacon block. This differs
 // from the above method by not using fork data from the state and instead retrieving it
 // via the respective epoch.

beacon-chain/core/capella/upgrade.go

@@ -42,6 +42,10 @@ func UpgradeToCapella(state state.BeaconState) (state.BeaconState, error) {
 		return nil, err
 	}
 
+	hrs, err := state.HistoricalRoots()
+	if err != nil {
+		return nil, err
+	}
 	s := &ethpb.BeaconStateCapella{
 		GenesisTime:           state.GenesisTime(),
 		GenesisValidatorsRoot: state.GenesisValidatorsRoot(),
@@ -54,7 +58,7 @@ func UpgradeToCapella(state state.BeaconState) (state.BeaconState, error) {
 		LatestBlockHeader:           state.LatestBlockHeader(),
 		BlockRoots:                  state.BlockRoots(),
 		StateRoots:                  state.StateRoots(),
-		HistoricalRoots:             state.HistoricalRoots(),
+		HistoricalRoots:             hrs,
 		Eth1Data:                    state.Eth1Data(),
 		Eth1DataVotes:               state.Eth1DataVotes(),
 		Eth1DepositIndex:            state.Eth1DepositIndex(),

beacon-chain/core/deneb/upgrade.go

@@ -57,6 +57,10 @@ func UpgradeToDeneb(state state.BeaconState) (state.BeaconState, error) {
 	if err != nil {
 		return nil, err
 	}
+	historicalRoots, err := state.HistoricalRoots()
+	if err != nil {
+		return nil, err
+	}
 
 	s := &ethpb.BeaconStateDeneb{
 		GenesisTime:           state.GenesisTime(),
@@ -70,7 +74,7 @@ func UpgradeToDeneb(state state.BeaconState) (state.BeaconState, error) {
 		LatestBlockHeader:           state.LatestBlockHeader(),
 		BlockRoots:                  state.BlockRoots(),
 		StateRoots:                  state.StateRoots(),
-		HistoricalRoots:             state.HistoricalRoots(),
+		HistoricalRoots:             historicalRoots,
 		Eth1Data:                    state.Eth1Data(),
 		Eth1DataVotes:               state.Eth1DataVotes(),
 		Eth1DepositIndex:            state.Eth1DepositIndex(),

beacon-chain/core/deneb/upgrade_test.go

@@ -47,8 +47,10 @@ func TestUpgradeToDeneb(t *testing.T) {
 	require.NoError(t, err)
 	require.DeepSSZEqual(t, make([]uint64, numValidators), s)
 
-	hr1 := preForkState.HistoricalRoots()
-	hr2 := mSt.HistoricalRoots()
+	hr1, err := preForkState.HistoricalRoots()
+	require.NoError(t, err)
+	hr2, err := mSt.HistoricalRoots()
+	require.NoError(t, err)
 	require.DeepEqual(t, hr1, hr2)
 
 	f := mSt.Fork()

beacon-chain/core/electra/upgrade.go

@@ -170,6 +170,10 @@ func UpgradeToElectra(beaconState state.BeaconState) (state.BeaconState, error)
 	if err != nil {
 		return nil, err
 	}
+	historicalRoots, err := beaconState.HistoricalRoots()
+	if err != nil {
+		return nil, err
+	}
 	excessBlobGas, err := payloadHeader.ExcessBlobGas()
 	if err != nil {
 		return nil, err
@@ -219,7 +223,7 @@ func UpgradeToElectra(beaconState state.BeaconState) (state.BeaconState, error)
 		LatestBlockHeader:           beaconState.LatestBlockHeader(),
 		BlockRoots:                  beaconState.BlockRoots(),
 		StateRoots:                  beaconState.StateRoots(),
-		HistoricalRoots:             beaconState.HistoricalRoots(),
+		HistoricalRoots:             historicalRoots,
 		Eth1Data:                    beaconState.Eth1Data(),
 		Eth1DataVotes:               beaconState.Eth1DataVotes(),
 		Eth1DepositIndex:            beaconState.Eth1DepositIndex(),

beacon-chain/core/electra/upgrade_test.go

@@ -77,8 +77,10 @@ func TestUpgradeToElectra(t *testing.T) {
 	require.NoError(t, err)
 	require.DeepSSZEqual(t, make([]uint64, numValidators), s)
 
-	hr1 := preForkState.HistoricalRoots()
-	hr2 := mSt.HistoricalRoots()
+	hr1, err := preForkState.HistoricalRoots()
+	require.NoError(t, err)
+	hr2, err := mSt.HistoricalRoots()
+	require.NoError(t, err)
 	require.DeepEqual(t, hr1, hr2)
 
 	f := mSt.Fork()

beacon-chain/core/epoch/epoch_processing_test.go

@@ -148,7 +148,8 @@ func TestProcessFinalUpdates_CanProcess(t *testing.T) {
 	assert.DeepNotEqual(t, params.BeaconConfig().ZeroHash[:], mix, "latest RANDAO still zero hashes")
 
 	// Verify historical root accumulator was appended.
-	roots := newS.HistoricalRoots()
+	roots, err := newS.HistoricalRoots()
+	require.NoError(t, err)
 	assert.Equal(t, 1, len(roots), "Unexpected slashed balance")
 	currAtt, err := newS.CurrentEpochAttestations()
 	require.NoError(t, err)
@@ -378,7 +379,8 @@ func TestProcessHistoricalDataUpdate(t *testing.T) {
 				return st
 			},
 			verifier: func(st state.BeaconState) {
-				roots := st.HistoricalRoots()
+				roots, err := st.HistoricalRoots()
+				require.NoError(t, err)
 				require.Equal(t, 0, len(roots))
 			},
 		},
@@ -391,7 +393,8 @@ func TestProcessHistoricalDataUpdate(t *testing.T) {
 				return st
 			},
 			verifier: func(st state.BeaconState) {
-				roots := st.HistoricalRoots()
+				roots, err := st.HistoricalRoots()
+				require.NoError(t, err)
 				require.Equal(t, 1, len(roots))
 
 				b := &ethpb.HistoricalBatch{
@@ -428,7 +431,8 @@ func TestProcessHistoricalDataUpdate(t *testing.T) {
 					StateSummaryRoot: sr[:],
 				}
 				require.DeepEqual(t, b, summaries[0])
-				hrs := st.HistoricalRoots()
+				hrs, err := st.HistoricalRoots()
+				require.NoError(t, err)
 				require.DeepEqual(t, hrs, [][]byte{})
 			},
 		},

beacon-chain/core/execution/upgrade.go

@@ -35,6 +35,10 @@ func UpgradeToBellatrix(state state.BeaconState) (state.BeaconState, error) {
 		return nil, err
 	}
 
+	hrs, err := state.HistoricalRoots()
+	if err != nil {
+		return nil, err
+	}
 	s := &ethpb.BeaconStateBellatrix{
 		GenesisTime:           state.GenesisTime(),
 		GenesisValidatorsRoot: state.GenesisValidatorsRoot(),
@@ -47,7 +51,7 @@ func UpgradeToBellatrix(state state.BeaconState) (state.BeaconState, error) {
 		LatestBlockHeader:           state.LatestBlockHeader(),
 		BlockRoots:                  state.BlockRoots(),
 		StateRoots:                  state.StateRoots(),
-		HistoricalRoots:             state.HistoricalRoots(),
+		HistoricalRoots:             hrs,
 		Eth1Data:                    state.Eth1Data(),
 		Eth1DataVotes:               state.Eth1DataVotes(),
 		Eth1DepositIndex:            state.Eth1DepositIndex(),

beacon-chain/core/execution/upgrade_test.go

@@ -24,8 +24,10 @@ func TestUpgradeToBellatrix(t *testing.T) {
 	require.DeepSSZEqual(t, preForkState.LatestBlockHeader(), mSt.LatestBlockHeader())
 	require.DeepSSZEqual(t, preForkState.BlockRoots(), mSt.BlockRoots())
 	require.DeepSSZEqual(t, preForkState.StateRoots(), mSt.StateRoots())
-	r1 := preForkState.HistoricalRoots()
-	r2 := mSt.HistoricalRoots()
+	r1, err := preForkState.HistoricalRoots()
+	require.NoError(t, err)
+	r2, err := mSt.HistoricalRoots()
+	require.NoError(t, err)
 	require.DeepSSZEqual(t, r1, r2)
 	require.DeepSSZEqual(t, preForkState.Eth1Data(), mSt.Eth1Data())
 	require.DeepSSZEqual(t, preForkState.Eth1DataVotes(), mSt.Eth1DataVotes())

beacon-chain/core/feed/block/events.go

@@ -11,8 +11,6 @@ const (
 
 // ReceivedBlockData is the data sent with ReceivedBlock events.
 type ReceivedBlockData struct {
-	SignedBlock       interfaces.ReadOnlySignedBeaconBlock
-	CurrDependentRoot [32]byte
-	PrevDependentRoot [32]byte
-	IsOptimistic      bool
+	SignedBlock  interfaces.ReadOnlySignedBeaconBlock
+	IsOptimistic bool
 }

beacon-chain/core/feed/operation/events.go

@@ -37,8 +37,11 @@ const (
 	// SingleAttReceived is sent after a single attestation object is received from gossip or rpc
 	SingleAttReceived = 9
 
+	// DataColumnSidecarReceived is sent after a data column sidecar is received from gossip or rpc.
+	DataColumnSidecarReceived = 10
+
 	// BlockGossipReceived is sent after a block has been received from gossip or API that passes validation rules.
-	BlockGossipReceived = 10
+	BlockGossipReceived = 11
 )
 
 // UnAggregatedAttReceivedData is the data sent with UnaggregatedAttReceived events.
@@ -90,6 +93,11 @@ type SingleAttReceivedData struct {
 	Attestation ethpb.Att
 }
 
+// DataColumnSidecarReceivedData is the data sent with DataColumnSidecarReceived events.
+type DataColumnSidecarReceivedData struct {
+	DataColumn *blocks.VerifiedRODataColumn
+}
+
 // BlockGossipReceivedData is the data sent with BlockGossipReceived events.
 type BlockGossipReceivedData struct {
 	// SignedBlock is the block that was received.

beacon-chain/core/feed/state/events.go

@@ -43,10 +43,6 @@ type BlockProcessedData struct {
 	BlockRoot [32]byte
 	// SignedBlock is the physical processed block.
 	SignedBlock interfaces.ReadOnlySignedBeaconBlock
-	// CurrDependentRoot is the current dependent root
-	CurrDependentRoot [32]byte
-	// PrevDependentRoot is the previous dependent root
-	PrevDependentRoot [32]byte
 	// Verified is true if the block's BLS contents have been verified.
 	Verified bool
 	// Optimistic is true if the block is optimistic.

beacon-chain/core/fulu/upgrade.go

@@ -57,6 +57,10 @@ func UpgradeToFulu(beaconState state.BeaconState) (state.BeaconState, error) {
 	if err != nil {
 		return nil, err
 	}
+	historicalRoots, err := beaconState.HistoricalRoots()
+	if err != nil {
+		return nil, err
+	}
 	excessBlobGas, err := payloadHeader.ExcessBlobGas()
 	if err != nil {
 		return nil, err
@@ -114,7 +118,7 @@ func UpgradeToFulu(beaconState state.BeaconState) (state.BeaconState, error) {
 		LatestBlockHeader:           beaconState.LatestBlockHeader(),
 		BlockRoots:                  beaconState.BlockRoots(),
 		StateRoots:                  beaconState.StateRoots(),
-		HistoricalRoots:             beaconState.HistoricalRoots(),
+		HistoricalRoots:             historicalRoots,
 		Eth1Data:                    beaconState.Eth1Data(),
 		Eth1DataVotes:               beaconState.Eth1DataVotes(),
 		Eth1DepositIndex:            beaconState.Eth1DepositIndex(),

beacon-chain/core/fulu/upgrade_test.go

@@ -43,8 +43,10 @@ func TestUpgradeToFulu(t *testing.T) {
 	require.DeepSSZEqual(t, preForkState.BlockRoots(), mSt.BlockRoots())
 	require.DeepSSZEqual(t, preForkState.StateRoots(), mSt.StateRoots())
 
-	hr1 := preForkState.HistoricalRoots()
-	hr2 := mSt.HistoricalRoots()
+	hr1, err := preForkState.HistoricalRoots()
+	require.NoError(t, err)
+	hr2, err := mSt.HistoricalRoots()
+	require.NoError(t, err)
 	require.DeepEqual(t, hr1, hr2)
 
 	require.DeepSSZEqual(t, preForkState.Eth1Data(), mSt.Eth1Data())

beacon-chain/core/helpers/attestation.go

@@ -187,7 +187,9 @@ func ValidateAttestationTime(attSlot primitives.Slot, genesisTime time.Time, clo
 // VerifyCheckpointEpoch is within current epoch and previous epoch
 // with respect to current time. Returns true if it's within, false if it's not.
 func VerifyCheckpointEpoch(c *ethpb.Checkpoint, genesis time.Time) bool {
-	currentSlot := slots.CurrentSlot(uint64(genesis.Unix()))
+	now := uint64(prysmTime.Now().Unix())
+	genesisTime := uint64(genesis.Unix())
+	currentSlot := primitives.Slot((now - genesisTime) / params.BeaconConfig().SecondsPerSlot)
 	currentEpoch := slots.ToEpoch(currentSlot)
 
 	var prevEpoch primitives.Epoch

beacon-chain/core/helpers/beacon_committee.go

@@ -18,7 +18,6 @@ import (
 	"github.com/OffchainLabs/prysm/v6/crypto/hash"
 	"github.com/OffchainLabs/prysm/v6/encoding/bytesutil"
 	"github.com/OffchainLabs/prysm/v6/math"
-	"github.com/OffchainLabs/prysm/v6/monitoring/tracing/trace"
 	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
 	"github.com/OffchainLabs/prysm/v6/runtime/version"
 	"github.com/OffchainLabs/prysm/v6/time/slots"
@@ -119,9 +118,6 @@ func attestationCommittees(
 
 // BeaconCommittees returns the list of all beacon committees for a given state at a given slot.
 func BeaconCommittees(ctx context.Context, state state.ReadOnlyBeaconState, slot primitives.Slot) ([][]primitives.ValidatorIndex, error) {
-	ctx, span := trace.StartSpan(ctx, "helpers.BeaconCommittees")
-	defer span.End()
-
 	epoch := slots.ToEpoch(slot)
 	activeCount, err := ActiveValidatorCount(ctx, state, epoch)
 	if err != nil {
@@ -248,9 +244,6 @@ func BeaconCommittee(
 	slot primitives.Slot,
 	committeeIndex primitives.CommitteeIndex,
 ) ([]primitives.ValidatorIndex, error) {
-	ctx, span := trace.StartSpan(ctx, "helpers.BeaconCommittee")
-	defer span.End()
-
 	committee, err := committeeCache.Committee(ctx, slot, seed, committeeIndex)
 	if err != nil {
 		return nil, errors.Wrap(err, "could not interface with committee cache")
@@ -304,9 +297,6 @@ func verifyAssignmentEpoch(epoch primitives.Epoch, state state.BeaconState) erro
 // It verifies the validity of the epoch, then iterates through each slot in the epoch to determine the
 // proposer for that slot and assigns them accordingly.
 func ProposerAssignments(ctx context.Context, state state.BeaconState, epoch primitives.Epoch) (map[primitives.ValidatorIndex][]primitives.Slot, error) {
-	ctx, span := trace.StartSpan(ctx, "helpers.ProposerAssignments")
-	defer span.End()
-
 	// Verify if the epoch is valid for assignment based on the provided state.
 	if err := verifyAssignmentEpoch(epoch, state); err != nil {
 		return nil, err
@@ -355,9 +345,6 @@ func ProposerAssignments(ctx context.Context, state state.BeaconState, epoch pri
 // It retrieves active validator indices, determines the number of committees per slot, and computes
 // assignments for each validator based on their presence in the provided validators slice.
 func CommitteeAssignments(ctx context.Context, state state.BeaconState, epoch primitives.Epoch, validators []primitives.ValidatorIndex) (map[primitives.ValidatorIndex]*CommitteeAssignment, error) {
-	ctx, span := trace.StartSpan(ctx, "helpers.CommitteeAssignments")
-	defer span.End()
-
 	// Verify if the epoch is valid for assignment based on the provided state.
 	if err := verifyAssignmentEpoch(epoch, state); err != nil {
 		return nil, err
@@ -445,9 +432,6 @@ func CommitteeIndices(committeeBits bitfield.Bitfield) []primitives.CommitteeInd
 // UpdateCommitteeCache gets called at the beginning of every epoch to cache the committee shuffled indices
 // list with committee index and epoch number. It caches the shuffled indices for the input epoch.
 func UpdateCommitteeCache(ctx context.Context, state state.ReadOnlyBeaconState, e primitives.Epoch) error {
-	ctx, span := trace.StartSpan(ctx, "committeeCache.UpdateCommitteeCache")
-	defer span.End()
-
 	seed, err := Seed(state, e, params.BeaconConfig().DomainBeaconAttester)
 	if err != nil {
 		return err

beacon-chain/core/helpers/sync_committee_test.go

@@ -78,6 +78,7 @@ func TestIsCurrentEpochSyncCommittee_UsingCommittee(t *testing.T) {
 
 func TestIsCurrentEpochSyncCommittee_DoesNotExist(t *testing.T) {
 	helpers.ClearCache()
+	params.SetupTestConfigCleanup(t)
 
 	validators := make([]*ethpb.Validator, params.BeaconConfig().SyncCommitteeSize)
 	syncCommittee := &ethpb.SyncCommittee{
@@ -264,6 +265,7 @@ func TestCurrentEpochSyncSubcommitteeIndices_UsingCommittee(t *testing.T) {
 }
 
 func TestCurrentEpochSyncSubcommitteeIndices_DoesNotExist(t *testing.T) {
+	params.SetupTestConfigCleanup(t)
 	helpers.ClearCache()
 
 	validators := make([]*ethpb.Validator, params.BeaconConfig().SyncCommitteeSize)

beacon-chain/core/helpers/validators.go

@@ -102,9 +102,6 @@ func checkValidatorSlashable(activationEpoch, withdrawableEpoch primitives.Epoch
 //	  """
 //	  return [ValidatorIndex(i) for i, v in enumerate(state.validators) if is_active_validator(v, epoch)]
 func ActiveValidatorIndices(ctx context.Context, s state.ReadOnlyBeaconState, epoch primitives.Epoch) ([]primitives.ValidatorIndex, error) {
-	ctx, span := trace.StartSpan(ctx, "helpers.ActiveValidatorIndices")
-	defer span.End()
-
 	seed, err := Seed(s, epoch, params.BeaconConfig().DomainBeaconAttester)
 	if err != nil {
 		return nil, errors.Wrap(err, "could not get seed")

beacon-chain/core/peerdas/BUILD.bazel

@@ -7,9 +7,10 @@ go_library(
         "info.go",
         "metrics.go",
         "p2p_interface.go",
+        "peer_sampling.go",
         "reconstruction.go",
+        "util.go",
         "validator.go",
-        "verification.go",
     ],
     importpath = "github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas",
     visibility = ["//visibility:public"],
@@ -44,10 +45,10 @@ go_test(
         "das_core_test.go",
         "info_test.go",
         "p2p_interface_test.go",
+        "peer_sampling_test.go",
         "reconstruction_test.go",
         "utils_test.go",
         "validator_test.go",
-        "verification_test.go",
     ],
     deps = [
         ":go_default_library",
@@ -68,6 +69,5 @@ go_test(
         "@com_github_ethereum_go_ethereum//p2p/enr:go_default_library",
         "@com_github_pkg_errors//:go_default_library",
         "@com_github_sirupsen_logrus//:go_default_library",
-        "@org_golang_x_sync//errgroup:go_default_library",
     ],
 )

beacon-chain/core/peerdas/das_core.go

@@ -7,6 +7,7 @@ import (
 	"time"
 
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg"
+	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
 	"github.com/OffchainLabs/prysm/v6/config/params"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
@@ -20,14 +21,11 @@ import (
 
 var (
 	// Custom errors
-	ErrCustodyGroupTooLarge               = errors.New("custody group too large")
-	ErrCustodyGroupCountTooLarge          = errors.New("custody group count too large")
-	ErrSizeMismatch                       = errors.New("mismatch in the number of blob KZG commitments and cellsAndProofs")
-	ErrNotEnoughDataColumnSidecars        = errors.New("not enough columns")
-	ErrDataColumnSidecarsNotSortedByIndex = errors.New("data column sidecars are not sorted by index")
-	errWrongComputedCustodyGroupCount     = errors.New("wrong computed custody group count, should never happen")
+	ErrCustodyGroupTooLarge           = errors.New("custody group too large")
+	errCustodyGroupCountTooLarge      = errors.New("custody group count too large")
+	errWrongComputedCustodyGroupCount = errors.New("wrong computed custody group count, should never happen")
 
-	// maxUint256 is the maximum value of an uint256.
+	// maxUint256 is the maximum value of a uint256.
 	maxUint256 = &uint256.Int{math.MaxUint64, math.MaxUint64, math.MaxUint64, math.MaxUint64}
 )
 
@@ -40,29 +38,18 @@ const (
 
 // CustodyGroups computes the custody groups the node should participate in for custody.
 // https://github.com/ethereum/consensus-specs/blob/v1.5.0-beta.5/specs/fulu/das-core.md#get_custody_groups
-func CustodyGroups(nodeId enode.ID, custodyGroupCount uint64) ([]uint64, error) {
+func CustodyGroups(nodeId enode.ID, custodyGroupCount uint64) (map[uint64]bool, error) {
 	numberOfCustodyGroup := params.BeaconConfig().NumberOfCustodyGroups
 
 	// Check if the custody group count is larger than the number of custody groups.
 	if custodyGroupCount > numberOfCustodyGroup {
-		return nil, ErrCustodyGroupCountTooLarge
-	}
-
-	// Shortcut if all custody groups are needed.
-	if custodyGroupCount == numberOfCustodyGroup {
-		custodyGroups := make([]uint64, 0, numberOfCustodyGroup)
-		for i := range numberOfCustodyGroup {
-			custodyGroups = append(custodyGroups, i)
-		}
-
-		return custodyGroups, nil
+		return nil, errCustodyGroupCountTooLarge
 	}
 
+	custodyGroups := make(map[uint64]bool, custodyGroupCount)
 	one := uint256.NewInt(1)
 
-	custodyGroupsMap := make(map[uint64]bool, custodyGroupCount)
-	custodyGroups := make([]uint64, 0, custodyGroupCount)
-	for currentId := new(uint256.Int).SetBytes(nodeId.Bytes()); uint64(len(custodyGroups)) < custodyGroupCount; {
+	for currentId := new(uint256.Int).SetBytes(nodeId.Bytes()); uint64(len(custodyGroups)) < custodyGroupCount; currentId.Add(currentId, one) {
 		// Convert to big endian bytes.
 		currentIdBytesBigEndian := currentId.Bytes32()
 
@@ -73,24 +60,15 @@ func CustodyGroups(nodeId enode.ID, custodyGroupCount uint64) ([]uint64, error)
 		hashedCurrentId := hash.Hash(currentIdBytesLittleEndian)
 
 		// Get the custody group ID.
-		custodyGroup := binary.LittleEndian.Uint64(hashedCurrentId[:8]) % numberOfCustodyGroup
+		custodyGroupId := binary.LittleEndian.Uint64(hashedCurrentId[:8]) % numberOfCustodyGroup
 
 		// Add the custody group to the map.
-		if !custodyGroupsMap[custodyGroup] {
-			custodyGroupsMap[custodyGroup] = true
-			custodyGroups = append(custodyGroups, custodyGroup)
-		}
+		custodyGroups[custodyGroupId] = true
 
+		// Overflow prevention.
 		if currentId.Cmp(maxUint256) == 0 {
-			// Overflow prevention.
 			currentId = uint256.NewInt(0)
-		} else {
-			// Increment the current ID.
-			currentId.Add(currentId, one)
 		}
-
-		// Sort the custody groups.
-		slices.Sort[[]uint64](custodyGroups)
 	}
 
 	// Final check.
@@ -107,8 +85,8 @@ func ComputeColumnsForCustodyGroup(custodyGroup uint64) ([]uint64, error) {
 	beaconConfig := params.BeaconConfig()
 	numberOfCustodyGroup := beaconConfig.NumberOfCustodyGroups
 
-	if custodyGroup >= numberOfCustodyGroup {
-		return nil, ErrCustodyGroupTooLarge
+	if custodyGroup > numberOfCustodyGroup {
+		return nil, errCustodyGroupCountTooLarge
 	}
 
 	numberOfColumns := beaconConfig.NumberOfColumns
@@ -125,11 +103,10 @@ func ComputeColumnsForCustodyGroup(custodyGroup uint64) ([]uint64, error) {
 }
 
 // DataColumnSidecars computes the data column sidecars from the signed block, cells and cell proofs.
-// The returned value contains pointers to function parameters.
-// (If the caller alterates `cellsAndProofs` afterwards, the returned value will be modified as well.)
 // https://github.com/ethereum/consensus-specs/blob/v1.5.0-beta.3/specs/fulu/das-core.md#get_data_column_sidecars
 func DataColumnSidecars(signedBlock interfaces.ReadOnlySignedBeaconBlock, cellsAndProofs []kzg.CellsAndProofs) ([]*ethpb.DataColumnSidecar, error) {
-	if signedBlock == nil || signedBlock.IsNil() || len(cellsAndProofs) == 0 {
+	start := time.Now()
+	if signedBlock == nil || len(cellsAndProofs) == 0 {
 		return nil, nil
 	}
 
@@ -141,7 +118,7 @@ func DataColumnSidecars(signedBlock interfaces.ReadOnlySignedBeaconBlock, cellsA
 	}
 
 	if len(blobKzgCommitments) != len(cellsAndProofs) {
-		return nil, ErrSizeMismatch
+		return nil, errors.New("mismatch in the number of blob KZG commitments and cellsAndProofs")
 	}
 
 	signedBlockHeader, err := signedBlock.Header()
@@ -154,87 +131,9 @@ func DataColumnSidecars(signedBlock interfaces.ReadOnlySignedBeaconBlock, cellsA
 		return nil, errors.Wrap(err, "merkle proof ZKG commitments")
 	}
 
-	dataColumnSidecars, err := dataColumnsSidecars(signedBlockHeader, blobKzgCommitments, kzgCommitmentsInclusionProof, cellsAndProofs)
-	if err != nil {
-		return nil, errors.Wrap(err, "data column sidecars")
-	}
-
-	return dataColumnSidecars, nil
-}
-
-// ComputeCustodyGroupForColumn computes the custody group for a given column.
-// It is the reciprocal function of ComputeColumnsForCustodyGroup.
-func ComputeCustodyGroupForColumn(columnIndex uint64) (uint64, error) {
-	beaconConfig := params.BeaconConfig()
-	numberOfColumns := beaconConfig.NumberOfColumns
-	numberOfCustodyGroups := beaconConfig.NumberOfCustodyGroups
-
-	if columnIndex >= numberOfColumns {
-		return 0, ErrIndexTooLarge
-	}
-
-	return columnIndex % numberOfCustodyGroups, nil
-}
-
-// CustodyGroupSamplingSize returns the number of custody groups the node should sample from.
-// https://github.com/ethereum/consensus-specs/blob/v1.5.0-beta.5/specs/fulu/das-core.md#custody-sampling
-func (custodyInfo *CustodyInfo) CustodyGroupSamplingSize(ct CustodyType) uint64 {
-	custodyGroupCount := custodyInfo.TargetGroupCount.Get()
-
-	if ct == Actual {
-		custodyGroupCount = custodyInfo.ActualGroupCount()
-	}
-
-	samplesPerSlot := params.BeaconConfig().SamplesPerSlot
-	return max(samplesPerSlot, custodyGroupCount)
-}
-
-// CustodyColumns computes the custody columns from the custody groups.
-func CustodyColumns(custodyGroups []uint64) (map[uint64]bool, error) {
-	numberOfCustodyGroups := params.BeaconConfig().NumberOfCustodyGroups
-
-	custodyGroupCount := len(custodyGroups)
-
-	// Compute the columns for each custody group.
-	columns := make(map[uint64]bool, custodyGroupCount)
-	for _, group := range custodyGroups {
-		if group >= numberOfCustodyGroups {
-			return nil, ErrCustodyGroupTooLarge
-		}
-
-		groupColumns, err := ComputeColumnsForCustodyGroup(group)
-		if err != nil {
-			return nil, errors.Wrap(err, "compute columns for custody group")
-		}
-
-		for _, column := range groupColumns {
-			columns[column] = true
-		}
-	}
-
-	return columns, nil
-}
-
-// dataColumnsSidecars computes the data column sidecars from the signed block header, the blob KZG commiments,
-// the KZG commitment includion proofs and cells and cell proofs.
-// The returned value contains pointers to function parameters.
-// (If the caller alterates input parameters afterwards, the returned value will be modified as well.)
-func dataColumnsSidecars(
-	signedBlockHeader *ethpb.SignedBeaconBlockHeader,
-	blobKzgCommitments [][]byte,
-	kzgCommitmentsInclusionProof [][]byte,
-	cellsAndProofs []kzg.CellsAndProofs,
-) ([]*ethpb.DataColumnSidecar, error) {
-	start := time.Now()
-	if len(blobKzgCommitments) != len(cellsAndProofs) {
-		return nil, ErrSizeMismatch
-	}
-
-	numberOfColumns := params.BeaconConfig().NumberOfColumns
-
 	blobsCount := len(cellsAndProofs)
-	sidecars := make([]*ethpb.DataColumnSidecar, 0, numberOfColumns)
-	for columnIndex := range numberOfColumns {
+	sidecars := make([]*ethpb.DataColumnSidecar, 0, fieldparams.NumberOfColumns)
+	for columnIndex := range uint64(fieldparams.NumberOfColumns) {
 		column := make([]kzg.Cell, 0, blobsCount)
 		kzgProofOfColumn := make([]kzg.Proof, 0, blobsCount)
 
@@ -256,7 +155,8 @@ func dataColumnsSidecars(
 
 		kzgProofOfColumnBytes := make([][]byte, 0, blobsCount)
 		for _, kzgProof := range kzgProofOfColumn {
-			kzgProofOfColumnBytes = append(kzgProofOfColumnBytes, kzgProof[:])
+			copiedProof := kzgProof
+			kzgProofOfColumnBytes = append(kzgProofOfColumnBytes, copiedProof[:])
 		}
 
 		sidecar := &ethpb.DataColumnSidecar{
@@ -274,3 +174,179 @@ func dataColumnsSidecars(
 	dataColumnComputationTime.Observe(float64(time.Since(start).Milliseconds()))
 	return sidecars, nil
 }
+
+// Blobs extract blobs from `dataColumnsSidecar`.
+// This can be seen as the reciprocal function of DataColumnSidecars.
+// `dataColumnsSidecar` needs to contain the datacolumns corresponding to the non-extended matrix,
+// else an error will be returned.
+// (`dataColumnsSidecar` can contain extra columns, but they will be ignored.)
+func Blobs(indices map[uint64]bool, dataColumnsSidecar []*ethpb.DataColumnSidecar) ([]*blocks.VerifiedROBlob, error) {
+	columnCount := fieldparams.NumberOfColumns
+
+	neededColumnCount := columnCount / 2
+
+	// Check if all needed columns are present.
+	sliceIndexFromColumnIndex := make(map[uint64]int, len(dataColumnsSidecar))
+	for i := range dataColumnsSidecar {
+		dataColumnSideCar := dataColumnsSidecar[i]
+		index := dataColumnSideCar.Index
+
+		if index < uint64(neededColumnCount) {
+			sliceIndexFromColumnIndex[index] = i
+		}
+	}
+
+	actualColumnCount := len(sliceIndexFromColumnIndex)
+
+	// Get missing columns.
+	if actualColumnCount < neededColumnCount {
+		missingColumns := make(map[int]bool, neededColumnCount-actualColumnCount)
+		for i := range neededColumnCount {
+			if _, ok := sliceIndexFromColumnIndex[uint64(i)]; !ok {
+				missingColumns[i] = true
+			}
+		}
+
+		missingColumnsSlice := make([]int, 0, len(missingColumns))
+		for i := range missingColumns {
+			missingColumnsSlice = append(missingColumnsSlice, i)
+		}
+
+		slices.Sort[[]int](missingColumnsSlice)
+		return nil, errors.Errorf("some columns are missing: %v", missingColumnsSlice)
+	}
+
+	// It is safe to retrieve the first column since we already checked that `dataColumnsSidecar` is not empty.
+	firstDataColumnSidecar := dataColumnsSidecar[0]
+
+	blobCount := uint64(len(firstDataColumnSidecar.Column))
+
+	// Check all colums have te same length.
+	for i := range dataColumnsSidecar {
+		if uint64(len(dataColumnsSidecar[i].Column)) != blobCount {
+			return nil, errors.Errorf("mismatch in the length of the data columns, expected %d, got %d", blobCount, len(dataColumnsSidecar[i].Column))
+		}
+	}
+
+	// Reconstruct verified RO blobs from columns.
+	verifiedROBlobs := make([]*blocks.VerifiedROBlob, 0, blobCount)
+
+	// Populate and filter indices.
+	indicesSlice := populateAndFilterIndices(indices, blobCount)
+
+	for _, blobIndex := range indicesSlice {
+		var blob kzg.Blob
+
+		// Compute the content of the blob.
+		for columnIndex := range neededColumnCount {
+			sliceIndex, ok := sliceIndexFromColumnIndex[uint64(columnIndex)]
+			if !ok {
+				return nil, errors.Errorf("missing column %d, this should never happen", columnIndex)
+			}
+
+			dataColumnSideCar := dataColumnsSidecar[sliceIndex]
+			cell := dataColumnSideCar.Column[blobIndex]
+
+			for i := 0; i < len(cell); i++ {
+				blob[columnIndex*kzg.BytesPerCell+i] = cell[i]
+			}
+		}
+
+		// Retrieve the blob KZG commitment.
+		blobKZGCommitment := kzg.Commitment(firstDataColumnSidecar.KzgCommitments[blobIndex])
+
+		// Compute the blob KZG proof.
+		blobKzgProof, err := kzg.ComputeBlobKZGProof(&blob, blobKZGCommitment)
+		if err != nil {
+			return nil, errors.Wrap(err, "compute blob KZG proof")
+		}
+
+		blobSidecar := &ethpb.BlobSidecar{
+			Index:                    blobIndex,
+			Blob:                     blob[:],
+			KzgCommitment:            blobKZGCommitment[:],
+			KzgProof:                 blobKzgProof[:],
+			SignedBlockHeader:        firstDataColumnSidecar.SignedBlockHeader,
+			CommitmentInclusionProof: firstDataColumnSidecar.KzgCommitmentsInclusionProof,
+		}
+
+		roBlob, err := blocks.NewROBlob(blobSidecar)
+		if err != nil {
+			return nil, errors.Wrap(err, "new RO blob")
+		}
+
+		verifiedROBlob := blocks.NewVerifiedROBlob(roBlob)
+		verifiedROBlobs = append(verifiedROBlobs, &verifiedROBlob)
+	}
+
+	return verifiedROBlobs, nil
+}
+
+// CustodyGroupSamplingSize returns the number of custody groups the node should sample from.
+// https://github.com/ethereum/consensus-specs/blob/v1.5.0-beta.5/specs/fulu/das-core.md#custody-sampling
+func (custodyInfo *CustodyInfo) CustodyGroupSamplingSize(ct CustodyType) uint64 {
+	custodyGroupCount := custodyInfo.TargetGroupCount.Get()
+
+	if ct == Actual {
+		custodyGroupCount = custodyInfo.ActualGroupCount()
+	}
+
+	samplesPerSlot := params.BeaconConfig().SamplesPerSlot
+	return max(samplesPerSlot, custodyGroupCount)
+}
+
+// CustodyColumns computes the custody columns from the custody groups.
+func CustodyColumns(custodyGroups map[uint64]bool) (map[uint64]bool, error) {
+	numberOfCustodyGroups := params.BeaconConfig().NumberOfCustodyGroups
+
+	custodyGroupCount := len(custodyGroups)
+
+	// Compute the columns for each custody group.
+	columns := make(map[uint64]bool, custodyGroupCount)
+	for group := range custodyGroups {
+		if group >= numberOfCustodyGroups {
+			return nil, ErrCustodyGroupTooLarge
+		}
+
+		groupColumns, err := ComputeColumnsForCustodyGroup(group)
+		if err != nil {
+			return nil, errors.Wrap(err, "compute columns for custody group")
+		}
+
+		for _, column := range groupColumns {
+			columns[column] = true
+		}
+	}
+
+	return columns, nil
+}
+
+// populateAndFilterIndices returns a sorted slices of indices, setting all indices if none are provided,
+// and filtering out indices higher than the blob count.
+func populateAndFilterIndices(indices map[uint64]bool, blobCount uint64) []uint64 {
+	// If no indices are provided, provide all blobs.
+	if len(indices) == 0 {
+		for i := range blobCount {
+			indices[i] = true
+		}
+	}
+
+	// Filter blobs index higher than the blob count.
+	filteredIndices := make(map[uint64]bool, len(indices))
+	for i := range indices {
+		if i < blobCount {
+			filteredIndices[i] = true
+		}
+	}
+
+	// Transform set to slice.
+	indicesSlice := make([]uint64, 0, len(filteredIndices))
+	for i := range filteredIndices {
+		indicesSlice = append(indicesSlice, i)
+	}
+
+	// Sort the indices.
+	slices.Sort[[]uint64](indicesSlice)
+
+	return indicesSlice
+}

beacon-chain/core/peerdas/das_core_test.go

@@ -5,103 +5,153 @@ import (
 
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
-	"github.com/OffchainLabs/prysm/v6/config/params"
+	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
 	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
 	"github.com/OffchainLabs/prysm/v6/testing/require"
 	"github.com/OffchainLabs/prysm/v6/testing/util"
-	"github.com/ethereum/go-ethereum/p2p/enode"
+	"github.com/pkg/errors"
 )
 
-func TestCustodyGroups(t *testing.T) {
-	// --------------------------------------------
-	// The happy path is unit tested in spec tests.
-	// --------------------------------------------
-	numberOfCustodyGroup := params.BeaconConfig().NumberOfCustodyGroups
-	_, err := peerdas.CustodyGroups(enode.ID{}, numberOfCustodyGroup+1)
-	require.ErrorIs(t, err, peerdas.ErrCustodyGroupCountTooLarge)
-}
-
-func TestComputeColumnsForCustodyGroup(t *testing.T) {
-	// --------------------------------------------
-	// The happy path is unit tested in spec tests.
-	// --------------------------------------------
-	numberOfCustodyGroup := params.BeaconConfig().NumberOfCustodyGroups
-	_, err := peerdas.ComputeColumnsForCustodyGroup(numberOfCustodyGroup)
-	require.ErrorIs(t, err, peerdas.ErrCustodyGroupTooLarge)
-}
+// ---------------------------------------------------------------
+// ( CustodyGroups is unit tested in spec tests.                 )
+// ( ComputeColumnsForCustodyGroup is unit tested in spec tests. )
+// ---------------------------------------------------------------
 
 func TestDataColumnSidecars(t *testing.T) {
-	t.Run("nil signed block", func(t *testing.T) {
-		var expected []*ethpb.DataColumnSidecar = nil
-		actual, err := peerdas.DataColumnSidecars(nil, []kzg.CellsAndProofs{})
-		require.NoError(t, err)
+	var expected []*ethpb.DataColumnSidecar = nil
+	actual, err := peerdas.DataColumnSidecars(nil, []kzg.CellsAndProofs{})
+	require.NoError(t, err)
 
-		require.DeepSSZEqual(t, expected, actual)
-	})
-
-	t.Run("empty cells and proofs", func(t *testing.T) {
-		// Create a protobuf signed beacon block.
-		signedBeaconBlockPb := util.NewBeaconBlockDeneb()
-
-		// Create a signed beacon block from the protobuf.
-		signedBeaconBlock, err := blocks.NewSignedBeaconBlock(signedBeaconBlockPb)
-		require.NoError(t, err)
-
-		actual, err := peerdas.DataColumnSidecars(signedBeaconBlock, []kzg.CellsAndProofs{})
-		require.NoError(t, err)
-		require.IsNil(t, actual)
-	})
-
-	t.Run("sizes mismatch", func(t *testing.T) {
-		// Create a protobuf signed beacon block.
-		signedBeaconBlockPb := util.NewBeaconBlockDeneb()
-
-		// Create a signed beacon block from the protobuf.
-		signedBeaconBlock, err := blocks.NewSignedBeaconBlock(signedBeaconBlockPb)
-		require.NoError(t, err)
-
-		// Create cells and proofs.
-		cellsAndProofs := make([]kzg.CellsAndProofs, 1)
-
-		_, err = peerdas.DataColumnSidecars(signedBeaconBlock, cellsAndProofs)
-		require.ErrorIs(t, err, peerdas.ErrSizeMismatch)
-	})
+	require.DeepSSZEqual(t, expected, actual)
 }
 
-func TestComputeCustodyGroupForColumn(t *testing.T) {
-	params.SetupTestConfigCleanup(t)
-	config := params.BeaconConfig()
-	config.NumberOfColumns = 128
-	config.NumberOfCustodyGroups = 64
-	params.OverrideBeaconConfig(config)
+func TestBlobs(t *testing.T) {
+	blobsIndice := map[uint64]bool{}
 
-	t.Run("index too large", func(t *testing.T) {
-		_, err := peerdas.ComputeCustodyGroupForColumn(1_000_000)
-		require.ErrorIs(t, err, peerdas.ErrIndexTooLarge)
-	})
+	almostAllColumns := make([]*ethpb.DataColumnSidecar, 0, fieldparams.NumberOfColumns/2)
+	for i := 2; i < fieldparams.NumberOfColumns/2+2; i++ {
+		almostAllColumns = append(almostAllColumns, &ethpb.DataColumnSidecar{
+			Index: uint64(i),
+		})
+	}
 
-	t.Run("nominal", func(t *testing.T) {
-		expected := uint64(2)
-		actual, err := peerdas.ComputeCustodyGroupForColumn(2)
+	testCases := []struct {
+		name     string
+		input    []*ethpb.DataColumnSidecar
+		expected []*blocks.VerifiedROBlob
+		err      error
+	}{
+		{
+			name:     "empty input",
+			input:    []*ethpb.DataColumnSidecar{},
+			expected: nil,
+			err:      errors.New("some columns are missing: [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63]"),
+		},
+		{
+			name:     "missing columns",
+			input:    almostAllColumns,
+			expected: nil,
+			err:      errors.New("some columns are missing: [0 1]"),
+		},
+	}
+
+	for _, tc := range testCases {
+		t.Run(tc.name, func(t *testing.T) {
+			actual, err := peerdas.Blobs(blobsIndice, tc.input)
+			if tc.err != nil {
+				require.Equal(t, tc.err.Error(), err.Error())
+			} else {
+				require.NoError(t, err)
+			}
+			require.DeepSSZEqual(t, tc.expected, actual)
+		})
+	}
+}
+
+func TestDataColumnsSidecarsBlobsRoundtrip(t *testing.T) {
+	const blobCount = 5
+	blobsIndex := map[uint64]bool{}
+
+	// Start the trusted setup.
+	err := kzg.Start()
+	require.NoError(t, err)
+
+	// Create a protobuf signed beacon block.
+	signedBeaconBlockPb := util.NewBeaconBlockDeneb()
+
+	// Generate random blobs and their corresponding commitments and proofs.
+	blobs := make([]kzg.Blob, 0, blobCount)
+	blobKzgCommitments := make([]*kzg.Commitment, 0, blobCount)
+	blobKzgProofs := make([]*kzg.Proof, 0, blobCount)
+
+	for blobIndex := range blobCount {
+		// Create a random blob.
+		blob := getRandBlob(int64(blobIndex))
+		blobs = append(blobs, blob)
+
+		// Generate a blobKZGCommitment for the blob.
+		blobKZGCommitment, proof, err := generateCommitmentAndProof(&blob)
 		require.NoError(t, err)
-		require.Equal(t, expected, actual)
 
-		expected = uint64(3)
-		actual, err = peerdas.ComputeCustodyGroupForColumn(3)
-		require.NoError(t, err)
-		require.Equal(t, expected, actual)
+		blobKzgCommitments = append(blobKzgCommitments, blobKZGCommitment)
+		blobKzgProofs = append(blobKzgProofs, proof)
+	}
 
-		expected = uint64(2)
-		actual, err = peerdas.ComputeCustodyGroupForColumn(66)
-		require.NoError(t, err)
-		require.Equal(t, expected, actual)
+	// Set the commitments into the block.
+	blobZkgCommitmentsBytes := make([][]byte, 0, blobCount)
+	for _, blobKZGCommitment := range blobKzgCommitments {
+		blobZkgCommitmentsBytes = append(blobZkgCommitmentsBytes, blobKZGCommitment[:])
+	}
 
-		expected = uint64(3)
-		actual, err = peerdas.ComputeCustodyGroupForColumn(67)
+	signedBeaconBlockPb.Block.Body.BlobKzgCommitments = blobZkgCommitmentsBytes
+
+	// Generate verified RO blobs.
+	verifiedROBlobs := make([]*blocks.VerifiedROBlob, 0, blobCount)
+
+	// Create a signed beacon block from the protobuf.
+	signedBeaconBlock, err := blocks.NewSignedBeaconBlock(signedBeaconBlockPb)
+	require.NoError(t, err)
+
+	commitmentInclusionProof, err := blocks.MerkleProofKZGCommitments(signedBeaconBlock.Block().Body())
+	require.NoError(t, err)
+
+	for blobIndex := range blobCount {
+		blob := blobs[blobIndex]
+		blobKZGCommitment := blobKzgCommitments[blobIndex]
+		blobKzgProof := blobKzgProofs[blobIndex]
+
+		// Get the signed beacon block header.
+		signedBeaconBlockHeader, err := signedBeaconBlock.Header()
 		require.NoError(t, err)
-		require.Equal(t, expected, actual)
-	})
+
+		blobSidecar := &ethpb.BlobSidecar{
+			Index:                    uint64(blobIndex),
+			Blob:                     blob[:],
+			KzgCommitment:            blobKZGCommitment[:],
+			KzgProof:                 blobKzgProof[:],
+			SignedBlockHeader:        signedBeaconBlockHeader,
+			CommitmentInclusionProof: commitmentInclusionProof,
+		}
+
+		roBlob, err := blocks.NewROBlob(blobSidecar)
+		require.NoError(t, err)
+
+		verifiedROBlob := blocks.NewVerifiedROBlob(roBlob)
+		verifiedROBlobs = append(verifiedROBlobs, &verifiedROBlob)
+	}
+
+	// Compute data columns sidecars from the signed beacon block and from the blobs.
+	cellsAndProofs := util.GenerateCellsAndProofs(t, blobs)
+	dataColumnsSidecar, err := peerdas.DataColumnSidecars(signedBeaconBlock, cellsAndProofs)
+	require.NoError(t, err)
+
+	// Compute the blobs from the data columns sidecar.
+	roundtripBlobs, err := peerdas.Blobs(blobsIndex, dataColumnsSidecar)
+	require.NoError(t, err)
+
+	// Check that the blobs are the same.
+	require.DeepSSZEqual(t, verifiedROBlobs, roundtripBlobs)
 }
 
 func TestCustodyGroupSamplingSize(t *testing.T) {
@@ -162,12 +212,12 @@ func TestCustodyGroupSamplingSize(t *testing.T) {
 
 func TestCustodyColumns(t *testing.T) {
 	t.Run("group too large", func(t *testing.T) {
-		_, err := peerdas.CustodyColumns([]uint64{1_000_000})
+		_, err := peerdas.CustodyColumns(map[uint64]bool{1_000_000: true})
 		require.ErrorIs(t, err, peerdas.ErrCustodyGroupTooLarge)
 	})
 
 	t.Run("nominal", func(t *testing.T) {
-		input := []uint64{1, 2}
+		input := map[uint64]bool{1: true, 2: true}
 		expected := map[uint64]bool{1: true, 2: true}
 
 		actual, err := peerdas.CustodyColumns(input)

beacon-chain/core/peerdas/info.go

@@ -91,14 +91,8 @@ func Info(nodeID enode.ID, custodyGroupCount uint64) (*info, bool, error) {
 	// Compute data columns subnets.
 	dataColumnsSubnets := DataColumnSubnets(custodyColumns)
 
-	// Convert the custody groups to a map.
-	custodyGroupsMap := make(map[uint64]bool, len(custodyGroups))
-	for _, group := range custodyGroups {
-		custodyGroupsMap[group] = true
-	}
-
 	result := &info{
-		CustodyGroups:      custodyGroupsMap,
+		CustodyGroups:      custodyGroups,
 		CustodyColumns:     custodyColumns,
 		DataColumnsSubnets: dataColumnsSubnets,
 	}
@@ -117,7 +111,7 @@ func (custodyInfo *CustodyInfo) ActualGroupCount() uint64 {
 // CustodyGroupCount returns the number of groups we should participate in for custody.
 func (tcgc *targetCustodyGroupCount) Get() uint64 {
 	// If subscribed to all subnets, return the number of custody groups.
-	if flags.Get().SubscribeAllDataSubnets {
+	if flags.Get().SubscribeAllColumnSubnets {
 		return params.BeaconConfig().NumberOfCustodyGroups
 	}
 
@@ -144,7 +138,7 @@ func (tcgc *targetCustodyGroupCount) SetValidatorsCustodyRequirement(value uint6
 // Get returns the to advertise custody group count.
 func (tacgc *toAdverstiseCustodyGroupCount) Get() uint64 {
 	// If subscribed to all subnets, return the number of custody groups.
-	if flags.Get().SubscribeAllDataSubnets {
+	if flags.Get().SubscribeAllColumnSubnets {
 		return params.BeaconConfig().NumberOfCustodyGroups
 	}
 

beacon-chain/core/peerdas/info_test.go

@@ -35,7 +35,7 @@ func TestTargetCustodyGroupCount(t *testing.T) {
 		expected                     uint64
 	}{
 		{
-			name:                         "subscribed to all data subnets",
+			name:                         "subscribed to all subnets",
 			subscribeToAllColumns:        true,
 			validatorsCustodyRequirement: 100,
 			expected:                     128,
@@ -60,7 +60,7 @@ func TestTargetCustodyGroupCount(t *testing.T) {
 			if tc.subscribeToAllColumns {
 				resetFlags := flags.Get()
 				gFlags := new(flags.GlobalFlags)
-				gFlags.SubscribeAllDataSubnets = true
+				gFlags.SubscribeToAllSubnets = true
 				flags.Init(gFlags)
 				defer flags.Init(resetFlags)
 			}
@@ -112,7 +112,7 @@ func TestToAdvertiseCustodyGroupCount(t *testing.T) {
 			if tc.subscribeToAllColumns {
 				resetFlags := flags.Get()
 				gFlags := new(flags.GlobalFlags)
-				gFlags.SubscribeAllDataSubnets = true
+				gFlags.SubscribeAllColumnSubnets = true
 				flags.Init(gFlags)
 				defer flags.Init(resetFlags)
 			}

beacon-chain/core/peerdas/metrics.go

@@ -9,6 +9,6 @@ var dataColumnComputationTime = promauto.NewHistogram(
 	prometheus.HistogramOpts{
 		Name:    "beacon_data_column_sidecar_computation_milliseconds",
 		Help:    "Captures the time taken to compute data column sidecars from blobs.",
-		Buckets: []float64{25, 50, 100, 250, 500, 750, 1000},
+		Buckets: []float64{100, 250, 500, 750, 1000, 1500, 2000, 4000, 8000, 12000, 16000},
 	},
 )

beacon-chain/core/peerdas/p2p_interface.go

@@ -146,6 +146,21 @@ func DataColumnSubnets(dataColumns map[uint64]bool) map[uint64]bool {
 	return subnets
 }
 
+// ComputeCustodyGroupForColumn computes the custody group for a given column.
+// It is the reciprocal function of ComputeColumnsForCustodyGroup.
+func ComputeCustodyGroupForColumn(columnIndex uint64) (uint64, error) {
+	beaconConfig := params.BeaconConfig()
+	numberOfColumns := beaconConfig.NumberOfColumns
+	numberOfCustodyGroups := beaconConfig.NumberOfCustodyGroups
+
+	if columnIndex >= numberOfColumns {
+		return 0, ErrIndexTooLarge
+	}
+
+	columnsPerGroup := numberOfColumns / numberOfCustodyGroups
+	return columnIndex / columnsPerGroup, nil
+}
+
 // CustodyGroupCountFromRecord extracts the custody group count from an ENR record.
 func CustodyGroupCountFromRecord(record *enr.Record) (uint64, error) {
 	if record == nil {

beacon-chain/core/peerdas/p2p_interface_test.go

@@ -2,7 +2,6 @@ package peerdas_test
 
 import (
 	"crypto/rand"
-	"fmt"
 	"testing"
 
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg"
@@ -53,15 +52,51 @@ func TestVerifyDataColumnSidecar(t *testing.T) {
 }
 
 func TestVerifyDataColumnSidecarKZGProofs(t *testing.T) {
-	const (
-		blobCount = 6
-		seed      = 0
-	)
 	err := kzg.Start()
 	require.NoError(t, err)
 
+	generateSidecars := func(t *testing.T) []*ethpb.DataColumnSidecar {
+		const blobCount = int64(6)
+
+		dbBlock := util.NewBeaconBlockDeneb()
+
+		commitments := make([][]byte, 0, blobCount)
+		blobs := make([]kzg.Blob, 0, blobCount)
+
+		for i := range blobCount {
+			blob := getRandBlob(i)
+			commitment, _, err := generateCommitmentAndProof(&blob)
+			require.NoError(t, err)
+
+			commitments = append(commitments, commitment[:])
+			blobs = append(blobs, blob)
+		}
+
+		dbBlock.Block.Body.BlobKzgCommitments = commitments
+		sBlock, err := blocks.NewSignedBeaconBlock(dbBlock)
+		require.NoError(t, err)
+
+		cellsAndProofs := util.GenerateCellsAndProofs(t, blobs)
+		sidecars, err := peerdas.DataColumnSidecars(sBlock, cellsAndProofs)
+		require.NoError(t, err)
+
+		return sidecars
+	}
+
+	generateRODataColumnSidecars := func(t *testing.T, sidecars []*ethpb.DataColumnSidecar) []blocks.RODataColumn {
+		roDataColumnSidecars := make([]blocks.RODataColumn, 0, len(sidecars))
+		for _, sidecar := range sidecars {
+			roCol, err := blocks.NewRODataColumn(sidecar)
+			require.NoError(t, err)
+
+			roDataColumnSidecars = append(roDataColumnSidecars, roCol)
+		}
+
+		return roDataColumnSidecars
+	}
+
 	t.Run("invalid proof", func(t *testing.T) {
-		sidecars := generateRandomSidecars(t, seed, blobCount)
+		sidecars := generateSidecars(t)
 		sidecars[0].Column[0][0]++ // It is OK to overflow
 		roDataColumnSidecars := generateRODataColumnSidecars(t, sidecars)
 
@@ -70,7 +105,7 @@ func TestVerifyDataColumnSidecarKZGProofs(t *testing.T) {
 	})
 
 	t.Run("nominal", func(t *testing.T) {
-		sidecars := generateRandomSidecars(t, seed, blobCount)
+		sidecars := generateSidecars(t)
 		roDataColumnSidecars := generateRODataColumnSidecars(t, sidecars)
 
 		err := peerdas.VerifyDataColumnsSidecarKZGProofs(roDataColumnSidecars)
@@ -223,6 +258,36 @@ func TestDataColumnSubnets(t *testing.T) {
 	}
 }
 
+func TestComputeCustodyGroupForColumn(t *testing.T) {
+	params.SetupTestConfigCleanup(t)
+	config := params.BeaconConfig()
+	config.NumberOfColumns = 128
+	config.NumberOfCustodyGroups = 64
+	params.OverrideBeaconConfig(config)
+
+	t.Run("index too large", func(t *testing.T) {
+		_, err := peerdas.ComputeCustodyGroupForColumn(1_000_000)
+		require.ErrorIs(t, err, peerdas.ErrIndexTooLarge)
+	})
+
+	t.Run("nominal", func(t *testing.T) {
+		expected := uint64(1)
+		actual, err := peerdas.ComputeCustodyGroupForColumn(2)
+		require.NoError(t, err)
+		require.Equal(t, expected, actual)
+
+		expected = uint64(1)
+		actual, err = peerdas.ComputeCustodyGroupForColumn(3)
+		require.NoError(t, err)
+		require.Equal(t, expected, actual)
+
+		expected = uint64(2)
+		actual, err = peerdas.ComputeCustodyGroupForColumn(4)
+		require.NoError(t, err)
+		require.Equal(t, expected, actual)
+	})
+}
+
 func TestCustodyGroupCountFromRecord(t *testing.T) {
 	t.Run("nil record", func(t *testing.T) {
 		_, err := peerdas.CustodyGroupCountFromRecord(nil)
@@ -246,96 +311,6 @@ func TestCustodyGroupCountFromRecord(t *testing.T) {
 	})
 }
 
-func BenchmarkVerifyDataColumnSidecarKZGProofs_SameCommitments_NoBatch(b *testing.B) {
-	const blobCount = 12
-	err := kzg.Start()
-	require.NoError(b, err)
-
-	b.StopTimer()
-	b.ResetTimer()
-	for i := range int64(b.N) {
-		// Generate new random sidecars to ensure the KZG backend does not cache anything.
-		sidecars := generateRandomSidecars(b, i, blobCount)
-		roDataColumnSidecars := generateRODataColumnSidecars(b, sidecars)
-
-		for _, sidecar := range roDataColumnSidecars {
-			sidecars := []blocks.RODataColumn{sidecar}
-			b.StartTimer()
-			err := peerdas.VerifyDataColumnsSidecarKZGProofs(sidecars)
-			b.StopTimer()
-			require.NoError(b, err)
-		}
-	}
-}
-
-func BenchmarkVerifyDataColumnSidecarKZGProofs_DiffCommitments_Batch(b *testing.B) {
-	const blobCount = 12
-	numberOfColumns := int64(params.BeaconConfig().NumberOfColumns)
-	err := kzg.Start()
-	require.NoError(b, err)
-
-	columnsCounts := []int64{1, 2, 4, 8, 16, 32, 64, 128}
-
-	for i, columnsCount := range columnsCounts {
-		b.Run(fmt.Sprintf("columnsCount_%d", columnsCount), func(b *testing.B) {
-			b.StopTimer()
-			b.ResetTimer()
-
-			for j := range int64(b.N) {
-				allSidecars := make([]*ethpb.DataColumnSidecar, 0, numberOfColumns)
-				for k := int64(0); k < numberOfColumns; k += columnsCount {
-					// Use different seeds to generate different blobs/commitments
-					seed := int64(b.N*i) + numberOfColumns*j + blobCount*k
-					sidecars := generateRandomSidecars(b, seed, blobCount)
-
-					// Pick sidecars.
-					allSidecars = append(allSidecars, sidecars[k:k+columnsCount]...)
-				}
-
-				roDataColumnSidecars := generateRODataColumnSidecars(b, allSidecars)
-
-				b.StartTimer()
-				err := peerdas.VerifyDataColumnsSidecarKZGProofs(roDataColumnSidecars)
-				b.StopTimer()
-				require.NoError(b, err)
-			}
-		})
-	}
-}
-
-func BenchmarkVerifyDataColumnSidecarKZGProofs_DiffCommitments_Batch4(b *testing.B) {
-	const (
-		blobCount = 12
-
-		// columnsCount*batchCount = 128
-		columnsCount = 4
-		batchCount   = 32
-	)
-
-	err := kzg.Start()
-	require.NoError(b, err)
-
-	b.StopTimer()
-	b.ResetTimer()
-
-	for i := range int64(b.N) {
-		allSidecars := make([][]blocks.RODataColumn, 0, batchCount)
-		for j := range int64(batchCount) {
-			// Use different seeds to generate different blobs/commitments
-			sidecars := generateRandomSidecars(b, int64(batchCount)*i+j*blobCount, blobCount)
-			roDataColumnSidecars := generateRODataColumnSidecars(b, sidecars[:columnsCount])
-			allSidecars = append(allSidecars, roDataColumnSidecars)
-		}
-
-		for _, sidecars := range allSidecars {
-			b.StartTimer()
-			err := peerdas.VerifyDataColumnsSidecarKZGProofs(sidecars)
-			b.StopTimer()
-			require.NoError(b, err)
-		}
-	}
-}
-
 func createTestSidecar(t *testing.T, index uint64, column, kzgCommitments, kzgProofs [][]byte) blocks.RODataColumn {
 	pbSignedBeaconBlock := util.NewBeaconBlockDeneb()
 	signedBeaconBlock, err := blocks.NewSignedBeaconBlock(pbSignedBeaconBlock)
@@ -357,42 +332,3 @@ func createTestSidecar(t *testing.T, index uint64, column, kzgCommitments, kzgPr
 
 	return roSidecar
 }
-
-func generateRandomSidecars(t testing.TB, seed, blobCount int64) []*ethpb.DataColumnSidecar {
-	dbBlock := util.NewBeaconBlockDeneb()
-
-	commitments := make([][]byte, 0, blobCount)
-	blobs := make([]kzg.Blob, 0, blobCount)
-
-	for i := range blobCount {
-		subSeed := seed + i
-		blob := getRandBlob(subSeed)
-		commitment, err := generateCommitment(&blob)
-		require.NoError(t, err)
-
-		commitments = append(commitments, commitment[:])
-		blobs = append(blobs, blob)
-	}
-
-	dbBlock.Block.Body.BlobKzgCommitments = commitments
-	sBlock, err := blocks.NewSignedBeaconBlock(dbBlock)
-	require.NoError(t, err)
-
-	cellsAndProofs := util.GenerateCellsAndProofs(t, blobs)
-	sidecars, err := peerdas.DataColumnSidecars(sBlock, cellsAndProofs)
-	require.NoError(t, err)
-
-	return sidecars
-}
-
-func generateRODataColumnSidecars(t testing.TB, sidecars []*ethpb.DataColumnSidecar) []blocks.RODataColumn {
-	roDataColumnSidecars := make([]blocks.RODataColumn, 0, len(sidecars))
-	for _, sidecar := range sidecars {
-		roCol, err := blocks.NewRODataColumn(sidecar)
-		require.NoError(t, err)
-
-		roDataColumnSidecars = append(roDataColumnSidecars, roCol)
-	}
-
-	return roDataColumnSidecars
-}

beacon-chain/core/peerdas/peer_sampling.go

@@ -0,0 +1,56 @@
+package peerdas
+
+import (
+	"math/big"
+
+	"github.com/OffchainLabs/prysm/v6/config/params"
+)
+
+// ExtendedSampleCount computes, for a given number of samples per slot and allowed failures the
+// number of samples we should actually query from peers.
+// https://github.com/ethereum/consensus-specs/blob/v1.5.0-beta.5/specs/fulu/peer-sampling.md#get_extended_sample_count
+func ExtendedSampleCount(samplesPerSlot, allowedFailures uint64) uint64 {
+	// Retrieve the columns count
+	columnsCount := params.BeaconConfig().NumberOfColumns
+
+	// If half of the columns are missing, we are able to reconstruct the data.
+	// If half of the columns + 1 are missing, we are not able to reconstruct the data.
+	// This is the smallest worst case.
+	worstCaseMissing := columnsCount/2 + 1
+
+	// Compute the false positive threshold.
+	falsePositiveThreshold := HypergeomCDF(0, columnsCount, worstCaseMissing, samplesPerSlot)
+
+	var sampleCount uint64
+
+	// Finally, compute the extended sample count.
+	for sampleCount = samplesPerSlot; sampleCount < columnsCount+1; sampleCount++ {
+		if HypergeomCDF(allowedFailures, columnsCount, worstCaseMissing, sampleCount) <= falsePositiveThreshold {
+			break
+		}
+	}
+
+	return sampleCount
+}
+
+// HypergeomCDF computes the hypergeometric cumulative distribution function.
+// https://en.wikipedia.org/wiki/Hypergeometric_distribution
+func HypergeomCDF(k, M, n, N uint64) float64 {
+	denominatorInt := new(big.Int).Binomial(int64(M), int64(N)) // lint:ignore uintcast
+	denominator := new(big.Float).SetInt(denominatorInt)
+
+	rBig := big.NewFloat(0)
+
+	for i := uint64(0); i < k+1; i++ {
+		a := new(big.Int).Binomial(int64(n), int64(i)) // lint:ignore uintcast
+		b := new(big.Int).Binomial(int64(M-n), int64(N-i))
+		numeratorInt := new(big.Int).Mul(a, b)
+		numerator := new(big.Float).SetInt(numeratorInt)
+		item := new(big.Float).Quo(numerator, denominator)
+		rBig.Add(rBig, item)
+	}
+
+	r, _ := rBig.Float64()
+
+	return r
+}

beacon-chain/core/peerdas/peer_sampling_test.go

@@ -0,0 +1,60 @@
+package peerdas_test
+
+import (
+	"testing"
+
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
+	"github.com/OffchainLabs/prysm/v6/testing/require"
+)
+
+func TestExtendedSampleCount(t *testing.T) {
+	const samplesPerSlot = 16
+
+	testCases := []struct {
+		name                string
+		allowedMissings     uint64
+		extendedSampleCount uint64
+	}{
+		{name: "allowedMissings=0", allowedMissings: 0, extendedSampleCount: 16},
+		{name: "allowedMissings=1", allowedMissings: 1, extendedSampleCount: 20},
+		{name: "allowedMissings=2", allowedMissings: 2, extendedSampleCount: 24},
+		{name: "allowedMissings=3", allowedMissings: 3, extendedSampleCount: 27},
+		{name: "allowedMissings=4", allowedMissings: 4, extendedSampleCount: 29},
+		{name: "allowedMissings=5", allowedMissings: 5, extendedSampleCount: 32},
+		{name: "allowedMissings=6", allowedMissings: 6, extendedSampleCount: 35},
+		{name: "allowedMissings=7", allowedMissings: 7, extendedSampleCount: 37},
+		{name: "allowedMissings=8", allowedMissings: 8, extendedSampleCount: 40},
+		{name: "allowedMissings=9", allowedMissings: 9, extendedSampleCount: 42},
+		{name: "allowedMissings=10", allowedMissings: 10, extendedSampleCount: 44},
+		{name: "allowedMissings=11", allowedMissings: 11, extendedSampleCount: 47},
+		{name: "allowedMissings=12", allowedMissings: 12, extendedSampleCount: 49},
+		{name: "allowedMissings=13", allowedMissings: 13, extendedSampleCount: 51},
+		{name: "allowedMissings=14", allowedMissings: 14, extendedSampleCount: 53},
+		{name: "allowedMissings=15", allowedMissings: 15, extendedSampleCount: 55},
+		{name: "allowedMissings=16", allowedMissings: 16, extendedSampleCount: 57},
+		{name: "allowedMissings=17", allowedMissings: 17, extendedSampleCount: 59},
+		{name: "allowedMissings=18", allowedMissings: 18, extendedSampleCount: 61},
+		{name: "allowedMissings=19", allowedMissings: 19, extendedSampleCount: 63},
+		{name: "allowedMissings=20", allowedMissings: 20, extendedSampleCount: 65},
+	}
+
+	for _, tc := range testCases {
+		t.Run(tc.name, func(t *testing.T) {
+			result := peerdas.ExtendedSampleCount(samplesPerSlot, tc.allowedMissings)
+			require.Equal(t, tc.extendedSampleCount, result)
+		})
+	}
+}
+
+func TestHypergeomCDF(t *testing.T) {
+	// Test case from https://en.wikipedia.org/wiki/Hypergeometric_distribution
+	// Population size: 1000, number of successes in population: 500, sample size: 10, number of successes in sample: 5
+	// Expected result: 0.072
+	const (
+		expected = 0.0796665913283742
+		margin   = 0.000001
+	)
+
+	actual := peerdas.HypergeomCDF(5, 128, 65, 16)
+	require.Equal(t, true, expected-margin <= actual && actual <= expected+margin)
+}

beacon-chain/core/peerdas/reconstruction.go

@@ -4,319 +4,144 @@ import (
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg"
 	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
 	"github.com/OffchainLabs/prysm/v6/config/params"
-	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
-	"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
 	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
 	"github.com/pkg/errors"
 	"golang.org/x/sync/errgroup"
 )
 
-var (
-	ErrColumnLengthsDiffer      = errors.New("columns do not have the same length")
-	ErrBlobIndexTooHigh         = errors.New("blob index is too high")
-	ErrBlockRootMismatch        = errors.New("block root mismatch")
-	ErrBlobsCellsProofsMismatch = errors.New("blobs and cells proofs mismatch")
-)
-
-// MinimumColumnsCountToReconstruct return the minimum number of columns needed to proceed to a reconstruction.
-func MinimumColumnsCountToReconstruct() uint64 {
-	// If the number of columns is odd, then we need total / 2 + 1 columns to reconstruct.
-	// If the number of columns is even, then we need total / 2 columns to reconstruct.
-	return (params.BeaconConfig().NumberOfColumns + 1) / 2
+// CanSelfReconstruct returns true if the node can self-reconstruct all the data columns from its custody group count.
+func CanSelfReconstruct(custodyGroupCount uint64) bool {
+	total := params.BeaconConfig().NumberOfCustodyGroups
+	// If total is odd, then we need total / 2 + 1 columns to reconstruct.
+	// If total is even, then we need total / 2 columns to reconstruct.
+	custodyGroupsNeeded := total/2 + total%2
+	return custodyGroupCount >= custodyGroupsNeeded
 }
 
-// ReconstructDataColumnSidecars reconstructs all the data column sidecars from the given input data column sidecars.
-// All input sidecars must be committed to the same block.
-// `inVerifiedRoSidecars` should contain enough (unique) sidecars to reconstruct the missing columns.
-func ReconstructDataColumnSidecars(inVerifiedRoSidecars []blocks.VerifiedRODataColumn) ([]blocks.VerifiedRODataColumn, error) {
-	// Check if there is at least one input sidecar.
-	if len(inVerifiedRoSidecars) == 0 {
-		return nil, ErrNotEnoughDataColumnSidecars
+// RecoverCellsAndProofs recovers the cells and proofs from the data column sidecars.
+func RecoverCellsAndProofs(
+	dataColumnSideCars []*ethpb.DataColumnSidecar,
+	blockRoot [fieldparams.RootLength]byte,
+) ([]kzg.CellsAndProofs, error) {
+	var wg errgroup.Group
+
+	dataColumnSideCarsCount := len(dataColumnSideCars)
+
+	if dataColumnSideCarsCount == 0 {
+		return nil, errors.New("no data column sidecars")
 	}
 
-	// Safely retrieve the first sidecar as a reference.
-	referenceSidecar := inVerifiedRoSidecars[0]
+	// Check if all columns have the same length.
+	blobCount := len(dataColumnSideCars[0].Column)
+	for _, sidecar := range dataColumnSideCars {
+		length := len(sidecar.Column)
 
-	// Check if all columns have the same length and are commmitted to the same block.
-	blobCount := len(referenceSidecar.Column)
-	blockRoot := referenceSidecar.BlockRoot()
-	for _, sidecar := range inVerifiedRoSidecars[1:] {
-		if len(sidecar.Column) != blobCount {
-			return nil, ErrColumnLengthsDiffer
-		}
-
-		if sidecar.BlockRoot() != blockRoot {
-			return nil, ErrBlockRootMismatch
+		if length != blobCount {
+			return nil, errors.New("columns do not have the same length")
 		}
 	}
 
-	// Deduplicate sidecars.
-	sidecarByIndex := make(map[uint64]blocks.VerifiedRODataColumn, len(inVerifiedRoSidecars))
-	for _, inVerifiedRoSidecar := range inVerifiedRoSidecars {
-		sidecarByIndex[inVerifiedRoSidecar.Index] = inVerifiedRoSidecar
-	}
-
-	// Check if there is enough sidecars to reconstruct the missing columns.
-	sidecarCount := len(sidecarByIndex)
-	if uint64(sidecarCount) < MinimumColumnsCountToReconstruct() {
-		return nil, ErrNotEnoughDataColumnSidecars
-	}
-
-	// Sidecars are verified and are committed to the same block.
-	// All signed block headers, KZG commitments, and inclusion proofs are the same.
-	signedBlockHeader := referenceSidecar.SignedBlockHeader
-	kzgCommitments := referenceSidecar.KzgCommitments
-	kzgCommitmentsInclusionProof := referenceSidecar.KzgCommitmentsInclusionProof
-
 	// Recover cells and compute proofs in parallel.
-	var wg errgroup.Group
-	cellsAndProofs := make([]kzg.CellsAndProofs, blobCount)
-	for blobIndex := range uint64(blobCount) {
-		wg.Go(func() error {
-			cellsIndices := make([]uint64, 0, sidecarCount)
-			cells := make([]kzg.Cell, 0, sidecarCount)
+	recoveredCellsAndProofs := make([]kzg.CellsAndProofs, blobCount)
+
+	for blobIndex := 0; blobIndex < blobCount; blobIndex++ {
+		bIndex := blobIndex
+		wg.Go(func() error {
+			cellsIndices := make([]uint64, 0, dataColumnSideCarsCount)
+			cells := make([]kzg.Cell, 0, dataColumnSideCarsCount)
+
+			for _, sidecar := range dataColumnSideCars {
+				// Build the cell indices.
+				cellsIndices = append(cellsIndices, sidecar.Index)
+
+				// Get the cell.
+				column := sidecar.Column
+				cell := column[bIndex]
 
-			for columnIndex, sidecar := range sidecarByIndex {
-				cell := sidecar.Column[blobIndex]
 				cells = append(cells, kzg.Cell(cell))
-				cellsIndices = append(cellsIndices, columnIndex)
 			}
 
 			// Recover the cells and proofs for the corresponding blob
-			cellsAndProofsForBlob, err := kzg.RecoverCellsAndKZGProofs(cellsIndices, cells)
+			cellsAndProofs, err := kzg.RecoverCellsAndKZGProofs(cellsIndices, cells)
 
 			if err != nil {
-				return errors.Wrapf(err, "recover cells and KZG proofs for blob %d", blobIndex)
+				return errors.Wrapf(err, "recover cells and KZG proofs for blob %d", bIndex)
 			}
 
-			// It is safe for multiple goroutines to concurrently write to the same slice,
-			// as long as they are writing to different indices, which is the case here.
-			cellsAndProofs[blobIndex] = cellsAndProofsForBlob
+			recoveredCellsAndProofs[bIndex] = cellsAndProofs
 			return nil
 		})
 	}
 
 	if err := wg.Wait(); err != nil {
-		return nil, errors.Wrap(err, "wait for RecoverCellsAndKZGProofs")
+		return nil, err
 	}
 
-	outSidecars, err := dataColumnsSidecars(signedBlockHeader, kzgCommitments, kzgCommitmentsInclusionProof, cellsAndProofs)
-	if err != nil {
-		return nil, errors.Wrap(err, "data column sidecars from items")
-	}
-
-	// Input sidecars are verified, and we reconstructed ourselves the missing sidecars.
-	// As a consequence, reconstructed sidecars are also verified.
-	outVerifiedRoSidecars := make([]blocks.VerifiedRODataColumn, 0, len(outSidecars))
-	for _, sidecar := range outSidecars {
-		roSidecar, err := blocks.NewRODataColumnWithRoot(sidecar, blockRoot)
-		if err != nil {
-			return nil, errors.Wrap(err, "new RO data column with root")
-		}
-
-		verifiedRoSidecar := blocks.NewVerifiedRODataColumn(roSidecar)
-		outVerifiedRoSidecars = append(outVerifiedRoSidecars, verifiedRoSidecar)
-	}
-
-	return outVerifiedRoSidecars, nil
+	return recoveredCellsAndProofs, nil
 }
 
-// ConstructDataColumnSidecars constructs data column sidecars from a block, (un-extended) blobs and
-// cell proofs corresponding the extended blobs. The main purpose of this function is to
-// construct data columns sidecars from data obtained from the execution client via:
-// - `engine_getBlobsV2` - https://github.com/ethereum/execution-apis/blob/main/src/engine/osaka.md#engine_getblobsv2, or
-// - `engine_getPayloadV5` - https://github.com/ethereum/execution-apis/blob/main/src/engine/osaka.md#engine_getpayloadv5
-// Note: In this function, to stick with the `BlobsBundleV2` format returned by the execution client in `engine_getPayloadV5`,
-// cell proofs are "flattened".
-func ConstructDataColumnSidecars(block interfaces.ReadOnlySignedBeaconBlock, blobs [][]byte, cellProofs [][]byte) ([]*ethpb.DataColumnSidecar, error) {
-	// Check if the cells count is equal to the cell proofs count.
-	numberOfColumns := params.BeaconConfig().NumberOfColumns
-	blobCount := uint64(len(blobs))
-	cellProofsCount := uint64(len(cellProofs))
-
-	cellsCount := blobCount * numberOfColumns
-	if cellsCount != cellProofsCount {
-		return nil, ErrBlobsCellsProofsMismatch
-	}
-
-	cellsAndProofs := make([]kzg.CellsAndProofs, 0, blobCount)
-	for i, blob := range blobs {
-		var kzgBlob kzg.Blob
-		if copy(kzgBlob[:], blob) != len(kzgBlob) {
-			return nil, errors.New("wrong blob size - should never happen")
-		}
-
-		// Compute the extended cells from the (non-extended) blob.
-		cells, err := kzg.ComputeCells(&kzgBlob)
-		if err != nil {
-			return nil, errors.Wrap(err, "compute cells")
-		}
-
-		var proofs []kzg.Proof
-		for idx := uint64(i) * numberOfColumns; idx < (uint64(i)+1)*numberOfColumns; idx++ {
-			var kzgProof kzg.Proof
-			if copy(kzgProof[:], cellProofs[idx]) != len(kzgProof) {
-				return nil, errors.New("wrong KZG proof size - should never happen")
-			}
-
-			proofs = append(proofs, kzgProof)
-		}
-
-		cellsProofs := kzg.CellsAndProofs{Cells: cells, Proofs: proofs}
-		cellsAndProofs = append(cellsAndProofs, cellsProofs)
-	}
-
-	dataColumnSidecars, err := DataColumnSidecars(block, cellsAndProofs)
-	if err != nil {
-		return nil, errors.Wrap(err, "data column sidcars")
-	}
-
-	return dataColumnSidecars, nil
-}
-
-// ReconstructBlobs constructs verified read only blobs sidecars from verified read only blob sidecars.
-// The following constraints must be satisfied:
-//   - All `dataColumnSidecars` has to be committed to the same block, and
-//   - `dataColumnSidecars` must be sorted by index and should not contain duplicates.
-//   - `dataColumnSidecars` must contain either all sidecars corresponding to (non-extended) blobs,
-//     or either enough sidecars to reconstruct the blobs.
-func ReconstructBlobs(block blocks.ROBlock, verifiedDataColumnSidecars []blocks.VerifiedRODataColumn, indices []int) ([]*blocks.VerifiedROBlob, error) {
-	// Return early if no blobs are requested.
-	if len(indices) == 0 {
+// DataColumnSidecarsForReconstruct is a TEMPORARY function until there is an official specification for it.
+// It is scheduled for deletion.
+func DataColumnSidecarsForReconstruct(
+	blobKzgCommitments [][]byte,
+	signedBlockHeader *ethpb.SignedBeaconBlockHeader,
+	kzgCommitmentsInclusionProof [][]byte,
+	cellsAndProofs []kzg.CellsAndProofs,
+) ([]*ethpb.DataColumnSidecar, error) {
+	// Each CellsAndProofs corresponds to a Blob
+	// So we can get the BlobCount by checking the length of CellsAndProofs
+	blobsCount := len(cellsAndProofs)
+	if blobsCount == 0 {
 		return nil, nil
 	}
 
-	if len(verifiedDataColumnSidecars) == 0 {
-		return nil, ErrNotEnoughDataColumnSidecars
-	}
+	// Get the column sidecars.
+	sidecars := make([]*ethpb.DataColumnSidecar, 0, fieldparams.NumberOfColumns)
+	for columnIndex := range fieldparams.NumberOfColumns {
+		column := make([]kzg.Cell, 0, blobsCount)
+		kzgProofOfColumn := make([]kzg.Proof, 0, blobsCount)
 
-	// Check if the sidecars are sorted by index and do not contain duplicates.
-	previousColumnIndex := verifiedDataColumnSidecars[0].Index
-	for _, dataColumnSidecar := range verifiedDataColumnSidecars[1:] {
-		columnIndex := dataColumnSidecar.Index
-		if columnIndex <= previousColumnIndex {
-			return nil, ErrDataColumnSidecarsNotSortedByIndex
-		}
+		for rowIndex := range blobsCount {
+			cellsForRow := cellsAndProofs[rowIndex].Cells
+			proofsForRow := cellsAndProofs[rowIndex].Proofs
 
-		previousColumnIndex = columnIndex
-	}
-
-	// Check if we have enough columns.
-	cellsPerBlob := fieldparams.CellsPerBlob
-	if len(verifiedDataColumnSidecars) < cellsPerBlob {
-		return nil, ErrNotEnoughDataColumnSidecars
-	}
-
-	// Check if the blob index is too high.
-	commitments, err := block.Block().Body().BlobKzgCommitments()
-	if err != nil {
-		return nil, errors.Wrap(err, "blob KZG commitments")
-	}
-
-	for _, blobIndex := range indices {
-		if blobIndex >= len(commitments) {
-			return nil, ErrBlobIndexTooHigh
-		}
-	}
-
-	// Check if the data column sidecars are aligned with the block.
-	dataColumnSidecars := make([]blocks.RODataColumn, 0, len(verifiedDataColumnSidecars))
-	for _, verifiedDataColumnSidecar := range verifiedDataColumnSidecars {
-		dataColumnSicecar := verifiedDataColumnSidecar.RODataColumn
-		dataColumnSidecars = append(dataColumnSidecars, dataColumnSicecar)
-	}
-
-	if err := DataColumnsAlignWithBlock(block, dataColumnSidecars); err != nil {
-		return nil, errors.Wrap(err, "data columns align with block")
-	}
-
-	// If all column sidecars corresponding to (non-extended) blobs are present, no need to reconstruct.
-	if verifiedDataColumnSidecars[cellsPerBlob-1].Index == uint64(cellsPerBlob-1) {
-		// Convert verified data column sidecars to verified blob sidecars.
-		blobSidecars, err := blobSidecarsFromDataColumnSidecars(block, verifiedDataColumnSidecars, indices)
-		if err != nil {
-			return nil, errors.Wrap(err, "blob sidecars from data column sidecars")
-		}
-
-		return blobSidecars, nil
-	}
-
-	// We need to reconstruct the blobs.
-	reconstructedDataColumnSidecars, err := ReconstructDataColumnSidecars(verifiedDataColumnSidecars)
-	if err != nil {
-		return nil, errors.Wrap(err, "reconstruct data column sidecars")
-	}
-
-	// Convert verified data column sidecars to verified blob sidecars.
-	blobSidecars, err := blobSidecarsFromDataColumnSidecars(block, reconstructedDataColumnSidecars, indices)
-	if err != nil {
-		return nil, errors.Wrap(err, "blob sidecars from data column sidecars")
-	}
-
-	return blobSidecars, nil
-}
-
-// blobSidecarsFromDataColumnSidecars converts verified data column sidecars to verified blob sidecars.
-func blobSidecarsFromDataColumnSidecars(roBlock blocks.ROBlock, dataColumnSidecars []blocks.VerifiedRODataColumn, indices []int) ([]*blocks.VerifiedROBlob, error) {
-	referenceSidecar := dataColumnSidecars[0]
-
-	kzgCommitments := referenceSidecar.KzgCommitments
-	signedBlockHeader := referenceSidecar.SignedBlockHeader
-
-	verifiedROBlobs := make([]*blocks.VerifiedROBlob, 0, len(indices))
-	for _, blobIndex := range indices {
-		var blob kzg.Blob
-
-		// Compute the content of the blob.
-		for columnIndex := range fieldparams.CellsPerBlob {
-			dataColumnSidecar := dataColumnSidecars[columnIndex]
-			cell := dataColumnSidecar.Column[blobIndex]
-			if copy(blob[kzg.BytesPerCell*columnIndex:], cell) != kzg.BytesPerCell {
-				return nil, errors.New("wrong cell size - should never happen")
+			if len(cellsForRow) != fieldparams.NumberOfColumns {
+				return nil, errors.Errorf("cells don't have the expected size: expected %d - actual %d", fieldparams.NumberOfColumns, len(cellsForRow))
 			}
+
+			cell := cellsForRow[columnIndex]
+			column = append(column, cell)
+
+			if len(proofsForRow) != fieldparams.NumberOfColumns {
+				return nil, errors.Errorf("proofs don't have the expected size: expected %d - actual %d", fieldparams.NumberOfColumns, len(proofsForRow))
+			}
+
+			kzgProof := proofsForRow[columnIndex]
+			kzgProofOfColumn = append(kzgProofOfColumn, kzgProof)
 		}
 
-		// Extract the KZG commitment.
-		var kzgCommitment kzg.Commitment
-		if copy(kzgCommitment[:], kzgCommitments[blobIndex]) != len(kzgCommitment) {
-			return nil, errors.New("wrong KZG commitment size - should never happen")
+		columnBytes := make([][]byte, 0, blobsCount)
+		for i := range column {
+			columnBytes = append(columnBytes, column[i][:])
 		}
 
-		// Compute the blob KZG proof.
-		blobKzgProof, err := kzg.ComputeBlobKZGProof(&blob, kzgCommitment)
-		if err != nil {
-			return nil, errors.Wrap(err, "compute blob KZG proof")
+		kzgProofOfColumnBytes := make([][]byte, 0, blobsCount)
+		for _, kzgProof := range kzgProofOfColumn {
+			copiedProof := kzgProof
+			kzgProofOfColumnBytes = append(kzgProofOfColumnBytes, copiedProof[:])
 		}
 
-		// Build the inclusion proof for the blob.
-		var kzgBlob kzg.Blob
-		if copy(kzgBlob[:], blob[:]) != len(kzgBlob) {
-			return nil, errors.New("wrong blob size - should never happen")
+		sidecar := &ethpb.DataColumnSidecar{
+			Index:                        uint64(columnIndex),
+			Column:                       columnBytes,
+			KzgCommitments:               blobKzgCommitments,
+			KzgProofs:                    kzgProofOfColumnBytes,
+			SignedBlockHeader:            signedBlockHeader,
+			KzgCommitmentsInclusionProof: kzgCommitmentsInclusionProof,
 		}
 
-		commitmentInclusionProof, err := blocks.MerkleProofKZGCommitment(roBlock.Block().Body(), blobIndex)
-		if err != nil {
-			return nil, errors.Wrap(err, "merkle proof KZG commitment")
-		}
-
-		// Build the blob sidecar.
-		blobSidecar := &ethpb.BlobSidecar{
-			Index:                    uint64(blobIndex),
-			Blob:                     blob[:],
-			KzgCommitment:            kzgCommitment[:],
-			KzgProof:                 blobKzgProof[:],
-			SignedBlockHeader:        signedBlockHeader,
-			CommitmentInclusionProof: commitmentInclusionProof,
-		}
-
-		roBlob, err := blocks.NewROBlob(blobSidecar)
-		if err != nil {
-			return nil, errors.Wrap(err, "new RO blob")
-		}
-
-		verifiedROBlob := blocks.NewVerifiedROBlob(roBlob)
-		verifiedROBlobs = append(verifiedROBlobs, &verifiedROBlob)
+		sidecars = append(sidecars, sidecar)
 	}
 
-	return verifiedROBlobs, nil
+	return sidecars, nil
 }

beacon-chain/core/peerdas/reconstruction_test.go

@@ -1,7 +1,6 @@
 package peerdas_test
 
 import (
-	"encoding/binary"
 	"testing"
 
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg"
@@ -12,30 +11,38 @@ import (
 	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
 	"github.com/OffchainLabs/prysm/v6/testing/require"
 	"github.com/OffchainLabs/prysm/v6/testing/util"
-	"github.com/pkg/errors"
-	"golang.org/x/sync/errgroup"
 )
 
-func TestMinimumColumnsCountToReconstruct(t *testing.T) {
+func TestCanSelfReconstruct(t *testing.T) {
 	testCases := []struct {
-		name            string
-		numberOfColumns uint64
-		expected        uint64
+		name                       string
+		totalNumberOfCustodyGroups uint64
+		custodyNumberOfGroups      uint64
+		expected                   bool
 	}{
 		{
-			name:            "numberOfColumns=128",
-			numberOfColumns: 128,
-			expected:        64,
+			name:                       "totalNumberOfCustodyGroups=64, custodyNumberOfGroups=31",
+			totalNumberOfCustodyGroups: 64,
+			custodyNumberOfGroups:      31,
+			expected:                   false,
 		},
 		{
-			name:            "numberOfColumns=129",
-			numberOfColumns: 129,
-			expected:        65,
+			name:                       "totalNumberOfCustodyGroups=64, custodyNumberOfGroups=32",
+			totalNumberOfCustodyGroups: 64,
+			custodyNumberOfGroups:      32,
+			expected:                   true,
 		},
 		{
-			name:            "numberOfColumns=130",
-			numberOfColumns: 130,
-			expected:        65,
+			name:                       "totalNumberOfCustodyGroups=65, custodyNumberOfGroups=32",
+			totalNumberOfCustodyGroups: 65,
+			custodyNumberOfGroups:      32,
+			expected:                   false,
+		},
+		{
+			name:                       "totalNumberOfCustodyGroups=63, custodyNumberOfGroups=33",
+			totalNumberOfCustodyGroups: 65,
+			custodyNumberOfGroups:      33,
+			expected:                   true,
 		},
 	}
 
@@ -44,278 +51,159 @@ func TestMinimumColumnsCountToReconstruct(t *testing.T) {
 			// Set the total number of columns.
 			params.SetupTestConfigCleanup(t)
 			cfg := params.BeaconConfig().Copy()
-			cfg.NumberOfColumns = tc.numberOfColumns
+			cfg.NumberOfCustodyGroups = tc.totalNumberOfCustodyGroups
 			params.OverrideBeaconConfig(cfg)
 
-			// Compute the minimum number of columns needed to reconstruct.
-			actual := peerdas.MinimumColumnsCountToReconstruct()
+			// Check if reconstuction is possible.
+			actual := peerdas.CanSelfReconstruct(tc.custodyNumberOfGroups)
 			require.Equal(t, tc.expected, actual)
 		})
 	}
 }
 
-func TestReconstructDataColumnSidecars(t *testing.T) {
-	// Start the trusted setup.
-	err := kzg.Start()
-	require.NoError(t, err)
+func TestReconstructionRoundTrip(t *testing.T) {
+	params.SetupTestConfigCleanup(t)
 
-	t.Run("empty input", func(t *testing.T) {
-		_, err := peerdas.ReconstructDataColumnSidecars(nil)
-		require.ErrorIs(t, err, peerdas.ErrNotEnoughDataColumnSidecars)
-	})
+	const blobCount = 5
 
-	t.Run("columns lengths differ", func(t *testing.T) {
-		_, _, verifiedRoSidecars := util.GenerateTestFuluBlockWithSidecars(t, 3)
+	var blockRoot [fieldparams.RootLength]byte
 
-		// Arbitrarily alter the column with index 3
-		verifiedRoSidecars[3].Column = verifiedRoSidecars[3].Column[1:]
+	signedBeaconBlockPb := util.NewBeaconBlockDeneb()
+	require.NoError(t, kzg.Start())
 
-		_, err := peerdas.ReconstructDataColumnSidecars(verifiedRoSidecars)
-		require.ErrorIs(t, err, peerdas.ErrColumnLengthsDiffer)
-	})
-
-	t.Run("roots differ", func(t *testing.T) {
-		_, _, verifiedRoSidecars := util.GenerateTestFuluBlockWithSidecars(t, 3, util.WithParentRoot([fieldparams.RootLength]byte{1}))
-		_, _, verifiedRoSidecarsAlter := util.GenerateTestFuluBlockWithSidecars(t, 3, util.WithParentRoot([fieldparams.RootLength]byte{2}))
-
-		// Arbitrarily alter the column with index 3
-		verifiedRoSidecars[3] = verifiedRoSidecarsAlter[3]
-		_, err := peerdas.ReconstructDataColumnSidecars(verifiedRoSidecars)
-		require.ErrorIs(t, err, peerdas.ErrBlockRootMismatch)
-	})
-
-	const blobCount = 6
-	signedBeaconBlockPb := util.NewBeaconBlockFulu()
-	block := signedBeaconBlockPb.Block
-
-	commitments := make([][]byte, 0, blobCount)
-	for i := range uint64(blobCount) {
-		var commitment [fieldparams.KzgCommitmentSize]byte
-		binary.BigEndian.PutUint64(commitment[:], i)
-		commitments = append(commitments, commitment[:])
-	}
-
-	block.Body.BlobKzgCommitments = commitments
-
-	t.Run("not enough columns to enable reconstruction", func(t *testing.T) {
-		_, _, verifiedRoSidecars := util.GenerateTestFuluBlockWithSidecars(t, 3)
-
-		minimum := peerdas.MinimumColumnsCountToReconstruct()
-		_, err := peerdas.ReconstructDataColumnSidecars(verifiedRoSidecars[:minimum-1])
-		require.ErrorIs(t, err, peerdas.ErrNotEnoughDataColumnSidecars)
-	})
-
-	t.Run("nominal", func(t *testing.T) {
-		// Build a full set of verified data column sidecars.
-		_, _, inputVerifiedRoSidecars := util.GenerateTestFuluBlockWithSidecars(t, 3)
-
-		// Arbitrarily keep only the even sicars.
-		filteredVerifiedRoSidecars := make([]blocks.VerifiedRODataColumn, 0, len(inputVerifiedRoSidecars)/2)
-		for i := 0; i < len(inputVerifiedRoSidecars); i += 2 {
-			filteredVerifiedRoSidecars = append(filteredVerifiedRoSidecars, inputVerifiedRoSidecars[i])
-		}
-
-		// Reconstruct the data column sidecars.
-		reconstructedVerifiedRoSidecars, err := peerdas.ReconstructDataColumnSidecars(filteredVerifiedRoSidecars)
+	// Generate random blobs and their corresponding commitments.
+	var (
+		blobsKzgCommitments [][]byte
+		blobs               []kzg.Blob
+	)
+	for i := range blobCount {
+		blob := getRandBlob(int64(i))
+		commitment, _, err := generateCommitmentAndProof(&blob)
 		require.NoError(t, err)
 
-		// Verify that the reconstructed sidecars are equal to the original ones.
-		require.DeepSSZEqual(t, inputVerifiedRoSidecars, reconstructedVerifiedRoSidecars)
-	})
-}
+		blobsKzgCommitments = append(blobsKzgCommitments, commitment[:])
+		blobs = append(blobs, blob)
+	}
 
-func TestConstructDataColumnSidecars(t *testing.T) {
-	const (
-		blobCount    = 3
-		cellsPerBlob = fieldparams.CellsPerBlob
-	)
+	// Generate a signed beacon block.
+	signedBeaconBlockPb.Block.Body.BlobKzgCommitments = blobsKzgCommitments
+	signedBeaconBlock, err := blocks.NewSignedBeaconBlock(signedBeaconBlockPb)
+	require.NoError(t, err)
+
+	// Get the signed beacon block header.
+	signedBeaconBlockHeader, err := signedBeaconBlock.Header()
+	require.NoError(t, err)
+
+	// Convert data columns sidecars from signed block and blobs.
+	cellsAndProofs := util.GenerateCellsAndProofs(t, blobs)
+	dataColumnSidecars, err := peerdas.DataColumnSidecars(signedBeaconBlock, cellsAndProofs)
+	require.NoError(t, err)
+
+	// Create verified RO data columns.
+	verifiedRoDataColumns := make([]*blocks.VerifiedRODataColumn, 0, blobCount)
+	for _, dataColumnSidecar := range dataColumnSidecars {
+		roDataColumn, err := blocks.NewRODataColumn(dataColumnSidecar)
+		require.NoError(t, err)
+
+		verifiedRoDataColumn := blocks.NewVerifiedRODataColumn(roDataColumn)
+		verifiedRoDataColumns = append(verifiedRoDataColumns, &verifiedRoDataColumn)
+	}
+
+	verifiedRoDataColumn := verifiedRoDataColumns[0]
 
 	numberOfColumns := params.BeaconConfig().NumberOfColumns
 
-	// Start the trusted setup.
-	err := kzg.Start()
-	require.NoError(t, err)
+	var noDataColumns []*ethpb.DataColumnSidecar
+	dataColumnsWithDifferentLengths := []*ethpb.DataColumnSidecar{
+		{Column: [][]byte{{}, {}}},
+		{Column: [][]byte{{}}},
+	}
+	notEnoughDataColumns := dataColumnSidecars[:numberOfColumns/2-1]
+	originalDataColumns := dataColumnSidecars[:numberOfColumns/2]
+	extendedDataColumns := dataColumnSidecars[numberOfColumns/2:]
+	evenDataColumns := make([]*ethpb.DataColumnSidecar, 0, numberOfColumns/2)
+	oddDataColumns := make([]*ethpb.DataColumnSidecar, 0, numberOfColumns/2)
+	allDataColumns := dataColumnSidecars
 
-	roBlock, _, baseVerifiedRoSidecars := util.GenerateTestFuluBlockWithSidecars(t, blobCount)
-
-	// Extract blobs and proofs from the sidecars.
-	blobs := make([][]byte, 0, blobCount)
-	cellProofs := make([][]byte, 0, cellsPerBlob)
-	for blobIndex := range blobCount {
-		blob := make([]byte, 0, cellsPerBlob)
-		for columnIndex := range cellsPerBlob {
-			cell := baseVerifiedRoSidecars[columnIndex].Column[blobIndex]
-			blob = append(blob, cell...)
-		}
-
-		blobs = append(blobs, blob)
-
-		for columnIndex := range numberOfColumns {
-			cellProof := baseVerifiedRoSidecars[columnIndex].KzgProofs[blobIndex]
-			cellProofs = append(cellProofs, cellProof)
+	for i, dataColumn := range dataColumnSidecars {
+		if i%2 == 0 {
+			evenDataColumns = append(evenDataColumns, dataColumn)
+		} else {
+			oddDataColumns = append(oddDataColumns, dataColumn)
 		}
 	}
 
-	actual, err := peerdas.ConstructDataColumnSidecars(roBlock, blobs, cellProofs)
-	require.NoError(t, err)
-
-	// Extract the base verified ro sidecars into sidecars.
-	expected := make([]*ethpb.DataColumnSidecar, 0, len(baseVerifiedRoSidecars))
-	for _, verifiedRoSidecar := range baseVerifiedRoSidecars {
-		expected = append(expected, verifiedRoSidecar.DataColumnSidecar)
+	testCases := []struct {
+		name               string
+		dataColumnsSidecar []*ethpb.DataColumnSidecar
+		isError            bool
+	}{
+		{
+			name:               "No data columns sidecars",
+			dataColumnsSidecar: noDataColumns,
+			isError:            true,
+		},
+		{
+			name:               "Data columns sidecar with different lengths",
+			dataColumnsSidecar: dataColumnsWithDifferentLengths,
+			isError:            true,
+		},
+		{
+			name:               "All columns are present (no actual need to reconstruct)",
+			dataColumnsSidecar: allDataColumns,
+			isError:            false,
+		},
+		{
+			name:               "Only original columns are present",
+			dataColumnsSidecar: originalDataColumns,
+			isError:            false,
+		},
+		{
+			name:               "Only extended columns are present",
+			dataColumnsSidecar: extendedDataColumns,
+			isError:            false,
+		},
+		{
+			name:               "Only even columns are present",
+			dataColumnsSidecar: evenDataColumns,
+			isError:            false,
+		},
+		{
+			name:               "Only odd columns are present",
+			dataColumnsSidecar: oddDataColumns,
+			isError:            false,
+		},
+		{
+			name:               "Not enough columns to reconstruct",
+			dataColumnsSidecar: notEnoughDataColumns,
+			isError:            true,
+		},
 	}
 
-	require.DeepSSZEqual(t, expected, actual)
-}
+	for _, tc := range testCases {
+		t.Run(tc.name, func(t *testing.T) {
+			// Recover cells and proofs from available data columns sidecars.
+			cellsAndProofs, err := peerdas.RecoverCellsAndProofs(tc.dataColumnsSidecar, blockRoot)
+			isError := (err != nil)
+			require.Equal(t, tc.isError, isError)
 
-func TestReconstructBlobs(t *testing.T) {
-	// Start the trusted setup.
-	err := kzg.Start()
-	require.NoError(t, err)
-
-	var emptyBlock blocks.ROBlock
-
-	t.Run("no index", func(t *testing.T) {
-		actual, err := peerdas.ReconstructBlobs(emptyBlock, nil, nil)
-		require.NoError(t, err)
-		require.IsNil(t, actual)
-	})
-
-	t.Run("empty input", func(t *testing.T) {
-		_, err := peerdas.ReconstructBlobs(emptyBlock, nil, []int{0})
-		require.ErrorIs(t, err, peerdas.ErrNotEnoughDataColumnSidecars)
-	})
-
-	t.Run("not sorted", func(t *testing.T) {
-		_, _, verifiedRoSidecars := util.GenerateTestFuluBlockWithSidecars(t, 3)
-
-		// Arbitrarily change the order of the sidecars.
-		verifiedRoSidecars[3], verifiedRoSidecars[2] = verifiedRoSidecars[2], verifiedRoSidecars[3]
-
-		_, err := peerdas.ReconstructBlobs(emptyBlock, verifiedRoSidecars, []int{0})
-		require.ErrorIs(t, err, peerdas.ErrDataColumnSidecarsNotSortedByIndex)
-	})
-
-	t.Run("not enough columns", func(t *testing.T) {
-		_, _, verifiedRoSidecars := util.GenerateTestFuluBlockWithSidecars(t, 3)
-
-		inputSidecars := verifiedRoSidecars[:fieldparams.CellsPerBlob-1]
-		_, err := peerdas.ReconstructBlobs(emptyBlock, inputSidecars, []int{0})
-		require.ErrorIs(t, err, peerdas.ErrNotEnoughDataColumnSidecars)
-	})
-
-	t.Run("index too high", func(t *testing.T) {
-		const blobCount = 3
-
-		roBlock, _, verifiedRoSidecars := util.GenerateTestFuluBlockWithSidecars(t, blobCount)
-
-		_, err := peerdas.ReconstructBlobs(roBlock, verifiedRoSidecars, []int{1, blobCount})
-		require.ErrorIs(t, err, peerdas.ErrBlobIndexTooHigh)
-	})
-
-	t.Run("not committed to the same block", func(t *testing.T) {
-		_, _, verifiedRoSidecars := util.GenerateTestFuluBlockWithSidecars(t, 3, util.WithParentRoot([fieldparams.RootLength]byte{1}))
-		roBlock, _, _ := util.GenerateTestFuluBlockWithSidecars(t, 3, util.WithParentRoot([fieldparams.RootLength]byte{2}))
-
-		_, err = peerdas.ReconstructBlobs(roBlock, verifiedRoSidecars, []int{0})
-		require.ErrorContains(t, peerdas.ErrRootMismatch.Error(), err)
-	})
-
-	t.Run("nominal", func(t *testing.T) {
-		const blobCount = 3
-		numberOfColumns := params.BeaconConfig().NumberOfColumns
-
-		roBlock, roBlobSidecars := util.GenerateTestElectraBlockWithSidecar(t, [fieldparams.RootLength]byte{}, 42, blobCount)
-
-		// Compute cells and proofs from blob sidecars.
-		var wg errgroup.Group
-		blobs := make([][]byte, blobCount)
-		cellsAndProofs := make([]kzg.CellsAndProofs, blobCount)
-		for i := range blobCount {
-			blob := roBlobSidecars[i].Blob
-			blobs[i] = blob
-
-			wg.Go(func() error {
-				var kzgBlob kzg.Blob
-				count := copy(kzgBlob[:], blob)
-				require.Equal(t, len(kzgBlob), count)
-
-				cp, err := kzg.ComputeCellsAndKZGProofs(&kzgBlob)
-				if err != nil {
-					return errors.Wrapf(err, "compute cells and kzg proofs for blob %d", i)
-				}
-
-				// It is safe for multiple goroutines to concurrently write to the same slice,
-				// as long as they are writing to different indices, which is the case here.
-				cellsAndProofs[i] = cp
-
-				return nil
-			})
-		}
-
-		err := wg.Wait()
-		require.NoError(t, err)
-
-		// Flatten proofs.
-		cellProofs := make([][]byte, 0, blobCount*numberOfColumns)
-		for _, cp := range cellsAndProofs {
-			for _, proof := range cp.Proofs {
-				cellProofs = append(cellProofs, proof[:])
+			if isError {
+				return
 			}
-		}
 
-		// Construct data column sidecars.
-		// It is OK to use the public function `ConstructDataColumnSidecars`, as long as
-		// `TestConstructDataColumnSidecars` tests pass.
-		dataColumnSidecars, err := peerdas.ConstructDataColumnSidecars(roBlock, blobs, cellProofs)
-		require.NoError(t, err)
+			// Recover all data columns sidecars from cells and proofs.
+			reconstructedDataColumnsSideCars, err := peerdas.DataColumnSidecarsForReconstruct(
+				blobsKzgCommitments,
+				signedBeaconBlockHeader,
+				verifiedRoDataColumn.KzgCommitmentsInclusionProof,
+				cellsAndProofs,
+			)
 
-		// Convert to verified data column sidecars.
-		verifiedRoSidecars := make([]blocks.VerifiedRODataColumn, 0, len(dataColumnSidecars))
-		for _, dataColumnSidecar := range dataColumnSidecars {
-			roSidecar, err := blocks.NewRODataColumn(dataColumnSidecar)
 			require.NoError(t, err)
 
-			verifiedRoSidecar := blocks.NewVerifiedRODataColumn(roSidecar)
-			verifiedRoSidecars = append(verifiedRoSidecars, verifiedRoSidecar)
-		}
-
-		indices := []int{2, 0}
-
-		t.Run("no reconstruction needed", func(t *testing.T) {
-			// Reconstruct blobs.
-			reconstructedVerifiedRoBlobSidecars, err := peerdas.ReconstructBlobs(roBlock, verifiedRoSidecars, indices)
-			require.NoError(t, err)
-
-			// Compare blobs.
-			for i, blobIndex := range indices {
-				expected := roBlobSidecars[blobIndex]
-				actual := reconstructedVerifiedRoBlobSidecars[i].ROBlob
-
-				require.DeepSSZEqual(t, expected, actual)
-			}
+			expected := dataColumnSidecars
+			actual := reconstructedDataColumnsSideCars
+			require.DeepSSZEqual(t, expected, actual)
 		})
-
-		t.Run("reconstruction needed", func(t *testing.T) {
-			// Arbitrarily keep only the even sidecars.
-			filteredSidecars := make([]blocks.VerifiedRODataColumn, 0, len(verifiedRoSidecars)/2)
-			for i := 0; i < len(verifiedRoSidecars); i += 2 {
-				filteredSidecars = append(filteredSidecars, verifiedRoSidecars[i])
-			}
-
-			// Reconstruct blobs.
-			reconstructedVerifiedRoBlobSidecars, err := peerdas.ReconstructBlobs(roBlock, filteredSidecars, indices)
-			require.NoError(t, err)
-
-			// Compare blobs.
-			for i, blobIndex := range indices {
-				expected := roBlobSidecars[blobIndex]
-				actual := reconstructedVerifiedRoBlobSidecars[i].ROBlob
-
-				require.DeepSSZEqual(t, expected, actual)
-			}
-		})
-
-	})
-
+	}
 }

beacon-chain/core/peerdas/util.go

@@ -0,0 +1,57 @@
+package peerdas
+
+import (
+	"fmt"
+
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg"
+	"github.com/OffchainLabs/prysm/v6/config/params"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
+	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
+	"github.com/OffchainLabs/prysm/v6/runtime/version"
+)
+
+// ConstructDataColumnSidecars constructs data column sidecars from a block, blobs and their cell proofs.
+// This is a convenience method as blob and cell proofs are common inputs.
+func ConstructDataColumnSidecars(block interfaces.SignedBeaconBlock, blobs [][]byte, cellProofs [][]byte) ([]*ethpb.DataColumnSidecar, error) {
+	// Check if the block is at least a Fulu block.
+	if block.Version() < version.Fulu {
+		return nil, nil
+	}
+
+	cellsAndProofs, err := constructCellsAndProofs(blobs, cellProofs)
+	if err != nil {
+		return nil, err
+	}
+
+	return DataColumnSidecars(block, cellsAndProofs)
+}
+
+func constructCellsAndProofs(blobs [][]byte, cellProofs [][]byte) ([]kzg.CellsAndProofs, error) {
+	numColumns := int(params.BeaconConfig().NumberOfColumns)
+	if len(blobs)*numColumns != len(cellProofs) {
+		return nil, fmt.Errorf("number of blobs and cell proofs do not match: %d * %d != %d", len(blobs), numColumns, len(cellProofs))
+	}
+
+	cellsAndProofs := make([]kzg.CellsAndProofs, 0, len(blobs))
+
+	for i, blob := range blobs {
+		var b kzg.Blob
+		copy(b[:], blob)
+		cells, err := kzg.ComputeCells(&b)
+		if err != nil {
+			return nil, err
+		}
+
+		var proofs []kzg.Proof
+		for idx := i * numColumns; idx < (i+1)*numColumns; idx++ {
+			proofs = append(proofs, kzg.Proof(cellProofs[idx]))
+		}
+
+		cellsAndProofs = append(cellsAndProofs, kzg.CellsAndProofs{
+			Cells:  cells,
+			Proofs: proofs,
+		})
+	}
+
+	return cellsAndProofs, nil
+}

beacon-chain/core/peerdas/utils_test.go

@@ -8,30 +8,18 @@ import (
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg"
 	"github.com/consensys/gnark-crypto/ecc/bls12-381/fr"
 	GoKZG "github.com/crate-crypto/go-kzg-4844"
-	"github.com/pkg/errors"
 	"github.com/sirupsen/logrus"
 )
 
-func generateCommitment(blob *kzg.Blob) (*kzg.Commitment, error) {
-	commitment, err := kzg.BlobToKZGCommitment(blob)
-	if err != nil {
-		return nil, errors.Wrap(err, "blob to kzg commitment")
-	}
-
-	return &commitment, nil
-}
-
 func generateCommitmentAndProof(blob *kzg.Blob) (*kzg.Commitment, *kzg.Proof, error) {
 	commitment, err := kzg.BlobToKZGCommitment(blob)
 	if err != nil {
 		return nil, nil, err
 	}
-
 	proof, err := kzg.ComputeBlobKZGProof(blob, commitment)
 	if err != nil {
 		return nil, nil, err
 	}
-
 	return &commitment, &proof, err
 }
 

beacon-chain/core/peerdas/validator.go

@@ -12,12 +12,12 @@ import (
 func ValidatorsCustodyRequirement(state beaconState.ReadOnlyBeaconState, validatorsIndex map[primitives.ValidatorIndex]bool) (uint64, error) {
 	totalNodeBalance := uint64(0)
 	for index := range validatorsIndex {
-		validator, err := state.ValidatorAtIndexReadOnly(index)
+		balance, err := state.BalanceAtIndex(index)
 		if err != nil {
-			return 0, errors.Wrapf(err, "validator at index %v", index)
+			return 0, errors.Wrapf(err, "balance at index for validator index %v", index)
 		}
 
-		totalNodeBalance += validator.EffectiveBalance()
+		totalNodeBalance += balance
 	}
 
 	beaconConfig := params.BeaconConfig()

beacon-chain/core/peerdas/validator_test.go

@@ -30,13 +30,9 @@ func TestValidatorsCustodyRequirement(t *testing.T) {
 
 	for _, tc := range testCases {
 		t.Run(tc.name, func(t *testing.T) {
-			validators := make([]*ethpb.Validator, 0, tc.count)
+			balances := make([]uint64, 0, tc.count)
 			for range tc.count {
-				validator := &ethpb.Validator{
-					EffectiveBalance: balance,
-				}
-
-				validators = append(validators, validator)
+				balances = append(balances, balance)
 			}
 
 			validatorsIndex := make(map[primitives.ValidatorIndex]bool)
@@ -44,7 +40,7 @@ func TestValidatorsCustodyRequirement(t *testing.T) {
 				validatorsIndex[primitives.ValidatorIndex(i)] = true
 			}
 
-			beaconState, err := state_native.InitializeFromProtoFulu(&ethpb.BeaconStateElectra{Validators: validators})
+			beaconState, err := state_native.InitializeFromProtoFulu(&ethpb.BeaconStateElectra{Balances: balances})
 			require.NoError(t, err)
 
 			actual, err := peerdas.ValidatorsCustodyRequirement(beaconState, validatorsIndex)

beacon-chain/core/peerdas/verification_test.go

@@ -1,77 +0,0 @@
-package peerdas_test
-
-import (
-	"testing"
-
-	"github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg"
-	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
-	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
-	"github.com/OffchainLabs/prysm/v6/config/params"
-	"github.com/OffchainLabs/prysm/v6/testing/require"
-	"github.com/OffchainLabs/prysm/v6/testing/util"
-)
-
-func TestDataColumnsAlignWithBlock(t *testing.T) {
-	// Start the trusted setup.
-	err := kzg.Start()
-	require.NoError(t, err)
-
-	t.Run("pre fulu", func(t *testing.T) {
-		block, _ := util.GenerateTestElectraBlockWithSidecar(t, [fieldparams.RootLength]byte{}, 0, 0)
-		err := peerdas.DataColumnsAlignWithBlock(block, nil)
-		require.NoError(t, err)
-	})
-
-	t.Run("too many commitmnets", func(t *testing.T) {
-		params.SetupTestConfigCleanup(t)
-		config := params.BeaconConfig()
-		config.BlobSchedule = []params.BlobScheduleEntry{{}}
-		params.OverrideBeaconConfig(config)
-
-		block, _, _ := util.GenerateTestFuluBlockWithSidecars(t, 3)
-		err := peerdas.DataColumnsAlignWithBlock(block, nil)
-		require.ErrorIs(t, err, peerdas.ErrTooManyCommitments)
-	})
-
-	t.Run("root mismatch", func(t *testing.T) {
-		_, sidecars, _ := util.GenerateTestFuluBlockWithSidecars(t, 2)
-		block, _, _ := util.GenerateTestFuluBlockWithSidecars(t, 0)
-		err := peerdas.DataColumnsAlignWithBlock(block, sidecars)
-		require.ErrorIs(t, err, peerdas.ErrRootMismatch)
-	})
-
-	t.Run("column size mismatch", func(t *testing.T) {
-		block, sidecars, _ := util.GenerateTestFuluBlockWithSidecars(t, 2)
-		sidecars[0].Column = [][]byte{}
-		err := peerdas.DataColumnsAlignWithBlock(block, sidecars)
-		require.ErrorIs(t, err, peerdas.ErrBlockColumnSizeMismatch)
-	})
-
-	t.Run("KZG commitments size mismatch", func(t *testing.T) {
-		block, sidecars, _ := util.GenerateTestFuluBlockWithSidecars(t, 2)
-		sidecars[0].KzgCommitments = [][]byte{}
-		err := peerdas.DataColumnsAlignWithBlock(block, sidecars)
-		require.ErrorIs(t, err, peerdas.ErrBlockColumnSizeMismatch)
-	})
-
-	t.Run("KZG proofs mismatch", func(t *testing.T) {
-		block, sidecars, _ := util.GenerateTestFuluBlockWithSidecars(t, 2)
-		sidecars[0].KzgProofs = [][]byte{}
-		err := peerdas.DataColumnsAlignWithBlock(block, sidecars)
-		require.ErrorIs(t, err, peerdas.ErrBlockColumnSizeMismatch)
-	})
-
-	t.Run("commitment mismatch", func(t *testing.T) {
-		block, _, _ := util.GenerateTestFuluBlockWithSidecars(t, 2)
-		_, alteredSidecars, _ := util.GenerateTestFuluBlockWithSidecars(t, 2)
-		alteredSidecars[1].KzgCommitments[0][0]++ // Overflow is OK
-		err := peerdas.DataColumnsAlignWithBlock(block, alteredSidecars)
-		require.ErrorIs(t, err, peerdas.ErrCommitmentMismatch)
-	})
-
-	t.Run("nominal", func(t *testing.T) {
-		block, sidecars, _ := util.GenerateTestFuluBlockWithSidecars(t, 2)
-		err := peerdas.DataColumnsAlignWithBlock(block, sidecars)
-		require.NoError(t, err)
-	})
-}

beacon-chain/core/time/slot_epoch.go

@@ -53,6 +53,11 @@ func HigherEqualThanAltairVersionAndEpoch(s state.BeaconState, e primitives.Epoc
 	return s.Version() >= version.Altair && e >= params.BeaconConfig().AltairForkEpoch
 }
 
+// PeerDASIsActive checks whether peerDAS is active at the provided slot.
+func PeerDASIsActive(slot primitives.Slot) bool {
+	return params.FuluEnabled() && slots.ToEpoch(slot) >= params.BeaconConfig().FuluForkEpoch
+}
+
 // CanUpgradeToAltair returns true if the input `slot` can upgrade to Altair.
 // Spec code:
 // If state.slot % SLOTS_PER_EPOCH == 0 and compute_epoch_at_slot(state.slot) == ALTAIR_FORK_EPOCH

beacon-chain/core/transition/BUILD.bazel

@@ -46,7 +46,6 @@ go_library(
         "//proto/engine/v1:go_default_library",
         "//proto/prysm/v1alpha1:go_default_library",
         "//runtime/version:go_default_library",
-        "//time/slots:go_default_library",
         "@com_github_pkg_errors//:go_default_library",
         "@com_github_prometheus_client_golang//prometheus:go_default_library",
         "@com_github_prometheus_client_golang//prometheus/promauto:go_default_library",

beacon-chain/core/transition/transition.go

@@ -27,9 +27,7 @@ import (
 	"github.com/OffchainLabs/prysm/v6/monitoring/tracing"
 	prysmTrace "github.com/OffchainLabs/prysm/v6/monitoring/tracing/trace"
 	"github.com/OffchainLabs/prysm/v6/runtime/version"
-	"github.com/OffchainLabs/prysm/v6/time/slots"
 	"github.com/pkg/errors"
-	"github.com/sirupsen/logrus"
 	"go.opentelemetry.io/otel/trace"
 )
 
@@ -293,8 +291,6 @@ func ProcessSlotsCore(ctx context.Context, span trace.Span, state state.BeaconSt
 			tracing.AnnotateError(span, err)
 			return nil, errors.Wrap(err, "failed to upgrade state")
 		}
-
-		logBlobLimitIncrease(state.Slot())
 	}
 	return state, nil
 }
@@ -511,19 +507,3 @@ func ProcessEpochPrecompute(ctx context.Context, state state.BeaconState) (state
 	}
 	return state, nil
 }
-
-func logBlobLimitIncrease(slot primitives.Slot) {
-	if !slots.IsEpochStart(slot) {
-		return
-	}
-
-	epoch := slots.ToEpoch(slot)
-	for _, entry := range params.BeaconConfig().BlobSchedule {
-		if entry.Epoch == epoch {
-			log.WithFields(logrus.Fields{
-				"epoch":     epoch,
-				"blobLimit": entry.MaxBlobsPerBlock,
-			}).Info("Blob limit updated")
-		}
-	}
-}

beacon-chain/das/BUILD.bazel

@@ -4,21 +4,26 @@ go_library(
     name = "go_default_library",
     srcs = [
         "availability.go",
+        "availability_columns.go",
         "cache.go",
+        "data_column_cache.go",
         "iface.go",
         "mock.go",
     ],
     importpath = "github.com/OffchainLabs/prysm/v6/beacon-chain/das",
     visibility = ["//visibility:public"],
     deps = [
+        "//beacon-chain/core/peerdas:go_default_library",
         "//beacon-chain/db/filesystem:go_default_library",
         "//beacon-chain/verification:go_default_library",
+        "//config/fieldparams:go_default_library",
         "//config/params:go_default_library",
         "//consensus-types/blocks:go_default_library",
         "//consensus-types/primitives:go_default_library",
         "//runtime/logging:go_default_library",
         "//runtime/version:go_default_library",
         "//time/slots:go_default_library",
+        "@com_github_ethereum_go_ethereum//p2p/enode:go_default_library",
         "@com_github_pkg_errors//:go_default_library",
         "@com_github_sirupsen_logrus//:go_default_library",
     ],
@@ -27,13 +32,18 @@ go_library(
 go_test(
     name = "go_default_test",
     srcs = [
+        "availability_columns_test.go",
         "availability_test.go",
         "cache_test.go",
+        "data_column_cache_test.go",
     ],
     embed = [":go_default_library"],
     deps = [
+        "//beacon-chain/core/peerdas:go_default_library",
         "//beacon-chain/db/filesystem:go_default_library",
         "//beacon-chain/verification:go_default_library",
+        "//cmd/beacon-chain/flags:go_default_library",
+        "//config/fieldparams:go_default_library",
         "//config/params:go_default_library",
         "//consensus-types/blocks:go_default_library",
         "//consensus-types/primitives:go_default_library",
@@ -41,6 +51,7 @@ go_test(
         "//testing/require:go_default_library",
         "//testing/util:go_default_library",
         "//time/slots:go_default_library",
+        "@com_github_ethereum_go_ethereum//p2p/enode:go_default_library",
         "@com_github_pkg_errors//:go_default_library",
     ],
 )

beacon-chain/das/availability.go

@@ -53,10 +53,16 @@ func NewLazilyPersistentStore(store *filesystem.BlobStorage, verifier BlobBatchV
 // Persist adds blobs to the working blob cache. Blobs stored in this cache will be persisted
 // for at least as long as the node is running. Once IsDataAvailable succeeds, all blobs referenced
 // by the given block are guaranteed to be persisted for the remainder of the retention period.
-func (s *LazilyPersistentStore) Persist(current primitives.Slot, sc ...blocks.ROBlob) error {
-	if len(sc) == 0 {
+func (s *LazilyPersistentStore) Persist(current primitives.Slot, scg ...blocks.ROSidecar) error {
+	if len(scg) == 0 {
 		return nil
 	}
+
+	sc, err := blocks.BlobSidecarsFromSidecars(scg)
+	if err != nil {
+		return errors.Wrap(err, "blob sidecars from sidecars")
+	}
+
 	if len(sc) > 1 {
 		first := sc[0].BlockRoot()
 		for i := 1; i < len(sc); i++ {

beacon-chain/das/availability_columns.go

@@ -0,0 +1,200 @@
+package das
+
+import (
+	"context"
+
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/db/filesystem"
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/verification"
+	"github.com/OffchainLabs/prysm/v6/config/params"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
+	"github.com/OffchainLabs/prysm/v6/runtime/version"
+	"github.com/OffchainLabs/prysm/v6/time/slots"
+	"github.com/ethereum/go-ethereum/p2p/enode"
+	errors "github.com/pkg/errors"
+)
+
+// LazilyPersistentStoreColumn is an implementation of AvailabilityStore to be used when batch syncing data columns.
+// This implementation will hold any data columns passed to Persist until the IsDataAvailable is called for their
+// block, at which time they will undergo full verification and be saved to the disk.
+type LazilyPersistentStoreColumn struct {
+	store                  *filesystem.DataColumnStorage
+	nodeID                 enode.ID
+	cache                  *dataColumnCache
+	custodyInfo            *peerdas.CustodyInfo
+	newDataColumnsVerifier verification.NewDataColumnsVerifier
+}
+
+var _ AvailabilityStore = &LazilyPersistentStoreColumn{}
+
+// DataColumnsVerifier enables LazilyPersistentStoreColumn to manage the verification process
+// going from RODataColumn->VerifiedRODataColumn, while avoiding the decision of which individual verifications
+// to run and in what order. Since LazilyPersistentStoreColumn always tries to verify and save data columns only when
+// they are all available, the interface takes a slice of data column sidecars.
+type DataColumnsVerifier interface {
+	VerifiedRODataColumns(ctx context.Context, blk blocks.ROBlock, scs []blocks.RODataColumn) ([]blocks.VerifiedRODataColumn, error)
+}
+
+func NewLazilyPersistentStoreColumn(store *filesystem.DataColumnStorage, nodeID enode.ID, newDataColumnsVerifier verification.NewDataColumnsVerifier, custodyInfo *peerdas.CustodyInfo) *LazilyPersistentStoreColumn {
+	return &LazilyPersistentStoreColumn{
+		store:                  store,
+		nodeID:                 nodeID,
+		cache:                  newDataColumnCache(),
+		custodyInfo:            custodyInfo,
+		newDataColumnsVerifier: newDataColumnsVerifier,
+	}
+}
+
+// PersistColumns adds columns to the working column cache. Columns stored in this cache will be persisted
+// for at least as long as the node is running. Once IsDataAvailable succeeds, all columns referenced
+// by the given block are guaranteed to be persisted for the remainder of the retention period.
+func (s *LazilyPersistentStoreColumn) Persist(current primitives.Slot, sidecars ...blocks.ROSidecar) error {
+	if len(sidecars) == 0 {
+		return nil
+	}
+
+	dataColumnSidecars, err := blocks.DataColumnSidecarsFromSidecars(sidecars)
+	if err != nil {
+		return errors.Wrap(err, "blob sidecars from sidecars")
+	}
+
+	// It is safe to retrieve the first sidecar.
+	firstSidecar := dataColumnSidecars[0]
+
+	if len(sidecars) > 1 {
+		firstRoot := firstSidecar.BlockRoot()
+		for _, sidecar := range dataColumnSidecars[1:] {
+			if sidecar.BlockRoot() != firstRoot {
+				return errMixedRoots
+			}
+		}
+	}
+
+	firstSidecarEpoch, currentEpoch := slots.ToEpoch(firstSidecar.Slot()), slots.ToEpoch(current)
+	if !params.WithinDAPeriod(firstSidecarEpoch, currentEpoch) {
+		return nil
+	}
+
+	key := dataColumnCacheKey{slot: firstSidecar.Slot(), root: firstSidecar.BlockRoot()}
+	entry := s.cache.ensure(key)
+
+	for i := range sidecars {
+		if err := entry.stash(&dataColumnSidecars[i]); err != nil {
+			return errors.Wrap(err, "stash DataColumnSidecar")
+		}
+	}
+
+	return nil
+}
+
+// IsDataAvailable returns nil if all the commitments in the given block are persisted to the db and have been verified.
+// DataColumnsSidecars already in the db are assumed to have been previously verified against the block.
+func (s *LazilyPersistentStoreColumn) IsDataAvailable(ctx context.Context, currentSlot primitives.Slot, block blocks.ROBlock) error {
+	blockCommitments, err := s.fullCommitmentsToCheck(s.nodeID, block, currentSlot)
+	if err != nil {
+		return errors.Wrapf(err, "full commitments to check with block root `%#x` and current slot `%d`", block.Root(), currentSlot)
+	}
+
+	// Return early for blocks that do not have any commitments.
+	if blockCommitments.count() == 0 {
+		return nil
+	}
+
+	// Get the root of the block.
+	blockRoot := block.Root()
+
+	// Build the cache key for the block.
+	key := dataColumnCacheKey{slot: block.Block().Slot(), root: blockRoot}
+
+	// Retrieve the cache entry for the block, or create an empty one if it doesn't exist.
+	entry := s.cache.ensure(key)
+
+	// Delete the cache entry for the block at the end.
+	defer s.cache.delete(key)
+
+	// Set the disk summary for the block in the cache entry.
+	entry.setDiskSummary(s.store.Summary(blockRoot))
+
+	// Verify we have all the expected sidecars, and fail fast if any are missing or inconsistent.
+	// We don't try to salvage problematic batches because this indicates a misbehaving peer and we'd rather
+	// ignore their response and decrease their peer score.
+	roDataColumns, err := entry.filter(blockRoot, blockCommitments)
+	if err != nil {
+		return errors.Wrap(err, "filter")
+	}
+
+	// https://github.com/ethereum/consensus-specs/blob/dev/specs/fulu/p2p-interface.md#datacolumnsidecarsbyrange-v1
+	verifier := s.newDataColumnsVerifier(roDataColumns, verification.ByRangeRequestDataColumnSidecarRequirements)
+
+	if err := verifier.Valid(); err != nil {
+		return errors.Wrap(err, "valid")
+	}
+
+	if err := verifier.SidecarInclusionProven(); err != nil {
+		return errors.Wrap(err, "sidecar inclusion proven")
+	}
+
+	if err := verifier.SidecarKzgProofVerified(); err != nil {
+		return errors.Wrap(err, "sidecar KZG proof verified")
+	}
+
+	verifiedRoDataColumns, err := verifier.VerifiedRODataColumns()
+	if err != nil {
+		return errors.Wrap(err, "verified RO data columns - should never happen")
+	}
+
+	if err := s.store.Save(verifiedRoDataColumns); err != nil {
+		return errors.Wrap(err, "save data column sidecars")
+	}
+
+	return nil
+}
+
+// fullCommitmentsToCheck returns the commitments to check for a given block.
+func (s *LazilyPersistentStoreColumn) fullCommitmentsToCheck(nodeID enode.ID, block blocks.ROBlock, currentSlot primitives.Slot) (*safeCommitmentsArray, error) {
+	// Return early for blocks that are pre-Fulu.
+	if block.Version() < version.Fulu {
+		return &safeCommitmentsArray{}, nil
+	}
+
+	// Compute the block epoch.
+	blockSlot := block.Block().Slot()
+	blockEpoch := slots.ToEpoch(blockSlot)
+
+	// Compute the current spoch.
+	currentEpoch := slots.ToEpoch(currentSlot)
+
+	// Return early if the request is out of the MIN_EPOCHS_FOR_DATA_COLUMN_SIDECARS_REQUESTS window.
+	if !params.WithinDAPeriod(blockEpoch, currentEpoch) {
+		return &safeCommitmentsArray{}, nil
+	}
+
+	// Retrieve the KZG commitments for the block.
+	kzgCommitments, err := block.Block().Body().BlobKzgCommitments()
+	if err != nil {
+		return nil, errors.Wrap(err, "blob KZG commitments")
+	}
+
+	// Return early if there are no commitments in the block.
+	if len(kzgCommitments) == 0 {
+		return &safeCommitmentsArray{}, nil
+	}
+
+	// Retrieve the groups count.
+	custodyGroupCount := s.custodyInfo.ActualGroupCount()
+
+	// Retrieve peer info.
+	peerInfo, _, err := peerdas.Info(nodeID, custodyGroupCount)
+	if err != nil {
+		return nil, errors.Wrap(err, "peer info")
+	}
+
+	// Create a safe commitments array for the custody columns.
+	commitmentsArray := &safeCommitmentsArray{}
+	for column := range peerInfo.CustodyColumns {
+		commitmentsArray[column] = kzgCommitments
+	}
+
+	return commitmentsArray, nil
+}

beacon-chain/das/availability_columns_test.go

@@ -0,0 +1,301 @@
+package das
+
+import (
+	"context"
+	"testing"
+
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/db/filesystem"
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/verification"
+	"github.com/OffchainLabs/prysm/v6/cmd/beacon-chain/flags"
+	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
+	"github.com/OffchainLabs/prysm/v6/config/params"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
+	"github.com/OffchainLabs/prysm/v6/encoding/bytesutil"
+	"github.com/OffchainLabs/prysm/v6/testing/require"
+	"github.com/OffchainLabs/prysm/v6/testing/util"
+	"github.com/OffchainLabs/prysm/v6/time/slots"
+	"github.com/ethereum/go-ethereum/p2p/enode"
+)
+
+func roSidecarsFromDataColumnParamsByBlockRoot(t *testing.T, dataColumnParamsByBlockRoot verification.DataColumnsParamsByRoot) ([]blocks.ROSidecar, []blocks.RODataColumn) {
+	roDataColumns, _ := verification.CreateTestVerifiedRoDataColumnSidecars(t, dataColumnParamsByBlockRoot)
+
+	roSidecars := make([]blocks.ROSidecar, 0, len(roDataColumns))
+	for _, roDataColumn := range roDataColumns {
+		roSidecars = append(roSidecars, blocks.NewSidecarFromDataColumnSidecar(roDataColumn))
+	}
+
+	return roSidecars, roDataColumns
+}
+
+func newSignedRoBlock(t *testing.T, signedBeaconBlock interface{}) blocks.ROBlock {
+	sb, err := blocks.NewSignedBeaconBlock(signedBeaconBlock)
+	require.NoError(t, err)
+
+	rb, err := blocks.NewROBlock(sb)
+	require.NoError(t, err)
+
+	return rb
+}
+
+var commitments = [][]byte{
+	bytesutil.PadTo([]byte("a"), 48),
+	bytesutil.PadTo([]byte("b"), 48),
+	bytesutil.PadTo([]byte("c"), 48),
+	bytesutil.PadTo([]byte("d"), 48),
+}
+
+func TestPersist(t *testing.T) {
+	t.Run("no sidecars", func(t *testing.T) {
+		dataColumnStorage := filesystem.NewEphemeralDataColumnStorage(t)
+		lazilyPersistentStoreColumns := NewLazilyPersistentStoreColumn(dataColumnStorage, enode.ID{}, nil, &peerdas.CustodyInfo{})
+		err := lazilyPersistentStoreColumns.Persist(0)
+		require.NoError(t, err)
+		require.Equal(t, 0, len(lazilyPersistentStoreColumns.cache.entries))
+	})
+
+	t.Run("mixed roots", func(t *testing.T) {
+		dataColumnStorage := filesystem.NewEphemeralDataColumnStorage(t)
+
+		dataColumnParamsByBlockRoot := map[[fieldparams.RootLength]byte][]verification.DataColumnParams{
+			{1}: {{ColumnIndex: 1}},
+			{2}: {{ColumnIndex: 2}},
+		}
+
+		roSidecars, _ := roSidecarsFromDataColumnParamsByBlockRoot(t, dataColumnParamsByBlockRoot)
+		lazilyPersistentStoreColumns := NewLazilyPersistentStoreColumn(dataColumnStorage, enode.ID{}, nil, &peerdas.CustodyInfo{})
+
+		err := lazilyPersistentStoreColumns.Persist(0, roSidecars...)
+		require.ErrorIs(t, err, errMixedRoots)
+		require.Equal(t, 0, len(lazilyPersistentStoreColumns.cache.entries))
+	})
+
+	t.Run("outside DA period", func(t *testing.T) {
+		dataColumnStorage := filesystem.NewEphemeralDataColumnStorage(t)
+
+		dataColumnParamsByBlockRoot := map[[fieldparams.RootLength]byte][]verification.DataColumnParams{
+			{1}: {{ColumnIndex: 1}},
+		}
+
+		roSidecars, _ := roSidecarsFromDataColumnParamsByBlockRoot(t, dataColumnParamsByBlockRoot)
+		lazilyPersistentStoreColumns := NewLazilyPersistentStoreColumn(dataColumnStorage, enode.ID{}, nil, &peerdas.CustodyInfo{})
+
+		err := lazilyPersistentStoreColumns.Persist(1_000_000, roSidecars...)
+		require.NoError(t, err)
+		require.Equal(t, 0, len(lazilyPersistentStoreColumns.cache.entries))
+	})
+
+	t.Run("nominal", func(t *testing.T) {
+		dataColumnStorage := filesystem.NewEphemeralDataColumnStorage(t)
+
+		dataColumnParamsByBlockRoot := map[[fieldparams.RootLength]byte][]verification.DataColumnParams{
+			{}: {{ColumnIndex: 1}, {ColumnIndex: 5}},
+		}
+
+		roSidecars, roDataColumns := roSidecarsFromDataColumnParamsByBlockRoot(t, dataColumnParamsByBlockRoot)
+		lazilyPersistentStoreColumns := NewLazilyPersistentStoreColumn(dataColumnStorage, enode.ID{}, nil, &peerdas.CustodyInfo{})
+
+		err := lazilyPersistentStoreColumns.Persist(0, roSidecars...)
+		require.NoError(t, err)
+		require.Equal(t, 1, len(lazilyPersistentStoreColumns.cache.entries))
+
+		key := dataColumnCacheKey{slot: 0, root: [32]byte{}}
+		entry := lazilyPersistentStoreColumns.cache.entries[key]
+
+		// A call to Persist does NOT save the sidecars to disk.
+		require.Equal(t, uint64(0), entry.diskSummary.Count())
+
+		require.DeepSSZEqual(t, roDataColumns[0], *entry.scs[1])
+		require.DeepSSZEqual(t, roDataColumns[1], *entry.scs[5])
+
+		for i, roDataColumn := range entry.scs {
+			if map[int]bool{1: true, 5: true}[i] {
+				continue
+			}
+
+			require.IsNil(t, roDataColumn)
+		}
+	})
+}
+
+func TestIsDataAvailable(t *testing.T) {
+	newDataColumnsVerifier := func(dataColumnSidecars []blocks.RODataColumn, _ []verification.Requirement) verification.DataColumnsVerifier {
+		return &mockDataColumnsVerifier{t: t, dataColumnSidecars: dataColumnSidecars}
+	}
+
+	ctx := context.Background()
+
+	t.Run("without commitments", func(t *testing.T) {
+		signedBeaconBlockFulu := util.NewBeaconBlockFulu()
+		signedRoBlock := newSignedRoBlock(t, signedBeaconBlockFulu)
+
+		dataColumnStorage := filesystem.NewEphemeralDataColumnStorage(t)
+		lazilyPersistentStoreColumns := NewLazilyPersistentStoreColumn(dataColumnStorage, enode.ID{}, newDataColumnsVerifier, &peerdas.CustodyInfo{})
+
+		err := lazilyPersistentStoreColumns.IsDataAvailable(ctx, 0 /*current slot*/, signedRoBlock)
+		require.NoError(t, err)
+	})
+
+	t.Run("with commitments", func(t *testing.T) {
+		signedBeaconBlockFulu := util.NewBeaconBlockFulu()
+		signedBeaconBlockFulu.Block.Body.BlobKzgCommitments = commitments
+		signedRoBlock := newSignedRoBlock(t, signedBeaconBlockFulu)
+		root := signedRoBlock.Root()
+
+		dataColumnStorage := filesystem.NewEphemeralDataColumnStorage(t)
+		lazilyPersistentStoreColumns := NewLazilyPersistentStoreColumn(dataColumnStorage, enode.ID{}, newDataColumnsVerifier, &peerdas.CustodyInfo{})
+
+		indices := [...]uint64{1, 17, 87, 102}
+		dataColumnsParams := make([]verification.DataColumnParams, 0, len(indices))
+		for _, index := range indices {
+			dataColumnParams := verification.DataColumnParams{
+				ColumnIndex:    index,
+				KzgCommitments: commitments,
+			}
+
+			dataColumnsParams = append(dataColumnsParams, dataColumnParams)
+		}
+
+		dataColumnsParamsByBlockRoot := verification.DataColumnsParamsByRoot{root: dataColumnsParams}
+		_, verifiedRoDataColumns := verification.CreateTestVerifiedRoDataColumnSidecars(t, dataColumnsParamsByBlockRoot)
+
+		key := dataColumnCacheKey{root: root}
+		entry := lazilyPersistentStoreColumns.cache.ensure(key)
+		defer lazilyPersistentStoreColumns.cache.delete(key)
+
+		for _, verifiedRoDataColumn := range verifiedRoDataColumns {
+			err := entry.stash(&verifiedRoDataColumn.RODataColumn)
+			require.NoError(t, err)
+		}
+
+		err := lazilyPersistentStoreColumns.IsDataAvailable(ctx, 0 /*current slot*/, signedRoBlock)
+		require.NoError(t, err)
+
+		actual, err := dataColumnStorage.Get(root, indices[:])
+		require.NoError(t, err)
+
+		summary := dataColumnStorage.Summary(root)
+		require.Equal(t, uint64(len(indices)), summary.Count())
+		require.DeepSSZEqual(t, verifiedRoDataColumns, actual)
+	})
+}
+
+func TestFullCommitmentsToCheck(t *testing.T) {
+	windowSlots, err := slots.EpochEnd(params.BeaconConfig().MinEpochsForDataColumnSidecarsRequest)
+	require.NoError(t, err)
+
+	testCases := []struct {
+		name        string
+		commitments [][]byte
+		block       func(*testing.T) blocks.ROBlock
+		slot        primitives.Slot
+	}{
+		{
+			name: "Pre-Fulu block",
+			block: func(t *testing.T) blocks.ROBlock {
+				return newSignedRoBlock(t, util.NewBeaconBlockElectra())
+			},
+		},
+		{
+			name: "Commitments outside data availability window",
+			block: func(t *testing.T) blocks.ROBlock {
+				beaconBlockElectra := util.NewBeaconBlockElectra()
+
+				// Block is from slot 0, "current slot" is window size +1 (so outside the window)
+				beaconBlockElectra.Block.Body.BlobKzgCommitments = commitments
+
+				return newSignedRoBlock(t, beaconBlockElectra)
+			},
+			slot: windowSlots + 1,
+		},
+		{
+			name: "Commitments within data availability window",
+			block: func(t *testing.T) blocks.ROBlock {
+				signedBeaconBlockFulu := util.NewBeaconBlockFulu()
+				signedBeaconBlockFulu.Block.Body.BlobKzgCommitments = commitments
+				signedBeaconBlockFulu.Block.Slot = 100
+
+				return newSignedRoBlock(t, signedBeaconBlockFulu)
+			},
+			commitments: commitments,
+			slot:        100,
+		},
+	}
+
+	for _, tc := range testCases {
+		t.Run(tc.name, func(t *testing.T) {
+			resetFlags := flags.Get()
+			gFlags := new(flags.GlobalFlags)
+			gFlags.SubscribeAllColumnSubnets = true
+			flags.Init(gFlags)
+			defer flags.Init(resetFlags)
+
+			b := tc.block(t)
+			s := NewLazilyPersistentStoreColumn(nil, enode.ID{}, nil, &peerdas.CustodyInfo{})
+
+			commitmentsArray, err := s.fullCommitmentsToCheck(enode.ID{}, b, tc.slot)
+			require.NoError(t, err)
+
+			for _, commitments := range commitmentsArray {
+				require.DeepEqual(t, tc.commitments, commitments)
+			}
+		})
+	}
+}
+
+type mockDataColumnsVerifier struct {
+	t                                                                        *testing.T
+	dataColumnSidecars                                                       []blocks.RODataColumn
+	validCalled, SidecarInclusionProvenCalled, SidecarKzgProofVerifiedCalled bool
+}
+
+var _ verification.DataColumnsVerifier = &mockDataColumnsVerifier{}
+
+func (m *mockDataColumnsVerifier) VerifiedRODataColumns() ([]blocks.VerifiedRODataColumn, error) {
+	require.Equal(m.t, true, m.validCalled && m.SidecarInclusionProvenCalled && m.SidecarKzgProofVerifiedCalled)
+
+	verifiedDataColumnSidecars := make([]blocks.VerifiedRODataColumn, 0, len(m.dataColumnSidecars))
+	for _, dataColumnSidecar := range m.dataColumnSidecars {
+		verifiedDataColumnSidecar := blocks.NewVerifiedRODataColumn(dataColumnSidecar)
+		verifiedDataColumnSidecars = append(verifiedDataColumnSidecars, verifiedDataColumnSidecar)
+	}
+
+	return verifiedDataColumnSidecars, nil
+}
+
+func (m *mockDataColumnsVerifier) SatisfyRequirement(verification.Requirement) {}
+
+func (m *mockDataColumnsVerifier) Valid() error {
+	m.validCalled = true
+	return nil
+}
+
+func (m *mockDataColumnsVerifier) CorrectSubnet(expectedTopics []string) error      { return nil }
+func (m *mockDataColumnsVerifier) NotFromFutureSlot() error                         { return nil }
+func (m *mockDataColumnsVerifier) SlotAboveFinalized() error                        { return nil }
+func (m *mockDataColumnsVerifier) ValidProposerSignature(ctx context.Context) error { return nil }
+
+func (m *mockDataColumnsVerifier) SidecarParentSeen(parentSeen func([fieldparams.RootLength]byte) bool) error {
+	return nil
+}
+
+func (m *mockDataColumnsVerifier) SidecarParentValid(badParent func([fieldparams.RootLength]byte) bool) error {
+	return nil
+}
+
+func (m *mockDataColumnsVerifier) SidecarParentSlotLower() error       { return nil }
+func (m *mockDataColumnsVerifier) SidecarDescendsFromFinalized() error { return nil }
+
+func (m *mockDataColumnsVerifier) SidecarInclusionProven() error {
+	m.SidecarInclusionProvenCalled = true
+	return nil
+}
+
+func (m *mockDataColumnsVerifier) SidecarKzgProofVerified() error {
+	m.SidecarKzgProofVerifiedCalled = true
+	return nil
+}
+
+func (m *mockDataColumnsVerifier) SidecarProposerExpected(ctx context.Context) error { return nil }

beacon-chain/das/availability_test.go

@@ -116,9 +116,11 @@ func TestLazilyPersistent_Missing(t *testing.T) {
 	ctx := context.Background()
 	store := filesystem.NewEphemeralBlobStorage(t)
 
-	blk, scs := util.GenerateTestDenebBlockWithSidecar(t, [32]byte{}, 1, 3)
+	blk, blobSidecars := util.GenerateTestDenebBlockWithSidecar(t, [32]byte{}, 1, 3)
 
-	mbv := &mockBlobBatchVerifier{t: t, scs: scs}
+	scs := blocks.NewSidecarsFromBlobSidecars(blobSidecars)
+
+	mbv := &mockBlobBatchVerifier{t: t, scs: blobSidecars}
 	as := NewLazilyPersistentStore(store, mbv)
 
 	// Only one commitment persisted, should return error with other indices
@@ -141,12 +143,14 @@ func TestLazilyPersistent_Mismatch(t *testing.T) {
 	ctx := context.Background()
 	store := filesystem.NewEphemeralBlobStorage(t)
 
-	blk, scs := util.GenerateTestDenebBlockWithSidecar(t, [32]byte{}, 1, 3)
+	blk, blobSidecars := util.GenerateTestDenebBlockWithSidecar(t, [32]byte{}, 1, 3)
 
 	mbv := &mockBlobBatchVerifier{t: t, err: errors.New("kzg check should not run")}
-	scs[0].KzgCommitment = bytesutil.PadTo([]byte("nope"), 48)
+	blobSidecars[0].KzgCommitment = bytesutil.PadTo([]byte("nope"), 48)
 	as := NewLazilyPersistentStore(store, mbv)
 
+	scs := blocks.NewSidecarsFromBlobSidecars(blobSidecars)
+
 	// Only one commitment persisted, should return error with other indices
 	require.NoError(t, as.Persist(1, scs[0]))
 	err := as.IsDataAvailable(ctx, 1, blk)
@@ -155,7 +159,10 @@ func TestLazilyPersistent_Mismatch(t *testing.T) {
 }
 
 func TestLazyPersistOnceCommitted(t *testing.T) {
-	_, scs := util.GenerateTestDenebBlockWithSidecar(t, [32]byte{}, 1, 6)
+	_, blobSidecars := util.GenerateTestDenebBlockWithSidecar(t, [32]byte{}, 1, 6)
+
+	scs := blocks.NewSidecarsFromBlobSidecars(blobSidecars)
+
 	as := NewLazilyPersistentStore(filesystem.NewEphemeralBlobStorage(t), &mockBlobBatchVerifier{})
 	// stashes as expected
 	require.NoError(t, as.Persist(1, scs...))
@@ -163,10 +170,13 @@ func TestLazyPersistOnceCommitted(t *testing.T) {
 	require.ErrorIs(t, as.Persist(1, scs...), ErrDuplicateSidecar)
 
 	// ignores index out of bound
-	scs[0].Index = 6
-	require.ErrorIs(t, as.Persist(1, scs[0]), errIndexOutOfBounds)
+	blobSidecars[0].Index = 6
+	require.ErrorIs(t, as.Persist(1, blocks.NewSidecarFromBlobSidecar(blobSidecars[0])), errIndexOutOfBounds)
+
+	_, moreBlobSidecars := util.GenerateTestDenebBlockWithSidecar(t, [32]byte{}, 1, 4)
+
+	more := blocks.NewSidecarsFromBlobSidecars(moreBlobSidecars)
 
-	_, more := util.GenerateTestDenebBlockWithSidecar(t, [32]byte{}, 1, 4)
 	// ignores sidecars before the retention period
 	slotOOB, err := slots.EpochStart(params.BeaconConfig().MinEpochsForBlobsSidecarsRequest)
 	require.NoError(t, err)

beacon-chain/das/cache.go

@@ -10,12 +10,7 @@ import (
 	"github.com/pkg/errors"
 )
 
-var (
-	ErrDuplicateSidecar   = errors.New("duplicate sidecar stashed in AvailabilityStore")
-	errIndexOutOfBounds   = errors.New("sidecar.index > MAX_BLOBS_PER_BLOCK")
-	errCommitmentMismatch = errors.New("KzgCommitment of sidecar in cache did not match block commitment")
-	errMissingSidecar     = errors.New("no sidecar in cache for block commitment")
-)
+var errIndexOutOfBounds = errors.New("sidecar.index > MAX_BLOBS_PER_BLOCK")
 
 // cacheKey includes the slot so that we can easily iterate through the cache and compare
 // slots for eviction purposes. Whether the input is the block or the sidecar, we always have

beacon-chain/das/data_column_cache.go

@@ -0,0 +1,135 @@
+package das
+
+import (
+	"reflect"
+
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/db/filesystem"
+	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
+	"github.com/pkg/errors"
+)
+
+var (
+	ErrDuplicateSidecar   = errors.New("duplicate sidecar stashed in AvailabilityStore")
+	errColumnIndexTooHigh = errors.New("column index too high")
+	errCommitmentMismatch = errors.New("KzgCommitment of sidecar in cache did not match block commitment")
+	errMissingSidecar     = errors.New("no sidecar in cache for block commitment")
+)
+
+// dataColumnCacheKey includes the slot so that we can easily iterate through the cache and compare
+// slots for eviction purposes. Whether the input is the block or the sidecar, we always have
+// the root+slot when interacting with the cache, so it isn't an inconvenience to use both.
+type dataColumnCacheKey struct {
+	slot primitives.Slot
+	root [32]byte
+}
+
+type dataColumnCache struct {
+	entries map[dataColumnCacheKey]*dataColumnCacheEntry
+}
+
+func newDataColumnCache() *dataColumnCache {
+	return &dataColumnCache{entries: make(map[dataColumnCacheKey]*dataColumnCacheEntry)}
+}
+
+// ensure returns the entry for the given key, creating it if it isn't already present.
+func (c *dataColumnCache) ensure(key dataColumnCacheKey) *dataColumnCacheEntry {
+	entry, ok := c.entries[key]
+	if !ok {
+		entry = &dataColumnCacheEntry{}
+		c.entries[key] = entry
+	}
+
+	return entry
+}
+
+// delete removes the cache entry from the cache.
+func (c *dataColumnCache) delete(key dataColumnCacheKey) {
+	delete(c.entries, key)
+}
+
+// dataColumnCacheEntry holds a fixed-length cache of BlobSidecars.
+type dataColumnCacheEntry struct {
+	scs         [fieldparams.NumberOfColumns]*blocks.RODataColumn
+	diskSummary filesystem.DataColumnStorageSummary
+}
+
+func (e *dataColumnCacheEntry) setDiskSummary(sum filesystem.DataColumnStorageSummary) {
+	e.diskSummary = sum
+}
+
+// stash adds an item to the in-memory cache of DataColumnSidecars.
+// Only the first DataColumnSidecar of a given Index will be kept in the cache.
+// stash will return an error if the given data colunn is already in the cache, or if the Index is out of bounds.
+func (e *dataColumnCacheEntry) stash(sc *blocks.RODataColumn) error {
+	if sc.Index >= fieldparams.NumberOfColumns {
+		return errors.Wrapf(errColumnIndexTooHigh, "index=%d", sc.Index)
+	}
+
+	if e.scs[sc.Index] != nil {
+		return errors.Wrapf(ErrDuplicateSidecar, "root=%#x, index=%d, commitment=%#x", sc.BlockRoot(), sc.Index, sc.KzgCommitments)
+	}
+
+	e.scs[sc.Index] = sc
+
+	return nil
+}
+
+func (e *dataColumnCacheEntry) filter(root [32]byte, commitmentsArray *safeCommitmentsArray) ([]blocks.RODataColumn, error) {
+	nonEmptyIndices := commitmentsArray.nonEmptyIndices()
+	if e.diskSummary.AllAvailable(nonEmptyIndices) {
+		return nil, nil
+	}
+
+	commitmentsCount := commitmentsArray.count()
+	sidecars := make([]blocks.RODataColumn, 0, commitmentsCount)
+
+	for i := range nonEmptyIndices {
+		if e.diskSummary.HasIndex(i) {
+			continue
+		}
+
+		if e.scs[i] == nil {
+			return nil, errors.Wrapf(errMissingSidecar, "root=%#x, index=%#x", root, i)
+		}
+
+		if !reflect.DeepEqual(commitmentsArray[i], e.scs[i].KzgCommitments) {
+			return nil, errors.Wrapf(errCommitmentMismatch, "root=%#x, index=%#x, commitment=%#x, block commitment=%#x", root, i, e.scs[i].KzgCommitments, commitmentsArray[i])
+		}
+
+		sidecars = append(sidecars, *e.scs[i])
+	}
+
+	return sidecars, nil
+}
+
+// safeCommitmentsArray is a fixed size array of commitments.
+// This is helpful for avoiding gratuitous bounds checks.
+type safeCommitmentsArray [fieldparams.NumberOfColumns][][]byte
+
+// count returns the number of commitments in the array.
+func (s *safeCommitmentsArray) count() int {
+	count := 0
+
+	for i := range s {
+		if s[i] != nil {
+			count++
+		}
+	}
+
+	return count
+}
+
+// nonEmptyIndices returns a map of indices that are non-nil in the array.
+func (s *safeCommitmentsArray) nonEmptyIndices() map[uint64]bool {
+	columns := make(map[uint64]bool)
+
+	for i := range s {
+		if s[i] != nil {
+			columns[uint64(i)] = true
+		}
+	}
+
+	return columns
+}

beacon-chain/das/data_column_cache_test.go

@@ -0,0 +1,124 @@
+package das
+
+import (
+	"testing"
+
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/db/filesystem"
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/verification"
+	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
+	"github.com/OffchainLabs/prysm/v6/testing/require"
+)
+
+func TestEnsureDeleteSetDiskSummary(t *testing.T) {
+	c := newDataColumnCache()
+	key := dataColumnCacheKey{}
+	entry := c.ensure(key)
+	require.DeepEqual(t, dataColumnCacheEntry{}, *entry)
+
+	diskSummary := filesystem.NewDataColumnStorageSummary(42, [fieldparams.NumberOfColumns]bool{true})
+	entry.setDiskSummary(diskSummary)
+	entry = c.ensure(key)
+	require.DeepEqual(t, dataColumnCacheEntry{diskSummary: diskSummary}, *entry)
+
+	c.delete(key)
+	entry = c.ensure(key)
+	require.DeepEqual(t, dataColumnCacheEntry{}, *entry)
+}
+
+func TestStash(t *testing.T) {
+	t.Run("Index too high", func(t *testing.T) {
+		dataColumnParamsByBlockRoot := verification.DataColumnsParamsByRoot{{1}: {{ColumnIndex: 10_000}}}
+		roDataColumns, _ := verification.CreateTestVerifiedRoDataColumnSidecars(t, dataColumnParamsByBlockRoot)
+
+		var entry dataColumnCacheEntry
+		err := entry.stash(&roDataColumns[0])
+		require.NotNil(t, err)
+	})
+
+	t.Run("Nominal and already existing", func(t *testing.T) {
+		dataColumnParamsByBlockRoot := verification.DataColumnsParamsByRoot{{1}: {{ColumnIndex: 1}}}
+		roDataColumns, _ := verification.CreateTestVerifiedRoDataColumnSidecars(t, dataColumnParamsByBlockRoot)
+
+		var entry dataColumnCacheEntry
+		err := entry.stash(&roDataColumns[0])
+		require.NoError(t, err)
+
+		require.DeepEqual(t, roDataColumns[0], entry.scs[1])
+
+		err = entry.stash(&roDataColumns[0])
+		require.NotNil(t, err)
+	})
+}
+
+func TestFilterDataColumns(t *testing.T) {
+	t.Run("All available", func(t *testing.T) {
+		commitmentsArray := safeCommitmentsArray{nil, [][]byte{[]byte{1}}, nil, [][]byte{[]byte{3}}}
+
+		diskSummary := filesystem.NewDataColumnStorageSummary(42, [fieldparams.NumberOfColumns]bool{false, true, false, true})
+
+		dataColumnCacheEntry := dataColumnCacheEntry{diskSummary: diskSummary}
+
+		actual, err := dataColumnCacheEntry.filter([fieldparams.RootLength]byte{}, &commitmentsArray)
+		require.NoError(t, err)
+		require.IsNil(t, actual)
+	})
+
+	t.Run("Some scs missing", func(t *testing.T) {
+		commitmentsArray := safeCommitmentsArray{nil, [][]byte{[]byte{1}}}
+
+		diskSummary := filesystem.NewDataColumnStorageSummary(42, [fieldparams.NumberOfColumns]bool{})
+
+		dataColumnCacheEntry := dataColumnCacheEntry{diskSummary: diskSummary}
+
+		_, err := dataColumnCacheEntry.filter([fieldparams.RootLength]byte{}, &commitmentsArray)
+		require.NotNil(t, err)
+	})
+
+	t.Run("Commitments not equal", func(t *testing.T) {
+		root := [fieldparams.RootLength]byte{}
+		commitmentsArray := safeCommitmentsArray{nil, [][]byte{[]byte{1}}}
+
+		dataColumnParamsByBlockRoot := verification.DataColumnsParamsByRoot{root: {{ColumnIndex: 1}}}
+		roDataColumns, _ := verification.CreateTestVerifiedRoDataColumnSidecars(t, dataColumnParamsByBlockRoot)
+
+		var scs [fieldparams.NumberOfColumns]*blocks.RODataColumn
+		scs[1] = &roDataColumns[0]
+
+		dataColumnCacheEntry := dataColumnCacheEntry{scs: scs}
+
+		_, err := dataColumnCacheEntry.filter(root, &commitmentsArray)
+		require.NotNil(t, err)
+	})
+
+	t.Run("Nominal", func(t *testing.T) {
+		root := [fieldparams.RootLength]byte{}
+		commitmentsArray := safeCommitmentsArray{nil, [][]byte{[]byte{1}}, nil, [][]byte{[]byte{3}}}
+
+		diskSummary := filesystem.NewDataColumnStorageSummary(42, [fieldparams.NumberOfColumns]bool{false, true})
+
+		dataColumnParamsByBlockRoot := verification.DataColumnsParamsByRoot{root: {{ColumnIndex: 3, KzgCommitments: [][]byte{[]byte{3}}}}}
+		expected, _ := verification.CreateTestVerifiedRoDataColumnSidecars(t, dataColumnParamsByBlockRoot)
+
+		var scs [fieldparams.NumberOfColumns]*blocks.RODataColumn
+		scs[3] = &expected[0]
+
+		dataColumnCacheEntry := dataColumnCacheEntry{scs: scs, diskSummary: diskSummary}
+
+		actual, err := dataColumnCacheEntry.filter(root, &commitmentsArray)
+		require.NoError(t, err)
+
+		require.DeepEqual(t, expected, actual)
+	})
+}
+
+func TestCount(t *testing.T) {
+	s := safeCommitmentsArray{nil, [][]byte{[]byte{1}}, nil, [][]byte{[]byte{3}}}
+	require.Equal(t, 2, s.count())
+}
+
+func TestNonEmptyIndices(t *testing.T) {
+	s := safeCommitmentsArray{nil, [][]byte{[]byte{10}}, nil, [][]byte{[]byte{20}}}
+	actual := s.nonEmptyIndices()
+	require.DeepEqual(t, map[uint64]bool{1: true, 3: true}, actual)
+}

beacon-chain/das/iface.go

@@ -15,5 +15,5 @@ import (
 // durably persisted before returning a non-error value.
 type AvailabilityStore interface {
 	IsDataAvailable(ctx context.Context, current primitives.Slot, b blocks.ROBlock) error
-	Persist(current primitives.Slot, sc ...blocks.ROBlob) error
+	Persist(current primitives.Slot, sc ...blocks.ROSidecar) error
 }

beacon-chain/das/mock.go

@@ -5,6 +5,7 @@ import (
 
 	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
+	errors "github.com/pkg/errors"
 )
 
 // MockAvailabilityStore is an implementation of AvailabilityStore that can be used by other packages in tests.
@@ -24,9 +25,13 @@ func (m *MockAvailabilityStore) IsDataAvailable(ctx context.Context, current pri
 }
 
 // Persist satisfies the corresponding method of the AvailabilityStore interface in a way that is useful for tests.
-func (m *MockAvailabilityStore) Persist(current primitives.Slot, sc ...blocks.ROBlob) error {
+func (m *MockAvailabilityStore) Persist(current primitives.Slot, sc ...blocks.ROSidecar) error {
+	blobSidecars, err := blocks.BlobSidecarsFromSidecars(sc)
+	if err != nil {
+		return errors.Wrap(err, "blob sidecars from sidecars")
+	}
 	if m.PersistBlobsCallback != nil {
-		return m.PersistBlobsCallback(current, sc...)
+		return m.PersistBlobsCallback(current, blobSidecars...)
 	}
 	return nil
 }

beacon-chain/db/filesystem/BUILD.bazel

@@ -30,6 +30,7 @@ go_library(
         "//consensus-types/primitives:go_default_library",
         "//encoding/bytesutil:go_default_library",
         "//io/file:go_default_library",
+        "//proto/prysm/v1alpha1:go_default_library",
         "//runtime/logging:go_default_library",
         "//time/slots:go_default_library",
         "@com_github_ethereum_go_ethereum//common/hexutil:go_default_library",

beacon-chain/db/filesystem/data_column.go

@@ -28,17 +28,19 @@ import (
 )
 
 const (
-	version                  = 0x01
-	versionOffset            = 0                           // bytes
-	versionSize              = 1                           // bytes
-	sidecarByteLenOffset     = versionOffset + versionSize // (Offset of the encoded size of the SSZ encoded data column sidecar)
-	sidecarByteLenSize       = 4                           // bytes (Size of the encoded size of the SSZ encoded data column sidecar)
-	mandatoryNumberOfColumns = 128                         // 2**7
-	indicesOffset            = sidecarByteLenOffset + sidecarByteLenSize
-	nonZeroOffset            = mandatoryNumberOfColumns
-	headerSize               = versionSize + sidecarByteLenSize + mandatoryNumberOfColumns
-	dataColumnsFileExtension = "sszs"
-	prunePeriod              = 1 * time.Minute
+	version                                      = 0x01
+	versionOffset                                = 0           // bytes
+	versionSize                                  = 1           // bytes
+	maxSszEncodedDataColumnSidecarSize           = 536_870_912 // 2**(4*8) / 8 bytes
+	encodedSszEncodedDataColumnSidecarSizeOffset = versionOffset + versionSize
+	encodedSszEncodedDataColumnSidecarSizeSize   = 4   // bytes (size of the encoded size of the SSZ encoded data column sidecar)
+	mandatoryNumberOfColumns                     = 128 // 2**7
+	indicesOffset                                = encodedSszEncodedDataColumnSidecarSizeOffset + encodedSszEncodedDataColumnSidecarSizeSize
+	indicesSize                                  = mandatoryNumberOfColumns
+	nonZeroOffset                                = mandatoryNumberOfColumns
+	headerSize                                   = versionSize + encodedSszEncodedDataColumnSidecarSizeSize + mandatoryNumberOfColumns
+	dataColumnsFileExtension                     = "sszs"
+	prunePeriod                                  = 1 * time.Minute
 )
 
 var (
@@ -59,17 +61,16 @@ type (
 		retentionEpochs primitives.Epoch
 		fs              afero.Fs
 		cache           *dataColumnStorageSummaryCache
-		dataColumnFeed  *event.Feed
+		DataColumnFeed  *event.Feed
+		muChans         map[[fieldparams.RootLength]byte]*muChan
+		mu              sync.Mutex // protect muChans
 		pruneMu         sync.RWMutex
-
-		mu      sync.Mutex // protects muChans
-		muChans map[[fieldparams.RootLength]byte]*muChan
 	}
 
 	// DataColumnStorageOption is a functional option for configuring a DataColumnStorage.
 	DataColumnStorageOption func(*DataColumnStorage) error
 
-	// DataColumnsIdent is a collection of unique identifiers for data column sidecars.
+	// DataColumnIdent is the unique identifier for a data column sidecar.
 	DataColumnsIdent struct {
 		Root    [fieldparams.RootLength]byte
 		Epoch   primitives.Epoch
@@ -129,7 +130,7 @@ func WithDataColumnFs(fs afero.Fs) DataColumnStorageOption {
 // initialized once per beacon node.
 func NewDataColumnStorage(ctx context.Context, opts ...DataColumnStorageOption) (*DataColumnStorage, error) {
 	storage := &DataColumnStorage{
-		dataColumnFeed: new(event.Feed),
+		DataColumnFeed: new(event.Feed),
 		muChans:        make(map[[fieldparams.RootLength]byte]*muChan),
 	}
 
@@ -194,7 +195,7 @@ func (dcs *DataColumnStorage) WarmCache() {
 		}
 
 		// Open the data column filesystem file.
-		f, err := dcs.fs.Open(path)
+		file, err := dcs.fs.Open(path)
 		if err != nil {
 			log.WithError(err).Error("Error encountered while opening data column filesystem file")
 			return nil
@@ -203,14 +204,14 @@ func (dcs *DataColumnStorage) WarmCache() {
 		// Close the file.
 		defer func() {
 			// Overwrite the existing error only if it is nil, since the close error is less important.
-			closeErr := f.Close()
+			closeErr := file.Close()
 			if closeErr != nil && err == nil {
 				err = closeErr
 			}
 		}()
 
 		// Read the metadata of the file.
-		metadata, err := dcs.metadata(f)
+		metadata, err := dcs.metadata(file)
 		if err != nil {
 			log.WithError(err).Error("Error encountered while reading metadata from data column filesystem file")
 			return nil
@@ -313,7 +314,7 @@ func (dcs *DataColumnStorage) Save(dataColumnSidecars []blocks.VerifiedRODataCol
 		}
 
 		// Notify the data column feed.
-		dcs.dataColumnFeed.Send(dataColumnsIdent)
+		dcs.DataColumnFeed.Send(dataColumnsIdent)
 	}
 
 	dataColumnSaveLatency.Observe(float64(time.Since(startTime).Milliseconds()))
@@ -321,23 +322,6 @@ func (dcs *DataColumnStorage) Save(dataColumnSidecars []blocks.VerifiedRODataCol
 	return nil
 }
 
-// Subscribe subscribes to the data column feed.
-// It returns the subscription and a 1-size buffer channel to receive data column sidecars.
-// It is the responsibility of the caller to:
-// - call `subscription.Unsubscribe` when done, and to
-// - read from the channel as fast as possible until the channel is closed.
-// A call to `Save` will buffer a new value to the channel.
-// If a call to `Save` is done while a value is already in the buffer of the channel:
-// - the call to `Save` will block until the new value can be bufferized in the channel, and
-// - all other subscribers won't be notified until the new value can be bufferized in the channel.
-func (dcs *DataColumnStorage) Subscribe() (event.Subscription, <-chan DataColumnsIdent) {
-	// Subscribe to newly data columns stored in the database.
-	identsChan := make(chan DataColumnsIdent, 1)
-	subscription := dcs.dataColumnFeed.Subscribe(identsChan)
-
-	return subscription, identsChan
-}
-
 // saveFilesystem saves data column sidecars into the database.
 // This function expects all data column sidecars to belong to the same block.
 func (dcs *DataColumnStorage) saveFilesystem(root [fieldparams.RootLength]byte, epoch primitives.Epoch, dataColumnSidecars []blocks.VerifiedRODataColumn) error {
@@ -347,7 +331,7 @@ func (dcs *DataColumnStorage) saveFilesystem(root [fieldparams.RootLength]byte,
 	dcs.pruneMu.RLock()
 	defer dcs.pruneMu.RUnlock()
 
-	fileMu, toStore := dcs.fileMutexChan(root)
+	fileMu, toStore := dcs.fileMutex(root)
 	toStore <- dataColumnSidecars
 
 	fileMu.Lock()
@@ -383,7 +367,7 @@ func (dcs *DataColumnStorage) Get(root [fieldparams.RootLength]byte, indices []u
 	dcs.pruneMu.RLock()
 	defer dcs.pruneMu.RUnlock()
 
-	fileMu, _ := dcs.fileMutexChan(root)
+	fileMu, _ := dcs.fileMutex(root)
 	fileMu.RLock()
 	defer fileMu.RUnlock()
 
@@ -403,11 +387,6 @@ func (dcs *DataColumnStorage) Get(root [fieldparams.RootLength]byte, indices []u
 		return nil, nil
 	}
 
-	// Exit early if no data column sidecars for this root is stored.
-	if !summary.HasAtLeastOneIndex(indices) {
-		return nil, nil
-	}
-
 	// Compute the file path.
 	filePath := filePath(root, summary.epoch)
 
@@ -465,7 +444,7 @@ func (dcs *DataColumnStorage) Remove(blockRoot [fieldparams.RootLength]byte) err
 	dcs.pruneMu.RLock()
 	defer dcs.pruneMu.RUnlock()
 
-	fileMu, _ := dcs.fileMutexChan(blockRoot)
+	fileMu, _ := dcs.fileMutex(blockRoot)
 	fileMu.Lock()
 	defer fileMu.Unlock()
 
@@ -510,6 +489,9 @@ func (dcs *DataColumnStorage) Clear() error {
 
 // prune clean the cache, the filesystem and mutexes.
 func (dcs *DataColumnStorage) prune() {
+	dcs.mu.Lock()
+	defer dcs.mu.Unlock()
+
 	highestStoredEpoch := dcs.cache.HighestEpoch()
 
 	// Check if we need to prune.
@@ -586,8 +568,6 @@ func (dcs *DataColumnStorage) prune() {
 		}
 	}
 
-	dcs.mu.Lock()
-	defer dcs.mu.Unlock()
 	clear(dcs.muChans)
 }
 
@@ -670,7 +650,7 @@ func (dcs *DataColumnStorage) saveDataColumnSidecarsExistingFile(filePath string
 
 	// Save indices to the file.
 	indices := metadata.indices.raw()
-	count, err := file.WriteAt(indices[:], int64(versionSize+sidecarByteLenSize))
+	count, err := file.WriteAt(indices[:], int64(versionSize+encodedSszEncodedDataColumnSidecarSizeSize))
 	if err != nil {
 		return errors.Wrap(err, "write indices")
 	}
@@ -781,7 +761,7 @@ func (dcs *DataColumnStorage) saveDataColumnSidecarsNewFile(filePath string, inp
 	}()
 
 	// Encode the SSZ encoded data column sidecar size.
-	var encodedSszEncodedDataColumnSidecarSize [sidecarByteLenSize]byte
+	var encodedSszEncodedDataColumnSidecarSize [encodedSszEncodedDataColumnSidecarSizeSize]byte
 	binary.BigEndian.PutUint32(encodedSszEncodedDataColumnSidecarSize[:], uint32(sszEncodedDataColumnSidecarRefSize))
 
 	// Get the raw indices.
@@ -834,7 +814,7 @@ func (dcs *DataColumnStorage) metadata(file afero.File) (*metadata, error) {
 	}
 
 	// DataColumnSidecar is a variable sized ssz object, but all data columns for a block will be the same size.
-	encodedSszEncodedDataColumnSidecarSize := header[sidecarByteLenOffset : sidecarByteLenOffset+sidecarByteLenSize]
+	encodedSszEncodedDataColumnSidecarSize := header[encodedSszEncodedDataColumnSidecarSizeOffset : encodedSszEncodedDataColumnSidecarSizeOffset+encodedSszEncodedDataColumnSidecarSizeSize]
 
 	// Convert the SSZ encoded data column sidecar size to an int.
 	sszEncodedDataColumnSidecarSize := binary.BigEndian.Uint32(encodedSszEncodedDataColumnSidecarSize)
@@ -861,7 +841,7 @@ func (dcs *DataColumnStorage) metadata(file afero.File) (*metadata, error) {
 	return metadata, nil
 }
 
-func (dcs *DataColumnStorage) fileMutexChan(root [fieldparams.RootLength]byte) (*sync.RWMutex, chan []blocks.VerifiedRODataColumn) {
+func (dcs *DataColumnStorage) fileMutex(root [fieldparams.RootLength]byte) (*sync.RWMutex, chan []blocks.VerifiedRODataColumn) {
 	dcs.mu.Lock()
 	defer dcs.mu.Unlock()
 

beacon-chain/db/filesystem/data_column_cache.go

@@ -33,17 +33,6 @@ func (s DataColumnStorageSummary) HasIndex(index uint64) bool {
 	return s.mask[index]
 }
 
-// HasAtLeastOneIndex returns true if at least one of the DataColumnSidecars at the given indices is available in the filesystem.
-func (s DataColumnStorageSummary) HasAtLeastOneIndex(indices []uint64) bool {
-	for _, index := range indices {
-		if s.mask[index] {
-			return true
-		}
-	}
-
-	return false
-}
-
 // Count returns the number of available data columns.
 func (s DataColumnStorageSummary) Count() uint64 {
 	count := uint64(0)
@@ -72,18 +61,6 @@ func (s DataColumnStorageSummary) AllAvailable(indices map[uint64]bool) bool {
 	return true
 }
 
-// Stored returns a map of all stored data columns.
-func (s DataColumnStorageSummary) Stored() map[uint64]bool {
-	stored := make(map[uint64]bool, fieldparams.NumberOfColumns)
-	for index, exists := range s.mask {
-		if exists {
-			stored[uint64(index)] = true
-		}
-	}
-
-	return stored
-}
-
 // DataColumnStorageSummarizer can be used to receive a summary of metadata about data columns on disk for a given root.
 // The DataColumnStorageSummary can be used to check which indices (if any) are available for a given block by root.
 type DataColumnStorageSummarizer interface {
@@ -145,12 +122,11 @@ func (sc *dataColumnStorageSummaryCache) set(dataColumnsIdent DataColumnsIdent)
 		count++
 
 		summary.mask[index] = true
+		sc.cache[dataColumnsIdent.Root] = summary
 		sc.lowestCachedEpoch = min(sc.lowestCachedEpoch, dataColumnsIdent.Epoch)
 		sc.highestCachedEpoch = max(sc.highestCachedEpoch, dataColumnsIdent.Epoch)
 	}
 
-	sc.cache[dataColumnsIdent.Root] = summary
-
 	countFloat := float64(count)
 	sc.dataColumnCount += countFloat
 	dataColumnDiskCount.Set(sc.dataColumnCount)

beacon-chain/db/filesystem/data_column_cache_test.go

@@ -22,16 +22,6 @@ func TestHasIndex(t *testing.T) {
 	require.Equal(t, true, hasIndex)
 }
 
-func TestHasAtLeastOneIndex(t *testing.T) {
-	summary := NewDataColumnStorageSummary(0, [fieldparams.NumberOfColumns]bool{false, true})
-
-	hasAtLeastOneIndex := summary.HasAtLeastOneIndex([]uint64{3, 1, 2})
-	require.Equal(t, true, hasAtLeastOneIndex)
-
-	hasAtLeastOneIndex = summary.HasAtLeastOneIndex([]uint64{3, 4, 2})
-	require.Equal(t, false, hasAtLeastOneIndex)
-}
-
 func TestCount(t *testing.T) {
 	summary := NewDataColumnStorageSummary(0, [fieldparams.NumberOfColumns]bool{false, true, false, true})
 
@@ -61,18 +51,6 @@ func TestAllAvailableDataColumns(t *testing.T) {
 	require.Equal(t, true, allAvailable)
 }
 
-func TestStored(t *testing.T) {
-	summary := NewDataColumnStorageSummary(0, [fieldparams.NumberOfColumns]bool{false, true, true, false})
-
-	expected := map[uint64]bool{1: true, 2: true}
-	actual := summary.Stored()
-
-	require.Equal(t, len(expected), len(actual))
-	for k, v := range expected {
-		require.Equal(t, v, actual[k])
-	}
-}
-
 func TestSummary(t *testing.T) {
 	root := [fieldparams.RootLength]byte{}
 

beacon-chain/db/filesystem/data_column_test.go

@@ -6,11 +6,11 @@ import (
 	"os"
 	"testing"
 
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/verification"
 	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
 	"github.com/OffchainLabs/prysm/v6/config/params"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
 	"github.com/OffchainLabs/prysm/v6/testing/require"
-	"github.com/OffchainLabs/prysm/v6/testing/util"
 	"github.com/spf13/afero"
 )
 
@@ -39,9 +39,9 @@ func TestWarmCache(t *testing.T) {
 	)
 	require.NoError(t, err)
 
-	_, verifiedRoDataColumnSidecars := util.CreateTestVerifiedRoDataColumnSidecars(
+	_, verifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
 		t,
-		util.DataColumnsParamsByRoot{
+		verification.DataColumnsParamsByRoot{
 			{0}: {
 				{Slot: 33, ColumnIndex: 2, DataColumn: []byte{1, 2, 3}}, // Period 0 - Epoch 1
 				{Slot: 33, ColumnIndex: 4, DataColumn: []byte{2, 3, 4}}, // Period 0 - Epoch 1
@@ -111,9 +111,9 @@ func TestSaveDataColumnsSidecars(t *testing.T) {
 		params.OverrideBeaconConfig(cfg)
 		params.SetupTestConfigCleanup(t)
 
-		_, verifiedRoDataColumnSidecars := util.CreateTestVerifiedRoDataColumnSidecars(
+		_, verifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
 			t,
-			util.DataColumnsParamsByRoot{
+			verification.DataColumnsParamsByRoot{
 				{}: {{ColumnIndex: 12}, {ColumnIndex: 1_000_000}, {ColumnIndex: 48}},
 			},
 		)
@@ -124,9 +124,9 @@ func TestSaveDataColumnsSidecars(t *testing.T) {
 	})
 
 	t.Run("one of the column index is too large", func(t *testing.T) {
-		_, verifiedRoDataColumnSidecars := util.CreateTestVerifiedRoDataColumnSidecars(
+		_, verifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
 			t,
-			util.DataColumnsParamsByRoot{{}: {{ColumnIndex: 12}, {ColumnIndex: 1_000_000}, {ColumnIndex: 48}}},
+			verification.DataColumnsParamsByRoot{{}: {{ColumnIndex: 12}, {ColumnIndex: 1_000_000}, {ColumnIndex: 48}}},
 		)
 
 		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
@@ -135,9 +135,9 @@ func TestSaveDataColumnsSidecars(t *testing.T) {
 	})
 
 	t.Run("different slots", func(t *testing.T) {
-		_, verifiedRoDataColumnSidecars := util.CreateTestVerifiedRoDataColumnSidecars(
+		_, verifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
 			t,
-			util.DataColumnsParamsByRoot{
+			verification.DataColumnsParamsByRoot{
 				{}: {
 					{Slot: 1, ColumnIndex: 12, DataColumn: []byte{1, 2, 3}},
 					{Slot: 2, ColumnIndex: 12, DataColumn: []byte{1, 2, 3}},
@@ -151,9 +151,9 @@ func TestSaveDataColumnsSidecars(t *testing.T) {
 	})
 
 	t.Run("new file - no data columns to save", func(t *testing.T) {
-		_, verifiedRoDataColumnSidecars := util.CreateTestVerifiedRoDataColumnSidecars(
+		_, verifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
 			t,
-			util.DataColumnsParamsByRoot{{}: {}},
+			verification.DataColumnsParamsByRoot{{}: {}},
 		)
 
 		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
@@ -162,9 +162,9 @@ func TestSaveDataColumnsSidecars(t *testing.T) {
 	})
 
 	t.Run("new file - different data column size", func(t *testing.T) {
-		_, verifiedRoDataColumnSidecars := util.CreateTestVerifiedRoDataColumnSidecars(
+		_, verifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
 			t,
-			util.DataColumnsParamsByRoot{
+			verification.DataColumnsParamsByRoot{
 				{}: {
 					{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}},
 					{ColumnIndex: 11, DataColumn: []byte{1, 2, 3, 4}},
@@ -178,9 +178,9 @@ func TestSaveDataColumnsSidecars(t *testing.T) {
 	})
 
 	t.Run("existing file - wrong incoming SSZ encoded size", func(t *testing.T) {
-		_, verifiedRoDataColumnSidecars := util.CreateTestVerifiedRoDataColumnSidecars(
+		_, verifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
 			t,
-			util.DataColumnsParamsByRoot{{1}: {{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}}}},
+			verification.DataColumnsParamsByRoot{{1}: {{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}}}},
 		)
 
 		// Save data columns into a file.
@@ -190,9 +190,9 @@ func TestSaveDataColumnsSidecars(t *testing.T) {
 
 		// Build a data column sidecar for the same block but with a different
 		// column index and an different SSZ encoded size.
-		_, verifiedRoDataColumnSidecars = util.CreateTestVerifiedRoDataColumnSidecars(
+		_, verifiedRoDataColumnSidecars = verification.CreateTestVerifiedRoDataColumnSidecars(
 			t,
-			util.DataColumnsParamsByRoot{{1}: {{ColumnIndex: 13, DataColumn: []byte{1, 2, 3, 4}}}},
+			verification.DataColumnsParamsByRoot{{1}: {{ColumnIndex: 13, DataColumn: []byte{1, 2, 3, 4}}}},
 		)
 
 		// Try to rewrite the file.
@@ -201,9 +201,9 @@ func TestSaveDataColumnsSidecars(t *testing.T) {
 	})
 
 	t.Run("nominal", func(t *testing.T) {
-		_, inputVerifiedRoDataColumnSidecars := util.CreateTestVerifiedRoDataColumnSidecars(
+		_, inputVerifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
 			t,
-			util.DataColumnsParamsByRoot{
+			verification.DataColumnsParamsByRoot{
 				{1}: {
 					{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}},
 					{ColumnIndex: 11, DataColumn: []byte{3, 4, 5}},
@@ -221,9 +221,9 @@ func TestSaveDataColumnsSidecars(t *testing.T) {
 		err := dataColumnStorage.Save(inputVerifiedRoDataColumnSidecars)
 		require.NoError(t, err)
 
-		_, inputVerifiedRoDataColumnSidecars = util.CreateTestVerifiedRoDataColumnSidecars(
+		_, inputVerifiedRoDataColumnSidecars = verification.CreateTestVerifiedRoDataColumnSidecars(
 			t,
-			util.DataColumnsParamsByRoot{
+			verification.DataColumnsParamsByRoot{
 				{1}: {
 					{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}}, // OK if duplicate
 					{ColumnIndex: 15, DataColumn: []byte{2, 3, 4}},
@@ -243,7 +243,7 @@ func TestSaveDataColumnsSidecars(t *testing.T) {
 			fileName         string
 			blockRoot        [fieldparams.RootLength]byte
 			expectedIndices  [mandatoryNumberOfColumns]byte
-			dataColumnParams []util.DataColumnParams
+			dataColumnParams []verification.DataColumnParams
 		}
 
 		fixtures := []fixture{
@@ -255,7 +255,7 @@ func TestSaveDataColumnsSidecars(t *testing.T) {
 					0, 0, 0, nonZeroOffset + 1, nonZeroOffset, nonZeroOffset + 2, 0, nonZeroOffset + 3,
 					// The rest is filled with zeroes.
 				},
-				dataColumnParams: []util.DataColumnParams{
+				dataColumnParams: []verification.DataColumnParams{
 					{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}},
 					{ColumnIndex: 11, DataColumn: []byte{3, 4, 5}},
 					{ColumnIndex: 13, DataColumn: []byte{6, 7, 8}},
@@ -271,7 +271,7 @@ func TestSaveDataColumnsSidecars(t *testing.T) {
 					0, 0, 0, 0, nonZeroOffset, nonZeroOffset + 1, 0, 0,
 					// The rest is filled with zeroes.
 				},
-				dataColumnParams: []util.DataColumnParams{
+				dataColumnParams: []verification.DataColumnParams{
 					{ColumnIndex: 12, DataColumn: []byte{3, 4, 5}},
 					{ColumnIndex: 13, DataColumn: []byte{6, 7, 8}},
 				},
@@ -283,7 +283,7 @@ func TestSaveDataColumnsSidecars(t *testing.T) {
 					0, 0, nonZeroOffset + 1, 0, 0, 0, nonZeroOffset, 0,
 					// The rest is filled with zeroes.
 				},
-				dataColumnParams: []util.DataColumnParams{
+				dataColumnParams: []verification.DataColumnParams{
 					{ColumnIndex: 6, DataColumn: []byte{3, 4, 5}},
 					{ColumnIndex: 2, DataColumn: []byte{6, 7, 8}},
 				},
@@ -292,9 +292,9 @@ func TestSaveDataColumnsSidecars(t *testing.T) {
 
 		for _, fixture := range fixtures {
 			// Build expected data column sidecars.
-			_, expectedDataColumnSidecars := util.CreateTestVerifiedRoDataColumnSidecars(
+			_, expectedDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
 				t,
-				util.DataColumnsParamsByRoot{fixture.blockRoot: fixture.dataColumnParams},
+				verification.DataColumnsParamsByRoot{fixture.blockRoot: fixture.dataColumnParams},
 			)
 
 			// Build expected bytes.
@@ -312,7 +312,7 @@ func TestSaveDataColumnsSidecars(t *testing.T) {
 				sszEncodedDataColumnSidecars = append(sszEncodedDataColumnSidecars, sszEncodedDataColumnSidecar...)
 			}
 
-			var encodedSszEncodedDataColumnSidecarSize [sidecarByteLenSize]byte
+			var encodedSszEncodedDataColumnSidecarSize [encodedSszEncodedDataColumnSidecarSizeSize]byte
 			binary.BigEndian.PutUint32(encodedSszEncodedDataColumnSidecarSize[:], uint32(sszEncodedDataColumnSidecarSize))
 
 			expectedBytes := make([]byte, 0, headerSize+dataColumnSidecarsCount*sszEncodedDataColumnSidecarSize)
@@ -352,7 +352,7 @@ func TestSaveDataColumnsSidecars(t *testing.T) {
 }
 
 func TestGetDataColumnSidecars(t *testing.T) {
-	t.Run("root not found", func(t *testing.T) {
+	t.Run("not found", func(t *testing.T) {
 		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
 
 		verifiedRODataColumnSidecars, err := dataColumnStorage.Get([fieldparams.RootLength]byte{1}, []uint64{12, 13, 14})
@@ -360,30 +360,10 @@ func TestGetDataColumnSidecars(t *testing.T) {
 		require.Equal(t, 0, len(verifiedRODataColumnSidecars))
 	})
 
-	t.Run("indices not found", func(t *testing.T) {
-		_, savedVerifiedRoDataColumnSidecars := util.CreateTestVerifiedRoDataColumnSidecars(
-			t,
-			util.DataColumnsParamsByRoot{
-				{1}: {
-					{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}},
-					{ColumnIndex: 14, DataColumn: []byte{2, 3, 4}},
-				},
-			},
-		)
-
-		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
-		err := dataColumnStorage.Save(savedVerifiedRoDataColumnSidecars)
-		require.NoError(t, err)
-
-		verifiedRODataColumnSidecars, err := dataColumnStorage.Get([fieldparams.RootLength]byte{1}, []uint64{3, 1, 2})
-		require.NoError(t, err)
-		require.Equal(t, 0, len(verifiedRODataColumnSidecars))
-	})
-
 	t.Run("nominal", func(t *testing.T) {
-		_, expectedVerifiedRoDataColumnSidecars := util.CreateTestVerifiedRoDataColumnSidecars(
+		_, expectedVerifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
 			t,
-			util.DataColumnsParamsByRoot{
+			verification.DataColumnsParamsByRoot{
 				{1}: {
 					{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}},
 					{ColumnIndex: 14, DataColumn: []byte{2, 3, 4}},
@@ -413,9 +393,9 @@ func TestRemove(t *testing.T) {
 	})
 
 	t.Run("nominal", func(t *testing.T) {
-		_, inputVerifiedRoDataColumnSidecars := util.CreateTestVerifiedRoDataColumnSidecars(
+		_, inputVerifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
 			t,
-			util.DataColumnsParamsByRoot{
+			verification.DataColumnsParamsByRoot{
 				{1}: {
 					{Slot: 32, ColumnIndex: 10, DataColumn: []byte{1, 2, 3}},
 					{Slot: 32, ColumnIndex: 11, DataColumn: []byte{2, 3, 4}},
@@ -453,9 +433,9 @@ func TestRemove(t *testing.T) {
 }
 
 func TestClear(t *testing.T) {
-	_, inputVerifiedRoDataColumnSidecars := util.CreateTestVerifiedRoDataColumnSidecars(
+	_, inputVerifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
 		t,
-		util.DataColumnsParamsByRoot{
+		verification.DataColumnsParamsByRoot{
 			{1}: {{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}}},
 			{2}: {{ColumnIndex: 13, DataColumn: []byte{6, 7, 8}}},
 		},
@@ -491,9 +471,9 @@ func TestClear(t *testing.T) {
 
 func TestMetadata(t *testing.T) {
 	t.Run("wrong version", func(t *testing.T) {
-		_, verifiedRoDataColumnSidecars := util.CreateTestVerifiedRoDataColumnSidecars(
+		_, verifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
 			t,
-			util.DataColumnsParamsByRoot{
+			verification.DataColumnsParamsByRoot{
 				{1}: {{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}}},
 			},
 		)
@@ -642,9 +622,9 @@ func TestPrune(t *testing.T) {
 
 			return true
 		}
-		_, verifiedRoDataColumnSidecars := util.CreateTestVerifiedRoDataColumnSidecars(
+		_, verifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
 			t,
-			util.DataColumnsParamsByRoot{
+			verification.DataColumnsParamsByRoot{
 				{0}: {
 					{Slot: 33, ColumnIndex: 2, DataColumn: []byte{1, 2, 3}}, // Period 0 - Epoch 1
 					{Slot: 33, ColumnIndex: 4, DataColumn: []byte{2, 3, 4}}, // Period 0 - Epoch 1
@@ -718,9 +698,9 @@ func TestPrune(t *testing.T) {
 		require.NoError(t, err)
 		require.Equal(t, true, compareSlices([]string{"0x0500000000000000000000000000000000000000000000000000000000000000.sszs"}, dirs))
 
-		_, verifiedRoDataColumnSidecars = util.CreateTestVerifiedRoDataColumnSidecars(
+		_, verifiedRoDataColumnSidecars = verification.CreateTestVerifiedRoDataColumnSidecars(
 			t,
-			util.DataColumnsParamsByRoot{
+			verification.DataColumnsParamsByRoot{
 				{6}: {{Slot: 451_141, ColumnIndex: 2, DataColumn: []byte{1, 2, 3}}}, // Period 3 - Epoch 14_098
 			},
 		)

beacon-chain/db/filesystem/iteration_test.go

@@ -1,7 +1,6 @@
 package filesystem
 
 import (
-	"bytes"
 	"fmt"
 	"io"
 	"math"
@@ -220,28 +219,6 @@ func TestListDir(t *testing.T) {
 	}
 }
 
-func TestSlotFromBlob(t *testing.T) {
-	cases := []struct {
-		slot primitives.Slot
-	}{
-		{slot: 0},
-		{slot: 2},
-		{slot: 1123581321},
-		{slot: math.MaxUint64},
-	}
-	for _, c := range cases {
-		t.Run(fmt.Sprintf("slot %d", c.slot), func(t *testing.T) {
-			_, sidecars := util.GenerateTestDenebBlockWithSidecar(t, [32]byte{}, c.slot, 1)
-			sc := sidecars[0]
-			enc, err := sc.MarshalSSZ()
-			require.NoError(t, err)
-			slot, err := slotFromBlob(bytes.NewReader(enc))
-			require.NoError(t, err)
-			require.Equal(t, c.slot, slot)
-		})
-	}
-}
-
 func TestIterationComplete(t *testing.T) {
 	targets := []migrationTestTarget{
 		{