Seperate subscribe data columns from attestation and sync committee subnets

* Validate parent block exists before signature

* `ValidProposerSignature`: Add comment

---------

Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>

CHANGELOG.md

@@ -4,64 +4,6 @@ All notable changes to this project will be documented in this file.
 
 The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
 
-## [v6.0.0](https://github.com/prysmaticlabs/prysm/compare/v5.3.2...v6.0.0) - 2025-04-21
-
-This release introduces Mainnet support for the upcoming Electra + Prague (Pectra) fork. The fork is scheduled for mainnet epoch 364032 (May 7, 2025, 10:05:11 UTC). You MUST update Prysm Beacon Node, Prysm Validator Client, and your execution layer client to the Pectra ready release prior to the fork to stay on the correct chain.
-
-Besides Pectra, we have more light client API support, cleanups, and a few bugfixes. Please review the changelog below and update your client as soon as practical before May 7.
-
-This release is **mandatory** for all operators before May 7.
-
-### Added
-
-- Implemented validator identities Beacon API endpoint. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15086)
-- Add SSZ support to light client updates by range API. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15082)
-- Add light client ssz types to the spec test. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15097)
-- Added the ability for execution requests to be tested in e2e with electra. [[PR]](https://github.com/prysmaticlabs/prysm/pull/14971)
-- Add warning messages for gas limit ranges that might be problematic. Low gas limits (≤10% of default) may cause transactions to fail, while high gas limits (>150% of default) could lead to block propagation issues. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15078)
-- Add light client store object to the beacon node object. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15120)
-- prysmctl option in wrapper script to generate devnet ssz. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15145)
-- Add support for Electra fork epoch. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15132)
-
-### Changed
-
-- The validator client will no longer use the full list of committee values but instead use the committee length and validator committee index. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15039)
-- Remove the header `Content-Disposition` from the `httputil.WriteSSZ` function. No `filename` parameter is needed anymore. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15092)
-- Sort attestations in proposer block by reward. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15093)
-- More efficient query method for stategen to retrieve blocks between a given state and the replay target block. This avoids attempting to look up blocks that are not needed for head replay queries, which may be missing due to a previous rollback bug. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15063)
-- removed old web3signer metrics in favor for a universal one. [[PR]](https://github.com/prysmaticlabs/prysm/pull/14920)
-- Deprecated everything related with the gRPC API. [[PR]](https://github.com/prysmaticlabs/prysm/pull/14944)
-- Migrate Prysm repo to Offchain Labs organization ahead of Pectra upgrade v6. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15140)
-
-### Deprecated
-
-- deprecates and removes usage of the `--trace` flag and`--cpuprofile` flag in favor of just using `--pprof`. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15083)
-
-### Removed
-
-- Remove /eth/v1/beacon/states/head/committees call when getting duties. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15039)
-- Removed unused hack scripts. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15157)
-- Remove `disable-committee-aware-packing` flag. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15162)
-- Remove deprecated flags for the major release. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15165)
-- Removed Beacon API endpoints which have been deprecated at the Deneb fork. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15166)
-
-### Fixed
-
-- The `--rpc` flag will now properly enable the keymanager APIs without web. The `--web` will enable both validator api endpoints and web. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15080)
-- Use latest state to pack attestation. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15113)
-- Clean up dangling block index entries for blocks that were previously deleted by incomplete cleanup code. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15040)
-- Fixed to use io stream instead of stream read. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15089)
-- When using a DV, send all aggregations for a slot and committee. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15110)
-- Fixed a bug in consolidation request processing. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15122)
-- Fix State Getter for pending withdrawal balance. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15123)
-- Fixed a bug in checking for attestation lengths in our block. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15134)
-- Fix Committee Index Check For Aggregates. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15146)
-- Fix filtering by committee index post-Electra in `ListAttestationsV2`. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15148)
-- Peers giving invalid data in range syncing are now downscored. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15149)
-- Adding fork guard to attestation api endpoints so that it doesn't accidentally include wrong attestation types in the pool. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15161)
-- fixed underflow with balances in leaking edge case with expected withdrawals. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15191)
-- Attribute block and blob issues to correct peers during range syncing. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15173)
-
 ## [v5.3.2](https://github.com/prysmaticlabs/prysm/compare/v5.3.1...v5.3.2) - 2025-03-25
 
 This release introduces support for the `Hoodi` testnet.
@@ -3313,4 +3255,4 @@ There are no security updates in this release.
 
 # Older than v2.0.0
 
-For changelog history for releases older than v2.0.0, please refer to https://github.com/prysmaticlabs/prysm/releases
+For changelog history for releases older than v2.0.0, please refer to https://github.com/prysmaticlabs/prysm/releases

WORKSPACE

@@ -1,7 +1,7 @@
 workspace(name = "prysm")
 
-load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
 load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")
+load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
 
 http_archive(
     name = "rules_pkg",
@@ -16,8 +16,6 @@ load("@rules_pkg//:deps.bzl", "rules_pkg_dependencies")
 
 rules_pkg_dependencies()
 
-load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
-
 http_archive(
     name = "toolchains_protoc",
     sha256 = "abb1540f8a9e045422730670ebb2f25b41fa56ca5a7cf795175a110a0a68f4ad",

api/client/event/event_stream.go

@@ -14,10 +14,22 @@ import (
 )
 
 const (
-	EventHead = "head"
-
-	EventError           = "error"
-	EventConnectionError = "connection_error"
+	EventHead                        = "head"
+	EventBlock                       = "block"
+	EventAttestation                 = "attestation"
+	EventVoluntaryExit               = "voluntary_exit"
+	EventBlsToExecutionChange        = "bls_to_execution_change"
+	EventProposerSlashing            = "proposer_slashing"
+	EventAttesterSlashing            = "attester_slashing"
+	EventFinalizedCheckpoint         = "finalized_checkpoint"
+	EventChainReorg                  = "chain_reorg"
+	EventContributionAndProof        = "contribution_and_proof"
+	EventLightClientFinalityUpdate   = "light_client_finality_update"
+	EventLightClientOptimisticUpdate = "light_client_optimistic_update"
+	EventPayloadAttributes           = "payload_attributes"
+	EventBlobSidecar                 = "blob_sidecar"
+	EventError                       = "error"
+	EventConnectionError             = "connection_error"
 )
 
 var (

api/server/structs/BUILD.bazel

@@ -57,7 +57,6 @@ go_test(
     deps = [
         "//proto/engine/v1:go_default_library",
         "//proto/prysm/v1alpha1:go_default_library",
-        "//testing/assert:go_default_library",
         "//testing/require:go_default_library",
         "@com_github_ethereum_go_ethereum//common:go_default_library",
         "@com_github_ethereum_go_ethereum//common/hexutil:go_default_library",

api/server/structs/conversions_state.go

@@ -26,7 +26,10 @@ func BeaconStateFromConsensus(st beaconState.BeaconState) (*BeaconState, error)
 	for i, r := range srcSr {
 		sr[i] = hexutil.Encode(r)
 	}
-	srcHr := st.HistoricalRoots()
+	srcHr, err := st.HistoricalRoots()
+	if err != nil {
+		return nil, err
+	}
 	hr := make([]string, len(srcHr))
 	for i, r := range srcHr {
 		hr[i] = hexutil.Encode(r)
@@ -113,7 +116,10 @@ func BeaconStateAltairFromConsensus(st beaconState.BeaconState) (*BeaconStateAlt
 	for i, r := range srcSr {
 		sr[i] = hexutil.Encode(r)
 	}
-	srcHr := st.HistoricalRoots()
+	srcHr, err := st.HistoricalRoots()
+	if err != nil {
+		return nil, err
+	}
 	hr := make([]string, len(srcHr))
 	for i, r := range srcHr {
 		hr[i] = hexutil.Encode(r)
@@ -219,7 +225,10 @@ func BeaconStateBellatrixFromConsensus(st beaconState.BeaconState) (*BeaconState
 	for i, r := range srcSr {
 		sr[i] = hexutil.Encode(r)
 	}
-	srcHr := st.HistoricalRoots()
+	srcHr, err := st.HistoricalRoots()
+	if err != nil {
+		return nil, err
+	}
 	hr := make([]string, len(srcHr))
 	for i, r := range srcHr {
 		hr[i] = hexutil.Encode(r)
@@ -338,7 +347,10 @@ func BeaconStateCapellaFromConsensus(st beaconState.BeaconState) (*BeaconStateCa
 	for i, r := range srcSr {
 		sr[i] = hexutil.Encode(r)
 	}
-	srcHr := st.HistoricalRoots()
+	srcHr, err := st.HistoricalRoots()
+	if err != nil {
+		return nil, err
+	}
 	hr := make([]string, len(srcHr))
 	for i, r := range srcHr {
 		hr[i] = hexutil.Encode(r)
@@ -476,7 +488,10 @@ func BeaconStateDenebFromConsensus(st beaconState.BeaconState) (*BeaconStateDene
 	for i, r := range srcSr {
 		sr[i] = hexutil.Encode(r)
 	}
-	srcHr := st.HistoricalRoots()
+	srcHr, err := st.HistoricalRoots()
+	if err != nil {
+		return nil, err
+	}
 	hr := make([]string, len(srcHr))
 	for i, r := range srcHr {
 		hr[i] = hexutil.Encode(r)
@@ -614,7 +629,10 @@ func BeaconStateElectraFromConsensus(st beaconState.BeaconState) (*BeaconStateEl
 	for i, r := range srcSr {
 		sr[i] = hexutil.Encode(r)
 	}
-	srcHr := st.HistoricalRoots()
+	srcHr, err := st.HistoricalRoots()
+	if err != nil {
+		return nil, err
+	}
 	hr := make([]string, len(srcHr))
 	for i, r := range srcHr {
 		hr[i] = hexutil.Encode(r)
@@ -797,7 +815,10 @@ func BeaconStateFuluFromConsensus(st beaconState.BeaconState) (*BeaconStateFulu,
 	for i, r := range srcSr {
 		sr[i] = hexutil.Encode(r)
 	}
-	srcHr := st.HistoricalRoots()
+	srcHr, err := st.HistoricalRoots()
+	if err != nil {
+		return nil, err
+	}
 	hr := make([]string, len(srcHr))
 	for i, r := range srcHr {
 		hr[i] = hexutil.Encode(r)

api/server/structs/conversions_test.go

@@ -4,9 +4,7 @@ import (
 	"testing"
 
 	eth "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
-	"github.com/OffchainLabs/prysm/v6/testing/assert"
 	"github.com/OffchainLabs/prysm/v6/testing/require"
-	"github.com/ethereum/go-ethereum/common/hexutil"
 )
 
 func TestDepositSnapshotFromConsensus(t *testing.T) {
@@ -104,266 +102,12 @@ func TestProposerSlashing_ToConsensus(t *testing.T) {
 	require.ErrorContains(t, errNilValue.Error(), err)
 }
 
-func TestProposerSlashing_FromConsensus(t *testing.T) {
-	input := []*eth.ProposerSlashing{
-		{
-			Header_1: &eth.SignedBeaconBlockHeader{
-				Header: &eth.BeaconBlockHeader{
-					Slot:          1,
-					ProposerIndex: 2,
-					ParentRoot:    []byte{3},
-					StateRoot:     []byte{4},
-					BodyRoot:      []byte{5},
-				},
-				Signature: []byte{6},
-			},
-			Header_2: &eth.SignedBeaconBlockHeader{
-				Header: &eth.BeaconBlockHeader{
-					Slot:          7,
-					ProposerIndex: 8,
-					ParentRoot:    []byte{9},
-					StateRoot:     []byte{10},
-					BodyRoot:      []byte{11},
-				},
-				Signature: []byte{12},
-			},
-		},
-		{
-			Header_1: &eth.SignedBeaconBlockHeader{
-				Header: &eth.BeaconBlockHeader{
-					Slot:          13,
-					ProposerIndex: 14,
-					ParentRoot:    []byte{15},
-					StateRoot:     []byte{16},
-					BodyRoot:      []byte{17},
-				},
-				Signature: []byte{18},
-			},
-			Header_2: &eth.SignedBeaconBlockHeader{
-				Header: &eth.BeaconBlockHeader{
-					Slot:          19,
-					ProposerIndex: 20,
-					ParentRoot:    []byte{21},
-					StateRoot:     []byte{22},
-					BodyRoot:      []byte{23},
-				},
-				Signature: []byte{24},
-			},
-		},
-	}
-
-	expectedResult := []*ProposerSlashing{
-		{
-			SignedHeader1: &SignedBeaconBlockHeader{
-				Message: &BeaconBlockHeader{
-					Slot:          "1",
-					ProposerIndex: "2",
-					ParentRoot:    hexutil.Encode([]byte{3}),
-					StateRoot:     hexutil.Encode([]byte{4}),
-					BodyRoot:      hexutil.Encode([]byte{5}),
-				},
-				Signature: hexutil.Encode([]byte{6}),
-			},
-			SignedHeader2: &SignedBeaconBlockHeader{
-				Message: &BeaconBlockHeader{
-					Slot:          "7",
-					ProposerIndex: "8",
-					ParentRoot:    hexutil.Encode([]byte{9}),
-					StateRoot:     hexutil.Encode([]byte{10}),
-					BodyRoot:      hexutil.Encode([]byte{11}),
-				},
-				Signature: hexutil.Encode([]byte{12}),
-			},
-		},
-		{
-			SignedHeader1: &SignedBeaconBlockHeader{
-				Message: &BeaconBlockHeader{
-					Slot:          "13",
-					ProposerIndex: "14",
-					ParentRoot:    hexutil.Encode([]byte{15}),
-					StateRoot:     hexutil.Encode([]byte{16}),
-					BodyRoot:      hexutil.Encode([]byte{17}),
-				},
-				Signature: hexutil.Encode([]byte{18}),
-			},
-			SignedHeader2: &SignedBeaconBlockHeader{
-				Message: &BeaconBlockHeader{
-					Slot:          "19",
-					ProposerIndex: "20",
-					ParentRoot:    hexutil.Encode([]byte{21}),
-					StateRoot:     hexutil.Encode([]byte{22}),
-					BodyRoot:      hexutil.Encode([]byte{23}),
-				},
-				Signature: hexutil.Encode([]byte{24}),
-			},
-		},
-	}
-
-	result := ProposerSlashingsFromConsensus(input)
-	assert.DeepEqual(t, expectedResult, result)
-}
-
 func TestAttesterSlashing_ToConsensus(t *testing.T) {
 	a := &AttesterSlashing{Attestation1: nil, Attestation2: nil}
 	_, err := a.ToConsensus()
 	require.ErrorContains(t, errNilValue.Error(), err)
 }
 
-func TestAttesterSlashing_FromConsensus(t *testing.T) {
-	input := []*eth.AttesterSlashing{
-		{
-			Attestation_1: &eth.IndexedAttestation{
-				AttestingIndices: []uint64{1, 2},
-				Data: &eth.AttestationData{
-					Slot:            3,
-					CommitteeIndex:  4,
-					BeaconBlockRoot: []byte{5},
-					Source: &eth.Checkpoint{
-						Epoch: 6,
-						Root:  []byte{7},
-					},
-					Target: &eth.Checkpoint{
-						Epoch: 8,
-						Root:  []byte{9},
-					},
-				},
-				Signature: []byte{10},
-			},
-			Attestation_2: &eth.IndexedAttestation{
-				AttestingIndices: []uint64{11, 12},
-				Data: &eth.AttestationData{
-					Slot:            13,
-					CommitteeIndex:  14,
-					BeaconBlockRoot: []byte{15},
-					Source: &eth.Checkpoint{
-						Epoch: 16,
-						Root:  []byte{17},
-					},
-					Target: &eth.Checkpoint{
-						Epoch: 18,
-						Root:  []byte{19},
-					},
-				},
-				Signature: []byte{20},
-			},
-		},
-		{
-			Attestation_1: &eth.IndexedAttestation{
-				AttestingIndices: []uint64{21, 22},
-				Data: &eth.AttestationData{
-					Slot:            23,
-					CommitteeIndex:  24,
-					BeaconBlockRoot: []byte{25},
-					Source: &eth.Checkpoint{
-						Epoch: 26,
-						Root:  []byte{27},
-					},
-					Target: &eth.Checkpoint{
-						Epoch: 28,
-						Root:  []byte{29},
-					},
-				},
-				Signature: []byte{30},
-			},
-			Attestation_2: &eth.IndexedAttestation{
-				AttestingIndices: []uint64{31, 32},
-				Data: &eth.AttestationData{
-					Slot:            33,
-					CommitteeIndex:  34,
-					BeaconBlockRoot: []byte{35},
-					Source: &eth.Checkpoint{
-						Epoch: 36,
-						Root:  []byte{37},
-					},
-					Target: &eth.Checkpoint{
-						Epoch: 38,
-						Root:  []byte{39},
-					},
-				},
-				Signature: []byte{40},
-			},
-		},
-	}
-
-	expectedResult := []*AttesterSlashing{
-		{
-			Attestation1: &IndexedAttestation{
-				AttestingIndices: []string{"1", "2"},
-				Data: &AttestationData{
-					Slot:            "3",
-					CommitteeIndex:  "4",
-					BeaconBlockRoot: hexutil.Encode([]byte{5}),
-					Source: &Checkpoint{
-						Epoch: "6",
-						Root:  hexutil.Encode([]byte{7}),
-					},
-					Target: &Checkpoint{
-						Epoch: "8",
-						Root:  hexutil.Encode([]byte{9}),
-					},
-				},
-				Signature: hexutil.Encode([]byte{10}),
-			},
-			Attestation2: &IndexedAttestation{
-				AttestingIndices: []string{"11", "12"},
-				Data: &AttestationData{
-					Slot:            "13",
-					CommitteeIndex:  "14",
-					BeaconBlockRoot: hexutil.Encode([]byte{15}),
-					Source: &Checkpoint{
-						Epoch: "16",
-						Root:  hexutil.Encode([]byte{17}),
-					},
-					Target: &Checkpoint{
-						Epoch: "18",
-						Root:  hexutil.Encode([]byte{19}),
-					},
-				},
-				Signature: hexutil.Encode([]byte{20}),
-			},
-		},
-		{
-			Attestation1: &IndexedAttestation{
-				AttestingIndices: []string{"21", "22"},
-				Data: &AttestationData{
-					Slot:            "23",
-					CommitteeIndex:  "24",
-					BeaconBlockRoot: hexutil.Encode([]byte{25}),
-					Source: &Checkpoint{
-						Epoch: "26",
-						Root:  hexutil.Encode([]byte{27}),
-					},
-					Target: &Checkpoint{
-						Epoch: "28",
-						Root:  hexutil.Encode([]byte{29}),
-					},
-				},
-				Signature: hexutil.Encode([]byte{30}),
-			},
-			Attestation2: &IndexedAttestation{
-				AttestingIndices: []string{"31", "32"},
-				Data: &AttestationData{
-					Slot:            "33",
-					CommitteeIndex:  "34",
-					BeaconBlockRoot: hexutil.Encode([]byte{35}),
-					Source: &Checkpoint{
-						Epoch: "36",
-						Root:  hexutil.Encode([]byte{37}),
-					},
-					Target: &Checkpoint{
-						Epoch: "38",
-						Root:  hexutil.Encode([]byte{39}),
-					},
-				},
-				Signature: hexutil.Encode([]byte{40}),
-			},
-		},
-	}
-
-	result := AttesterSlashingsFromConsensus(input)
-	assert.DeepEqual(t, expectedResult, result)
-}
-
 func TestIndexedAttestation_ToConsensus(t *testing.T) {
 	a := &IndexedAttestation{
 		AttestingIndices: []string{"1"},

api/server/structs/endpoints_beacon.go

@@ -263,13 +263,6 @@ type ChainHead struct {
 	OptimisticStatus           bool   `json:"optimistic_status"`
 }
 
-type GetPendingConsolidationsResponse struct {
-	Version             string                  `json:"version"`
-	ExecutionOptimistic bool                    `json:"execution_optimistic"`
-	Finalized           bool                    `json:"finalized"`
-	Data                []*PendingConsolidation `json:"data"`
-}
-
 type GetPendingDepositsResponse struct {
 	Version             string            `json:"version"`
 	ExecutionOptimistic bool              `json:"execution_optimistic"`

api/server/structs/other.go

@@ -253,7 +253,7 @@ type PendingDeposit struct {
 }
 
 type PendingPartialWithdrawal struct {
-	Index             string `json:"validator_index"`
+	Index             string `json:"index"`
 	Amount            string `json:"amount"`
 	WithdrawableEpoch string `json:"withdrawable_epoch"`
 }

beacon-chain/blockchain/BUILD.bazel

@@ -25,6 +25,7 @@ go_library(
         "receive_attestation.go",
         "receive_blob.go",
         "receive_block.go",
+        "receive_data_column.go",
         "service.go",
         "setup_forchoice.go",
         "tracked_proposer.go",
@@ -50,6 +51,7 @@ go_library(
         "//beacon-chain/core/feed/state:go_default_library",
         "//beacon-chain/core/helpers:go_default_library",
         "//beacon-chain/core/light-client:go_default_library",
+        "//beacon-chain/core/peerdas:go_default_library",
         "//beacon-chain/core/signing:go_default_library",
         "//beacon-chain/core/time:go_default_library",
         "//beacon-chain/core/transition:go_default_library",
@@ -146,6 +148,7 @@ go_test(
         "//beacon-chain/core/feed/state:go_default_library",
         "//beacon-chain/core/helpers:go_default_library",
         "//beacon-chain/core/light-client:go_default_library",
+        "//beacon-chain/core/peerdas:go_default_library",
         "//beacon-chain/core/signing:go_default_library",
         "//beacon-chain/core/transition:go_default_library",
         "//beacon-chain/das:go_default_library",

beacon-chain/blockchain/error.go

@@ -40,10 +40,12 @@ var (
 	errNotGenesisRoot = errors.New("root is not the genesis block root")
 	// errBlacklistedBlock is returned when a block is blacklisted as invalid.
 	errBlacklistedRoot = verification.AsVerificationFailure(errors.New("block root is blacklisted"))
+	// errMaxBlobsExceeded is returned when the number of blobs in a block exceeds the maximum allowed.
+	errMaxBlobsExceeded = verification.AsVerificationFailure(errors.New("Expected commitments in block exceeds MAX_BLOBS_PER_BLOCK"))
+	// errMaxDataColumnsExceeded is returned when the number of data columns exceeds the maximum allowed.
+	errMaxDataColumnsExceeded = verification.AsVerificationFailure(errors.New("Expected data columns for node exceeds NUMBER_OF_COLUMNS"))
 )
 
-var errMaxBlobsExceeded = verification.AsVerificationFailure(errors.New("Expected commitments in block exceeds MAX_BLOBS_PER_BLOCK"))
-
 // An invalid block is the block that fails state transition based on the core protocol rules.
 // The beacon node shall not be accepting nor building blocks that branch off from an invalid block.
 // Some examples of invalid blocks are:

beacon-chain/blockchain/execution_engine.go

@@ -184,17 +184,13 @@ func (s *Service) notifyForkchoiceUpdate(ctx context.Context, arg *fcuConfig) (*
 	return payloadID, nil
 }
 
-func (s *Service) firePayloadAttributesEvent(f event.SubscriberSender, block interfaces.ReadOnlySignedBeaconBlock, root [32]byte, nextSlot primitives.Slot) {
-	// If we're syncing a block in the past and init-sync is still running, we shouldn't fire this event.
-	if !s.cfg.SyncChecker.Synced() {
-		return
-	}
+func firePayloadAttributesEvent(_ context.Context, f event.SubscriberSender, nextSlot primitives.Slot) {
 	// the fcu args have differing amounts of completeness based on the code path,
 	// and there is work we only want to do if a client is actually listening to the events beacon api endpoint.
 	// temporary solution: just fire a blank event and fill in the details in the api handler.
 	f.Send(&feed.Event{
 		Type: statefeed.PayloadAttributes,
-		Data: payloadattribute.EventData{HeadBlock: block, HeadRoot: root, ProposalSlot: nextSlot},
+		Data: payloadattribute.EventData{ProposalSlot: nextSlot},
 	})
 }
 
@@ -439,6 +435,9 @@ func (s *Service) removeInvalidBlockAndState(ctx context.Context, blkRoots [][32
 			// Blobs may not exist for some blocks, leading to deletion failures. Log such errors at debug level.
 			log.WithError(err).Debug("Could not remove blob from blob storage")
 		}
+		if err := s.dataColumnStorage.Remove(root); err != nil {
+			log.WithError(err).Debug("Could not remove data columns from data column storage")
+		}
 	}
 	return nil
 }

beacon-chain/blockchain/forkchoice_update_execution.go

@@ -102,7 +102,7 @@ func (s *Service) forkchoiceUpdateWithExecution(ctx context.Context, args *fcuCo
 		log.WithError(err).Error("could not save head")
 	}
 
-	go s.firePayloadAttributesEvent(s.cfg.StateNotifier.StateFeed(), args.headBlock, args.headRoot, s.CurrentSlot()+1)
+	go firePayloadAttributesEvent(ctx, s.cfg.StateNotifier.StateFeed(), s.CurrentSlot()+1)
 
 	// Only need to prune attestations from pool if the head has changed.
 	s.pruneAttsFromPool(s.ctx, args.headState, args.headBlock)

beacon-chain/blockchain/kzg/BUILD.bazel

@@ -7,7 +7,7 @@ go_library(
         "trusted_setup.go",
         "validation.go",
     ],
-    embedsrcs = ["trusted_setup_4096.json"],
+    embedsrcs = ["trusted_setup.json"],
     importpath = "github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg",
     visibility = ["//visibility:public"],
     deps = [

beacon-chain/blockchain/kzg/kzg.go

@@ -39,8 +39,7 @@ type CellsAndProofs struct {
 
 // BlobToKZGCommitment computes a KZG commitment from a given blob.
 func BlobToKZGCommitment(blob *Blob) (Commitment, error) {
-	var kzgBlob kzg4844.Blob
-	copy(kzgBlob[:], blob[:])
+	kzgBlob := kzg4844.Blob(*blob)
 
 	commitment, err := kzg4844.BlobToCommitment(&kzgBlob)
 	if err != nil {
@@ -52,10 +51,9 @@ func BlobToKZGCommitment(blob *Blob) (Commitment, error) {
 
 // ComputeCells computes the (extended) cells from a given blob.
 func ComputeCells(blob *Blob) ([]Cell, error) {
-	var ckzgBlob ckzg4844.Blob
-	copy(ckzgBlob[:], blob[:])
+	ckzgBlob := (*ckzg4844.Blob)(blob)
 
-	ckzgCells, err := ckzg4844.ComputeCells(&ckzgBlob)
+	ckzgCells, err := ckzg4844.ComputeCells(ckzgBlob)
 	if err != nil {
 		return nil, errors.Wrap(err, "compute cells")
 	}
@@ -70,8 +68,7 @@ func ComputeCells(blob *Blob) ([]Cell, error) {
 
 // ComputeBlobKZGProof computes the blob KZG proof from a given blob and its commitment.
 func ComputeBlobKZGProof(blob *Blob, commitment Commitment) (Proof, error) {
-	var kzgBlob kzg4844.Blob
-	copy(kzgBlob[:], blob[:])
+	kzgBlob := kzg4844.Blob(*blob)
 
 	proof, err := kzg4844.ComputeBlobProof(&kzgBlob, kzg4844.Commitment(commitment))
 	if err != nil {
@@ -82,10 +79,9 @@ func ComputeBlobKZGProof(blob *Blob, commitment Commitment) (Proof, error) {
 
 // ComputeCellsAndKZGProofs computes the cells and cells KZG proofs from a given blob.
 func ComputeCellsAndKZGProofs(blob *Blob) (CellsAndProofs, error) {
-	var ckzgBlob ckzg4844.Blob
-	copy(ckzgBlob[:], blob[:])
+	ckzgBlob := (*ckzg4844.Blob)(blob)
 
-	ckzgCells, ckzgProofs, err := ckzg4844.ComputeCellsAndKZGProofs(&ckzgBlob)
+	ckzgCells, ckzgProofs, err := ckzg4844.ComputeCellsAndKZGProofs(ckzgBlob)
 	if err != nil {
 		return CellsAndProofs{}, err
 	}
@@ -93,7 +89,7 @@ func ComputeCellsAndKZGProofs(blob *Blob) (CellsAndProofs, error) {
 	return makeCellsAndProofs(ckzgCells[:], ckzgProofs[:])
 }
 
-// VerifyCellKZGProofBatch verifies the KZG proofs for a given slice of commitments, cells indices, cells and proofs.
+// VerifyBlobKZGProof verifies the KZG proofs for a given slice of commitments, cells indices, cells and proofs.
 // Note: It is way more efficient to call once this function with big slices than calling it multiple times with small slices.
 func VerifyCellKZGProofBatch(commitmentsBytes []Bytes48, cellIndices []uint64, cells []Cell, proofsBytes []Bytes48) (bool, error) {
 	// Convert `Cell` type to `ckzg4844.Cell`
@@ -128,9 +124,8 @@ func makeCellsAndProofs(ckzgCells []ckzg4844.Cell, ckzgProofs []ckzg4844.KZGProo
 		return CellsAndProofs{}, errors.New("different number of cells/proofs")
 	}
 
-	cells := make([]Cell, 0, len(ckzgCells))
-	proofs := make([]Proof, 0, len(ckzgProofs))
-
+	var cells []Cell
+	var proofs []Proof
 	for i := range ckzgCells {
 		cells = append(cells, Cell(ckzgCells[i]))
 		proofs = append(proofs, Proof(ckzgProofs[i]))

beacon-chain/blockchain/kzg/trusted_setup.go

@@ -11,8 +11,7 @@ import (
 )
 
 var (
-	// https://github.com/ethereum/consensus-specs/blob/dev/presets/mainnet/trusted_setups/trusted_setup_4096.json
-	//go:embed trusted_setup_4096.json
+	//go:embed trusted_setup.json
 	embeddedTrustedSetup []byte // 1.2Mb
 	kzgContext           *GoKZG.Context
 	kzgLoaded            bool
@@ -30,11 +29,9 @@ func Start() error {
 	if err != nil {
 		return errors.Wrap(err, "could not parse trusted setup JSON")
 	}
-
 	kzgContext, err = GoKZG.NewContext4096(&GoKZG.JSONTrustedSetup{
 		SetupG2:         trustedSetup.G2Monomial[:],
-		SetupG1Lagrange: trustedSetup.G1Lagrange,
-	})
+		SetupG1Lagrange: trustedSetup.G1Lagrange})
 	if err != nil {
 		return errors.Wrap(err, "could not initialize go-kzg context")
 	}

beacon-chain/blockchain/mock_test.go

@@ -20,7 +20,6 @@ func testServiceOptsWithDB(t *testing.T) []Option {
 		WithForkChoiceStore(fcs),
 		WithClockSynchronizer(cs),
 		WithStateNotifier(&mock.MockStateNotifier{RecordEvents: true}),
-		WithSyncChecker(&mock.MockSyncChecker{}),
 	}
 }
 

beacon-chain/blockchain/options.go

@@ -1,10 +1,13 @@
 package blockchain
 
 import (
+	"time"
+
 	"github.com/OffchainLabs/prysm/v6/async/event"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/cache"
 	statefeed "github.com/OffchainLabs/prysm/v6/beacon-chain/core/feed/state"
 	lightclient "github.com/OffchainLabs/prysm/v6/beacon-chain/core/light-client"
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/db"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/db/filesystem"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/execution"
@@ -127,9 +130,9 @@ func WithBLSToExecPool(p blstoexec.PoolManager) Option {
 }
 
 // WithP2PBroadcaster to broadcast messages after appropriate processing.
-func WithP2PBroadcaster(p p2p.Broadcaster) Option {
+func WithP2PBroadcaster(p p2p.Acceser) Option {
 	return func(s *Service) error {
-		s.cfg.P2p = p
+		s.cfg.P2P = p
 		return nil
 	}
 }
@@ -208,6 +211,14 @@ func WithBlobStorage(b *filesystem.BlobStorage) Option {
 	}
 }
 
+// WithDataColumnStorage sets the data column storage backend for the blockchain service.
+func WithDataColumnStorage(b *filesystem.DataColumnStorage) Option {
+	return func(s *Service) error {
+		s.dataColumnStorage = b
+		return nil
+	}
+}
+
 func WithSyncChecker(checker Checker) Option {
 	return func(s *Service) error {
 		s.cfg.SyncChecker = checker
@@ -215,6 +226,13 @@ func WithSyncChecker(checker Checker) Option {
 	}
 }
 
+func WithCustodyInfo(custodyInfo *peerdas.CustodyInfo) Option {
+	return func(s *Service) error {
+		s.cfg.CustodyInfo = custodyInfo
+		return nil
+	}
+}
+
 func WithSlasherEnabled(enabled bool) Option {
 	return func(s *Service) error {
 		s.slasherEnabled = enabled
@@ -222,6 +240,13 @@ func WithSlasherEnabled(enabled bool) Option {
 	}
 }
 
+func WithGenesisTime(genesisTime time.Time) Option {
+	return func(s *Service) error {
+		s.genesisTime = genesisTime
+		return nil
+	}
+}
+
 func WithLightClientStore(lcs *lightclient.Store) Option {
 	return func(s *Service) error {
 		s.lcStore = lcs

beacon-chain/blockchain/process_block.go

@@ -3,10 +3,12 @@ package blockchain
 import (
 	"context"
 	"fmt"
+	"slices"
 	"time"
 
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/blocks"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
 	coreTime "github.com/OffchainLabs/prysm/v6/beacon-chain/core/time"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/transition"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/das"
@@ -239,8 +241,9 @@ func (s *Service) onBlockBatch(ctx context.Context, blks []consensusblocks.ROBlo
 				return err
 			}
 		}
+
 		if err := avs.IsDataAvailable(ctx, s.CurrentSlot(), b); err != nil {
-			return errors.Wrapf(err, "could not validate blob data availability at slot %d", b.Block().Slot())
+			return errors.Wrapf(err, "could not validate sidecar availability at slot %d", b.Block().Slot())
 		}
 		args := &forkchoicetypes.BlockAndCheckpoints{Block: b,
 			JustifiedCheckpoint: jCheckpoints[i],
@@ -601,20 +604,206 @@ func missingIndices(bs *filesystem.BlobStorage, root [32]byte, expected [][]byte
 	return missing, nil
 }
 
-// isDataAvailable blocks until all BlobSidecars committed to in the block are available,
-// or an error or context cancellation occurs. A nil result means that the data availability check is successful.
-// The function will first check the database to see if all sidecars have been persisted. If any
-// sidecars are missing, it will then read from the blobNotifier channel for the given root until the channel is
-// closed, the context hits cancellation/timeout, or notifications have been received for all the missing sidecars.
-func (s *Service) isDataAvailable(ctx context.Context, root [32]byte, signed interfaces.ReadOnlySignedBeaconBlock) error {
-	if signed.Version() < version.Deneb {
-		return nil
+func missingDataColumns(bs *filesystem.DataColumnStorage, root [32]byte, expected map[uint64]bool) (map[uint64]bool, error) {
+	if len(expected) == 0 {
+		return nil, nil
 	}
 
-	block := signed.Block()
+	if len(expected) > int(params.BeaconConfig().NumberOfColumns) {
+		return nil, errMaxDataColumnsExceeded
+	}
+
+	// Get a summary of the data columns stored in the database.
+	summary := bs.Summary(root)
+
+	// Check all expected data columns against the summary.
+	missing := make(map[uint64]bool)
+	for column := range expected {
+		if !summary.HasIndex(column) {
+			missing[column] = true
+		}
+	}
+
+	return missing, nil
+}
+
+// isDataAvailable blocks until all sidecars committed to in the block are available,
+// or an error or context cancellation occurs. A nil result means that the data availability check is successful.
+// The function will first check the database to see if all sidecars have been persisted. If any
+// sidecars are missing, it will then read from the sidecar notifier channel for the given root until the channel is
+// closed, the context hits cancellation/timeout, or notifications have been received for all the missing sidecars.
+func (s *Service) isDataAvailable(ctx context.Context, root [32]byte, signedBlock interfaces.ReadOnlySignedBeaconBlock) error {
+	block := signedBlock.Block()
 	if block == nil {
 		return errors.New("invalid nil beacon block")
 	}
+
+	blockVersion := block.Version()
+	if blockVersion >= version.Fulu {
+		return s.areDataColumnsAvailable(ctx, root, block)
+	}
+
+	if blockVersion >= version.Deneb {
+		return s.areBlobsAvailable(ctx, root, block)
+	}
+
+	return nil
+}
+
+// areDataColumnsAvailable blocks until all data columns committed to in the block are available,
+// or an error or context cancellation occurs. A nil result means that the data availability check is successful.
+func (s *Service) areDataColumnsAvailable(ctx context.Context, root [fieldparams.RootLength]byte, block interfaces.ReadOnlyBeaconBlock) error {
+	// We are only required to check within MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS
+	blockSlot, currentSlot := block.Slot(), s.CurrentSlot()
+	blockEpoch, currentEpoch := slots.ToEpoch(blockSlot), slots.ToEpoch(currentSlot)
+
+	if !params.WithinDAPeriod(blockEpoch, currentEpoch) {
+		return nil
+	}
+
+	body := block.Body()
+	if body == nil {
+		return errors.New("invalid nil beacon block body")
+	}
+
+	kzgCommitments, err := body.BlobKzgCommitments()
+	if err != nil {
+		return errors.Wrap(err, "blob KZG commitments")
+	}
+
+	// If block has not commitments there is nothing to wait for.
+	if len(kzgCommitments) == 0 {
+		return nil
+	}
+
+	// All columns to sample need to be available for the block to be considered available.
+	// https://github.com/ethereum/consensus-specs/blob/v1.5.0-alpha.10/specs/fulu/das-core.md#custody-sampling
+	nodeID := s.cfg.P2P.NodeID()
+
+	// Prevent custody group count to change during the rest of the function.
+	s.cfg.CustodyInfo.Mut.RLock()
+	defer s.cfg.CustodyInfo.Mut.RUnlock()
+
+	// Get the custody group sampling size for the node.
+	custodyGroupSamplingSize := s.cfg.CustodyInfo.CustodyGroupSamplingSize(peerdas.Actual)
+	peerInfo, _, err := peerdas.Info(nodeID, custodyGroupSamplingSize)
+	if err != nil {
+		return errors.Wrap(err, "peer info")
+	}
+
+	// Subscribe to newly data columns stored in the database.
+	identsChan := make(chan filesystem.DataColumnsIdent)
+	subscription := s.dataColumnStorage.DataColumnFeed.Subscribe(identsChan)
+	defer subscription.Unsubscribe()
+
+	// Get the count of data columns we already have in the store.
+	summary := s.dataColumnStorage.Summary(root)
+	storedDataColumnsCount := summary.Count()
+
+	// As soon as we have more than half of the data columns, we can reconstruct the missing ones.
+	// We don't need to wait for the rest of the data columns to declare the block as available.
+	if peerdas.CanSelfReconstruct(storedDataColumnsCount) {
+		return nil
+	}
+
+	// Get a map of data column indices that are not currently available.
+	missingMap, err := missingDataColumns(s.dataColumnStorage, root, peerInfo.CustodyColumns)
+	if err != nil {
+		return errors.Wrap(err, "missing data columns")
+	}
+
+	// If there are no missing indices, all data column sidecars are available.
+	// This is the happy path.
+	if len(missingMap) == 0 {
+		return nil
+	}
+
+	// Log for DA checks that cross over into the next slot; helpful for debugging.
+	nextSlot := slots.BeginsAt(block.Slot()+1, s.genesisTime)
+
+	// Avoid logging if DA check is called after next slot start.
+	if nextSlot.After(time.Now()) {
+		timer := time.AfterFunc(time.Until(nextSlot), func() {
+			missingMapCount := uint64(len(missingMap))
+
+			if missingMapCount == 0 {
+				return
+			}
+
+			var (
+				expected interface{} = "all"
+				missing  interface{} = "all"
+			)
+
+			numberOfColumns := params.BeaconConfig().NumberOfColumns
+			colMapCount := uint64(len(peerInfo.CustodyColumns))
+
+			if colMapCount < numberOfColumns {
+				expected = uint64MapToSortedSlice(peerInfo.CustodyColumns)
+			}
+
+			if missingMapCount < numberOfColumns {
+				missing = uint64MapToSortedSlice(missingMap)
+			}
+
+			log.WithFields(logrus.Fields{
+				"slot":            block.Slot(),
+				"root":            fmt.Sprintf("%#x", root),
+				"columnsExpected": expected,
+				"columnsWaiting":  missing,
+			}).Warning("Data columns still missing at slot end")
+		})
+		defer timer.Stop()
+	}
+
+	for {
+		select {
+		case idents := <-identsChan:
+			if idents.Root != root {
+				// This is not the root we are looking for.
+				continue
+			}
+
+			for _, index := range idents.Indices {
+				// This is a data column we are expecting.
+				if _, ok := missingMap[index]; ok {
+					storedDataColumnsCount++
+				}
+
+				// As soon as we have more than half of the data columns, we can reconstruct the missing ones.
+				// We don't need to wait for the rest of the data columns to declare the block as available.
+				if peerdas.CanSelfReconstruct(storedDataColumnsCount) {
+					return nil
+				}
+
+				// Remove the index from the missing map.
+				delete(missingMap, index)
+
+				// Return if there is no more missing data columns.
+				if len(missingMap) == 0 {
+					return nil
+				}
+			}
+
+		case <-ctx.Done():
+			var missingIndices interface{} = "all"
+			numberOfColumns := params.BeaconConfig().NumberOfColumns
+			missingIndicesCount := uint64(len(missingMap))
+
+			if missingIndicesCount < numberOfColumns {
+				missingIndices = uint64MapToSortedSlice(missingMap)
+			}
+
+			return errors.Wrapf(ctx.Err(), "data column sidecars slot: %d, BlockRoot: %#x, missing %v", block.Slot(), root, missingIndices)
+		}
+	}
+}
+
+// areBlobsAvailable blocks until all BlobSidecars committed to in the block are available,
+// or an error or context cancellation occurs. A nil result means that the data availability check is successful.
+func (s *Service) areBlobsAvailable(ctx context.Context, root [fieldparams.RootLength]byte, block interfaces.ReadOnlyBeaconBlock) error {
+	blockSlot := block.Slot()
+
 	// We are only required to check within MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS
 	if !params.WithinDAPeriod(slots.ToEpoch(block.Slot()), slots.ToEpoch(s.CurrentSlot())) {
 		return nil
@@ -636,7 +825,7 @@ func (s *Service) isDataAvailable(ctx context.Context, root [32]byte, signed int
 	// get a map of BlobSidecar indices that are not currently available.
 	missing, err := missingIndices(s.blobStorage, root, kzgCommitments, block.Slot())
 	if err != nil {
-		return err
+		return errors.Wrap(err, "missing indices")
 	}
 	// If there are no missing indices, all BlobSidecars are available.
 	if len(missing) == 0 {
@@ -648,15 +837,20 @@ func (s *Service) isDataAvailable(ctx context.Context, root [32]byte, signed int
 	nc := s.blobNotifiers.forRoot(root, block.Slot())
 
 	// Log for DA checks that cross over into the next slot; helpful for debugging.
-	nextSlot := slots.BeginsAt(signed.Block().Slot()+1, s.genesisTime)
+	nextSlot := slots.BeginsAt(block.Slot()+1, s.genesisTime)
 	// Avoid logging if DA check is called after next slot start.
 	if nextSlot.After(time.Now()) {
 		nst := time.AfterFunc(time.Until(nextSlot), func() {
 			if len(missing) == 0 {
 				return
 			}
-			log.WithFields(daCheckLogFields(root, signed.Block().Slot(), expected, len(missing))).
-				Error("Still waiting for DA check at slot end.")
+
+			log.WithFields(logrus.Fields{
+				"slot":          blockSlot,
+				"root":          fmt.Sprintf("%#x", root),
+				"blobsExpected": expected,
+				"blobsWaiting":  len(missing),
+			}).Error("Still waiting for blobs DA check at slot end.")
 		})
 		defer nst.Stop()
 	}
@@ -678,13 +872,14 @@ func (s *Service) isDataAvailable(ctx context.Context, root [32]byte, signed int
 	}
 }
 
-func daCheckLogFields(root [32]byte, slot primitives.Slot, expected, missing int) logrus.Fields {
-	return logrus.Fields{
-		"slot":          slot,
-		"root":          fmt.Sprintf("%#x", root),
-		"blobsExpected": expected,
-		"blobsWaiting":  missing,
+// uint64MapToSortedSlice produces a sorted uint64 slice from a map.
+func uint64MapToSortedSlice(input map[uint64]bool) []uint64 {
+	output := make([]uint64, 0, len(input))
+	for idx := range input {
+		output = append(output, idx)
 	}
+	slices.Sort[[]uint64](output)
+	return output
 }
 
 // lateBlockTasks  is called 4 seconds into the slot and performs tasks
@@ -729,13 +924,9 @@ func (s *Service) lateBlockTasks(ctx context.Context) {
 	attribute := s.getPayloadAttribute(ctx, headState, s.CurrentSlot()+1, headRoot[:])
 	// return early if we are not proposing next slot
 	if attribute.IsEmpty() {
-		headBlock, err := s.headBlock()
-		if err != nil {
-			log.WithError(err).WithField("head_root", headRoot).Error("unable to retrieve head block to fire payload attributes event")
-		}
 		// notifyForkchoiceUpdate fires the payload attribute event. But in this case, we won't
 		// call notifyForkchoiceUpdate, so the event is fired here.
-		go s.firePayloadAttributesEvent(s.cfg.StateNotifier.StateFeed(), headBlock, headRoot, s.CurrentSlot()+1)
+		go firePayloadAttributesEvent(ctx, s.cfg.StateNotifier.StateFeed(), s.CurrentSlot()+1)
 		return
 	}
 
@@ -770,7 +961,7 @@ func (s *Service) waitForSync() error {
 	}
 }
 
-func (s *Service) handleInvalidExecutionError(ctx context.Context, err error, blockRoot [32]byte, parentRoot [32]byte) error {
+func (s *Service) handleInvalidExecutionError(ctx context.Context, err error, blockRoot, parentRoot [32]byte) error {
 	if IsInvalidBlock(err) && InvalidBlockLVH(err) != [32]byte{} {
 		return s.pruneInvalidBlock(ctx, blockRoot, parentRoot, InvalidBlockLVH(err))
 	}

beacon-chain/blockchain/process_block_helpers.go

@@ -103,29 +103,15 @@ func (s *Service) sendStateFeedOnBlock(cfg *postBlockProcessConfig) {
 		log.WithError(err).Debug("Could not check if block is optimistic")
 		optimistic = true
 	}
-	currEpoch := slots.ToEpoch(s.CurrentSlot())
-	currDependenRoot, err := s.cfg.ForkChoiceStore.DependentRoot(currEpoch)
-	if err != nil {
-		log.WithError(err).Debug("Could not get dependent root")
-	}
-	prevDependentRoot := [32]byte{}
-	if currEpoch > 0 {
-		prevDependentRoot, err = s.cfg.ForkChoiceStore.DependentRoot(currEpoch - 1)
-		if err != nil {
-			log.WithError(err).Debug("Could not get previous dependent root")
-		}
-	}
 	// Send notification of the processed block to the state feed.
 	s.cfg.StateNotifier.StateFeed().Send(&feed.Event{
 		Type: statefeed.BlockProcessed,
 		Data: &statefeed.BlockProcessedData{
-			Slot:              cfg.roblock.Block().Slot(),
-			BlockRoot:         cfg.roblock.Root(),
-			SignedBlock:       cfg.roblock,
-			CurrDependentRoot: currDependenRoot,
-			PrevDependentRoot: prevDependentRoot,
-			Verified:          true,
-			Optimistic:        optimistic,
+			Slot:        cfg.roblock.Block().Slot(),
+			BlockRoot:   cfg.roblock.Root(),
+			SignedBlock: cfg.roblock,
+			Verified:    true,
+			Optimistic:  optimistic,
 		},
 	})
 }
@@ -309,11 +295,6 @@ func (s *Service) processLightClientFinalityUpdate(
 		Type: statefeed.LightClientFinalityUpdate,
 		Data: newUpdate,
 	})
-
-	if err = s.cfg.P2p.BroadcastLightClientFinalityUpdate(ctx, newUpdate); err != nil {
-		return errors.Wrap(err, "could not broadcast light client finality update")
-	}
-
 	return nil
 }
 
@@ -363,10 +344,6 @@ func (s *Service) processLightClientOptimisticUpdate(ctx context.Context, signed
 		Data: newUpdate,
 	})
 
-	if err = s.cfg.P2p.BroadcastLightClientOptimisticUpdate(ctx, newUpdate); err != nil {
-		return errors.Wrap(err, "could not broadcast light client optimistic update")
-	}
-
 	return nil
 }
 

beacon-chain/blockchain/process_block_test.go

@@ -2,6 +2,7 @@ package blockchain
 
 import (
 	"context"
+	"crypto/rand"
 	"fmt"
 	"math/big"
 	"strconv"
@@ -13,6 +14,7 @@ import (
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/blocks"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
 	lightClient "github.com/OffchainLabs/prysm/v6/beacon-chain/core/light-client"
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/signing"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/transition"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/das"
@@ -24,8 +26,8 @@ import (
 	doublylinkedtree "github.com/OffchainLabs/prysm/v6/beacon-chain/forkchoice/doubly-linked-tree"
 	forkchoicetypes "github.com/OffchainLabs/prysm/v6/beacon-chain/forkchoice/types"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/operations/attestations/kv"
-	mockp2p "github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/testing"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/state"
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/verification"
 	"github.com/OffchainLabs/prysm/v6/config/features"
 	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
 	"github.com/OffchainLabs/prysm/v6/config/params"
@@ -3246,6 +3248,154 @@ func TestSaveLightClientBootstrap(t *testing.T) {
 	reset()
 }
 
+type testIsAvailableParams struct {
+	options                 []Option
+	blobKzgCommitmentsCount uint64
+	columnsToSave           []uint64
+}
+
+func testIsAvailableSetup(t *testing.T, params testIsAvailableParams) (context.Context, context.CancelFunc, *Service, [fieldparams.RootLength]byte, interfaces.SignedBeaconBlock) {
+	ctx, cancel := context.WithCancel(context.Background())
+	dataColumnStorage := filesystem.NewEphemeralDataColumnStorage(t)
+
+	options := append(params.options, WithDataColumnStorage(dataColumnStorage))
+	service, _ := minimalTestService(t, options...)
+
+	genesisState, secretKeys := util.DeterministicGenesisStateElectra(t, 32 /*validator count*/)
+
+	err := service.saveGenesisData(ctx, genesisState)
+	require.NoError(t, err)
+
+	conf := util.DefaultBlockGenConfig()
+	conf.NumBlobKzgCommitments = params.blobKzgCommitmentsCount
+
+	signedBeaconBlock, err := util.GenerateFullBlockFulu(genesisState, secretKeys, conf, 10 /*block slot*/)
+	require.NoError(t, err)
+
+	root, err := signedBeaconBlock.Block.HashTreeRoot()
+	require.NoError(t, err)
+
+	dataColumnsParams := make([]verification.DataColumnParams, 0, len(params.columnsToSave))
+	for _, i := range params.columnsToSave {
+		dataColumnParam := verification.DataColumnParams{ColumnIndex: i}
+		dataColumnsParams = append(dataColumnsParams, dataColumnParam)
+	}
+
+	dataColumnParamsByBlockRoot := verification.DataColumnsParamsByRoot{root: dataColumnsParams}
+	_, verifiedRODataColumns := verification.CreateTestVerifiedRoDataColumnSidecars(t, dataColumnParamsByBlockRoot)
+
+	err = dataColumnStorage.Save(verifiedRODataColumns)
+	require.NoError(t, err)
+
+	signed, err := consensusblocks.NewSignedBeaconBlock(signedBeaconBlock)
+	require.NoError(t, err)
+
+	return ctx, cancel, service, root, signed
+}
+
+func TestIsDataAvailable(t *testing.T) {
+	t.Run("Fulu - out of retention window", func(t *testing.T) {
+		params := testIsAvailableParams{options: []Option{WithGenesisTime(time.Unix(0, 0))}}
+		ctx, _, service, root, signed := testIsAvailableSetup(t, params)
+
+		err := service.isDataAvailable(ctx, root, signed)
+		require.NoError(t, err)
+	})
+
+	t.Run("Fulu - no commitment in blocks", func(t *testing.T) {
+		ctx, _, service, root, signed := testIsAvailableSetup(t, testIsAvailableParams{})
+
+		err := service.isDataAvailable(ctx, root, signed)
+		require.NoError(t, err)
+	})
+
+	t.Run("Fulu - more than half of the columns in custody", func(t *testing.T) {
+		halfNumberOfColumns := params.BeaconConfig().NumberOfColumns / 2
+		indices := make([]uint64, 0, halfNumberOfColumns)
+		for i := range halfNumberOfColumns {
+			indices = append(indices, i)
+		}
+
+		params := testIsAvailableParams{
+			options:                 []Option{WithCustodyInfo(&peerdas.CustodyInfo{})},
+			columnsToSave:           indices,
+			blobKzgCommitmentsCount: 3,
+		}
+
+		ctx, _, service, root, signed := testIsAvailableSetup(t, params)
+
+		err := service.isDataAvailable(ctx, root, signed)
+		require.NoError(t, err)
+	})
+
+	t.Run("Fulu - no missing data columns", func(t *testing.T) {
+		params := testIsAvailableParams{
+			options:                 []Option{WithCustodyInfo(&peerdas.CustodyInfo{})},
+			columnsToSave:           []uint64{1, 17, 19, 42, 75, 87, 102, 117, 119}, // 119 is not needed
+			blobKzgCommitmentsCount: 3,
+		}
+
+		ctx, _, service, root, signed := testIsAvailableSetup(t, params)
+
+		err := service.isDataAvailable(ctx, root, signed)
+		require.NoError(t, err)
+	})
+
+	t.Run("Fulu - some initially missing data columns (no reconstruction)", func(t *testing.T) {
+		var custodyInfo peerdas.CustodyInfo
+		custodyInfo.TargetGroupCount.SetValidatorsCustodyRequirement(128)
+		custodyInfo.ToAdvertiseGroupCount.Set(128)
+
+		testParams := testIsAvailableParams{
+			options:                 []Option{WithCustodyInfo(&custodyInfo)},
+			blobKzgCommitmentsCount: 3,
+		}
+
+		ctx, _, service, root, signed := testIsAvailableSetup(t, testParams)
+
+		// If needed, generate a random root that is different from the block root.
+		var randomRoot [fieldparams.RootLength]byte
+		for randomRoot == root {
+			randomRootSlice := make([]byte, fieldparams.RootLength)
+			_, err := rand.Read(randomRootSlice)
+			require.NoError(t, err)
+			copy(randomRoot[:], randomRootSlice)
+		}
+
+		// TODO: Achieve the same result without using time.AfterFunc.
+		time.AfterFunc(10*time.Millisecond, func() {
+			halfNumberOfColumns := params.BeaconConfig().NumberOfColumns / 2
+			indices := make([]uint64, 0, halfNumberOfColumns)
+			for i := range halfNumberOfColumns {
+				indices = append(indices, i)
+			}
+
+			withSomeRequiredColumns := filesystem.DataColumnsIdent{Root: root, Indices: indices}
+			service.dataColumnStorage.DataColumnFeed.Send(withSomeRequiredColumns)
+		})
+
+		err := service.isDataAvailable(ctx, root, signed)
+		require.NoError(t, err)
+	})
+
+	t.Run("Fulu - some columns are definitively missing", func(t *testing.T) {
+		params := testIsAvailableParams{
+			options:                 []Option{WithCustodyInfo(&peerdas.CustodyInfo{})},
+			blobKzgCommitmentsCount: 3,
+		}
+
+		ctx, cancel, service, root, signed := testIsAvailableSetup(t, params)
+
+		// TODO: Achieve the same result without using time.AfterFunc.
+		time.AfterFunc(10*time.Millisecond, func() {
+			cancel()
+		})
+
+		err := service.isDataAvailable(ctx, root, signed)
+		require.NotNil(t, err)
+	})
+}
+
 func setupLightClientTestRequirements(ctx context.Context, t *testing.T, s *Service, v int, options ...util.LightClientOption) (*util.TestLightClient, *postBlockProcessConfig) {
 	var l *util.TestLightClient
 	switch v {
@@ -3310,7 +3460,6 @@ func TestProcessLightClientOptimisticUpdate(t *testing.T) {
 	params.OverrideBeaconConfig(beaconCfg)
 
 	s, tr := minimalTestService(t)
-	s.cfg.P2p = &mockp2p.FakeP2P{}
 	ctx := tr.ctx
 
 	testCases := []struct {
@@ -3446,7 +3595,6 @@ func TestProcessLightClientFinalityUpdate(t *testing.T) {
 	params.OverrideBeaconConfig(beaconCfg)
 
 	s, tr := minimalTestService(t)
-	s.cfg.P2p = &mockp2p.FakeP2P{}
 	ctx := tr.ctx
 
 	testCases := []struct {

beacon-chain/blockchain/receive_block.go

@@ -52,6 +52,13 @@ type BlobReceiver interface {
 	ReceiveBlob(context.Context, blocks.VerifiedROBlob) error
 }
 
+// DataColumnReceiver interface defines the methods of chain service for receiving new
+// data columns
+type DataColumnReceiver interface {
+	ReceiveDataColumn(blocks.VerifiedRODataColumn) error
+	ReceiveDataColumns([]blocks.VerifiedRODataColumn) error
+}
+
 // SlashingReceiver interface defines the methods of chain service for receiving validated slashing over the wire.
 type SlashingReceiver interface {
 	ReceiveAttesterSlashing(ctx context.Context, slashing ethpb.AttSlashing)
@@ -74,6 +81,7 @@ func (s *Service) ReceiveBlock(ctx context.Context, block interfaces.ReadOnlySig
 		log.WithField("blockRoot", fmt.Sprintf("%#x", blockRoot)).Debug("Ignoring already synced block")
 		return nil
 	}
+
 	receivedTime := time.Now()
 	s.blockBeingSynced.set(blockRoot)
 	defer s.blockBeingSynced.unset(blockRoot)
@@ -82,6 +90,7 @@ func (s *Service) ReceiveBlock(ctx context.Context, block interfaces.ReadOnlySig
 	if err != nil {
 		return err
 	}
+
 	preState, err := s.getBlockPreState(ctx, blockCopy.Block())
 	if err != nil {
 		return errors.Wrap(err, "could not get block's prestate")
@@ -97,10 +106,12 @@ func (s *Service) ReceiveBlock(ctx context.Context, block interfaces.ReadOnlySig
 	if err != nil {
 		return err
 	}
+
 	daWaitedTime, err := s.handleDA(ctx, blockCopy, blockRoot, avs)
 	if err != nil {
 		return err
 	}
+
 	// Defragment the state before continuing block processing.
 	s.defragmentState(postState)
 
@@ -229,24 +240,32 @@ func (s *Service) handleDA(
 	block interfaces.SignedBeaconBlock,
 	blockRoot [32]byte,
 	avs das.AvailabilityStore,
-) (time.Duration, error) {
-	daStartTime := time.Now()
-	if avs != nil {
-		rob, err := blocks.NewROBlockWithRoot(block, blockRoot)
-		if err != nil {
-			return 0, err
+) (elapsed time.Duration, err error) {
+	defer func(start time.Time) {
+		elapsed := time.Since(start)
+
+		if err == nil {
+			dataAvailWaitedTime.Observe(float64(elapsed.Milliseconds()))
 		}
-		if err := avs.IsDataAvailable(ctx, s.CurrentSlot(), rob); err != nil {
-			return 0, errors.Wrap(err, "could not validate blob data availability (AvailabilityStore.IsDataAvailable)")
-		}
-	} else {
-		if err := s.isDataAvailable(ctx, blockRoot, block); err != nil {
-			return 0, errors.Wrap(err, "could not validate blob data availability")
+	}(time.Now())
+
+	if avs == nil {
+		if err = s.isDataAvailable(ctx, blockRoot, block); err != nil {
+			return
 		}
+
+		return
 	}
-	daWaitedTime := time.Since(daStartTime)
-	dataAvailWaitedTime.Observe(float64(daWaitedTime.Milliseconds()))
-	return daWaitedTime, nil
+
+	var rob blocks.ROBlock
+	rob, err = blocks.NewROBlockWithRoot(block, blockRoot)
+	if err != nil {
+		return
+	}
+
+	err = avs.IsDataAvailable(ctx, s.CurrentSlot(), rob)
+
+	return
 }
 
 func (s *Service) reportPostBlockProcessing(

beacon-chain/blockchain/receive_data_column.go

@@ -0,0 +1,25 @@
+package blockchain
+
+import (
+	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
+	"github.com/pkg/errors"
+)
+
+// ReceiveDataColumns receives a batch of data columns.
+func (s *Service) ReceiveDataColumns(dataColumnSidecars []blocks.VerifiedRODataColumn) error {
+	if err := s.dataColumnStorage.Save(dataColumnSidecars); err != nil {
+		return errors.Wrap(err, "save data column sidecars")
+	}
+
+	return nil
+}
+
+// ReceiveDataColumn receives a single data column.
+// (It is only a wrapper around ReceiveDataColumns.)
+func (s *Service) ReceiveDataColumn(dataColumnSidecar blocks.VerifiedRODataColumn) error {
+	if err := s.dataColumnStorage.Save([]blocks.VerifiedRODataColumn{dataColumnSidecar}); err != nil {
+		return errors.Wrap(err, "save data column sidecars")
+	}
+
+	return nil
+}

beacon-chain/blockchain/service.go

@@ -16,6 +16,7 @@ import (
 	statefeed "github.com/OffchainLabs/prysm/v6/beacon-chain/core/feed/state"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
 	lightClient "github.com/OffchainLabs/prysm/v6/beacon-chain/core/light-client"
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
 	coreTime "github.com/OffchainLabs/prysm/v6/beacon-chain/core/time"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/transition"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/db"
@@ -30,6 +31,7 @@ import (
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/startup"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/state"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/state/stategen"
+	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
 	"github.com/OffchainLabs/prysm/v6/config/params"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
@@ -64,6 +66,7 @@ type Service struct {
 	blobNotifiers        *blobNotifierMap
 	blockBeingSynced     *currentlySyncingBlock
 	blobStorage          *filesystem.BlobStorage
+	dataColumnStorage    *filesystem.DataColumnStorage
 	slasherEnabled       bool
 	lcStore              *lightClient.Store
 }
@@ -81,7 +84,7 @@ type config struct {
 	ExitPool                voluntaryexits.PoolManager
 	SlashingPool            slashings.PoolManager
 	BLSToExecPool           blstoexec.PoolManager
-	P2p                     p2p.Broadcaster
+	P2P                     p2p.Acceser
 	MaxRoutines             int
 	StateNotifier           statefeed.Notifier
 	ForkChoiceStore         f.ForkChoicer
@@ -93,6 +96,7 @@ type config struct {
 	FinalizedStateAtStartUp state.BeaconState
 	ExecutionEngineCaller   execution.EngineCaller
 	SyncChecker             Checker
+	CustodyInfo             *peerdas.CustodyInfo
 }
 
 // Checker is an interface used to determine if a node is in initial sync
@@ -106,22 +110,26 @@ var ErrMissingClockSetter = errors.New("blockchain Service initialized without a
 type blobNotifierMap struct {
 	sync.RWMutex
 	notifiers map[[32]byte]chan uint64
-	seenIndex map[[32]byte][]bool
+	// TODO: Separate blobs from data columns
+	// seenIndex map[[32]byte][]bool
+	seenIndex map[[32]byte][fieldparams.NumberOfColumns]bool
 }
 
 // notifyIndex notifies a blob by its index for a given root.
 // It uses internal maps to keep track of seen indices and notifier channels.
 func (bn *blobNotifierMap) notifyIndex(root [32]byte, idx uint64, slot primitives.Slot) {
-	maxBlobsPerBlock := params.BeaconConfig().MaxBlobsPerBlock(slot)
-	if idx >= uint64(maxBlobsPerBlock) {
-		return
-	}
+	// TODO: Separate blobs from data columns
+	// maxBlobsPerBlock := params.BeaconConfig().MaxBlobsPerBlock(slot)
+	// if idx >= uint64(maxBlobsPerBlock) {
+	// 	return
+	// }
 
 	bn.Lock()
 	seen := bn.seenIndex[root]
-	if seen == nil {
-		seen = make([]bool, maxBlobsPerBlock)
-	}
+	// TODO: Separate blobs from data columns
+	// if seen == nil {
+	// seen = make([]bool, maxBlobsPerBlock)
+	// }
 	if seen[idx] {
 		bn.Unlock()
 		return
@@ -132,7 +140,9 @@ func (bn *blobNotifierMap) notifyIndex(root [32]byte, idx uint64, slot primitive
 	// Retrieve or create the notifier channel for the given root.
 	c, ok := bn.notifiers[root]
 	if !ok {
-		c = make(chan uint64, maxBlobsPerBlock)
+		// TODO: Separate blobs from data columns
+		// c = make(chan uint64, maxBlobsPerBlock)
+		c = make(chan uint64, fieldparams.NumberOfColumns)
 		bn.notifiers[root] = c
 	}
 
@@ -142,12 +152,15 @@ func (bn *blobNotifierMap) notifyIndex(root [32]byte, idx uint64, slot primitive
 }
 
 func (bn *blobNotifierMap) forRoot(root [32]byte, slot primitives.Slot) chan uint64 {
-	maxBlobsPerBlock := params.BeaconConfig().MaxBlobsPerBlock(slot)
+	// TODO: Separate blobs from data columns
+	// maxBlobsPerBlock := params.BeaconConfig().MaxBlobsPerBlock(slot)
 	bn.Lock()
 	defer bn.Unlock()
 	c, ok := bn.notifiers[root]
 	if !ok {
-		c = make(chan uint64, maxBlobsPerBlock)
+		// TODO: Separate blobs from data columns
+		// c = make(chan uint64, maxBlobsPerBlock)
+		c = make(chan uint64, fieldparams.NumberOfColumns)
 		bn.notifiers[root] = c
 	}
 	return c
@@ -173,7 +186,9 @@ func NewService(ctx context.Context, opts ...Option) (*Service, error) {
 	ctx, cancel := context.WithCancel(ctx)
 	bn := &blobNotifierMap{
 		notifiers: make(map[[32]byte]chan uint64),
-		seenIndex: make(map[[32]byte][]bool),
+		// TODO: Separate blobs from data columns
+		// seenIndex: make(map[[32]byte][]bool),
+		seenIndex: make(map[[32]byte][fieldparams.NumberOfColumns]bool),
 	}
 	srv := &Service{
 		ctx:                  ctx,

beacon-chain/blockchain/service_test.go

@@ -97,7 +97,7 @@ func setupBeaconChain(t *testing.T, beaconDB db.Database) *Service {
 		WithAttestationPool(attestations.NewPool()),
 		WithSlashingPool(slashings.NewPool()),
 		WithExitPool(voluntaryexits.NewPool()),
-		WithP2PBroadcaster(&mockBroadcaster{}),
+		WithP2PBroadcaster(&mockAccesser{}),
 		WithStateNotifier(&mockBeaconNode{}),
 		WithForkChoiceStore(fc),
 		WithAttestationService(attService),
@@ -587,7 +587,9 @@ func (s *MockClockSetter) SetClock(g *startup.Clock) error {
 func TestNotifyIndex(t *testing.T) {
 	// Initialize a blobNotifierMap
 	bn := &blobNotifierMap{
-		seenIndex: make(map[[32]byte][]bool),
+		// TODO: Separate blobs from data columns
+		// seenIndex: make(map[[32]byte][]bool),
+		seenIndex: make(map[[32]byte][fieldparams.NumberOfColumns]bool),
 		notifiers: make(map[[32]byte]chan uint64),
 	}
 

beacon-chain/blockchain/setup_test.go

@@ -20,9 +20,10 @@ import (
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/operations/attestations"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/operations/blstoexec"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p"
+	p2pTesting "github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/testing"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/startup"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/state/stategen"
-	"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
+	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
 	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
 	"github.com/OffchainLabs/prysm/v6/testing/require"
 	"google.golang.org/protobuf/proto"
@@ -47,6 +48,11 @@ type mockBroadcaster struct {
 	broadcastCalled bool
 }
 
+type mockAccesser struct {
+	mockBroadcaster
+	p2pTesting.MockPeerManager
+}
+
 func (mb *mockBroadcaster) Broadcast(_ context.Context, _ proto.Message) error {
 	mb.broadcastCalled = true
 	return nil
@@ -67,12 +73,7 @@ func (mb *mockBroadcaster) BroadcastBlob(_ context.Context, _ uint64, _ *ethpb.B
 	return nil
 }
 
-func (mb *mockBroadcaster) BroadcastLightClientOptimisticUpdate(_ context.Context, _ interfaces.LightClientOptimisticUpdate) error {
-	mb.broadcastCalled = true
-	return nil
-}
-
-func (mb *mockBroadcaster) BroadcastLightClientFinalityUpdate(_ context.Context, _ interfaces.LightClientFinalityUpdate) error {
+func (mb *mockBroadcaster) BroadcastDataColumn(_ context.Context, _ [fieldparams.RootLength]byte, _ uint64, _ *ethpb.DataColumnSidecar) error {
 	mb.broadcastCalled = true
 	return nil
 }
@@ -132,8 +133,10 @@ func minimalTestService(t *testing.T, opts ...Option) (*Service, *testServiceReq
 		WithDepositCache(dc),
 		WithTrackedValidatorsCache(cache.NewTrackedValidatorsCache()),
 		WithBlobStorage(filesystem.NewEphemeralBlobStorage(t)),
+		WithDataColumnStorage(filesystem.NewEphemeralDataColumnStorage(t)),
 		WithSyncChecker(mock.MockChecker{}),
 		WithExecutionEngineCaller(&mockExecution.EngineClient{}),
+		WithP2PBroadcaster(&mockAccesser{}),
 		WithLightClientStore(&lightclient.Store{}),
 	}
 	// append the variadic opts so they override the defaults by being processed afterwards

beacon-chain/blockchain/testing/mock.go

@@ -53,7 +53,6 @@ type ChainService struct {
 	InitSyncBlockRoots          map[[32]byte]bool
 	DB                          db.Database
 	State                       state.BeaconState
-	HeadStateErr                error
 	Block                       interfaces.ReadOnlySignedBeaconBlock
 	VerifyBlkDescendantErr      error
 	stateNotifier               statefeed.Notifier
@@ -75,6 +74,7 @@ type ChainService struct {
 	BlockSlot                   primitives.Slot
 	SyncingRoot                 [32]byte
 	Blobs                       []blocks.VerifiedROBlob
+	DataColumns                 []blocks.VerifiedRODataColumn
 	TargetRoot                  [32]byte
 	MockHeadSlot                *primitives.Slot
 }
@@ -365,9 +365,6 @@ func (s *ChainService) HeadState(context.Context) (state.BeaconState, error) {
 
 // HeadStateReadOnly mocks HeadStateReadOnly method in chain service.
 func (s *ChainService) HeadStateReadOnly(context.Context) (state.ReadOnlyBeaconState, error) {
-	if s.HeadStateErr != nil {
-		return nil, s.HeadStateErr
-	}
 	return s.State, nil
 }
 
@@ -715,18 +712,18 @@ func (c *ChainService) ReceiveBlob(_ context.Context, b blocks.VerifiedROBlob) e
 	return nil
 }
 
+// ReceiveDataColumn implements the same method in chain service
+func (c *ChainService) ReceiveDataColumn(dc blocks.VerifiedRODataColumn) error {
+	c.DataColumns = append(c.DataColumns, dc)
+	return nil
+}
+
+// ReceiveDataColumns implements the same method in chain service
+func (*ChainService) ReceiveDataColumns(_ []blocks.VerifiedRODataColumn) error {
+	return nil
+}
+
 // TargetRootForEpoch mocks the same method in the chain service
 func (c *ChainService) TargetRootForEpoch(_ [32]byte, _ primitives.Epoch) ([32]byte, error) {
 	return c.TargetRoot, nil
 }
-
-// MockSyncChecker is a mock implementation of blockchain.Checker.
-// We can't make an assertion here that this is true because that would create a circular dependency.
-type MockSyncChecker struct {
-	synced bool
-}
-
-// Synced satisfies the blockchain.Checker interface.
-func (m *MockSyncChecker) Synced() bool {
-	return m.synced
-}

beacon-chain/core/altair/upgrade.go

@@ -67,6 +67,10 @@ func UpgradeToAltair(ctx context.Context, state state.BeaconState) (state.Beacon
 	epoch := time.CurrentEpoch(state)
 
 	numValidators := state.NumValidators()
+	hrs, err := state.HistoricalRoots()
+	if err != nil {
+		return nil, err
+	}
 	s := &ethpb.BeaconStateAltair{
 		GenesisTime:           state.GenesisTime(),
 		GenesisValidatorsRoot: state.GenesisValidatorsRoot(),
@@ -79,7 +83,7 @@ func UpgradeToAltair(ctx context.Context, state state.BeaconState) (state.Beacon
 		LatestBlockHeader:           state.LatestBlockHeader(),
 		BlockRoots:                  state.BlockRoots(),
 		StateRoots:                  state.StateRoots(),
-		HistoricalRoots:             state.HistoricalRoots(),
+		HistoricalRoots:             hrs,
 		Eth1Data:                    state.Eth1Data(),
 		Eth1DataVotes:               state.Eth1DataVotes(),
 		Eth1DepositIndex:            state.Eth1DepositIndex(),

beacon-chain/core/altair/upgrade_test.go

@@ -82,8 +82,10 @@ func TestUpgradeToAltair(t *testing.T) {
 	require.DeepSSZEqual(t, preForkState.LatestBlockHeader(), aState.LatestBlockHeader())
 	require.DeepSSZEqual(t, preForkState.BlockRoots(), aState.BlockRoots())
 	require.DeepSSZEqual(t, preForkState.StateRoots(), aState.StateRoots())
-	r1 := preForkState.HistoricalRoots()
-	r2 := aState.HistoricalRoots()
+	r1, err := preForkState.HistoricalRoots()
+	require.NoError(t, err)
+	r2, err := aState.HistoricalRoots()
+	require.NoError(t, err)
 	require.DeepSSZEqual(t, r1, r2)
 	require.DeepSSZEqual(t, preForkState.Eth1Data(), aState.Eth1Data())
 	require.DeepSSZEqual(t, preForkState.Eth1DataVotes(), aState.Eth1DataVotes())

beacon-chain/core/blocks/payload.go

@@ -12,6 +12,7 @@ import (
 	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
+	"github.com/OffchainLabs/prysm/v6/encoding/bytesutil"
 	"github.com/OffchainLabs/prysm/v6/runtime/version"
 	"github.com/OffchainLabs/prysm/v6/time/slots"
 	"github.com/pkg/errors"
@@ -229,13 +230,29 @@ func verifyBlobCommitmentCount(slot primitives.Slot, body interfaces.ReadOnlyBea
 	if body.Version() < version.Deneb {
 		return nil
 	}
+
 	kzgs, err := body.BlobKzgCommitments()
 	if err != nil {
 		return err
 	}
-	maxBlobsPerBlock := params.BeaconConfig().MaxBlobsPerBlock(slot)
-	if len(kzgs) > maxBlobsPerBlock {
-		return fmt.Errorf("too many kzg commitments in block: %d", len(kzgs))
+
+	commitmentCount, maxBlobsPerBlock := len(kzgs), params.BeaconConfig().MaxBlobsPerBlock(slot)
+	if commitmentCount > maxBlobsPerBlock {
+		return fmt.Errorf("too many kzg commitments in block: actual count %d - max allowed %d", commitmentCount, maxBlobsPerBlock)
 	}
+
 	return nil
 }
+
+// GetBlockPayloadHash returns the hash of the execution payload of the block
+func GetBlockPayloadHash(blk interfaces.ReadOnlyBeaconBlock) ([32]byte, error) {
+	var payloadHash [32]byte
+	if IsPreBellatrixVersion(blk.Version()) {
+		return payloadHash, nil
+	}
+	payload, err := blk.Body().Execution()
+	if err != nil {
+		return payloadHash, err
+	}
+	return bytesutil.ToBytes32(payload.BlockHash()), nil
+}

beacon-chain/core/blocks/payload_test.go

@@ -926,8 +926,10 @@ func TestVerifyBlobCommitmentCount(t *testing.T) {
 	require.NoError(t, err)
 	require.NoError(t, blocks.VerifyBlobCommitmentCount(rb.Slot(), rb.Body()))
 
-	b = &ethpb.BeaconBlockDeneb{Body: &ethpb.BeaconBlockBodyDeneb{BlobKzgCommitments: make([][]byte, params.BeaconConfig().MaxBlobsPerBlock(rb.Slot())+1)}}
+	maxCommitmentsPerBlock := params.BeaconConfig().MaxBlobsPerBlock(rb.Slot())
+
+	b = &ethpb.BeaconBlockDeneb{Body: &ethpb.BeaconBlockBodyDeneb{BlobKzgCommitments: make([][]byte, maxCommitmentsPerBlock+1)}}
 	rb, err = consensusblocks.NewBeaconBlock(b)
 	require.NoError(t, err)
-	require.ErrorContains(t, fmt.Sprintf("too many kzg commitments in block: %d", params.BeaconConfig().MaxBlobsPerBlock(rb.Slot())+1), blocks.VerifyBlobCommitmentCount(rb.Slot(), rb.Body()))
+	require.ErrorContains(t, fmt.Sprintf("too many kzg commitments in block: actual count %d - max allowed %d", maxCommitmentsPerBlock+1, maxCommitmentsPerBlock), blocks.VerifyBlobCommitmentCount(rb.Slot(), rb.Body()))
 }

beacon-chain/core/blocks/proposer_slashing.go

@@ -16,9 +16,6 @@ import (
 	"google.golang.org/protobuf/proto"
 )
 
-// ErrCouldNotVerifyBlockHeader is returned when a block header's signature cannot be verified.
-var ErrCouldNotVerifyBlockHeader = errors.New("could not verify beacon block header")
-
 type slashValidatorFunc func(
 	ctx context.Context,
 	st state.BeaconState,
@@ -117,7 +114,7 @@ func VerifyProposerSlashing(
 	for _, header := range headers {
 		if err := signing.ComputeDomainVerifySigningRoot(beaconState, pIdx, slots.ToEpoch(hSlot),
 			header.Header, params.BeaconConfig().DomainBeaconProposer, header.Signature); err != nil {
-			return errors.Wrap(ErrCouldNotVerifyBlockHeader, err.Error())
+			return errors.Wrap(err, "could not verify beacon block header")
 		}
 	}
 	return nil

beacon-chain/core/blocks/signature.go

@@ -96,6 +96,24 @@ func VerifyBlockHeaderSignature(beaconState state.BeaconState, header *ethpb.Sig
 	return signing.VerifyBlockHeaderSigningRoot(header.Header, proposerPubKey, header.Signature, domain)
 }
 
+func VerifyBlockHeaderSignatureUsingCurrentFork(beaconState state.BeaconState, header *ethpb.SignedBeaconBlockHeader) error {
+	currentEpoch := slots.ToEpoch(header.Header.Slot)
+	fork, err := forks.Fork(currentEpoch)
+	if err != nil {
+		return err
+	}
+	domain, err := signing.Domain(fork, currentEpoch, params.BeaconConfig().DomainBeaconProposer, beaconState.GenesisValidatorsRoot())
+	if err != nil {
+		return err
+	}
+	proposer, err := beaconState.ValidatorAtIndex(header.Header.ProposerIndex)
+	if err != nil {
+		return err
+	}
+	proposerPubKey := proposer.PublicKey
+	return signing.VerifyBlockHeaderSigningRoot(header.Header, proposerPubKey, header.Signature, domain)
+}
+
 // VerifyBlockSignatureUsingCurrentFork verifies the proposer signature of a beacon block. This differs
 // from the above method by not using fork data from the state and instead retrieving it
 // via the respective epoch.

beacon-chain/core/capella/upgrade.go

@@ -42,6 +42,10 @@ func UpgradeToCapella(state state.BeaconState) (state.BeaconState, error) {
 		return nil, err
 	}
 
+	hrs, err := state.HistoricalRoots()
+	if err != nil {
+		return nil, err
+	}
 	s := &ethpb.BeaconStateCapella{
 		GenesisTime:           state.GenesisTime(),
 		GenesisValidatorsRoot: state.GenesisValidatorsRoot(),
@@ -54,7 +58,7 @@ func UpgradeToCapella(state state.BeaconState) (state.BeaconState, error) {
 		LatestBlockHeader:           state.LatestBlockHeader(),
 		BlockRoots:                  state.BlockRoots(),
 		StateRoots:                  state.StateRoots(),
-		HistoricalRoots:             state.HistoricalRoots(),
+		HistoricalRoots:             hrs,
 		Eth1Data:                    state.Eth1Data(),
 		Eth1DataVotes:               state.Eth1DataVotes(),
 		Eth1DepositIndex:            state.Eth1DepositIndex(),

beacon-chain/core/deneb/upgrade.go

@@ -57,6 +57,10 @@ func UpgradeToDeneb(state state.BeaconState) (state.BeaconState, error) {
 	if err != nil {
 		return nil, err
 	}
+	historicalRoots, err := state.HistoricalRoots()
+	if err != nil {
+		return nil, err
+	}
 
 	s := &ethpb.BeaconStateDeneb{
 		GenesisTime:           state.GenesisTime(),
@@ -70,7 +74,7 @@ func UpgradeToDeneb(state state.BeaconState) (state.BeaconState, error) {
 		LatestBlockHeader:           state.LatestBlockHeader(),
 		BlockRoots:                  state.BlockRoots(),
 		StateRoots:                  state.StateRoots(),
-		HistoricalRoots:             state.HistoricalRoots(),
+		HistoricalRoots:             historicalRoots,
 		Eth1Data:                    state.Eth1Data(),
 		Eth1DataVotes:               state.Eth1DataVotes(),
 		Eth1DepositIndex:            state.Eth1DepositIndex(),

beacon-chain/core/deneb/upgrade_test.go

@@ -47,8 +47,10 @@ func TestUpgradeToDeneb(t *testing.T) {
 	require.NoError(t, err)
 	require.DeepSSZEqual(t, make([]uint64, numValidators), s)
 
-	hr1 := preForkState.HistoricalRoots()
-	hr2 := mSt.HistoricalRoots()
+	hr1, err := preForkState.HistoricalRoots()
+	require.NoError(t, err)
+	hr2, err := mSt.HistoricalRoots()
+	require.NoError(t, err)
 	require.DeepEqual(t, hr1, hr2)
 
 	f := mSt.Fork()

beacon-chain/core/electra/upgrade.go

@@ -170,6 +170,10 @@ func UpgradeToElectra(beaconState state.BeaconState) (state.BeaconState, error)
 	if err != nil {
 		return nil, err
 	}
+	historicalRoots, err := beaconState.HistoricalRoots()
+	if err != nil {
+		return nil, err
+	}
 	excessBlobGas, err := payloadHeader.ExcessBlobGas()
 	if err != nil {
 		return nil, err
@@ -219,7 +223,7 @@ func UpgradeToElectra(beaconState state.BeaconState) (state.BeaconState, error)
 		LatestBlockHeader:           beaconState.LatestBlockHeader(),
 		BlockRoots:                  beaconState.BlockRoots(),
 		StateRoots:                  beaconState.StateRoots(),
-		HistoricalRoots:             beaconState.HistoricalRoots(),
+		HistoricalRoots:             historicalRoots,
 		Eth1Data:                    beaconState.Eth1Data(),
 		Eth1DataVotes:               beaconState.Eth1DataVotes(),
 		Eth1DepositIndex:            beaconState.Eth1DepositIndex(),

beacon-chain/core/electra/upgrade_test.go

@@ -77,8 +77,10 @@ func TestUpgradeToElectra(t *testing.T) {
 	require.NoError(t, err)
 	require.DeepSSZEqual(t, make([]uint64, numValidators), s)
 
-	hr1 := preForkState.HistoricalRoots()
-	hr2 := mSt.HistoricalRoots()
+	hr1, err := preForkState.HistoricalRoots()
+	require.NoError(t, err)
+	hr2, err := mSt.HistoricalRoots()
+	require.NoError(t, err)
 	require.DeepEqual(t, hr1, hr2)
 
 	f := mSt.Fork()

beacon-chain/core/epoch/epoch_processing_test.go

@@ -148,7 +148,8 @@ func TestProcessFinalUpdates_CanProcess(t *testing.T) {
 	assert.DeepNotEqual(t, params.BeaconConfig().ZeroHash[:], mix, "latest RANDAO still zero hashes")
 
 	// Verify historical root accumulator was appended.
-	roots := newS.HistoricalRoots()
+	roots, err := newS.HistoricalRoots()
+	require.NoError(t, err)
 	assert.Equal(t, 1, len(roots), "Unexpected slashed balance")
 	currAtt, err := newS.CurrentEpochAttestations()
 	require.NoError(t, err)
@@ -378,7 +379,8 @@ func TestProcessHistoricalDataUpdate(t *testing.T) {
 				return st
 			},
 			verifier: func(st state.BeaconState) {
-				roots := st.HistoricalRoots()
+				roots, err := st.HistoricalRoots()
+				require.NoError(t, err)
 				require.Equal(t, 0, len(roots))
 			},
 		},
@@ -391,7 +393,8 @@ func TestProcessHistoricalDataUpdate(t *testing.T) {
 				return st
 			},
 			verifier: func(st state.BeaconState) {
-				roots := st.HistoricalRoots()
+				roots, err := st.HistoricalRoots()
+				require.NoError(t, err)
 				require.Equal(t, 1, len(roots))
 
 				b := &ethpb.HistoricalBatch{
@@ -428,7 +431,8 @@ func TestProcessHistoricalDataUpdate(t *testing.T) {
 					StateSummaryRoot: sr[:],
 				}
 				require.DeepEqual(t, b, summaries[0])
-				hrs := st.HistoricalRoots()
+				hrs, err := st.HistoricalRoots()
+				require.NoError(t, err)
 				require.DeepEqual(t, hrs, [][]byte{})
 			},
 		},

beacon-chain/core/execution/upgrade.go

@@ -35,6 +35,10 @@ func UpgradeToBellatrix(state state.BeaconState) (state.BeaconState, error) {
 		return nil, err
 	}
 
+	hrs, err := state.HistoricalRoots()
+	if err != nil {
+		return nil, err
+	}
 	s := &ethpb.BeaconStateBellatrix{
 		GenesisTime:           state.GenesisTime(),
 		GenesisValidatorsRoot: state.GenesisValidatorsRoot(),
@@ -47,7 +51,7 @@ func UpgradeToBellatrix(state state.BeaconState) (state.BeaconState, error) {
 		LatestBlockHeader:           state.LatestBlockHeader(),
 		BlockRoots:                  state.BlockRoots(),
 		StateRoots:                  state.StateRoots(),
-		HistoricalRoots:             state.HistoricalRoots(),
+		HistoricalRoots:             hrs,
 		Eth1Data:                    state.Eth1Data(),
 		Eth1DataVotes:               state.Eth1DataVotes(),
 		Eth1DepositIndex:            state.Eth1DepositIndex(),

beacon-chain/core/execution/upgrade_test.go

@@ -24,8 +24,10 @@ func TestUpgradeToBellatrix(t *testing.T) {
 	require.DeepSSZEqual(t, preForkState.LatestBlockHeader(), mSt.LatestBlockHeader())
 	require.DeepSSZEqual(t, preForkState.BlockRoots(), mSt.BlockRoots())
 	require.DeepSSZEqual(t, preForkState.StateRoots(), mSt.StateRoots())
-	r1 := preForkState.HistoricalRoots()
-	r2 := mSt.HistoricalRoots()
+	r1, err := preForkState.HistoricalRoots()
+	require.NoError(t, err)
+	r2, err := mSt.HistoricalRoots()
+	require.NoError(t, err)
 	require.DeepSSZEqual(t, r1, r2)
 	require.DeepSSZEqual(t, preForkState.Eth1Data(), mSt.Eth1Data())
 	require.DeepSSZEqual(t, preForkState.Eth1DataVotes(), mSt.Eth1DataVotes())

beacon-chain/core/feed/block/events.go

@@ -11,8 +11,6 @@ const (
 
 // ReceivedBlockData is the data sent with ReceivedBlock events.
 type ReceivedBlockData struct {
-	SignedBlock       interfaces.ReadOnlySignedBeaconBlock
-	CurrDependentRoot [32]byte
-	PrevDependentRoot [32]byte
-	IsOptimistic      bool
+	SignedBlock  interfaces.ReadOnlySignedBeaconBlock
+	IsOptimistic bool
 }

beacon-chain/core/feed/operation/events.go

@@ -37,8 +37,11 @@ const (
 	// SingleAttReceived is sent after a single attestation object is received from gossip or rpc
 	SingleAttReceived = 9
 
+	// DataColumnSidecarReceived is sent after a data column sidecar is received from gossip or rpc.
+	DataColumnSidecarReceived = 10
+
 	// BlockGossipReceived is sent after a block has been received from gossip or API that passes validation rules.
-	BlockGossipReceived = 10
+	BlockGossipReceived = 11
 )
 
 // UnAggregatedAttReceivedData is the data sent with UnaggregatedAttReceived events.
@@ -90,6 +93,11 @@ type SingleAttReceivedData struct {
 	Attestation ethpb.Att
 }
 
+// DataColumnSidecarReceivedData is the data sent with DataColumnSidecarReceived events.
+type DataColumnSidecarReceivedData struct {
+	DataColumn *blocks.VerifiedRODataColumn
+}
+
 // BlockGossipReceivedData is the data sent with BlockGossipReceived events.
 type BlockGossipReceivedData struct {
 	// SignedBlock is the block that was received.

beacon-chain/core/feed/state/events.go

@@ -43,10 +43,6 @@ type BlockProcessedData struct {
 	BlockRoot [32]byte
 	// SignedBlock is the physical processed block.
 	SignedBlock interfaces.ReadOnlySignedBeaconBlock
-	// CurrDependentRoot is the current dependent root
-	CurrDependentRoot [32]byte
-	// PrevDependentRoot is the previous dependent root
-	PrevDependentRoot [32]byte
 	// Verified is true if the block's BLS contents have been verified.
 	Verified bool
 	// Optimistic is true if the block is optimistic.

beacon-chain/core/fulu/upgrade.go

@@ -57,6 +57,10 @@ func UpgradeToFulu(beaconState state.BeaconState) (state.BeaconState, error) {
 	if err != nil {
 		return nil, err
 	}
+	historicalRoots, err := beaconState.HistoricalRoots()
+	if err != nil {
+		return nil, err
+	}
 	excessBlobGas, err := payloadHeader.ExcessBlobGas()
 	if err != nil {
 		return nil, err
@@ -114,7 +118,7 @@ func UpgradeToFulu(beaconState state.BeaconState) (state.BeaconState, error) {
 		LatestBlockHeader:           beaconState.LatestBlockHeader(),
 		BlockRoots:                  beaconState.BlockRoots(),
 		StateRoots:                  beaconState.StateRoots(),
-		HistoricalRoots:             beaconState.HistoricalRoots(),
+		HistoricalRoots:             historicalRoots,
 		Eth1Data:                    beaconState.Eth1Data(),
 		Eth1DataVotes:               beaconState.Eth1DataVotes(),
 		Eth1DepositIndex:            beaconState.Eth1DepositIndex(),

beacon-chain/core/fulu/upgrade_test.go

@@ -43,8 +43,10 @@ func TestUpgradeToFulu(t *testing.T) {
 	require.DeepSSZEqual(t, preForkState.BlockRoots(), mSt.BlockRoots())
 	require.DeepSSZEqual(t, preForkState.StateRoots(), mSt.StateRoots())
 
-	hr1 := preForkState.HistoricalRoots()
-	hr2 := mSt.HistoricalRoots()
+	hr1, err := preForkState.HistoricalRoots()
+	require.NoError(t, err)
+	hr2, err := mSt.HistoricalRoots()
+	require.NoError(t, err)
 	require.DeepEqual(t, hr1, hr2)
 
 	require.DeepSSZEqual(t, preForkState.Eth1Data(), mSt.Eth1Data())

beacon-chain/core/helpers/beacon_committee.go

@@ -18,7 +18,6 @@ import (
 	"github.com/OffchainLabs/prysm/v6/crypto/hash"
 	"github.com/OffchainLabs/prysm/v6/encoding/bytesutil"
 	"github.com/OffchainLabs/prysm/v6/math"
-	"github.com/OffchainLabs/prysm/v6/monitoring/tracing/trace"
 	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
 	"github.com/OffchainLabs/prysm/v6/runtime/version"
 	"github.com/OffchainLabs/prysm/v6/time/slots"
@@ -298,9 +297,6 @@ func verifyAssignmentEpoch(epoch primitives.Epoch, state state.BeaconState) erro
 // It verifies the validity of the epoch, then iterates through each slot in the epoch to determine the
 // proposer for that slot and assigns them accordingly.
 func ProposerAssignments(ctx context.Context, state state.BeaconState, epoch primitives.Epoch) (map[primitives.ValidatorIndex][]primitives.Slot, error) {
-	ctx, span := trace.StartSpan(ctx, "helpers.ProposerAssignments")
-	defer span.End()
-
 	// Verify if the epoch is valid for assignment based on the provided state.
 	if err := verifyAssignmentEpoch(epoch, state); err != nil {
 		return nil, err
@@ -349,9 +345,6 @@ func ProposerAssignments(ctx context.Context, state state.BeaconState, epoch pri
 // It retrieves active validator indices, determines the number of committees per slot, and computes
 // assignments for each validator based on their presence in the provided validators slice.
 func CommitteeAssignments(ctx context.Context, state state.BeaconState, epoch primitives.Epoch, validators []primitives.ValidatorIndex) (map[primitives.ValidatorIndex]*CommitteeAssignment, error) {
-	ctx, span := trace.StartSpan(ctx, "helpers.CommitteeAssignments")
-	defer span.End()
-
 	// Verify if the epoch is valid for assignment based on the provided state.
 	if err := verifyAssignmentEpoch(epoch, state); err != nil {
 		return nil, err

beacon-chain/core/helpers/sync_committee_test.go

@@ -78,6 +78,7 @@ func TestIsCurrentEpochSyncCommittee_UsingCommittee(t *testing.T) {
 
 func TestIsCurrentEpochSyncCommittee_DoesNotExist(t *testing.T) {
 	helpers.ClearCache()
+	params.SetupTestConfigCleanup(t)
 
 	validators := make([]*ethpb.Validator, params.BeaconConfig().SyncCommitteeSize)
 	syncCommittee := &ethpb.SyncCommittee{
@@ -264,6 +265,7 @@ func TestCurrentEpochSyncSubcommitteeIndices_UsingCommittee(t *testing.T) {
 }
 
 func TestCurrentEpochSyncSubcommitteeIndices_DoesNotExist(t *testing.T) {
+	params.SetupTestConfigCleanup(t)
 	helpers.ClearCache()
 
 	validators := make([]*ethpb.Validator, params.BeaconConfig().SyncCommitteeSize)

beacon-chain/core/peerdas/BUILD.bazel

@@ -7,6 +7,7 @@ go_library(
         "info.go",
         "metrics.go",
         "p2p_interface.go",
+        "peer_sampling.go",
         "reconstruction.go",
         "util.go",
         "validator.go",
@@ -44,6 +45,7 @@ go_test(
         "das_core_test.go",
         "info_test.go",
         "p2p_interface_test.go",
+        "peer_sampling_test.go",
         "reconstruction_test.go",
         "utils_test.go",
         "validator_test.go",

beacon-chain/core/peerdas/das_core.go

@@ -7,6 +7,7 @@ import (
 	"time"
 
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg"
+	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
 	"github.com/OffchainLabs/prysm/v6/config/params"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
@@ -21,11 +22,10 @@ import (
 var (
 	// Custom errors
 	ErrCustodyGroupTooLarge           = errors.New("custody group too large")
-	ErrCustodyGroupCountTooLarge      = errors.New("custody group count too large")
-	ErrMismatchSize                   = errors.New("mismatch in the number of blob KZG commitments and cellsAndProofs")
+	errCustodyGroupCountTooLarge      = errors.New("custody group count too large")
 	errWrongComputedCustodyGroupCount = errors.New("wrong computed custody group count, should never happen")
 
-	// maxUint256 is the maximum value of an uint256.
+	// maxUint256 is the maximum value of a uint256.
 	maxUint256 = &uint256.Int{math.MaxUint64, math.MaxUint64, math.MaxUint64, math.MaxUint64}
 )
 
@@ -38,29 +38,18 @@ const (
 
 // CustodyGroups computes the custody groups the node should participate in for custody.
 // https://github.com/ethereum/consensus-specs/blob/v1.5.0-beta.5/specs/fulu/das-core.md#get_custody_groups
-func CustodyGroups(nodeId enode.ID, custodyGroupCount uint64) ([]uint64, error) {
+func CustodyGroups(nodeId enode.ID, custodyGroupCount uint64) (map[uint64]bool, error) {
 	numberOfCustodyGroup := params.BeaconConfig().NumberOfCustodyGroups
 
 	// Check if the custody group count is larger than the number of custody groups.
 	if custodyGroupCount > numberOfCustodyGroup {
-		return nil, ErrCustodyGroupCountTooLarge
-	}
-
-	// Shortcut if all custody groups are needed.
-	if custodyGroupCount == numberOfCustodyGroup {
-		custodyGroups := make([]uint64, 0, numberOfCustodyGroup)
-		for i := range numberOfCustodyGroup {
-			custodyGroups = append(custodyGroups, i)
-		}
-
-		return custodyGroups, nil
+		return nil, errCustodyGroupCountTooLarge
 	}
 
+	custodyGroups := make(map[uint64]bool, custodyGroupCount)
 	one := uint256.NewInt(1)
 
-	custodyGroupsMap := make(map[uint64]bool, custodyGroupCount)
-	custodyGroups := make([]uint64, 0, custodyGroupCount)
-	for currentId := new(uint256.Int).SetBytes(nodeId.Bytes()); uint64(len(custodyGroups)) < custodyGroupCount; {
+	for currentId := new(uint256.Int).SetBytes(nodeId.Bytes()); uint64(len(custodyGroups)) < custodyGroupCount; currentId.Add(currentId, one) {
 		// Convert to big endian bytes.
 		currentIdBytesBigEndian := currentId.Bytes32()
 
@@ -71,24 +60,15 @@ func CustodyGroups(nodeId enode.ID, custodyGroupCount uint64) ([]uint64, error)
 		hashedCurrentId := hash.Hash(currentIdBytesLittleEndian)
 
 		// Get the custody group ID.
-		custodyGroup := binary.LittleEndian.Uint64(hashedCurrentId[:8]) % numberOfCustodyGroup
+		custodyGroupId := binary.LittleEndian.Uint64(hashedCurrentId[:8]) % numberOfCustodyGroup
 
 		// Add the custody group to the map.
-		if !custodyGroupsMap[custodyGroup] {
-			custodyGroupsMap[custodyGroup] = true
-			custodyGroups = append(custodyGroups, custodyGroup)
-		}
+		custodyGroups[custodyGroupId] = true
 
+		// Overflow prevention.
 		if currentId.Cmp(maxUint256) == 0 {
-			// Overflow prevention.
 			currentId = uint256.NewInt(0)
-		} else {
-			// Increment the current ID.
-			currentId.Add(currentId, one)
 		}
-
-		// Sort the custody groups.
-		slices.Sort[[]uint64](custodyGroups)
 	}
 
 	// Final check.
@@ -105,8 +85,8 @@ func ComputeColumnsForCustodyGroup(custodyGroup uint64) ([]uint64, error) {
 	beaconConfig := params.BeaconConfig()
 	numberOfCustodyGroup := beaconConfig.NumberOfCustodyGroups
 
-	if custodyGroup >= numberOfCustodyGroup {
-		return nil, ErrCustodyGroupTooLarge
+	if custodyGroup > numberOfCustodyGroup {
+		return nil, errCustodyGroupCountTooLarge
 	}
 
 	numberOfColumns := beaconConfig.NumberOfColumns
@@ -123,11 +103,10 @@ func ComputeColumnsForCustodyGroup(custodyGroup uint64) ([]uint64, error) {
 }
 
 // DataColumnSidecars computes the data column sidecars from the signed block, cells and cell proofs.
-// The returned value contains pointers to function parameters.
-// (If the caller alterates `cellsAndProofs` afterwards, the returned value will be modified as well.)
 // https://github.com/ethereum/consensus-specs/blob/v1.5.0-beta.3/specs/fulu/das-core.md#get_data_column_sidecars
 func DataColumnSidecars(signedBlock interfaces.ReadOnlySignedBeaconBlock, cellsAndProofs []kzg.CellsAndProofs) ([]*ethpb.DataColumnSidecar, error) {
-	if signedBlock == nil || signedBlock.IsNil() || len(cellsAndProofs) == 0 {
+	start := time.Now()
+	if signedBlock == nil || len(cellsAndProofs) == 0 {
 		return nil, nil
 	}
 
@@ -139,7 +118,7 @@ func DataColumnSidecars(signedBlock interfaces.ReadOnlySignedBeaconBlock, cellsA
 	}
 
 	if len(blobKzgCommitments) != len(cellsAndProofs) {
-		return nil, ErrMismatchSize
+		return nil, errors.New("mismatch in the number of blob KZG commitments and cellsAndProofs")
 	}
 
 	signedBlockHeader, err := signedBlock.Header()
@@ -152,34 +131,9 @@ func DataColumnSidecars(signedBlock interfaces.ReadOnlySignedBeaconBlock, cellsA
 		return nil, errors.Wrap(err, "merkle proof ZKG commitments")
 	}
 
-	dataColumnSidecars, err := DataColumnsSidecarsFromItems(signedBlockHeader, blobKzgCommitments, kzgCommitmentsInclusionProof, cellsAndProofs)
-	if err != nil {
-		return nil, errors.Wrap(err, "data column sidecars from items")
-	}
-
-	return dataColumnSidecars, nil
-}
-
-// DataColumnsSidecarsFromItems computes the data column sidecars from the signed block header, the blob KZG commiments,
-// the KZG commitment includion proofs and cells and cell proofs.
-// The returned value contains pointers to function parameters.
-// (If the caller alterates input parameters afterwards, the returned value will be modified as well.)
-func DataColumnsSidecarsFromItems(
-	signedBlockHeader *ethpb.SignedBeaconBlockHeader,
-	blobKzgCommitments [][]byte,
-	kzgCommitmentsInclusionProof [][]byte,
-	cellsAndProofs []kzg.CellsAndProofs,
-) ([]*ethpb.DataColumnSidecar, error) {
-	start := time.Now()
-	if len(blobKzgCommitments) != len(cellsAndProofs) {
-		return nil, ErrMismatchSize
-	}
-
-	numberOfColumns := params.BeaconConfig().NumberOfColumns
-
 	blobsCount := len(cellsAndProofs)
-	sidecars := make([]*ethpb.DataColumnSidecar, 0, numberOfColumns)
-	for columnIndex := range numberOfColumns {
+	sidecars := make([]*ethpb.DataColumnSidecar, 0, fieldparams.NumberOfColumns)
+	for columnIndex := range uint64(fieldparams.NumberOfColumns) {
 		column := make([]kzg.Cell, 0, blobsCount)
 		kzgProofOfColumn := make([]kzg.Proof, 0, blobsCount)
 
@@ -201,7 +155,8 @@ func DataColumnsSidecarsFromItems(
 
 		kzgProofOfColumnBytes := make([][]byte, 0, blobsCount)
 		for _, kzgProof := range kzgProofOfColumn {
-			kzgProofOfColumnBytes = append(kzgProofOfColumnBytes, kzgProof[:])
+			copiedProof := kzgProof
+			kzgProofOfColumnBytes = append(kzgProofOfColumnBytes, copiedProof[:])
 		}
 
 		sidecar := &ethpb.DataColumnSidecar{
@@ -220,30 +175,15 @@ func DataColumnsSidecarsFromItems(
 	return sidecars, nil
 }
 
-// ComputeCustodyGroupForColumn computes the custody group for a given column.
-// It is the reciprocal function of ComputeColumnsForCustodyGroup.
-func ComputeCustodyGroupForColumn(columnIndex uint64) (uint64, error) {
-	beaconConfig := params.BeaconConfig()
-	numberOfColumns := beaconConfig.NumberOfColumns
-	numberOfCustodyGroups := beaconConfig.NumberOfCustodyGroups
-
-	if columnIndex >= numberOfColumns {
-		return 0, ErrIndexTooLarge
-	}
-
-	return columnIndex % numberOfCustodyGroups, nil
-}
-
 // Blobs extract blobs from `dataColumnsSidecar`.
 // This can be seen as the reciprocal function of DataColumnSidecars.
 // `dataColumnsSidecar` needs to contain the datacolumns corresponding to the non-extended matrix,
 // else an error will be returned.
 // (`dataColumnsSidecar` can contain extra columns, but they will be ignored.)
 func Blobs(indices map[uint64]bool, dataColumnsSidecar []*ethpb.DataColumnSidecar) ([]*blocks.VerifiedROBlob, error) {
-	numberOfColumns := params.BeaconConfig().NumberOfColumns
+	columnCount := fieldparams.NumberOfColumns
 
-	// Compute the number of needed columns, including the number of columns is odd case.
-	neededColumnCount := (numberOfColumns + 1) / 2
+	neededColumnCount := columnCount / 2
 
 	// Check if all needed columns are present.
 	sliceIndexFromColumnIndex := make(map[uint64]int, len(dataColumnsSidecar))
@@ -251,24 +191,28 @@ func Blobs(indices map[uint64]bool, dataColumnsSidecar []*ethpb.DataColumnSideca
 		dataColumnSideCar := dataColumnsSidecar[i]
 		index := dataColumnSideCar.Index
 
-		if index < neededColumnCount {
+		if index < uint64(neededColumnCount) {
 			sliceIndexFromColumnIndex[index] = i
 		}
 	}
 
-	actualColumnCount := uint64(len(sliceIndexFromColumnIndex))
+	actualColumnCount := len(sliceIndexFromColumnIndex)
 
 	// Get missing columns.
 	if actualColumnCount < neededColumnCount {
-		var missingColumnsSlice []uint64
-
+		missingColumns := make(map[int]bool, neededColumnCount-actualColumnCount)
 		for i := range neededColumnCount {
-			if _, ok := sliceIndexFromColumnIndex[i]; !ok {
-				missingColumnsSlice = append(missingColumnsSlice, i)
+			if _, ok := sliceIndexFromColumnIndex[uint64(i)]; !ok {
+				missingColumns[i] = true
 			}
 		}
 
-		slices.Sort[[]uint64](missingColumnsSlice)
+		missingColumnsSlice := make([]int, 0, len(missingColumns))
+		for i := range missingColumns {
+			missingColumnsSlice = append(missingColumnsSlice, i)
+		}
+
+		slices.Sort[[]int](missingColumnsSlice)
 		return nil, errors.Errorf("some columns are missing: %v", missingColumnsSlice)
 	}
 
@@ -295,7 +239,7 @@ func Blobs(indices map[uint64]bool, dataColumnsSidecar []*ethpb.DataColumnSideca
 
 		// Compute the content of the blob.
 		for columnIndex := range neededColumnCount {
-			sliceIndex, ok := sliceIndexFromColumnIndex[columnIndex]
+			sliceIndex, ok := sliceIndexFromColumnIndex[uint64(columnIndex)]
 			if !ok {
 				return nil, errors.Errorf("missing column %d, this should never happen", columnIndex)
 			}
@@ -303,8 +247,8 @@ func Blobs(indices map[uint64]bool, dataColumnsSidecar []*ethpb.DataColumnSideca
 			dataColumnSideCar := dataColumnsSidecar[sliceIndex]
 			cell := dataColumnSideCar.Column[blobIndex]
 
-			for i := range cell {
-				blob[columnIndex*kzg.BytesPerCell+uint64(i)] = cell[i]
+			for i := 0; i < len(cell); i++ {
+				blob[columnIndex*kzg.BytesPerCell+i] = cell[i]
 			}
 		}
 
@@ -352,14 +296,14 @@ func (custodyInfo *CustodyInfo) CustodyGroupSamplingSize(ct CustodyType) uint64
 }
 
 // CustodyColumns computes the custody columns from the custody groups.
-func CustodyColumns(custodyGroups []uint64) (map[uint64]bool, error) {
+func CustodyColumns(custodyGroups map[uint64]bool) (map[uint64]bool, error) {
 	numberOfCustodyGroups := params.BeaconConfig().NumberOfCustodyGroups
 
 	custodyGroupCount := len(custodyGroups)
 
 	// Compute the columns for each custody group.
 	columns := make(map[uint64]bool, custodyGroupCount)
-	for _, group := range custodyGroups {
+	for group := range custodyGroups {
 		if group >= numberOfCustodyGroups {
 			return nil, ErrCustodyGroupTooLarge
 		}
@@ -388,13 +332,19 @@ func populateAndFilterIndices(indices map[uint64]bool, blobCount uint64) []uint6
 	}
 
 	// Filter blobs index higher than the blob count.
-	indicesSlice := make([]uint64, 0, len(indices))
+	filteredIndices := make(map[uint64]bool, len(indices))
 	for i := range indices {
 		if i < blobCount {
-			indicesSlice = append(indicesSlice, i)
+			filteredIndices[i] = true
 		}
 	}
 
+	// Transform set to slice.
+	indicesSlice := make([]uint64, 0, len(filteredIndices))
+	for i := range filteredIndices {
+		indicesSlice = append(indicesSlice, i)
+	}
+
 	// Sort the indices.
 	slices.Sort[[]uint64](indicesSlice)
 

beacon-chain/core/peerdas/das_core_test.go

@@ -5,115 +5,34 @@ import (
 
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
-	"github.com/OffchainLabs/prysm/v6/config/params"
+	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
 	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
 	"github.com/OffchainLabs/prysm/v6/testing/require"
 	"github.com/OffchainLabs/prysm/v6/testing/util"
-	"github.com/ethereum/go-ethereum/p2p/enode"
 	"github.com/pkg/errors"
 )
 
-func TestCustodyGroups(t *testing.T) {
-	// The happy path is unit tested in spec tests.
-	numberOfCustodyGroup := params.BeaconConfig().NumberOfCustodyGroups
-	_, err := peerdas.CustodyGroups(enode.ID{}, numberOfCustodyGroup+1)
-	require.ErrorIs(t, err, peerdas.ErrCustodyGroupCountTooLarge)
-}
-
-func TestComputeColumnsForCustodyGroup(t *testing.T) {
-	// The happy path is unit tested in spec tests.
-	numberOfCustodyGroup := params.BeaconConfig().NumberOfCustodyGroups
-	_, err := peerdas.ComputeColumnsForCustodyGroup(numberOfCustodyGroup)
-	require.ErrorIs(t, err, peerdas.ErrCustodyGroupTooLarge)
-}
+// ---------------------------------------------------------------
+// ( CustodyGroups is unit tested in spec tests.                 )
+// ( ComputeColumnsForCustodyGroup is unit tested in spec tests. )
+// ---------------------------------------------------------------
 
 func TestDataColumnSidecars(t *testing.T) {
-	t.Run("nil signed block", func(t *testing.T) {
-		var expected []*ethpb.DataColumnSidecar = nil
-		actual, err := peerdas.DataColumnSidecars(nil, []kzg.CellsAndProofs{})
-		require.NoError(t, err)
+	var expected []*ethpb.DataColumnSidecar = nil
+	actual, err := peerdas.DataColumnSidecars(nil, []kzg.CellsAndProofs{})
+	require.NoError(t, err)
 
-		require.DeepSSZEqual(t, expected, actual)
-	})
-
-	t.Run("empty cells and proofs", func(t *testing.T) {
-		// Create a protobuf signed beacon block.
-		signedBeaconBlockPb := util.NewBeaconBlockDeneb()
-
-		// Create a signed beacon block from the protobuf.
-		signedBeaconBlock, err := blocks.NewSignedBeaconBlock(signedBeaconBlockPb)
-		require.NoError(t, err)
-
-		actual, err := peerdas.DataColumnSidecars(signedBeaconBlock, []kzg.CellsAndProofs{})
-		require.NoError(t, err)
-		require.IsNil(t, actual)
-	})
-
-	t.Run("sizes mismatch", func(t *testing.T) {
-		// Create a protobuf signed beacon block.
-		signedBeaconBlockPb := util.NewBeaconBlockDeneb()
-
-		// Create a signed beacon block from the protobuf.
-		signedBeaconBlock, err := blocks.NewSignedBeaconBlock(signedBeaconBlockPb)
-		require.NoError(t, err)
-
-		// Create cells and proofs.
-		cellsAndProofs := make([]kzg.CellsAndProofs, 1)
-
-		_, err = peerdas.DataColumnSidecars(signedBeaconBlock, cellsAndProofs)
-		require.ErrorIs(t, err, peerdas.ErrMismatchSize)
-	})
-}
-
-// --------------------------------------------------------------------------------------------------------------------------------------
-// DataColumnsSidecarsFromItems is tested as part of the DataColumnSidecars tests, in the TestDataColumnsSidecarsBlobsRoundtrip function.
-// --------------------------------------------------------------------------------------------------------------------------------------
-
-func TestComputeCustodyGroupForColumn(t *testing.T) {
-	params.SetupTestConfigCleanup(t)
-	config := params.BeaconConfig()
-	config.NumberOfColumns = 128
-	config.NumberOfCustodyGroups = 64
-	params.OverrideBeaconConfig(config)
-
-	t.Run("index too large", func(t *testing.T) {
-		_, err := peerdas.ComputeCustodyGroupForColumn(1_000_000)
-		require.ErrorIs(t, err, peerdas.ErrIndexTooLarge)
-	})
-
-	t.Run("nominal", func(t *testing.T) {
-		expected := uint64(2)
-		actual, err := peerdas.ComputeCustodyGroupForColumn(2)
-		require.NoError(t, err)
-		require.Equal(t, expected, actual)
-
-		expected = uint64(3)
-		actual, err = peerdas.ComputeCustodyGroupForColumn(3)
-		require.NoError(t, err)
-		require.Equal(t, expected, actual)
-
-		expected = uint64(2)
-		actual, err = peerdas.ComputeCustodyGroupForColumn(66)
-		require.NoError(t, err)
-		require.Equal(t, expected, actual)
-
-		expected = uint64(3)
-		actual, err = peerdas.ComputeCustodyGroupForColumn(67)
-		require.NoError(t, err)
-		require.Equal(t, expected, actual)
-	})
+	require.DeepSSZEqual(t, expected, actual)
 }
 
 func TestBlobs(t *testing.T) {
 	blobsIndice := map[uint64]bool{}
 
-	numberOfColumns := params.BeaconConfig().NumberOfColumns
-
-	almostAllColumns := make([]*ethpb.DataColumnSidecar, 0, numberOfColumns/2)
-	for i := uint64(2); i < numberOfColumns/2+2; i++ {
+	almostAllColumns := make([]*ethpb.DataColumnSidecar, 0, fieldparams.NumberOfColumns/2)
+	for i := 2; i < fieldparams.NumberOfColumns/2+2; i++ {
 		almostAllColumns = append(almostAllColumns, &ethpb.DataColumnSidecar{
-			Index: i,
+			Index: uint64(i),
 		})
 	}
 
@@ -293,12 +212,12 @@ func TestCustodyGroupSamplingSize(t *testing.T) {
 
 func TestCustodyColumns(t *testing.T) {
 	t.Run("group too large", func(t *testing.T) {
-		_, err := peerdas.CustodyColumns([]uint64{1_000_000})
+		_, err := peerdas.CustodyColumns(map[uint64]bool{1_000_000: true})
 		require.ErrorIs(t, err, peerdas.ErrCustodyGroupTooLarge)
 	})
 
 	t.Run("nominal", func(t *testing.T) {
-		input := []uint64{1, 2}
+		input := map[uint64]bool{1: true, 2: true}
 		expected := map[uint64]bool{1: true, 2: true}
 
 		actual, err := peerdas.CustodyColumns(input)

beacon-chain/core/peerdas/info.go

@@ -91,14 +91,8 @@ func Info(nodeID enode.ID, custodyGroupCount uint64) (*info, bool, error) {
 	// Compute data columns subnets.
 	dataColumnsSubnets := DataColumnSubnets(custodyColumns)
 
-	// Convert the custody groups to a map.
-	custodyGroupsMap := make(map[uint64]bool, len(custodyGroups))
-	for _, group := range custodyGroups {
-		custodyGroupsMap[group] = true
-	}
-
 	result := &info{
-		CustodyGroups:      custodyGroupsMap,
+		CustodyGroups:      custodyGroups,
 		CustodyColumns:     custodyColumns,
 		DataColumnsSubnets: dataColumnsSubnets,
 	}
@@ -117,7 +111,7 @@ func (custodyInfo *CustodyInfo) ActualGroupCount() uint64 {
 // CustodyGroupCount returns the number of groups we should participate in for custody.
 func (tcgc *targetCustodyGroupCount) Get() uint64 {
 	// If subscribed to all subnets, return the number of custody groups.
-	if flags.Get().SubscribeToAllSubnets {
+	if flags.Get().SubscribeAllColumnSubnets {
 		return params.BeaconConfig().NumberOfCustodyGroups
 	}
 
@@ -144,7 +138,7 @@ func (tcgc *targetCustodyGroupCount) SetValidatorsCustodyRequirement(value uint6
 // Get returns the to advertise custody group count.
 func (tacgc *toAdverstiseCustodyGroupCount) Get() uint64 {
 	// If subscribed to all subnets, return the number of custody groups.
-	if flags.Get().SubscribeToAllSubnets {
+	if flags.Get().SubscribeAllColumnSubnets {
 		return params.BeaconConfig().NumberOfCustodyGroups
 	}
 

beacon-chain/core/peerdas/info_test.go

@@ -30,25 +30,25 @@ func TestInfo(t *testing.T) {
 func TestTargetCustodyGroupCount(t *testing.T) {
 	testCases := []struct {
 		name                         string
-		subscribeToAllSubnets        bool
+		subscribeToAllColumns        bool
 		validatorsCustodyRequirement uint64
 		expected                     uint64
 	}{
 		{
 			name:                         "subscribed to all subnets",
-			subscribeToAllSubnets:        true,
+			subscribeToAllColumns:        true,
 			validatorsCustodyRequirement: 100,
 			expected:                     128,
 		},
 		{
 			name:                         "no validators attached",
-			subscribeToAllSubnets:        false,
+			subscribeToAllColumns:        false,
 			validatorsCustodyRequirement: 0,
 			expected:                     4,
 		},
 		{
 			name:                         "some validators attached",
-			subscribeToAllSubnets:        false,
+			subscribeToAllColumns:        false,
 			validatorsCustodyRequirement: 100,
 			expected:                     100,
 		},
@@ -57,7 +57,7 @@ func TestTargetCustodyGroupCount(t *testing.T) {
 	for _, tc := range testCases {
 		t.Run(tc.name, func(t *testing.T) {
 			// Subscribe to all subnets if needed.
-			if tc.subscribeToAllSubnets {
+			if tc.subscribeToAllColumns {
 				resetFlags := flags.Get()
 				gFlags := new(flags.GlobalFlags)
 				gFlags.SubscribeToAllSubnets = true
@@ -82,25 +82,25 @@ func TestTargetCustodyGroupCount(t *testing.T) {
 func TestToAdvertiseCustodyGroupCount(t *testing.T) {
 	testCases := []struct {
 		name                         string
-		subscribeToAllSubnets        bool
+		subscribeToAllColumns        bool
 		toAdvertiseCustodyGroupCount uint64
 		expected                     uint64
 	}{
 		{
 			name:                         "subscribed to all subnets",
-			subscribeToAllSubnets:        true,
+			subscribeToAllColumns:        true,
 			toAdvertiseCustodyGroupCount: 100,
 			expected:                     128,
 		},
 		{
 			name:                         "higher than custody requirement",
-			subscribeToAllSubnets:        false,
+			subscribeToAllColumns:        false,
 			toAdvertiseCustodyGroupCount: 100,
 			expected:                     100,
 		},
 		{
 			name:                         "lower than custody requirement",
-			subscribeToAllSubnets:        false,
+			subscribeToAllColumns:        false,
 			toAdvertiseCustodyGroupCount: 1,
 			expected:                     4,
 		},
@@ -109,10 +109,10 @@ func TestToAdvertiseCustodyGroupCount(t *testing.T) {
 	for _, tc := range testCases {
 		t.Run(tc.name, func(t *testing.T) {
 			// Subscribe to all subnets if needed.
-			if tc.subscribeToAllSubnets {
+			if tc.subscribeToAllColumns {
 				resetFlags := flags.Get()
 				gFlags := new(flags.GlobalFlags)
-				gFlags.SubscribeToAllSubnets = true
+				gFlags.SubscribeAllColumnSubnets = true
 				flags.Init(gFlags)
 				defer flags.Init(resetFlags)
 			}

beacon-chain/core/peerdas/p2p_interface.go

@@ -146,6 +146,21 @@ func DataColumnSubnets(dataColumns map[uint64]bool) map[uint64]bool {
 	return subnets
 }
 
+// ComputeCustodyGroupForColumn computes the custody group for a given column.
+// It is the reciprocal function of ComputeColumnsForCustodyGroup.
+func ComputeCustodyGroupForColumn(columnIndex uint64) (uint64, error) {
+	beaconConfig := params.BeaconConfig()
+	numberOfColumns := beaconConfig.NumberOfColumns
+	numberOfCustodyGroups := beaconConfig.NumberOfCustodyGroups
+
+	if columnIndex >= numberOfColumns {
+		return 0, ErrIndexTooLarge
+	}
+
+	columnsPerGroup := numberOfColumns / numberOfCustodyGroups
+	return columnIndex / columnsPerGroup, nil
+}
+
 // CustodyGroupCountFromRecord extracts the custody group count from an ENR record.
 func CustodyGroupCountFromRecord(record *enr.Record) (uint64, error) {
 	if record == nil {

beacon-chain/core/peerdas/p2p_interface_test.go

@@ -258,6 +258,36 @@ func TestDataColumnSubnets(t *testing.T) {
 	}
 }
 
+func TestComputeCustodyGroupForColumn(t *testing.T) {
+	params.SetupTestConfigCleanup(t)
+	config := params.BeaconConfig()
+	config.NumberOfColumns = 128
+	config.NumberOfCustodyGroups = 64
+	params.OverrideBeaconConfig(config)
+
+	t.Run("index too large", func(t *testing.T) {
+		_, err := peerdas.ComputeCustodyGroupForColumn(1_000_000)
+		require.ErrorIs(t, err, peerdas.ErrIndexTooLarge)
+	})
+
+	t.Run("nominal", func(t *testing.T) {
+		expected := uint64(1)
+		actual, err := peerdas.ComputeCustodyGroupForColumn(2)
+		require.NoError(t, err)
+		require.Equal(t, expected, actual)
+
+		expected = uint64(1)
+		actual, err = peerdas.ComputeCustodyGroupForColumn(3)
+		require.NoError(t, err)
+		require.Equal(t, expected, actual)
+
+		expected = uint64(2)
+		actual, err = peerdas.ComputeCustodyGroupForColumn(4)
+		require.NoError(t, err)
+		require.Equal(t, expected, actual)
+	})
+}
+
 func TestCustodyGroupCountFromRecord(t *testing.T) {
 	t.Run("nil record", func(t *testing.T) {
 		_, err := peerdas.CustodyGroupCountFromRecord(nil)

beacon-chain/core/peerdas/peer_sampling.go

@@ -0,0 +1,56 @@
+package peerdas
+
+import (
+	"math/big"
+
+	"github.com/OffchainLabs/prysm/v6/config/params"
+)
+
+// ExtendedSampleCount computes, for a given number of samples per slot and allowed failures the
+// number of samples we should actually query from peers.
+// https://github.com/ethereum/consensus-specs/blob/v1.5.0-beta.5/specs/fulu/peer-sampling.md#get_extended_sample_count
+func ExtendedSampleCount(samplesPerSlot, allowedFailures uint64) uint64 {
+	// Retrieve the columns count
+	columnsCount := params.BeaconConfig().NumberOfColumns
+
+	// If half of the columns are missing, we are able to reconstruct the data.
+	// If half of the columns + 1 are missing, we are not able to reconstruct the data.
+	// This is the smallest worst case.
+	worstCaseMissing := columnsCount/2 + 1
+
+	// Compute the false positive threshold.
+	falsePositiveThreshold := HypergeomCDF(0, columnsCount, worstCaseMissing, samplesPerSlot)
+
+	var sampleCount uint64
+
+	// Finally, compute the extended sample count.
+	for sampleCount = samplesPerSlot; sampleCount < columnsCount+1; sampleCount++ {
+		if HypergeomCDF(allowedFailures, columnsCount, worstCaseMissing, sampleCount) <= falsePositiveThreshold {
+			break
+		}
+	}
+
+	return sampleCount
+}
+
+// HypergeomCDF computes the hypergeometric cumulative distribution function.
+// https://en.wikipedia.org/wiki/Hypergeometric_distribution
+func HypergeomCDF(k, M, n, N uint64) float64 {
+	denominatorInt := new(big.Int).Binomial(int64(M), int64(N)) // lint:ignore uintcast
+	denominator := new(big.Float).SetInt(denominatorInt)
+
+	rBig := big.NewFloat(0)
+
+	for i := uint64(0); i < k+1; i++ {
+		a := new(big.Int).Binomial(int64(n), int64(i)) // lint:ignore uintcast
+		b := new(big.Int).Binomial(int64(M-n), int64(N-i))
+		numeratorInt := new(big.Int).Mul(a, b)
+		numerator := new(big.Float).SetInt(numeratorInt)
+		item := new(big.Float).Quo(numerator, denominator)
+		rBig.Add(rBig, item)
+	}
+
+	r, _ := rBig.Float64()
+
+	return r
+}

beacon-chain/core/peerdas/peer_sampling_test.go

@@ -0,0 +1,60 @@
+package peerdas_test
+
+import (
+	"testing"
+
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
+	"github.com/OffchainLabs/prysm/v6/testing/require"
+)
+
+func TestExtendedSampleCount(t *testing.T) {
+	const samplesPerSlot = 16
+
+	testCases := []struct {
+		name                string
+		allowedMissings     uint64
+		extendedSampleCount uint64
+	}{
+		{name: "allowedMissings=0", allowedMissings: 0, extendedSampleCount: 16},
+		{name: "allowedMissings=1", allowedMissings: 1, extendedSampleCount: 20},
+		{name: "allowedMissings=2", allowedMissings: 2, extendedSampleCount: 24},
+		{name: "allowedMissings=3", allowedMissings: 3, extendedSampleCount: 27},
+		{name: "allowedMissings=4", allowedMissings: 4, extendedSampleCount: 29},
+		{name: "allowedMissings=5", allowedMissings: 5, extendedSampleCount: 32},
+		{name: "allowedMissings=6", allowedMissings: 6, extendedSampleCount: 35},
+		{name: "allowedMissings=7", allowedMissings: 7, extendedSampleCount: 37},
+		{name: "allowedMissings=8", allowedMissings: 8, extendedSampleCount: 40},
+		{name: "allowedMissings=9", allowedMissings: 9, extendedSampleCount: 42},
+		{name: "allowedMissings=10", allowedMissings: 10, extendedSampleCount: 44},
+		{name: "allowedMissings=11", allowedMissings: 11, extendedSampleCount: 47},
+		{name: "allowedMissings=12", allowedMissings: 12, extendedSampleCount: 49},
+		{name: "allowedMissings=13", allowedMissings: 13, extendedSampleCount: 51},
+		{name: "allowedMissings=14", allowedMissings: 14, extendedSampleCount: 53},
+		{name: "allowedMissings=15", allowedMissings: 15, extendedSampleCount: 55},
+		{name: "allowedMissings=16", allowedMissings: 16, extendedSampleCount: 57},
+		{name: "allowedMissings=17", allowedMissings: 17, extendedSampleCount: 59},
+		{name: "allowedMissings=18", allowedMissings: 18, extendedSampleCount: 61},
+		{name: "allowedMissings=19", allowedMissings: 19, extendedSampleCount: 63},
+		{name: "allowedMissings=20", allowedMissings: 20, extendedSampleCount: 65},
+	}
+
+	for _, tc := range testCases {
+		t.Run(tc.name, func(t *testing.T) {
+			result := peerdas.ExtendedSampleCount(samplesPerSlot, tc.allowedMissings)
+			require.Equal(t, tc.extendedSampleCount, result)
+		})
+	}
+}
+
+func TestHypergeomCDF(t *testing.T) {
+	// Test case from https://en.wikipedia.org/wiki/Hypergeometric_distribution
+	// Population size: 1000, number of successes in population: 500, sample size: 10, number of successes in sample: 5
+	// Expected result: 0.072
+	const (
+		expected = 0.0796665913283742
+		margin   = 0.000001
+	)
+
+	actual := peerdas.HypergeomCDF(5, 128, 65, 16)
+	require.Equal(t, true, expected-margin <= actual && actual <= expected+margin)
+}

beacon-chain/core/peerdas/reconstruction.go

@@ -2,6 +2,7 @@ package peerdas
 
 import (
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg"
+	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
 	"github.com/OffchainLabs/prysm/v6/config/params"
 	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
 	"github.com/pkg/errors"
@@ -13,11 +14,15 @@ func CanSelfReconstruct(custodyGroupCount uint64) bool {
 	total := params.BeaconConfig().NumberOfCustodyGroups
 	// If total is odd, then we need total / 2 + 1 columns to reconstruct.
 	// If total is even, then we need total / 2 columns to reconstruct.
-	return custodyGroupCount >= (total+1)/2
+	custodyGroupsNeeded := total/2 + total%2
+	return custodyGroupCount >= custodyGroupsNeeded
 }
 
 // RecoverCellsAndProofs recovers the cells and proofs from the data column sidecars.
-func RecoverCellsAndProofs(dataColumnSideCars []*ethpb.DataColumnSidecar) ([]kzg.CellsAndProofs, error) {
+func RecoverCellsAndProofs(
+	dataColumnSideCars []*ethpb.DataColumnSidecar,
+	blockRoot [fieldparams.RootLength]byte,
+) ([]kzg.CellsAndProofs, error) {
 	var wg errgroup.Group
 
 	dataColumnSideCarsCount := len(dataColumnSideCars)
@@ -74,3 +79,69 @@ func RecoverCellsAndProofs(dataColumnSideCars []*ethpb.DataColumnSidecar) ([]kzg
 
 	return recoveredCellsAndProofs, nil
 }
+
+// DataColumnSidecarsForReconstruct is a TEMPORARY function until there is an official specification for it.
+// It is scheduled for deletion.
+func DataColumnSidecarsForReconstruct(
+	blobKzgCommitments [][]byte,
+	signedBlockHeader *ethpb.SignedBeaconBlockHeader,
+	kzgCommitmentsInclusionProof [][]byte,
+	cellsAndProofs []kzg.CellsAndProofs,
+) ([]*ethpb.DataColumnSidecar, error) {
+	// Each CellsAndProofs corresponds to a Blob
+	// So we can get the BlobCount by checking the length of CellsAndProofs
+	blobsCount := len(cellsAndProofs)
+	if blobsCount == 0 {
+		return nil, nil
+	}
+
+	// Get the column sidecars.
+	sidecars := make([]*ethpb.DataColumnSidecar, 0, fieldparams.NumberOfColumns)
+	for columnIndex := range fieldparams.NumberOfColumns {
+		column := make([]kzg.Cell, 0, blobsCount)
+		kzgProofOfColumn := make([]kzg.Proof, 0, blobsCount)
+
+		for rowIndex := range blobsCount {
+			cellsForRow := cellsAndProofs[rowIndex].Cells
+			proofsForRow := cellsAndProofs[rowIndex].Proofs
+
+			if len(cellsForRow) != fieldparams.NumberOfColumns {
+				return nil, errors.Errorf("cells don't have the expected size: expected %d - actual %d", fieldparams.NumberOfColumns, len(cellsForRow))
+			}
+
+			cell := cellsForRow[columnIndex]
+			column = append(column, cell)
+
+			if len(proofsForRow) != fieldparams.NumberOfColumns {
+				return nil, errors.Errorf("proofs don't have the expected size: expected %d - actual %d", fieldparams.NumberOfColumns, len(proofsForRow))
+			}
+
+			kzgProof := proofsForRow[columnIndex]
+			kzgProofOfColumn = append(kzgProofOfColumn, kzgProof)
+		}
+
+		columnBytes := make([][]byte, 0, blobsCount)
+		for i := range column {
+			columnBytes = append(columnBytes, column[i][:])
+		}
+
+		kzgProofOfColumnBytes := make([][]byte, 0, blobsCount)
+		for _, kzgProof := range kzgProofOfColumn {
+			copiedProof := kzgProof
+			kzgProofOfColumnBytes = append(kzgProofOfColumnBytes, copiedProof[:])
+		}
+
+		sidecar := &ethpb.DataColumnSidecar{
+			Index:                        uint64(columnIndex),
+			Column:                       columnBytes,
+			KzgCommitments:               blobKzgCommitments,
+			KzgProofs:                    kzgProofOfColumnBytes,
+			SignedBlockHeader:            signedBlockHeader,
+			KzgCommitmentsInclusionProof: kzgCommitmentsInclusionProof,
+		}
+
+		sidecars = append(sidecars, sidecar)
+	}
+
+	return sidecars, nil
+}

beacon-chain/core/peerdas/reconstruction_test.go

@@ -3,9 +3,14 @@ package peerdas_test
 import (
 	"testing"
 
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
+	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
 	"github.com/OffchainLabs/prysm/v6/config/params"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
+	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
 	"github.com/OffchainLabs/prysm/v6/testing/require"
+	"github.com/OffchainLabs/prysm/v6/testing/util"
 )
 
 func TestCanSelfReconstruct(t *testing.T) {
@@ -55,3 +60,150 @@ func TestCanSelfReconstruct(t *testing.T) {
 		})
 	}
 }
+
+func TestReconstructionRoundTrip(t *testing.T) {
+	params.SetupTestConfigCleanup(t)
+
+	const blobCount = 5
+
+	var blockRoot [fieldparams.RootLength]byte
+
+	signedBeaconBlockPb := util.NewBeaconBlockDeneb()
+	require.NoError(t, kzg.Start())
+
+	// Generate random blobs and their corresponding commitments.
+	var (
+		blobsKzgCommitments [][]byte
+		blobs               []kzg.Blob
+	)
+	for i := range blobCount {
+		blob := getRandBlob(int64(i))
+		commitment, _, err := generateCommitmentAndProof(&blob)
+		require.NoError(t, err)
+
+		blobsKzgCommitments = append(blobsKzgCommitments, commitment[:])
+		blobs = append(blobs, blob)
+	}
+
+	// Generate a signed beacon block.
+	signedBeaconBlockPb.Block.Body.BlobKzgCommitments = blobsKzgCommitments
+	signedBeaconBlock, err := blocks.NewSignedBeaconBlock(signedBeaconBlockPb)
+	require.NoError(t, err)
+
+	// Get the signed beacon block header.
+	signedBeaconBlockHeader, err := signedBeaconBlock.Header()
+	require.NoError(t, err)
+
+	// Convert data columns sidecars from signed block and blobs.
+	cellsAndProofs := util.GenerateCellsAndProofs(t, blobs)
+	dataColumnSidecars, err := peerdas.DataColumnSidecars(signedBeaconBlock, cellsAndProofs)
+	require.NoError(t, err)
+
+	// Create verified RO data columns.
+	verifiedRoDataColumns := make([]*blocks.VerifiedRODataColumn, 0, blobCount)
+	for _, dataColumnSidecar := range dataColumnSidecars {
+		roDataColumn, err := blocks.NewRODataColumn(dataColumnSidecar)
+		require.NoError(t, err)
+
+		verifiedRoDataColumn := blocks.NewVerifiedRODataColumn(roDataColumn)
+		verifiedRoDataColumns = append(verifiedRoDataColumns, &verifiedRoDataColumn)
+	}
+
+	verifiedRoDataColumn := verifiedRoDataColumns[0]
+
+	numberOfColumns := params.BeaconConfig().NumberOfColumns
+
+	var noDataColumns []*ethpb.DataColumnSidecar
+	dataColumnsWithDifferentLengths := []*ethpb.DataColumnSidecar{
+		{Column: [][]byte{{}, {}}},
+		{Column: [][]byte{{}}},
+	}
+	notEnoughDataColumns := dataColumnSidecars[:numberOfColumns/2-1]
+	originalDataColumns := dataColumnSidecars[:numberOfColumns/2]
+	extendedDataColumns := dataColumnSidecars[numberOfColumns/2:]
+	evenDataColumns := make([]*ethpb.DataColumnSidecar, 0, numberOfColumns/2)
+	oddDataColumns := make([]*ethpb.DataColumnSidecar, 0, numberOfColumns/2)
+	allDataColumns := dataColumnSidecars
+
+	for i, dataColumn := range dataColumnSidecars {
+		if i%2 == 0 {
+			evenDataColumns = append(evenDataColumns, dataColumn)
+		} else {
+			oddDataColumns = append(oddDataColumns, dataColumn)
+		}
+	}
+
+	testCases := []struct {
+		name               string
+		dataColumnsSidecar []*ethpb.DataColumnSidecar
+		isError            bool
+	}{
+		{
+			name:               "No data columns sidecars",
+			dataColumnsSidecar: noDataColumns,
+			isError:            true,
+		},
+		{
+			name:               "Data columns sidecar with different lengths",
+			dataColumnsSidecar: dataColumnsWithDifferentLengths,
+			isError:            true,
+		},
+		{
+			name:               "All columns are present (no actual need to reconstruct)",
+			dataColumnsSidecar: allDataColumns,
+			isError:            false,
+		},
+		{
+			name:               "Only original columns are present",
+			dataColumnsSidecar: originalDataColumns,
+			isError:            false,
+		},
+		{
+			name:               "Only extended columns are present",
+			dataColumnsSidecar: extendedDataColumns,
+			isError:            false,
+		},
+		{
+			name:               "Only even columns are present",
+			dataColumnsSidecar: evenDataColumns,
+			isError:            false,
+		},
+		{
+			name:               "Only odd columns are present",
+			dataColumnsSidecar: oddDataColumns,
+			isError:            false,
+		},
+		{
+			name:               "Not enough columns to reconstruct",
+			dataColumnsSidecar: notEnoughDataColumns,
+			isError:            true,
+		},
+	}
+
+	for _, tc := range testCases {
+		t.Run(tc.name, func(t *testing.T) {
+			// Recover cells and proofs from available data columns sidecars.
+			cellsAndProofs, err := peerdas.RecoverCellsAndProofs(tc.dataColumnsSidecar, blockRoot)
+			isError := (err != nil)
+			require.Equal(t, tc.isError, isError)
+
+			if isError {
+				return
+			}
+
+			// Recover all data columns sidecars from cells and proofs.
+			reconstructedDataColumnsSideCars, err := peerdas.DataColumnSidecarsForReconstruct(
+				blobsKzgCommitments,
+				signedBeaconBlockHeader,
+				verifiedRoDataColumn.KzgCommitmentsInclusionProof,
+				cellsAndProofs,
+			)
+
+			require.NoError(t, err)
+
+			expected := dataColumnSidecars
+			actual := reconstructedDataColumnsSideCars
+			require.DeepSSZEqual(t, expected, actual)
+		})
+	}
+}

beacon-chain/core/peerdas/util.go

@@ -8,7 +8,6 @@ import (
 	"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
 	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
 	"github.com/OffchainLabs/prysm/v6/runtime/version"
-	"github.com/pkg/errors"
 )
 
 // ConstructDataColumnSidecars constructs data column sidecars from a block, blobs and their cell proofs.
@@ -19,9 +18,18 @@ func ConstructDataColumnSidecars(block interfaces.SignedBeaconBlock, blobs [][]b
 		return nil, nil
 	}
 
-	numberOfColumns := params.BeaconConfig().NumberOfColumns
-	if uint64(len(blobs))*numberOfColumns != uint64(len(cellProofs)) {
-		return nil, fmt.Errorf("number of blobs and cell proofs do not match: %d * %d != %d", len(blobs), numberOfColumns, len(cellProofs))
+	cellsAndProofs, err := constructCellsAndProofs(blobs, cellProofs)
+	if err != nil {
+		return nil, err
+	}
+
+	return DataColumnSidecars(block, cellsAndProofs)
+}
+
+func constructCellsAndProofs(blobs [][]byte, cellProofs [][]byte) ([]kzg.CellsAndProofs, error) {
+	numColumns := int(params.BeaconConfig().NumberOfColumns)
+	if len(blobs)*numColumns != len(cellProofs) {
+		return nil, fmt.Errorf("number of blobs and cell proofs do not match: %d * %d != %d", len(blobs), numColumns, len(cellProofs))
 	}
 
 	cellsAndProofs := make([]kzg.CellsAndProofs, 0, len(blobs))
@@ -35,7 +43,7 @@ func ConstructDataColumnSidecars(block interfaces.SignedBeaconBlock, blobs [][]b
 		}
 
 		var proofs []kzg.Proof
-		for idx := uint64(i) * numberOfColumns; idx < (uint64(i)+1)*numberOfColumns; idx++ {
+		for idx := i * numColumns; idx < (i+1)*numColumns; idx++ {
 			proofs = append(proofs, kzg.Proof(cellProofs[idx]))
 		}
 
@@ -45,10 +53,5 @@ func ConstructDataColumnSidecars(block interfaces.SignedBeaconBlock, blobs [][]b
 		})
 	}
 
-	dataColumnSidecars, err := DataColumnSidecars(block, cellsAndProofs)
-	if err != nil {
-		return nil, errors.Wrap(err, "data column sidcars")
-	}
-
-	return dataColumnSidecars, nil
+	return cellsAndProofs, nil
 }

beacon-chain/core/peerdas/validator.go

@@ -12,12 +12,12 @@ import (
 func ValidatorsCustodyRequirement(state beaconState.ReadOnlyBeaconState, validatorsIndex map[primitives.ValidatorIndex]bool) (uint64, error) {
 	totalNodeBalance := uint64(0)
 	for index := range validatorsIndex {
-		validator, err := state.ValidatorAtIndexReadOnly(index)
+		balance, err := state.BalanceAtIndex(index)
 		if err != nil {
-			return 0, errors.Wrapf(err, "validator at index %v", index)
+			return 0, errors.Wrapf(err, "balance at index for validator index %v", index)
 		}
 
-		totalNodeBalance += validator.EffectiveBalance()
+		totalNodeBalance += balance
 	}
 
 	beaconConfig := params.BeaconConfig()

beacon-chain/core/peerdas/validator_test.go

@@ -30,13 +30,9 @@ func TestValidatorsCustodyRequirement(t *testing.T) {
 
 	for _, tc := range testCases {
 		t.Run(tc.name, func(t *testing.T) {
-			validators := make([]*ethpb.Validator, 0, tc.count)
+			balances := make([]uint64, 0, tc.count)
 			for range tc.count {
-				validator := &ethpb.Validator{
-					EffectiveBalance: balance,
-				}
-
-				validators = append(validators, validator)
+				balances = append(balances, balance)
 			}
 
 			validatorsIndex := make(map[primitives.ValidatorIndex]bool)
@@ -44,7 +40,7 @@ func TestValidatorsCustodyRequirement(t *testing.T) {
 				validatorsIndex[primitives.ValidatorIndex(i)] = true
 			}
 
-			beaconState, err := state_native.InitializeFromProtoFulu(&ethpb.BeaconStateElectra{Validators: validators})
+			beaconState, err := state_native.InitializeFromProtoFulu(&ethpb.BeaconStateElectra{Balances: balances})
 			require.NoError(t, err)
 
 			actual, err := peerdas.ValidatorsCustodyRequirement(beaconState, validatorsIndex)

beacon-chain/core/time/slot_epoch.go

@@ -53,6 +53,11 @@ func HigherEqualThanAltairVersionAndEpoch(s state.BeaconState, e primitives.Epoc
 	return s.Version() >= version.Altair && e >= params.BeaconConfig().AltairForkEpoch
 }
 
+// PeerDASIsActive checks whether peerDAS is active at the provided slot.
+func PeerDASIsActive(slot primitives.Slot) bool {
+	return params.FuluEnabled() && slots.ToEpoch(slot) >= params.BeaconConfig().FuluForkEpoch
+}
+
 // CanUpgradeToAltair returns true if the input `slot` can upgrade to Altair.
 // Spec code:
 // If state.slot % SLOTS_PER_EPOCH == 0 and compute_epoch_at_slot(state.slot) == ALTAIR_FORK_EPOCH

beacon-chain/das/BUILD.bazel

@@ -4,21 +4,26 @@ go_library(
     name = "go_default_library",
     srcs = [
         "availability.go",
+        "availability_columns.go",
         "cache.go",
+        "data_column_cache.go",
         "iface.go",
         "mock.go",
     ],
     importpath = "github.com/OffchainLabs/prysm/v6/beacon-chain/das",
     visibility = ["//visibility:public"],
     deps = [
+        "//beacon-chain/core/peerdas:go_default_library",
         "//beacon-chain/db/filesystem:go_default_library",
         "//beacon-chain/verification:go_default_library",
+        "//config/fieldparams:go_default_library",
         "//config/params:go_default_library",
         "//consensus-types/blocks:go_default_library",
         "//consensus-types/primitives:go_default_library",
         "//runtime/logging:go_default_library",
         "//runtime/version:go_default_library",
         "//time/slots:go_default_library",
+        "@com_github_ethereum_go_ethereum//p2p/enode:go_default_library",
         "@com_github_pkg_errors//:go_default_library",
         "@com_github_sirupsen_logrus//:go_default_library",
     ],
@@ -27,13 +32,18 @@ go_library(
 go_test(
     name = "go_default_test",
     srcs = [
+        "availability_columns_test.go",
         "availability_test.go",
         "cache_test.go",
+        "data_column_cache_test.go",
     ],
     embed = [":go_default_library"],
     deps = [
+        "//beacon-chain/core/peerdas:go_default_library",
         "//beacon-chain/db/filesystem:go_default_library",
         "//beacon-chain/verification:go_default_library",
+        "//cmd/beacon-chain/flags:go_default_library",
+        "//config/fieldparams:go_default_library",
         "//config/params:go_default_library",
         "//consensus-types/blocks:go_default_library",
         "//consensus-types/primitives:go_default_library",
@@ -41,6 +51,7 @@ go_test(
         "//testing/require:go_default_library",
         "//testing/util:go_default_library",
         "//time/slots:go_default_library",
+        "@com_github_ethereum_go_ethereum//p2p/enode:go_default_library",
         "@com_github_pkg_errors//:go_default_library",
     ],
 )

beacon-chain/das/availability.go

@@ -53,10 +53,16 @@ func NewLazilyPersistentStore(store *filesystem.BlobStorage, verifier BlobBatchV
 // Persist adds blobs to the working blob cache. Blobs stored in this cache will be persisted
 // for at least as long as the node is running. Once IsDataAvailable succeeds, all blobs referenced
 // by the given block are guaranteed to be persisted for the remainder of the retention period.
-func (s *LazilyPersistentStore) Persist(current primitives.Slot, sc ...blocks.ROBlob) error {
-	if len(sc) == 0 {
+func (s *LazilyPersistentStore) Persist(current primitives.Slot, scg ...blocks.ROSidecar) error {
+	if len(scg) == 0 {
 		return nil
 	}
+
+	sc, err := blocks.BlobSidecarsFromSidecars(scg)
+	if err != nil {
+		return errors.Wrap(err, "blob sidecars from sidecars")
+	}
+
 	if len(sc) > 1 {
 		first := sc[0].BlockRoot()
 		for i := 1; i < len(sc); i++ {

beacon-chain/das/availability_columns.go

@@ -0,0 +1,200 @@
+package das
+
+import (
+	"context"
+
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/db/filesystem"
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/verification"
+	"github.com/OffchainLabs/prysm/v6/config/params"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
+	"github.com/OffchainLabs/prysm/v6/runtime/version"
+	"github.com/OffchainLabs/prysm/v6/time/slots"
+	"github.com/ethereum/go-ethereum/p2p/enode"
+	errors "github.com/pkg/errors"
+)
+
+// LazilyPersistentStoreColumn is an implementation of AvailabilityStore to be used when batch syncing data columns.
+// This implementation will hold any data columns passed to Persist until the IsDataAvailable is called for their
+// block, at which time they will undergo full verification and be saved to the disk.
+type LazilyPersistentStoreColumn struct {
+	store                  *filesystem.DataColumnStorage
+	nodeID                 enode.ID
+	cache                  *dataColumnCache
+	custodyInfo            *peerdas.CustodyInfo
+	newDataColumnsVerifier verification.NewDataColumnsVerifier
+}
+
+var _ AvailabilityStore = &LazilyPersistentStoreColumn{}
+
+// DataColumnsVerifier enables LazilyPersistentStoreColumn to manage the verification process
+// going from RODataColumn->VerifiedRODataColumn, while avoiding the decision of which individual verifications
+// to run and in what order. Since LazilyPersistentStoreColumn always tries to verify and save data columns only when
+// they are all available, the interface takes a slice of data column sidecars.
+type DataColumnsVerifier interface {
+	VerifiedRODataColumns(ctx context.Context, blk blocks.ROBlock, scs []blocks.RODataColumn) ([]blocks.VerifiedRODataColumn, error)
+}
+
+func NewLazilyPersistentStoreColumn(store *filesystem.DataColumnStorage, nodeID enode.ID, newDataColumnsVerifier verification.NewDataColumnsVerifier, custodyInfo *peerdas.CustodyInfo) *LazilyPersistentStoreColumn {
+	return &LazilyPersistentStoreColumn{
+		store:                  store,
+		nodeID:                 nodeID,
+		cache:                  newDataColumnCache(),
+		custodyInfo:            custodyInfo,
+		newDataColumnsVerifier: newDataColumnsVerifier,
+	}
+}
+
+// PersistColumns adds columns to the working column cache. Columns stored in this cache will be persisted
+// for at least as long as the node is running. Once IsDataAvailable succeeds, all columns referenced
+// by the given block are guaranteed to be persisted for the remainder of the retention period.
+func (s *LazilyPersistentStoreColumn) Persist(current primitives.Slot, sidecars ...blocks.ROSidecar) error {
+	if len(sidecars) == 0 {
+		return nil
+	}
+
+	dataColumnSidecars, err := blocks.DataColumnSidecarsFromSidecars(sidecars)
+	if err != nil {
+		return errors.Wrap(err, "blob sidecars from sidecars")
+	}
+
+	// It is safe to retrieve the first sidecar.
+	firstSidecar := dataColumnSidecars[0]
+
+	if len(sidecars) > 1 {
+		firstRoot := firstSidecar.BlockRoot()
+		for _, sidecar := range dataColumnSidecars[1:] {
+			if sidecar.BlockRoot() != firstRoot {
+				return errMixedRoots
+			}
+		}
+	}
+
+	firstSidecarEpoch, currentEpoch := slots.ToEpoch(firstSidecar.Slot()), slots.ToEpoch(current)
+	if !params.WithinDAPeriod(firstSidecarEpoch, currentEpoch) {
+		return nil
+	}
+
+	key := dataColumnCacheKey{slot: firstSidecar.Slot(), root: firstSidecar.BlockRoot()}
+	entry := s.cache.ensure(key)
+
+	for i := range sidecars {
+		if err := entry.stash(&dataColumnSidecars[i]); err != nil {
+			return errors.Wrap(err, "stash DataColumnSidecar")
+		}
+	}
+
+	return nil
+}
+
+// IsDataAvailable returns nil if all the commitments in the given block are persisted to the db and have been verified.
+// DataColumnsSidecars already in the db are assumed to have been previously verified against the block.
+func (s *LazilyPersistentStoreColumn) IsDataAvailable(ctx context.Context, currentSlot primitives.Slot, block blocks.ROBlock) error {
+	blockCommitments, err := s.fullCommitmentsToCheck(s.nodeID, block, currentSlot)
+	if err != nil {
+		return errors.Wrapf(err, "full commitments to check with block root `%#x` and current slot `%d`", block.Root(), currentSlot)
+	}
+
+	// Return early for blocks that do not have any commitments.
+	if blockCommitments.count() == 0 {
+		return nil
+	}
+
+	// Get the root of the block.
+	blockRoot := block.Root()
+
+	// Build the cache key for the block.
+	key := dataColumnCacheKey{slot: block.Block().Slot(), root: blockRoot}
+
+	// Retrieve the cache entry for the block, or create an empty one if it doesn't exist.
+	entry := s.cache.ensure(key)
+
+	// Delete the cache entry for the block at the end.
+	defer s.cache.delete(key)
+
+	// Set the disk summary for the block in the cache entry.
+	entry.setDiskSummary(s.store.Summary(blockRoot))
+
+	// Verify we have all the expected sidecars, and fail fast if any are missing or inconsistent.
+	// We don't try to salvage problematic batches because this indicates a misbehaving peer and we'd rather
+	// ignore their response and decrease their peer score.
+	roDataColumns, err := entry.filter(blockRoot, blockCommitments)
+	if err != nil {
+		return errors.Wrap(err, "filter")
+	}
+
+	// https://github.com/ethereum/consensus-specs/blob/dev/specs/fulu/p2p-interface.md#datacolumnsidecarsbyrange-v1
+	verifier := s.newDataColumnsVerifier(roDataColumns, verification.ByRangeRequestDataColumnSidecarRequirements)
+
+	if err := verifier.Valid(); err != nil {
+		return errors.Wrap(err, "valid")
+	}
+
+	if err := verifier.SidecarInclusionProven(); err != nil {
+		return errors.Wrap(err, "sidecar inclusion proven")
+	}
+
+	if err := verifier.SidecarKzgProofVerified(); err != nil {
+		return errors.Wrap(err, "sidecar KZG proof verified")
+	}
+
+	verifiedRoDataColumns, err := verifier.VerifiedRODataColumns()
+	if err != nil {
+		return errors.Wrap(err, "verified RO data columns - should never happen")
+	}
+
+	if err := s.store.Save(verifiedRoDataColumns); err != nil {
+		return errors.Wrap(err, "save data column sidecars")
+	}
+
+	return nil
+}
+
+// fullCommitmentsToCheck returns the commitments to check for a given block.
+func (s *LazilyPersistentStoreColumn) fullCommitmentsToCheck(nodeID enode.ID, block blocks.ROBlock, currentSlot primitives.Slot) (*safeCommitmentsArray, error) {
+	// Return early for blocks that are pre-Fulu.
+	if block.Version() < version.Fulu {
+		return &safeCommitmentsArray{}, nil
+	}
+
+	// Compute the block epoch.
+	blockSlot := block.Block().Slot()
+	blockEpoch := slots.ToEpoch(blockSlot)
+
+	// Compute the current spoch.
+	currentEpoch := slots.ToEpoch(currentSlot)
+
+	// Return early if the request is out of the MIN_EPOCHS_FOR_DATA_COLUMN_SIDECARS_REQUESTS window.
+	if !params.WithinDAPeriod(blockEpoch, currentEpoch) {
+		return &safeCommitmentsArray{}, nil
+	}
+
+	// Retrieve the KZG commitments for the block.
+	kzgCommitments, err := block.Block().Body().BlobKzgCommitments()
+	if err != nil {
+		return nil, errors.Wrap(err, "blob KZG commitments")
+	}
+
+	// Return early if there are no commitments in the block.
+	if len(kzgCommitments) == 0 {
+		return &safeCommitmentsArray{}, nil
+	}
+
+	// Retrieve the groups count.
+	custodyGroupCount := s.custodyInfo.ActualGroupCount()
+
+	// Retrieve peer info.
+	peerInfo, _, err := peerdas.Info(nodeID, custodyGroupCount)
+	if err != nil {
+		return nil, errors.Wrap(err, "peer info")
+	}
+
+	// Create a safe commitments array for the custody columns.
+	commitmentsArray := &safeCommitmentsArray{}
+	for column := range peerInfo.CustodyColumns {
+		commitmentsArray[column] = kzgCommitments
+	}
+
+	return commitmentsArray, nil
+}

beacon-chain/das/availability_columns_test.go

@@ -0,0 +1,301 @@
+package das
+
+import (
+	"context"
+	"testing"
+
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/db/filesystem"
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/verification"
+	"github.com/OffchainLabs/prysm/v6/cmd/beacon-chain/flags"
+	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
+	"github.com/OffchainLabs/prysm/v6/config/params"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
+	"github.com/OffchainLabs/prysm/v6/encoding/bytesutil"
+	"github.com/OffchainLabs/prysm/v6/testing/require"
+	"github.com/OffchainLabs/prysm/v6/testing/util"
+	"github.com/OffchainLabs/prysm/v6/time/slots"
+	"github.com/ethereum/go-ethereum/p2p/enode"
+)
+
+func roSidecarsFromDataColumnParamsByBlockRoot(t *testing.T, dataColumnParamsByBlockRoot verification.DataColumnsParamsByRoot) ([]blocks.ROSidecar, []blocks.RODataColumn) {
+	roDataColumns, _ := verification.CreateTestVerifiedRoDataColumnSidecars(t, dataColumnParamsByBlockRoot)
+
+	roSidecars := make([]blocks.ROSidecar, 0, len(roDataColumns))
+	for _, roDataColumn := range roDataColumns {
+		roSidecars = append(roSidecars, blocks.NewSidecarFromDataColumnSidecar(roDataColumn))
+	}
+
+	return roSidecars, roDataColumns
+}
+
+func newSignedRoBlock(t *testing.T, signedBeaconBlock interface{}) blocks.ROBlock {
+	sb, err := blocks.NewSignedBeaconBlock(signedBeaconBlock)
+	require.NoError(t, err)
+
+	rb, err := blocks.NewROBlock(sb)
+	require.NoError(t, err)
+
+	return rb
+}
+
+var commitments = [][]byte{
+	bytesutil.PadTo([]byte("a"), 48),
+	bytesutil.PadTo([]byte("b"), 48),
+	bytesutil.PadTo([]byte("c"), 48),
+	bytesutil.PadTo([]byte("d"), 48),
+}
+
+func TestPersist(t *testing.T) {
+	t.Run("no sidecars", func(t *testing.T) {
+		dataColumnStorage := filesystem.NewEphemeralDataColumnStorage(t)
+		lazilyPersistentStoreColumns := NewLazilyPersistentStoreColumn(dataColumnStorage, enode.ID{}, nil, &peerdas.CustodyInfo{})
+		err := lazilyPersistentStoreColumns.Persist(0)
+		require.NoError(t, err)
+		require.Equal(t, 0, len(lazilyPersistentStoreColumns.cache.entries))
+	})
+
+	t.Run("mixed roots", func(t *testing.T) {
+		dataColumnStorage := filesystem.NewEphemeralDataColumnStorage(t)
+
+		dataColumnParamsByBlockRoot := map[[fieldparams.RootLength]byte][]verification.DataColumnParams{
+			{1}: {{ColumnIndex: 1}},
+			{2}: {{ColumnIndex: 2}},
+		}
+
+		roSidecars, _ := roSidecarsFromDataColumnParamsByBlockRoot(t, dataColumnParamsByBlockRoot)
+		lazilyPersistentStoreColumns := NewLazilyPersistentStoreColumn(dataColumnStorage, enode.ID{}, nil, &peerdas.CustodyInfo{})
+
+		err := lazilyPersistentStoreColumns.Persist(0, roSidecars...)
+		require.ErrorIs(t, err, errMixedRoots)
+		require.Equal(t, 0, len(lazilyPersistentStoreColumns.cache.entries))
+	})
+
+	t.Run("outside DA period", func(t *testing.T) {
+		dataColumnStorage := filesystem.NewEphemeralDataColumnStorage(t)
+
+		dataColumnParamsByBlockRoot := map[[fieldparams.RootLength]byte][]verification.DataColumnParams{
+			{1}: {{ColumnIndex: 1}},
+		}
+
+		roSidecars, _ := roSidecarsFromDataColumnParamsByBlockRoot(t, dataColumnParamsByBlockRoot)
+		lazilyPersistentStoreColumns := NewLazilyPersistentStoreColumn(dataColumnStorage, enode.ID{}, nil, &peerdas.CustodyInfo{})
+
+		err := lazilyPersistentStoreColumns.Persist(1_000_000, roSidecars...)
+		require.NoError(t, err)
+		require.Equal(t, 0, len(lazilyPersistentStoreColumns.cache.entries))
+	})
+
+	t.Run("nominal", func(t *testing.T) {
+		dataColumnStorage := filesystem.NewEphemeralDataColumnStorage(t)
+
+		dataColumnParamsByBlockRoot := map[[fieldparams.RootLength]byte][]verification.DataColumnParams{
+			{}: {{ColumnIndex: 1}, {ColumnIndex: 5}},
+		}
+
+		roSidecars, roDataColumns := roSidecarsFromDataColumnParamsByBlockRoot(t, dataColumnParamsByBlockRoot)
+		lazilyPersistentStoreColumns := NewLazilyPersistentStoreColumn(dataColumnStorage, enode.ID{}, nil, &peerdas.CustodyInfo{})
+
+		err := lazilyPersistentStoreColumns.Persist(0, roSidecars...)
+		require.NoError(t, err)
+		require.Equal(t, 1, len(lazilyPersistentStoreColumns.cache.entries))
+
+		key := dataColumnCacheKey{slot: 0, root: [32]byte{}}
+		entry := lazilyPersistentStoreColumns.cache.entries[key]
+
+		// A call to Persist does NOT save the sidecars to disk.
+		require.Equal(t, uint64(0), entry.diskSummary.Count())
+
+		require.DeepSSZEqual(t, roDataColumns[0], *entry.scs[1])
+		require.DeepSSZEqual(t, roDataColumns[1], *entry.scs[5])
+
+		for i, roDataColumn := range entry.scs {
+			if map[int]bool{1: true, 5: true}[i] {
+				continue
+			}
+
+			require.IsNil(t, roDataColumn)
+		}
+	})
+}
+
+func TestIsDataAvailable(t *testing.T) {
+	newDataColumnsVerifier := func(dataColumnSidecars []blocks.RODataColumn, _ []verification.Requirement) verification.DataColumnsVerifier {
+		return &mockDataColumnsVerifier{t: t, dataColumnSidecars: dataColumnSidecars}
+	}
+
+	ctx := context.Background()
+
+	t.Run("without commitments", func(t *testing.T) {
+		signedBeaconBlockFulu := util.NewBeaconBlockFulu()
+		signedRoBlock := newSignedRoBlock(t, signedBeaconBlockFulu)
+
+		dataColumnStorage := filesystem.NewEphemeralDataColumnStorage(t)
+		lazilyPersistentStoreColumns := NewLazilyPersistentStoreColumn(dataColumnStorage, enode.ID{}, newDataColumnsVerifier, &peerdas.CustodyInfo{})
+
+		err := lazilyPersistentStoreColumns.IsDataAvailable(ctx, 0 /*current slot*/, signedRoBlock)
+		require.NoError(t, err)
+	})
+
+	t.Run("with commitments", func(t *testing.T) {
+		signedBeaconBlockFulu := util.NewBeaconBlockFulu()
+		signedBeaconBlockFulu.Block.Body.BlobKzgCommitments = commitments
+		signedRoBlock := newSignedRoBlock(t, signedBeaconBlockFulu)
+		root := signedRoBlock.Root()
+
+		dataColumnStorage := filesystem.NewEphemeralDataColumnStorage(t)
+		lazilyPersistentStoreColumns := NewLazilyPersistentStoreColumn(dataColumnStorage, enode.ID{}, newDataColumnsVerifier, &peerdas.CustodyInfo{})
+
+		indices := [...]uint64{1, 17, 87, 102}
+		dataColumnsParams := make([]verification.DataColumnParams, 0, len(indices))
+		for _, index := range indices {
+			dataColumnParams := verification.DataColumnParams{
+				ColumnIndex:    index,
+				KzgCommitments: commitments,
+			}
+
+			dataColumnsParams = append(dataColumnsParams, dataColumnParams)
+		}
+
+		dataColumnsParamsByBlockRoot := verification.DataColumnsParamsByRoot{root: dataColumnsParams}
+		_, verifiedRoDataColumns := verification.CreateTestVerifiedRoDataColumnSidecars(t, dataColumnsParamsByBlockRoot)
+
+		key := dataColumnCacheKey{root: root}
+		entry := lazilyPersistentStoreColumns.cache.ensure(key)
+		defer lazilyPersistentStoreColumns.cache.delete(key)
+
+		for _, verifiedRoDataColumn := range verifiedRoDataColumns {
+			err := entry.stash(&verifiedRoDataColumn.RODataColumn)
+			require.NoError(t, err)
+		}
+
+		err := lazilyPersistentStoreColumns.IsDataAvailable(ctx, 0 /*current slot*/, signedRoBlock)
+		require.NoError(t, err)
+
+		actual, err := dataColumnStorage.Get(root, indices[:])
+		require.NoError(t, err)
+
+		summary := dataColumnStorage.Summary(root)
+		require.Equal(t, uint64(len(indices)), summary.Count())
+		require.DeepSSZEqual(t, verifiedRoDataColumns, actual)
+	})
+}
+
+func TestFullCommitmentsToCheck(t *testing.T) {
+	windowSlots, err := slots.EpochEnd(params.BeaconConfig().MinEpochsForDataColumnSidecarsRequest)
+	require.NoError(t, err)
+
+	testCases := []struct {
+		name        string
+		commitments [][]byte
+		block       func(*testing.T) blocks.ROBlock
+		slot        primitives.Slot
+	}{
+		{
+			name: "Pre-Fulu block",
+			block: func(t *testing.T) blocks.ROBlock {
+				return newSignedRoBlock(t, util.NewBeaconBlockElectra())
+			},
+		},
+		{
+			name: "Commitments outside data availability window",
+			block: func(t *testing.T) blocks.ROBlock {
+				beaconBlockElectra := util.NewBeaconBlockElectra()
+
+				// Block is from slot 0, "current slot" is window size +1 (so outside the window)
+				beaconBlockElectra.Block.Body.BlobKzgCommitments = commitments
+
+				return newSignedRoBlock(t, beaconBlockElectra)
+			},
+			slot: windowSlots + 1,
+		},
+		{
+			name: "Commitments within data availability window",
+			block: func(t *testing.T) blocks.ROBlock {
+				signedBeaconBlockFulu := util.NewBeaconBlockFulu()
+				signedBeaconBlockFulu.Block.Body.BlobKzgCommitments = commitments
+				signedBeaconBlockFulu.Block.Slot = 100
+
+				return newSignedRoBlock(t, signedBeaconBlockFulu)
+			},
+			commitments: commitments,
+			slot:        100,
+		},
+	}
+
+	for _, tc := range testCases {
+		t.Run(tc.name, func(t *testing.T) {
+			resetFlags := flags.Get()
+			gFlags := new(flags.GlobalFlags)
+			gFlags.SubscribeAllColumnSubnets = true
+			flags.Init(gFlags)
+			defer flags.Init(resetFlags)
+
+			b := tc.block(t)
+			s := NewLazilyPersistentStoreColumn(nil, enode.ID{}, nil, &peerdas.CustodyInfo{})
+
+			commitmentsArray, err := s.fullCommitmentsToCheck(enode.ID{}, b, tc.slot)
+			require.NoError(t, err)
+
+			for _, commitments := range commitmentsArray {
+				require.DeepEqual(t, tc.commitments, commitments)
+			}
+		})
+	}
+}
+
+type mockDataColumnsVerifier struct {
+	t                                                                        *testing.T
+	dataColumnSidecars                                                       []blocks.RODataColumn
+	validCalled, SidecarInclusionProvenCalled, SidecarKzgProofVerifiedCalled bool
+}
+
+var _ verification.DataColumnsVerifier = &mockDataColumnsVerifier{}
+
+func (m *mockDataColumnsVerifier) VerifiedRODataColumns() ([]blocks.VerifiedRODataColumn, error) {
+	require.Equal(m.t, true, m.validCalled && m.SidecarInclusionProvenCalled && m.SidecarKzgProofVerifiedCalled)
+
+	verifiedDataColumnSidecars := make([]blocks.VerifiedRODataColumn, 0, len(m.dataColumnSidecars))
+	for _, dataColumnSidecar := range m.dataColumnSidecars {
+		verifiedDataColumnSidecar := blocks.NewVerifiedRODataColumn(dataColumnSidecar)
+		verifiedDataColumnSidecars = append(verifiedDataColumnSidecars, verifiedDataColumnSidecar)
+	}
+
+	return verifiedDataColumnSidecars, nil
+}
+
+func (m *mockDataColumnsVerifier) SatisfyRequirement(verification.Requirement) {}
+
+func (m *mockDataColumnsVerifier) Valid() error {
+	m.validCalled = true
+	return nil
+}
+
+func (m *mockDataColumnsVerifier) CorrectSubnet(expectedTopics []string) error      { return nil }
+func (m *mockDataColumnsVerifier) NotFromFutureSlot() error                         { return nil }
+func (m *mockDataColumnsVerifier) SlotAboveFinalized() error                        { return nil }
+func (m *mockDataColumnsVerifier) ValidProposerSignature(ctx context.Context) error { return nil }
+
+func (m *mockDataColumnsVerifier) SidecarParentSeen(parentSeen func([fieldparams.RootLength]byte) bool) error {
+	return nil
+}
+
+func (m *mockDataColumnsVerifier) SidecarParentValid(badParent func([fieldparams.RootLength]byte) bool) error {
+	return nil
+}
+
+func (m *mockDataColumnsVerifier) SidecarParentSlotLower() error       { return nil }
+func (m *mockDataColumnsVerifier) SidecarDescendsFromFinalized() error { return nil }
+
+func (m *mockDataColumnsVerifier) SidecarInclusionProven() error {
+	m.SidecarInclusionProvenCalled = true
+	return nil
+}
+
+func (m *mockDataColumnsVerifier) SidecarKzgProofVerified() error {
+	m.SidecarKzgProofVerifiedCalled = true
+	return nil
+}
+
+func (m *mockDataColumnsVerifier) SidecarProposerExpected(ctx context.Context) error { return nil }

beacon-chain/das/availability_test.go

@@ -116,9 +116,11 @@ func TestLazilyPersistent_Missing(t *testing.T) {
 	ctx := context.Background()
 	store := filesystem.NewEphemeralBlobStorage(t)
 
-	blk, scs := util.GenerateTestDenebBlockWithSidecar(t, [32]byte{}, 1, 3)
+	blk, blobSidecars := util.GenerateTestDenebBlockWithSidecar(t, [32]byte{}, 1, 3)
 
-	mbv := &mockBlobBatchVerifier{t: t, scs: scs}
+	scs := blocks.NewSidecarsFromBlobSidecars(blobSidecars)
+
+	mbv := &mockBlobBatchVerifier{t: t, scs: blobSidecars}
 	as := NewLazilyPersistentStore(store, mbv)
 
 	// Only one commitment persisted, should return error with other indices
@@ -141,12 +143,14 @@ func TestLazilyPersistent_Mismatch(t *testing.T) {
 	ctx := context.Background()
 	store := filesystem.NewEphemeralBlobStorage(t)
 
-	blk, scs := util.GenerateTestDenebBlockWithSidecar(t, [32]byte{}, 1, 3)
+	blk, blobSidecars := util.GenerateTestDenebBlockWithSidecar(t, [32]byte{}, 1, 3)
 
 	mbv := &mockBlobBatchVerifier{t: t, err: errors.New("kzg check should not run")}
-	scs[0].KzgCommitment = bytesutil.PadTo([]byte("nope"), 48)
+	blobSidecars[0].KzgCommitment = bytesutil.PadTo([]byte("nope"), 48)
 	as := NewLazilyPersistentStore(store, mbv)
 
+	scs := blocks.NewSidecarsFromBlobSidecars(blobSidecars)
+
 	// Only one commitment persisted, should return error with other indices
 	require.NoError(t, as.Persist(1, scs[0]))
 	err := as.IsDataAvailable(ctx, 1, blk)
@@ -155,7 +159,10 @@ func TestLazilyPersistent_Mismatch(t *testing.T) {
 }
 
 func TestLazyPersistOnceCommitted(t *testing.T) {
-	_, scs := util.GenerateTestDenebBlockWithSidecar(t, [32]byte{}, 1, 6)
+	_, blobSidecars := util.GenerateTestDenebBlockWithSidecar(t, [32]byte{}, 1, 6)
+
+	scs := blocks.NewSidecarsFromBlobSidecars(blobSidecars)
+
 	as := NewLazilyPersistentStore(filesystem.NewEphemeralBlobStorage(t), &mockBlobBatchVerifier{})
 	// stashes as expected
 	require.NoError(t, as.Persist(1, scs...))
@@ -163,10 +170,13 @@ func TestLazyPersistOnceCommitted(t *testing.T) {
 	require.ErrorIs(t, as.Persist(1, scs...), ErrDuplicateSidecar)
 
 	// ignores index out of bound
-	scs[0].Index = 6
-	require.ErrorIs(t, as.Persist(1, scs[0]), errIndexOutOfBounds)
+	blobSidecars[0].Index = 6
+	require.ErrorIs(t, as.Persist(1, blocks.NewSidecarFromBlobSidecar(blobSidecars[0])), errIndexOutOfBounds)
+
+	_, moreBlobSidecars := util.GenerateTestDenebBlockWithSidecar(t, [32]byte{}, 1, 4)
+
+	more := blocks.NewSidecarsFromBlobSidecars(moreBlobSidecars)
 
-	_, more := util.GenerateTestDenebBlockWithSidecar(t, [32]byte{}, 1, 4)
 	// ignores sidecars before the retention period
 	slotOOB, err := slots.EpochStart(params.BeaconConfig().MinEpochsForBlobsSidecarsRequest)
 	require.NoError(t, err)

beacon-chain/das/cache.go

@@ -10,12 +10,7 @@ import (
 	"github.com/pkg/errors"
 )
 
-var (
-	ErrDuplicateSidecar   = errors.New("duplicate sidecar stashed in AvailabilityStore")
-	errIndexOutOfBounds   = errors.New("sidecar.index > MAX_BLOBS_PER_BLOCK")
-	errCommitmentMismatch = errors.New("KzgCommitment of sidecar in cache did not match block commitment")
-	errMissingSidecar     = errors.New("no sidecar in cache for block commitment")
-)
+var errIndexOutOfBounds = errors.New("sidecar.index > MAX_BLOBS_PER_BLOCK")
 
 // cacheKey includes the slot so that we can easily iterate through the cache and compare
 // slots for eviction purposes. Whether the input is the block or the sidecar, we always have

beacon-chain/das/data_column_cache.go

@@ -0,0 +1,135 @@
+package das
+
+import (
+	"reflect"
+
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/db/filesystem"
+	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
+	"github.com/pkg/errors"
+)
+
+var (
+	ErrDuplicateSidecar   = errors.New("duplicate sidecar stashed in AvailabilityStore")
+	errColumnIndexTooHigh = errors.New("column index too high")
+	errCommitmentMismatch = errors.New("KzgCommitment of sidecar in cache did not match block commitment")
+	errMissingSidecar     = errors.New("no sidecar in cache for block commitment")
+)
+
+// dataColumnCacheKey includes the slot so that we can easily iterate through the cache and compare
+// slots for eviction purposes. Whether the input is the block or the sidecar, we always have
+// the root+slot when interacting with the cache, so it isn't an inconvenience to use both.
+type dataColumnCacheKey struct {
+	slot primitives.Slot
+	root [32]byte
+}
+
+type dataColumnCache struct {
+	entries map[dataColumnCacheKey]*dataColumnCacheEntry
+}
+
+func newDataColumnCache() *dataColumnCache {
+	return &dataColumnCache{entries: make(map[dataColumnCacheKey]*dataColumnCacheEntry)}
+}
+
+// ensure returns the entry for the given key, creating it if it isn't already present.
+func (c *dataColumnCache) ensure(key dataColumnCacheKey) *dataColumnCacheEntry {
+	entry, ok := c.entries[key]
+	if !ok {
+		entry = &dataColumnCacheEntry{}
+		c.entries[key] = entry
+	}
+
+	return entry
+}
+
+// delete removes the cache entry from the cache.
+func (c *dataColumnCache) delete(key dataColumnCacheKey) {
+	delete(c.entries, key)
+}
+
+// dataColumnCacheEntry holds a fixed-length cache of BlobSidecars.
+type dataColumnCacheEntry struct {
+	scs         [fieldparams.NumberOfColumns]*blocks.RODataColumn
+	diskSummary filesystem.DataColumnStorageSummary
+}
+
+func (e *dataColumnCacheEntry) setDiskSummary(sum filesystem.DataColumnStorageSummary) {
+	e.diskSummary = sum
+}
+
+// stash adds an item to the in-memory cache of DataColumnSidecars.
+// Only the first DataColumnSidecar of a given Index will be kept in the cache.
+// stash will return an error if the given data colunn is already in the cache, or if the Index is out of bounds.
+func (e *dataColumnCacheEntry) stash(sc *blocks.RODataColumn) error {
+	if sc.Index >= fieldparams.NumberOfColumns {
+		return errors.Wrapf(errColumnIndexTooHigh, "index=%d", sc.Index)
+	}
+
+	if e.scs[sc.Index] != nil {
+		return errors.Wrapf(ErrDuplicateSidecar, "root=%#x, index=%d, commitment=%#x", sc.BlockRoot(), sc.Index, sc.KzgCommitments)
+	}
+
+	e.scs[sc.Index] = sc
+
+	return nil
+}
+
+func (e *dataColumnCacheEntry) filter(root [32]byte, commitmentsArray *safeCommitmentsArray) ([]blocks.RODataColumn, error) {
+	nonEmptyIndices := commitmentsArray.nonEmptyIndices()
+	if e.diskSummary.AllAvailable(nonEmptyIndices) {
+		return nil, nil
+	}
+
+	commitmentsCount := commitmentsArray.count()
+	sidecars := make([]blocks.RODataColumn, 0, commitmentsCount)
+
+	for i := range nonEmptyIndices {
+		if e.diskSummary.HasIndex(i) {
+			continue
+		}
+
+		if e.scs[i] == nil {
+			return nil, errors.Wrapf(errMissingSidecar, "root=%#x, index=%#x", root, i)
+		}
+
+		if !reflect.DeepEqual(commitmentsArray[i], e.scs[i].KzgCommitments) {
+			return nil, errors.Wrapf(errCommitmentMismatch, "root=%#x, index=%#x, commitment=%#x, block commitment=%#x", root, i, e.scs[i].KzgCommitments, commitmentsArray[i])
+		}
+
+		sidecars = append(sidecars, *e.scs[i])
+	}
+
+	return sidecars, nil
+}
+
+// safeCommitmentsArray is a fixed size array of commitments.
+// This is helpful for avoiding gratuitous bounds checks.
+type safeCommitmentsArray [fieldparams.NumberOfColumns][][]byte
+
+// count returns the number of commitments in the array.
+func (s *safeCommitmentsArray) count() int {
+	count := 0
+
+	for i := range s {
+		if s[i] != nil {
+			count++
+		}
+	}
+
+	return count
+}
+
+// nonEmptyIndices returns a map of indices that are non-nil in the array.
+func (s *safeCommitmentsArray) nonEmptyIndices() map[uint64]bool {
+	columns := make(map[uint64]bool)
+
+	for i := range s {
+		if s[i] != nil {
+			columns[uint64(i)] = true
+		}
+	}
+
+	return columns
+}

beacon-chain/das/data_column_cache_test.go

@@ -0,0 +1,124 @@
+package das
+
+import (
+	"testing"
+
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/db/filesystem"
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/verification"
+	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
+	"github.com/OffchainLabs/prysm/v6/testing/require"
+)
+
+func TestEnsureDeleteSetDiskSummary(t *testing.T) {
+	c := newDataColumnCache()
+	key := dataColumnCacheKey{}
+	entry := c.ensure(key)
+	require.DeepEqual(t, dataColumnCacheEntry{}, *entry)
+
+	diskSummary := filesystem.NewDataColumnStorageSummary(42, [fieldparams.NumberOfColumns]bool{true})
+	entry.setDiskSummary(diskSummary)
+	entry = c.ensure(key)
+	require.DeepEqual(t, dataColumnCacheEntry{diskSummary: diskSummary}, *entry)
+
+	c.delete(key)
+	entry = c.ensure(key)
+	require.DeepEqual(t, dataColumnCacheEntry{}, *entry)
+}
+
+func TestStash(t *testing.T) {
+	t.Run("Index too high", func(t *testing.T) {
+		dataColumnParamsByBlockRoot := verification.DataColumnsParamsByRoot{{1}: {{ColumnIndex: 10_000}}}
+		roDataColumns, _ := verification.CreateTestVerifiedRoDataColumnSidecars(t, dataColumnParamsByBlockRoot)
+
+		var entry dataColumnCacheEntry
+		err := entry.stash(&roDataColumns[0])
+		require.NotNil(t, err)
+	})
+
+	t.Run("Nominal and already existing", func(t *testing.T) {
+		dataColumnParamsByBlockRoot := verification.DataColumnsParamsByRoot{{1}: {{ColumnIndex: 1}}}
+		roDataColumns, _ := verification.CreateTestVerifiedRoDataColumnSidecars(t, dataColumnParamsByBlockRoot)
+
+		var entry dataColumnCacheEntry
+		err := entry.stash(&roDataColumns[0])
+		require.NoError(t, err)
+
+		require.DeepEqual(t, roDataColumns[0], entry.scs[1])
+
+		err = entry.stash(&roDataColumns[0])
+		require.NotNil(t, err)
+	})
+}
+
+func TestFilterDataColumns(t *testing.T) {
+	t.Run("All available", func(t *testing.T) {
+		commitmentsArray := safeCommitmentsArray{nil, [][]byte{[]byte{1}}, nil, [][]byte{[]byte{3}}}
+
+		diskSummary := filesystem.NewDataColumnStorageSummary(42, [fieldparams.NumberOfColumns]bool{false, true, false, true})
+
+		dataColumnCacheEntry := dataColumnCacheEntry{diskSummary: diskSummary}
+
+		actual, err := dataColumnCacheEntry.filter([fieldparams.RootLength]byte{}, &commitmentsArray)
+		require.NoError(t, err)
+		require.IsNil(t, actual)
+	})
+
+	t.Run("Some scs missing", func(t *testing.T) {
+		commitmentsArray := safeCommitmentsArray{nil, [][]byte{[]byte{1}}}
+
+		diskSummary := filesystem.NewDataColumnStorageSummary(42, [fieldparams.NumberOfColumns]bool{})
+
+		dataColumnCacheEntry := dataColumnCacheEntry{diskSummary: diskSummary}
+
+		_, err := dataColumnCacheEntry.filter([fieldparams.RootLength]byte{}, &commitmentsArray)
+		require.NotNil(t, err)
+	})
+
+	t.Run("Commitments not equal", func(t *testing.T) {
+		root := [fieldparams.RootLength]byte{}
+		commitmentsArray := safeCommitmentsArray{nil, [][]byte{[]byte{1}}}
+
+		dataColumnParamsByBlockRoot := verification.DataColumnsParamsByRoot{root: {{ColumnIndex: 1}}}
+		roDataColumns, _ := verification.CreateTestVerifiedRoDataColumnSidecars(t, dataColumnParamsByBlockRoot)
+
+		var scs [fieldparams.NumberOfColumns]*blocks.RODataColumn
+		scs[1] = &roDataColumns[0]
+
+		dataColumnCacheEntry := dataColumnCacheEntry{scs: scs}
+
+		_, err := dataColumnCacheEntry.filter(root, &commitmentsArray)
+		require.NotNil(t, err)
+	})
+
+	t.Run("Nominal", func(t *testing.T) {
+		root := [fieldparams.RootLength]byte{}
+		commitmentsArray := safeCommitmentsArray{nil, [][]byte{[]byte{1}}, nil, [][]byte{[]byte{3}}}
+
+		diskSummary := filesystem.NewDataColumnStorageSummary(42, [fieldparams.NumberOfColumns]bool{false, true})
+
+		dataColumnParamsByBlockRoot := verification.DataColumnsParamsByRoot{root: {{ColumnIndex: 3, KzgCommitments: [][]byte{[]byte{3}}}}}
+		expected, _ := verification.CreateTestVerifiedRoDataColumnSidecars(t, dataColumnParamsByBlockRoot)
+
+		var scs [fieldparams.NumberOfColumns]*blocks.RODataColumn
+		scs[3] = &expected[0]
+
+		dataColumnCacheEntry := dataColumnCacheEntry{scs: scs, diskSummary: diskSummary}
+
+		actual, err := dataColumnCacheEntry.filter(root, &commitmentsArray)
+		require.NoError(t, err)
+
+		require.DeepEqual(t, expected, actual)
+	})
+}
+
+func TestCount(t *testing.T) {
+	s := safeCommitmentsArray{nil, [][]byte{[]byte{1}}, nil, [][]byte{[]byte{3}}}
+	require.Equal(t, 2, s.count())
+}
+
+func TestNonEmptyIndices(t *testing.T) {
+	s := safeCommitmentsArray{nil, [][]byte{[]byte{10}}, nil, [][]byte{[]byte{20}}}
+	actual := s.nonEmptyIndices()
+	require.DeepEqual(t, map[uint64]bool{1: true, 3: true}, actual)
+}

beacon-chain/das/iface.go

@@ -15,5 +15,5 @@ import (
 // durably persisted before returning a non-error value.
 type AvailabilityStore interface {
 	IsDataAvailable(ctx context.Context, current primitives.Slot, b blocks.ROBlock) error
-	Persist(current primitives.Slot, sc ...blocks.ROBlob) error
+	Persist(current primitives.Slot, sc ...blocks.ROSidecar) error
 }

beacon-chain/das/mock.go

@@ -5,6 +5,7 @@ import (
 
 	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
+	errors "github.com/pkg/errors"
 )
 
 // MockAvailabilityStore is an implementation of AvailabilityStore that can be used by other packages in tests.
@@ -24,9 +25,13 @@ func (m *MockAvailabilityStore) IsDataAvailable(ctx context.Context, current pri
 }
 
 // Persist satisfies the corresponding method of the AvailabilityStore interface in a way that is useful for tests.
-func (m *MockAvailabilityStore) Persist(current primitives.Slot, sc ...blocks.ROBlob) error {
+func (m *MockAvailabilityStore) Persist(current primitives.Slot, sc ...blocks.ROSidecar) error {
+	blobSidecars, err := blocks.BlobSidecarsFromSidecars(sc)
+	if err != nil {
+		return errors.Wrap(err, "blob sidecars from sidecars")
+	}
 	if m.PersistBlobsCallback != nil {
-		return m.PersistBlobsCallback(current, sc...)
+		return m.PersistBlobsCallback(current, blobSidecars...)
 	}
 	return nil
 }

beacon-chain/db/filesystem/BUILD.bazel

@@ -5,6 +5,9 @@ go_library(
     srcs = [
         "blob.go",
         "cache.go",
+        "data_column.go",
+        "data_column_cache.go",
+        "doc.go",
         "iteration.go",
         "layout.go",
         "layout_by_epoch.go",
@@ -17,6 +20,8 @@ go_library(
     importpath = "github.com/OffchainLabs/prysm/v6/beacon-chain/db/filesystem",
     visibility = ["//visibility:public"],
     deps = [
+        "//async:go_default_library",
+        "//async/event:go_default_library",
         "//beacon-chain/db:go_default_library",
         "//beacon-chain/verification:go_default_library",
         "//config/fieldparams:go_default_library",
@@ -25,6 +30,7 @@ go_library(
         "//consensus-types/primitives:go_default_library",
         "//encoding/bytesutil:go_default_library",
         "//io/file:go_default_library",
+        "//proto/prysm/v1alpha1:go_default_library",
         "//runtime/logging:go_default_library",
         "//time/slots:go_default_library",
         "@com_github_ethereum_go_ethereum//common/hexutil:go_default_library",
@@ -41,6 +47,8 @@ go_test(
     srcs = [
         "blob_test.go",
         "cache_test.go",
+        "data_column_cache_test.go",
+        "data_column_test.go",
         "iteration_test.go",
         "layout_test.go",
         "migration_test.go",
@@ -50,6 +58,7 @@ go_test(
     deps = [
         "//beacon-chain/db:go_default_library",
         "//beacon-chain/verification:go_default_library",
+        "//config/fieldparams:go_default_library",
         "//config/params:go_default_library",
         "//consensus-types/primitives:go_default_library",
         "//encoding/bytesutil:go_default_library",

beacon-chain/db/filesystem/data_column_cache.go

@@ -0,0 +1,219 @@
+package filesystem
+
+import (
+	"sync"
+
+	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
+	"github.com/OffchainLabs/prysm/v6/config/params"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
+	"github.com/pkg/errors"
+)
+
+var errDataColumnIndexOutOfBounds = errors.New("data column index too high")
+
+// DataColumnStorageSummary represents cached information about the DataColumnSidecars on disk for each root the cache knows about.
+type DataColumnStorageSummary struct {
+	epoch primitives.Epoch
+	mask  [fieldparams.NumberOfColumns]bool
+}
+
+// NewDataColumnStorageSummary creates a new DataColumnStorageSummary for a given epoch and mask.
+func NewDataColumnStorageSummary(epoch primitives.Epoch, mask [fieldparams.NumberOfColumns]bool) DataColumnStorageSummary {
+	return DataColumnStorageSummary{
+		epoch: epoch,
+		mask:  mask,
+	}
+}
+
+// HasIndex returns true if the DataColumnSidecar at the given index is available in the filesystem.
+func (s DataColumnStorageSummary) HasIndex(index uint64) bool {
+	if index >= uint64(fieldparams.NumberOfColumns) {
+		return false
+	}
+	return s.mask[index]
+}
+
+// Count returns the number of available data columns.
+func (s DataColumnStorageSummary) Count() uint64 {
+	count := uint64(0)
+
+	for _, available := range s.mask {
+		if available {
+			count++
+		}
+	}
+
+	return count
+}
+
+// AllAvailable returns true if we have all data columns for corresponding indices.
+func (s DataColumnStorageSummary) AllAvailable(indices map[uint64]bool) bool {
+	if len(indices) > len(s.mask) {
+		return false
+	}
+
+	for index := range indices {
+		if !s.mask[index] {
+			return false
+		}
+	}
+
+	return true
+}
+
+// DataColumnStorageSummarizer can be used to receive a summary of metadata about data columns on disk for a given root.
+// The DataColumnStorageSummary can be used to check which indices (if any) are available for a given block by root.
+type DataColumnStorageSummarizer interface {
+	Summary(root [fieldparams.RootLength]byte) DataColumnStorageSummary
+}
+
+type dataColumnStorageSummaryCache struct {
+	mu                 sync.RWMutex
+	dataColumnCount    float64
+	lowestCachedEpoch  primitives.Epoch
+	highestCachedEpoch primitives.Epoch
+	cache              map[[fieldparams.RootLength]byte]DataColumnStorageSummary
+}
+
+var _ DataColumnStorageSummarizer = &dataColumnStorageSummaryCache{}
+
+func newDataColumnStorageSummaryCache() *dataColumnStorageSummaryCache {
+	return &dataColumnStorageSummaryCache{
+		cache:             make(map[[fieldparams.RootLength]byte]DataColumnStorageSummary),
+		lowestCachedEpoch: params.BeaconConfig().FarFutureEpoch,
+	}
+}
+
+// Summary returns the DataColumnStorageSummary for `root`.
+// The DataColumnStorageSummary can be used to check for the presence of DataColumnSidecars based on Index.
+func (sc *dataColumnStorageSummaryCache) Summary(root [fieldparams.RootLength]byte) DataColumnStorageSummary {
+	sc.mu.RLock()
+	defer sc.mu.RUnlock()
+
+	return sc.cache[root]
+}
+
+func (sc *dataColumnStorageSummaryCache) HighestEpoch() primitives.Epoch {
+	sc.mu.RLock()
+	defer sc.mu.RUnlock()
+	return sc.highestCachedEpoch
+}
+
+// set updates the cache.
+func (sc *dataColumnStorageSummaryCache) set(dataColumnsIdent DataColumnsIdent) error {
+	numberOfColumns := params.BeaconConfig().NumberOfColumns
+
+	sc.mu.Lock()
+	defer sc.mu.Unlock()
+
+	summary := sc.cache[dataColumnsIdent.Root]
+	summary.epoch = dataColumnsIdent.Epoch
+
+	count := uint64(0)
+	for _, index := range dataColumnsIdent.Indices {
+		if index >= numberOfColumns {
+			return errDataColumnIndexOutOfBounds
+		}
+
+		if summary.mask[index] {
+			continue
+		}
+
+		count++
+
+		summary.mask[index] = true
+		sc.cache[dataColumnsIdent.Root] = summary
+		sc.lowestCachedEpoch = min(sc.lowestCachedEpoch, dataColumnsIdent.Epoch)
+		sc.highestCachedEpoch = max(sc.highestCachedEpoch, dataColumnsIdent.Epoch)
+	}
+
+	countFloat := float64(count)
+	sc.dataColumnCount += countFloat
+	dataColumnDiskCount.Set(sc.dataColumnCount)
+	dataColumnWrittenCounter.Add(countFloat)
+
+	return nil
+}
+
+// get returns the DataColumnStorageSummary for the given block root.
+// If the root is not in the cache, the second return value will be false.
+func (sc *dataColumnStorageSummaryCache) get(blockRoot [fieldparams.RootLength]byte) (DataColumnStorageSummary, bool) {
+	sc.mu.RLock()
+	defer sc.mu.RUnlock()
+
+	v, ok := sc.cache[blockRoot]
+	return v, ok
+}
+
+// evict removes the DataColumnStorageSummary for the given block root from the cache.
+func (s *dataColumnStorageSummaryCache) evict(blockRoot [fieldparams.RootLength]byte) int {
+	deleted := 0
+
+	s.mu.Lock()
+	defer s.mu.Unlock()
+
+	summary, ok := s.cache[blockRoot]
+	if !ok {
+		return 0
+	}
+
+	for i := range summary.mask {
+		if summary.mask[i] {
+			deleted += 1
+		}
+	}
+
+	delete(s.cache, blockRoot)
+	if deleted > 0 {
+		s.dataColumnCount -= float64(deleted)
+		dataColumnDiskCount.Set(s.dataColumnCount)
+	}
+
+	// The lowest and highest cached epoch may no longer be valid here,
+	// but is not worth the effort to recalculate.
+
+	return deleted
+}
+
+// pruneUpTo removes all entries from the cache up to the given target epoch included.
+func (sc *dataColumnStorageSummaryCache) pruneUpTo(targetEpoch primitives.Epoch) uint64 {
+	sc.mu.Lock()
+	defer sc.mu.Unlock()
+
+	prunedCount := uint64(0)
+	newLowestCachedEpoch := params.BeaconConfig().FarFutureEpoch
+	newHighestCachedEpoch := primitives.Epoch(0)
+
+	for blockRoot, summary := range sc.cache {
+		epoch := summary.epoch
+
+		if epoch > targetEpoch {
+			newLowestCachedEpoch = min(newLowestCachedEpoch, epoch)
+			newHighestCachedEpoch = max(newHighestCachedEpoch, epoch)
+		}
+
+		if epoch <= targetEpoch {
+			for i := range summary.mask {
+				if summary.mask[i] {
+					prunedCount += 1
+				}
+			}
+
+			delete(sc.cache, blockRoot)
+		}
+	}
+
+	if prunedCount > 0 {
+		sc.lowestCachedEpoch = newLowestCachedEpoch
+		sc.highestCachedEpoch = newHighestCachedEpoch
+		sc.dataColumnCount -= float64(prunedCount)
+		dataColumnDiskCount.Set(sc.dataColumnCount)
+	}
+
+	return prunedCount
+}
+
+// clear removes all entries from the cache.
+func (sc *dataColumnStorageSummaryCache) clear() uint64 {
+	return sc.pruneUpTo(params.BeaconConfig().FarFutureEpoch)
+}

beacon-chain/db/filesystem/data_column_cache_test.go

@@ -0,0 +1,235 @@
+package filesystem
+
+import (
+	"testing"
+
+	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
+	"github.com/OffchainLabs/prysm/v6/config/params"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
+	"github.com/OffchainLabs/prysm/v6/testing/require"
+)
+
+func TestHasIndex(t *testing.T) {
+	summary := NewDataColumnStorageSummary(0, [fieldparams.NumberOfColumns]bool{false, true})
+
+	hasIndex := summary.HasIndex(1_000_000)
+	require.Equal(t, false, hasIndex)
+
+	hasIndex = summary.HasIndex(0)
+	require.Equal(t, false, hasIndex)
+
+	hasIndex = summary.HasIndex(1)
+	require.Equal(t, true, hasIndex)
+}
+
+func TestCount(t *testing.T) {
+	summary := NewDataColumnStorageSummary(0, [fieldparams.NumberOfColumns]bool{false, true, false, true})
+
+	count := summary.Count()
+	require.Equal(t, uint64(2), count)
+}
+
+func TestAllAvailableDataColumns(t *testing.T) {
+	const count = uint64(1_000)
+
+	summary := NewDataColumnStorageSummary(0, [fieldparams.NumberOfColumns]bool{false, true, false, true})
+
+	indices := make(map[uint64]bool, count)
+	for i := range count {
+		indices[i] = true
+	}
+
+	allAvailable := summary.AllAvailable(indices)
+	require.Equal(t, false, allAvailable)
+
+	indices = map[uint64]bool{1: true, 2: true}
+	allAvailable = summary.AllAvailable(indices)
+	require.Equal(t, false, allAvailable)
+
+	indices = map[uint64]bool{1: true, 3: true}
+	allAvailable = summary.AllAvailable(indices)
+	require.Equal(t, true, allAvailable)
+}
+
+func TestSummary(t *testing.T) {
+	root := [fieldparams.RootLength]byte{}
+
+	summaryCache := newDataColumnStorageSummaryCache()
+	expected := NewDataColumnStorageSummary(0, [fieldparams.NumberOfColumns]bool{})
+	actual := summaryCache.Summary(root)
+	require.DeepEqual(t, expected, actual)
+
+	summaryCache = newDataColumnStorageSummaryCache()
+	expected = NewDataColumnStorageSummary(0, [fieldparams.NumberOfColumns]bool{true, false, true, false})
+	summaryCache.cache[root] = expected
+	actual = summaryCache.Summary(root)
+	require.DeepEqual(t, expected, actual)
+}
+
+func TestHighestEpoch(t *testing.T) {
+	root1 := [fieldparams.RootLength]byte{1}
+	root2 := [fieldparams.RootLength]byte{2}
+	root3 := [fieldparams.RootLength]byte{3}
+
+	summaryCache := newDataColumnStorageSummaryCache()
+	actual := summaryCache.HighestEpoch()
+	require.Equal(t, primitives.Epoch(0), actual)
+
+	err := summaryCache.set(DataColumnsIdent{Root: root1, Epoch: 42, Indices: []uint64{1, 3}})
+	require.NoError(t, err)
+	require.Equal(t, primitives.Epoch(42), summaryCache.HighestEpoch())
+
+	err = summaryCache.set(DataColumnsIdent{Root: root2, Epoch: 43, Indices: []uint64{1, 3}})
+	require.NoError(t, err)
+	require.Equal(t, primitives.Epoch(43), summaryCache.HighestEpoch())
+
+	err = summaryCache.set(DataColumnsIdent{Root: root3, Epoch: 40, Indices: []uint64{1, 3}})
+	require.NoError(t, err)
+	require.Equal(t, primitives.Epoch(43), summaryCache.HighestEpoch())
+}
+
+func TestSet(t *testing.T) {
+	t.Run("Index out of bounds", func(t *testing.T) {
+		summaryCache := newDataColumnStorageSummaryCache()
+		err := summaryCache.set(DataColumnsIdent{Indices: []uint64{1_000_000}})
+		require.ErrorIs(t, err, errDataColumnIndexOutOfBounds)
+		require.Equal(t, params.BeaconConfig().FarFutureEpoch, summaryCache.lowestCachedEpoch)
+		require.Equal(t, 0, len(summaryCache.cache))
+	})
+
+	t.Run("Nominal", func(t *testing.T) {
+		root1 := [fieldparams.RootLength]byte{1}
+		root2 := [fieldparams.RootLength]byte{2}
+
+		summaryCache := newDataColumnStorageSummaryCache()
+
+		err := summaryCache.set(DataColumnsIdent{Root: root1, Epoch: 42, Indices: []uint64{1, 3}})
+		require.NoError(t, err)
+		require.Equal(t, primitives.Epoch(42), summaryCache.lowestCachedEpoch)
+		require.Equal(t, 1, len(summaryCache.cache))
+		expected := DataColumnStorageSummary{epoch: 42, mask: [fieldparams.NumberOfColumns]bool{false, true, false, true}}
+		actual := summaryCache.cache[root1]
+		require.DeepEqual(t, expected, actual)
+
+		err = summaryCache.set(DataColumnsIdent{Root: root1, Epoch: 42, Indices: []uint64{0, 1}})
+		require.NoError(t, err)
+		require.Equal(t, primitives.Epoch(42), summaryCache.lowestCachedEpoch)
+		require.Equal(t, 1, len(summaryCache.cache))
+		expected = DataColumnStorageSummary{epoch: 42, mask: [fieldparams.NumberOfColumns]bool{true, true, false, true}}
+		actual = summaryCache.cache[root1]
+		require.DeepEqual(t, expected, actual)
+
+		err = summaryCache.set(DataColumnsIdent{Root: root2, Epoch: 43, Indices: []uint64{1}})
+		require.NoError(t, err)
+		require.Equal(t, primitives.Epoch(42), summaryCache.lowestCachedEpoch) // Epoch 42 is still the lowest
+		require.Equal(t, 2, len(summaryCache.cache))
+		expected = DataColumnStorageSummary{epoch: 43, mask: [fieldparams.NumberOfColumns]bool{false, true}}
+		actual = summaryCache.cache[root2]
+		require.DeepEqual(t, expected, actual)
+	})
+}
+
+func TestGet(t *testing.T) {
+	t.Run("Not in cache", func(t *testing.T) {
+		summaryCache := newDataColumnStorageSummaryCache()
+		root := [fieldparams.RootLength]byte{}
+		_, ok := summaryCache.get(root)
+		require.Equal(t, false, ok)
+	})
+
+	t.Run("In cache", func(t *testing.T) {
+		root := [fieldparams.RootLength]byte{}
+		summaryCache := newDataColumnStorageSummaryCache()
+		summaryCache.cache[root] = NewDataColumnStorageSummary(42, [fieldparams.NumberOfColumns]bool{true, false, true, false})
+		actual, ok := summaryCache.get(root)
+		require.Equal(t, true, ok)
+		expected := NewDataColumnStorageSummary(42, [fieldparams.NumberOfColumns]bool{true, false, true, false})
+		require.DeepEqual(t, expected, actual)
+	})
+}
+
+func TestEvict(t *testing.T) {
+	t.Run("No eviction", func(t *testing.T) {
+		root := [fieldparams.RootLength]byte{}
+		summaryCache := newDataColumnStorageSummaryCache()
+
+		evicted := summaryCache.evict(root)
+		require.Equal(t, 0, evicted)
+	})
+
+	t.Run("Eviction", func(t *testing.T) {
+		root1 := [fieldparams.RootLength]byte{1}
+		root2 := [fieldparams.RootLength]byte{2}
+		summaryCache := newDataColumnStorageSummaryCache()
+		summaryCache.cache[root1] = NewDataColumnStorageSummary(42, [fieldparams.NumberOfColumns]bool{true, false, true, false})
+		summaryCache.cache[root2] = NewDataColumnStorageSummary(43, [fieldparams.NumberOfColumns]bool{false, true, false, true})
+
+		evicted := summaryCache.evict(root1)
+		require.Equal(t, 2, evicted)
+		require.Equal(t, 1, len(summaryCache.cache))
+
+		_, ok := summaryCache.cache[root1]
+		require.Equal(t, false, ok)
+
+		_, ok = summaryCache.cache[root2]
+		require.Equal(t, true, ok)
+	})
+}
+
+func TestPruneUpTo(t *testing.T) {
+	t.Run("No pruning", func(t *testing.T) {
+		summaryCache := newDataColumnStorageSummaryCache()
+		err := summaryCache.set(DataColumnsIdent{Root: [fieldparams.RootLength]byte{1}, Epoch: 42, Indices: []uint64{1}})
+		require.NoError(t, err)
+
+		err = summaryCache.set(DataColumnsIdent{Root: [fieldparams.RootLength]byte{2}, Epoch: 43, Indices: []uint64{2, 4}})
+		require.NoError(t, err)
+
+		count := summaryCache.pruneUpTo(41)
+		require.Equal(t, uint64(0), count)
+		require.Equal(t, 2, len(summaryCache.cache))
+		require.Equal(t, primitives.Epoch(42), summaryCache.lowestCachedEpoch)
+	})
+
+	t.Run("Pruning", func(t *testing.T) {
+		summaryCache := newDataColumnStorageSummaryCache()
+		err := summaryCache.set(DataColumnsIdent{Root: [fieldparams.RootLength]byte{1}, Epoch: 42, Indices: []uint64{1}})
+		require.NoError(t, err)
+
+		err = summaryCache.set(DataColumnsIdent{Root: [fieldparams.RootLength]byte{2}, Epoch: 44, Indices: []uint64{2, 4}})
+		require.NoError(t, err)
+
+		err = summaryCache.set(DataColumnsIdent{Root: [fieldparams.RootLength]byte{3}, Epoch: 45, Indices: []uint64{2, 4}})
+		require.NoError(t, err)
+
+		count := summaryCache.pruneUpTo(42)
+		require.Equal(t, uint64(1), count)
+		require.Equal(t, 2, len(summaryCache.cache))
+		require.Equal(t, primitives.Epoch(44), summaryCache.lowestCachedEpoch)
+
+		count = summaryCache.pruneUpTo(45)
+		require.Equal(t, uint64(4), count)
+		require.Equal(t, 0, len(summaryCache.cache))
+		require.Equal(t, params.BeaconConfig().FarFutureEpoch, summaryCache.lowestCachedEpoch)
+		require.Equal(t, primitives.Epoch(0), summaryCache.highestCachedEpoch)
+
+	})
+
+	t.Run("Clear", func(t *testing.T) {
+		summaryCache := newDataColumnStorageSummaryCache()
+		err := summaryCache.set(DataColumnsIdent{Root: [fieldparams.RootLength]byte{1}, Epoch: 42, Indices: []uint64{1}})
+		require.NoError(t, err)
+
+		err = summaryCache.set(DataColumnsIdent{Root: [fieldparams.RootLength]byte{2}, Epoch: 44, Indices: []uint64{2, 4}})
+		require.NoError(t, err)
+
+		err = summaryCache.set(DataColumnsIdent{Root: [fieldparams.RootLength]byte{3}, Epoch: 45, Indices: []uint64{2, 4}})
+		require.NoError(t, err)
+
+		count := summaryCache.clear()
+		require.Equal(t, uint64(5), count)
+		require.Equal(t, 0, len(summaryCache.cache))
+		require.Equal(t, params.BeaconConfig().FarFutureEpoch, summaryCache.lowestCachedEpoch)
+		require.Equal(t, primitives.Epoch(0), summaryCache.highestCachedEpoch)
+	})
+}

beacon-chain/db/filesystem/data_column_test.go

@@ -0,0 +1,742 @@
+package filesystem
+
+import (
+	"context"
+	"encoding/binary"
+	"os"
+	"testing"
+
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/verification"
+	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
+	"github.com/OffchainLabs/prysm/v6/config/params"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
+	"github.com/OffchainLabs/prysm/v6/testing/require"
+	"github.com/spf13/afero"
+)
+
+func TestNewDataColumnStorage(t *testing.T) {
+	ctx := context.Background()
+
+	t.Run("No base path", func(t *testing.T) {
+		_, err := NewDataColumnStorage(ctx)
+		require.ErrorIs(t, err, errNoBasePath)
+	})
+
+	t.Run("Nominal", func(t *testing.T) {
+		dir := t.TempDir()
+
+		storage, err := NewDataColumnStorage(ctx, WithDataColumnBasePath(dir))
+		require.NoError(t, err)
+		require.Equal(t, dir, storage.base)
+	})
+}
+
+func TestWarmCache(t *testing.T) {
+	storage, err := NewDataColumnStorage(
+		context.Background(),
+		WithDataColumnBasePath(t.TempDir()),
+		WithDataColumnRetentionEpochs(10_000),
+	)
+	require.NoError(t, err)
+
+	_, verifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
+		t,
+		verification.DataColumnsParamsByRoot{
+			{0}: {
+				{Slot: 33, ColumnIndex: 2, DataColumn: []byte{1, 2, 3}}, // Period 0 - Epoch 1
+				{Slot: 33, ColumnIndex: 4, DataColumn: []byte{2, 3, 4}}, // Period 0 - Epoch 1
+			},
+			{1}: {
+				{Slot: 128_002, ColumnIndex: 2, DataColumn: []byte{1, 2, 3}}, // Period 0 - Epoch 4000
+				{Slot: 128_002, ColumnIndex: 4, DataColumn: []byte{2, 3, 4}}, // Period 0 - Epoch 4000
+			},
+			{2}: {
+				{Slot: 128_003, ColumnIndex: 1, DataColumn: []byte{1, 2, 3}}, // Period 0 - Epoch 4000
+				{Slot: 128_003, ColumnIndex: 3, DataColumn: []byte{2, 3, 4}}, // Period 0 - Epoch 4000
+			},
+			{3}: {
+				{Slot: 128_034, ColumnIndex: 2, DataColumn: []byte{1, 2, 3}}, // Period 0 - Epoch 4001
+				{Slot: 128_034, ColumnIndex: 4, DataColumn: []byte{2, 3, 4}}, // Period 0 - Epoch 4001
+			},
+			{4}: {
+				{Slot: 131_138, ColumnIndex: 2, DataColumn: []byte{1, 2, 3}}, // Period 1 - Epoch 4098
+			},
+			{5}: {
+				{Slot: 131_138, ColumnIndex: 1, DataColumn: []byte{1, 2, 3}}, // Period 1 - Epoch 4098
+			},
+			{6}: {
+				{Slot: 131_168, ColumnIndex: 0, DataColumn: []byte{1, 2, 3}}, // Period 1 - Epoch 4099
+			},
+		},
+	)
+
+	err = storage.Save(verifiedRoDataColumnSidecars)
+	require.NoError(t, err)
+
+	storage.retentionEpochs = 4_096
+
+	storage.WarmCache()
+	require.Equal(t, primitives.Epoch(4_000), storage.cache.lowestCachedEpoch)
+	require.Equal(t, 6, len(storage.cache.cache))
+
+	summary, ok := storage.cache.get([fieldparams.RootLength]byte{1})
+	require.Equal(t, true, ok)
+	require.DeepEqual(t, DataColumnStorageSummary{epoch: 4_000, mask: [fieldparams.NumberOfColumns]bool{false, false, true, false, true}}, summary)
+
+	summary, ok = storage.cache.get([fieldparams.RootLength]byte{2})
+	require.Equal(t, true, ok)
+	require.DeepEqual(t, DataColumnStorageSummary{epoch: 4_000, mask: [fieldparams.NumberOfColumns]bool{false, true, false, true}}, summary)
+
+	summary, ok = storage.cache.get([fieldparams.RootLength]byte{3})
+	require.Equal(t, true, ok)
+	require.DeepEqual(t, DataColumnStorageSummary{epoch: 4_001, mask: [fieldparams.NumberOfColumns]bool{false, false, true, false, true}}, summary)
+
+	summary, ok = storage.cache.get([fieldparams.RootLength]byte{4})
+	require.Equal(t, true, ok)
+	require.DeepEqual(t, DataColumnStorageSummary{epoch: 4_098, mask: [fieldparams.NumberOfColumns]bool{false, false, true}}, summary)
+
+	summary, ok = storage.cache.get([fieldparams.RootLength]byte{5})
+	require.Equal(t, true, ok)
+	require.DeepEqual(t, DataColumnStorageSummary{epoch: 4_098, mask: [fieldparams.NumberOfColumns]bool{false, true}}, summary)
+
+	summary, ok = storage.cache.get([fieldparams.RootLength]byte{6})
+	require.Equal(t, true, ok)
+	require.DeepEqual(t, DataColumnStorageSummary{epoch: 4_099, mask: [fieldparams.NumberOfColumns]bool{true}}, summary)
+}
+
+func TestSaveDataColumnsSidecars(t *testing.T) {
+	t.Run("wrong numbers of columns", func(t *testing.T) {
+		cfg := params.BeaconConfig().Copy()
+		cfg.NumberOfColumns = 0
+		params.OverrideBeaconConfig(cfg)
+		params.SetupTestConfigCleanup(t)
+
+		_, verifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
+			t,
+			verification.DataColumnsParamsByRoot{
+				{}: {{ColumnIndex: 12}, {ColumnIndex: 1_000_000}, {ColumnIndex: 48}},
+			},
+		)
+
+		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
+		err := dataColumnStorage.Save(verifiedRoDataColumnSidecars)
+		require.ErrorIs(t, err, errWrongNumberOfColumns)
+	})
+
+	t.Run("one of the column index is too large", func(t *testing.T) {
+		_, verifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
+			t,
+			verification.DataColumnsParamsByRoot{{}: {{ColumnIndex: 12}, {ColumnIndex: 1_000_000}, {ColumnIndex: 48}}},
+		)
+
+		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
+		err := dataColumnStorage.Save(verifiedRoDataColumnSidecars)
+		require.ErrorIs(t, err, errDataColumnIndexTooLarge)
+	})
+
+	t.Run("different slots", func(t *testing.T) {
+		_, verifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
+			t,
+			verification.DataColumnsParamsByRoot{
+				{}: {
+					{Slot: 1, ColumnIndex: 12, DataColumn: []byte{1, 2, 3}},
+					{Slot: 2, ColumnIndex: 12, DataColumn: []byte{1, 2, 3}},
+				},
+			},
+		)
+
+		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
+		err := dataColumnStorage.Save(verifiedRoDataColumnSidecars)
+		require.ErrorIs(t, err, errDataColumnSidecarsFromDifferentSlots)
+	})
+
+	t.Run("new file - no data columns to save", func(t *testing.T) {
+		_, verifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
+			t,
+			verification.DataColumnsParamsByRoot{{}: {}},
+		)
+
+		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
+		err := dataColumnStorage.Save(verifiedRoDataColumnSidecars)
+		require.NoError(t, err)
+	})
+
+	t.Run("new file - different data column size", func(t *testing.T) {
+		_, verifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
+			t,
+			verification.DataColumnsParamsByRoot{
+				{}: {
+					{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}},
+					{ColumnIndex: 11, DataColumn: []byte{1, 2, 3, 4}},
+				},
+			},
+		)
+
+		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
+		err := dataColumnStorage.Save(verifiedRoDataColumnSidecars)
+		require.ErrorIs(t, err, errWrongSszEncodedDataColumnSidecarSize)
+	})
+
+	t.Run("existing file - wrong incoming SSZ encoded size", func(t *testing.T) {
+		_, verifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
+			t,
+			verification.DataColumnsParamsByRoot{{1}: {{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}}}},
+		)
+
+		// Save data columns into a file.
+		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
+		err := dataColumnStorage.Save(verifiedRoDataColumnSidecars)
+		require.NoError(t, err)
+
+		// Build a data column sidecar for the same block but with a different
+		// column index and an different SSZ encoded size.
+		_, verifiedRoDataColumnSidecars = verification.CreateTestVerifiedRoDataColumnSidecars(
+			t,
+			verification.DataColumnsParamsByRoot{{1}: {{ColumnIndex: 13, DataColumn: []byte{1, 2, 3, 4}}}},
+		)
+
+		// Try to rewrite the file.
+		err = dataColumnStorage.Save(verifiedRoDataColumnSidecars)
+		require.ErrorIs(t, err, errWrongSszEncodedDataColumnSidecarSize)
+	})
+
+	t.Run("nominal", func(t *testing.T) {
+		_, inputVerifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
+			t,
+			verification.DataColumnsParamsByRoot{
+				{1}: {
+					{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}},
+					{ColumnIndex: 11, DataColumn: []byte{3, 4, 5}},
+					{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}}, // OK if duplicate
+					{ColumnIndex: 13, DataColumn: []byte{6, 7, 8}},
+				},
+				{2}: {
+					{ColumnIndex: 12, DataColumn: []byte{3, 4, 5}},
+					{ColumnIndex: 13, DataColumn: []byte{6, 7, 8}},
+				},
+			},
+		)
+
+		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
+		err := dataColumnStorage.Save(inputVerifiedRoDataColumnSidecars)
+		require.NoError(t, err)
+
+		_, inputVerifiedRoDataColumnSidecars = verification.CreateTestVerifiedRoDataColumnSidecars(
+			t,
+			verification.DataColumnsParamsByRoot{
+				{1}: {
+					{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}}, // OK if duplicate
+					{ColumnIndex: 15, DataColumn: []byte{2, 3, 4}},
+					{ColumnIndex: 1, DataColumn: []byte{2, 3, 4}},
+				},
+				{3}: {
+					{ColumnIndex: 6, DataColumn: []byte{3, 4, 5}},
+					{ColumnIndex: 2, DataColumn: []byte{6, 7, 8}},
+				},
+			},
+		)
+
+		err = dataColumnStorage.Save(inputVerifiedRoDataColumnSidecars)
+		require.NoError(t, err)
+
+		type fixture struct {
+			fileName         string
+			blockRoot        [fieldparams.RootLength]byte
+			expectedIndices  [mandatoryNumberOfColumns]byte
+			dataColumnParams []verification.DataColumnParams
+		}
+
+		fixtures := []fixture{
+			{
+				fileName:  "0/0/0x0100000000000000000000000000000000000000000000000000000000000000.sszs",
+				blockRoot: [fieldparams.RootLength]byte{1},
+				expectedIndices: [mandatoryNumberOfColumns]byte{
+					0, nonZeroOffset + 4, 0, 0, 0, 0, 0, 0,
+					0, 0, 0, nonZeroOffset + 1, nonZeroOffset, nonZeroOffset + 2, 0, nonZeroOffset + 3,
+					// The rest is filled with zeroes.
+				},
+				dataColumnParams: []verification.DataColumnParams{
+					{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}},
+					{ColumnIndex: 11, DataColumn: []byte{3, 4, 5}},
+					{ColumnIndex: 13, DataColumn: []byte{6, 7, 8}},
+					{ColumnIndex: 15, DataColumn: []byte{2, 3, 4}},
+					{ColumnIndex: 1, DataColumn: []byte{2, 3, 4}},
+				},
+			},
+			{
+				fileName:  "0/0/0x0200000000000000000000000000000000000000000000000000000000000000.sszs",
+				blockRoot: [fieldparams.RootLength]byte{2},
+				expectedIndices: [mandatoryNumberOfColumns]byte{
+					0, 0, 0, 0, 0, 0, 0, 0,
+					0, 0, 0, 0, nonZeroOffset, nonZeroOffset + 1, 0, 0,
+					// The rest is filled with zeroes.
+				},
+				dataColumnParams: []verification.DataColumnParams{
+					{ColumnIndex: 12, DataColumn: []byte{3, 4, 5}},
+					{ColumnIndex: 13, DataColumn: []byte{6, 7, 8}},
+				},
+			},
+			{
+				fileName:  "0/0/0x0300000000000000000000000000000000000000000000000000000000000000.sszs",
+				blockRoot: [fieldparams.RootLength]byte{3},
+				expectedIndices: [mandatoryNumberOfColumns]byte{
+					0, 0, nonZeroOffset + 1, 0, 0, 0, nonZeroOffset, 0,
+					// The rest is filled with zeroes.
+				},
+				dataColumnParams: []verification.DataColumnParams{
+					{ColumnIndex: 6, DataColumn: []byte{3, 4, 5}},
+					{ColumnIndex: 2, DataColumn: []byte{6, 7, 8}},
+				},
+			},
+		}
+
+		for _, fixture := range fixtures {
+			// Build expected data column sidecars.
+			_, expectedDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
+				t,
+				verification.DataColumnsParamsByRoot{fixture.blockRoot: fixture.dataColumnParams},
+			)
+
+			// Build expected bytes.
+			firstSszEncodedDataColumnSidecar, err := expectedDataColumnSidecars[0].MarshalSSZ()
+			require.NoError(t, err)
+
+			dataColumnSidecarsCount := len(expectedDataColumnSidecars)
+			sszEncodedDataColumnSidecarSize := len(firstSszEncodedDataColumnSidecar)
+
+			sszEncodedDataColumnSidecars := make([]byte, 0, dataColumnSidecarsCount*sszEncodedDataColumnSidecarSize)
+			sszEncodedDataColumnSidecars = append(sszEncodedDataColumnSidecars, firstSszEncodedDataColumnSidecar...)
+			for _, dataColumnSidecar := range expectedDataColumnSidecars[1:] {
+				sszEncodedDataColumnSidecar, err := dataColumnSidecar.MarshalSSZ()
+				require.NoError(t, err)
+				sszEncodedDataColumnSidecars = append(sszEncodedDataColumnSidecars, sszEncodedDataColumnSidecar...)
+			}
+
+			var encodedSszEncodedDataColumnSidecarSize [encodedSszEncodedDataColumnSidecarSizeSize]byte
+			binary.BigEndian.PutUint32(encodedSszEncodedDataColumnSidecarSize[:], uint32(sszEncodedDataColumnSidecarSize))
+
+			expectedBytes := make([]byte, 0, headerSize+dataColumnSidecarsCount*sszEncodedDataColumnSidecarSize)
+			expectedBytes = append(expectedBytes, []byte{0x01}...)
+			expectedBytes = append(expectedBytes, encodedSszEncodedDataColumnSidecarSize[:]...)
+			expectedBytes = append(expectedBytes, fixture.expectedIndices[:]...)
+			expectedBytes = append(expectedBytes, sszEncodedDataColumnSidecars...)
+
+			// Check the actual content of the file.
+			actualBytes, err := afero.ReadFile(dataColumnStorage.fs, fixture.fileName)
+			require.NoError(t, err)
+			require.DeepSSZEqual(t, expectedBytes, actualBytes)
+
+			// Check the summary.
+			indices := map[uint64]bool{}
+			for _, dataColumnParam := range fixture.dataColumnParams {
+				indices[dataColumnParam.ColumnIndex] = true
+			}
+
+			summary := dataColumnStorage.Summary(fixture.blockRoot)
+			for index := range uint64(mandatoryNumberOfColumns) {
+				require.Equal(t, indices[index], summary.HasIndex(index))
+			}
+
+			err = dataColumnStorage.Remove(fixture.blockRoot)
+			require.NoError(t, err)
+
+			summary = dataColumnStorage.Summary(fixture.blockRoot)
+			for index := range uint64(mandatoryNumberOfColumns) {
+				require.Equal(t, false, summary.HasIndex(index))
+			}
+
+			_, err = afero.ReadFile(dataColumnStorage.fs, fixture.fileName)
+			require.ErrorIs(t, err, os.ErrNotExist)
+		}
+	})
+}
+
+func TestGetDataColumnSidecars(t *testing.T) {
+	t.Run("not found", func(t *testing.T) {
+		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
+
+		verifiedRODataColumnSidecars, err := dataColumnStorage.Get([fieldparams.RootLength]byte{1}, []uint64{12, 13, 14})
+		require.NoError(t, err)
+		require.Equal(t, 0, len(verifiedRODataColumnSidecars))
+	})
+
+	t.Run("nominal", func(t *testing.T) {
+		_, expectedVerifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
+			t,
+			verification.DataColumnsParamsByRoot{
+				{1}: {
+					{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}},
+					{ColumnIndex: 14, DataColumn: []byte{2, 3, 4}},
+				},
+			},
+		)
+
+		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
+		err := dataColumnStorage.Save(expectedVerifiedRoDataColumnSidecars)
+		require.NoError(t, err)
+
+		verifiedRODataColumnSidecars, err := dataColumnStorage.Get([fieldparams.RootLength]byte{1}, nil)
+		require.NoError(t, err)
+		require.DeepSSZEqual(t, expectedVerifiedRoDataColumnSidecars, verifiedRODataColumnSidecars)
+
+		verifiedRODataColumnSidecars, err = dataColumnStorage.Get([fieldparams.RootLength]byte{1}, []uint64{12, 13, 14})
+		require.NoError(t, err)
+		require.DeepSSZEqual(t, expectedVerifiedRoDataColumnSidecars, verifiedRODataColumnSidecars)
+	})
+}
+
+func TestRemove(t *testing.T) {
+	t.Run("not found", func(t *testing.T) {
+		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
+		err := dataColumnStorage.Remove([fieldparams.RootLength]byte{1})
+		require.NoError(t, err)
+	})
+
+	t.Run("nominal", func(t *testing.T) {
+		_, inputVerifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
+			t,
+			verification.DataColumnsParamsByRoot{
+				{1}: {
+					{Slot: 32, ColumnIndex: 10, DataColumn: []byte{1, 2, 3}},
+					{Slot: 32, ColumnIndex: 11, DataColumn: []byte{2, 3, 4}},
+				},
+				{2}: {
+					{Slot: 33, ColumnIndex: 10, DataColumn: []byte{1, 2, 3}},
+					{Slot: 33, ColumnIndex: 11, DataColumn: []byte{2, 3, 4}},
+				},
+			},
+		)
+
+		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
+		err := dataColumnStorage.Save(inputVerifiedRoDataColumnSidecars)
+		require.NoError(t, err)
+
+		err = dataColumnStorage.Remove([fieldparams.RootLength]byte{1})
+		require.NoError(t, err)
+
+		summary := dataColumnStorage.Summary([fieldparams.RootLength]byte{1})
+		require.Equal(t, primitives.Epoch(0), summary.epoch)
+		require.Equal(t, uint64(0), summary.Count())
+
+		summary = dataColumnStorage.Summary([fieldparams.RootLength]byte{2})
+		require.Equal(t, primitives.Epoch(1), summary.epoch)
+		require.Equal(t, uint64(2), summary.Count())
+
+		actual, err := dataColumnStorage.Get([fieldparams.RootLength]byte{1}, nil)
+		require.NoError(t, err)
+		require.Equal(t, 0, len(actual))
+
+		actual, err = dataColumnStorage.Get([fieldparams.RootLength]byte{2}, nil)
+		require.NoError(t, err)
+		require.Equal(t, 2, len(actual))
+	})
+}
+
+func TestClear(t *testing.T) {
+	_, inputVerifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
+		t,
+		verification.DataColumnsParamsByRoot{
+			{1}: {{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}}},
+			{2}: {{ColumnIndex: 13, DataColumn: []byte{6, 7, 8}}},
+		},
+	)
+
+	_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
+	err := dataColumnStorage.Save(inputVerifiedRoDataColumnSidecars)
+	require.NoError(t, err)
+
+	filePaths := []string{
+		"0/0/0x0100000000000000000000000000000000000000000000000000000000000000.sszs",
+		"0/0/0x0200000000000000000000000000000000000000000000000000000000000000.sszs",
+	}
+
+	for _, filePath := range filePaths {
+		_, err = afero.ReadFile(dataColumnStorage.fs, filePath)
+		require.NoError(t, err)
+	}
+
+	err = dataColumnStorage.Clear()
+	require.NoError(t, err)
+
+	summary := dataColumnStorage.Summary([fieldparams.RootLength]byte{1})
+	for index := range uint64(mandatoryNumberOfColumns) {
+		require.Equal(t, false, summary.HasIndex(index))
+	}
+
+	for _, filePath := range filePaths {
+		_, err = afero.ReadFile(dataColumnStorage.fs, filePath)
+		require.ErrorIs(t, err, os.ErrNotExist)
+	}
+}
+
+func TestMetadata(t *testing.T) {
+	t.Run("wrong version", func(t *testing.T) {
+		_, verifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
+			t,
+			verification.DataColumnsParamsByRoot{
+				{1}: {{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}}},
+			},
+		)
+
+		// Save data columns into a file.
+		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
+		err := dataColumnStorage.Save(verifiedRoDataColumnSidecars)
+		require.NoError(t, err)
+
+		// Alter the version.
+		const filePath = "0/0/0x0100000000000000000000000000000000000000000000000000000000000000.sszs"
+		file, err := dataColumnStorage.fs.OpenFile(filePath, os.O_WRONLY, os.FileMode(0600))
+		require.NoError(t, err)
+
+		count, err := file.Write([]byte{42})
+		require.NoError(t, err)
+		require.Equal(t, 1, count)
+
+		// Try to read the metadata.
+		_, err = dataColumnStorage.metadata(file)
+		require.ErrorIs(t, err, errWrongVersion)
+
+		err = file.Close()
+		require.NoError(t, err)
+	})
+}
+
+func TestNewStorageIndices(t *testing.T) {
+	t.Run("wrong number of columns", func(t *testing.T) {
+		_, err := newStorageIndices(nil)
+		require.ErrorIs(t, err, errWrongNumberOfColumns)
+	})
+
+	t.Run("nominal", func(t *testing.T) {
+		var indices [mandatoryNumberOfColumns]byte
+		indices[0] = 1
+
+		storageIndices, err := newStorageIndices(indices[:])
+		require.NoError(t, err)
+		require.Equal(t, indices, storageIndices.indices)
+	})
+}
+
+func TestStorageIndicesGet(t *testing.T) {
+	t.Run("index too large", func(t *testing.T) {
+		var indices storageIndices
+		_, _, err := indices.get(1_000_000)
+		require.ErrorIs(t, errDataColumnIndexTooLarge, err)
+	})
+
+	t.Run("index not set", func(t *testing.T) {
+		const expected = false
+		var indices storageIndices
+		actual, _, err := indices.get(0)
+		require.NoError(t, err)
+		require.Equal(t, expected, actual)
+	})
+
+	t.Run("index set", func(t *testing.T) {
+		const (
+			expectedOk       = true
+			expectedPosition = int64(3)
+		)
+
+		indices := storageIndices{indices: [mandatoryNumberOfColumns]byte{0, 131}}
+		actualOk, actualPosition, err := indices.get(1)
+		require.NoError(t, err)
+		require.Equal(t, expectedOk, actualOk)
+		require.Equal(t, expectedPosition, actualPosition)
+	})
+}
+
+func TestStorageIndicesLen(t *testing.T) {
+	const expected = int64(2)
+	indices := storageIndices{count: 2}
+	actual := indices.len()
+	require.Equal(t, expected, actual)
+}
+
+func TestStorageIndicesAll(t *testing.T) {
+	expectedIndices := []uint64{1, 3}
+	indices := storageIndices{indices: [mandatoryNumberOfColumns]byte{0, 131, 0, 128}}
+	actualIndices := indices.all()
+	require.DeepEqual(t, expectedIndices, actualIndices)
+}
+
+func TestStorageIndicesSet(t *testing.T) {
+	t.Run("data column index too large", func(t *testing.T) {
+		var indices storageIndices
+		err := indices.set(1_000_000, 0)
+		require.ErrorIs(t, errDataColumnIndexTooLarge, err)
+	})
+
+	t.Run("position too large", func(t *testing.T) {
+		var indices storageIndices
+		err := indices.set(0, 255)
+		require.ErrorIs(t, errDataColumnIndexTooLarge, err)
+	})
+
+	t.Run("nominal", func(t *testing.T) {
+		expected := [mandatoryNumberOfColumns]byte{0, 0, 128, 0, 131}
+		var storageIndices storageIndices
+		require.Equal(t, int64(0), storageIndices.len())
+
+		err := storageIndices.set(2, 1)
+		require.NoError(t, err)
+		require.Equal(t, int64(1), storageIndices.len())
+
+		err = storageIndices.set(4, 3)
+		require.NoError(t, err)
+		require.Equal(t, int64(2), storageIndices.len())
+
+		err = storageIndices.set(2, 0)
+		require.NoError(t, err)
+		require.Equal(t, int64(2), storageIndices.len())
+
+		actual := storageIndices.indices
+		require.Equal(t, expected, actual)
+	})
+}
+
+func TestPrune(t *testing.T) {
+	t.Run(("nothing to prune"), func(t *testing.T) {
+		dir := t.TempDir()
+		dataColumnStorage, err := NewDataColumnStorage(context.Background(), WithDataColumnBasePath(dir))
+		require.NoError(t, err)
+
+		dataColumnStorage.prune()
+	})
+	t.Run("nominal", func(t *testing.T) {
+		var compareSlices = func(left, right []string) bool {
+			if len(left) != len(right) {
+				return false
+			}
+
+			leftMap := make(map[string]bool, len(left))
+			for _, leftItem := range left {
+				leftMap[leftItem] = true
+			}
+
+			for _, rightItem := range right {
+				if _, ok := leftMap[rightItem]; !ok {
+					return false
+				}
+			}
+
+			return true
+		}
+		_, verifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
+			t,
+			verification.DataColumnsParamsByRoot{
+				{0}: {
+					{Slot: 33, ColumnIndex: 2, DataColumn: []byte{1, 2, 3}}, // Period 0 - Epoch 1
+					{Slot: 33, ColumnIndex: 4, DataColumn: []byte{2, 3, 4}}, // Period 0 - Epoch 1
+				},
+				{1}: {
+					{Slot: 128_002, ColumnIndex: 2, DataColumn: []byte{1, 2, 3}}, // Period 0 - Epoch 4000
+					{Slot: 128_002, ColumnIndex: 4, DataColumn: []byte{2, 3, 4}}, // Period 0 - Epoch 4000
+				},
+				{2}: {
+					{Slot: 128_003, ColumnIndex: 1, DataColumn: []byte{1, 2, 3}}, // Period 0 - Epoch 4000
+					{Slot: 128_003, ColumnIndex: 3, DataColumn: []byte{2, 3, 4}}, // Period 0 - Epoch 4000
+				},
+				{3}: {
+					{Slot: 131_138, ColumnIndex: 2, DataColumn: []byte{1, 2, 3}}, // Period 1 - Epoch 4098
+					{Slot: 131_138, ColumnIndex: 3, DataColumn: []byte{1, 2, 3}}, // Period 1 - Epoch 4098
+				},
+				{4}: {
+					{Slot: 131_169, ColumnIndex: 2, DataColumn: []byte{1, 2, 3}}, // Period 1 - Epoch 4099
+					{Slot: 131_169, ColumnIndex: 3, DataColumn: []byte{1, 2, 3}}, // Period 1 - Epoch 4099
+				},
+				{5}: {
+					{Slot: 262_144, ColumnIndex: 2, DataColumn: []byte{1, 2, 3}}, // Period 2 - Epoch 8192
+					{Slot: 262_144, ColumnIndex: 3, DataColumn: []byte{1, 2, 3}}, // Period 2 - Epoch 8292
+				},
+			},
+		)
+
+		dir := t.TempDir()
+		dataColumnStorage, err := NewDataColumnStorage(context.Background(), WithDataColumnBasePath(dir), WithDataColumnRetentionEpochs(10_000))
+		require.NoError(t, err)
+
+		err = dataColumnStorage.Save(verifiedRoDataColumnSidecars)
+		require.NoError(t, err)
+
+		dirs, err := listDir(dataColumnStorage.fs, ".")
+		require.NoError(t, err)
+		require.Equal(t, true, compareSlices([]string{"0", "1", "2"}, dirs))
+
+		dirs, err = listDir(dataColumnStorage.fs, "0")
+		require.NoError(t, err)
+		require.Equal(t, true, compareSlices([]string{"1", "4000"}, dirs))
+
+		dirs, err = listDir(dataColumnStorage.fs, "1")
+		require.NoError(t, err)
+		require.Equal(t, true, compareSlices([]string{"4099", "4098"}, dirs))
+
+		dirs, err = listDir(dataColumnStorage.fs, "2")
+		require.NoError(t, err)
+		require.Equal(t, true, compareSlices([]string{"8192"}, dirs))
+
+		dirs, err = listDir(dataColumnStorage.fs, "0/1")
+		require.NoError(t, err)
+		require.Equal(t, true, compareSlices([]string{"0x0000000000000000000000000000000000000000000000000000000000000000.sszs"}, dirs))
+
+		dirs, err = listDir(dataColumnStorage.fs, "0/4000")
+		require.NoError(t, err)
+		require.Equal(t, true, compareSlices([]string{
+			"0x0200000000000000000000000000000000000000000000000000000000000000.sszs",
+			"0x0100000000000000000000000000000000000000000000000000000000000000.sszs",
+		}, dirs))
+
+		dirs, err = listDir(dataColumnStorage.fs, "1/4098")
+		require.NoError(t, err)
+		require.Equal(t, true, compareSlices([]string{"0x0300000000000000000000000000000000000000000000000000000000000000.sszs"}, dirs))
+
+		dirs, err = listDir(dataColumnStorage.fs, "1/4099")
+		require.NoError(t, err)
+		require.Equal(t, true, compareSlices([]string{"0x0400000000000000000000000000000000000000000000000000000000000000.sszs"}, dirs))
+
+		dirs, err = listDir(dataColumnStorage.fs, "2/8192")
+		require.NoError(t, err)
+		require.Equal(t, true, compareSlices([]string{"0x0500000000000000000000000000000000000000000000000000000000000000.sszs"}, dirs))
+
+		_, verifiedRoDataColumnSidecars = verification.CreateTestVerifiedRoDataColumnSidecars(
+			t,
+			verification.DataColumnsParamsByRoot{
+				{6}: {{Slot: 451_141, ColumnIndex: 2, DataColumn: []byte{1, 2, 3}}}, // Period 3 - Epoch 14_098
+			},
+		)
+
+		err = dataColumnStorage.Save(verifiedRoDataColumnSidecars)
+		require.NoError(t, err)
+
+		// dataColumnStorage.prune(14_098)
+		dataColumnStorage.prune()
+
+		dirs, err = listDir(dataColumnStorage.fs, ".")
+		require.NoError(t, err)
+		require.Equal(t, true, compareSlices([]string{"1", "2", "3"}, dirs))
+
+		dirs, err = listDir(dataColumnStorage.fs, "1")
+		require.NoError(t, err)
+		require.Equal(t, true, compareSlices([]string{"4099"}, dirs))
+
+		dirs, err = listDir(dataColumnStorage.fs, "2")
+		require.NoError(t, err)
+		require.Equal(t, true, compareSlices([]string{"8192"}, dirs))
+
+		dirs, err = listDir(dataColumnStorage.fs, "3")
+		require.NoError(t, err)
+		require.Equal(t, true, compareSlices([]string{"14098"}, dirs))
+
+		dirs, err = listDir(dataColumnStorage.fs, "1/4099")
+		require.NoError(t, err)
+		require.Equal(t, true, compareSlices([]string{"0x0400000000000000000000000000000000000000000000000000000000000000.sszs"}, dirs))
+
+		dirs, err = listDir(dataColumnStorage.fs, "2/8192")
+		require.NoError(t, err)
+		require.Equal(t, true, compareSlices([]string{"0x0500000000000000000000000000000000000000000000000000000000000000.sszs"}, dirs))
+
+		dirs, err = listDir(dataColumnStorage.fs, "3/14098")
+		require.NoError(t, err)
+		require.Equal(t, true, compareSlices([]string{"0x0600000000000000000000000000000000000000000000000000000000000000.sszs"}, dirs))
+	})
+}

beacon-chain/db/filesystem/doc.go

@@ -0,0 +1,104 @@
+package filesystem
+
+// nolint:dupword
+/*
+Data column sidecars storage documentation
+==========================================
+
+File organisation
+-----------------
+- The first byte represents the version of the file structure (up to 0xff = 255).
+  We set it to 0x01.
+  Note: This is not strictly needed, but it will help a lot if, in the future,
+        we want to modify the file structure.
+- The next 4 bytes represents the size of a SSZ encoded data column sidecar.
+  (See the `Computation of the maximum size of a DataColumnSidecar` section to a description
+  of how this value is computed).
+- The next 128 bytes represent the index in the file of a given column.
+  The first bit of each byte in the index is set to 0 if there is no data column,
+  and set to 1 if there is a data column.
+  The remaining 7 bits (from 0 to 127) represent the index of the data column.
+  This sentinel bit is needed to distinguish between the column with index 0 and no column.
+  Example: If the column with index 5 is in the 3th position in the file, then indices[5] = 0x80 + 0x03 = 0x83.
+- The rest of the file is a repeat of the SSZ encoded data columns sidecars.
+
+
+|------------------------------------------|------------------------------------------------------------------------------------|
+|                Byte offset               |                                     Description                                    |
+|------------------------------------------|------------------------------------------------------------------------------------|
+|                                        0 | version (1 byte) | sszEncodedDataColumnSidecarSize (4 bytes) | indices (128 bytes) |
+|133 +   0*sszEncodedDataColumnSidecarSize | sszEncodedDataColumnSidecar (sszEncodedDataColumnSidecarSize bytes)                |
+|133 +   1*sszEncodedDataColumnSidecarSize | sszEncodedDataColumnSidecar (sszEncodedDataColumnSidecarSize bytes)                |
+|133 +   2*sszEncodedDataColumnSidecarSize | sszEncodedDataColumnSidecar (sszEncodedDataColumnSidecarSize bytes)                |
+|                  ...                     |                                 ...                                                |
+|133 + 127*sszEncodedDataColumnSidecarSize | sszEncodedDataColumnSidecar (sszEncodedDataColumnSidecarSize bytes)                |
+|------------------------------------------|------------------------------------------------------------------------------------|
+
+Each file is named after the block root where the data columns were data columns are committed to.
+Example: `0x259c6d2f6a0bb75e2405cea7cb248e5663dc26b9404fd3bcd777afc20de91c1e.sszs`
+
+Database organisation
+---------------------
+SSZ encoded data column sidecars are stored following the `by-epoch` layout.
+- The first layer is a directory corresponding to the `period`, which corresponds to the epoch divided by the 4096.
+- The second layer is a directory corresponding to the epoch.
+- Then all files are stored in the epoch directory.
+
+Example:
+data-columns
+├── 0
+│   ├── 3638
+│   │   ├── 0x259c6d2f6a0bb75e2405cea7cb248e5663dc26b9404fd3bcd777afc20de91c1e.sszs
+│   │   ├── 0x2a855b1f6e9a2f04f8383e336325bf7d5ba02d1eab3ef90ef183736f8c768533.sszs
+│   │   ├── ...
+│   │   ├── 0xeb78e2b2350a71c640f1e96fea9e42f38e65705ab7e6e100c8bc9c589f2c5f2b.sszs
+│   │   └── 0xeb7ee68da988fd20d773d45aad01dd62527734367a146e2b048715bd68a4e370.sszs
+│   └── 3639
+│       ├── 0x0fd231fe95e57936fa44f6c712c490b9e337a481b661dfd46768901e90444330.sszs
+│       ├── 0x1bf5edff6b6ba2b65b1db325ff3312bbb57da461ef2ae651bd741af851aada3a.sszs
+│       ├── ...
+│       ├── 0xa156a527e631f858fee79fab7ef1fde3f6117a2e1201d47c09fbab0c6780c937.sszs
+│       └── 0xcd80bc535ddc467dea1d19e0c39c1160875ccd1989061bcd8ce206e3c1261c87.sszs
+└── 1
+    ├── 4096
+    │   ├── 0x0d244009093e2bedb72eb265280290199e8c7bf1d90d7583c41af40d9f662269.sszs
+    │   ├── 0x11f420928d8de41c50e735caab0369996824a5299c5f054e097965855925697d.sszs
+    │   ├── ...
+    │   ├── 0xbe91fc782877ed400d95c02c61aebfdd592635d11f8e64c94b46abd84f45c967.sszs
+    │   └── 0xf246189f078f02d30173ff74605cf31c9e65b5e463275ebdbeb40476638135ff.sszs
+    └── 4097
+        ├── 0x454d000674793c479e90504c0fe9827b50bb176ae022dab4e37d6a21471ab570.sszs
+        ├── 0xac5eb7437d7190c48cfa863e3c45f96a7f8af371d47ac12ccda07129a06af763.sszs
+        ├── ...
+        ├── 0xb7df30561d9d92ab5fafdd96bca8b44526497c8debf0fc425c7a0770b2abeb83.sszs
+        └── 0xc1dd0b1ae847b6ec62303a36d08c6a4a2e9e3ec4be3ff70551972a0ee3de9c14.sszs
+
+Computation of the maximum size of a DataColumnSidecar
+------------------------------------------------------
+https://github.com/ethereum/consensus-specs/blob/dev/specs/fulu/das-core.md#datacolumnsidecar
+
+
+class DataColumnSidecar(Container):
+    index: ColumnIndex  # Index of column in extended matrix
+    column: List[Cell, MAX_BLOB_COMMITMENTS_PER_BLOCK]
+    kzg_commitments: List[KZGCommitment, MAX_BLOB_COMMITMENTS_PER_BLOCK]
+    kzg_proofs: List[KZGProof, MAX_BLOB_COMMITMENTS_PER_BLOCK]
+    signed_block_header: SignedBeaconBlockHeader
+    kzg_commitments_inclusion_proof: Vector[Bytes32, KZG_COMMITMENTS_INCLUSION_PROOF_DEPTH]
+
+
+- index: 2 bytes (ColumnIndex)
+- `column`: 4,096 (MAX_BLOB_COMMITMENTS_PER_BLOCK) * 64 (FIELD_ELEMENTS_PER_CELL) * 32 bytes (BYTES_PER_FIELD_ELEMENT) = 8,388,608 bytes
+- kzg_commitments: 4,096 (MAX_BLOB_COMMITMENTS_PER_BLOCK) * 48 bytes (KZGCommitment) = 196,608 bytes
+- kzg_proofs: 4,096 (MAX_BLOB_COMMITMENTS_PER_BLOCK) * 48 bytes (KZGProof) = 196,608 bytes
+- signed_block_header: 2 bytes (Slot) + 2 bytes (ValidatorIndex) + 3 * 2 bytes (Root) + 96 bytes (BLSSignature) = 106 bytes
+- kzg_commitments_inclusion_proof: 4 (KZG_COMMITMENTS_INCLUSION_PROOF_DEPTH) * 32 bytes = 128 bytes
+
+TOTAL: 8,782,060 bytes = 70,256,480 bits
+log(70,256,480) / log(2) ~= 26.07
+
+==> 32 bits (4 bytes) are enough to store the maximum size of a data column sidecar.
+
+The maximum size of an SSZ encoded data column can be 2**32 bits = 536,879,912 bytes,
+which left a room of 536,879,912 bytes - 8,782,060 bytes ~= 503 mega bytes to store the extra data needed by SSZ encoding (which is more than enough.)
+*/

beacon-chain/db/filesystem/iteration_test.go

@@ -1,7 +1,6 @@
 package filesystem
 
 import (
-	"bytes"
 	"fmt"
 	"io"
 	"math"
@@ -220,28 +219,6 @@ func TestListDir(t *testing.T) {
 	}
 }
 
-func TestSlotFromBlob(t *testing.T) {
-	cases := []struct {
-		slot primitives.Slot
-	}{
-		{slot: 0},
-		{slot: 2},
-		{slot: 1123581321},
-		{slot: math.MaxUint64},
-	}
-	for _, c := range cases {
-		t.Run(fmt.Sprintf("slot %d", c.slot), func(t *testing.T) {
-			_, sidecars := util.GenerateTestDenebBlockWithSidecar(t, [32]byte{}, c.slot, 1)
-			sc := sidecars[0]
-			enc, err := sc.MarshalSSZ()
-			require.NoError(t, err)
-			slot, err := slotFromBlob(bytes.NewReader(enc))
-			require.NoError(t, err)
-			require.Equal(t, c.slot, slot)
-		})
-	}
-}
-
 func TestIterationComplete(t *testing.T) {
 	targets := []migrationTestTarget{
 		{

beacon-chain/db/filesystem/layout_flat.go

@@ -1,12 +1,11 @@
 package filesystem
 
 import (
-	"encoding/binary"
-	"io"
 	"os"
 	"path"
 
 	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
+	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
 	"github.com/OffchainLabs/prysm/v6/time/slots"
 	"github.com/pkg/errors"
 	"github.com/spf13/afero"
@@ -115,29 +114,25 @@ func (l *flatLayout) pruneBefore(before primitives.Epoch) (*pruneSummary, error)
 
 // Read slot from marshaled BlobSidecar data in the given file. See slotFromBlob for details.
 func slotFromFile(name string, fs afero.Fs) (primitives.Slot, error) {
-	f, err := fs.Open(name)
+	// Read whole file, try to unmarshal it as data columns sidecar, if it fails, try to unmarshal it as blob sidecar.
+	content, err := afero.ReadFile(fs, name)
 	if err != nil {
 		return 0, err
 	}
-	defer func() {
-		if err := f.Close(); err != nil {
-			log.WithError(err).Errorf("Could not close blob file")
-		}
-	}()
-	return slotFromBlob(f)
-}
 
-// slotFromBlob reads the ssz data of a file at the specified offset (8 + 131072 + 48 + 48 = 131176 bytes),
-// which is calculated based on the size of the BlobSidecar struct and is based on the size of the fields
-// preceding the slot information within SignedBeaconBlockHeader.
-func slotFromBlob(at io.ReaderAt) (primitives.Slot, error) {
-	b := make([]byte, 8)
-	_, err := at.ReadAt(b, 131176)
-	if err != nil {
-		return 0, err
+	dataColumnSidecar := &ethpb.DataColumnSidecar{}
+	err = dataColumnSidecar.UnmarshalSSZ(content)
+	if err == nil {
+		return dataColumnSidecar.SignedBlockHeader.Header.Slot, nil
 	}
-	rawSlot := binary.LittleEndian.Uint64(b)
-	return primitives.Slot(rawSlot), nil
+
+	blobSidecar := &ethpb.BlobSidecar{}
+	err = blobSidecar.UnmarshalSSZ(content)
+	if err == nil {
+		return blobSidecar.SignedBlockHeader.Header.Slot, nil
+	}
+
+	return 0, errors.New("failed to unmarshal as blob sidecar or data column sidecar")
 }
 
 type flatSlotReader struct {

beacon-chain/db/filesystem/metrics.go

@@ -6,6 +6,7 @@ import (
 )
 
 var (
+	// Blobs
 	blobBuckets     = []float64{3, 5, 7, 9, 11, 13}
 	blobSaveLatency = promauto.NewHistogram(prometheus.HistogramOpts{
 		Name:    "blob_storage_save_latency",
@@ -33,4 +34,29 @@ var (
 		Name: "blob_disk_bytes",
 		Help: "Approximate number of bytes occupied by blobs in storage",
 	})
+
+	// Data columns
+	dataColumnBuckets     = []float64{3, 5, 7, 9, 11, 13}
+	dataColumnSaveLatency = promauto.NewHistogram(prometheus.HistogramOpts{
+		Name:    "data_column_storage_save_latency",
+		Help:    "Latency of DataColumnSidecar storage save operations in milliseconds",
+		Buckets: dataColumnBuckets,
+	})
+	dataColumnFetchLatency = promauto.NewHistogram(prometheus.HistogramOpts{
+		Name:    "data_column_storage_get_latency",
+		Help:    "Latency of DataColumnSidecar storage get operations in milliseconds",
+		Buckets: dataColumnBuckets,
+	})
+	dataColumnPrunedCounter = promauto.NewCounter(prometheus.CounterOpts{
+		Name: "data_column_pruned",
+		Help: "Number of DataColumnSidecar pruned.",
+	})
+	dataColumnWrittenCounter = promauto.NewCounter(prometheus.CounterOpts{
+		Name: "data_column_written",
+		Help: "Number of DataColumnSidecar written",
+	})
+	dataColumnDiskCount = promauto.NewGauge(prometheus.GaugeOpts{
+		Name: "data_column_disk_count",
+		Help: "Approximate number of data columns in storage",
+	})
 )

beacon-chain/db/filesystem/mock.go

@@ -1,14 +1,19 @@
 package filesystem
 
 import (
+	"context"
 	"testing"
 
+	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
 	"github.com/OffchainLabs/prysm/v6/config/params"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
 	"github.com/OffchainLabs/prysm/v6/time/slots"
 	"github.com/spf13/afero"
 )
 
+// Blobs
+// -----
+
 // NewEphemeralBlobStorage should only be used for tests.
 // The instance of BlobStorage returned is backed by an in-memory virtual filesystem,
 // improving test performance and simplifying cleanup.
@@ -75,3 +80,52 @@ func NewMockBlobStorageSummarizer(t *testing.T, set map[[32]byte][]int) BlobStor
 	}
 	return c
 }
+
+// Data columns
+// ------------
+
+// NewEphemeralDataColumnStorage should only be used for tests.
+// The instance of DataColumnStorage returned is backed by an in-memory virtual filesystem,
+// improving test performance and simplifying cleanup.
+func NewEphemeralDataColumnStorage(t testing.TB, opts ...DataColumnStorageOption) *DataColumnStorage {
+	return NewWarmedEphemeralDataColumnStorageUsingFs(t, afero.NewMemMapFs(), opts...)
+}
+
+// NewEphemeralDataColumnStorageAndFs can be used by tests that want access to the virtual filesystem
+// in order to interact with it outside the parameters of the DataColumnStorage API.
+func NewEphemeralDataColumnStorageAndFs(t testing.TB, opts ...DataColumnStorageOption) (afero.Fs, *DataColumnStorage) {
+	fs := afero.NewMemMapFs()
+	dcs := NewWarmedEphemeralDataColumnStorageUsingFs(t, fs, opts...)
+	return fs, dcs
+}
+
+func NewWarmedEphemeralDataColumnStorageUsingFs(t testing.TB, fs afero.Fs, opts ...DataColumnStorageOption) *DataColumnStorage {
+	bs := NewEphemeralDataColumnStorageUsingFs(t, fs, opts...)
+	bs.WarmCache()
+	return bs
+}
+
+func NewEphemeralDataColumnStorageUsingFs(t testing.TB, fs afero.Fs, opts ...DataColumnStorageOption) *DataColumnStorage {
+	opts = append(opts,
+		WithDataColumnRetentionEpochs(params.BeaconConfig().MinEpochsForBlobsSidecarsRequest),
+		WithDataColumnFs(fs),
+	)
+
+	bs, err := NewDataColumnStorage(context.Background(), opts...)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	return bs
+}
+
+func NewMockDataColumnStorageSummarizer(t *testing.T, set map[[fieldparams.RootLength]byte][]uint64) DataColumnStorageSummarizer {
+	c := newDataColumnStorageSummaryCache()
+	for root, indices := range set {
+		if err := c.set(DataColumnsIdent{Root: root, Epoch: 0, Indices: indices}); err != nil {
+			t.Fatal(err)
+		}
+	}
+
+	return c
+}

beacon-chain/db/kv/BUILD.bazel

@@ -59,7 +59,7 @@ go_library(
         "//runtime/version:go_default_library",
         "//time:go_default_library",
         "//time/slots:go_default_library",
-        "@com_github_dgraph_io_ristretto_v2//:go_default_library",
+        "@com_github_dgraph_io_ristretto//:go_default_library",
         "@com_github_ethereum_go_ethereum//common:go_default_library",
         "@com_github_ethereum_go_ethereum//common/hexutil:go_default_library",
         "@com_github_golang_snappy//:go_default_library",

beacon-chain/db/kv/blocks.go

@@ -40,7 +40,7 @@ func (s *Store) Block(ctx context.Context, blockRoot [32]byte) (interfaces.ReadO
 
 func (s *Store) getBlock(ctx context.Context, blockRoot [32]byte, tx *bolt.Tx) (interfaces.ReadOnlySignedBeaconBlock, error) {
 	if v, ok := s.blockCache.Get(string(blockRoot[:])); v != nil && ok {
-		return v, nil
+		return v.(interfaces.ReadOnlySignedBeaconBlock), nil
 	}
 	// This method allows the caller to pass in its tx if one is already open.
 	// Or if a nil value is used, a transaction will be managed intenally.

beacon-chain/db/kv/error.go

@@ -16,6 +16,9 @@ var ErrNotFoundOriginBlockRoot = errors.Wrap(ErrNotFound, "OriginBlockRoot")
 // ErrNotFoundGenesisBlockRoot means no genesis block root was found, indicating the db was not initialized with genesis
 var ErrNotFoundGenesisBlockRoot = errors.Wrap(ErrNotFound, "OriginGenesisRoot")
 
+// ErrNotFoundBackfillBlockRoot is an error specifically for the origin block root getter
+var ErrNotFoundBackfillBlockRoot = errors.Wrap(ErrNotFound, "BackfillBlockRoot")
+
 // ErrNotFoundFeeRecipient is a not found error specifically for the fee recipient getter
 var ErrNotFoundFeeRecipient = errors.Wrap(ErrNotFound, "fee recipient")
 

beacon-chain/db/kv/kv.go

@@ -13,10 +13,8 @@ import (
 	"github.com/OffchainLabs/prysm/v6/config/features"
 	"github.com/OffchainLabs/prysm/v6/config/params"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
-	"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
 	"github.com/OffchainLabs/prysm/v6/io/file"
-	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
-	"github.com/dgraph-io/ristretto/v2"
+	"github.com/dgraph-io/ristretto"
 	"github.com/pkg/errors"
 	"github.com/prometheus/client_golang/prometheus"
 	"github.com/prometheus/client_golang/prometheus/promauto"
@@ -88,8 +86,8 @@ var blockedBuckets = [][]byte{
 type Store struct {
 	db                  *bolt.DB
 	databasePath        string
-	blockCache          *ristretto.Cache[string, interfaces.ReadOnlySignedBeaconBlock]
-	validatorEntryCache *ristretto.Cache[[]byte, *ethpb.Validator]
+	blockCache          *ristretto.Cache
+	validatorEntryCache *ristretto.Cache
 	stateSummaryCache   *stateSummaryCache
 	ctx                 context.Context
 }
@@ -158,7 +156,7 @@ func NewKVStore(ctx context.Context, dirPath string, opts ...KVStoreOption) (*St
 		return nil, err
 	}
 	boltDB.AllocSize = boltAllocSize
-	blockCache, err := ristretto.NewCache(&ristretto.Config[string, interfaces.ReadOnlySignedBeaconBlock]{
+	blockCache, err := ristretto.NewCache(&ristretto.Config{
 		NumCounters: 1000,           // number of keys to track frequency of (1000).
 		MaxCost:     BlockCacheSize, // maximum cost of cache (1000 Blocks).
 		BufferItems: 64,             // number of keys per Get buffer.
@@ -167,7 +165,7 @@ func NewKVStore(ctx context.Context, dirPath string, opts ...KVStoreOption) (*St
 		return nil, err
 	}
 
-	validatorCache, err := ristretto.NewCache(&ristretto.Config[[]byte, *ethpb.Validator]{
+	validatorCache, err := ristretto.NewCache(&ristretto.Config{
 		NumCounters: NumOfValidatorEntries, // number of entries in cache (2 Million).
 		MaxCost:     ValidatorEntryMaxCost, // maximum size of the cache (64Mb)
 		BufferItems: 64,                    // number of keys per Get buffer.

beacon-chain/db/kv/state.go

@@ -744,9 +744,14 @@ func (s *Store) validatorEntries(ctx context.Context, blockRoot [32]byte) ([]*et
 			// get the entry bytes from the cache or from the DB.
 			v, ok := s.validatorEntryCache.Get(key)
 			if ok {
-				valEntry := v
-				validatorEntries = append(validatorEntries, valEntry)
-				validatorEntryCacheHit.Inc()
+				valEntry, vType := v.(*ethpb.Validator)
+				if vType {
+					validatorEntries = append(validatorEntries, valEntry)
+					validatorEntryCacheHit.Inc()
+				} else {
+					// this should never happen, but anyway it's good to bail out if one happens.
+					return errors.New("validator cache does not have proper object type")
+				}
 			} else {
 				// not in cache, so get it from the DB, decode it and add to the entry list.
 				valEntryBytes := valBkt.Get(key)

beacon-chain/db/kv/state_test.go

@@ -321,11 +321,15 @@ func TestState_CanSaveRetrieveValidatorEntriesFromCache(t *testing.T) {
 		hash, hashErr := stateValidators[i].HashTreeRoot()
 		assert.NoError(t, hashErr)
 
-		data, ok := db.validatorEntryCache.Get(hash[:])
+		data, ok := db.validatorEntryCache.Get(string(hash[:]))
 		assert.Equal(t, true, ok)
 		require.NotNil(t, data)
 
-		require.DeepSSZEqual(t, stateValidators[i], data, "validator entry is not matching")
+		valEntry, vType := data.(*ethpb.Validator)
+		assert.Equal(t, true, vType)
+		require.NotNil(t, valEntry)
+
+		require.DeepSSZEqual(t, stateValidators[i], valEntry, "validator entry is not matching")
 	}
 
 	// check if all the validator entries are still intact in the validator entry bucket.
@@ -443,7 +447,7 @@ func TestState_DeleteState(t *testing.T) {
 		assert.NoError(t, hashErr)
 		v, found := db.validatorEntryCache.Get(hash[:])
 		require.Equal(t, false, found)
-		require.IsNil(t, v)
+		require.Equal(t, nil, v)
 	}
 
 	// check if the index of the first state is deleted.

beacon-chain/execution/BUILD.bazel

@@ -25,12 +25,14 @@ go_library(
         "//testing/spectest:__subpackages__",
     ],
     deps = [
+        "//beacon-chain/blockchain/kzg:go_default_library",
         "//beacon-chain/cache:go_default_library",
         "//beacon-chain/cache/depositsnapshot:go_default_library",
         "//beacon-chain/core/altair:go_default_library",
         "//beacon-chain/core/feed:go_default_library",
         "//beacon-chain/core/feed/state:go_default_library",
         "//beacon-chain/core/helpers:go_default_library",
+        "//beacon-chain/core/peerdas:go_default_library",
         "//beacon-chain/core/transition:go_default_library",
         "//beacon-chain/db:go_default_library",
         "//beacon-chain/execution/types:go_default_library",
@@ -97,6 +99,7 @@ go_test(
     embed = [":go_default_library"],
     deps = [
         "//async/event:go_default_library",
+        "//beacon-chain/blockchain/kzg:go_default_library",
         "//beacon-chain/cache/depositsnapshot:go_default_library",
         "//beacon-chain/core/feed:go_default_library",
         "//beacon-chain/core/feed/state:go_default_library",

beacon-chain/execution/engine_client.go

@@ -7,6 +7,8 @@ import (
 	"strings"
 	"time"
 
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg"
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/execution/types"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/verification"
 	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
@@ -44,11 +46,16 @@ var (
 		GetPayloadMethodV3,
 		GetPayloadBodiesByHashV1,
 		GetPayloadBodiesByRangeV1,
+		GetBlobsV1,
 	}
 	electraEngineEndpoints = []string{
 		NewPayloadMethodV4,
 		GetPayloadMethodV4,
 	}
+	fuluEngineEndpoints = []string{
+		GetPayloadMethodV5,
+		GetBlobsV2,
+	}
 )
 
 const (
@@ -73,6 +80,8 @@ const (
 	GetPayloadMethodV3 = "engine_getPayloadV3"
 	// GetPayloadMethodV4 is the get payload method added for electra
 	GetPayloadMethodV4 = "engine_getPayloadV4"
+	// GetPayloadMethodV5 is the get payload method added for fulu
+	GetPayloadMethodV5 = "engine_getPayloadV5"
 	// BlockByHashMethod request string for JSON-RPC.
 	BlockByHashMethod = "eth_getBlockByHash"
 	// BlockByNumberMethod request string for JSON-RPC.
@@ -85,8 +94,15 @@ const (
 	ExchangeCapabilities = "engine_exchangeCapabilities"
 	// GetBlobsV1 request string for JSON-RPC.
 	GetBlobsV1 = "engine_getBlobsV1"
+	// GetBlobsV2 request string for JSON-RPC.
+	GetBlobsV2 = "engine_getBlobsV2"
 	// Defines the seconds before timing out engine endpoints with non-block execution semantics.
-	defaultEngineTimeout = time.Second
+	// TODO: Remove temporarily needed hack since geth takes an input blobs txs with blobs proofs, and
+	// does the heavy lifting of building cells proofs, while normally this is done by the tx sender.
+	// This is a cool hack because it lets the CL to act as if the tx sender actually computed the cells proofs.
+	// The only counter part is the `engine_getPayloadv<x>` takes a lot of time.
+	// defaultEngineTimeout = time.Second
+	defaultEngineTimeout = 2 * time.Second
 )
 
 var errInvalidPayloadBodyResponse = errors.New("engine api payload body response is invalid")
@@ -108,6 +124,7 @@ type Reconstructor interface {
 		ctx context.Context, blindedBlocks []interfaces.ReadOnlySignedBeaconBlock,
 	) ([]interfaces.SignedBeaconBlock, error)
 	ReconstructBlobSidecars(ctx context.Context, block interfaces.ReadOnlySignedBeaconBlock, blockRoot [32]byte, hi func(uint64) bool) ([]blocks.VerifiedROBlob, error)
+	ReconstructDataColumnSidecars(ctx context.Context, block interfaces.ReadOnlySignedBeaconBlock, blockRoot [32]byte) ([]blocks.VerifiedRODataColumn, error)
 }
 
 // EngineCaller defines a client that can interact with an Ethereum
@@ -257,6 +274,9 @@ func (s *Service) ForkchoiceUpdated(
 
 func getPayloadMethodAndMessage(slot primitives.Slot) (string, proto.Message) {
 	pe := slots.ToEpoch(slot)
+	if pe >= params.BeaconConfig().FuluForkEpoch {
+		return GetPayloadMethodV5, &pb.ExecutionBundleFulu{}
+	}
 	if pe >= params.BeaconConfig().ElectraForkEpoch {
 		return GetPayloadMethodV4, &pb.ExecutionBundleElectra{}
 	}
@@ -289,7 +309,7 @@ func (s *Service) GetPayload(ctx context.Context, payloadId [8]byte, slot primit
 	}
 	res, err := blocks.NewGetPayloadResponse(result)
 	if err != nil {
-		return nil, err
+		return nil, errors.Wrap(err, "new get payload response")
 	}
 	return res, nil
 }
@@ -302,6 +322,9 @@ func (s *Service) ExchangeCapabilities(ctx context.Context) ([]string, error) {
 	if params.ElectraEnabled() {
 		supportedEngineEndpoints = append(supportedEngineEndpoints, electraEngineEndpoints...)
 	}
+	if params.FuluEnabled() {
+		supportedEngineEndpoints = append(supportedEngineEndpoints, fuluEngineEndpoints...)
+	}
 	var result []string
 	err := s.rpcClient.CallContext(ctx, &result, ExchangeCapabilities, supportedEngineEndpoints)
 	if err != nil {
@@ -495,9 +518,10 @@ func (s *Service) HeaderByNumber(ctx context.Context, number *big.Int) (*types.H
 func (s *Service) GetBlobs(ctx context.Context, versionedHashes []common.Hash) ([]*pb.BlobAndProof, error) {
 	ctx, span := trace.StartSpan(ctx, "powchain.engine-api-client.GetBlobs")
 	defer span.End()
+
 	// If the execution engine does not support `GetBlobsV1`, return early to prevent encountering an error later.
 	if !s.capabilityCache.has(GetBlobsV1) {
-		return nil, nil
+		return nil, errors.New(fmt.Sprintf("%s is not supported", GetBlobsV1))
 	}
 
 	result := make([]*pb.BlobAndProof, len(versionedHashes))
@@ -505,6 +529,19 @@ func (s *Service) GetBlobs(ctx context.Context, versionedHashes []common.Hash) (
 	return result, handleRPCError(err)
 }
 
+func (s *Service) GetBlobsV2(ctx context.Context, versionedHashes []common.Hash) ([]*pb.BlobAndProofV2, error) {
+	ctx, span := trace.StartSpan(ctx, "powchain.engine-api-client.GetBlobsV2")
+	defer span.End()
+
+	if !s.capabilityCache.has(GetBlobsV2) {
+		return nil, errors.New(fmt.Sprintf("%s is not supported", GetBlobsV2))
+	}
+
+	result := make([]*pb.BlobAndProofV2, len(versionedHashes))
+	err := s.rpcClient.CallContext(ctx, &result, GetBlobsV2, versionedHashes)
+	return result, handleRPCError(err)
+}
+
 // ReconstructFullBlock takes in a blinded beacon block and reconstructs
 // a beacon block with a full execution payload via the engine API.
 func (s *Service) ReconstructFullBlock(
@@ -615,6 +652,85 @@ func (s *Service) ReconstructBlobSidecars(ctx context.Context, block interfaces.
 	return verifiedBlobs, nil
 }
 
+// ReconstructDataColumnSidecars reconstructs the verified data column sidecars for a given beacon block.
+// It retrieves the KZG commitments from the block body, fetches the associated blobs and cell proofs from the EL,
+// and constructs the corresponding verified read-only data column sidecars.
+func (s *Service) ReconstructDataColumnSidecars(ctx context.Context, block interfaces.ReadOnlySignedBeaconBlock, blockRoot [32]byte) ([]blocks.VerifiedRODataColumn, error) {
+	blockBody := block.Block().Body()
+	kzgCommitments, err := blockBody.BlobKzgCommitments()
+	if err != nil {
+		return nil, wrapWithBlockRoot(err, blockRoot, "blob KZG commitments")
+	}
+
+	// Collect KZG hashes for all blobs
+	var kzgHashes []common.Hash
+	for _, commitment := range kzgCommitments {
+		kzgHashes = append(kzgHashes, primitives.ConvertKzgCommitmentToVersionedHash(commitment))
+	}
+
+	// Fetch all blobsAndCellsProofs from EL
+	blobAndProofV2s, err := s.GetBlobsV2(ctx, kzgHashes)
+	if err != nil {
+		return nil, wrapWithBlockRoot(err, blockRoot, "get blobs V2")
+	}
+
+	var cellsAndProofs []kzg.CellsAndProofs
+	for _, blobAndProof := range blobAndProofV2s {
+		if blobAndProof == nil {
+			return nil, wrapWithBlockRoot(errors.New("unable to reconstruct data column sidecars, did not get all blobs from EL"), blockRoot, "")
+		}
+
+		var blob kzg.Blob
+		copy(blob[:], blobAndProof.Blob)
+		cells, err := kzg.ComputeCells(&blob)
+		if err != nil {
+			return nil, wrapWithBlockRoot(err, blockRoot, "could not compute cells")
+		}
+
+		proofs := make([]kzg.Proof, len(blobAndProof.KzgProofs))
+		for i, proof := range blobAndProof.KzgProofs {
+			proofs[i] = kzg.Proof(proof)
+		}
+		cellsAndProofs = append(cellsAndProofs, kzg.CellsAndProofs{
+			Cells:  cells,
+			Proofs: proofs,
+		})
+	}
+
+	header, err := block.Header()
+	if err != nil {
+		return nil, wrapWithBlockRoot(err, blockRoot, "could not get header")
+	}
+
+	kzgCommitmentsInclusionProof, err := blocks.MerkleProofKZGCommitments(blockBody)
+	if err != nil {
+		return nil, wrapWithBlockRoot(err, blockRoot, "could not get Merkle proof for KZG commitments")
+	}
+
+	dataColumnSidecars, err := peerdas.DataColumnSidecarsForReconstruct(kzgCommitments, header, kzgCommitmentsInclusionProof, cellsAndProofs)
+	if err != nil {
+		return nil, wrapWithBlockRoot(err, blockRoot, "could not reconstruct data column sidecars")
+	}
+
+	verifiedRODataColumns := make([]blocks.VerifiedRODataColumn, len(dataColumnSidecars))
+	for i, dataColumnSidecar := range dataColumnSidecars {
+		roDataColumn, err := blocks.NewRODataColumnWithRoot(dataColumnSidecar, blockRoot)
+		if err != nil {
+			return nil, wrapWithBlockRoot(err, blockRoot, "new read-only data column with root")
+		}
+
+		// We trust the execution layer we are connected to, so we can upgrade the read only data column sidecar into a verified one.
+		verifiedRODataColumns[i] = blocks.NewVerifiedRODataColumn(roDataColumn)
+	}
+
+	log.WithFields(logrus.Fields{
+		"root": fmt.Sprintf("%#x", blockRoot),
+		"slot": block.Block().Slot(),
+	}).Debug("Data columns successfully reconstructed from EL")
+
+	return verifiedRODataColumns, nil
+}
+
 func fullPayloadFromPayloadBody(
 	header interfaces.ExecutionData, body *pb.ExecutionPayloadBody, bVersion int,
 ) (interfaces.ExecutionData, error) {
@@ -902,3 +1018,8 @@ func toBlockNumArg(number *big.Int) string {
 	}
 	return hexutil.EncodeBig(number)
 }
+
+// wrapWithBlockRoot returns a new error with the given block root.
+func wrapWithBlockRoot(err error, blockRoot [32]byte, message string) error {
+	return errors.Wrap(err, fmt.Sprintf("%s for block %#x", message, blockRoot))
+}

beacon-chain/execution/engine_client_test.go

@@ -13,6 +13,7 @@ import (
 	"strings"
 	"testing"
 
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/db/filesystem"
 	mocks "github.com/OffchainLabs/prysm/v6/beacon-chain/execution/testing"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/verification"
@@ -167,6 +168,7 @@ func TestClient_HTTP(t *testing.T) {
 	cfg.CapellaForkEpoch = 1
 	cfg.DenebForkEpoch = 2
 	cfg.ElectraForkEpoch = 3
+	cfg.FuluForkEpoch = 4
 	params.OverrideBeaconConfig(cfg)
 
 	t.Run(GetPayloadMethod, func(t *testing.T) {
@@ -317,11 +319,11 @@ func TestClient_HTTP(t *testing.T) {
 		require.DeepEqual(t, uint64(2), g)
 
 		commitments := [][]byte{bytesutil.PadTo([]byte("commitment1"), fieldparams.BLSPubkeyLength), bytesutil.PadTo([]byte("commitment2"), fieldparams.BLSPubkeyLength)}
-		require.DeepEqual(t, commitments, resp.BlobsBundle.KzgCommitments)
+		require.DeepEqual(t, commitments, resp.BlobsBundler.GetKzgCommitments())
 		proofs := [][]byte{bytesutil.PadTo([]byte("proof1"), fieldparams.BLSPubkeyLength), bytesutil.PadTo([]byte("proof2"), fieldparams.BLSPubkeyLength)}
-		require.DeepEqual(t, proofs, resp.BlobsBundle.Proofs)
+		require.DeepEqual(t, proofs, resp.BlobsBundler.GetProofs())
 		blobs := [][]byte{bytesutil.PadTo([]byte("a"), fieldparams.BlobLength), bytesutil.PadTo([]byte("b"), fieldparams.BlobLength)}
-		require.DeepEqual(t, blobs, resp.BlobsBundle.Blobs)
+		require.DeepEqual(t, blobs, resp.BlobsBundler.GetBlobs())
 	})
 	t.Run(GetPayloadMethodV4, func(t *testing.T) {
 		payloadId := [8]byte{1}
@@ -372,11 +374,11 @@ func TestClient_HTTP(t *testing.T) {
 		require.DeepEqual(t, uint64(2), g)
 
 		commitments := [][]byte{bytesutil.PadTo([]byte("commitment1"), fieldparams.BLSPubkeyLength), bytesutil.PadTo([]byte("commitment2"), fieldparams.BLSPubkeyLength)}
-		require.DeepEqual(t, commitments, resp.BlobsBundle.KzgCommitments)
+		require.DeepEqual(t, commitments, resp.BlobsBundler.GetKzgCommitments())
 		proofs := [][]byte{bytesutil.PadTo([]byte("proof1"), fieldparams.BLSPubkeyLength), bytesutil.PadTo([]byte("proof2"), fieldparams.BLSPubkeyLength)}
-		require.DeepEqual(t, proofs, resp.BlobsBundle.Proofs)
+		require.DeepEqual(t, proofs, resp.BlobsBundler.GetProofs())
 		blobs := [][]byte{bytesutil.PadTo([]byte("a"), fieldparams.BlobLength), bytesutil.PadTo([]byte("b"), fieldparams.BlobLength)}
-		require.DeepEqual(t, blobs, resp.BlobsBundle.Blobs)
+		require.DeepEqual(t, blobs, resp.BlobsBundler.GetBlobs())
 		requests := &pb.ExecutionRequests{
 			Deposits: []*pb.DepositRequest{
 				{
@@ -405,7 +407,52 @@ func TestClient_HTTP(t *testing.T) {
 
 		require.DeepEqual(t, requests, resp.ExecutionRequests)
 	})
+	t.Run(GetPayloadMethodV5, func(t *testing.T) {
+		payloadId := [8]byte{1}
+		want, ok := fix["ExecutionBundleFulu"].(*pb.GetPayloadV5ResponseJson)
+		require.Equal(t, true, ok)
+		srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+			w.Header().Set("Content-Type", "application/json")
+			defer func() {
+				require.NoError(t, r.Body.Close())
+			}()
+			enc, err := io.ReadAll(r.Body)
+			require.NoError(t, err)
+			jsonRequestString := string(enc)
 
+			reqArg, err := json.Marshal(pb.PayloadIDBytes(payloadId))
+			require.NoError(t, err)
+
+			// We expect the JSON string RPC request contains the right arguments.
+			require.Equal(t, true, strings.Contains(
+				jsonRequestString, string(reqArg),
+			))
+			resp := map[string]interface{}{
+				"jsonrpc": "2.0",
+				"id":      1,
+				"result":  want,
+			}
+			err = json.NewEncoder(w).Encode(resp)
+			require.NoError(t, err)
+		}))
+		defer srv.Close()
+
+		rpcClient, err := rpc.DialHTTP(srv.URL)
+		require.NoError(t, err)
+		defer rpcClient.Close()
+
+		client := &Service{}
+		client.rpcClient = rpcClient
+
+		// We call the RPC method via HTTP and expect a proper result.
+		resp, err := client.GetPayload(ctx, payloadId, 4*params.BeaconConfig().SlotsPerEpoch)
+		require.NoError(t, err)
+		_, ok = resp.BlobsBundler.(*pb.BlobsBundleV2)
+		if !ok {
+			t.Logf("resp.BlobsBundler has unexpected type: %T", resp.BlobsBundler)
+		}
+		require.Equal(t, ok, true)
+	})
 	t.Run(ForkchoiceUpdatedMethod+" VALID status", func(t *testing.T) {
 		forkChoiceState := &pb.ForkchoiceState{
 			HeadBlockHash:      []byte("head"),
@@ -1539,6 +1586,7 @@ func fixtures() map[string]interface{} {
 		"ExecutionPayloadCapellaWithValue":  s.ExecutionPayloadWithValueCapella,
 		"ExecutionPayloadDenebWithValue":    s.ExecutionPayloadWithValueDeneb,
 		"ExecutionBundleElectra":            s.ExecutionBundleElectra,
+		"ExecutionBundleFulu":               s.ExecutionBundleFulu,
 		"ValidPayloadStatus":                s.ValidPayloadStatus,
 		"InvalidBlockHashStatus":            s.InvalidBlockHashStatus,
 		"AcceptedStatus":                    s.AcceptedStatus,
@@ -1774,6 +1822,36 @@ func fixturesStruct() *payloadFixtures {
 			append([]byte{pb.WithdrawalRequestType}, withdrawalRequestBytes...),
 			append([]byte{pb.ConsolidationRequestType}, consolidationRequestBytes...)},
 	}
+	executionBundleFixtureFulu := &pb.GetPayloadV5ResponseJson{
+		ShouldOverrideBuilder: true,
+		ExecutionPayload: &pb.ExecutionPayloadDenebJSON{
+			ParentHash:    &common.Hash{'a'},
+			FeeRecipient:  &common.Address{'b'},
+			StateRoot:     &common.Hash{'c'},
+			ReceiptsRoot:  &common.Hash{'d'},
+			LogsBloom:     &hexutil.Bytes{'e'},
+			PrevRandao:    &common.Hash{'f'},
+			BaseFeePerGas: "0x123",
+			BlockHash:     &common.Hash{'g'},
+			Transactions:  []hexutil.Bytes{{'h'}},
+			Withdrawals:   []*pb.Withdrawal{},
+			BlockNumber:   &hexUint,
+			GasLimit:      &hexUint,
+			GasUsed:       &hexUint,
+			Timestamp:     &hexUint,
+			BlobGasUsed:   &bgu,
+			ExcessBlobGas: &ebg,
+		},
+		BlockValue: "0x11fffffffff",
+		BlobsBundle: &pb.BlobBundleV2JSON{
+			Commitments: []hexutil.Bytes{[]byte("commitment1"), []byte("commitment2")},
+			Proofs:      []hexutil.Bytes{[]byte("proof1"), []byte("proof2")},
+			Blobs:       []hexutil.Bytes{{'a'}, {'b'}},
+		},
+		ExecutionRequests: []hexutil.Bytes{append([]byte{pb.DepositRequestType}, depositRequestBytes...),
+			append([]byte{pb.WithdrawalRequestType}, withdrawalRequestBytes...),
+			append([]byte{pb.ConsolidationRequestType}, consolidationRequestBytes...)},
+	}
 	parent := bytesutil.PadTo([]byte("parentHash"), fieldparams.RootLength)
 	sha3Uncles := bytesutil.PadTo([]byte("sha3Uncles"), fieldparams.RootLength)
 	miner := bytesutil.PadTo([]byte("miner"), fieldparams.FeeRecipientLength)
@@ -1868,6 +1946,7 @@ func fixturesStruct() *payloadFixtures {
 		ExecutionPayloadWithValueCapella:  executionPayloadWithValueFixtureCapella,
 		ExecutionPayloadWithValueDeneb:    executionPayloadWithValueFixtureDeneb,
 		ExecutionBundleElectra:            executionBundleFixtureElectra,
+		ExecutionBundleFulu:               executionBundleFixtureFulu,
 		ValidPayloadStatus:                validStatus,
 		InvalidBlockHashStatus:            inValidBlockHashStatus,
 		AcceptedStatus:                    acceptedStatus,
@@ -1892,6 +1971,7 @@ type payloadFixtures struct {
 	ExecutionPayloadWithValueCapella  *pb.GetPayloadV2ResponseJson
 	ExecutionPayloadWithValueDeneb    *pb.GetPayloadV3ResponseJson
 	ExecutionBundleElectra            *pb.GetPayloadV4ResponseJson
+	ExecutionBundleFulu               *pb.GetPayloadV5ResponseJson
 	ValidPayloadStatus                *pb.PayloadStatus
 	InvalidBlockHashStatus            *pb.PayloadStatus
 	AcceptedStatus                    *pb.PayloadStatus
@@ -2424,7 +2504,7 @@ func TestReconstructBlobSidecars(t *testing.T) {
 	t.Run("get-blobs end point is not supported", func(t *testing.T) {
 		hi := mockSummary(t, []bool{true, true, true, true, true, false})
 		verifiedBlobs, err := client.ReconstructBlobSidecars(ctx, sb, r, hi)
-		require.NoError(t, err)
+		require.ErrorContains(t, "engine_getBlobsV1 is not supported", err)
 		require.Equal(t, 0, len(verifiedBlobs))
 	})
 
@@ -2476,6 +2556,64 @@ func TestReconstructBlobSidecars(t *testing.T) {
 	})
 }
 
+func TestReconstructDataColumnSidecars(t *testing.T) {
+	// Start the trusted setup.
+	err := kzg.Start()
+	require.NoError(t, err)
+
+	// Setup right fork epoch
+	params.SetupTestConfigCleanup(t)
+	cfg := params.BeaconConfig().Copy()
+	cfg.CapellaForkEpoch = 1
+	cfg.DenebForkEpoch = 2
+	cfg.ElectraForkEpoch = 3
+	cfg.FuluForkEpoch = 4
+	params.OverrideBeaconConfig(cfg)
+
+	client := &Service{capabilityCache: &capabilityCache{}}
+	b := util.NewBeaconBlockFulu()
+	b.Block.Slot = 4 * params.BeaconConfig().SlotsPerEpoch
+	kzgCommitments := createRandomKzgCommitments(t, 6)
+	b.Block.Body.BlobKzgCommitments = kzgCommitments
+	r, err := b.Block.HashTreeRoot()
+	require.NoError(t, err)
+	sb, err := blocks.NewSignedBeaconBlock(b)
+	require.NoError(t, err)
+
+	ctx := context.Background()
+
+	t.Run("GetBlobsV2 is not supported", func(t *testing.T) {
+		_, err := client.ReconstructDataColumnSidecars(ctx, sb, r)
+		require.ErrorContains(t, "get blobs V2 for block", err)
+	})
+
+	t.Run("receiving all blobs", func(t *testing.T) {
+		blobMasks := []bool{true, true, true, true, true, true}
+		srv := createBlobServerV2(t, 6, blobMasks)
+		defer srv.Close()
+
+		rpcClient, client := setupRpcClientV2(t, srv.URL, client)
+		defer rpcClient.Close()
+
+		dataColumns, err := client.ReconstructDataColumnSidecars(ctx, sb, r)
+		require.NoError(t, err)
+		require.Equal(t, 128, len(dataColumns))
+	})
+
+	t.Run("missing some blobs", func(t *testing.T) {
+		blobMasks := []bool{false, true, true, true, true, true}
+		srv := createBlobServerV2(t, 6, blobMasks)
+		defer srv.Close()
+
+		rpcClient, client := setupRpcClientV2(t, srv.URL, client)
+		defer rpcClient.Close()
+
+		dataColumns, err := client.ReconstructDataColumnSidecars(ctx, sb, r)
+		require.ErrorContains(t, "unable to reconstruct data column sidecars, did not get all blobs from EL", err)
+		require.Equal(t, 0, len(dataColumns))
+	})
+}
+
 func createRandomKzgCommitments(t *testing.T, num int) [][]byte {
 	kzgCommitments := make([][]byte, num)
 	for i := range kzgCommitments {
@@ -2511,6 +2649,42 @@ func createBlobServer(t *testing.T, numBlobs int, callbackFuncs ...func()) *http
 	}))
 }
 
+func createBlobServerV2(t *testing.T, numBlobs int, blobMasks []bool) *httptest.Server {
+	return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		w.Header().Set("Content-Type", "application/json")
+		defer func() {
+			require.NoError(t, r.Body.Close())
+		}()
+
+		require.Equal(t, len(blobMasks), numBlobs)
+
+		blobAndCellProofs := make([]*pb.BlobAndProofV2Json, numBlobs)
+		for i := range blobAndCellProofs {
+			if !blobMasks[i] {
+				continue
+			}
+
+			blobAndCellProofs[i] = &pb.BlobAndProofV2Json{
+				Blob:      []byte("0xblob"),
+				KzgProofs: []hexutil.Bytes{},
+			}
+			for j := 0; j < int(params.BeaconConfig().NumberOfColumns); j++ {
+				cellProof := make([]byte, 48)
+				blobAndCellProofs[i].KzgProofs = append(blobAndCellProofs[i].KzgProofs, cellProof)
+			}
+		}
+
+		respJSON := map[string]interface{}{
+			"jsonrpc": "2.0",
+			"id":      1,
+			"result":  blobAndCellProofs,
+		}
+
+		err := json.NewEncoder(w).Encode(respJSON)
+		require.NoError(t, err)
+	}))
+}
+
 func setupRpcClient(t *testing.T, url string, client *Service) (*rpc.Client, *Service) {
 	rpcClient, err := rpc.DialHTTP(url)
 	require.NoError(t, err)
@@ -2522,6 +2696,12 @@ func setupRpcClient(t *testing.T, url string, client *Service) (*rpc.Client, *Se
 	return rpcClient, client
 }
 
+func setupRpcClientV2(t *testing.T, url string, client *Service) (*rpc.Client, *Service) {
+	rpcClient, client := setupRpcClient(t, url, client)
+	client.capabilityCache = &capabilityCache{capabilities: map[string]interface{}{GetBlobsV2: nil}}
+	return rpcClient, client
+}
+
 func testNewBlobVerifier() verification.NewBlobVerifier {
 	return func(b blocks.ROBlob, reqs []verification.Requirement) verification.BlobVerifier {
 		return &verification.MockBlobVerifier{

beacon-chain/execution/testing/mock_engine_client.go

@@ -38,6 +38,8 @@ type EngineClient struct {
 	ErrGetPayload               error
 	BlobSidecars                []blocks.VerifiedROBlob
 	ErrorBlobSidecars           error
+	DataColumnSidecars          []blocks.VerifiedRODataColumn
+	ErrorDataColumnSidecars     error
 }
 
 // NewPayload --
@@ -113,6 +115,10 @@ func (e *EngineClient) ReconstructBlobSidecars(context.Context, interfaces.ReadO
 	return e.BlobSidecars, e.ErrorBlobSidecars
 }
 
+func (e *EngineClient) ReconstructDataColumnSidecars(ctx context.Context, block interfaces.ReadOnlySignedBeaconBlock, blockRoot [32]byte) ([]blocks.VerifiedRODataColumn, error) {
+	return e.DataColumnSidecars, e.ErrorDataColumnSidecars
+}
+
 // GetTerminalBlockHash --
 func (e *EngineClient) GetTerminalBlockHash(ctx context.Context, transitionTime uint64) ([]byte, bool, error) {
 	ttd := new(big.Int)

beacon-chain/node/BUILD.bazel

@@ -23,6 +23,7 @@ go_library(
         "//beacon-chain/cache:go_default_library",
         "//beacon-chain/cache/depositsnapshot:go_default_library",
         "//beacon-chain/core/light-client:go_default_library",
+        "//beacon-chain/core/peerdas:go_default_library",
         "//beacon-chain/db:go_default_library",
         "//beacon-chain/db/filesystem:go_default_library",
         "//beacon-chain/db/kv:go_default_library",

beacon-chain/node/node.go

@@ -25,6 +25,7 @@ import (
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/cache"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/cache/depositsnapshot"
 	lightclient "github.com/OffchainLabs/prysm/v6/beacon-chain/core/light-client"
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/db"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/db/filesystem"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/db/kv"
@@ -86,44 +87,47 @@ type serviceFlagOpts struct {
 // full PoS node. It handles the lifecycle of the entire system and registers
 // services to a service registry.
 type BeaconNode struct {
-	cliCtx                  *cli.Context
-	ctx                     context.Context
-	cancel                  context.CancelFunc
-	services                *runtime.ServiceRegistry
-	lock                    sync.RWMutex
-	stop                    chan struct{} // Channel to wait for termination notifications.
-	db                      db.Database
-	slasherDB               db.SlasherDatabase
-	attestationCache        *cache.AttestationCache
-	attestationPool         attestations.Pool
-	exitPool                voluntaryexits.PoolManager
-	slashingsPool           slashings.PoolManager
-	syncCommitteePool       synccommittee.Pool
-	blsToExecPool           blstoexec.PoolManager
-	depositCache            cache.DepositCache
-	trackedValidatorsCache  *cache.TrackedValidatorsCache
-	payloadIDCache          *cache.PayloadIDCache
-	stateFeed               *event.Feed
-	blockFeed               *event.Feed
-	opFeed                  *event.Feed
-	stateGen                *stategen.State
-	collector               *bcnodeCollector
-	slasherBlockHeadersFeed *event.Feed
-	slasherAttestationsFeed *event.Feed
-	finalizedStateAtStartUp state.BeaconState
-	serviceFlagOpts         *serviceFlagOpts
-	GenesisInitializer      genesis.Initializer
-	CheckpointInitializer   checkpoint.Initializer
-	forkChoicer             forkchoice.ForkChoicer
-	clockWaiter             startup.ClockWaiter
-	BackfillOpts            []backfill.ServiceOption
-	initialSyncComplete     chan struct{}
-	BlobStorage             *filesystem.BlobStorage
-	BlobStorageOptions      []filesystem.BlobStorageOption
-	verifyInitWaiter        *verification.InitializerWaiter
-	syncChecker             *initialsync.SyncChecker
-	slasherEnabled          bool
-	lcStore                 *lightclient.Store
+	cliCtx                   *cli.Context
+	ctx                      context.Context
+	cancel                   context.CancelFunc
+	services                 *runtime.ServiceRegistry
+	lock                     sync.RWMutex
+	stop                     chan struct{} // Channel to wait for termination notifications.
+	db                       db.Database
+	slasherDB                db.SlasherDatabase
+	attestationCache         *cache.AttestationCache
+	attestationPool          attestations.Pool
+	exitPool                 voluntaryexits.PoolManager
+	slashingsPool            slashings.PoolManager
+	syncCommitteePool        synccommittee.Pool
+	blsToExecPool            blstoexec.PoolManager
+	depositCache             cache.DepositCache
+	trackedValidatorsCache   *cache.TrackedValidatorsCache
+	payloadIDCache           *cache.PayloadIDCache
+	stateFeed                *event.Feed
+	blockFeed                *event.Feed
+	opFeed                   *event.Feed
+	stateGen                 *stategen.State
+	collector                *bcnodeCollector
+	slasherBlockHeadersFeed  *event.Feed
+	slasherAttestationsFeed  *event.Feed
+	finalizedStateAtStartUp  state.BeaconState
+	serviceFlagOpts          *serviceFlagOpts
+	GenesisInitializer       genesis.Initializer
+	CheckpointInitializer    checkpoint.Initializer
+	forkChoicer              forkchoice.ForkChoicer
+	clockWaiter              startup.ClockWaiter
+	BackfillOpts             []backfill.ServiceOption
+	initialSyncComplete      chan struct{}
+	BlobStorage              *filesystem.BlobStorage
+	BlobStorageOptions       []filesystem.BlobStorageOption
+	DataColumnStorage        *filesystem.DataColumnStorage
+	DataColumnStorageOptions []filesystem.DataColumnStorageOption
+	custodyInfo              *peerdas.CustodyInfo
+	verifyInitWaiter         *verification.InitializerWaiter
+	syncChecker              *initialsync.SyncChecker
+	slasherEnabled           bool
+	lcStore                  *lightclient.Store
 }
 
 // New creates a new node instance, sets up configuration options, and registers
@@ -161,6 +165,7 @@ func New(cliCtx *cli.Context, cancel context.CancelFunc, opts ...Option) (*Beaco
 		serviceFlagOpts:         &serviceFlagOpts{},
 		initialSyncComplete:     make(chan struct{}),
 		syncChecker:             &initialsync.SyncChecker{},
+		custodyInfo:             &peerdas.CustodyInfo{},
 		slasherEnabled:          cliCtx.Bool(flags.SlasherFlag.Name),
 		lcStore:                 &lightclient.Store{},
 	}
@@ -190,6 +195,15 @@ func New(cliCtx *cli.Context, cancel context.CancelFunc, opts ...Option) (*Beaco
 		beacon.BlobStorage = blobs
 	}
 
+	if beacon.DataColumnStorage == nil {
+		dataColumnStorage, err := filesystem.NewDataColumnStorage(cliCtx.Context, beacon.DataColumnStorageOptions...)
+		if err != nil {
+			return nil, errors.Wrap(err, "new data column storage")
+		}
+
+		beacon.DataColumnStorage = dataColumnStorage
+	}
+
 	bfs, err := startBaseServices(cliCtx, beacon, depositAddress)
 	if err != nil {
 		return nil, errors.Wrap(err, "could not start modules")
@@ -280,7 +294,9 @@ func startBaseServices(cliCtx *cli.Context, beacon *BeaconNode, depositAddress s
 	if err := beacon.startDB(cliCtx, depositAddress); err != nil {
 		return nil, errors.Wrap(err, "could not start DB")
 	}
+
 	beacon.BlobStorage.WarmCache()
+	beacon.DataColumnStorage.WarmCache()
 
 	log.Debugln("Starting Slashing DB")
 	if err := beacon.startSlasherDB(cliCtx); err != nil {
@@ -491,6 +507,10 @@ func (b *BeaconNode) clearDB(clearDB, forceClearDB bool, d *kv.Store, dbPath str
 			return nil, errors.Wrap(err, "could not clear blob storage")
 		}
 
+		if err := b.DataColumnStorage.Clear(); err != nil {
+			return nil, errors.Wrap(err, "could not clear data column storage")
+		}
+
 		d, err = kv.NewKVStore(b.ctx, dbPath)
 		if err != nil {
 			return nil, errors.Wrap(err, "could not create new database")
@@ -697,6 +717,7 @@ func (b *BeaconNode) registerP2P(cliCtx *cli.Context) error {
 		StateNotifier:        b,
 		DB:                   b.db,
 		ClockWaiter:          b.clockWaiter,
+		CustodyInfo:          b.custodyInfo,
 	})
 	if err != nil {
 		return err
@@ -775,9 +796,11 @@ func (b *BeaconNode) registerBlockchainService(fc forkchoice.ForkChoicer, gs *st
 		blockchain.WithClockSynchronizer(gs),
 		blockchain.WithSyncComplete(syncComplete),
 		blockchain.WithBlobStorage(b.BlobStorage),
+		blockchain.WithDataColumnStorage(b.DataColumnStorage),
 		blockchain.WithTrackedValidatorsCache(b.trackedValidatorsCache),
 		blockchain.WithPayloadIDCache(b.payloadIDCache),
 		blockchain.WithSyncChecker(b.syncChecker),
+		blockchain.WithCustodyInfo(b.custodyInfo),
 		blockchain.WithSlasherEnabled(b.slasherEnabled),
 		blockchain.WithLightClientStore(b.lcStore),
 	)
@@ -862,10 +885,12 @@ func (b *BeaconNode) registerSyncService(initialSyncComplete chan struct{}, bFil
 		regularsync.WithInitialSyncComplete(initialSyncComplete),
 		regularsync.WithStateNotifier(b),
 		regularsync.WithBlobStorage(b.BlobStorage),
+		regularsync.WithDataColumnStorage(b.DataColumnStorage),
 		regularsync.WithVerifierWaiter(b.verifyInitWaiter),
 		regularsync.WithAvailableBlocker(bFillStore),
+		regularsync.WithTrackedValidatorsCache(b.trackedValidatorsCache),
+		regularsync.WithCustodyInfo(b.custodyInfo),
 		regularsync.WithSlasherEnabled(b.slasherEnabled),
-		regularsync.WithLightClientStore(b.lcStore),
 	)
 	return b.services.RegisterService(rs)
 }
@@ -889,6 +914,8 @@ func (b *BeaconNode) registerInitialSyncService(complete chan struct{}) error {
 		ClockWaiter:         b.clockWaiter,
 		InitialSyncComplete: complete,
 		BlobStorage:         b.BlobStorage,
+		DataColumnStorage:   b.DataColumnStorage,
+		CustodyInfo:         b.custodyInfo,
 	}, opts...)
 	return b.services.RegisterService(is)
 }
@@ -983,6 +1010,7 @@ func (b *BeaconNode) registerRPCService(router *http.ServeMux) error {
 		FinalizationFetcher:       chainService,
 		BlockReceiver:             chainService,
 		BlobReceiver:              chainService,
+		DataColumnReceiver:        chainService,
 		AttestationReceiver:       chainService,
 		GenesisTimeFetcher:        chainService,
 		GenesisFetcher:            chainService,
@@ -1010,6 +1038,7 @@ func (b *BeaconNode) registerRPCService(router *http.ServeMux) error {
 		Router:                    router,
 		ClockWaiter:               b.clockWaiter,
 		BlobStorage:               b.BlobStorage,
+		DataColumnStorage:         b.DataColumnStorage,
 		TrackedValidatorsCache:    b.trackedValidatorsCache,
 		PayloadIDCache:            b.payloadIDCache,
 		LCStore:                   b.lcStore,
@@ -1032,7 +1061,6 @@ func (b *BeaconNode) registerPrometheusService(_ *cli.Context) error {
 	}
 
 	service := prometheus.NewService(
-		b.cliCtx.Context,
 		fmt.Sprintf("%s:%d", b.cliCtx.String(cmd.MonitoringHostFlag.Name), b.cliCtx.Int(flags.MonitoringPortFlag.Name)),
 		b.services,
 		additionalHandlers...,
@@ -1151,6 +1179,7 @@ func (b *BeaconNode) registerPrunerService(cliCtx *cli.Context) error {
 
 func (b *BeaconNode) RegisterBackfillService(cliCtx *cli.Context, bfs *backfill.Store) error {
 	pa := peers.NewAssigner(b.fetchP2P().Peers(), b.forkChoicer)
+	// TODO: Add backfill for data column storage
 	bf, err := backfill.NewService(cliCtx.Context, bfs, b.BlobStorage, b.clockWaiter, b.fetchP2P(), pa, b.BackfillOpts...)
 	if err != nil {
 		return errors.Wrap(err, "error initializing backfill service")

beacon-chain/node/node_test.go

@@ -54,7 +54,7 @@ func TestNodeClose_OK(t *testing.T) {
 	cmd.ValidatorMonitorIndicesFlag.Value.SetInt(1)
 	ctx, cancel := newCliContextWithCancel(&app, set)
 
-	node, err := New(ctx, cancel, WithBlobStorage(filesystem.NewEphemeralBlobStorage(t)))
+	node, err := New(ctx, cancel, WithBlobStorage(filesystem.NewEphemeralBlobStorage(t)), WithDataColumnStorage(filesystem.NewEphemeralDataColumnStorage(t)))
 	require.NoError(t, err)
 
 	node.Close()
@@ -75,7 +75,7 @@ func TestNodeStart_Ok(t *testing.T) {
 	node, err := New(ctx, cancel, WithBlockchainFlagOptions([]blockchain.Option{}),
 		WithBuilderFlagOptions([]builder.Option{}),
 		WithExecutionChainOptions([]execution.Option{}),
-		WithBlobStorage(filesystem.NewEphemeralBlobStorage(t)))
+		WithBlobStorage(filesystem.NewEphemeralBlobStorage(t)), WithDataColumnStorage(filesystem.NewEphemeralDataColumnStorage(t)))
 	require.NoError(t, err)
 	node.services = &runtime.ServiceRegistry{}
 	go func() {
@@ -99,7 +99,7 @@ func TestNodeStart_SyncChecker(t *testing.T) {
 	node, err := New(ctx, cancel, WithBlockchainFlagOptions([]blockchain.Option{}),
 		WithBuilderFlagOptions([]builder.Option{}),
 		WithExecutionChainOptions([]execution.Option{}),
-		WithBlobStorage(filesystem.NewEphemeralBlobStorage(t)))
+		WithBlobStorage(filesystem.NewEphemeralBlobStorage(t)), WithDataColumnStorage(filesystem.NewEphemeralDataColumnStorage(t)))
 	require.NoError(t, err)
 	go func() {
 		node.Start()
@@ -130,7 +130,7 @@ func TestClearDB(t *testing.T) {
 	context, cancel := newCliContextWithCancel(&app, set)
 	options := []Option{
 		WithExecutionChainOptions([]execution.Option{execution.WithHttpEndpoint(endpoint)}),
-		WithBlobStorage(filesystem.NewEphemeralBlobStorage(t)),
+		WithBlobStorage(filesystem.NewEphemeralBlobStorage(t)), WithDataColumnStorage(filesystem.NewEphemeralDataColumnStorage(t)),
 	}
 	_, err = New(context, cancel, options...)
 	require.NoError(t, err)

beacon-chain/node/options.go

@@ -50,3 +50,20 @@ func WithBlobStorageOptions(opt ...filesystem.BlobStorageOption) Option {
 		return nil
 	}
 }
+
+// WithDataColumnStorage sets the DataColumnStorage backend for the BeaconNode
+func WithDataColumnStorage(bs *filesystem.DataColumnStorage) Option {
+	return func(bn *BeaconNode) error {
+		bn.DataColumnStorage = bs
+		return nil
+	}
+}
+
+// WithDataColumnStorageOptions appends 1 or more filesystem.DataColumnStorageOption on the beacon node,
+// to be used when initializing data column storage.
+func WithDataColumnStorageOptions(opt ...filesystem.DataColumnStorageOption) Option {
+	return func(bn *BeaconNode) error {
+		bn.DataColumnStorageOptions = append(bn.DataColumnStorageOptions, opt...)
+		return nil
+	}
+}

beacon-chain/p2p/BUILD.bazel

@@ -7,6 +7,7 @@ go_library(
         "broadcaster.go",
         "config.go",
         "connection_gater.go",
+        "custody.go",
         "dial_relay_node.go",
         "discovery.go",
         "doc.go",
@@ -45,6 +46,7 @@ go_library(
         "//beacon-chain/core/altair:go_default_library",
         "//beacon-chain/core/feed/state:go_default_library",
         "//beacon-chain/core/helpers:go_default_library",
+        "//beacon-chain/core/peerdas:go_default_library",
         "//beacon-chain/core/time:go_default_library",
         "//beacon-chain/db:go_default_library",
         "//beacon-chain/p2p/encoder:go_default_library",
@@ -55,8 +57,8 @@ go_library(
         "//beacon-chain/startup:go_default_library",
         "//cmd/beacon-chain/flags:go_default_library",
         "//config/features:go_default_library",
+        "//config/fieldparams:go_default_library",
         "//config/params:go_default_library",
-        "//consensus-types/interfaces:go_default_library",
         "//consensus-types/primitives:go_default_library",
         "//consensus-types/wrapper:go_default_library",
         "//container/leaky-bucket:go_default_library",
@@ -117,6 +119,7 @@ go_test(
         "addr_factory_test.go",
         "broadcaster_test.go",
         "connection_gater_test.go",
+        "custody_test.go",
         "dial_relay_node_test.go",
         "discovery_test.go",
         "fork_test.go",
@@ -138,10 +141,11 @@ go_test(
     flaky = True,
     tags = ["requires-network"],
     deps = [
+        "//beacon-chain/blockchain/kzg:go_default_library",
         "//beacon-chain/blockchain/testing:go_default_library",
         "//beacon-chain/cache:go_default_library",
         "//beacon-chain/core/helpers:go_default_library",
-        "//beacon-chain/core/light-client:go_default_library",
+        "//beacon-chain/core/peerdas:go_default_library",
         "//beacon-chain/core/signing:go_default_library",
         "//beacon-chain/db/testing:go_default_library",
         "//beacon-chain/p2p/encoder:go_default_library",
@@ -154,7 +158,7 @@ go_test(
         "//cmd/beacon-chain/flags:go_default_library",
         "//config/fieldparams:go_default_library",
         "//config/params:go_default_library",
-        "//consensus-types/interfaces:go_default_library",
+        "//consensus-types/blocks:go_default_library",
         "//consensus-types/primitives:go_default_library",
         "//consensus-types/wrapper:go_default_library",
         "//container/leaky-bucket:go_default_library",
@@ -165,13 +169,12 @@ go_test(
         "//network/forks:go_default_library",
         "//proto/eth/v1:go_default_library",
         "//proto/prysm/v1alpha1:go_default_library",
+        "//proto/prysm/v1alpha1/metadata:go_default_library",
         "//proto/testing:go_default_library",
-        "//runtime/version:go_default_library",
         "//testing/assert:go_default_library",
         "//testing/require:go_default_library",
         "//testing/util:go_default_library",
         "//time:go_default_library",
-        "//time/slots:go_default_library",
         "@com_github_ethereum_go_ethereum//crypto:go_default_library",
         "@com_github_ethereum_go_ethereum//p2p/discover:go_default_library",
         "@com_github_ethereum_go_ethereum//p2p/enode:go_default_library",

beacon-chain/p2p/broadcaster.go

@@ -9,12 +9,11 @@ import (
 
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/altair"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
+	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
 	"github.com/OffchainLabs/prysm/v6/config/params"
-	"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
 	"github.com/OffchainLabs/prysm/v6/crypto/hash"
 	"github.com/OffchainLabs/prysm/v6/monitoring/tracing"
 	"github.com/OffchainLabs/prysm/v6/monitoring/tracing/trace"
-	"github.com/OffchainLabs/prysm/v6/network/forks"
 	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
 	"github.com/OffchainLabs/prysm/v6/time/slots"
 	"github.com/pkg/errors"
@@ -98,7 +97,12 @@ func (s *Service) BroadcastSyncCommitteeMessage(ctx context.Context, subnet uint
 	return nil
 }
 
-func (s *Service) internalBroadcastAttestation(ctx context.Context, subnet uint64, att ethpb.Att, forkDigest [4]byte) {
+func (s *Service) internalBroadcastAttestation(
+	ctx context.Context,
+	subnet uint64,
+	att ethpb.Att,
+	forkDigest [fieldparams.VersionLength]byte,
+) {
 	_, span := trace.StartSpan(ctx, "p2p.internalBroadcastAttestation")
 	defer span.End()
 	ctx = trace.NewContext(context.Background(), span) // clear parent context / deadline.
@@ -154,7 +158,7 @@ func (s *Service) internalBroadcastAttestation(ctx context.Context, subnet uint6
 	}
 }
 
-func (s *Service) broadcastSyncCommittee(ctx context.Context, subnet uint64, sMsg *ethpb.SyncCommitteeMessage, forkDigest [4]byte) {
+func (s *Service) broadcastSyncCommittee(ctx context.Context, subnet uint64, sMsg *ethpb.SyncCommitteeMessage, forkDigest [fieldparams.VersionLength]byte) {
 	_, span := trace.StartSpan(ctx, "p2p.broadcastSyncCommittee")
 	defer span.End()
 	ctx = trace.NewContext(context.Background(), span) // clear parent context / deadline.
@@ -230,7 +234,12 @@ func (s *Service) BroadcastBlob(ctx context.Context, subnet uint64, blob *ethpb.
 	return nil
 }
 
-func (s *Service) internalBroadcastBlob(ctx context.Context, subnet uint64, blobSidecar *ethpb.BlobSidecar, forkDigest [4]byte) {
+func (s *Service) internalBroadcastBlob(
+	ctx context.Context,
+	subnet uint64,
+	blobSidecar *ethpb.BlobSidecar,
+	forkDigest [fieldparams.VersionLength]byte,
+) {
 	_, span := trace.StartSpan(ctx, "p2p.internalBroadcastBlob")
 	defer span.End()
 	ctx = trace.NewContext(context.Background(), span) // clear parent context / deadline.
@@ -245,7 +254,7 @@ func (s *Service) internalBroadcastBlob(ctx context.Context, subnet uint64, blob
 	s.subnetLocker(wrappedSubIdx).RUnlock()
 
 	if !hasPeer {
-		blobSidecarCommitteeBroadcastAttempts.Inc()
+		blobSidecarBroadcastAttempts.Inc()
 		if err := func() error {
 			s.subnetLocker(wrappedSubIdx).Lock()
 			defer s.subnetLocker(wrappedSubIdx).Unlock()
@@ -254,7 +263,7 @@ func (s *Service) internalBroadcastBlob(ctx context.Context, subnet uint64, blob
 				return err
 			}
 			if ok {
-				blobSidecarCommitteeBroadcasts.Inc()
+				blobSidecarBroadcasts.Inc()
 				return nil
 			}
 			return errors.New("failed to find peers for subnet")
@@ -270,56 +279,118 @@ func (s *Service) internalBroadcastBlob(ctx context.Context, subnet uint64, blob
 	}
 }
 
-func (s *Service) BroadcastLightClientOptimisticUpdate(ctx context.Context, update interfaces.LightClientOptimisticUpdate) error {
-	ctx, span := trace.StartSpan(ctx, "p2p.BroadcastLightClientOptimisticUpdate")
+// BroadcastDataColumn broadcasts a data column to the p2p network, the message is assumed to be
+// broadcasted to the current fork and to the input column subnet.
+// TODO: Add tests
+func (s *Service) BroadcastDataColumn(
+	ctx context.Context,
+	root [fieldparams.RootLength]byte,
+	columnSubnet uint64,
+	dataColumnSidecar *ethpb.DataColumnSidecar,
+) error {
+	// Add tracing to the function.
+	ctx, span := trace.StartSpan(ctx, "p2p.BroadcastBlob")
 	defer span.End()
 
-	if update == nil || update.IsNil() {
-		return errors.New("attempted to broadcast nil light client optimistic update")
+	// Ensure the data column sidecar is not nil.
+	if dataColumnSidecar == nil {
+		return errors.Errorf("attempted to broadcast nil data column sidecar at subnet %d", columnSubnet)
 	}
 
-	forkDigest, err := forks.ForkDigestFromEpoch(slots.ToEpoch(update.AttestedHeader().Beacon().Slot), s.genesisValidatorsRoot)
+	// Retrieve the current fork digest.
+	forkDigest, err := s.currentForkDigest()
 	if err != nil {
-		err := errors.Wrap(err, "could not retrieve fork digest")
+		err := errors.Wrap(err, "current fork digest")
 		tracing.AnnotateError(span, err)
 		return err
 	}
 
-	if err := s.broadcastObject(ctx, update, lcOptimisticToTopic(forkDigest)); err != nil {
-		log.WithError(err).Debug("Failed to broadcast light client optimistic update")
-		err := errors.Wrap(err, "could not publish message")
-		tracing.AnnotateError(span, err)
-		return err
-	}
-	log.Debug("Successfully broadcast light client optimistic update")
+	// Non-blocking broadcast, with attempts to discover a column subnet peer if none available.
+	go s.internalBroadcastDataColumn(ctx, root, columnSubnet, dataColumnSidecar, forkDigest)
 
 	return nil
 }
 
-func (s *Service) BroadcastLightClientFinalityUpdate(ctx context.Context, update interfaces.LightClientFinalityUpdate) error {
-	ctx, span := trace.StartSpan(ctx, "p2p.BroadcastLightClientFinalityUpdate")
+func (s *Service) internalBroadcastDataColumn(
+	ctx context.Context,
+	root [fieldparams.RootLength]byte,
+	columnSubnet uint64,
+	dataColumnSidecar *ethpb.DataColumnSidecar,
+	forkDigest [fieldparams.VersionLength]byte,
+) {
+	// Add tracing to the function.
+	_, span := trace.StartSpan(ctx, "p2p.internalBroadcastDataColumn")
 	defer span.End()
 
-	if update == nil || update.IsNil() {
-		return errors.New("attempted to broadcast nil light client finality update")
+	// Increase the number of broadcast attempts.
+	dataColumnSidecarBroadcastAttempts.Inc()
+
+	// Clear parent context / deadline.
+	ctx = trace.NewContext(context.Background(), span)
+
+	// Define a one-slot length context timeout.
+	oneSlot := time.Duration(params.BeaconConfig().SecondsPerSlot) * time.Second
+	ctx, cancel := context.WithTimeout(ctx, oneSlot)
+	defer cancel()
+
+	// Build the topic corresponding to this column subnet and this fork digest.
+	topic := dataColumnSubnetToTopic(columnSubnet, forkDigest)
+
+	// Compute the wrapped subnet index.
+	wrappedSubIdx := columnSubnet + dataColumnSubnetVal
+
+	// Check if we have peers with this subnet.
+	hasPeer := func() bool {
+		s.subnetLocker(wrappedSubIdx).RLock()
+		defer s.subnetLocker(wrappedSubIdx).RUnlock()
+
+		return s.hasPeerWithSubnet(topic)
+	}()
+
+	// If no peers are found, attempt to find peers with this subnet.
+	if !hasPeer {
+		if err := func() error {
+			s.subnetLocker(wrappedSubIdx).Lock()
+			defer s.subnetLocker(wrappedSubIdx).Unlock()
+
+			ok, err := s.FindPeersWithSubnet(ctx, topic, columnSubnet, 1 /*threshold*/)
+			if err != nil {
+				return errors.Wrap(err, "find peers for subnet")
+			}
+
+			if ok {
+				return nil
+			}
+			return errors.New("failed to find peers for subnet")
+		}(); err != nil {
+			log.WithError(err).Error("Failed to find peers")
+			tracing.AnnotateError(span, err)
+		}
 	}
 
-	forkDigest, err := forks.ForkDigestFromEpoch(slots.ToEpoch(update.AttestedHeader().Beacon().Slot), s.genesisValidatorsRoot)
+	// Broadcast the data column sidecar to the network.
+	if err := s.broadcastObject(ctx, dataColumnSidecar, topic); err != nil {
+		log.WithError(err).Error("Failed to broadcast data column sidecar")
+		tracing.AnnotateError(span, err)
+	}
+
+	header := dataColumnSidecar.SignedBlockHeader.GetHeader()
+	slot := header.GetSlot()
+
+	slotStartTime, err := slots.ToTime(uint64(s.genesisTime.Unix()), slot)
 	if err != nil {
-		err := errors.Wrap(err, "could not retrieve fork digest")
-		tracing.AnnotateError(span, err)
-		return err
+		log.WithError(err).Error("Failed to convert slot to time")
 	}
 
-	if err := s.broadcastObject(ctx, update, lcFinalityToTopic(forkDigest)); err != nil {
-		log.WithError(err).Debug("Failed to broadcast light client finality update")
-		err := errors.Wrap(err, "could not publish message")
-		tracing.AnnotateError(span, err)
-		return err
-	}
-	log.Debug("Successfully broadcast light client finality update")
+	log.WithFields(logrus.Fields{
+		"slot":               slot,
+		"timeSinceSlotStart": time.Since(slotStartTime),
+		"root":               fmt.Sprintf("%#x", root),
+		"columnSubnet":       columnSubnet,
+	}).Debug("Broadcasted data column sidecar")
 
-	return nil
+	// Increase the number of successful broadcasts.
+	dataColumnSidecarBroadcasts.Inc()
 }
 
 // method to broadcast messages to other peers in our gossip mesh.
@@ -351,22 +422,18 @@ func (s *Service) broadcastObject(ctx context.Context, obj ssz.Marshaler, topic
 	return nil
 }
 
-func attestationToTopic(subnet uint64, forkDigest [4]byte) string {
+func attestationToTopic(subnet uint64, forkDigest [fieldparams.VersionLength]byte) string {
 	return fmt.Sprintf(AttestationSubnetTopicFormat, forkDigest, subnet)
 }
 
-func syncCommitteeToTopic(subnet uint64, forkDigest [4]byte) string {
+func syncCommitteeToTopic(subnet uint64, forkDigest [fieldparams.VersionLength]byte) string {
 	return fmt.Sprintf(SyncCommitteeSubnetTopicFormat, forkDigest, subnet)
 }
 
-func blobSubnetToTopic(subnet uint64, forkDigest [4]byte) string {
+func blobSubnetToTopic(subnet uint64, forkDigest [fieldparams.VersionLength]byte) string {
 	return fmt.Sprintf(BlobSubnetTopicFormat, forkDigest, subnet)
 }
 
-func lcOptimisticToTopic(forkDigest [4]byte) string {
-	return fmt.Sprintf(LightClientOptimisticUpdateTopicFormat, forkDigest)
-}
-
-func lcFinalityToTopic(forkDigest [4]byte) string {
-	return fmt.Sprintf(LightClientFinalityUpdateTopicFormat, forkDigest)
+func dataColumnSubnetToTopic(subnet uint64, forkDigest [fieldparams.VersionLength]byte) string {
+	return fmt.Sprintf(DataColumnSubnetTopicFormat, forkDigest, subnet)
 }

beacon-chain/p2p/broadcaster_test.go

@@ -9,23 +9,21 @@ import (
 	"testing"
 	"time"
 
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
-	lightClient "github.com/OffchainLabs/prysm/v6/beacon-chain/core/light-client"
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/peers"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/peers/scorers"
 	p2ptest "github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/testing"
 	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
-	"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/wrapper"
 	"github.com/OffchainLabs/prysm/v6/encoding/bytesutil"
-	"github.com/OffchainLabs/prysm/v6/network/forks"
 	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
 	testpb "github.com/OffchainLabs/prysm/v6/proto/testing"
-	"github.com/OffchainLabs/prysm/v6/runtime/version"
 	"github.com/OffchainLabs/prysm/v6/testing/assert"
 	"github.com/OffchainLabs/prysm/v6/testing/require"
 	"github.com/OffchainLabs/prysm/v6/testing/util"
-	"github.com/OffchainLabs/prysm/v6/time/slots"
 	pubsub "github.com/libp2p/go-libp2p-pubsub"
 	"github.com/libp2p/go-libp2p/core/host"
 	"github.com/prysmaticlabs/go-bitfield"
@@ -522,11 +520,12 @@ func TestService_BroadcastBlob(t *testing.T) {
 	require.Equal(t, false, util.WaitTimeout(&wg, 1*time.Second), "Failed to receive pubsub within 1s")
 }
 
-func TestService_BroadcastLightClientOptimisticUpdate(t *testing.T) {
+func TestService_BroadcastDataColumn(t *testing.T) {
+	require.NoError(t, kzg.Start())
 	p1 := p2ptest.NewTestP2P(t)
 	p2 := p2ptest.NewTestP2P(t)
 	p1.Connect(p2)
-	require.NotEqual(t, 0, len(p1.BHost.Network().Peers()))
+	require.NotEqual(t, 0, len(p1.BHost.Network().Peers()), "No peers")
 
 	p := &Service{
 		host:                  p1.BHost,
@@ -542,14 +541,36 @@ func TestService_BroadcastLightClientOptimisticUpdate(t *testing.T) {
 		}),
 	}
 
-	l := util.NewTestLightClient(t, version.Altair)
-	msg, err := lightClient.NewLightClientOptimisticUpdateFromBeaconState(l.Ctx, l.State.Slot(), l.State, l.Block, l.AttestedState, l.AttestedBlock)
+	var (
+		comts [][]byte
+		blobs []kzg.Blob
+	)
+	for i := int64(0); i < 6; i++ {
+		blob := kzg.Blob(util.GetRandBlob(i))
+		commitment, err := kzg.BlobToKZGCommitment(&blob)
+		require.NoError(t, err)
+		comts = append(comts, commitment[:])
+		blobs = append(blobs, blob)
+	}
+
+	b := util.NewBeaconBlockFulu()
+	b.Block.Body.BlobKzgCommitments = comts
+	sb, err := blocks.NewSignedBeaconBlock(b)
+	require.NoError(t, err)
+	for i := range blobs {
+		blobs[i] = kzg.Blob(util.GetRandBlob(int64(i)))
+	}
+	cellsAndProofs := util.GenerateCellsAndProofs(t, blobs)
+	sidecars, err := peerdas.DataColumnSidecars(sb, cellsAndProofs)
 	require.NoError(t, err)
 
-	GossipTypeMapping[reflect.TypeOf(msg)] = LightClientOptimisticUpdateTopicFormat
-	digest, err := forks.ForkDigestFromEpoch(slots.ToEpoch(msg.AttestedHeader().Beacon().Slot), p.genesisValidatorsRoot)
+	sidecar := sidecars[0]
+	subnet := uint64(0)
+	topic := DataColumnSubnetTopicFormat
+	GossipTypeMapping[reflect.TypeOf(sidecar)] = topic
+	digest, err := p.currentForkDigest()
 	require.NoError(t, err)
-	topic := fmt.Sprintf(LightClientOptimisticUpdateTopicFormat, digest)
+	topic = fmt.Sprintf(topic, digest, subnet)
 
 	// External peer subscribes to the topic.
 	topic += p.Encoding().ProtocolSuffix()
@@ -566,92 +587,20 @@ func TestService_BroadcastLightClientOptimisticUpdate(t *testing.T) {
 		ctx, cancel := context.WithTimeout(context.Background(), 1*time.Second)
 		defer cancel()
 
-		incomingMessage, err := sub.Next(ctx)
+		msg, err := sub.Next(ctx)
 		require.NoError(t, err)
 
-		result := &ethpb.LightClientOptimisticUpdateAltair{}
-		require.NoError(t, p.Encoding().DecodeGossip(incomingMessage.Data, result))
-		if !proto.Equal(result, msg.Proto()) {
-			tt.Errorf("Did not receive expected message, got %+v, wanted %+v", result, msg)
-		}
+		result := &ethpb.DataColumnSidecar{}
+		require.NoError(t, p.Encoding().DecodeGossip(msg.Data, result))
+		require.DeepEqual(t, result, sidecar)
 	}(t)
 
-	// Broadcasting nil should fail.
+	// Attempt to broadcast nil object should fail.
 	ctx := context.Background()
-	require.ErrorContains(t, "attempted to broadcast nil", p.BroadcastLightClientOptimisticUpdate(ctx, nil))
-	var nilUpdate interfaces.LightClientOptimisticUpdate
-	require.ErrorContains(t, "attempted to broadcast nil", p.BroadcastLightClientOptimisticUpdate(ctx, nilUpdate))
+	var root [fieldparams.RootLength]byte
+	require.ErrorContains(t, "attempted to broadcast nil", p.BroadcastDataColumn(ctx, root, subnet, nil))
 
 	// Broadcast to peers and wait.
-	require.NoError(t, p.BroadcastLightClientOptimisticUpdate(ctx, msg))
-	if util.WaitTimeout(&wg, 1*time.Second) {
-		t.Error("Failed to receive pubsub within 1s")
-	}
-}
-
-func TestService_BroadcastLightClientFinalityUpdate(t *testing.T) {
-	p1 := p2ptest.NewTestP2P(t)
-	p2 := p2ptest.NewTestP2P(t)
-	p1.Connect(p2)
-	require.NotEqual(t, 0, len(p1.BHost.Network().Peers()))
-
-	p := &Service{
-		host:                  p1.BHost,
-		pubsub:                p1.PubSub(),
-		joinedTopics:          map[string]*pubsub.Topic{},
-		cfg:                   &Config{},
-		genesisTime:           time.Now(),
-		genesisValidatorsRoot: bytesutil.PadTo([]byte{'A'}, 32),
-		subnetsLock:           make(map[uint64]*sync.RWMutex),
-		subnetsLockLock:       sync.Mutex{},
-		peers: peers.NewStatus(context.Background(), &peers.StatusConfig{
-			ScorerParams: &scorers.Config{},
-		}),
-	}
-
-	l := util.NewTestLightClient(t, version.Altair)
-	msg, err := lightClient.NewLightClientFinalityUpdateFromBeaconState(l.Ctx, l.State.Slot(), l.State, l.Block, l.AttestedState, l.AttestedBlock, l.FinalizedBlock)
-	require.NoError(t, err)
-
-	GossipTypeMapping[reflect.TypeOf(msg)] = LightClientFinalityUpdateTopicFormat
-	digest, err := forks.ForkDigestFromEpoch(slots.ToEpoch(msg.AttestedHeader().Beacon().Slot), p.genesisValidatorsRoot)
-	require.NoError(t, err)
-	topic := fmt.Sprintf(LightClientFinalityUpdateTopicFormat, digest)
-
-	// External peer subscribes to the topic.
-	topic += p.Encoding().ProtocolSuffix()
-	sub, err := p2.SubscribeToTopic(topic)
-	require.NoError(t, err)
-
-	time.Sleep(50 * time.Millisecond) // libp2p fails without this delay...
-
-	// Async listen for the pubsub, must be before the broadcast.
-	var wg sync.WaitGroup
-	wg.Add(1)
-	go func(tt *testing.T) {
-		defer wg.Done()
-		ctx, cancel := context.WithTimeout(context.Background(), 1*time.Second)
-		defer cancel()
-
-		incomingMessage, err := sub.Next(ctx)
-		require.NoError(t, err)
-
-		result := &ethpb.LightClientFinalityUpdateAltair{}
-		require.NoError(t, p.Encoding().DecodeGossip(incomingMessage.Data, result))
-		if !proto.Equal(result, msg.Proto()) {
-			tt.Errorf("Did not receive expected message, got %+v, wanted %+v", result, msg)
-		}
-	}(t)
-
-	// Broadcasting nil should fail.
-	ctx := context.Background()
-	require.ErrorContains(t, "attempted to broadcast nil", p.BroadcastLightClientFinalityUpdate(ctx, nil))
-	var nilUpdate interfaces.LightClientFinalityUpdate
-	require.ErrorContains(t, "attempted to broadcast nil", p.BroadcastLightClientFinalityUpdate(ctx, nilUpdate))
-
-	// Broadcast to peers and wait.
-	require.NoError(t, p.BroadcastLightClientFinalityUpdate(ctx, msg))
-	if util.WaitTimeout(&wg, 1*time.Second) {
-		t.Error("Failed to receive pubsub within 1s")
-	}
+	require.NoError(t, p.BroadcastDataColumn(ctx, root, subnet, sidecar))
+	require.Equal(t, false, util.WaitTimeout(&wg, 1*time.Second), "Failed to receive pubsub within 1s")
 }