fix conflicts

* Add additional spans for tracing GetDuties

* Changelog fragment

CHANGELOG.md

@@ -4,6 +4,64 @@ All notable changes to this project will be documented in this file.
 
 The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
 
+## [v6.0.0](https://github.com/prysmaticlabs/prysm/compare/v5.3.2...v6.0.0) - 2025-04-21
+
+This release introduces Mainnet support for the upcoming Electra + Prague (Pectra) fork. The fork is scheduled for mainnet epoch 364032 (May 7, 2025, 10:05:11 UTC). You MUST update Prysm Beacon Node, Prysm Validator Client, and your execution layer client to the Pectra ready release prior to the fork to stay on the correct chain.
+
+Besides Pectra, we have more light client API support, cleanups, and a few bugfixes. Please review the changelog below and update your client as soon as practical before May 7.
+
+This release is **mandatory** for all operators before May 7.
+
+### Added
+
+- Implemented validator identities Beacon API endpoint. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15086)
+- Add SSZ support to light client updates by range API. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15082)
+- Add light client ssz types to the spec test. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15097)
+- Added the ability for execution requests to be tested in e2e with electra. [[PR]](https://github.com/prysmaticlabs/prysm/pull/14971)
+- Add warning messages for gas limit ranges that might be problematic. Low gas limits (≤10% of default) may cause transactions to fail, while high gas limits (>150% of default) could lead to block propagation issues. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15078)
+- Add light client store object to the beacon node object. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15120)
+- prysmctl option in wrapper script to generate devnet ssz. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15145)
+- Add support for Electra fork epoch. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15132)
+
+### Changed
+
+- The validator client will no longer use the full list of committee values but instead use the committee length and validator committee index. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15039)
+- Remove the header `Content-Disposition` from the `httputil.WriteSSZ` function. No `filename` parameter is needed anymore. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15092)
+- Sort attestations in proposer block by reward. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15093)
+- More efficient query method for stategen to retrieve blocks between a given state and the replay target block. This avoids attempting to look up blocks that are not needed for head replay queries, which may be missing due to a previous rollback bug. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15063)
+- removed old web3signer metrics in favor for a universal one. [[PR]](https://github.com/prysmaticlabs/prysm/pull/14920)
+- Deprecated everything related with the gRPC API. [[PR]](https://github.com/prysmaticlabs/prysm/pull/14944)
+- Migrate Prysm repo to Offchain Labs organization ahead of Pectra upgrade v6. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15140)
+
+### Deprecated
+
+- deprecates and removes usage of the `--trace` flag and`--cpuprofile` flag in favor of just using `--pprof`. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15083)
+
+### Removed
+
+- Remove /eth/v1/beacon/states/head/committees call when getting duties. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15039)
+- Removed unused hack scripts. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15157)
+- Remove `disable-committee-aware-packing` flag. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15162)
+- Remove deprecated flags for the major release. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15165)
+- Removed Beacon API endpoints which have been deprecated at the Deneb fork. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15166)
+
+### Fixed
+
+- The `--rpc` flag will now properly enable the keymanager APIs without web. The `--web` will enable both validator api endpoints and web. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15080)
+- Use latest state to pack attestation. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15113)
+- Clean up dangling block index entries for blocks that were previously deleted by incomplete cleanup code. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15040)
+- Fixed to use io stream instead of stream read. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15089)
+- When using a DV, send all aggregations for a slot and committee. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15110)
+- Fixed a bug in consolidation request processing. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15122)
+- Fix State Getter for pending withdrawal balance. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15123)
+- Fixed a bug in checking for attestation lengths in our block. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15134)
+- Fix Committee Index Check For Aggregates. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15146)
+- Fix filtering by committee index post-Electra in `ListAttestationsV2`. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15148)
+- Peers giving invalid data in range syncing are now downscored. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15149)
+- Adding fork guard to attestation api endpoints so that it doesn't accidentally include wrong attestation types in the pool. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15161)
+- fixed underflow with balances in leaking edge case with expected withdrawals. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15191)
+- Attribute block and blob issues to correct peers during range syncing. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15173)
+
 ## [v5.3.2](https://github.com/prysmaticlabs/prysm/compare/v5.3.1...v5.3.2) - 2025-03-25
 
 This release introduces support for the `Hoodi` testnet.
@@ -3255,4 +3313,4 @@ There are no security updates in this release.
 
 # Older than v2.0.0
 
-For changelog history for releases older than v2.0.0, please refer to https://github.com/prysmaticlabs/prysm/releases
+For changelog history for releases older than v2.0.0, please refer to https://github.com/prysmaticlabs/prysm/releases

WORKSPACE

@@ -1,7 +1,7 @@
 workspace(name = "prysm")
 
-load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")
 load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
+load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")
 
 http_archive(
     name = "rules_pkg",
@@ -16,6 +16,8 @@ load("@rules_pkg//:deps.bzl", "rules_pkg_dependencies")
 
 rules_pkg_dependencies()
 
+load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
+
 http_archive(
     name = "toolchains_protoc",
     sha256 = "abb1540f8a9e045422730670ebb2f25b41fa56ca5a7cf795175a110a0a68f4ad",
@@ -253,7 +255,7 @@ filegroup(
     url = "https://github.com/ethereum/EIPs/archive/5480440fe51742ed23342b68cf106cefd427e39d.tar.gz",
 )
 
-consensus_spec_version = "v1.5.0-beta.5"
+consensus_spec_version = "v1.5.0"
 
 bls_test_version = "v0.1.1"
 
@@ -269,7 +271,7 @@ filegroup(
     visibility = ["//visibility:public"],
 )
     """,
-    integrity = "sha256-H+Pt4z+HCVDnEBAv814yvsjR7f5l1IpumjFoTj2XnLE=",
+    integrity = "sha256-JljxS/if/t0qvGWcf5CgsX+72fj90yGTg/uEgC56y7U=",
     url = "https://github.com/ethereum/consensus-spec-tests/releases/download/%s/general.tar.gz" % consensus_spec_version,
 )
 
@@ -285,7 +287,7 @@ filegroup(
     visibility = ["//visibility:public"],
 )
     """,
-    integrity = "sha256-Dqiwf5BG7yYyURGf+i87AIdArAyztvcgjoi2kSxrGvo=",
+    integrity = "sha256-NRba2h4zqb2LAXyDPglHTtkT4gVyuwpY708XmwXKXV8=",
     url = "https://github.com/ethereum/consensus-spec-tests/releases/download/%s/minimal.tar.gz" % consensus_spec_version,
 )
 
@@ -301,7 +303,7 @@ filegroup(
     visibility = ["//visibility:public"],
 )
     """,
-    integrity = "sha256-xrmsFF243pzXHAjh1EQYKS9gtcwmtHK3wRZDSLlVVRk=",
+    integrity = "sha256-hpbtKUbc3NHtVcUPk/Zm+Hn57G2ijI9qvXJwl9hc/tM=",
     url = "https://github.com/ethereum/consensus-spec-tests/releases/download/%s/mainnet.tar.gz" % consensus_spec_version,
 )
 
@@ -316,7 +318,7 @@ filegroup(
     visibility = ["//visibility:public"],
 )
     """,
-    integrity = "sha256-c+gGapqifCvFtmtxfhOwieBDO2Syxp13GECWEpWM/Ho=",
+    integrity = "sha256-Wy3YcJxoXiKQwrGgJecrtjtdokc4X/VUNBmyQXJf0Oc=",
     strip_prefix = "consensus-specs-" + consensus_spec_version[1:],
     url = "https://github.com/ethereum/consensus-specs/archive/refs/tags/%s.tar.gz" % consensus_spec_version,
 )

api/server/structs/BUILD.bazel

@@ -57,6 +57,7 @@ go_test(
     deps = [
         "//proto/engine/v1:go_default_library",
         "//proto/prysm/v1alpha1:go_default_library",
+        "//testing/assert:go_default_library",
         "//testing/require:go_default_library",
         "@com_github_ethereum_go_ethereum//common:go_default_library",
         "@com_github_ethereum_go_ethereum//common/hexutil:go_default_library",

api/server/structs/conversions_state.go

@@ -26,10 +26,7 @@ func BeaconStateFromConsensus(st beaconState.BeaconState) (*BeaconState, error)
 	for i, r := range srcSr {
 		sr[i] = hexutil.Encode(r)
 	}
-	srcHr, err := st.HistoricalRoots()
-	if err != nil {
-		return nil, err
-	}
+	srcHr := st.HistoricalRoots()
 	hr := make([]string, len(srcHr))
 	for i, r := range srcHr {
 		hr[i] = hexutil.Encode(r)
@@ -116,10 +113,7 @@ func BeaconStateAltairFromConsensus(st beaconState.BeaconState) (*BeaconStateAlt
 	for i, r := range srcSr {
 		sr[i] = hexutil.Encode(r)
 	}
-	srcHr, err := st.HistoricalRoots()
-	if err != nil {
-		return nil, err
-	}
+	srcHr := st.HistoricalRoots()
 	hr := make([]string, len(srcHr))
 	for i, r := range srcHr {
 		hr[i] = hexutil.Encode(r)
@@ -225,10 +219,7 @@ func BeaconStateBellatrixFromConsensus(st beaconState.BeaconState) (*BeaconState
 	for i, r := range srcSr {
 		sr[i] = hexutil.Encode(r)
 	}
-	srcHr, err := st.HistoricalRoots()
-	if err != nil {
-		return nil, err
-	}
+	srcHr := st.HistoricalRoots()
 	hr := make([]string, len(srcHr))
 	for i, r := range srcHr {
 		hr[i] = hexutil.Encode(r)
@@ -347,10 +338,7 @@ func BeaconStateCapellaFromConsensus(st beaconState.BeaconState) (*BeaconStateCa
 	for i, r := range srcSr {
 		sr[i] = hexutil.Encode(r)
 	}
-	srcHr, err := st.HistoricalRoots()
-	if err != nil {
-		return nil, err
-	}
+	srcHr := st.HistoricalRoots()
 	hr := make([]string, len(srcHr))
 	for i, r := range srcHr {
 		hr[i] = hexutil.Encode(r)
@@ -488,10 +476,7 @@ func BeaconStateDenebFromConsensus(st beaconState.BeaconState) (*BeaconStateDene
 	for i, r := range srcSr {
 		sr[i] = hexutil.Encode(r)
 	}
-	srcHr, err := st.HistoricalRoots()
-	if err != nil {
-		return nil, err
-	}
+	srcHr := st.HistoricalRoots()
 	hr := make([]string, len(srcHr))
 	for i, r := range srcHr {
 		hr[i] = hexutil.Encode(r)
@@ -629,10 +614,7 @@ func BeaconStateElectraFromConsensus(st beaconState.BeaconState) (*BeaconStateEl
 	for i, r := range srcSr {
 		sr[i] = hexutil.Encode(r)
 	}
-	srcHr, err := st.HistoricalRoots()
-	if err != nil {
-		return nil, err
-	}
+	srcHr := st.HistoricalRoots()
 	hr := make([]string, len(srcHr))
 	for i, r := range srcHr {
 		hr[i] = hexutil.Encode(r)
@@ -815,10 +797,7 @@ func BeaconStateFuluFromConsensus(st beaconState.BeaconState) (*BeaconStateFulu,
 	for i, r := range srcSr {
 		sr[i] = hexutil.Encode(r)
 	}
-	srcHr, err := st.HistoricalRoots()
-	if err != nil {
-		return nil, err
-	}
+	srcHr := st.HistoricalRoots()
 	hr := make([]string, len(srcHr))
 	for i, r := range srcHr {
 		hr[i] = hexutil.Encode(r)

api/server/structs/conversions_test.go

@@ -4,7 +4,9 @@ import (
 	"testing"
 
 	eth "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
+	"github.com/OffchainLabs/prysm/v6/testing/assert"
 	"github.com/OffchainLabs/prysm/v6/testing/require"
+	"github.com/ethereum/go-ethereum/common/hexutil"
 )
 
 func TestDepositSnapshotFromConsensus(t *testing.T) {
@@ -102,12 +104,266 @@ func TestProposerSlashing_ToConsensus(t *testing.T) {
 	require.ErrorContains(t, errNilValue.Error(), err)
 }
 
+func TestProposerSlashing_FromConsensus(t *testing.T) {
+	input := []*eth.ProposerSlashing{
+		{
+			Header_1: &eth.SignedBeaconBlockHeader{
+				Header: &eth.BeaconBlockHeader{
+					Slot:          1,
+					ProposerIndex: 2,
+					ParentRoot:    []byte{3},
+					StateRoot:     []byte{4},
+					BodyRoot:      []byte{5},
+				},
+				Signature: []byte{6},
+			},
+			Header_2: &eth.SignedBeaconBlockHeader{
+				Header: &eth.BeaconBlockHeader{
+					Slot:          7,
+					ProposerIndex: 8,
+					ParentRoot:    []byte{9},
+					StateRoot:     []byte{10},
+					BodyRoot:      []byte{11},
+				},
+				Signature: []byte{12},
+			},
+		},
+		{
+			Header_1: &eth.SignedBeaconBlockHeader{
+				Header: &eth.BeaconBlockHeader{
+					Slot:          13,
+					ProposerIndex: 14,
+					ParentRoot:    []byte{15},
+					StateRoot:     []byte{16},
+					BodyRoot:      []byte{17},
+				},
+				Signature: []byte{18},
+			},
+			Header_2: &eth.SignedBeaconBlockHeader{
+				Header: &eth.BeaconBlockHeader{
+					Slot:          19,
+					ProposerIndex: 20,
+					ParentRoot:    []byte{21},
+					StateRoot:     []byte{22},
+					BodyRoot:      []byte{23},
+				},
+				Signature: []byte{24},
+			},
+		},
+	}
+
+	expectedResult := []*ProposerSlashing{
+		{
+			SignedHeader1: &SignedBeaconBlockHeader{
+				Message: &BeaconBlockHeader{
+					Slot:          "1",
+					ProposerIndex: "2",
+					ParentRoot:    hexutil.Encode([]byte{3}),
+					StateRoot:     hexutil.Encode([]byte{4}),
+					BodyRoot:      hexutil.Encode([]byte{5}),
+				},
+				Signature: hexutil.Encode([]byte{6}),
+			},
+			SignedHeader2: &SignedBeaconBlockHeader{
+				Message: &BeaconBlockHeader{
+					Slot:          "7",
+					ProposerIndex: "8",
+					ParentRoot:    hexutil.Encode([]byte{9}),
+					StateRoot:     hexutil.Encode([]byte{10}),
+					BodyRoot:      hexutil.Encode([]byte{11}),
+				},
+				Signature: hexutil.Encode([]byte{12}),
+			},
+		},
+		{
+			SignedHeader1: &SignedBeaconBlockHeader{
+				Message: &BeaconBlockHeader{
+					Slot:          "13",
+					ProposerIndex: "14",
+					ParentRoot:    hexutil.Encode([]byte{15}),
+					StateRoot:     hexutil.Encode([]byte{16}),
+					BodyRoot:      hexutil.Encode([]byte{17}),
+				},
+				Signature: hexutil.Encode([]byte{18}),
+			},
+			SignedHeader2: &SignedBeaconBlockHeader{
+				Message: &BeaconBlockHeader{
+					Slot:          "19",
+					ProposerIndex: "20",
+					ParentRoot:    hexutil.Encode([]byte{21}),
+					StateRoot:     hexutil.Encode([]byte{22}),
+					BodyRoot:      hexutil.Encode([]byte{23}),
+				},
+				Signature: hexutil.Encode([]byte{24}),
+			},
+		},
+	}
+
+	result := ProposerSlashingsFromConsensus(input)
+	assert.DeepEqual(t, expectedResult, result)
+}
+
 func TestAttesterSlashing_ToConsensus(t *testing.T) {
 	a := &AttesterSlashing{Attestation1: nil, Attestation2: nil}
 	_, err := a.ToConsensus()
 	require.ErrorContains(t, errNilValue.Error(), err)
 }
 
+func TestAttesterSlashing_FromConsensus(t *testing.T) {
+	input := []*eth.AttesterSlashing{
+		{
+			Attestation_1: &eth.IndexedAttestation{
+				AttestingIndices: []uint64{1, 2},
+				Data: &eth.AttestationData{
+					Slot:            3,
+					CommitteeIndex:  4,
+					BeaconBlockRoot: []byte{5},
+					Source: &eth.Checkpoint{
+						Epoch: 6,
+						Root:  []byte{7},
+					},
+					Target: &eth.Checkpoint{
+						Epoch: 8,
+						Root:  []byte{9},
+					},
+				},
+				Signature: []byte{10},
+			},
+			Attestation_2: &eth.IndexedAttestation{
+				AttestingIndices: []uint64{11, 12},
+				Data: &eth.AttestationData{
+					Slot:            13,
+					CommitteeIndex:  14,
+					BeaconBlockRoot: []byte{15},
+					Source: &eth.Checkpoint{
+						Epoch: 16,
+						Root:  []byte{17},
+					},
+					Target: &eth.Checkpoint{
+						Epoch: 18,
+						Root:  []byte{19},
+					},
+				},
+				Signature: []byte{20},
+			},
+		},
+		{
+			Attestation_1: &eth.IndexedAttestation{
+				AttestingIndices: []uint64{21, 22},
+				Data: &eth.AttestationData{
+					Slot:            23,
+					CommitteeIndex:  24,
+					BeaconBlockRoot: []byte{25},
+					Source: &eth.Checkpoint{
+						Epoch: 26,
+						Root:  []byte{27},
+					},
+					Target: &eth.Checkpoint{
+						Epoch: 28,
+						Root:  []byte{29},
+					},
+				},
+				Signature: []byte{30},
+			},
+			Attestation_2: &eth.IndexedAttestation{
+				AttestingIndices: []uint64{31, 32},
+				Data: &eth.AttestationData{
+					Slot:            33,
+					CommitteeIndex:  34,
+					BeaconBlockRoot: []byte{35},
+					Source: &eth.Checkpoint{
+						Epoch: 36,
+						Root:  []byte{37},
+					},
+					Target: &eth.Checkpoint{
+						Epoch: 38,
+						Root:  []byte{39},
+					},
+				},
+				Signature: []byte{40},
+			},
+		},
+	}
+
+	expectedResult := []*AttesterSlashing{
+		{
+			Attestation1: &IndexedAttestation{
+				AttestingIndices: []string{"1", "2"},
+				Data: &AttestationData{
+					Slot:            "3",
+					CommitteeIndex:  "4",
+					BeaconBlockRoot: hexutil.Encode([]byte{5}),
+					Source: &Checkpoint{
+						Epoch: "6",
+						Root:  hexutil.Encode([]byte{7}),
+					},
+					Target: &Checkpoint{
+						Epoch: "8",
+						Root:  hexutil.Encode([]byte{9}),
+					},
+				},
+				Signature: hexutil.Encode([]byte{10}),
+			},
+			Attestation2: &IndexedAttestation{
+				AttestingIndices: []string{"11", "12"},
+				Data: &AttestationData{
+					Slot:            "13",
+					CommitteeIndex:  "14",
+					BeaconBlockRoot: hexutil.Encode([]byte{15}),
+					Source: &Checkpoint{
+						Epoch: "16",
+						Root:  hexutil.Encode([]byte{17}),
+					},
+					Target: &Checkpoint{
+						Epoch: "18",
+						Root:  hexutil.Encode([]byte{19}),
+					},
+				},
+				Signature: hexutil.Encode([]byte{20}),
+			},
+		},
+		{
+			Attestation1: &IndexedAttestation{
+				AttestingIndices: []string{"21", "22"},
+				Data: &AttestationData{
+					Slot:            "23",
+					CommitteeIndex:  "24",
+					BeaconBlockRoot: hexutil.Encode([]byte{25}),
+					Source: &Checkpoint{
+						Epoch: "26",
+						Root:  hexutil.Encode([]byte{27}),
+					},
+					Target: &Checkpoint{
+						Epoch: "28",
+						Root:  hexutil.Encode([]byte{29}),
+					},
+				},
+				Signature: hexutil.Encode([]byte{30}),
+			},
+			Attestation2: &IndexedAttestation{
+				AttestingIndices: []string{"31", "32"},
+				Data: &AttestationData{
+					Slot:            "33",
+					CommitteeIndex:  "34",
+					BeaconBlockRoot: hexutil.Encode([]byte{35}),
+					Source: &Checkpoint{
+						Epoch: "36",
+						Root:  hexutil.Encode([]byte{37}),
+					},
+					Target: &Checkpoint{
+						Epoch: "38",
+						Root:  hexutil.Encode([]byte{39}),
+					},
+				},
+				Signature: hexutil.Encode([]byte{40}),
+			},
+		},
+	}
+
+	result := AttesterSlashingsFromConsensus(input)
+	assert.DeepEqual(t, expectedResult, result)
+}
+
 func TestIndexedAttestation_ToConsensus(t *testing.T) {
 	a := &IndexedAttestation{
 		AttestingIndices: []string{"1"},

api/server/structs/endpoints_beacon.go

@@ -263,6 +263,13 @@ type ChainHead struct {
 	OptimisticStatus           bool   `json:"optimistic_status"`
 }
 
+type GetPendingConsolidationsResponse struct {
+	Version             string                  `json:"version"`
+	ExecutionOptimistic bool                    `json:"execution_optimistic"`
+	Finalized           bool                    `json:"finalized"`
+	Data                []*PendingConsolidation `json:"data"`
+}
+
 type GetPendingDepositsResponse struct {
 	Version             string            `json:"version"`
 	ExecutionOptimistic bool              `json:"execution_optimistic"`

api/server/structs/other.go

@@ -253,7 +253,7 @@ type PendingDeposit struct {
 }
 
 type PendingPartialWithdrawal struct {
-	Index             string `json:"index"`
+	Index             string `json:"validator_index"`
 	Amount            string `json:"amount"`
 	WithdrawableEpoch string `json:"withdrawable_epoch"`
 }

beacon-chain/blockchain/BUILD.bazel

@@ -25,7 +25,6 @@ go_library(
         "receive_attestation.go",
         "receive_blob.go",
         "receive_block.go",
-        "receive_data_column.go",
         "service.go",
         "setup_forchoice.go",
         "tracked_proposer.go",
@@ -51,7 +50,6 @@ go_library(
         "//beacon-chain/core/feed/state:go_default_library",
         "//beacon-chain/core/helpers:go_default_library",
         "//beacon-chain/core/light-client:go_default_library",
-        "//beacon-chain/core/peerdas:go_default_library",
         "//beacon-chain/core/signing:go_default_library",
         "//beacon-chain/core/time:go_default_library",
         "//beacon-chain/core/transition:go_default_library",
@@ -148,7 +146,6 @@ go_test(
         "//beacon-chain/core/feed/state:go_default_library",
         "//beacon-chain/core/helpers:go_default_library",
         "//beacon-chain/core/light-client:go_default_library",
-        "//beacon-chain/core/peerdas:go_default_library",
         "//beacon-chain/core/signing:go_default_library",
         "//beacon-chain/core/transition:go_default_library",
         "//beacon-chain/das:go_default_library",

beacon-chain/blockchain/error.go

@@ -40,12 +40,10 @@ var (
 	errNotGenesisRoot = errors.New("root is not the genesis block root")
 	// errBlacklistedBlock is returned when a block is blacklisted as invalid.
 	errBlacklistedRoot = verification.AsVerificationFailure(errors.New("block root is blacklisted"))
-	// errMaxBlobsExceeded is returned when the number of blobs in a block exceeds the maximum allowed.
-	errMaxBlobsExceeded = verification.AsVerificationFailure(errors.New("Expected commitments in block exceeds MAX_BLOBS_PER_BLOCK"))
-	// errMaxDataColumnsExceeded is returned when the number of data columns exceeds the maximum allowed.
-	errMaxDataColumnsExceeded = verification.AsVerificationFailure(errors.New("Expected data columns for node exceeds NUMBER_OF_COLUMNS"))
 )
 
+var errMaxBlobsExceeded = verification.AsVerificationFailure(errors.New("Expected commitments in block exceeds MAX_BLOBS_PER_BLOCK"))
+
 // An invalid block is the block that fails state transition based on the core protocol rules.
 // The beacon node shall not be accepting nor building blocks that branch off from an invalid block.
 // Some examples of invalid blocks are:

beacon-chain/blockchain/execution_engine.go

@@ -184,13 +184,17 @@ func (s *Service) notifyForkchoiceUpdate(ctx context.Context, arg *fcuConfig) (*
 	return payloadID, nil
 }
 
-func firePayloadAttributesEvent(_ context.Context, f event.SubscriberSender, nextSlot primitives.Slot) {
+func (s *Service) firePayloadAttributesEvent(f event.SubscriberSender, block interfaces.ReadOnlySignedBeaconBlock, root [32]byte, nextSlot primitives.Slot) {
+	// If we're syncing a block in the past and init-sync is still running, we shouldn't fire this event.
+	if !s.cfg.SyncChecker.Synced() {
+		return
+	}
 	// the fcu args have differing amounts of completeness based on the code path,
 	// and there is work we only want to do if a client is actually listening to the events beacon api endpoint.
 	// temporary solution: just fire a blank event and fill in the details in the api handler.
 	f.Send(&feed.Event{
 		Type: statefeed.PayloadAttributes,
-		Data: payloadattribute.EventData{ProposalSlot: nextSlot},
+		Data: payloadattribute.EventData{HeadBlock: block, HeadRoot: root, ProposalSlot: nextSlot},
 	})
 }
 
@@ -435,9 +439,6 @@ func (s *Service) removeInvalidBlockAndState(ctx context.Context, blkRoots [][32
 			// Blobs may not exist for some blocks, leading to deletion failures. Log such errors at debug level.
 			log.WithError(err).Debug("Could not remove blob from blob storage")
 		}
-		if err := s.dataColumnStorage.Remove(root); err != nil {
-			log.WithError(err).Debug("Could not remove data columns from data column storage")
-		}
 	}
 	return nil
 }

beacon-chain/blockchain/forkchoice_update_execution.go

@@ -102,7 +102,7 @@ func (s *Service) forkchoiceUpdateWithExecution(ctx context.Context, args *fcuCo
 		log.WithError(err).Error("could not save head")
 	}
 
-	go firePayloadAttributesEvent(ctx, s.cfg.StateNotifier.StateFeed(), s.CurrentSlot()+1)
+	go s.firePayloadAttributesEvent(s.cfg.StateNotifier.StateFeed(), args.headBlock, args.headRoot, s.CurrentSlot()+1)
 
 	// Only need to prune attestations from pool if the head has changed.
 	s.pruneAttsFromPool(s.ctx, args.headState, args.headBlock)

beacon-chain/blockchain/kzg/BUILD.bazel

@@ -7,7 +7,7 @@ go_library(
         "trusted_setup.go",
         "validation.go",
     ],
-    embedsrcs = ["trusted_setup.json"],
+    embedsrcs = ["trusted_setup_4096.json"],
     importpath = "github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg",
     visibility = ["//visibility:public"],
     deps = [

beacon-chain/blockchain/kzg/kzg.go

@@ -39,7 +39,8 @@ type CellsAndProofs struct {
 
 // BlobToKZGCommitment computes a KZG commitment from a given blob.
 func BlobToKZGCommitment(blob *Blob) (Commitment, error) {
-	kzgBlob := kzg4844.Blob(*blob)
+	var kzgBlob kzg4844.Blob
+	copy(kzgBlob[:], blob[:])
 
 	commitment, err := kzg4844.BlobToCommitment(&kzgBlob)
 	if err != nil {
@@ -51,9 +52,10 @@ func BlobToKZGCommitment(blob *Blob) (Commitment, error) {
 
 // ComputeCells computes the (extended) cells from a given blob.
 func ComputeCells(blob *Blob) ([]Cell, error) {
-	ckzgBlob := (*ckzg4844.Blob)(blob)
+	var ckzgBlob ckzg4844.Blob
+	copy(ckzgBlob[:], blob[:])
 
-	ckzgCells, err := ckzg4844.ComputeCells(ckzgBlob)
+	ckzgCells, err := ckzg4844.ComputeCells(&ckzgBlob)
 	if err != nil {
 		return nil, errors.Wrap(err, "compute cells")
 	}
@@ -68,7 +70,8 @@ func ComputeCells(blob *Blob) ([]Cell, error) {
 
 // ComputeBlobKZGProof computes the blob KZG proof from a given blob and its commitment.
 func ComputeBlobKZGProof(blob *Blob, commitment Commitment) (Proof, error) {
-	kzgBlob := kzg4844.Blob(*blob)
+	var kzgBlob kzg4844.Blob
+	copy(kzgBlob[:], blob[:])
 
 	proof, err := kzg4844.ComputeBlobProof(&kzgBlob, kzg4844.Commitment(commitment))
 	if err != nil {
@@ -79,9 +82,10 @@ func ComputeBlobKZGProof(blob *Blob, commitment Commitment) (Proof, error) {
 
 // ComputeCellsAndKZGProofs computes the cells and cells KZG proofs from a given blob.
 func ComputeCellsAndKZGProofs(blob *Blob) (CellsAndProofs, error) {
-	ckzgBlob := (*ckzg4844.Blob)(blob)
+	var ckzgBlob ckzg4844.Blob
+	copy(ckzgBlob[:], blob[:])
 
-	ckzgCells, ckzgProofs, err := ckzg4844.ComputeCellsAndKZGProofs(ckzgBlob)
+	ckzgCells, ckzgProofs, err := ckzg4844.ComputeCellsAndKZGProofs(&ckzgBlob)
 	if err != nil {
 		return CellsAndProofs{}, err
 	}
@@ -89,7 +93,7 @@ func ComputeCellsAndKZGProofs(blob *Blob) (CellsAndProofs, error) {
 	return makeCellsAndProofs(ckzgCells[:], ckzgProofs[:])
 }
 
-// VerifyBlobKZGProof verifies the KZG proofs for a given slice of commitments, cells indices, cells and proofs.
+// VerifyCellKZGProofBatch verifies the KZG proofs for a given slice of commitments, cells indices, cells and proofs.
 // Note: It is way more efficient to call once this function with big slices than calling it multiple times with small slices.
 func VerifyCellKZGProofBatch(commitmentsBytes []Bytes48, cellIndices []uint64, cells []Cell, proofsBytes []Bytes48) (bool, error) {
 	// Convert `Cell` type to `ckzg4844.Cell`
@@ -124,8 +128,9 @@ func makeCellsAndProofs(ckzgCells []ckzg4844.Cell, ckzgProofs []ckzg4844.KZGProo
 		return CellsAndProofs{}, errors.New("different number of cells/proofs")
 	}
 
-	var cells []Cell
-	var proofs []Proof
+	cells := make([]Cell, 0, len(ckzgCells))
+	proofs := make([]Proof, 0, len(ckzgProofs))
+
 	for i := range ckzgCells {
 		cells = append(cells, Cell(ckzgCells[i]))
 		proofs = append(proofs, Proof(ckzgProofs[i]))

beacon-chain/blockchain/kzg/trusted_setup.go

@@ -11,7 +11,8 @@ import (
 )
 
 var (
-	//go:embed trusted_setup.json
+	// https://github.com/ethereum/consensus-specs/blob/dev/presets/mainnet/trusted_setups/trusted_setup_4096.json
+	//go:embed trusted_setup_4096.json
 	embeddedTrustedSetup []byte // 1.2Mb
 	kzgContext           *GoKZG.Context
 	kzgLoaded            bool
@@ -29,9 +30,11 @@ func Start() error {
 	if err != nil {
 		return errors.Wrap(err, "could not parse trusted setup JSON")
 	}
+
 	kzgContext, err = GoKZG.NewContext4096(&GoKZG.JSONTrustedSetup{
 		SetupG2:         trustedSetup.G2Monomial[:],
-		SetupG1Lagrange: trustedSetup.G1Lagrange})
+		SetupG1Lagrange: trustedSetup.G1Lagrange,
+	})
 	if err != nil {
 		return errors.Wrap(err, "could not initialize go-kzg context")
 	}

beacon-chain/blockchain/mock_test.go

@@ -20,6 +20,7 @@ func testServiceOptsWithDB(t *testing.T) []Option {
 		WithForkChoiceStore(fcs),
 		WithClockSynchronizer(cs),
 		WithStateNotifier(&mock.MockStateNotifier{RecordEvents: true}),
+		WithSyncChecker(&mock.MockSyncChecker{}),
 	}
 }
 

beacon-chain/blockchain/options.go

@@ -1,13 +1,10 @@
 package blockchain
 
 import (
-	"time"
-
 	"github.com/OffchainLabs/prysm/v6/async/event"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/cache"
 	statefeed "github.com/OffchainLabs/prysm/v6/beacon-chain/core/feed/state"
 	lightclient "github.com/OffchainLabs/prysm/v6/beacon-chain/core/light-client"
-	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/db"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/db/filesystem"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/execution"
@@ -130,9 +127,9 @@ func WithBLSToExecPool(p blstoexec.PoolManager) Option {
 }
 
 // WithP2PBroadcaster to broadcast messages after appropriate processing.
-func WithP2PBroadcaster(p p2p.Acceser) Option {
+func WithP2PBroadcaster(p p2p.Broadcaster) Option {
 	return func(s *Service) error {
-		s.cfg.P2P = p
+		s.cfg.P2p = p
 		return nil
 	}
 }
@@ -211,14 +208,6 @@ func WithBlobStorage(b *filesystem.BlobStorage) Option {
 	}
 }
 
-// WithDataColumnStorage sets the data column storage backend for the blockchain service.
-func WithDataColumnStorage(b *filesystem.DataColumnStorage) Option {
-	return func(s *Service) error {
-		s.dataColumnStorage = b
-		return nil
-	}
-}
-
 func WithSyncChecker(checker Checker) Option {
 	return func(s *Service) error {
 		s.cfg.SyncChecker = checker
@@ -226,13 +215,6 @@ func WithSyncChecker(checker Checker) Option {
 	}
 }
 
-func WithCustodyInfo(custodyInfo *peerdas.CustodyInfo) Option {
-	return func(s *Service) error {
-		s.cfg.CustodyInfo = custodyInfo
-		return nil
-	}
-}
-
 func WithSlasherEnabled(enabled bool) Option {
 	return func(s *Service) error {
 		s.slasherEnabled = enabled
@@ -240,13 +222,6 @@ func WithSlasherEnabled(enabled bool) Option {
 	}
 }
 
-func WithGenesisTime(genesisTime time.Time) Option {
-	return func(s *Service) error {
-		s.genesisTime = genesisTime
-		return nil
-	}
-}
-
 func WithLightClientStore(lcs *lightclient.Store) Option {
 	return func(s *Service) error {
 		s.lcStore = lcs

beacon-chain/blockchain/process_block.go

@@ -3,12 +3,10 @@ package blockchain
 import (
 	"context"
 	"fmt"
-	"slices"
 	"time"
 
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/blocks"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
-	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
 	coreTime "github.com/OffchainLabs/prysm/v6/beacon-chain/core/time"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/transition"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/das"
@@ -241,9 +239,8 @@ func (s *Service) onBlockBatch(ctx context.Context, blks []consensusblocks.ROBlo
 				return err
 			}
 		}
-
 		if err := avs.IsDataAvailable(ctx, s.CurrentSlot(), b); err != nil {
-			return errors.Wrapf(err, "could not validate sidecar availability at slot %d", b.Block().Slot())
+			return errors.Wrapf(err, "could not validate blob data availability at slot %d", b.Block().Slot())
 		}
 		args := &forkchoicetypes.BlockAndCheckpoints{Block: b,
 			JustifiedCheckpoint: jCheckpoints[i],
@@ -604,206 +601,20 @@ func missingIndices(bs *filesystem.BlobStorage, root [32]byte, expected [][]byte
 	return missing, nil
 }
 
-func missingDataColumns(bs *filesystem.DataColumnStorage, root [32]byte, expected map[uint64]bool) (map[uint64]bool, error) {
-	if len(expected) == 0 {
-		return nil, nil
-	}
-
-	if len(expected) > int(params.BeaconConfig().NumberOfColumns) {
-		return nil, errMaxDataColumnsExceeded
-	}
-
-	// Get a summary of the data columns stored in the database.
-	summary := bs.Summary(root)
-
-	// Check all expected data columns against the summary.
-	missing := make(map[uint64]bool)
-	for column := range expected {
-		if !summary.HasIndex(column) {
-			missing[column] = true
-		}
-	}
-
-	return missing, nil
-}
-
-// isDataAvailable blocks until all sidecars committed to in the block are available,
+// isDataAvailable blocks until all BlobSidecars committed to in the block are available,
 // or an error or context cancellation occurs. A nil result means that the data availability check is successful.
 // The function will first check the database to see if all sidecars have been persisted. If any
-// sidecars are missing, it will then read from the sidecar notifier channel for the given root until the channel is
+// sidecars are missing, it will then read from the blobNotifier channel for the given root until the channel is
 // closed, the context hits cancellation/timeout, or notifications have been received for all the missing sidecars.
-func (s *Service) isDataAvailable(ctx context.Context, root [32]byte, signedBlock interfaces.ReadOnlySignedBeaconBlock) error {
-	block := signedBlock.Block()
+func (s *Service) isDataAvailable(ctx context.Context, root [32]byte, signed interfaces.ReadOnlySignedBeaconBlock) error {
+	if signed.Version() < version.Deneb {
+		return nil
+	}
+
+	block := signed.Block()
 	if block == nil {
 		return errors.New("invalid nil beacon block")
 	}
-
-	blockVersion := block.Version()
-	if blockVersion >= version.Fulu {
-		return s.areDataColumnsAvailable(ctx, root, block)
-	}
-
-	if blockVersion >= version.Deneb {
-		return s.areBlobsAvailable(ctx, root, block)
-	}
-
-	return nil
-}
-
-// areDataColumnsAvailable blocks until all data columns committed to in the block are available,
-// or an error or context cancellation occurs. A nil result means that the data availability check is successful.
-func (s *Service) areDataColumnsAvailable(ctx context.Context, root [fieldparams.RootLength]byte, block interfaces.ReadOnlyBeaconBlock) error {
-	// We are only required to check within MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS
-	blockSlot, currentSlot := block.Slot(), s.CurrentSlot()
-	blockEpoch, currentEpoch := slots.ToEpoch(blockSlot), slots.ToEpoch(currentSlot)
-
-	if !params.WithinDAPeriod(blockEpoch, currentEpoch) {
-		return nil
-	}
-
-	body := block.Body()
-	if body == nil {
-		return errors.New("invalid nil beacon block body")
-	}
-
-	kzgCommitments, err := body.BlobKzgCommitments()
-	if err != nil {
-		return errors.Wrap(err, "blob KZG commitments")
-	}
-
-	// If block has not commitments there is nothing to wait for.
-	if len(kzgCommitments) == 0 {
-		return nil
-	}
-
-	// All columns to sample need to be available for the block to be considered available.
-	// https://github.com/ethereum/consensus-specs/blob/v1.5.0-alpha.10/specs/fulu/das-core.md#custody-sampling
-	nodeID := s.cfg.P2P.NodeID()
-
-	// Prevent custody group count to change during the rest of the function.
-	s.cfg.CustodyInfo.Mut.RLock()
-	defer s.cfg.CustodyInfo.Mut.RUnlock()
-
-	// Get the custody group sampling size for the node.
-	custodyGroupSamplingSize := s.cfg.CustodyInfo.CustodyGroupSamplingSize(peerdas.Actual)
-	peerInfo, _, err := peerdas.Info(nodeID, custodyGroupSamplingSize)
-	if err != nil {
-		return errors.Wrap(err, "peer info")
-	}
-
-	// Subscribe to newly data columns stored in the database.
-	identsChan := make(chan filesystem.DataColumnsIdent)
-	subscription := s.dataColumnStorage.DataColumnFeed.Subscribe(identsChan)
-	defer subscription.Unsubscribe()
-
-	// Get the count of data columns we already have in the store.
-	summary := s.dataColumnStorage.Summary(root)
-	storedDataColumnsCount := summary.Count()
-
-	// As soon as we have more than half of the data columns, we can reconstruct the missing ones.
-	// We don't need to wait for the rest of the data columns to declare the block as available.
-	if peerdas.CanSelfReconstruct(storedDataColumnsCount) {
-		return nil
-	}
-
-	// Get a map of data column indices that are not currently available.
-	missingMap, err := missingDataColumns(s.dataColumnStorage, root, peerInfo.CustodyColumns)
-	if err != nil {
-		return errors.Wrap(err, "missing data columns")
-	}
-
-	// If there are no missing indices, all data column sidecars are available.
-	// This is the happy path.
-	if len(missingMap) == 0 {
-		return nil
-	}
-
-	// Log for DA checks that cross over into the next slot; helpful for debugging.
-	nextSlot := slots.BeginsAt(block.Slot()+1, s.genesisTime)
-
-	// Avoid logging if DA check is called after next slot start.
-	if nextSlot.After(time.Now()) {
-		timer := time.AfterFunc(time.Until(nextSlot), func() {
-			missingMapCount := uint64(len(missingMap))
-
-			if missingMapCount == 0 {
-				return
-			}
-
-			var (
-				expected interface{} = "all"
-				missing  interface{} = "all"
-			)
-
-			numberOfColumns := params.BeaconConfig().NumberOfColumns
-			colMapCount := uint64(len(peerInfo.CustodyColumns))
-
-			if colMapCount < numberOfColumns {
-				expected = uint64MapToSortedSlice(peerInfo.CustodyColumns)
-			}
-
-			if missingMapCount < numberOfColumns {
-				missing = uint64MapToSortedSlice(missingMap)
-			}
-
-			log.WithFields(logrus.Fields{
-				"slot":            block.Slot(),
-				"root":            fmt.Sprintf("%#x", root),
-				"columnsExpected": expected,
-				"columnsWaiting":  missing,
-			}).Warning("Data columns still missing at slot end")
-		})
-		defer timer.Stop()
-	}
-
-	for {
-		select {
-		case idents := <-identsChan:
-			if idents.Root != root {
-				// This is not the root we are looking for.
-				continue
-			}
-
-			for _, index := range idents.Indices {
-				// This is a data column we are expecting.
-				if _, ok := missingMap[index]; ok {
-					storedDataColumnsCount++
-				}
-
-				// As soon as we have more than half of the data columns, we can reconstruct the missing ones.
-				// We don't need to wait for the rest of the data columns to declare the block as available.
-				if peerdas.CanSelfReconstruct(storedDataColumnsCount) {
-					return nil
-				}
-
-				// Remove the index from the missing map.
-				delete(missingMap, index)
-
-				// Return if there is no more missing data columns.
-				if len(missingMap) == 0 {
-					return nil
-				}
-			}
-
-		case <-ctx.Done():
-			var missingIndices interface{} = "all"
-			numberOfColumns := params.BeaconConfig().NumberOfColumns
-			missingIndicesCount := uint64(len(missingMap))
-
-			if missingIndicesCount < numberOfColumns {
-				missingIndices = uint64MapToSortedSlice(missingMap)
-			}
-
-			return errors.Wrapf(ctx.Err(), "data column sidecars slot: %d, BlockRoot: %#x, missing %v", block.Slot(), root, missingIndices)
-		}
-	}
-}
-
-// areBlobsAvailable blocks until all BlobSidecars committed to in the block are available,
-// or an error or context cancellation occurs. A nil result means that the data availability check is successful.
-func (s *Service) areBlobsAvailable(ctx context.Context, root [fieldparams.RootLength]byte, block interfaces.ReadOnlyBeaconBlock) error {
-	blockSlot := block.Slot()
-
 	// We are only required to check within MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS
 	if !params.WithinDAPeriod(slots.ToEpoch(block.Slot()), slots.ToEpoch(s.CurrentSlot())) {
 		return nil
@@ -825,7 +636,7 @@ func (s *Service) areBlobsAvailable(ctx context.Context, root [fieldparams.RootL
 	// get a map of BlobSidecar indices that are not currently available.
 	missing, err := missingIndices(s.blobStorage, root, kzgCommitments, block.Slot())
 	if err != nil {
-		return errors.Wrap(err, "missing indices")
+		return err
 	}
 	// If there are no missing indices, all BlobSidecars are available.
 	if len(missing) == 0 {
@@ -837,20 +648,15 @@ func (s *Service) areBlobsAvailable(ctx context.Context, root [fieldparams.RootL
 	nc := s.blobNotifiers.forRoot(root, block.Slot())
 
 	// Log for DA checks that cross over into the next slot; helpful for debugging.
-	nextSlot := slots.BeginsAt(block.Slot()+1, s.genesisTime)
+	nextSlot := slots.BeginsAt(signed.Block().Slot()+1, s.genesisTime)
 	// Avoid logging if DA check is called after next slot start.
 	if nextSlot.After(time.Now()) {
 		nst := time.AfterFunc(time.Until(nextSlot), func() {
 			if len(missing) == 0 {
 				return
 			}
-
-			log.WithFields(logrus.Fields{
-				"slot":          blockSlot,
-				"root":          fmt.Sprintf("%#x", root),
-				"blobsExpected": expected,
-				"blobsWaiting":  len(missing),
-			}).Error("Still waiting for blobs DA check at slot end.")
+			log.WithFields(daCheckLogFields(root, signed.Block().Slot(), expected, len(missing))).
+				Error("Still waiting for DA check at slot end.")
 		})
 		defer nst.Stop()
 	}
@@ -872,14 +678,13 @@ func (s *Service) areBlobsAvailable(ctx context.Context, root [fieldparams.RootL
 	}
 }
 
-// uint64MapToSortedSlice produces a sorted uint64 slice from a map.
-func uint64MapToSortedSlice(input map[uint64]bool) []uint64 {
-	output := make([]uint64, 0, len(input))
-	for idx := range input {
-		output = append(output, idx)
+func daCheckLogFields(root [32]byte, slot primitives.Slot, expected, missing int) logrus.Fields {
+	return logrus.Fields{
+		"slot":          slot,
+		"root":          fmt.Sprintf("%#x", root),
+		"blobsExpected": expected,
+		"blobsWaiting":  missing,
 	}
-	slices.Sort[[]uint64](output)
-	return output
 }
 
 // lateBlockTasks  is called 4 seconds into the slot and performs tasks
@@ -924,9 +729,13 @@ func (s *Service) lateBlockTasks(ctx context.Context) {
 	attribute := s.getPayloadAttribute(ctx, headState, s.CurrentSlot()+1, headRoot[:])
 	// return early if we are not proposing next slot
 	if attribute.IsEmpty() {
+		headBlock, err := s.headBlock()
+		if err != nil {
+			log.WithError(err).WithField("head_root", headRoot).Error("unable to retrieve head block to fire payload attributes event")
+		}
 		// notifyForkchoiceUpdate fires the payload attribute event. But in this case, we won't
 		// call notifyForkchoiceUpdate, so the event is fired here.
-		go firePayloadAttributesEvent(ctx, s.cfg.StateNotifier.StateFeed(), s.CurrentSlot()+1)
+		go s.firePayloadAttributesEvent(s.cfg.StateNotifier.StateFeed(), headBlock, headRoot, s.CurrentSlot()+1)
 		return
 	}
 
@@ -961,7 +770,7 @@ func (s *Service) waitForSync() error {
 	}
 }
 
-func (s *Service) handleInvalidExecutionError(ctx context.Context, err error, blockRoot, parentRoot [32]byte) error {
+func (s *Service) handleInvalidExecutionError(ctx context.Context, err error, blockRoot [32]byte, parentRoot [32]byte) error {
 	if IsInvalidBlock(err) && InvalidBlockLVH(err) != [32]byte{} {
 		return s.pruneInvalidBlock(ctx, blockRoot, parentRoot, InvalidBlockLVH(err))
 	}

beacon-chain/blockchain/process_block_helpers.go

@@ -103,15 +103,29 @@ func (s *Service) sendStateFeedOnBlock(cfg *postBlockProcessConfig) {
 		log.WithError(err).Debug("Could not check if block is optimistic")
 		optimistic = true
 	}
+	currEpoch := slots.ToEpoch(s.CurrentSlot())
+	currDependenRoot, err := s.cfg.ForkChoiceStore.DependentRoot(currEpoch)
+	if err != nil {
+		log.WithError(err).Debug("Could not get dependent root")
+	}
+	prevDependentRoot := [32]byte{}
+	if currEpoch > 0 {
+		prevDependentRoot, err = s.cfg.ForkChoiceStore.DependentRoot(currEpoch - 1)
+		if err != nil {
+			log.WithError(err).Debug("Could not get previous dependent root")
+		}
+	}
 	// Send notification of the processed block to the state feed.
 	s.cfg.StateNotifier.StateFeed().Send(&feed.Event{
 		Type: statefeed.BlockProcessed,
 		Data: &statefeed.BlockProcessedData{
-			Slot:        cfg.roblock.Block().Slot(),
-			BlockRoot:   cfg.roblock.Root(),
-			SignedBlock: cfg.roblock,
-			Verified:    true,
-			Optimistic:  optimistic,
+			Slot:              cfg.roblock.Block().Slot(),
+			BlockRoot:         cfg.roblock.Root(),
+			SignedBlock:       cfg.roblock,
+			CurrDependentRoot: currDependenRoot,
+			PrevDependentRoot: prevDependentRoot,
+			Verified:          true,
+			Optimistic:        optimistic,
 		},
 	})
 }
@@ -295,6 +309,11 @@ func (s *Service) processLightClientFinalityUpdate(
 		Type: statefeed.LightClientFinalityUpdate,
 		Data: newUpdate,
 	})
+
+	if err = s.cfg.P2p.BroadcastLightClientFinalityUpdate(ctx, newUpdate); err != nil {
+		return errors.Wrap(err, "could not broadcast light client finality update")
+	}
+
 	return nil
 }
 
@@ -344,6 +363,10 @@ func (s *Service) processLightClientOptimisticUpdate(ctx context.Context, signed
 		Data: newUpdate,
 	})
 
+	if err = s.cfg.P2p.BroadcastLightClientOptimisticUpdate(ctx, newUpdate); err != nil {
+		return errors.Wrap(err, "could not broadcast light client optimistic update")
+	}
+
 	return nil
 }
 

beacon-chain/blockchain/process_block_test.go

@@ -2,7 +2,6 @@ package blockchain
 
 import (
 	"context"
-	"crypto/rand"
 	"fmt"
 	"math/big"
 	"strconv"
@@ -14,7 +13,6 @@ import (
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/blocks"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
 	lightClient "github.com/OffchainLabs/prysm/v6/beacon-chain/core/light-client"
-	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/signing"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/transition"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/das"
@@ -26,8 +24,8 @@ import (
 	doublylinkedtree "github.com/OffchainLabs/prysm/v6/beacon-chain/forkchoice/doubly-linked-tree"
 	forkchoicetypes "github.com/OffchainLabs/prysm/v6/beacon-chain/forkchoice/types"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/operations/attestations/kv"
+	mockp2p "github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/testing"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/state"
-	"github.com/OffchainLabs/prysm/v6/beacon-chain/verification"
 	"github.com/OffchainLabs/prysm/v6/config/features"
 	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
 	"github.com/OffchainLabs/prysm/v6/config/params"
@@ -3248,154 +3246,6 @@ func TestSaveLightClientBootstrap(t *testing.T) {
 	reset()
 }
 
-type testIsAvailableParams struct {
-	options                 []Option
-	blobKzgCommitmentsCount uint64
-	columnsToSave           []uint64
-}
-
-func testIsAvailableSetup(t *testing.T, params testIsAvailableParams) (context.Context, context.CancelFunc, *Service, [fieldparams.RootLength]byte, interfaces.SignedBeaconBlock) {
-	ctx, cancel := context.WithCancel(context.Background())
-	dataColumnStorage := filesystem.NewEphemeralDataColumnStorage(t)
-
-	options := append(params.options, WithDataColumnStorage(dataColumnStorage))
-	service, _ := minimalTestService(t, options...)
-
-	genesisState, secretKeys := util.DeterministicGenesisStateElectra(t, 32 /*validator count*/)
-
-	err := service.saveGenesisData(ctx, genesisState)
-	require.NoError(t, err)
-
-	conf := util.DefaultBlockGenConfig()
-	conf.NumBlobKzgCommitments = params.blobKzgCommitmentsCount
-
-	signedBeaconBlock, err := util.GenerateFullBlockFulu(genesisState, secretKeys, conf, 10 /*block slot*/)
-	require.NoError(t, err)
-
-	root, err := signedBeaconBlock.Block.HashTreeRoot()
-	require.NoError(t, err)
-
-	dataColumnsParams := make([]verification.DataColumnParams, 0, len(params.columnsToSave))
-	for _, i := range params.columnsToSave {
-		dataColumnParam := verification.DataColumnParams{ColumnIndex: i}
-		dataColumnsParams = append(dataColumnsParams, dataColumnParam)
-	}
-
-	dataColumnParamsByBlockRoot := verification.DataColumnsParamsByRoot{root: dataColumnsParams}
-	_, verifiedRODataColumns := verification.CreateTestVerifiedRoDataColumnSidecars(t, dataColumnParamsByBlockRoot)
-
-	err = dataColumnStorage.Save(verifiedRODataColumns)
-	require.NoError(t, err)
-
-	signed, err := consensusblocks.NewSignedBeaconBlock(signedBeaconBlock)
-	require.NoError(t, err)
-
-	return ctx, cancel, service, root, signed
-}
-
-func TestIsDataAvailable(t *testing.T) {
-	t.Run("Fulu - out of retention window", func(t *testing.T) {
-		params := testIsAvailableParams{options: []Option{WithGenesisTime(time.Unix(0, 0))}}
-		ctx, _, service, root, signed := testIsAvailableSetup(t, params)
-
-		err := service.isDataAvailable(ctx, root, signed)
-		require.NoError(t, err)
-	})
-
-	t.Run("Fulu - no commitment in blocks", func(t *testing.T) {
-		ctx, _, service, root, signed := testIsAvailableSetup(t, testIsAvailableParams{})
-
-		err := service.isDataAvailable(ctx, root, signed)
-		require.NoError(t, err)
-	})
-
-	t.Run("Fulu - more than half of the columns in custody", func(t *testing.T) {
-		halfNumberOfColumns := params.BeaconConfig().NumberOfColumns / 2
-		indices := make([]uint64, 0, halfNumberOfColumns)
-		for i := range halfNumberOfColumns {
-			indices = append(indices, i)
-		}
-
-		params := testIsAvailableParams{
-			options:                 []Option{WithCustodyInfo(&peerdas.CustodyInfo{})},
-			columnsToSave:           indices,
-			blobKzgCommitmentsCount: 3,
-		}
-
-		ctx, _, service, root, signed := testIsAvailableSetup(t, params)
-
-		err := service.isDataAvailable(ctx, root, signed)
-		require.NoError(t, err)
-	})
-
-	t.Run("Fulu - no missing data columns", func(t *testing.T) {
-		params := testIsAvailableParams{
-			options:                 []Option{WithCustodyInfo(&peerdas.CustodyInfo{})},
-			columnsToSave:           []uint64{1, 17, 19, 42, 75, 87, 102, 117, 119}, // 119 is not needed
-			blobKzgCommitmentsCount: 3,
-		}
-
-		ctx, _, service, root, signed := testIsAvailableSetup(t, params)
-
-		err := service.isDataAvailable(ctx, root, signed)
-		require.NoError(t, err)
-	})
-
-	t.Run("Fulu - some initially missing data columns (no reconstruction)", func(t *testing.T) {
-		var custodyInfo peerdas.CustodyInfo
-		custodyInfo.TargetGroupCount.SetValidatorsCustodyRequirement(128)
-		custodyInfo.ToAdvertiseGroupCount.Set(128)
-
-		testParams := testIsAvailableParams{
-			options:                 []Option{WithCustodyInfo(&custodyInfo)},
-			blobKzgCommitmentsCount: 3,
-		}
-
-		ctx, _, service, root, signed := testIsAvailableSetup(t, testParams)
-
-		// If needed, generate a random root that is different from the block root.
-		var randomRoot [fieldparams.RootLength]byte
-		for randomRoot == root {
-			randomRootSlice := make([]byte, fieldparams.RootLength)
-			_, err := rand.Read(randomRootSlice)
-			require.NoError(t, err)
-			copy(randomRoot[:], randomRootSlice)
-		}
-
-		// TODO: Achieve the same result without using time.AfterFunc.
-		time.AfterFunc(10*time.Millisecond, func() {
-			halfNumberOfColumns := params.BeaconConfig().NumberOfColumns / 2
-			indices := make([]uint64, 0, halfNumberOfColumns)
-			for i := range halfNumberOfColumns {
-				indices = append(indices, i)
-			}
-
-			withSomeRequiredColumns := filesystem.DataColumnsIdent{Root: root, Indices: indices}
-			service.dataColumnStorage.DataColumnFeed.Send(withSomeRequiredColumns)
-		})
-
-		err := service.isDataAvailable(ctx, root, signed)
-		require.NoError(t, err)
-	})
-
-	t.Run("Fulu - some columns are definitively missing", func(t *testing.T) {
-		params := testIsAvailableParams{
-			options:                 []Option{WithCustodyInfo(&peerdas.CustodyInfo{})},
-			blobKzgCommitmentsCount: 3,
-		}
-
-		ctx, cancel, service, root, signed := testIsAvailableSetup(t, params)
-
-		// TODO: Achieve the same result without using time.AfterFunc.
-		time.AfterFunc(10*time.Millisecond, func() {
-			cancel()
-		})
-
-		err := service.isDataAvailable(ctx, root, signed)
-		require.NotNil(t, err)
-	})
-}
-
 func setupLightClientTestRequirements(ctx context.Context, t *testing.T, s *Service, v int, options ...util.LightClientOption) (*util.TestLightClient, *postBlockProcessConfig) {
 	var l *util.TestLightClient
 	switch v {
@@ -3460,6 +3310,7 @@ func TestProcessLightClientOptimisticUpdate(t *testing.T) {
 	params.OverrideBeaconConfig(beaconCfg)
 
 	s, tr := minimalTestService(t)
+	s.cfg.P2p = &mockp2p.FakeP2P{}
 	ctx := tr.ctx
 
 	testCases := []struct {
@@ -3595,6 +3446,7 @@ func TestProcessLightClientFinalityUpdate(t *testing.T) {
 	params.OverrideBeaconConfig(beaconCfg)
 
 	s, tr := minimalTestService(t)
+	s.cfg.P2p = &mockp2p.FakeP2P{}
 	ctx := tr.ctx
 
 	testCases := []struct {

beacon-chain/blockchain/receive_block.go

@@ -52,13 +52,6 @@ type BlobReceiver interface {
 	ReceiveBlob(context.Context, blocks.VerifiedROBlob) error
 }
 
-// DataColumnReceiver interface defines the methods of chain service for receiving new
-// data columns
-type DataColumnReceiver interface {
-	ReceiveDataColumn(blocks.VerifiedRODataColumn) error
-	ReceiveDataColumns([]blocks.VerifiedRODataColumn) error
-}
-
 // SlashingReceiver interface defines the methods of chain service for receiving validated slashing over the wire.
 type SlashingReceiver interface {
 	ReceiveAttesterSlashing(ctx context.Context, slashing ethpb.AttSlashing)
@@ -81,7 +74,6 @@ func (s *Service) ReceiveBlock(ctx context.Context, block interfaces.ReadOnlySig
 		log.WithField("blockRoot", fmt.Sprintf("%#x", blockRoot)).Debug("Ignoring already synced block")
 		return nil
 	}
-
 	receivedTime := time.Now()
 	s.blockBeingSynced.set(blockRoot)
 	defer s.blockBeingSynced.unset(blockRoot)
@@ -90,7 +82,6 @@ func (s *Service) ReceiveBlock(ctx context.Context, block interfaces.ReadOnlySig
 	if err != nil {
 		return err
 	}
-
 	preState, err := s.getBlockPreState(ctx, blockCopy.Block())
 	if err != nil {
 		return errors.Wrap(err, "could not get block's prestate")
@@ -106,12 +97,10 @@ func (s *Service) ReceiveBlock(ctx context.Context, block interfaces.ReadOnlySig
 	if err != nil {
 		return err
 	}
-
 	daWaitedTime, err := s.handleDA(ctx, blockCopy, blockRoot, avs)
 	if err != nil {
 		return err
 	}
-
 	// Defragment the state before continuing block processing.
 	s.defragmentState(postState)
 
@@ -240,32 +229,24 @@ func (s *Service) handleDA(
 	block interfaces.SignedBeaconBlock,
 	blockRoot [32]byte,
 	avs das.AvailabilityStore,
-) (elapsed time.Duration, err error) {
-	defer func(start time.Time) {
-		elapsed := time.Since(start)
-
-		if err == nil {
-			dataAvailWaitedTime.Observe(float64(elapsed.Milliseconds()))
+) (time.Duration, error) {
+	daStartTime := time.Now()
+	if avs != nil {
+		rob, err := blocks.NewROBlockWithRoot(block, blockRoot)
+		if err != nil {
+			return 0, err
 		}
-	}(time.Now())
-
-	if avs == nil {
-		if err = s.isDataAvailable(ctx, blockRoot, block); err != nil {
-			return
+		if err := avs.IsDataAvailable(ctx, s.CurrentSlot(), rob); err != nil {
+			return 0, errors.Wrap(err, "could not validate blob data availability (AvailabilityStore.IsDataAvailable)")
+		}
+	} else {
+		if err := s.isDataAvailable(ctx, blockRoot, block); err != nil {
+			return 0, errors.Wrap(err, "could not validate blob data availability")
 		}
-
-		return
 	}
-
-	var rob blocks.ROBlock
-	rob, err = blocks.NewROBlockWithRoot(block, blockRoot)
-	if err != nil {
-		return
-	}
-
-	err = avs.IsDataAvailable(ctx, s.CurrentSlot(), rob)
-
-	return
+	daWaitedTime := time.Since(daStartTime)
+	dataAvailWaitedTime.Observe(float64(daWaitedTime.Milliseconds()))
+	return daWaitedTime, nil
 }
 
 func (s *Service) reportPostBlockProcessing(

beacon-chain/blockchain/receive_data_column.go

@@ -1,25 +0,0 @@
-package blockchain
-
-import (
-	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
-	"github.com/pkg/errors"
-)
-
-// ReceiveDataColumns receives a batch of data columns.
-func (s *Service) ReceiveDataColumns(dataColumnSidecars []blocks.VerifiedRODataColumn) error {
-	if err := s.dataColumnStorage.Save(dataColumnSidecars); err != nil {
-		return errors.Wrap(err, "save data column sidecars")
-	}
-
-	return nil
-}
-
-// ReceiveDataColumn receives a single data column.
-// (It is only a wrapper around ReceiveDataColumns.)
-func (s *Service) ReceiveDataColumn(dataColumnSidecar blocks.VerifiedRODataColumn) error {
-	if err := s.dataColumnStorage.Save([]blocks.VerifiedRODataColumn{dataColumnSidecar}); err != nil {
-		return errors.Wrap(err, "save data column sidecars")
-	}
-
-	return nil
-}

beacon-chain/blockchain/service.go

@@ -16,7 +16,6 @@ import (
 	statefeed "github.com/OffchainLabs/prysm/v6/beacon-chain/core/feed/state"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
 	lightClient "github.com/OffchainLabs/prysm/v6/beacon-chain/core/light-client"
-	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
 	coreTime "github.com/OffchainLabs/prysm/v6/beacon-chain/core/time"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/transition"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/db"
@@ -31,7 +30,6 @@ import (
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/startup"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/state"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/state/stategen"
-	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
 	"github.com/OffchainLabs/prysm/v6/config/params"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
@@ -66,7 +64,6 @@ type Service struct {
 	blobNotifiers        *blobNotifierMap
 	blockBeingSynced     *currentlySyncingBlock
 	blobStorage          *filesystem.BlobStorage
-	dataColumnStorage    *filesystem.DataColumnStorage
 	slasherEnabled       bool
 	lcStore              *lightClient.Store
 }
@@ -84,7 +81,7 @@ type config struct {
 	ExitPool                voluntaryexits.PoolManager
 	SlashingPool            slashings.PoolManager
 	BLSToExecPool           blstoexec.PoolManager
-	P2P                     p2p.Acceser
+	P2p                     p2p.Broadcaster
 	MaxRoutines             int
 	StateNotifier           statefeed.Notifier
 	ForkChoiceStore         f.ForkChoicer
@@ -96,7 +93,6 @@ type config struct {
 	FinalizedStateAtStartUp state.BeaconState
 	ExecutionEngineCaller   execution.EngineCaller
 	SyncChecker             Checker
-	CustodyInfo             *peerdas.CustodyInfo
 }
 
 // Checker is an interface used to determine if a node is in initial sync
@@ -110,26 +106,22 @@ var ErrMissingClockSetter = errors.New("blockchain Service initialized without a
 type blobNotifierMap struct {
 	sync.RWMutex
 	notifiers map[[32]byte]chan uint64
-	// TODO: Separate blobs from data columns
-	// seenIndex map[[32]byte][]bool
-	seenIndex map[[32]byte][fieldparams.NumberOfColumns]bool
+	seenIndex map[[32]byte][]bool
 }
 
 // notifyIndex notifies a blob by its index for a given root.
 // It uses internal maps to keep track of seen indices and notifier channels.
 func (bn *blobNotifierMap) notifyIndex(root [32]byte, idx uint64, slot primitives.Slot) {
-	// TODO: Separate blobs from data columns
-	// maxBlobsPerBlock := params.BeaconConfig().MaxBlobsPerBlock(slot)
-	// if idx >= uint64(maxBlobsPerBlock) {
-	// 	return
-	// }
+	maxBlobsPerBlock := params.BeaconConfig().MaxBlobsPerBlock(slot)
+	if idx >= uint64(maxBlobsPerBlock) {
+		return
+	}
 
 	bn.Lock()
 	seen := bn.seenIndex[root]
-	// TODO: Separate blobs from data columns
-	// if seen == nil {
-	// seen = make([]bool, maxBlobsPerBlock)
-	// }
+	if seen == nil {
+		seen = make([]bool, maxBlobsPerBlock)
+	}
 	if seen[idx] {
 		bn.Unlock()
 		return
@@ -140,9 +132,7 @@ func (bn *blobNotifierMap) notifyIndex(root [32]byte, idx uint64, slot primitive
 	// Retrieve or create the notifier channel for the given root.
 	c, ok := bn.notifiers[root]
 	if !ok {
-		// TODO: Separate blobs from data columns
-		// c = make(chan uint64, maxBlobsPerBlock)
-		c = make(chan uint64, fieldparams.NumberOfColumns)
+		c = make(chan uint64, maxBlobsPerBlock)
 		bn.notifiers[root] = c
 	}
 
@@ -152,15 +142,12 @@ func (bn *blobNotifierMap) notifyIndex(root [32]byte, idx uint64, slot primitive
 }
 
 func (bn *blobNotifierMap) forRoot(root [32]byte, slot primitives.Slot) chan uint64 {
-	// TODO: Separate blobs from data columns
-	// maxBlobsPerBlock := params.BeaconConfig().MaxBlobsPerBlock(slot)
+	maxBlobsPerBlock := params.BeaconConfig().MaxBlobsPerBlock(slot)
 	bn.Lock()
 	defer bn.Unlock()
 	c, ok := bn.notifiers[root]
 	if !ok {
-		// TODO: Separate blobs from data columns
-		// c = make(chan uint64, maxBlobsPerBlock)
-		c = make(chan uint64, fieldparams.NumberOfColumns)
+		c = make(chan uint64, maxBlobsPerBlock)
 		bn.notifiers[root] = c
 	}
 	return c
@@ -186,9 +173,7 @@ func NewService(ctx context.Context, opts ...Option) (*Service, error) {
 	ctx, cancel := context.WithCancel(ctx)
 	bn := &blobNotifierMap{
 		notifiers: make(map[[32]byte]chan uint64),
-		// TODO: Separate blobs from data columns
-		// seenIndex: make(map[[32]byte][]bool),
-		seenIndex: make(map[[32]byte][fieldparams.NumberOfColumns]bool),
+		seenIndex: make(map[[32]byte][]bool),
 	}
 	srv := &Service{
 		ctx:                  ctx,

beacon-chain/blockchain/service_test.go

@@ -97,7 +97,7 @@ func setupBeaconChain(t *testing.T, beaconDB db.Database) *Service {
 		WithAttestationPool(attestations.NewPool()),
 		WithSlashingPool(slashings.NewPool()),
 		WithExitPool(voluntaryexits.NewPool()),
-		WithP2PBroadcaster(&mockAccesser{}),
+		WithP2PBroadcaster(&mockBroadcaster{}),
 		WithStateNotifier(&mockBeaconNode{}),
 		WithForkChoiceStore(fc),
 		WithAttestationService(attService),
@@ -587,9 +587,7 @@ func (s *MockClockSetter) SetClock(g *startup.Clock) error {
 func TestNotifyIndex(t *testing.T) {
 	// Initialize a blobNotifierMap
 	bn := &blobNotifierMap{
-		// TODO: Separate blobs from data columns
-		// seenIndex: make(map[[32]byte][]bool),
-		seenIndex: make(map[[32]byte][fieldparams.NumberOfColumns]bool),
+		seenIndex: make(map[[32]byte][]bool),
 		notifiers: make(map[[32]byte]chan uint64),
 	}
 

beacon-chain/blockchain/setup_test.go

@@ -20,10 +20,9 @@ import (
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/operations/attestations"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/operations/blstoexec"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p"
-	p2pTesting "github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/testing"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/startup"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/state/stategen"
-	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
 	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
 	"github.com/OffchainLabs/prysm/v6/testing/require"
 	"google.golang.org/protobuf/proto"
@@ -48,11 +47,6 @@ type mockBroadcaster struct {
 	broadcastCalled bool
 }
 
-type mockAccesser struct {
-	mockBroadcaster
-	p2pTesting.MockPeerManager
-}
-
 func (mb *mockBroadcaster) Broadcast(_ context.Context, _ proto.Message) error {
 	mb.broadcastCalled = true
 	return nil
@@ -73,7 +67,12 @@ func (mb *mockBroadcaster) BroadcastBlob(_ context.Context, _ uint64, _ *ethpb.B
 	return nil
 }
 
-func (mb *mockBroadcaster) BroadcastDataColumn(_ context.Context, _ [fieldparams.RootLength]byte, _ uint64, _ *ethpb.DataColumnSidecar) error {
+func (mb *mockBroadcaster) BroadcastLightClientOptimisticUpdate(_ context.Context, _ interfaces.LightClientOptimisticUpdate) error {
+	mb.broadcastCalled = true
+	return nil
+}
+
+func (mb *mockBroadcaster) BroadcastLightClientFinalityUpdate(_ context.Context, _ interfaces.LightClientFinalityUpdate) error {
 	mb.broadcastCalled = true
 	return nil
 }
@@ -133,10 +132,8 @@ func minimalTestService(t *testing.T, opts ...Option) (*Service, *testServiceReq
 		WithDepositCache(dc),
 		WithTrackedValidatorsCache(cache.NewTrackedValidatorsCache()),
 		WithBlobStorage(filesystem.NewEphemeralBlobStorage(t)),
-		WithDataColumnStorage(filesystem.NewEphemeralDataColumnStorage(t)),
 		WithSyncChecker(mock.MockChecker{}),
 		WithExecutionEngineCaller(&mockExecution.EngineClient{}),
-		WithP2PBroadcaster(&mockAccesser{}),
 		WithLightClientStore(&lightclient.Store{}),
 	}
 	// append the variadic opts so they override the defaults by being processed afterwards

beacon-chain/blockchain/testing/mock.go

@@ -53,6 +53,7 @@ type ChainService struct {
 	InitSyncBlockRoots          map[[32]byte]bool
 	DB                          db.Database
 	State                       state.BeaconState
+	HeadStateErr                error
 	Block                       interfaces.ReadOnlySignedBeaconBlock
 	VerifyBlkDescendantErr      error
 	stateNotifier               statefeed.Notifier
@@ -74,7 +75,6 @@ type ChainService struct {
 	BlockSlot                   primitives.Slot
 	SyncingRoot                 [32]byte
 	Blobs                       []blocks.VerifiedROBlob
-	DataColumns                 []blocks.VerifiedRODataColumn
 	TargetRoot                  [32]byte
 	MockHeadSlot                *primitives.Slot
 }
@@ -365,6 +365,9 @@ func (s *ChainService) HeadState(context.Context) (state.BeaconState, error) {
 
 // HeadStateReadOnly mocks HeadStateReadOnly method in chain service.
 func (s *ChainService) HeadStateReadOnly(context.Context) (state.ReadOnlyBeaconState, error) {
+	if s.HeadStateErr != nil {
+		return nil, s.HeadStateErr
+	}
 	return s.State, nil
 }
 
@@ -712,18 +715,18 @@ func (c *ChainService) ReceiveBlob(_ context.Context, b blocks.VerifiedROBlob) e
 	return nil
 }
 
-// ReceiveDataColumn implements the same method in chain service
-func (c *ChainService) ReceiveDataColumn(dc blocks.VerifiedRODataColumn) error {
-	c.DataColumns = append(c.DataColumns, dc)
-	return nil
-}
-
-// ReceiveDataColumns implements the same method in chain service
-func (*ChainService) ReceiveDataColumns(_ []blocks.VerifiedRODataColumn) error {
-	return nil
-}
-
 // TargetRootForEpoch mocks the same method in the chain service
 func (c *ChainService) TargetRootForEpoch(_ [32]byte, _ primitives.Epoch) ([32]byte, error) {
 	return c.TargetRoot, nil
 }
+
+// MockSyncChecker is a mock implementation of blockchain.Checker.
+// We can't make an assertion here that this is true because that would create a circular dependency.
+type MockSyncChecker struct {
+	synced bool
+}
+
+// Synced satisfies the blockchain.Checker interface.
+func (m *MockSyncChecker) Synced() bool {
+	return m.synced
+}

beacon-chain/core/altair/upgrade.go

@@ -67,10 +67,6 @@ func UpgradeToAltair(ctx context.Context, state state.BeaconState) (state.Beacon
 	epoch := time.CurrentEpoch(state)
 
 	numValidators := state.NumValidators()
-	hrs, err := state.HistoricalRoots()
-	if err != nil {
-		return nil, err
-	}
 	s := &ethpb.BeaconStateAltair{
 		GenesisTime:           state.GenesisTime(),
 		GenesisValidatorsRoot: state.GenesisValidatorsRoot(),
@@ -83,7 +79,7 @@ func UpgradeToAltair(ctx context.Context, state state.BeaconState) (state.Beacon
 		LatestBlockHeader:           state.LatestBlockHeader(),
 		BlockRoots:                  state.BlockRoots(),
 		StateRoots:                  state.StateRoots(),
-		HistoricalRoots:             hrs,
+		HistoricalRoots:             state.HistoricalRoots(),
 		Eth1Data:                    state.Eth1Data(),
 		Eth1DataVotes:               state.Eth1DataVotes(),
 		Eth1DepositIndex:            state.Eth1DepositIndex(),

beacon-chain/core/altair/upgrade_test.go

@@ -82,10 +82,8 @@ func TestUpgradeToAltair(t *testing.T) {
 	require.DeepSSZEqual(t, preForkState.LatestBlockHeader(), aState.LatestBlockHeader())
 	require.DeepSSZEqual(t, preForkState.BlockRoots(), aState.BlockRoots())
 	require.DeepSSZEqual(t, preForkState.StateRoots(), aState.StateRoots())
-	r1, err := preForkState.HistoricalRoots()
-	require.NoError(t, err)
-	r2, err := aState.HistoricalRoots()
-	require.NoError(t, err)
+	r1 := preForkState.HistoricalRoots()
+	r2 := aState.HistoricalRoots()
 	require.DeepSSZEqual(t, r1, r2)
 	require.DeepSSZEqual(t, preForkState.Eth1Data(), aState.Eth1Data())
 	require.DeepSSZEqual(t, preForkState.Eth1DataVotes(), aState.Eth1DataVotes())

beacon-chain/core/blocks/payload.go

@@ -230,17 +230,14 @@ func verifyBlobCommitmentCount(slot primitives.Slot, body interfaces.ReadOnlyBea
 	if body.Version() < version.Deneb {
 		return nil
 	}
-
 	kzgs, err := body.BlobKzgCommitments()
 	if err != nil {
 		return err
 	}
-
-	commitmentCount, maxBlobsPerBlock := len(kzgs), params.BeaconConfig().MaxBlobsPerBlock(slot)
-	if commitmentCount > maxBlobsPerBlock {
-		return fmt.Errorf("too many kzg commitments in block: actual count %d - max allowed %d", commitmentCount, maxBlobsPerBlock)
+	maxBlobsPerBlock := params.BeaconConfig().MaxBlobsPerBlock(slot)
+	if len(kzgs) > maxBlobsPerBlock {
+		return fmt.Errorf("too many kzg commitments in block: %d", len(kzgs))
 	}
-
 	return nil
 }
 

beacon-chain/core/blocks/payload_test.go

@@ -926,10 +926,8 @@ func TestVerifyBlobCommitmentCount(t *testing.T) {
 	require.NoError(t, err)
 	require.NoError(t, blocks.VerifyBlobCommitmentCount(rb.Slot(), rb.Body()))
 
-	maxCommitmentsPerBlock := params.BeaconConfig().MaxBlobsPerBlock(rb.Slot())
-
-	b = &ethpb.BeaconBlockDeneb{Body: &ethpb.BeaconBlockBodyDeneb{BlobKzgCommitments: make([][]byte, maxCommitmentsPerBlock+1)}}
+	b = &ethpb.BeaconBlockDeneb{Body: &ethpb.BeaconBlockBodyDeneb{BlobKzgCommitments: make([][]byte, params.BeaconConfig().MaxBlobsPerBlock(rb.Slot())+1)}}
 	rb, err = consensusblocks.NewBeaconBlock(b)
 	require.NoError(t, err)
-	require.ErrorContains(t, fmt.Sprintf("too many kzg commitments in block: actual count %d - max allowed %d", maxCommitmentsPerBlock+1, maxCommitmentsPerBlock), blocks.VerifyBlobCommitmentCount(rb.Slot(), rb.Body()))
+	require.ErrorContains(t, fmt.Sprintf("too many kzg commitments in block: %d", params.BeaconConfig().MaxBlobsPerBlock(rb.Slot())+1), blocks.VerifyBlobCommitmentCount(rb.Slot(), rb.Body()))
 }

beacon-chain/core/blocks/proposer_slashing.go

@@ -16,6 +16,9 @@ import (
 	"google.golang.org/protobuf/proto"
 )
 
+// ErrCouldNotVerifyBlockHeader is returned when a block header's signature cannot be verified.
+var ErrCouldNotVerifyBlockHeader = errors.New("could not verify beacon block header")
+
 type slashValidatorFunc func(
 	ctx context.Context,
 	st state.BeaconState,
@@ -114,7 +117,7 @@ func VerifyProposerSlashing(
 	for _, header := range headers {
 		if err := signing.ComputeDomainVerifySigningRoot(beaconState, pIdx, slots.ToEpoch(hSlot),
 			header.Header, params.BeaconConfig().DomainBeaconProposer, header.Signature); err != nil {
-			return errors.Wrap(err, "could not verify beacon block header")
+			return errors.Wrap(ErrCouldNotVerifyBlockHeader, err.Error())
 		}
 	}
 	return nil

beacon-chain/core/blocks/signature.go

@@ -96,24 +96,6 @@ func VerifyBlockHeaderSignature(beaconState state.BeaconState, header *ethpb.Sig
 	return signing.VerifyBlockHeaderSigningRoot(header.Header, proposerPubKey, header.Signature, domain)
 }
 
-func VerifyBlockHeaderSignatureUsingCurrentFork(beaconState state.BeaconState, header *ethpb.SignedBeaconBlockHeader) error {
-	currentEpoch := slots.ToEpoch(header.Header.Slot)
-	fork, err := forks.Fork(currentEpoch)
-	if err != nil {
-		return err
-	}
-	domain, err := signing.Domain(fork, currentEpoch, params.BeaconConfig().DomainBeaconProposer, beaconState.GenesisValidatorsRoot())
-	if err != nil {
-		return err
-	}
-	proposer, err := beaconState.ValidatorAtIndex(header.Header.ProposerIndex)
-	if err != nil {
-		return err
-	}
-	proposerPubKey := proposer.PublicKey
-	return signing.VerifyBlockHeaderSigningRoot(header.Header, proposerPubKey, header.Signature, domain)
-}
-
 // VerifyBlockSignatureUsingCurrentFork verifies the proposer signature of a beacon block. This differs
 // from the above method by not using fork data from the state and instead retrieving it
 // via the respective epoch.

beacon-chain/core/capella/upgrade.go

@@ -42,10 +42,6 @@ func UpgradeToCapella(state state.BeaconState) (state.BeaconState, error) {
 		return nil, err
 	}
 
-	hrs, err := state.HistoricalRoots()
-	if err != nil {
-		return nil, err
-	}
 	s := &ethpb.BeaconStateCapella{
 		GenesisTime:           state.GenesisTime(),
 		GenesisValidatorsRoot: state.GenesisValidatorsRoot(),
@@ -58,7 +54,7 @@ func UpgradeToCapella(state state.BeaconState) (state.BeaconState, error) {
 		LatestBlockHeader:           state.LatestBlockHeader(),
 		BlockRoots:                  state.BlockRoots(),
 		StateRoots:                  state.StateRoots(),
-		HistoricalRoots:             hrs,
+		HistoricalRoots:             state.HistoricalRoots(),
 		Eth1Data:                    state.Eth1Data(),
 		Eth1DataVotes:               state.Eth1DataVotes(),
 		Eth1DepositIndex:            state.Eth1DepositIndex(),

beacon-chain/core/deneb/upgrade.go

@@ -57,10 +57,6 @@ func UpgradeToDeneb(state state.BeaconState) (state.BeaconState, error) {
 	if err != nil {
 		return nil, err
 	}
-	historicalRoots, err := state.HistoricalRoots()
-	if err != nil {
-		return nil, err
-	}
 
 	s := &ethpb.BeaconStateDeneb{
 		GenesisTime:           state.GenesisTime(),
@@ -74,7 +70,7 @@ func UpgradeToDeneb(state state.BeaconState) (state.BeaconState, error) {
 		LatestBlockHeader:           state.LatestBlockHeader(),
 		BlockRoots:                  state.BlockRoots(),
 		StateRoots:                  state.StateRoots(),
-		HistoricalRoots:             historicalRoots,
+		HistoricalRoots:             state.HistoricalRoots(),
 		Eth1Data:                    state.Eth1Data(),
 		Eth1DataVotes:               state.Eth1DataVotes(),
 		Eth1DepositIndex:            state.Eth1DepositIndex(),

beacon-chain/core/deneb/upgrade_test.go

@@ -47,10 +47,8 @@ func TestUpgradeToDeneb(t *testing.T) {
 	require.NoError(t, err)
 	require.DeepSSZEqual(t, make([]uint64, numValidators), s)
 
-	hr1, err := preForkState.HistoricalRoots()
-	require.NoError(t, err)
-	hr2, err := mSt.HistoricalRoots()
-	require.NoError(t, err)
+	hr1 := preForkState.HistoricalRoots()
+	hr2 := mSt.HistoricalRoots()
 	require.DeepEqual(t, hr1, hr2)
 
 	f := mSt.Fork()

beacon-chain/core/electra/upgrade.go

@@ -170,10 +170,6 @@ func UpgradeToElectra(beaconState state.BeaconState) (state.BeaconState, error)
 	if err != nil {
 		return nil, err
 	}
-	historicalRoots, err := beaconState.HistoricalRoots()
-	if err != nil {
-		return nil, err
-	}
 	excessBlobGas, err := payloadHeader.ExcessBlobGas()
 	if err != nil {
 		return nil, err
@@ -223,7 +219,7 @@ func UpgradeToElectra(beaconState state.BeaconState) (state.BeaconState, error)
 		LatestBlockHeader:           beaconState.LatestBlockHeader(),
 		BlockRoots:                  beaconState.BlockRoots(),
 		StateRoots:                  beaconState.StateRoots(),
-		HistoricalRoots:             historicalRoots,
+		HistoricalRoots:             beaconState.HistoricalRoots(),
 		Eth1Data:                    beaconState.Eth1Data(),
 		Eth1DataVotes:               beaconState.Eth1DataVotes(),
 		Eth1DepositIndex:            beaconState.Eth1DepositIndex(),

beacon-chain/core/electra/upgrade_test.go

@@ -77,10 +77,8 @@ func TestUpgradeToElectra(t *testing.T) {
 	require.NoError(t, err)
 	require.DeepSSZEqual(t, make([]uint64, numValidators), s)
 
-	hr1, err := preForkState.HistoricalRoots()
-	require.NoError(t, err)
-	hr2, err := mSt.HistoricalRoots()
-	require.NoError(t, err)
+	hr1 := preForkState.HistoricalRoots()
+	hr2 := mSt.HistoricalRoots()
 	require.DeepEqual(t, hr1, hr2)
 
 	f := mSt.Fork()

beacon-chain/core/epoch/epoch_processing_test.go

@@ -148,8 +148,7 @@ func TestProcessFinalUpdates_CanProcess(t *testing.T) {
 	assert.DeepNotEqual(t, params.BeaconConfig().ZeroHash[:], mix, "latest RANDAO still zero hashes")
 
 	// Verify historical root accumulator was appended.
-	roots, err := newS.HistoricalRoots()
-	require.NoError(t, err)
+	roots := newS.HistoricalRoots()
 	assert.Equal(t, 1, len(roots), "Unexpected slashed balance")
 	currAtt, err := newS.CurrentEpochAttestations()
 	require.NoError(t, err)
@@ -379,8 +378,7 @@ func TestProcessHistoricalDataUpdate(t *testing.T) {
 				return st
 			},
 			verifier: func(st state.BeaconState) {
-				roots, err := st.HistoricalRoots()
-				require.NoError(t, err)
+				roots := st.HistoricalRoots()
 				require.Equal(t, 0, len(roots))
 			},
 		},
@@ -393,8 +391,7 @@ func TestProcessHistoricalDataUpdate(t *testing.T) {
 				return st
 			},
 			verifier: func(st state.BeaconState) {
-				roots, err := st.HistoricalRoots()
-				require.NoError(t, err)
+				roots := st.HistoricalRoots()
 				require.Equal(t, 1, len(roots))
 
 				b := &ethpb.HistoricalBatch{
@@ -431,8 +428,7 @@ func TestProcessHistoricalDataUpdate(t *testing.T) {
 					StateSummaryRoot: sr[:],
 				}
 				require.DeepEqual(t, b, summaries[0])
-				hrs, err := st.HistoricalRoots()
-				require.NoError(t, err)
+				hrs := st.HistoricalRoots()
 				require.DeepEqual(t, hrs, [][]byte{})
 			},
 		},

beacon-chain/core/execution/upgrade.go

@@ -35,10 +35,6 @@ func UpgradeToBellatrix(state state.BeaconState) (state.BeaconState, error) {
 		return nil, err
 	}
 
-	hrs, err := state.HistoricalRoots()
-	if err != nil {
-		return nil, err
-	}
 	s := &ethpb.BeaconStateBellatrix{
 		GenesisTime:           state.GenesisTime(),
 		GenesisValidatorsRoot: state.GenesisValidatorsRoot(),
@@ -51,7 +47,7 @@ func UpgradeToBellatrix(state state.BeaconState) (state.BeaconState, error) {
 		LatestBlockHeader:           state.LatestBlockHeader(),
 		BlockRoots:                  state.BlockRoots(),
 		StateRoots:                  state.StateRoots(),
-		HistoricalRoots:             hrs,
+		HistoricalRoots:             state.HistoricalRoots(),
 		Eth1Data:                    state.Eth1Data(),
 		Eth1DataVotes:               state.Eth1DataVotes(),
 		Eth1DepositIndex:            state.Eth1DepositIndex(),

beacon-chain/core/execution/upgrade_test.go

@@ -24,10 +24,8 @@ func TestUpgradeToBellatrix(t *testing.T) {
 	require.DeepSSZEqual(t, preForkState.LatestBlockHeader(), mSt.LatestBlockHeader())
 	require.DeepSSZEqual(t, preForkState.BlockRoots(), mSt.BlockRoots())
 	require.DeepSSZEqual(t, preForkState.StateRoots(), mSt.StateRoots())
-	r1, err := preForkState.HistoricalRoots()
-	require.NoError(t, err)
-	r2, err := mSt.HistoricalRoots()
-	require.NoError(t, err)
+	r1 := preForkState.HistoricalRoots()
+	r2 := mSt.HistoricalRoots()
 	require.DeepSSZEqual(t, r1, r2)
 	require.DeepSSZEqual(t, preForkState.Eth1Data(), mSt.Eth1Data())
 	require.DeepSSZEqual(t, preForkState.Eth1DataVotes(), mSt.Eth1DataVotes())

beacon-chain/core/feed/block/events.go

@@ -11,6 +11,8 @@ const (
 
 // ReceivedBlockData is the data sent with ReceivedBlock events.
 type ReceivedBlockData struct {
-	SignedBlock  interfaces.ReadOnlySignedBeaconBlock
-	IsOptimistic bool
+	SignedBlock       interfaces.ReadOnlySignedBeaconBlock
+	CurrDependentRoot [32]byte
+	PrevDependentRoot [32]byte
+	IsOptimistic      bool
 }

beacon-chain/core/feed/operation/events.go

@@ -37,11 +37,8 @@ const (
 	// SingleAttReceived is sent after a single attestation object is received from gossip or rpc
 	SingleAttReceived = 9
 
-	// DataColumnSidecarReceived is sent after a data column sidecar is received from gossip or rpc.
-	DataColumnSidecarReceived = 10
-
 	// BlockGossipReceived is sent after a block has been received from gossip or API that passes validation rules.
-	BlockGossipReceived = 11
+	BlockGossipReceived = 10
 )
 
 // UnAggregatedAttReceivedData is the data sent with UnaggregatedAttReceived events.
@@ -93,11 +90,6 @@ type SingleAttReceivedData struct {
 	Attestation ethpb.Att
 }
 
-// DataColumnSidecarReceivedData is the data sent with DataColumnSidecarReceived events.
-type DataColumnSidecarReceivedData struct {
-	DataColumn *blocks.VerifiedRODataColumn
-}
-
 // BlockGossipReceivedData is the data sent with BlockGossipReceived events.
 type BlockGossipReceivedData struct {
 	// SignedBlock is the block that was received.

beacon-chain/core/feed/state/events.go

@@ -43,6 +43,10 @@ type BlockProcessedData struct {
 	BlockRoot [32]byte
 	// SignedBlock is the physical processed block.
 	SignedBlock interfaces.ReadOnlySignedBeaconBlock
+	// CurrDependentRoot is the current dependent root
+	CurrDependentRoot [32]byte
+	// PrevDependentRoot is the previous dependent root
+	PrevDependentRoot [32]byte
 	// Verified is true if the block's BLS contents have been verified.
 	Verified bool
 	// Optimistic is true if the block is optimistic.

beacon-chain/core/fulu/upgrade.go

@@ -57,10 +57,6 @@ func UpgradeToFulu(beaconState state.BeaconState) (state.BeaconState, error) {
 	if err != nil {
 		return nil, err
 	}
-	historicalRoots, err := beaconState.HistoricalRoots()
-	if err != nil {
-		return nil, err
-	}
 	excessBlobGas, err := payloadHeader.ExcessBlobGas()
 	if err != nil {
 		return nil, err
@@ -118,7 +114,7 @@ func UpgradeToFulu(beaconState state.BeaconState) (state.BeaconState, error) {
 		LatestBlockHeader:           beaconState.LatestBlockHeader(),
 		BlockRoots:                  beaconState.BlockRoots(),
 		StateRoots:                  beaconState.StateRoots(),
-		HistoricalRoots:             historicalRoots,
+		HistoricalRoots:             beaconState.HistoricalRoots(),
 		Eth1Data:                    beaconState.Eth1Data(),
 		Eth1DataVotes:               beaconState.Eth1DataVotes(),
 		Eth1DepositIndex:            beaconState.Eth1DepositIndex(),

beacon-chain/core/fulu/upgrade_test.go

@@ -43,10 +43,8 @@ func TestUpgradeToFulu(t *testing.T) {
 	require.DeepSSZEqual(t, preForkState.BlockRoots(), mSt.BlockRoots())
 	require.DeepSSZEqual(t, preForkState.StateRoots(), mSt.StateRoots())
 
-	hr1, err := preForkState.HistoricalRoots()
-	require.NoError(t, err)
-	hr2, err := mSt.HistoricalRoots()
-	require.NoError(t, err)
+	hr1 := preForkState.HistoricalRoots()
+	hr2 := mSt.HistoricalRoots()
 	require.DeepEqual(t, hr1, hr2)
 
 	require.DeepSSZEqual(t, preForkState.Eth1Data(), mSt.Eth1Data())

beacon-chain/core/helpers/beacon_committee.go

@@ -18,6 +18,7 @@ import (
 	"github.com/OffchainLabs/prysm/v6/crypto/hash"
 	"github.com/OffchainLabs/prysm/v6/encoding/bytesutil"
 	"github.com/OffchainLabs/prysm/v6/math"
+	"github.com/OffchainLabs/prysm/v6/monitoring/tracing/trace"
 	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
 	"github.com/OffchainLabs/prysm/v6/runtime/version"
 	"github.com/OffchainLabs/prysm/v6/time/slots"
@@ -297,6 +298,9 @@ func verifyAssignmentEpoch(epoch primitives.Epoch, state state.BeaconState) erro
 // It verifies the validity of the epoch, then iterates through each slot in the epoch to determine the
 // proposer for that slot and assigns them accordingly.
 func ProposerAssignments(ctx context.Context, state state.BeaconState, epoch primitives.Epoch) (map[primitives.ValidatorIndex][]primitives.Slot, error) {
+	ctx, span := trace.StartSpan(ctx, "helpers.ProposerAssignments")
+	defer span.End()
+
 	// Verify if the epoch is valid for assignment based on the provided state.
 	if err := verifyAssignmentEpoch(epoch, state); err != nil {
 		return nil, err
@@ -345,6 +349,9 @@ func ProposerAssignments(ctx context.Context, state state.BeaconState, epoch pri
 // It retrieves active validator indices, determines the number of committees per slot, and computes
 // assignments for each validator based on their presence in the provided validators slice.
 func CommitteeAssignments(ctx context.Context, state state.BeaconState, epoch primitives.Epoch, validators []primitives.ValidatorIndex) (map[primitives.ValidatorIndex]*CommitteeAssignment, error) {
+	ctx, span := trace.StartSpan(ctx, "helpers.CommitteeAssignments")
+	defer span.End()
+
 	// Verify if the epoch is valid for assignment based on the provided state.
 	if err := verifyAssignmentEpoch(epoch, state); err != nil {
 		return nil, err

beacon-chain/core/helpers/sync_committee_test.go

@@ -78,7 +78,6 @@ func TestIsCurrentEpochSyncCommittee_UsingCommittee(t *testing.T) {
 
 func TestIsCurrentEpochSyncCommittee_DoesNotExist(t *testing.T) {
 	helpers.ClearCache()
-	params.SetupTestConfigCleanup(t)
 
 	validators := make([]*ethpb.Validator, params.BeaconConfig().SyncCommitteeSize)
 	syncCommittee := &ethpb.SyncCommittee{
@@ -265,7 +264,6 @@ func TestCurrentEpochSyncSubcommitteeIndices_UsingCommittee(t *testing.T) {
 }
 
 func TestCurrentEpochSyncSubcommitteeIndices_DoesNotExist(t *testing.T) {
-	params.SetupTestConfigCleanup(t)
 	helpers.ClearCache()
 
 	validators := make([]*ethpb.Validator, params.BeaconConfig().SyncCommitteeSize)

beacon-chain/core/peerdas/BUILD.bazel

@@ -7,7 +7,6 @@ go_library(
         "info.go",
         "metrics.go",
         "p2p_interface.go",
-        "peer_sampling.go",
         "reconstruction.go",
         "util.go",
         "validator.go",
@@ -45,7 +44,6 @@ go_test(
         "das_core_test.go",
         "info_test.go",
         "p2p_interface_test.go",
-        "peer_sampling_test.go",
         "reconstruction_test.go",
         "utils_test.go",
         "validator_test.go",

beacon-chain/core/peerdas/das_core.go

@@ -7,7 +7,6 @@ import (
 	"time"
 
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg"
-	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
 	"github.com/OffchainLabs/prysm/v6/config/params"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
@@ -22,10 +21,11 @@ import (
 var (
 	// Custom errors
 	ErrCustodyGroupTooLarge           = errors.New("custody group too large")
-	errCustodyGroupCountTooLarge      = errors.New("custody group count too large")
+	ErrCustodyGroupCountTooLarge      = errors.New("custody group count too large")
+	ErrMismatchSize                   = errors.New("mismatch in the number of blob KZG commitments and cellsAndProofs")
 	errWrongComputedCustodyGroupCount = errors.New("wrong computed custody group count, should never happen")
 
-	// maxUint256 is the maximum value of a uint256.
+	// maxUint256 is the maximum value of an uint256.
 	maxUint256 = &uint256.Int{math.MaxUint64, math.MaxUint64, math.MaxUint64, math.MaxUint64}
 )
 
@@ -38,18 +38,29 @@ const (
 
 // CustodyGroups computes the custody groups the node should participate in for custody.
 // https://github.com/ethereum/consensus-specs/blob/v1.5.0-beta.5/specs/fulu/das-core.md#get_custody_groups
-func CustodyGroups(nodeId enode.ID, custodyGroupCount uint64) (map[uint64]bool, error) {
+func CustodyGroups(nodeId enode.ID, custodyGroupCount uint64) ([]uint64, error) {
 	numberOfCustodyGroup := params.BeaconConfig().NumberOfCustodyGroups
 
 	// Check if the custody group count is larger than the number of custody groups.
 	if custodyGroupCount > numberOfCustodyGroup {
-		return nil, errCustodyGroupCountTooLarge
+		return nil, ErrCustodyGroupCountTooLarge
+	}
+
+	// Shortcut if all custody groups are needed.
+	if custodyGroupCount == numberOfCustodyGroup {
+		custodyGroups := make([]uint64, 0, numberOfCustodyGroup)
+		for i := range numberOfCustodyGroup {
+			custodyGroups = append(custodyGroups, i)
+		}
+
+		return custodyGroups, nil
 	}
 
-	custodyGroups := make(map[uint64]bool, custodyGroupCount)
 	one := uint256.NewInt(1)
 
-	for currentId := new(uint256.Int).SetBytes(nodeId.Bytes()); uint64(len(custodyGroups)) < custodyGroupCount; currentId.Add(currentId, one) {
+	custodyGroupsMap := make(map[uint64]bool, custodyGroupCount)
+	custodyGroups := make([]uint64, 0, custodyGroupCount)
+	for currentId := new(uint256.Int).SetBytes(nodeId.Bytes()); uint64(len(custodyGroups)) < custodyGroupCount; {
 		// Convert to big endian bytes.
 		currentIdBytesBigEndian := currentId.Bytes32()
 
@@ -60,15 +71,24 @@ func CustodyGroups(nodeId enode.ID, custodyGroupCount uint64) (map[uint64]bool,
 		hashedCurrentId := hash.Hash(currentIdBytesLittleEndian)
 
 		// Get the custody group ID.
-		custodyGroupId := binary.LittleEndian.Uint64(hashedCurrentId[:8]) % numberOfCustodyGroup
+		custodyGroup := binary.LittleEndian.Uint64(hashedCurrentId[:8]) % numberOfCustodyGroup
 
 		// Add the custody group to the map.
-		custodyGroups[custodyGroupId] = true
-
-		// Overflow prevention.
-		if currentId.Cmp(maxUint256) == 0 {
-			currentId = uint256.NewInt(0)
+		if !custodyGroupsMap[custodyGroup] {
+			custodyGroupsMap[custodyGroup] = true
+			custodyGroups = append(custodyGroups, custodyGroup)
 		}
+
+		if currentId.Cmp(maxUint256) == 0 {
+			// Overflow prevention.
+			currentId = uint256.NewInt(0)
+		} else {
+			// Increment the current ID.
+			currentId.Add(currentId, one)
+		}
+
+		// Sort the custody groups.
+		slices.Sort[[]uint64](custodyGroups)
 	}
 
 	// Final check.
@@ -85,8 +105,8 @@ func ComputeColumnsForCustodyGroup(custodyGroup uint64) ([]uint64, error) {
 	beaconConfig := params.BeaconConfig()
 	numberOfCustodyGroup := beaconConfig.NumberOfCustodyGroups
 
-	if custodyGroup > numberOfCustodyGroup {
-		return nil, errCustodyGroupCountTooLarge
+	if custodyGroup >= numberOfCustodyGroup {
+		return nil, ErrCustodyGroupTooLarge
 	}
 
 	numberOfColumns := beaconConfig.NumberOfColumns
@@ -103,10 +123,11 @@ func ComputeColumnsForCustodyGroup(custodyGroup uint64) ([]uint64, error) {
 }
 
 // DataColumnSidecars computes the data column sidecars from the signed block, cells and cell proofs.
+// The returned value contains pointers to function parameters.
+// (If the caller alterates `cellsAndProofs` afterwards, the returned value will be modified as well.)
 // https://github.com/ethereum/consensus-specs/blob/v1.5.0-beta.3/specs/fulu/das-core.md#get_data_column_sidecars
 func DataColumnSidecars(signedBlock interfaces.ReadOnlySignedBeaconBlock, cellsAndProofs []kzg.CellsAndProofs) ([]*ethpb.DataColumnSidecar, error) {
-	start := time.Now()
-	if signedBlock == nil || len(cellsAndProofs) == 0 {
+	if signedBlock == nil || signedBlock.IsNil() || len(cellsAndProofs) == 0 {
 		return nil, nil
 	}
 
@@ -118,7 +139,7 @@ func DataColumnSidecars(signedBlock interfaces.ReadOnlySignedBeaconBlock, cellsA
 	}
 
 	if len(blobKzgCommitments) != len(cellsAndProofs) {
-		return nil, errors.New("mismatch in the number of blob KZG commitments and cellsAndProofs")
+		return nil, ErrMismatchSize
 	}
 
 	signedBlockHeader, err := signedBlock.Header()
@@ -131,9 +152,34 @@ func DataColumnSidecars(signedBlock interfaces.ReadOnlySignedBeaconBlock, cellsA
 		return nil, errors.Wrap(err, "merkle proof ZKG commitments")
 	}
 
+	dataColumnSidecars, err := DataColumnsSidecarsFromItems(signedBlockHeader, blobKzgCommitments, kzgCommitmentsInclusionProof, cellsAndProofs)
+	if err != nil {
+		return nil, errors.Wrap(err, "data column sidecars from items")
+	}
+
+	return dataColumnSidecars, nil
+}
+
+// DataColumnsSidecarsFromItems computes the data column sidecars from the signed block header, the blob KZG commiments,
+// the KZG commitment includion proofs and cells and cell proofs.
+// The returned value contains pointers to function parameters.
+// (If the caller alterates input parameters afterwards, the returned value will be modified as well.)
+func DataColumnsSidecarsFromItems(
+	signedBlockHeader *ethpb.SignedBeaconBlockHeader,
+	blobKzgCommitments [][]byte,
+	kzgCommitmentsInclusionProof [][]byte,
+	cellsAndProofs []kzg.CellsAndProofs,
+) ([]*ethpb.DataColumnSidecar, error) {
+	start := time.Now()
+	if len(blobKzgCommitments) != len(cellsAndProofs) {
+		return nil, ErrMismatchSize
+	}
+
+	numberOfColumns := params.BeaconConfig().NumberOfColumns
+
 	blobsCount := len(cellsAndProofs)
-	sidecars := make([]*ethpb.DataColumnSidecar, 0, fieldparams.NumberOfColumns)
-	for columnIndex := range uint64(fieldparams.NumberOfColumns) {
+	sidecars := make([]*ethpb.DataColumnSidecar, 0, numberOfColumns)
+	for columnIndex := range numberOfColumns {
 		column := make([]kzg.Cell, 0, blobsCount)
 		kzgProofOfColumn := make([]kzg.Proof, 0, blobsCount)
 
@@ -155,8 +201,7 @@ func DataColumnSidecars(signedBlock interfaces.ReadOnlySignedBeaconBlock, cellsA
 
 		kzgProofOfColumnBytes := make([][]byte, 0, blobsCount)
 		for _, kzgProof := range kzgProofOfColumn {
-			copiedProof := kzgProof
-			kzgProofOfColumnBytes = append(kzgProofOfColumnBytes, copiedProof[:])
+			kzgProofOfColumnBytes = append(kzgProofOfColumnBytes, kzgProof[:])
 		}
 
 		sidecar := &ethpb.DataColumnSidecar{
@@ -175,15 +220,30 @@ func DataColumnSidecars(signedBlock interfaces.ReadOnlySignedBeaconBlock, cellsA
 	return sidecars, nil
 }
 
+// ComputeCustodyGroupForColumn computes the custody group for a given column.
+// It is the reciprocal function of ComputeColumnsForCustodyGroup.
+func ComputeCustodyGroupForColumn(columnIndex uint64) (uint64, error) {
+	beaconConfig := params.BeaconConfig()
+	numberOfColumns := beaconConfig.NumberOfColumns
+	numberOfCustodyGroups := beaconConfig.NumberOfCustodyGroups
+
+	if columnIndex >= numberOfColumns {
+		return 0, ErrIndexTooLarge
+	}
+
+	return columnIndex % numberOfCustodyGroups, nil
+}
+
 // Blobs extract blobs from `dataColumnsSidecar`.
 // This can be seen as the reciprocal function of DataColumnSidecars.
 // `dataColumnsSidecar` needs to contain the datacolumns corresponding to the non-extended matrix,
 // else an error will be returned.
 // (`dataColumnsSidecar` can contain extra columns, but they will be ignored.)
 func Blobs(indices map[uint64]bool, dataColumnsSidecar []*ethpb.DataColumnSidecar) ([]*blocks.VerifiedROBlob, error) {
-	columnCount := fieldparams.NumberOfColumns
+	numberOfColumns := params.BeaconConfig().NumberOfColumns
 
-	neededColumnCount := columnCount / 2
+	// Compute the number of needed columns, including the number of columns is odd case.
+	neededColumnCount := (numberOfColumns + 1) / 2
 
 	// Check if all needed columns are present.
 	sliceIndexFromColumnIndex := make(map[uint64]int, len(dataColumnsSidecar))
@@ -191,28 +251,24 @@ func Blobs(indices map[uint64]bool, dataColumnsSidecar []*ethpb.DataColumnSideca
 		dataColumnSideCar := dataColumnsSidecar[i]
 		index := dataColumnSideCar.Index
 
-		if index < uint64(neededColumnCount) {
+		if index < neededColumnCount {
 			sliceIndexFromColumnIndex[index] = i
 		}
 	}
 
-	actualColumnCount := len(sliceIndexFromColumnIndex)
+	actualColumnCount := uint64(len(sliceIndexFromColumnIndex))
 
 	// Get missing columns.
 	if actualColumnCount < neededColumnCount {
-		missingColumns := make(map[int]bool, neededColumnCount-actualColumnCount)
+		var missingColumnsSlice []uint64
+
 		for i := range neededColumnCount {
-			if _, ok := sliceIndexFromColumnIndex[uint64(i)]; !ok {
-				missingColumns[i] = true
+			if _, ok := sliceIndexFromColumnIndex[i]; !ok {
+				missingColumnsSlice = append(missingColumnsSlice, i)
 			}
 		}
 
-		missingColumnsSlice := make([]int, 0, len(missingColumns))
-		for i := range missingColumns {
-			missingColumnsSlice = append(missingColumnsSlice, i)
-		}
-
-		slices.Sort[[]int](missingColumnsSlice)
+		slices.Sort[[]uint64](missingColumnsSlice)
 		return nil, errors.Errorf("some columns are missing: %v", missingColumnsSlice)
 	}
 
@@ -239,7 +295,7 @@ func Blobs(indices map[uint64]bool, dataColumnsSidecar []*ethpb.DataColumnSideca
 
 		// Compute the content of the blob.
 		for columnIndex := range neededColumnCount {
-			sliceIndex, ok := sliceIndexFromColumnIndex[uint64(columnIndex)]
+			sliceIndex, ok := sliceIndexFromColumnIndex[columnIndex]
 			if !ok {
 				return nil, errors.Errorf("missing column %d, this should never happen", columnIndex)
 			}
@@ -247,8 +303,8 @@ func Blobs(indices map[uint64]bool, dataColumnsSidecar []*ethpb.DataColumnSideca
 			dataColumnSideCar := dataColumnsSidecar[sliceIndex]
 			cell := dataColumnSideCar.Column[blobIndex]
 
-			for i := 0; i < len(cell); i++ {
-				blob[columnIndex*kzg.BytesPerCell+i] = cell[i]
+			for i := range cell {
+				blob[columnIndex*kzg.BytesPerCell+uint64(i)] = cell[i]
 			}
 		}
 
@@ -296,14 +352,14 @@ func (custodyInfo *CustodyInfo) CustodyGroupSamplingSize(ct CustodyType) uint64
 }
 
 // CustodyColumns computes the custody columns from the custody groups.
-func CustodyColumns(custodyGroups map[uint64]bool) (map[uint64]bool, error) {
+func CustodyColumns(custodyGroups []uint64) (map[uint64]bool, error) {
 	numberOfCustodyGroups := params.BeaconConfig().NumberOfCustodyGroups
 
 	custodyGroupCount := len(custodyGroups)
 
 	// Compute the columns for each custody group.
 	columns := make(map[uint64]bool, custodyGroupCount)
-	for group := range custodyGroups {
+	for _, group := range custodyGroups {
 		if group >= numberOfCustodyGroups {
 			return nil, ErrCustodyGroupTooLarge
 		}
@@ -332,19 +388,13 @@ func populateAndFilterIndices(indices map[uint64]bool, blobCount uint64) []uint6
 	}
 
 	// Filter blobs index higher than the blob count.
-	filteredIndices := make(map[uint64]bool, len(indices))
+	indicesSlice := make([]uint64, 0, len(indices))
 	for i := range indices {
 		if i < blobCount {
-			filteredIndices[i] = true
+			indicesSlice = append(indicesSlice, i)
 		}
 	}
 
-	// Transform set to slice.
-	indicesSlice := make([]uint64, 0, len(filteredIndices))
-	for i := range filteredIndices {
-		indicesSlice = append(indicesSlice, i)
-	}
-
 	// Sort the indices.
 	slices.Sort[[]uint64](indicesSlice)
 

beacon-chain/core/peerdas/das_core_test.go

@@ -5,34 +5,115 @@ import (
 
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
-	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
+	"github.com/OffchainLabs/prysm/v6/config/params"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
 	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
 	"github.com/OffchainLabs/prysm/v6/testing/require"
 	"github.com/OffchainLabs/prysm/v6/testing/util"
+	"github.com/ethereum/go-ethereum/p2p/enode"
 	"github.com/pkg/errors"
 )
 
-// ---------------------------------------------------------------
-// ( CustodyGroups is unit tested in spec tests.                 )
-// ( ComputeColumnsForCustodyGroup is unit tested in spec tests. )
-// ---------------------------------------------------------------
+func TestCustodyGroups(t *testing.T) {
+	// The happy path is unit tested in spec tests.
+	numberOfCustodyGroup := params.BeaconConfig().NumberOfCustodyGroups
+	_, err := peerdas.CustodyGroups(enode.ID{}, numberOfCustodyGroup+1)
+	require.ErrorIs(t, err, peerdas.ErrCustodyGroupCountTooLarge)
+}
+
+func TestComputeColumnsForCustodyGroup(t *testing.T) {
+	// The happy path is unit tested in spec tests.
+	numberOfCustodyGroup := params.BeaconConfig().NumberOfCustodyGroups
+	_, err := peerdas.ComputeColumnsForCustodyGroup(numberOfCustodyGroup)
+	require.ErrorIs(t, err, peerdas.ErrCustodyGroupTooLarge)
+}
 
 func TestDataColumnSidecars(t *testing.T) {
-	var expected []*ethpb.DataColumnSidecar = nil
-	actual, err := peerdas.DataColumnSidecars(nil, []kzg.CellsAndProofs{})
-	require.NoError(t, err)
+	t.Run("nil signed block", func(t *testing.T) {
+		var expected []*ethpb.DataColumnSidecar = nil
+		actual, err := peerdas.DataColumnSidecars(nil, []kzg.CellsAndProofs{})
+		require.NoError(t, err)
 
-	require.DeepSSZEqual(t, expected, actual)
+		require.DeepSSZEqual(t, expected, actual)
+	})
+
+	t.Run("empty cells and proofs", func(t *testing.T) {
+		// Create a protobuf signed beacon block.
+		signedBeaconBlockPb := util.NewBeaconBlockDeneb()
+
+		// Create a signed beacon block from the protobuf.
+		signedBeaconBlock, err := blocks.NewSignedBeaconBlock(signedBeaconBlockPb)
+		require.NoError(t, err)
+
+		actual, err := peerdas.DataColumnSidecars(signedBeaconBlock, []kzg.CellsAndProofs{})
+		require.NoError(t, err)
+		require.IsNil(t, actual)
+	})
+
+	t.Run("sizes mismatch", func(t *testing.T) {
+		// Create a protobuf signed beacon block.
+		signedBeaconBlockPb := util.NewBeaconBlockDeneb()
+
+		// Create a signed beacon block from the protobuf.
+		signedBeaconBlock, err := blocks.NewSignedBeaconBlock(signedBeaconBlockPb)
+		require.NoError(t, err)
+
+		// Create cells and proofs.
+		cellsAndProofs := make([]kzg.CellsAndProofs, 1)
+
+		_, err = peerdas.DataColumnSidecars(signedBeaconBlock, cellsAndProofs)
+		require.ErrorIs(t, err, peerdas.ErrMismatchSize)
+	})
+}
+
+// --------------------------------------------------------------------------------------------------------------------------------------
+// DataColumnsSidecarsFromItems is tested as part of the DataColumnSidecars tests, in the TestDataColumnsSidecarsBlobsRoundtrip function.
+// --------------------------------------------------------------------------------------------------------------------------------------
+
+func TestComputeCustodyGroupForColumn(t *testing.T) {
+	params.SetupTestConfigCleanup(t)
+	config := params.BeaconConfig()
+	config.NumberOfColumns = 128
+	config.NumberOfCustodyGroups = 64
+	params.OverrideBeaconConfig(config)
+
+	t.Run("index too large", func(t *testing.T) {
+		_, err := peerdas.ComputeCustodyGroupForColumn(1_000_000)
+		require.ErrorIs(t, err, peerdas.ErrIndexTooLarge)
+	})
+
+	t.Run("nominal", func(t *testing.T) {
+		expected := uint64(2)
+		actual, err := peerdas.ComputeCustodyGroupForColumn(2)
+		require.NoError(t, err)
+		require.Equal(t, expected, actual)
+
+		expected = uint64(3)
+		actual, err = peerdas.ComputeCustodyGroupForColumn(3)
+		require.NoError(t, err)
+		require.Equal(t, expected, actual)
+
+		expected = uint64(2)
+		actual, err = peerdas.ComputeCustodyGroupForColumn(66)
+		require.NoError(t, err)
+		require.Equal(t, expected, actual)
+
+		expected = uint64(3)
+		actual, err = peerdas.ComputeCustodyGroupForColumn(67)
+		require.NoError(t, err)
+		require.Equal(t, expected, actual)
+	})
 }
 
 func TestBlobs(t *testing.T) {
 	blobsIndice := map[uint64]bool{}
 
-	almostAllColumns := make([]*ethpb.DataColumnSidecar, 0, fieldparams.NumberOfColumns/2)
-	for i := 2; i < fieldparams.NumberOfColumns/2+2; i++ {
+	numberOfColumns := params.BeaconConfig().NumberOfColumns
+
+	almostAllColumns := make([]*ethpb.DataColumnSidecar, 0, numberOfColumns/2)
+	for i := uint64(2); i < numberOfColumns/2+2; i++ {
 		almostAllColumns = append(almostAllColumns, &ethpb.DataColumnSidecar{
-			Index: uint64(i),
+			Index: i,
 		})
 	}
 
@@ -212,12 +293,12 @@ func TestCustodyGroupSamplingSize(t *testing.T) {
 
 func TestCustodyColumns(t *testing.T) {
 	t.Run("group too large", func(t *testing.T) {
-		_, err := peerdas.CustodyColumns(map[uint64]bool{1_000_000: true})
+		_, err := peerdas.CustodyColumns([]uint64{1_000_000})
 		require.ErrorIs(t, err, peerdas.ErrCustodyGroupTooLarge)
 	})
 
 	t.Run("nominal", func(t *testing.T) {
-		input := map[uint64]bool{1: true, 2: true}
+		input := []uint64{1, 2}
 		expected := map[uint64]bool{1: true, 2: true}
 
 		actual, err := peerdas.CustodyColumns(input)

beacon-chain/core/peerdas/info.go

@@ -91,8 +91,14 @@ func Info(nodeID enode.ID, custodyGroupCount uint64) (*info, bool, error) {
 	// Compute data columns subnets.
 	dataColumnsSubnets := DataColumnSubnets(custodyColumns)
 
+	// Convert the custody groups to a map.
+	custodyGroupsMap := make(map[uint64]bool, len(custodyGroups))
+	for _, group := range custodyGroups {
+		custodyGroupsMap[group] = true
+	}
+
 	result := &info{
-		CustodyGroups:      custodyGroups,
+		CustodyGroups:      custodyGroupsMap,
 		CustodyColumns:     custodyColumns,
 		DataColumnsSubnets: dataColumnsSubnets,
 	}
@@ -111,7 +117,7 @@ func (custodyInfo *CustodyInfo) ActualGroupCount() uint64 {
 // CustodyGroupCount returns the number of groups we should participate in for custody.
 func (tcgc *targetCustodyGroupCount) Get() uint64 {
 	// If subscribed to all subnets, return the number of custody groups.
-	if flags.Get().SubscribeAllColumnSubnets {
+	if flags.Get().SubscribeToAllSubnets {
 		return params.BeaconConfig().NumberOfCustodyGroups
 	}
 
@@ -138,7 +144,7 @@ func (tcgc *targetCustodyGroupCount) SetValidatorsCustodyRequirement(value uint6
 // Get returns the to advertise custody group count.
 func (tacgc *toAdverstiseCustodyGroupCount) Get() uint64 {
 	// If subscribed to all subnets, return the number of custody groups.
-	if flags.Get().SubscribeAllColumnSubnets {
+	if flags.Get().SubscribeToAllSubnets {
 		return params.BeaconConfig().NumberOfCustodyGroups
 	}
 

beacon-chain/core/peerdas/info_test.go

@@ -30,25 +30,25 @@ func TestInfo(t *testing.T) {
 func TestTargetCustodyGroupCount(t *testing.T) {
 	testCases := []struct {
 		name                         string
-		subscribeToAllColumns        bool
+		subscribeToAllSubnets        bool
 		validatorsCustodyRequirement uint64
 		expected                     uint64
 	}{
 		{
 			name:                         "subscribed to all subnets",
-			subscribeToAllColumns:        true,
+			subscribeToAllSubnets:        true,
 			validatorsCustodyRequirement: 100,
 			expected:                     128,
 		},
 		{
 			name:                         "no validators attached",
-			subscribeToAllColumns:        false,
+			subscribeToAllSubnets:        false,
 			validatorsCustodyRequirement: 0,
 			expected:                     4,
 		},
 		{
 			name:                         "some validators attached",
-			subscribeToAllColumns:        false,
+			subscribeToAllSubnets:        false,
 			validatorsCustodyRequirement: 100,
 			expected:                     100,
 		},
@@ -57,7 +57,7 @@ func TestTargetCustodyGroupCount(t *testing.T) {
 	for _, tc := range testCases {
 		t.Run(tc.name, func(t *testing.T) {
 			// Subscribe to all subnets if needed.
-			if tc.subscribeToAllColumns {
+			if tc.subscribeToAllSubnets {
 				resetFlags := flags.Get()
 				gFlags := new(flags.GlobalFlags)
 				gFlags.SubscribeToAllSubnets = true
@@ -82,25 +82,25 @@ func TestTargetCustodyGroupCount(t *testing.T) {
 func TestToAdvertiseCustodyGroupCount(t *testing.T) {
 	testCases := []struct {
 		name                         string
-		subscribeToAllColumns        bool
+		subscribeToAllSubnets        bool
 		toAdvertiseCustodyGroupCount uint64
 		expected                     uint64
 	}{
 		{
 			name:                         "subscribed to all subnets",
-			subscribeToAllColumns:        true,
+			subscribeToAllSubnets:        true,
 			toAdvertiseCustodyGroupCount: 100,
 			expected:                     128,
 		},
 		{
 			name:                         "higher than custody requirement",
-			subscribeToAllColumns:        false,
+			subscribeToAllSubnets:        false,
 			toAdvertiseCustodyGroupCount: 100,
 			expected:                     100,
 		},
 		{
 			name:                         "lower than custody requirement",
-			subscribeToAllColumns:        false,
+			subscribeToAllSubnets:        false,
 			toAdvertiseCustodyGroupCount: 1,
 			expected:                     4,
 		},
@@ -109,10 +109,10 @@ func TestToAdvertiseCustodyGroupCount(t *testing.T) {
 	for _, tc := range testCases {
 		t.Run(tc.name, func(t *testing.T) {
 			// Subscribe to all subnets if needed.
-			if tc.subscribeToAllColumns {
+			if tc.subscribeToAllSubnets {
 				resetFlags := flags.Get()
 				gFlags := new(flags.GlobalFlags)
-				gFlags.SubscribeAllColumnSubnets = true
+				gFlags.SubscribeToAllSubnets = true
 				flags.Init(gFlags)
 				defer flags.Init(resetFlags)
 			}

beacon-chain/core/peerdas/p2p_interface.go

@@ -146,21 +146,6 @@ func DataColumnSubnets(dataColumns map[uint64]bool) map[uint64]bool {
 	return subnets
 }
 
-// ComputeCustodyGroupForColumn computes the custody group for a given column.
-// It is the reciprocal function of ComputeColumnsForCustodyGroup.
-func ComputeCustodyGroupForColumn(columnIndex uint64) (uint64, error) {
-	beaconConfig := params.BeaconConfig()
-	numberOfColumns := beaconConfig.NumberOfColumns
-	numberOfCustodyGroups := beaconConfig.NumberOfCustodyGroups
-
-	if columnIndex >= numberOfColumns {
-		return 0, ErrIndexTooLarge
-	}
-
-	columnsPerGroup := numberOfColumns / numberOfCustodyGroups
-	return columnIndex / columnsPerGroup, nil
-}
-
 // CustodyGroupCountFromRecord extracts the custody group count from an ENR record.
 func CustodyGroupCountFromRecord(record *enr.Record) (uint64, error) {
 	if record == nil {

beacon-chain/core/peerdas/p2p_interface_test.go

@@ -258,36 +258,6 @@ func TestDataColumnSubnets(t *testing.T) {
 	}
 }
 
-func TestComputeCustodyGroupForColumn(t *testing.T) {
-	params.SetupTestConfigCleanup(t)
-	config := params.BeaconConfig()
-	config.NumberOfColumns = 128
-	config.NumberOfCustodyGroups = 64
-	params.OverrideBeaconConfig(config)
-
-	t.Run("index too large", func(t *testing.T) {
-		_, err := peerdas.ComputeCustodyGroupForColumn(1_000_000)
-		require.ErrorIs(t, err, peerdas.ErrIndexTooLarge)
-	})
-
-	t.Run("nominal", func(t *testing.T) {
-		expected := uint64(1)
-		actual, err := peerdas.ComputeCustodyGroupForColumn(2)
-		require.NoError(t, err)
-		require.Equal(t, expected, actual)
-
-		expected = uint64(1)
-		actual, err = peerdas.ComputeCustodyGroupForColumn(3)
-		require.NoError(t, err)
-		require.Equal(t, expected, actual)
-
-		expected = uint64(2)
-		actual, err = peerdas.ComputeCustodyGroupForColumn(4)
-		require.NoError(t, err)
-		require.Equal(t, expected, actual)
-	})
-}
-
 func TestCustodyGroupCountFromRecord(t *testing.T) {
 	t.Run("nil record", func(t *testing.T) {
 		_, err := peerdas.CustodyGroupCountFromRecord(nil)

beacon-chain/core/peerdas/peer_sampling.go

@@ -1,56 +0,0 @@
-package peerdas
-
-import (
-	"math/big"
-
-	"github.com/OffchainLabs/prysm/v6/config/params"
-)
-
-// ExtendedSampleCount computes, for a given number of samples per slot and allowed failures the
-// number of samples we should actually query from peers.
-// https://github.com/ethereum/consensus-specs/blob/v1.5.0-beta.5/specs/fulu/peer-sampling.md#get_extended_sample_count
-func ExtendedSampleCount(samplesPerSlot, allowedFailures uint64) uint64 {
-	// Retrieve the columns count
-	columnsCount := params.BeaconConfig().NumberOfColumns
-
-	// If half of the columns are missing, we are able to reconstruct the data.
-	// If half of the columns + 1 are missing, we are not able to reconstruct the data.
-	// This is the smallest worst case.
-	worstCaseMissing := columnsCount/2 + 1
-
-	// Compute the false positive threshold.
-	falsePositiveThreshold := HypergeomCDF(0, columnsCount, worstCaseMissing, samplesPerSlot)
-
-	var sampleCount uint64
-
-	// Finally, compute the extended sample count.
-	for sampleCount = samplesPerSlot; sampleCount < columnsCount+1; sampleCount++ {
-		if HypergeomCDF(allowedFailures, columnsCount, worstCaseMissing, sampleCount) <= falsePositiveThreshold {
-			break
-		}
-	}
-
-	return sampleCount
-}
-
-// HypergeomCDF computes the hypergeometric cumulative distribution function.
-// https://en.wikipedia.org/wiki/Hypergeometric_distribution
-func HypergeomCDF(k, M, n, N uint64) float64 {
-	denominatorInt := new(big.Int).Binomial(int64(M), int64(N)) // lint:ignore uintcast
-	denominator := new(big.Float).SetInt(denominatorInt)
-
-	rBig := big.NewFloat(0)
-
-	for i := uint64(0); i < k+1; i++ {
-		a := new(big.Int).Binomial(int64(n), int64(i)) // lint:ignore uintcast
-		b := new(big.Int).Binomial(int64(M-n), int64(N-i))
-		numeratorInt := new(big.Int).Mul(a, b)
-		numerator := new(big.Float).SetInt(numeratorInt)
-		item := new(big.Float).Quo(numerator, denominator)
-		rBig.Add(rBig, item)
-	}
-
-	r, _ := rBig.Float64()
-
-	return r
-}

beacon-chain/core/peerdas/peer_sampling_test.go

@@ -1,60 +0,0 @@
-package peerdas_test
-
-import (
-	"testing"
-
-	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
-	"github.com/OffchainLabs/prysm/v6/testing/require"
-)
-
-func TestExtendedSampleCount(t *testing.T) {
-	const samplesPerSlot = 16
-
-	testCases := []struct {
-		name                string
-		allowedMissings     uint64
-		extendedSampleCount uint64
-	}{
-		{name: "allowedMissings=0", allowedMissings: 0, extendedSampleCount: 16},
-		{name: "allowedMissings=1", allowedMissings: 1, extendedSampleCount: 20},
-		{name: "allowedMissings=2", allowedMissings: 2, extendedSampleCount: 24},
-		{name: "allowedMissings=3", allowedMissings: 3, extendedSampleCount: 27},
-		{name: "allowedMissings=4", allowedMissings: 4, extendedSampleCount: 29},
-		{name: "allowedMissings=5", allowedMissings: 5, extendedSampleCount: 32},
-		{name: "allowedMissings=6", allowedMissings: 6, extendedSampleCount: 35},
-		{name: "allowedMissings=7", allowedMissings: 7, extendedSampleCount: 37},
-		{name: "allowedMissings=8", allowedMissings: 8, extendedSampleCount: 40},
-		{name: "allowedMissings=9", allowedMissings: 9, extendedSampleCount: 42},
-		{name: "allowedMissings=10", allowedMissings: 10, extendedSampleCount: 44},
-		{name: "allowedMissings=11", allowedMissings: 11, extendedSampleCount: 47},
-		{name: "allowedMissings=12", allowedMissings: 12, extendedSampleCount: 49},
-		{name: "allowedMissings=13", allowedMissings: 13, extendedSampleCount: 51},
-		{name: "allowedMissings=14", allowedMissings: 14, extendedSampleCount: 53},
-		{name: "allowedMissings=15", allowedMissings: 15, extendedSampleCount: 55},
-		{name: "allowedMissings=16", allowedMissings: 16, extendedSampleCount: 57},
-		{name: "allowedMissings=17", allowedMissings: 17, extendedSampleCount: 59},
-		{name: "allowedMissings=18", allowedMissings: 18, extendedSampleCount: 61},
-		{name: "allowedMissings=19", allowedMissings: 19, extendedSampleCount: 63},
-		{name: "allowedMissings=20", allowedMissings: 20, extendedSampleCount: 65},
-	}
-
-	for _, tc := range testCases {
-		t.Run(tc.name, func(t *testing.T) {
-			result := peerdas.ExtendedSampleCount(samplesPerSlot, tc.allowedMissings)
-			require.Equal(t, tc.extendedSampleCount, result)
-		})
-	}
-}
-
-func TestHypergeomCDF(t *testing.T) {
-	// Test case from https://en.wikipedia.org/wiki/Hypergeometric_distribution
-	// Population size: 1000, number of successes in population: 500, sample size: 10, number of successes in sample: 5
-	// Expected result: 0.072
-	const (
-		expected = 0.0796665913283742
-		margin   = 0.000001
-	)
-
-	actual := peerdas.HypergeomCDF(5, 128, 65, 16)
-	require.Equal(t, true, expected-margin <= actual && actual <= expected+margin)
-}

beacon-chain/core/peerdas/reconstruction.go

@@ -2,7 +2,6 @@ package peerdas
 
 import (
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg"
-	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
 	"github.com/OffchainLabs/prysm/v6/config/params"
 	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
 	"github.com/pkg/errors"
@@ -14,15 +13,11 @@ func CanSelfReconstruct(custodyGroupCount uint64) bool {
 	total := params.BeaconConfig().NumberOfCustodyGroups
 	// If total is odd, then we need total / 2 + 1 columns to reconstruct.
 	// If total is even, then we need total / 2 columns to reconstruct.
-	custodyGroupsNeeded := total/2 + total%2
-	return custodyGroupCount >= custodyGroupsNeeded
+	return custodyGroupCount >= (total+1)/2
 }
 
 // RecoverCellsAndProofs recovers the cells and proofs from the data column sidecars.
-func RecoverCellsAndProofs(
-	dataColumnSideCars []*ethpb.DataColumnSidecar,
-	blockRoot [fieldparams.RootLength]byte,
-) ([]kzg.CellsAndProofs, error) {
+func RecoverCellsAndProofs(dataColumnSideCars []*ethpb.DataColumnSidecar) ([]kzg.CellsAndProofs, error) {
 	var wg errgroup.Group
 
 	dataColumnSideCarsCount := len(dataColumnSideCars)
@@ -79,69 +74,3 @@ func RecoverCellsAndProofs(
 
 	return recoveredCellsAndProofs, nil
 }
-
-// DataColumnSidecarsForReconstruct is a TEMPORARY function until there is an official specification for it.
-// It is scheduled for deletion.
-func DataColumnSidecarsForReconstruct(
-	blobKzgCommitments [][]byte,
-	signedBlockHeader *ethpb.SignedBeaconBlockHeader,
-	kzgCommitmentsInclusionProof [][]byte,
-	cellsAndProofs []kzg.CellsAndProofs,
-) ([]*ethpb.DataColumnSidecar, error) {
-	// Each CellsAndProofs corresponds to a Blob
-	// So we can get the BlobCount by checking the length of CellsAndProofs
-	blobsCount := len(cellsAndProofs)
-	if blobsCount == 0 {
-		return nil, nil
-	}
-
-	// Get the column sidecars.
-	sidecars := make([]*ethpb.DataColumnSidecar, 0, fieldparams.NumberOfColumns)
-	for columnIndex := range fieldparams.NumberOfColumns {
-		column := make([]kzg.Cell, 0, blobsCount)
-		kzgProofOfColumn := make([]kzg.Proof, 0, blobsCount)
-
-		for rowIndex := range blobsCount {
-			cellsForRow := cellsAndProofs[rowIndex].Cells
-			proofsForRow := cellsAndProofs[rowIndex].Proofs
-
-			if len(cellsForRow) != fieldparams.NumberOfColumns {
-				return nil, errors.Errorf("cells don't have the expected size: expected %d - actual %d", fieldparams.NumberOfColumns, len(cellsForRow))
-			}
-
-			cell := cellsForRow[columnIndex]
-			column = append(column, cell)
-
-			if len(proofsForRow) != fieldparams.NumberOfColumns {
-				return nil, errors.Errorf("proofs don't have the expected size: expected %d - actual %d", fieldparams.NumberOfColumns, len(proofsForRow))
-			}
-
-			kzgProof := proofsForRow[columnIndex]
-			kzgProofOfColumn = append(kzgProofOfColumn, kzgProof)
-		}
-
-		columnBytes := make([][]byte, 0, blobsCount)
-		for i := range column {
-			columnBytes = append(columnBytes, column[i][:])
-		}
-
-		kzgProofOfColumnBytes := make([][]byte, 0, blobsCount)
-		for _, kzgProof := range kzgProofOfColumn {
-			copiedProof := kzgProof
-			kzgProofOfColumnBytes = append(kzgProofOfColumnBytes, copiedProof[:])
-		}
-
-		sidecar := &ethpb.DataColumnSidecar{
-			Index:                        uint64(columnIndex),
-			Column:                       columnBytes,
-			KzgCommitments:               blobKzgCommitments,
-			KzgProofs:                    kzgProofOfColumnBytes,
-			SignedBlockHeader:            signedBlockHeader,
-			KzgCommitmentsInclusionProof: kzgCommitmentsInclusionProof,
-		}
-
-		sidecars = append(sidecars, sidecar)
-	}
-
-	return sidecars, nil
-}

beacon-chain/core/peerdas/reconstruction_test.go

@@ -3,14 +3,9 @@ package peerdas_test
 import (
 	"testing"
 
-	"github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
-	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
 	"github.com/OffchainLabs/prysm/v6/config/params"
-	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
-	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
 	"github.com/OffchainLabs/prysm/v6/testing/require"
-	"github.com/OffchainLabs/prysm/v6/testing/util"
 )
 
 func TestCanSelfReconstruct(t *testing.T) {
@@ -60,150 +55,3 @@ func TestCanSelfReconstruct(t *testing.T) {
 		})
 	}
 }
-
-func TestReconstructionRoundTrip(t *testing.T) {
-	params.SetupTestConfigCleanup(t)
-
-	const blobCount = 5
-
-	var blockRoot [fieldparams.RootLength]byte
-
-	signedBeaconBlockPb := util.NewBeaconBlockDeneb()
-	require.NoError(t, kzg.Start())
-
-	// Generate random blobs and their corresponding commitments.
-	var (
-		blobsKzgCommitments [][]byte
-		blobs               []kzg.Blob
-	)
-	for i := range blobCount {
-		blob := getRandBlob(int64(i))
-		commitment, _, err := generateCommitmentAndProof(&blob)
-		require.NoError(t, err)
-
-		blobsKzgCommitments = append(blobsKzgCommitments, commitment[:])
-		blobs = append(blobs, blob)
-	}
-
-	// Generate a signed beacon block.
-	signedBeaconBlockPb.Block.Body.BlobKzgCommitments = blobsKzgCommitments
-	signedBeaconBlock, err := blocks.NewSignedBeaconBlock(signedBeaconBlockPb)
-	require.NoError(t, err)
-
-	// Get the signed beacon block header.
-	signedBeaconBlockHeader, err := signedBeaconBlock.Header()
-	require.NoError(t, err)
-
-	// Convert data columns sidecars from signed block and blobs.
-	cellsAndProofs := util.GenerateCellsAndProofs(t, blobs)
-	dataColumnSidecars, err := peerdas.DataColumnSidecars(signedBeaconBlock, cellsAndProofs)
-	require.NoError(t, err)
-
-	// Create verified RO data columns.
-	verifiedRoDataColumns := make([]*blocks.VerifiedRODataColumn, 0, blobCount)
-	for _, dataColumnSidecar := range dataColumnSidecars {
-		roDataColumn, err := blocks.NewRODataColumn(dataColumnSidecar)
-		require.NoError(t, err)
-
-		verifiedRoDataColumn := blocks.NewVerifiedRODataColumn(roDataColumn)
-		verifiedRoDataColumns = append(verifiedRoDataColumns, &verifiedRoDataColumn)
-	}
-
-	verifiedRoDataColumn := verifiedRoDataColumns[0]
-
-	numberOfColumns := params.BeaconConfig().NumberOfColumns
-
-	var noDataColumns []*ethpb.DataColumnSidecar
-	dataColumnsWithDifferentLengths := []*ethpb.DataColumnSidecar{
-		{Column: [][]byte{{}, {}}},
-		{Column: [][]byte{{}}},
-	}
-	notEnoughDataColumns := dataColumnSidecars[:numberOfColumns/2-1]
-	originalDataColumns := dataColumnSidecars[:numberOfColumns/2]
-	extendedDataColumns := dataColumnSidecars[numberOfColumns/2:]
-	evenDataColumns := make([]*ethpb.DataColumnSidecar, 0, numberOfColumns/2)
-	oddDataColumns := make([]*ethpb.DataColumnSidecar, 0, numberOfColumns/2)
-	allDataColumns := dataColumnSidecars
-
-	for i, dataColumn := range dataColumnSidecars {
-		if i%2 == 0 {
-			evenDataColumns = append(evenDataColumns, dataColumn)
-		} else {
-			oddDataColumns = append(oddDataColumns, dataColumn)
-		}
-	}
-
-	testCases := []struct {
-		name               string
-		dataColumnsSidecar []*ethpb.DataColumnSidecar
-		isError            bool
-	}{
-		{
-			name:               "No data columns sidecars",
-			dataColumnsSidecar: noDataColumns,
-			isError:            true,
-		},
-		{
-			name:               "Data columns sidecar with different lengths",
-			dataColumnsSidecar: dataColumnsWithDifferentLengths,
-			isError:            true,
-		},
-		{
-			name:               "All columns are present (no actual need to reconstruct)",
-			dataColumnsSidecar: allDataColumns,
-			isError:            false,
-		},
-		{
-			name:               "Only original columns are present",
-			dataColumnsSidecar: originalDataColumns,
-			isError:            false,
-		},
-		{
-			name:               "Only extended columns are present",
-			dataColumnsSidecar: extendedDataColumns,
-			isError:            false,
-		},
-		{
-			name:               "Only even columns are present",
-			dataColumnsSidecar: evenDataColumns,
-			isError:            false,
-		},
-		{
-			name:               "Only odd columns are present",
-			dataColumnsSidecar: oddDataColumns,
-			isError:            false,
-		},
-		{
-			name:               "Not enough columns to reconstruct",
-			dataColumnsSidecar: notEnoughDataColumns,
-			isError:            true,
-		},
-	}
-
-	for _, tc := range testCases {
-		t.Run(tc.name, func(t *testing.T) {
-			// Recover cells and proofs from available data columns sidecars.
-			cellsAndProofs, err := peerdas.RecoverCellsAndProofs(tc.dataColumnsSidecar, blockRoot)
-			isError := (err != nil)
-			require.Equal(t, tc.isError, isError)
-
-			if isError {
-				return
-			}
-
-			// Recover all data columns sidecars from cells and proofs.
-			reconstructedDataColumnsSideCars, err := peerdas.DataColumnSidecarsForReconstruct(
-				blobsKzgCommitments,
-				signedBeaconBlockHeader,
-				verifiedRoDataColumn.KzgCommitmentsInclusionProof,
-				cellsAndProofs,
-			)
-
-			require.NoError(t, err)
-
-			expected := dataColumnSidecars
-			actual := reconstructedDataColumnsSideCars
-			require.DeepSSZEqual(t, expected, actual)
-		})
-	}
-}

beacon-chain/core/peerdas/util.go

@@ -8,6 +8,7 @@ import (
 	"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
 	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
 	"github.com/OffchainLabs/prysm/v6/runtime/version"
+	"github.com/pkg/errors"
 )
 
 // ConstructDataColumnSidecars constructs data column sidecars from a block, blobs and their cell proofs.
@@ -18,18 +19,9 @@ func ConstructDataColumnSidecars(block interfaces.SignedBeaconBlock, blobs [][]b
 		return nil, nil
 	}
 
-	cellsAndProofs, err := constructCellsAndProofs(blobs, cellProofs)
-	if err != nil {
-		return nil, err
-	}
-
-	return DataColumnSidecars(block, cellsAndProofs)
-}
-
-func constructCellsAndProofs(blobs [][]byte, cellProofs [][]byte) ([]kzg.CellsAndProofs, error) {
-	numColumns := int(params.BeaconConfig().NumberOfColumns)
-	if len(blobs)*numColumns != len(cellProofs) {
-		return nil, fmt.Errorf("number of blobs and cell proofs do not match: %d * %d != %d", len(blobs), numColumns, len(cellProofs))
+	numberOfColumns := params.BeaconConfig().NumberOfColumns
+	if uint64(len(blobs))*numberOfColumns != uint64(len(cellProofs)) {
+		return nil, fmt.Errorf("number of blobs and cell proofs do not match: %d * %d != %d", len(blobs), numberOfColumns, len(cellProofs))
 	}
 
 	cellsAndProofs := make([]kzg.CellsAndProofs, 0, len(blobs))
@@ -43,7 +35,7 @@ func constructCellsAndProofs(blobs [][]byte, cellProofs [][]byte) ([]kzg.CellsAn
 		}
 
 		var proofs []kzg.Proof
-		for idx := i * numColumns; idx < (i+1)*numColumns; idx++ {
+		for idx := uint64(i) * numberOfColumns; idx < (uint64(i)+1)*numberOfColumns; idx++ {
 			proofs = append(proofs, kzg.Proof(cellProofs[idx]))
 		}
 
@@ -53,5 +45,10 @@ func constructCellsAndProofs(blobs [][]byte, cellProofs [][]byte) ([]kzg.CellsAn
 		})
 	}
 
-	return cellsAndProofs, nil
+	dataColumnSidecars, err := DataColumnSidecars(block, cellsAndProofs)
+	if err != nil {
+		return nil, errors.Wrap(err, "data column sidcars")
+	}
+
+	return dataColumnSidecars, nil
 }

beacon-chain/core/peerdas/validator.go

@@ -12,12 +12,12 @@ import (
 func ValidatorsCustodyRequirement(state beaconState.ReadOnlyBeaconState, validatorsIndex map[primitives.ValidatorIndex]bool) (uint64, error) {
 	totalNodeBalance := uint64(0)
 	for index := range validatorsIndex {
-		balance, err := state.BalanceAtIndex(index)
+		validator, err := state.ValidatorAtIndexReadOnly(index)
 		if err != nil {
-			return 0, errors.Wrapf(err, "balance at index for validator index %v", index)
+			return 0, errors.Wrapf(err, "validator at index %v", index)
 		}
 
-		totalNodeBalance += balance
+		totalNodeBalance += validator.EffectiveBalance()
 	}
 
 	beaconConfig := params.BeaconConfig()

beacon-chain/core/peerdas/validator_test.go

@@ -30,9 +30,13 @@ func TestValidatorsCustodyRequirement(t *testing.T) {
 
 	for _, tc := range testCases {
 		t.Run(tc.name, func(t *testing.T) {
-			balances := make([]uint64, 0, tc.count)
+			validators := make([]*ethpb.Validator, 0, tc.count)
 			for range tc.count {
-				balances = append(balances, balance)
+				validator := &ethpb.Validator{
+					EffectiveBalance: balance,
+				}
+
+				validators = append(validators, validator)
 			}
 
 			validatorsIndex := make(map[primitives.ValidatorIndex]bool)
@@ -40,7 +44,7 @@ func TestValidatorsCustodyRequirement(t *testing.T) {
 				validatorsIndex[primitives.ValidatorIndex(i)] = true
 			}
 
-			beaconState, err := state_native.InitializeFromProtoFulu(&ethpb.BeaconStateElectra{Balances: balances})
+			beaconState, err := state_native.InitializeFromProtoFulu(&ethpb.BeaconStateElectra{Validators: validators})
 			require.NoError(t, err)
 
 			actual, err := peerdas.ValidatorsCustodyRequirement(beaconState, validatorsIndex)

beacon-chain/core/time/slot_epoch.go

@@ -53,11 +53,6 @@ func HigherEqualThanAltairVersionAndEpoch(s state.BeaconState, e primitives.Epoc
 	return s.Version() >= version.Altair && e >= params.BeaconConfig().AltairForkEpoch
 }
 
-// PeerDASIsActive checks whether peerDAS is active at the provided slot.
-func PeerDASIsActive(slot primitives.Slot) bool {
-	return params.FuluEnabled() && slots.ToEpoch(slot) >= params.BeaconConfig().FuluForkEpoch
-}
-
 // CanUpgradeToAltair returns true if the input `slot` can upgrade to Altair.
 // Spec code:
 // If state.slot % SLOTS_PER_EPOCH == 0 and compute_epoch_at_slot(state.slot) == ALTAIR_FORK_EPOCH

beacon-chain/das/BUILD.bazel

@@ -4,26 +4,21 @@ go_library(
     name = "go_default_library",
     srcs = [
         "availability.go",
-        "availability_columns.go",
         "cache.go",
-        "data_column_cache.go",
         "iface.go",
         "mock.go",
     ],
     importpath = "github.com/OffchainLabs/prysm/v6/beacon-chain/das",
     visibility = ["//visibility:public"],
     deps = [
-        "//beacon-chain/core/peerdas:go_default_library",
         "//beacon-chain/db/filesystem:go_default_library",
         "//beacon-chain/verification:go_default_library",
-        "//config/fieldparams:go_default_library",
         "//config/params:go_default_library",
         "//consensus-types/blocks:go_default_library",
         "//consensus-types/primitives:go_default_library",
         "//runtime/logging:go_default_library",
         "//runtime/version:go_default_library",
         "//time/slots:go_default_library",
-        "@com_github_ethereum_go_ethereum//p2p/enode:go_default_library",
         "@com_github_pkg_errors//:go_default_library",
         "@com_github_sirupsen_logrus//:go_default_library",
     ],
@@ -32,18 +27,13 @@ go_library(
 go_test(
     name = "go_default_test",
     srcs = [
-        "availability_columns_test.go",
         "availability_test.go",
         "cache_test.go",
-        "data_column_cache_test.go",
     ],
     embed = [":go_default_library"],
     deps = [
-        "//beacon-chain/core/peerdas:go_default_library",
         "//beacon-chain/db/filesystem:go_default_library",
         "//beacon-chain/verification:go_default_library",
-        "//cmd/beacon-chain/flags:go_default_library",
-        "//config/fieldparams:go_default_library",
         "//config/params:go_default_library",
         "//consensus-types/blocks:go_default_library",
         "//consensus-types/primitives:go_default_library",
@@ -51,7 +41,6 @@ go_test(
         "//testing/require:go_default_library",
         "//testing/util:go_default_library",
         "//time/slots:go_default_library",
-        "@com_github_ethereum_go_ethereum//p2p/enode:go_default_library",
         "@com_github_pkg_errors//:go_default_library",
     ],
 )

beacon-chain/das/availability.go

@@ -53,16 +53,10 @@ func NewLazilyPersistentStore(store *filesystem.BlobStorage, verifier BlobBatchV
 // Persist adds blobs to the working blob cache. Blobs stored in this cache will be persisted
 // for at least as long as the node is running. Once IsDataAvailable succeeds, all blobs referenced
 // by the given block are guaranteed to be persisted for the remainder of the retention period.
-func (s *LazilyPersistentStore) Persist(current primitives.Slot, scg ...blocks.ROSidecar) error {
-	if len(scg) == 0 {
+func (s *LazilyPersistentStore) Persist(current primitives.Slot, sc ...blocks.ROBlob) error {
+	if len(sc) == 0 {
 		return nil
 	}
-
-	sc, err := blocks.BlobSidecarsFromSidecars(scg)
-	if err != nil {
-		return errors.Wrap(err, "blob sidecars from sidecars")
-	}
-
 	if len(sc) > 1 {
 		first := sc[0].BlockRoot()
 		for i := 1; i < len(sc); i++ {

beacon-chain/das/availability_columns.go

@@ -1,200 +0,0 @@
-package das
-
-import (
-	"context"
-
-	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
-	"github.com/OffchainLabs/prysm/v6/beacon-chain/db/filesystem"
-	"github.com/OffchainLabs/prysm/v6/beacon-chain/verification"
-	"github.com/OffchainLabs/prysm/v6/config/params"
-	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
-	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
-	"github.com/OffchainLabs/prysm/v6/runtime/version"
-	"github.com/OffchainLabs/prysm/v6/time/slots"
-	"github.com/ethereum/go-ethereum/p2p/enode"
-	errors "github.com/pkg/errors"
-)
-
-// LazilyPersistentStoreColumn is an implementation of AvailabilityStore to be used when batch syncing data columns.
-// This implementation will hold any data columns passed to Persist until the IsDataAvailable is called for their
-// block, at which time they will undergo full verification and be saved to the disk.
-type LazilyPersistentStoreColumn struct {
-	store                  *filesystem.DataColumnStorage
-	nodeID                 enode.ID
-	cache                  *dataColumnCache
-	custodyInfo            *peerdas.CustodyInfo
-	newDataColumnsVerifier verification.NewDataColumnsVerifier
-}
-
-var _ AvailabilityStore = &LazilyPersistentStoreColumn{}
-
-// DataColumnsVerifier enables LazilyPersistentStoreColumn to manage the verification process
-// going from RODataColumn->VerifiedRODataColumn, while avoiding the decision of which individual verifications
-// to run and in what order. Since LazilyPersistentStoreColumn always tries to verify and save data columns only when
-// they are all available, the interface takes a slice of data column sidecars.
-type DataColumnsVerifier interface {
-	VerifiedRODataColumns(ctx context.Context, blk blocks.ROBlock, scs []blocks.RODataColumn) ([]blocks.VerifiedRODataColumn, error)
-}
-
-func NewLazilyPersistentStoreColumn(store *filesystem.DataColumnStorage, nodeID enode.ID, newDataColumnsVerifier verification.NewDataColumnsVerifier, custodyInfo *peerdas.CustodyInfo) *LazilyPersistentStoreColumn {
-	return &LazilyPersistentStoreColumn{
-		store:                  store,
-		nodeID:                 nodeID,
-		cache:                  newDataColumnCache(),
-		custodyInfo:            custodyInfo,
-		newDataColumnsVerifier: newDataColumnsVerifier,
-	}
-}
-
-// PersistColumns adds columns to the working column cache. Columns stored in this cache will be persisted
-// for at least as long as the node is running. Once IsDataAvailable succeeds, all columns referenced
-// by the given block are guaranteed to be persisted for the remainder of the retention period.
-func (s *LazilyPersistentStoreColumn) Persist(current primitives.Slot, sidecars ...blocks.ROSidecar) error {
-	if len(sidecars) == 0 {
-		return nil
-	}
-
-	dataColumnSidecars, err := blocks.DataColumnSidecarsFromSidecars(sidecars)
-	if err != nil {
-		return errors.Wrap(err, "blob sidecars from sidecars")
-	}
-
-	// It is safe to retrieve the first sidecar.
-	firstSidecar := dataColumnSidecars[0]
-
-	if len(sidecars) > 1 {
-		firstRoot := firstSidecar.BlockRoot()
-		for _, sidecar := range dataColumnSidecars[1:] {
-			if sidecar.BlockRoot() != firstRoot {
-				return errMixedRoots
-			}
-		}
-	}
-
-	firstSidecarEpoch, currentEpoch := slots.ToEpoch(firstSidecar.Slot()), slots.ToEpoch(current)
-	if !params.WithinDAPeriod(firstSidecarEpoch, currentEpoch) {
-		return nil
-	}
-
-	key := dataColumnCacheKey{slot: firstSidecar.Slot(), root: firstSidecar.BlockRoot()}
-	entry := s.cache.ensure(key)
-
-	for i := range sidecars {
-		if err := entry.stash(&dataColumnSidecars[i]); err != nil {
-			return errors.Wrap(err, "stash DataColumnSidecar")
-		}
-	}
-
-	return nil
-}
-
-// IsDataAvailable returns nil if all the commitments in the given block are persisted to the db and have been verified.
-// DataColumnsSidecars already in the db are assumed to have been previously verified against the block.
-func (s *LazilyPersistentStoreColumn) IsDataAvailable(ctx context.Context, currentSlot primitives.Slot, block blocks.ROBlock) error {
-	blockCommitments, err := s.fullCommitmentsToCheck(s.nodeID, block, currentSlot)
-	if err != nil {
-		return errors.Wrapf(err, "full commitments to check with block root `%#x` and current slot `%d`", block.Root(), currentSlot)
-	}
-
-	// Return early for blocks that do not have any commitments.
-	if blockCommitments.count() == 0 {
-		return nil
-	}
-
-	// Get the root of the block.
-	blockRoot := block.Root()
-
-	// Build the cache key for the block.
-	key := dataColumnCacheKey{slot: block.Block().Slot(), root: blockRoot}
-
-	// Retrieve the cache entry for the block, or create an empty one if it doesn't exist.
-	entry := s.cache.ensure(key)
-
-	// Delete the cache entry for the block at the end.
-	defer s.cache.delete(key)
-
-	// Set the disk summary for the block in the cache entry.
-	entry.setDiskSummary(s.store.Summary(blockRoot))
-
-	// Verify we have all the expected sidecars, and fail fast if any are missing or inconsistent.
-	// We don't try to salvage problematic batches because this indicates a misbehaving peer and we'd rather
-	// ignore their response and decrease their peer score.
-	roDataColumns, err := entry.filter(blockRoot, blockCommitments)
-	if err != nil {
-		return errors.Wrap(err, "filter")
-	}
-
-	// https://github.com/ethereum/consensus-specs/blob/dev/specs/fulu/p2p-interface.md#datacolumnsidecarsbyrange-v1
-	verifier := s.newDataColumnsVerifier(roDataColumns, verification.ByRangeRequestDataColumnSidecarRequirements)
-
-	if err := verifier.Valid(); err != nil {
-		return errors.Wrap(err, "valid")
-	}
-
-	if err := verifier.SidecarInclusionProven(); err != nil {
-		return errors.Wrap(err, "sidecar inclusion proven")
-	}
-
-	if err := verifier.SidecarKzgProofVerified(); err != nil {
-		return errors.Wrap(err, "sidecar KZG proof verified")
-	}
-
-	verifiedRoDataColumns, err := verifier.VerifiedRODataColumns()
-	if err != nil {
-		return errors.Wrap(err, "verified RO data columns - should never happen")
-	}
-
-	if err := s.store.Save(verifiedRoDataColumns); err != nil {
-		return errors.Wrap(err, "save data column sidecars")
-	}
-
-	return nil
-}
-
-// fullCommitmentsToCheck returns the commitments to check for a given block.
-func (s *LazilyPersistentStoreColumn) fullCommitmentsToCheck(nodeID enode.ID, block blocks.ROBlock, currentSlot primitives.Slot) (*safeCommitmentsArray, error) {
-	// Return early for blocks that are pre-Fulu.
-	if block.Version() < version.Fulu {
-		return &safeCommitmentsArray{}, nil
-	}
-
-	// Compute the block epoch.
-	blockSlot := block.Block().Slot()
-	blockEpoch := slots.ToEpoch(blockSlot)
-
-	// Compute the current spoch.
-	currentEpoch := slots.ToEpoch(currentSlot)
-
-	// Return early if the request is out of the MIN_EPOCHS_FOR_DATA_COLUMN_SIDECARS_REQUESTS window.
-	if !params.WithinDAPeriod(blockEpoch, currentEpoch) {
-		return &safeCommitmentsArray{}, nil
-	}
-
-	// Retrieve the KZG commitments for the block.
-	kzgCommitments, err := block.Block().Body().BlobKzgCommitments()
-	if err != nil {
-		return nil, errors.Wrap(err, "blob KZG commitments")
-	}
-
-	// Return early if there are no commitments in the block.
-	if len(kzgCommitments) == 0 {
-		return &safeCommitmentsArray{}, nil
-	}
-
-	// Retrieve the groups count.
-	custodyGroupCount := s.custodyInfo.ActualGroupCount()
-
-	// Retrieve peer info.
-	peerInfo, _, err := peerdas.Info(nodeID, custodyGroupCount)
-	if err != nil {
-		return nil, errors.Wrap(err, "peer info")
-	}
-
-	// Create a safe commitments array for the custody columns.
-	commitmentsArray := &safeCommitmentsArray{}
-	for column := range peerInfo.CustodyColumns {
-		commitmentsArray[column] = kzgCommitments
-	}
-
-	return commitmentsArray, nil
-}

beacon-chain/das/availability_columns_test.go

@@ -1,301 +0,0 @@
-package das
-
-import (
-	"context"
-	"testing"
-
-	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
-	"github.com/OffchainLabs/prysm/v6/beacon-chain/db/filesystem"
-	"github.com/OffchainLabs/prysm/v6/beacon-chain/verification"
-	"github.com/OffchainLabs/prysm/v6/cmd/beacon-chain/flags"
-	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
-	"github.com/OffchainLabs/prysm/v6/config/params"
-	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
-	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
-	"github.com/OffchainLabs/prysm/v6/encoding/bytesutil"
-	"github.com/OffchainLabs/prysm/v6/testing/require"
-	"github.com/OffchainLabs/prysm/v6/testing/util"
-	"github.com/OffchainLabs/prysm/v6/time/slots"
-	"github.com/ethereum/go-ethereum/p2p/enode"
-)
-
-func roSidecarsFromDataColumnParamsByBlockRoot(t *testing.T, dataColumnParamsByBlockRoot verification.DataColumnsParamsByRoot) ([]blocks.ROSidecar, []blocks.RODataColumn) {
-	roDataColumns, _ := verification.CreateTestVerifiedRoDataColumnSidecars(t, dataColumnParamsByBlockRoot)
-
-	roSidecars := make([]blocks.ROSidecar, 0, len(roDataColumns))
-	for _, roDataColumn := range roDataColumns {
-		roSidecars = append(roSidecars, blocks.NewSidecarFromDataColumnSidecar(roDataColumn))
-	}
-
-	return roSidecars, roDataColumns
-}
-
-func newSignedRoBlock(t *testing.T, signedBeaconBlock interface{}) blocks.ROBlock {
-	sb, err := blocks.NewSignedBeaconBlock(signedBeaconBlock)
-	require.NoError(t, err)
-
-	rb, err := blocks.NewROBlock(sb)
-	require.NoError(t, err)
-
-	return rb
-}
-
-var commitments = [][]byte{
-	bytesutil.PadTo([]byte("a"), 48),
-	bytesutil.PadTo([]byte("b"), 48),
-	bytesutil.PadTo([]byte("c"), 48),
-	bytesutil.PadTo([]byte("d"), 48),
-}
-
-func TestPersist(t *testing.T) {
-	t.Run("no sidecars", func(t *testing.T) {
-		dataColumnStorage := filesystem.NewEphemeralDataColumnStorage(t)
-		lazilyPersistentStoreColumns := NewLazilyPersistentStoreColumn(dataColumnStorage, enode.ID{}, nil, &peerdas.CustodyInfo{})
-		err := lazilyPersistentStoreColumns.Persist(0)
-		require.NoError(t, err)
-		require.Equal(t, 0, len(lazilyPersistentStoreColumns.cache.entries))
-	})
-
-	t.Run("mixed roots", func(t *testing.T) {
-		dataColumnStorage := filesystem.NewEphemeralDataColumnStorage(t)
-
-		dataColumnParamsByBlockRoot := map[[fieldparams.RootLength]byte][]verification.DataColumnParams{
-			{1}: {{ColumnIndex: 1}},
-			{2}: {{ColumnIndex: 2}},
-		}
-
-		roSidecars, _ := roSidecarsFromDataColumnParamsByBlockRoot(t, dataColumnParamsByBlockRoot)
-		lazilyPersistentStoreColumns := NewLazilyPersistentStoreColumn(dataColumnStorage, enode.ID{}, nil, &peerdas.CustodyInfo{})
-
-		err := lazilyPersistentStoreColumns.Persist(0, roSidecars...)
-		require.ErrorIs(t, err, errMixedRoots)
-		require.Equal(t, 0, len(lazilyPersistentStoreColumns.cache.entries))
-	})
-
-	t.Run("outside DA period", func(t *testing.T) {
-		dataColumnStorage := filesystem.NewEphemeralDataColumnStorage(t)
-
-		dataColumnParamsByBlockRoot := map[[fieldparams.RootLength]byte][]verification.DataColumnParams{
-			{1}: {{ColumnIndex: 1}},
-		}
-
-		roSidecars, _ := roSidecarsFromDataColumnParamsByBlockRoot(t, dataColumnParamsByBlockRoot)
-		lazilyPersistentStoreColumns := NewLazilyPersistentStoreColumn(dataColumnStorage, enode.ID{}, nil, &peerdas.CustodyInfo{})
-
-		err := lazilyPersistentStoreColumns.Persist(1_000_000, roSidecars...)
-		require.NoError(t, err)
-		require.Equal(t, 0, len(lazilyPersistentStoreColumns.cache.entries))
-	})
-
-	t.Run("nominal", func(t *testing.T) {
-		dataColumnStorage := filesystem.NewEphemeralDataColumnStorage(t)
-
-		dataColumnParamsByBlockRoot := map[[fieldparams.RootLength]byte][]verification.DataColumnParams{
-			{}: {{ColumnIndex: 1}, {ColumnIndex: 5}},
-		}
-
-		roSidecars, roDataColumns := roSidecarsFromDataColumnParamsByBlockRoot(t, dataColumnParamsByBlockRoot)
-		lazilyPersistentStoreColumns := NewLazilyPersistentStoreColumn(dataColumnStorage, enode.ID{}, nil, &peerdas.CustodyInfo{})
-
-		err := lazilyPersistentStoreColumns.Persist(0, roSidecars...)
-		require.NoError(t, err)
-		require.Equal(t, 1, len(lazilyPersistentStoreColumns.cache.entries))
-
-		key := dataColumnCacheKey{slot: 0, root: [32]byte{}}
-		entry := lazilyPersistentStoreColumns.cache.entries[key]
-
-		// A call to Persist does NOT save the sidecars to disk.
-		require.Equal(t, uint64(0), entry.diskSummary.Count())
-
-		require.DeepSSZEqual(t, roDataColumns[0], *entry.scs[1])
-		require.DeepSSZEqual(t, roDataColumns[1], *entry.scs[5])
-
-		for i, roDataColumn := range entry.scs {
-			if map[int]bool{1: true, 5: true}[i] {
-				continue
-			}
-
-			require.IsNil(t, roDataColumn)
-		}
-	})
-}
-
-func TestIsDataAvailable(t *testing.T) {
-	newDataColumnsVerifier := func(dataColumnSidecars []blocks.RODataColumn, _ []verification.Requirement) verification.DataColumnsVerifier {
-		return &mockDataColumnsVerifier{t: t, dataColumnSidecars: dataColumnSidecars}
-	}
-
-	ctx := context.Background()
-
-	t.Run("without commitments", func(t *testing.T) {
-		signedBeaconBlockFulu := util.NewBeaconBlockFulu()
-		signedRoBlock := newSignedRoBlock(t, signedBeaconBlockFulu)
-
-		dataColumnStorage := filesystem.NewEphemeralDataColumnStorage(t)
-		lazilyPersistentStoreColumns := NewLazilyPersistentStoreColumn(dataColumnStorage, enode.ID{}, newDataColumnsVerifier, &peerdas.CustodyInfo{})
-
-		err := lazilyPersistentStoreColumns.IsDataAvailable(ctx, 0 /*current slot*/, signedRoBlock)
-		require.NoError(t, err)
-	})
-
-	t.Run("with commitments", func(t *testing.T) {
-		signedBeaconBlockFulu := util.NewBeaconBlockFulu()
-		signedBeaconBlockFulu.Block.Body.BlobKzgCommitments = commitments
-		signedRoBlock := newSignedRoBlock(t, signedBeaconBlockFulu)
-		root := signedRoBlock.Root()
-
-		dataColumnStorage := filesystem.NewEphemeralDataColumnStorage(t)
-		lazilyPersistentStoreColumns := NewLazilyPersistentStoreColumn(dataColumnStorage, enode.ID{}, newDataColumnsVerifier, &peerdas.CustodyInfo{})
-
-		indices := [...]uint64{1, 17, 87, 102}
-		dataColumnsParams := make([]verification.DataColumnParams, 0, len(indices))
-		for _, index := range indices {
-			dataColumnParams := verification.DataColumnParams{
-				ColumnIndex:    index,
-				KzgCommitments: commitments,
-			}
-
-			dataColumnsParams = append(dataColumnsParams, dataColumnParams)
-		}
-
-		dataColumnsParamsByBlockRoot := verification.DataColumnsParamsByRoot{root: dataColumnsParams}
-		_, verifiedRoDataColumns := verification.CreateTestVerifiedRoDataColumnSidecars(t, dataColumnsParamsByBlockRoot)
-
-		key := dataColumnCacheKey{root: root}
-		entry := lazilyPersistentStoreColumns.cache.ensure(key)
-		defer lazilyPersistentStoreColumns.cache.delete(key)
-
-		for _, verifiedRoDataColumn := range verifiedRoDataColumns {
-			err := entry.stash(&verifiedRoDataColumn.RODataColumn)
-			require.NoError(t, err)
-		}
-
-		err := lazilyPersistentStoreColumns.IsDataAvailable(ctx, 0 /*current slot*/, signedRoBlock)
-		require.NoError(t, err)
-
-		actual, err := dataColumnStorage.Get(root, indices[:])
-		require.NoError(t, err)
-
-		summary := dataColumnStorage.Summary(root)
-		require.Equal(t, uint64(len(indices)), summary.Count())
-		require.DeepSSZEqual(t, verifiedRoDataColumns, actual)
-	})
-}
-
-func TestFullCommitmentsToCheck(t *testing.T) {
-	windowSlots, err := slots.EpochEnd(params.BeaconConfig().MinEpochsForDataColumnSidecarsRequest)
-	require.NoError(t, err)
-
-	testCases := []struct {
-		name        string
-		commitments [][]byte
-		block       func(*testing.T) blocks.ROBlock
-		slot        primitives.Slot
-	}{
-		{
-			name: "Pre-Fulu block",
-			block: func(t *testing.T) blocks.ROBlock {
-				return newSignedRoBlock(t, util.NewBeaconBlockElectra())
-			},
-		},
-		{
-			name: "Commitments outside data availability window",
-			block: func(t *testing.T) blocks.ROBlock {
-				beaconBlockElectra := util.NewBeaconBlockElectra()
-
-				// Block is from slot 0, "current slot" is window size +1 (so outside the window)
-				beaconBlockElectra.Block.Body.BlobKzgCommitments = commitments
-
-				return newSignedRoBlock(t, beaconBlockElectra)
-			},
-			slot: windowSlots + 1,
-		},
-		{
-			name: "Commitments within data availability window",
-			block: func(t *testing.T) blocks.ROBlock {
-				signedBeaconBlockFulu := util.NewBeaconBlockFulu()
-				signedBeaconBlockFulu.Block.Body.BlobKzgCommitments = commitments
-				signedBeaconBlockFulu.Block.Slot = 100
-
-				return newSignedRoBlock(t, signedBeaconBlockFulu)
-			},
-			commitments: commitments,
-			slot:        100,
-		},
-	}
-
-	for _, tc := range testCases {
-		t.Run(tc.name, func(t *testing.T) {
-			resetFlags := flags.Get()
-			gFlags := new(flags.GlobalFlags)
-			gFlags.SubscribeAllColumnSubnets = true
-			flags.Init(gFlags)
-			defer flags.Init(resetFlags)
-
-			b := tc.block(t)
-			s := NewLazilyPersistentStoreColumn(nil, enode.ID{}, nil, &peerdas.CustodyInfo{})
-
-			commitmentsArray, err := s.fullCommitmentsToCheck(enode.ID{}, b, tc.slot)
-			require.NoError(t, err)
-
-			for _, commitments := range commitmentsArray {
-				require.DeepEqual(t, tc.commitments, commitments)
-			}
-		})
-	}
-}
-
-type mockDataColumnsVerifier struct {
-	t                                                                        *testing.T
-	dataColumnSidecars                                                       []blocks.RODataColumn
-	validCalled, SidecarInclusionProvenCalled, SidecarKzgProofVerifiedCalled bool
-}
-
-var _ verification.DataColumnsVerifier = &mockDataColumnsVerifier{}
-
-func (m *mockDataColumnsVerifier) VerifiedRODataColumns() ([]blocks.VerifiedRODataColumn, error) {
-	require.Equal(m.t, true, m.validCalled && m.SidecarInclusionProvenCalled && m.SidecarKzgProofVerifiedCalled)
-
-	verifiedDataColumnSidecars := make([]blocks.VerifiedRODataColumn, 0, len(m.dataColumnSidecars))
-	for _, dataColumnSidecar := range m.dataColumnSidecars {
-		verifiedDataColumnSidecar := blocks.NewVerifiedRODataColumn(dataColumnSidecar)
-		verifiedDataColumnSidecars = append(verifiedDataColumnSidecars, verifiedDataColumnSidecar)
-	}
-
-	return verifiedDataColumnSidecars, nil
-}
-
-func (m *mockDataColumnsVerifier) SatisfyRequirement(verification.Requirement) {}
-
-func (m *mockDataColumnsVerifier) Valid() error {
-	m.validCalled = true
-	return nil
-}
-
-func (m *mockDataColumnsVerifier) CorrectSubnet(expectedTopics []string) error      { return nil }
-func (m *mockDataColumnsVerifier) NotFromFutureSlot() error                         { return nil }
-func (m *mockDataColumnsVerifier) SlotAboveFinalized() error                        { return nil }
-func (m *mockDataColumnsVerifier) ValidProposerSignature(ctx context.Context) error { return nil }
-
-func (m *mockDataColumnsVerifier) SidecarParentSeen(parentSeen func([fieldparams.RootLength]byte) bool) error {
-	return nil
-}
-
-func (m *mockDataColumnsVerifier) SidecarParentValid(badParent func([fieldparams.RootLength]byte) bool) error {
-	return nil
-}
-
-func (m *mockDataColumnsVerifier) SidecarParentSlotLower() error       { return nil }
-func (m *mockDataColumnsVerifier) SidecarDescendsFromFinalized() error { return nil }
-
-func (m *mockDataColumnsVerifier) SidecarInclusionProven() error {
-	m.SidecarInclusionProvenCalled = true
-	return nil
-}
-
-func (m *mockDataColumnsVerifier) SidecarKzgProofVerified() error {
-	m.SidecarKzgProofVerifiedCalled = true
-	return nil
-}
-
-func (m *mockDataColumnsVerifier) SidecarProposerExpected(ctx context.Context) error { return nil }

beacon-chain/das/availability_test.go

@@ -116,11 +116,9 @@ func TestLazilyPersistent_Missing(t *testing.T) {
 	ctx := context.Background()
 	store := filesystem.NewEphemeralBlobStorage(t)
 
-	blk, blobSidecars := util.GenerateTestDenebBlockWithSidecar(t, [32]byte{}, 1, 3)
+	blk, scs := util.GenerateTestDenebBlockWithSidecar(t, [32]byte{}, 1, 3)
 
-	scs := blocks.NewSidecarsFromBlobSidecars(blobSidecars)
-
-	mbv := &mockBlobBatchVerifier{t: t, scs: blobSidecars}
+	mbv := &mockBlobBatchVerifier{t: t, scs: scs}
 	as := NewLazilyPersistentStore(store, mbv)
 
 	// Only one commitment persisted, should return error with other indices
@@ -143,14 +141,12 @@ func TestLazilyPersistent_Mismatch(t *testing.T) {
 	ctx := context.Background()
 	store := filesystem.NewEphemeralBlobStorage(t)
 
-	blk, blobSidecars := util.GenerateTestDenebBlockWithSidecar(t, [32]byte{}, 1, 3)
+	blk, scs := util.GenerateTestDenebBlockWithSidecar(t, [32]byte{}, 1, 3)
 
 	mbv := &mockBlobBatchVerifier{t: t, err: errors.New("kzg check should not run")}
-	blobSidecars[0].KzgCommitment = bytesutil.PadTo([]byte("nope"), 48)
+	scs[0].KzgCommitment = bytesutil.PadTo([]byte("nope"), 48)
 	as := NewLazilyPersistentStore(store, mbv)
 
-	scs := blocks.NewSidecarsFromBlobSidecars(blobSidecars)
-
 	// Only one commitment persisted, should return error with other indices
 	require.NoError(t, as.Persist(1, scs[0]))
 	err := as.IsDataAvailable(ctx, 1, blk)
@@ -159,10 +155,7 @@ func TestLazilyPersistent_Mismatch(t *testing.T) {
 }
 
 func TestLazyPersistOnceCommitted(t *testing.T) {
-	_, blobSidecars := util.GenerateTestDenebBlockWithSidecar(t, [32]byte{}, 1, 6)
-
-	scs := blocks.NewSidecarsFromBlobSidecars(blobSidecars)
-
+	_, scs := util.GenerateTestDenebBlockWithSidecar(t, [32]byte{}, 1, 6)
 	as := NewLazilyPersistentStore(filesystem.NewEphemeralBlobStorage(t), &mockBlobBatchVerifier{})
 	// stashes as expected
 	require.NoError(t, as.Persist(1, scs...))
@@ -170,13 +163,10 @@ func TestLazyPersistOnceCommitted(t *testing.T) {
 	require.ErrorIs(t, as.Persist(1, scs...), ErrDuplicateSidecar)
 
 	// ignores index out of bound
-	blobSidecars[0].Index = 6
-	require.ErrorIs(t, as.Persist(1, blocks.NewSidecarFromBlobSidecar(blobSidecars[0])), errIndexOutOfBounds)
-
-	_, moreBlobSidecars := util.GenerateTestDenebBlockWithSidecar(t, [32]byte{}, 1, 4)
-
-	more := blocks.NewSidecarsFromBlobSidecars(moreBlobSidecars)
+	scs[0].Index = 6
+	require.ErrorIs(t, as.Persist(1, scs[0]), errIndexOutOfBounds)
 
+	_, more := util.GenerateTestDenebBlockWithSidecar(t, [32]byte{}, 1, 4)
 	// ignores sidecars before the retention period
 	slotOOB, err := slots.EpochStart(params.BeaconConfig().MinEpochsForBlobsSidecarsRequest)
 	require.NoError(t, err)

beacon-chain/das/cache.go

@@ -10,7 +10,12 @@ import (
 	"github.com/pkg/errors"
 )
 
-var errIndexOutOfBounds = errors.New("sidecar.index > MAX_BLOBS_PER_BLOCK")
+var (
+	ErrDuplicateSidecar   = errors.New("duplicate sidecar stashed in AvailabilityStore")
+	errIndexOutOfBounds   = errors.New("sidecar.index > MAX_BLOBS_PER_BLOCK")
+	errCommitmentMismatch = errors.New("KzgCommitment of sidecar in cache did not match block commitment")
+	errMissingSidecar     = errors.New("no sidecar in cache for block commitment")
+)
 
 // cacheKey includes the slot so that we can easily iterate through the cache and compare
 // slots for eviction purposes. Whether the input is the block or the sidecar, we always have

beacon-chain/das/data_column_cache.go

@@ -1,135 +0,0 @@
-package das
-
-import (
-	"reflect"
-
-	"github.com/OffchainLabs/prysm/v6/beacon-chain/db/filesystem"
-	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
-	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
-	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
-	"github.com/pkg/errors"
-)
-
-var (
-	ErrDuplicateSidecar   = errors.New("duplicate sidecar stashed in AvailabilityStore")
-	errColumnIndexTooHigh = errors.New("column index too high")
-	errCommitmentMismatch = errors.New("KzgCommitment of sidecar in cache did not match block commitment")
-	errMissingSidecar     = errors.New("no sidecar in cache for block commitment")
-)
-
-// dataColumnCacheKey includes the slot so that we can easily iterate through the cache and compare
-// slots for eviction purposes. Whether the input is the block or the sidecar, we always have
-// the root+slot when interacting with the cache, so it isn't an inconvenience to use both.
-type dataColumnCacheKey struct {
-	slot primitives.Slot
-	root [32]byte
-}
-
-type dataColumnCache struct {
-	entries map[dataColumnCacheKey]*dataColumnCacheEntry
-}
-
-func newDataColumnCache() *dataColumnCache {
-	return &dataColumnCache{entries: make(map[dataColumnCacheKey]*dataColumnCacheEntry)}
-}
-
-// ensure returns the entry for the given key, creating it if it isn't already present.
-func (c *dataColumnCache) ensure(key dataColumnCacheKey) *dataColumnCacheEntry {
-	entry, ok := c.entries[key]
-	if !ok {
-		entry = &dataColumnCacheEntry{}
-		c.entries[key] = entry
-	}
-
-	return entry
-}
-
-// delete removes the cache entry from the cache.
-func (c *dataColumnCache) delete(key dataColumnCacheKey) {
-	delete(c.entries, key)
-}
-
-// dataColumnCacheEntry holds a fixed-length cache of BlobSidecars.
-type dataColumnCacheEntry struct {
-	scs         [fieldparams.NumberOfColumns]*blocks.RODataColumn
-	diskSummary filesystem.DataColumnStorageSummary
-}
-
-func (e *dataColumnCacheEntry) setDiskSummary(sum filesystem.DataColumnStorageSummary) {
-	e.diskSummary = sum
-}
-
-// stash adds an item to the in-memory cache of DataColumnSidecars.
-// Only the first DataColumnSidecar of a given Index will be kept in the cache.
-// stash will return an error if the given data colunn is already in the cache, or if the Index is out of bounds.
-func (e *dataColumnCacheEntry) stash(sc *blocks.RODataColumn) error {
-	if sc.Index >= fieldparams.NumberOfColumns {
-		return errors.Wrapf(errColumnIndexTooHigh, "index=%d", sc.Index)
-	}
-
-	if e.scs[sc.Index] != nil {
-		return errors.Wrapf(ErrDuplicateSidecar, "root=%#x, index=%d, commitment=%#x", sc.BlockRoot(), sc.Index, sc.KzgCommitments)
-	}
-
-	e.scs[sc.Index] = sc
-
-	return nil
-}
-
-func (e *dataColumnCacheEntry) filter(root [32]byte, commitmentsArray *safeCommitmentsArray) ([]blocks.RODataColumn, error) {
-	nonEmptyIndices := commitmentsArray.nonEmptyIndices()
-	if e.diskSummary.AllAvailable(nonEmptyIndices) {
-		return nil, nil
-	}
-
-	commitmentsCount := commitmentsArray.count()
-	sidecars := make([]blocks.RODataColumn, 0, commitmentsCount)
-
-	for i := range nonEmptyIndices {
-		if e.diskSummary.HasIndex(i) {
-			continue
-		}
-
-		if e.scs[i] == nil {
-			return nil, errors.Wrapf(errMissingSidecar, "root=%#x, index=%#x", root, i)
-		}
-
-		if !reflect.DeepEqual(commitmentsArray[i], e.scs[i].KzgCommitments) {
-			return nil, errors.Wrapf(errCommitmentMismatch, "root=%#x, index=%#x, commitment=%#x, block commitment=%#x", root, i, e.scs[i].KzgCommitments, commitmentsArray[i])
-		}
-
-		sidecars = append(sidecars, *e.scs[i])
-	}
-
-	return sidecars, nil
-}
-
-// safeCommitmentsArray is a fixed size array of commitments.
-// This is helpful for avoiding gratuitous bounds checks.
-type safeCommitmentsArray [fieldparams.NumberOfColumns][][]byte
-
-// count returns the number of commitments in the array.
-func (s *safeCommitmentsArray) count() int {
-	count := 0
-
-	for i := range s {
-		if s[i] != nil {
-			count++
-		}
-	}
-
-	return count
-}
-
-// nonEmptyIndices returns a map of indices that are non-nil in the array.
-func (s *safeCommitmentsArray) nonEmptyIndices() map[uint64]bool {
-	columns := make(map[uint64]bool)
-
-	for i := range s {
-		if s[i] != nil {
-			columns[uint64(i)] = true
-		}
-	}
-
-	return columns
-}

beacon-chain/das/data_column_cache_test.go

@@ -1,124 +0,0 @@
-package das
-
-import (
-	"testing"
-
-	"github.com/OffchainLabs/prysm/v6/beacon-chain/db/filesystem"
-	"github.com/OffchainLabs/prysm/v6/beacon-chain/verification"
-	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
-	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
-	"github.com/OffchainLabs/prysm/v6/testing/require"
-)
-
-func TestEnsureDeleteSetDiskSummary(t *testing.T) {
-	c := newDataColumnCache()
-	key := dataColumnCacheKey{}
-	entry := c.ensure(key)
-	require.DeepEqual(t, dataColumnCacheEntry{}, *entry)
-
-	diskSummary := filesystem.NewDataColumnStorageSummary(42, [fieldparams.NumberOfColumns]bool{true})
-	entry.setDiskSummary(diskSummary)
-	entry = c.ensure(key)
-	require.DeepEqual(t, dataColumnCacheEntry{diskSummary: diskSummary}, *entry)
-
-	c.delete(key)
-	entry = c.ensure(key)
-	require.DeepEqual(t, dataColumnCacheEntry{}, *entry)
-}
-
-func TestStash(t *testing.T) {
-	t.Run("Index too high", func(t *testing.T) {
-		dataColumnParamsByBlockRoot := verification.DataColumnsParamsByRoot{{1}: {{ColumnIndex: 10_000}}}
-		roDataColumns, _ := verification.CreateTestVerifiedRoDataColumnSidecars(t, dataColumnParamsByBlockRoot)
-
-		var entry dataColumnCacheEntry
-		err := entry.stash(&roDataColumns[0])
-		require.NotNil(t, err)
-	})
-
-	t.Run("Nominal and already existing", func(t *testing.T) {
-		dataColumnParamsByBlockRoot := verification.DataColumnsParamsByRoot{{1}: {{ColumnIndex: 1}}}
-		roDataColumns, _ := verification.CreateTestVerifiedRoDataColumnSidecars(t, dataColumnParamsByBlockRoot)
-
-		var entry dataColumnCacheEntry
-		err := entry.stash(&roDataColumns[0])
-		require.NoError(t, err)
-
-		require.DeepEqual(t, roDataColumns[0], entry.scs[1])
-
-		err = entry.stash(&roDataColumns[0])
-		require.NotNil(t, err)
-	})
-}
-
-func TestFilterDataColumns(t *testing.T) {
-	t.Run("All available", func(t *testing.T) {
-		commitmentsArray := safeCommitmentsArray{nil, [][]byte{[]byte{1}}, nil, [][]byte{[]byte{3}}}
-
-		diskSummary := filesystem.NewDataColumnStorageSummary(42, [fieldparams.NumberOfColumns]bool{false, true, false, true})
-
-		dataColumnCacheEntry := dataColumnCacheEntry{diskSummary: diskSummary}
-
-		actual, err := dataColumnCacheEntry.filter([fieldparams.RootLength]byte{}, &commitmentsArray)
-		require.NoError(t, err)
-		require.IsNil(t, actual)
-	})
-
-	t.Run("Some scs missing", func(t *testing.T) {
-		commitmentsArray := safeCommitmentsArray{nil, [][]byte{[]byte{1}}}
-
-		diskSummary := filesystem.NewDataColumnStorageSummary(42, [fieldparams.NumberOfColumns]bool{})
-
-		dataColumnCacheEntry := dataColumnCacheEntry{diskSummary: diskSummary}
-
-		_, err := dataColumnCacheEntry.filter([fieldparams.RootLength]byte{}, &commitmentsArray)
-		require.NotNil(t, err)
-	})
-
-	t.Run("Commitments not equal", func(t *testing.T) {
-		root := [fieldparams.RootLength]byte{}
-		commitmentsArray := safeCommitmentsArray{nil, [][]byte{[]byte{1}}}
-
-		dataColumnParamsByBlockRoot := verification.DataColumnsParamsByRoot{root: {{ColumnIndex: 1}}}
-		roDataColumns, _ := verification.CreateTestVerifiedRoDataColumnSidecars(t, dataColumnParamsByBlockRoot)
-
-		var scs [fieldparams.NumberOfColumns]*blocks.RODataColumn
-		scs[1] = &roDataColumns[0]
-
-		dataColumnCacheEntry := dataColumnCacheEntry{scs: scs}
-
-		_, err := dataColumnCacheEntry.filter(root, &commitmentsArray)
-		require.NotNil(t, err)
-	})
-
-	t.Run("Nominal", func(t *testing.T) {
-		root := [fieldparams.RootLength]byte{}
-		commitmentsArray := safeCommitmentsArray{nil, [][]byte{[]byte{1}}, nil, [][]byte{[]byte{3}}}
-
-		diskSummary := filesystem.NewDataColumnStorageSummary(42, [fieldparams.NumberOfColumns]bool{false, true})
-
-		dataColumnParamsByBlockRoot := verification.DataColumnsParamsByRoot{root: {{ColumnIndex: 3, KzgCommitments: [][]byte{[]byte{3}}}}}
-		expected, _ := verification.CreateTestVerifiedRoDataColumnSidecars(t, dataColumnParamsByBlockRoot)
-
-		var scs [fieldparams.NumberOfColumns]*blocks.RODataColumn
-		scs[3] = &expected[0]
-
-		dataColumnCacheEntry := dataColumnCacheEntry{scs: scs, diskSummary: diskSummary}
-
-		actual, err := dataColumnCacheEntry.filter(root, &commitmentsArray)
-		require.NoError(t, err)
-
-		require.DeepEqual(t, expected, actual)
-	})
-}
-
-func TestCount(t *testing.T) {
-	s := safeCommitmentsArray{nil, [][]byte{[]byte{1}}, nil, [][]byte{[]byte{3}}}
-	require.Equal(t, 2, s.count())
-}
-
-func TestNonEmptyIndices(t *testing.T) {
-	s := safeCommitmentsArray{nil, [][]byte{[]byte{10}}, nil, [][]byte{[]byte{20}}}
-	actual := s.nonEmptyIndices()
-	require.DeepEqual(t, map[uint64]bool{1: true, 3: true}, actual)
-}

beacon-chain/das/iface.go

@@ -15,5 +15,5 @@ import (
 // durably persisted before returning a non-error value.
 type AvailabilityStore interface {
 	IsDataAvailable(ctx context.Context, current primitives.Slot, b blocks.ROBlock) error
-	Persist(current primitives.Slot, sc ...blocks.ROSidecar) error
+	Persist(current primitives.Slot, sc ...blocks.ROBlob) error
 }

beacon-chain/das/mock.go

@@ -5,7 +5,6 @@ import (
 
 	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
-	errors "github.com/pkg/errors"
 )
 
 // MockAvailabilityStore is an implementation of AvailabilityStore that can be used by other packages in tests.
@@ -25,13 +24,9 @@ func (m *MockAvailabilityStore) IsDataAvailable(ctx context.Context, current pri
 }
 
 // Persist satisfies the corresponding method of the AvailabilityStore interface in a way that is useful for tests.
-func (m *MockAvailabilityStore) Persist(current primitives.Slot, sc ...blocks.ROSidecar) error {
-	blobSidecars, err := blocks.BlobSidecarsFromSidecars(sc)
-	if err != nil {
-		return errors.Wrap(err, "blob sidecars from sidecars")
-	}
+func (m *MockAvailabilityStore) Persist(current primitives.Slot, sc ...blocks.ROBlob) error {
 	if m.PersistBlobsCallback != nil {
-		return m.PersistBlobsCallback(current, blobSidecars...)
+		return m.PersistBlobsCallback(current, sc...)
 	}
 	return nil
 }

beacon-chain/db/filesystem/BUILD.bazel

@@ -5,9 +5,6 @@ go_library(
     srcs = [
         "blob.go",
         "cache.go",
-        "data_column.go",
-        "data_column_cache.go",
-        "doc.go",
         "iteration.go",
         "layout.go",
         "layout_by_epoch.go",
@@ -20,8 +17,6 @@ go_library(
     importpath = "github.com/OffchainLabs/prysm/v6/beacon-chain/db/filesystem",
     visibility = ["//visibility:public"],
     deps = [
-        "//async:go_default_library",
-        "//async/event:go_default_library",
         "//beacon-chain/db:go_default_library",
         "//beacon-chain/verification:go_default_library",
         "//config/fieldparams:go_default_library",
@@ -30,7 +25,6 @@ go_library(
         "//consensus-types/primitives:go_default_library",
         "//encoding/bytesutil:go_default_library",
         "//io/file:go_default_library",
-        "//proto/prysm/v1alpha1:go_default_library",
         "//runtime/logging:go_default_library",
         "//time/slots:go_default_library",
         "@com_github_ethereum_go_ethereum//common/hexutil:go_default_library",
@@ -47,8 +41,6 @@ go_test(
     srcs = [
         "blob_test.go",
         "cache_test.go",
-        "data_column_cache_test.go",
-        "data_column_test.go",
         "iteration_test.go",
         "layout_test.go",
         "migration_test.go",
@@ -58,7 +50,6 @@ go_test(
     deps = [
         "//beacon-chain/db:go_default_library",
         "//beacon-chain/verification:go_default_library",
-        "//config/fieldparams:go_default_library",
         "//config/params:go_default_library",
         "//consensus-types/primitives:go_default_library",
         "//encoding/bytesutil:go_default_library",

beacon-chain/db/filesystem/data_column_cache.go

@@ -1,219 +0,0 @@
-package filesystem
-
-import (
-	"sync"
-
-	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
-	"github.com/OffchainLabs/prysm/v6/config/params"
-	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
-	"github.com/pkg/errors"
-)
-
-var errDataColumnIndexOutOfBounds = errors.New("data column index too high")
-
-// DataColumnStorageSummary represents cached information about the DataColumnSidecars on disk for each root the cache knows about.
-type DataColumnStorageSummary struct {
-	epoch primitives.Epoch
-	mask  [fieldparams.NumberOfColumns]bool
-}
-
-// NewDataColumnStorageSummary creates a new DataColumnStorageSummary for a given epoch and mask.
-func NewDataColumnStorageSummary(epoch primitives.Epoch, mask [fieldparams.NumberOfColumns]bool) DataColumnStorageSummary {
-	return DataColumnStorageSummary{
-		epoch: epoch,
-		mask:  mask,
-	}
-}
-
-// HasIndex returns true if the DataColumnSidecar at the given index is available in the filesystem.
-func (s DataColumnStorageSummary) HasIndex(index uint64) bool {
-	if index >= uint64(fieldparams.NumberOfColumns) {
-		return false
-	}
-	return s.mask[index]
-}
-
-// Count returns the number of available data columns.
-func (s DataColumnStorageSummary) Count() uint64 {
-	count := uint64(0)
-
-	for _, available := range s.mask {
-		if available {
-			count++
-		}
-	}
-
-	return count
-}
-
-// AllAvailable returns true if we have all data columns for corresponding indices.
-func (s DataColumnStorageSummary) AllAvailable(indices map[uint64]bool) bool {
-	if len(indices) > len(s.mask) {
-		return false
-	}
-
-	for index := range indices {
-		if !s.mask[index] {
-			return false
-		}
-	}
-
-	return true
-}
-
-// DataColumnStorageSummarizer can be used to receive a summary of metadata about data columns on disk for a given root.
-// The DataColumnStorageSummary can be used to check which indices (if any) are available for a given block by root.
-type DataColumnStorageSummarizer interface {
-	Summary(root [fieldparams.RootLength]byte) DataColumnStorageSummary
-}
-
-type dataColumnStorageSummaryCache struct {
-	mu                 sync.RWMutex
-	dataColumnCount    float64
-	lowestCachedEpoch  primitives.Epoch
-	highestCachedEpoch primitives.Epoch
-	cache              map[[fieldparams.RootLength]byte]DataColumnStorageSummary
-}
-
-var _ DataColumnStorageSummarizer = &dataColumnStorageSummaryCache{}
-
-func newDataColumnStorageSummaryCache() *dataColumnStorageSummaryCache {
-	return &dataColumnStorageSummaryCache{
-		cache:             make(map[[fieldparams.RootLength]byte]DataColumnStorageSummary),
-		lowestCachedEpoch: params.BeaconConfig().FarFutureEpoch,
-	}
-}
-
-// Summary returns the DataColumnStorageSummary for `root`.
-// The DataColumnStorageSummary can be used to check for the presence of DataColumnSidecars based on Index.
-func (sc *dataColumnStorageSummaryCache) Summary(root [fieldparams.RootLength]byte) DataColumnStorageSummary {
-	sc.mu.RLock()
-	defer sc.mu.RUnlock()
-
-	return sc.cache[root]
-}
-
-func (sc *dataColumnStorageSummaryCache) HighestEpoch() primitives.Epoch {
-	sc.mu.RLock()
-	defer sc.mu.RUnlock()
-	return sc.highestCachedEpoch
-}
-
-// set updates the cache.
-func (sc *dataColumnStorageSummaryCache) set(dataColumnsIdent DataColumnsIdent) error {
-	numberOfColumns := params.BeaconConfig().NumberOfColumns
-
-	sc.mu.Lock()
-	defer sc.mu.Unlock()
-
-	summary := sc.cache[dataColumnsIdent.Root]
-	summary.epoch = dataColumnsIdent.Epoch
-
-	count := uint64(0)
-	for _, index := range dataColumnsIdent.Indices {
-		if index >= numberOfColumns {
-			return errDataColumnIndexOutOfBounds
-		}
-
-		if summary.mask[index] {
-			continue
-		}
-
-		count++
-
-		summary.mask[index] = true
-		sc.cache[dataColumnsIdent.Root] = summary
-		sc.lowestCachedEpoch = min(sc.lowestCachedEpoch, dataColumnsIdent.Epoch)
-		sc.highestCachedEpoch = max(sc.highestCachedEpoch, dataColumnsIdent.Epoch)
-	}
-
-	countFloat := float64(count)
-	sc.dataColumnCount += countFloat
-	dataColumnDiskCount.Set(sc.dataColumnCount)
-	dataColumnWrittenCounter.Add(countFloat)
-
-	return nil
-}
-
-// get returns the DataColumnStorageSummary for the given block root.
-// If the root is not in the cache, the second return value will be false.
-func (sc *dataColumnStorageSummaryCache) get(blockRoot [fieldparams.RootLength]byte) (DataColumnStorageSummary, bool) {
-	sc.mu.RLock()
-	defer sc.mu.RUnlock()
-
-	v, ok := sc.cache[blockRoot]
-	return v, ok
-}
-
-// evict removes the DataColumnStorageSummary for the given block root from the cache.
-func (s *dataColumnStorageSummaryCache) evict(blockRoot [fieldparams.RootLength]byte) int {
-	deleted := 0
-
-	s.mu.Lock()
-	defer s.mu.Unlock()
-
-	summary, ok := s.cache[blockRoot]
-	if !ok {
-		return 0
-	}
-
-	for i := range summary.mask {
-		if summary.mask[i] {
-			deleted += 1
-		}
-	}
-
-	delete(s.cache, blockRoot)
-	if deleted > 0 {
-		s.dataColumnCount -= float64(deleted)
-		dataColumnDiskCount.Set(s.dataColumnCount)
-	}
-
-	// The lowest and highest cached epoch may no longer be valid here,
-	// but is not worth the effort to recalculate.
-
-	return deleted
-}
-
-// pruneUpTo removes all entries from the cache up to the given target epoch included.
-func (sc *dataColumnStorageSummaryCache) pruneUpTo(targetEpoch primitives.Epoch) uint64 {
-	sc.mu.Lock()
-	defer sc.mu.Unlock()
-
-	prunedCount := uint64(0)
-	newLowestCachedEpoch := params.BeaconConfig().FarFutureEpoch
-	newHighestCachedEpoch := primitives.Epoch(0)
-
-	for blockRoot, summary := range sc.cache {
-		epoch := summary.epoch
-
-		if epoch > targetEpoch {
-			newLowestCachedEpoch = min(newLowestCachedEpoch, epoch)
-			newHighestCachedEpoch = max(newHighestCachedEpoch, epoch)
-		}
-
-		if epoch <= targetEpoch {
-			for i := range summary.mask {
-				if summary.mask[i] {
-					prunedCount += 1
-				}
-			}
-
-			delete(sc.cache, blockRoot)
-		}
-	}
-
-	if prunedCount > 0 {
-		sc.lowestCachedEpoch = newLowestCachedEpoch
-		sc.highestCachedEpoch = newHighestCachedEpoch
-		sc.dataColumnCount -= float64(prunedCount)
-		dataColumnDiskCount.Set(sc.dataColumnCount)
-	}
-
-	return prunedCount
-}
-
-// clear removes all entries from the cache.
-func (sc *dataColumnStorageSummaryCache) clear() uint64 {
-	return sc.pruneUpTo(params.BeaconConfig().FarFutureEpoch)
-}

beacon-chain/db/filesystem/data_column_cache_test.go

@@ -1,235 +0,0 @@
-package filesystem
-
-import (
-	"testing"
-
-	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
-	"github.com/OffchainLabs/prysm/v6/config/params"
-	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
-	"github.com/OffchainLabs/prysm/v6/testing/require"
-)
-
-func TestHasIndex(t *testing.T) {
-	summary := NewDataColumnStorageSummary(0, [fieldparams.NumberOfColumns]bool{false, true})
-
-	hasIndex := summary.HasIndex(1_000_000)
-	require.Equal(t, false, hasIndex)
-
-	hasIndex = summary.HasIndex(0)
-	require.Equal(t, false, hasIndex)
-
-	hasIndex = summary.HasIndex(1)
-	require.Equal(t, true, hasIndex)
-}
-
-func TestCount(t *testing.T) {
-	summary := NewDataColumnStorageSummary(0, [fieldparams.NumberOfColumns]bool{false, true, false, true})
-
-	count := summary.Count()
-	require.Equal(t, uint64(2), count)
-}
-
-func TestAllAvailableDataColumns(t *testing.T) {
-	const count = uint64(1_000)
-
-	summary := NewDataColumnStorageSummary(0, [fieldparams.NumberOfColumns]bool{false, true, false, true})
-
-	indices := make(map[uint64]bool, count)
-	for i := range count {
-		indices[i] = true
-	}
-
-	allAvailable := summary.AllAvailable(indices)
-	require.Equal(t, false, allAvailable)
-
-	indices = map[uint64]bool{1: true, 2: true}
-	allAvailable = summary.AllAvailable(indices)
-	require.Equal(t, false, allAvailable)
-
-	indices = map[uint64]bool{1: true, 3: true}
-	allAvailable = summary.AllAvailable(indices)
-	require.Equal(t, true, allAvailable)
-}
-
-func TestSummary(t *testing.T) {
-	root := [fieldparams.RootLength]byte{}
-
-	summaryCache := newDataColumnStorageSummaryCache()
-	expected := NewDataColumnStorageSummary(0, [fieldparams.NumberOfColumns]bool{})
-	actual := summaryCache.Summary(root)
-	require.DeepEqual(t, expected, actual)
-
-	summaryCache = newDataColumnStorageSummaryCache()
-	expected = NewDataColumnStorageSummary(0, [fieldparams.NumberOfColumns]bool{true, false, true, false})
-	summaryCache.cache[root] = expected
-	actual = summaryCache.Summary(root)
-	require.DeepEqual(t, expected, actual)
-}
-
-func TestHighestEpoch(t *testing.T) {
-	root1 := [fieldparams.RootLength]byte{1}
-	root2 := [fieldparams.RootLength]byte{2}
-	root3 := [fieldparams.RootLength]byte{3}
-
-	summaryCache := newDataColumnStorageSummaryCache()
-	actual := summaryCache.HighestEpoch()
-	require.Equal(t, primitives.Epoch(0), actual)
-
-	err := summaryCache.set(DataColumnsIdent{Root: root1, Epoch: 42, Indices: []uint64{1, 3}})
-	require.NoError(t, err)
-	require.Equal(t, primitives.Epoch(42), summaryCache.HighestEpoch())
-
-	err = summaryCache.set(DataColumnsIdent{Root: root2, Epoch: 43, Indices: []uint64{1, 3}})
-	require.NoError(t, err)
-	require.Equal(t, primitives.Epoch(43), summaryCache.HighestEpoch())
-
-	err = summaryCache.set(DataColumnsIdent{Root: root3, Epoch: 40, Indices: []uint64{1, 3}})
-	require.NoError(t, err)
-	require.Equal(t, primitives.Epoch(43), summaryCache.HighestEpoch())
-}
-
-func TestSet(t *testing.T) {
-	t.Run("Index out of bounds", func(t *testing.T) {
-		summaryCache := newDataColumnStorageSummaryCache()
-		err := summaryCache.set(DataColumnsIdent{Indices: []uint64{1_000_000}})
-		require.ErrorIs(t, err, errDataColumnIndexOutOfBounds)
-		require.Equal(t, params.BeaconConfig().FarFutureEpoch, summaryCache.lowestCachedEpoch)
-		require.Equal(t, 0, len(summaryCache.cache))
-	})
-
-	t.Run("Nominal", func(t *testing.T) {
-		root1 := [fieldparams.RootLength]byte{1}
-		root2 := [fieldparams.RootLength]byte{2}
-
-		summaryCache := newDataColumnStorageSummaryCache()
-
-		err := summaryCache.set(DataColumnsIdent{Root: root1, Epoch: 42, Indices: []uint64{1, 3}})
-		require.NoError(t, err)
-		require.Equal(t, primitives.Epoch(42), summaryCache.lowestCachedEpoch)
-		require.Equal(t, 1, len(summaryCache.cache))
-		expected := DataColumnStorageSummary{epoch: 42, mask: [fieldparams.NumberOfColumns]bool{false, true, false, true}}
-		actual := summaryCache.cache[root1]
-		require.DeepEqual(t, expected, actual)
-
-		err = summaryCache.set(DataColumnsIdent{Root: root1, Epoch: 42, Indices: []uint64{0, 1}})
-		require.NoError(t, err)
-		require.Equal(t, primitives.Epoch(42), summaryCache.lowestCachedEpoch)
-		require.Equal(t, 1, len(summaryCache.cache))
-		expected = DataColumnStorageSummary{epoch: 42, mask: [fieldparams.NumberOfColumns]bool{true, true, false, true}}
-		actual = summaryCache.cache[root1]
-		require.DeepEqual(t, expected, actual)
-
-		err = summaryCache.set(DataColumnsIdent{Root: root2, Epoch: 43, Indices: []uint64{1}})
-		require.NoError(t, err)
-		require.Equal(t, primitives.Epoch(42), summaryCache.lowestCachedEpoch) // Epoch 42 is still the lowest
-		require.Equal(t, 2, len(summaryCache.cache))
-		expected = DataColumnStorageSummary{epoch: 43, mask: [fieldparams.NumberOfColumns]bool{false, true}}
-		actual = summaryCache.cache[root2]
-		require.DeepEqual(t, expected, actual)
-	})
-}
-
-func TestGet(t *testing.T) {
-	t.Run("Not in cache", func(t *testing.T) {
-		summaryCache := newDataColumnStorageSummaryCache()
-		root := [fieldparams.RootLength]byte{}
-		_, ok := summaryCache.get(root)
-		require.Equal(t, false, ok)
-	})
-
-	t.Run("In cache", func(t *testing.T) {
-		root := [fieldparams.RootLength]byte{}
-		summaryCache := newDataColumnStorageSummaryCache()
-		summaryCache.cache[root] = NewDataColumnStorageSummary(42, [fieldparams.NumberOfColumns]bool{true, false, true, false})
-		actual, ok := summaryCache.get(root)
-		require.Equal(t, true, ok)
-		expected := NewDataColumnStorageSummary(42, [fieldparams.NumberOfColumns]bool{true, false, true, false})
-		require.DeepEqual(t, expected, actual)
-	})
-}
-
-func TestEvict(t *testing.T) {
-	t.Run("No eviction", func(t *testing.T) {
-		root := [fieldparams.RootLength]byte{}
-		summaryCache := newDataColumnStorageSummaryCache()
-
-		evicted := summaryCache.evict(root)
-		require.Equal(t, 0, evicted)
-	})
-
-	t.Run("Eviction", func(t *testing.T) {
-		root1 := [fieldparams.RootLength]byte{1}
-		root2 := [fieldparams.RootLength]byte{2}
-		summaryCache := newDataColumnStorageSummaryCache()
-		summaryCache.cache[root1] = NewDataColumnStorageSummary(42, [fieldparams.NumberOfColumns]bool{true, false, true, false})
-		summaryCache.cache[root2] = NewDataColumnStorageSummary(43, [fieldparams.NumberOfColumns]bool{false, true, false, true})
-
-		evicted := summaryCache.evict(root1)
-		require.Equal(t, 2, evicted)
-		require.Equal(t, 1, len(summaryCache.cache))
-
-		_, ok := summaryCache.cache[root1]
-		require.Equal(t, false, ok)
-
-		_, ok = summaryCache.cache[root2]
-		require.Equal(t, true, ok)
-	})
-}
-
-func TestPruneUpTo(t *testing.T) {
-	t.Run("No pruning", func(t *testing.T) {
-		summaryCache := newDataColumnStorageSummaryCache()
-		err := summaryCache.set(DataColumnsIdent{Root: [fieldparams.RootLength]byte{1}, Epoch: 42, Indices: []uint64{1}})
-		require.NoError(t, err)
-
-		err = summaryCache.set(DataColumnsIdent{Root: [fieldparams.RootLength]byte{2}, Epoch: 43, Indices: []uint64{2, 4}})
-		require.NoError(t, err)
-
-		count := summaryCache.pruneUpTo(41)
-		require.Equal(t, uint64(0), count)
-		require.Equal(t, 2, len(summaryCache.cache))
-		require.Equal(t, primitives.Epoch(42), summaryCache.lowestCachedEpoch)
-	})
-
-	t.Run("Pruning", func(t *testing.T) {
-		summaryCache := newDataColumnStorageSummaryCache()
-		err := summaryCache.set(DataColumnsIdent{Root: [fieldparams.RootLength]byte{1}, Epoch: 42, Indices: []uint64{1}})
-		require.NoError(t, err)
-
-		err = summaryCache.set(DataColumnsIdent{Root: [fieldparams.RootLength]byte{2}, Epoch: 44, Indices: []uint64{2, 4}})
-		require.NoError(t, err)
-
-		err = summaryCache.set(DataColumnsIdent{Root: [fieldparams.RootLength]byte{3}, Epoch: 45, Indices: []uint64{2, 4}})
-		require.NoError(t, err)
-
-		count := summaryCache.pruneUpTo(42)
-		require.Equal(t, uint64(1), count)
-		require.Equal(t, 2, len(summaryCache.cache))
-		require.Equal(t, primitives.Epoch(44), summaryCache.lowestCachedEpoch)
-
-		count = summaryCache.pruneUpTo(45)
-		require.Equal(t, uint64(4), count)
-		require.Equal(t, 0, len(summaryCache.cache))
-		require.Equal(t, params.BeaconConfig().FarFutureEpoch, summaryCache.lowestCachedEpoch)
-		require.Equal(t, primitives.Epoch(0), summaryCache.highestCachedEpoch)
-
-	})
-
-	t.Run("Clear", func(t *testing.T) {
-		summaryCache := newDataColumnStorageSummaryCache()
-		err := summaryCache.set(DataColumnsIdent{Root: [fieldparams.RootLength]byte{1}, Epoch: 42, Indices: []uint64{1}})
-		require.NoError(t, err)
-
-		err = summaryCache.set(DataColumnsIdent{Root: [fieldparams.RootLength]byte{2}, Epoch: 44, Indices: []uint64{2, 4}})
-		require.NoError(t, err)
-
-		err = summaryCache.set(DataColumnsIdent{Root: [fieldparams.RootLength]byte{3}, Epoch: 45, Indices: []uint64{2, 4}})
-		require.NoError(t, err)
-
-		count := summaryCache.clear()
-		require.Equal(t, uint64(5), count)
-		require.Equal(t, 0, len(summaryCache.cache))
-		require.Equal(t, params.BeaconConfig().FarFutureEpoch, summaryCache.lowestCachedEpoch)
-		require.Equal(t, primitives.Epoch(0), summaryCache.highestCachedEpoch)
-	})
-}

beacon-chain/db/filesystem/data_column_test.go

@@ -1,742 +0,0 @@
-package filesystem
-
-import (
-	"context"
-	"encoding/binary"
-	"os"
-	"testing"
-
-	"github.com/OffchainLabs/prysm/v6/beacon-chain/verification"
-	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
-	"github.com/OffchainLabs/prysm/v6/config/params"
-	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
-	"github.com/OffchainLabs/prysm/v6/testing/require"
-	"github.com/spf13/afero"
-)
-
-func TestNewDataColumnStorage(t *testing.T) {
-	ctx := context.Background()
-
-	t.Run("No base path", func(t *testing.T) {
-		_, err := NewDataColumnStorage(ctx)
-		require.ErrorIs(t, err, errNoBasePath)
-	})
-
-	t.Run("Nominal", func(t *testing.T) {
-		dir := t.TempDir()
-
-		storage, err := NewDataColumnStorage(ctx, WithDataColumnBasePath(dir))
-		require.NoError(t, err)
-		require.Equal(t, dir, storage.base)
-	})
-}
-
-func TestWarmCache(t *testing.T) {
-	storage, err := NewDataColumnStorage(
-		context.Background(),
-		WithDataColumnBasePath(t.TempDir()),
-		WithDataColumnRetentionEpochs(10_000),
-	)
-	require.NoError(t, err)
-
-	_, verifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
-		t,
-		verification.DataColumnsParamsByRoot{
-			{0}: {
-				{Slot: 33, ColumnIndex: 2, DataColumn: []byte{1, 2, 3}}, // Period 0 - Epoch 1
-				{Slot: 33, ColumnIndex: 4, DataColumn: []byte{2, 3, 4}}, // Period 0 - Epoch 1
-			},
-			{1}: {
-				{Slot: 128_002, ColumnIndex: 2, DataColumn: []byte{1, 2, 3}}, // Period 0 - Epoch 4000
-				{Slot: 128_002, ColumnIndex: 4, DataColumn: []byte{2, 3, 4}}, // Period 0 - Epoch 4000
-			},
-			{2}: {
-				{Slot: 128_003, ColumnIndex: 1, DataColumn: []byte{1, 2, 3}}, // Period 0 - Epoch 4000
-				{Slot: 128_003, ColumnIndex: 3, DataColumn: []byte{2, 3, 4}}, // Period 0 - Epoch 4000
-			},
-			{3}: {
-				{Slot: 128_034, ColumnIndex: 2, DataColumn: []byte{1, 2, 3}}, // Period 0 - Epoch 4001
-				{Slot: 128_034, ColumnIndex: 4, DataColumn: []byte{2, 3, 4}}, // Period 0 - Epoch 4001
-			},
-			{4}: {
-				{Slot: 131_138, ColumnIndex: 2, DataColumn: []byte{1, 2, 3}}, // Period 1 - Epoch 4098
-			},
-			{5}: {
-				{Slot: 131_138, ColumnIndex: 1, DataColumn: []byte{1, 2, 3}}, // Period 1 - Epoch 4098
-			},
-			{6}: {
-				{Slot: 131_168, ColumnIndex: 0, DataColumn: []byte{1, 2, 3}}, // Period 1 - Epoch 4099
-			},
-		},
-	)
-
-	err = storage.Save(verifiedRoDataColumnSidecars)
-	require.NoError(t, err)
-
-	storage.retentionEpochs = 4_096
-
-	storage.WarmCache()
-	require.Equal(t, primitives.Epoch(4_000), storage.cache.lowestCachedEpoch)
-	require.Equal(t, 6, len(storage.cache.cache))
-
-	summary, ok := storage.cache.get([fieldparams.RootLength]byte{1})
-	require.Equal(t, true, ok)
-	require.DeepEqual(t, DataColumnStorageSummary{epoch: 4_000, mask: [fieldparams.NumberOfColumns]bool{false, false, true, false, true}}, summary)
-
-	summary, ok = storage.cache.get([fieldparams.RootLength]byte{2})
-	require.Equal(t, true, ok)
-	require.DeepEqual(t, DataColumnStorageSummary{epoch: 4_000, mask: [fieldparams.NumberOfColumns]bool{false, true, false, true}}, summary)
-
-	summary, ok = storage.cache.get([fieldparams.RootLength]byte{3})
-	require.Equal(t, true, ok)
-	require.DeepEqual(t, DataColumnStorageSummary{epoch: 4_001, mask: [fieldparams.NumberOfColumns]bool{false, false, true, false, true}}, summary)
-
-	summary, ok = storage.cache.get([fieldparams.RootLength]byte{4})
-	require.Equal(t, true, ok)
-	require.DeepEqual(t, DataColumnStorageSummary{epoch: 4_098, mask: [fieldparams.NumberOfColumns]bool{false, false, true}}, summary)
-
-	summary, ok = storage.cache.get([fieldparams.RootLength]byte{5})
-	require.Equal(t, true, ok)
-	require.DeepEqual(t, DataColumnStorageSummary{epoch: 4_098, mask: [fieldparams.NumberOfColumns]bool{false, true}}, summary)
-
-	summary, ok = storage.cache.get([fieldparams.RootLength]byte{6})
-	require.Equal(t, true, ok)
-	require.DeepEqual(t, DataColumnStorageSummary{epoch: 4_099, mask: [fieldparams.NumberOfColumns]bool{true}}, summary)
-}
-
-func TestSaveDataColumnsSidecars(t *testing.T) {
-	t.Run("wrong numbers of columns", func(t *testing.T) {
-		cfg := params.BeaconConfig().Copy()
-		cfg.NumberOfColumns = 0
-		params.OverrideBeaconConfig(cfg)
-		params.SetupTestConfigCleanup(t)
-
-		_, verifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
-			t,
-			verification.DataColumnsParamsByRoot{
-				{}: {{ColumnIndex: 12}, {ColumnIndex: 1_000_000}, {ColumnIndex: 48}},
-			},
-		)
-
-		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
-		err := dataColumnStorage.Save(verifiedRoDataColumnSidecars)
-		require.ErrorIs(t, err, errWrongNumberOfColumns)
-	})
-
-	t.Run("one of the column index is too large", func(t *testing.T) {
-		_, verifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
-			t,
-			verification.DataColumnsParamsByRoot{{}: {{ColumnIndex: 12}, {ColumnIndex: 1_000_000}, {ColumnIndex: 48}}},
-		)
-
-		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
-		err := dataColumnStorage.Save(verifiedRoDataColumnSidecars)
-		require.ErrorIs(t, err, errDataColumnIndexTooLarge)
-	})
-
-	t.Run("different slots", func(t *testing.T) {
-		_, verifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
-			t,
-			verification.DataColumnsParamsByRoot{
-				{}: {
-					{Slot: 1, ColumnIndex: 12, DataColumn: []byte{1, 2, 3}},
-					{Slot: 2, ColumnIndex: 12, DataColumn: []byte{1, 2, 3}},
-				},
-			},
-		)
-
-		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
-		err := dataColumnStorage.Save(verifiedRoDataColumnSidecars)
-		require.ErrorIs(t, err, errDataColumnSidecarsFromDifferentSlots)
-	})
-
-	t.Run("new file - no data columns to save", func(t *testing.T) {
-		_, verifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
-			t,
-			verification.DataColumnsParamsByRoot{{}: {}},
-		)
-
-		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
-		err := dataColumnStorage.Save(verifiedRoDataColumnSidecars)
-		require.NoError(t, err)
-	})
-
-	t.Run("new file - different data column size", func(t *testing.T) {
-		_, verifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
-			t,
-			verification.DataColumnsParamsByRoot{
-				{}: {
-					{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}},
-					{ColumnIndex: 11, DataColumn: []byte{1, 2, 3, 4}},
-				},
-			},
-		)
-
-		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
-		err := dataColumnStorage.Save(verifiedRoDataColumnSidecars)
-		require.ErrorIs(t, err, errWrongSszEncodedDataColumnSidecarSize)
-	})
-
-	t.Run("existing file - wrong incoming SSZ encoded size", func(t *testing.T) {
-		_, verifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
-			t,
-			verification.DataColumnsParamsByRoot{{1}: {{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}}}},
-		)
-
-		// Save data columns into a file.
-		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
-		err := dataColumnStorage.Save(verifiedRoDataColumnSidecars)
-		require.NoError(t, err)
-
-		// Build a data column sidecar for the same block but with a different
-		// column index and an different SSZ encoded size.
-		_, verifiedRoDataColumnSidecars = verification.CreateTestVerifiedRoDataColumnSidecars(
-			t,
-			verification.DataColumnsParamsByRoot{{1}: {{ColumnIndex: 13, DataColumn: []byte{1, 2, 3, 4}}}},
-		)
-
-		// Try to rewrite the file.
-		err = dataColumnStorage.Save(verifiedRoDataColumnSidecars)
-		require.ErrorIs(t, err, errWrongSszEncodedDataColumnSidecarSize)
-	})
-
-	t.Run("nominal", func(t *testing.T) {
-		_, inputVerifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
-			t,
-			verification.DataColumnsParamsByRoot{
-				{1}: {
-					{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}},
-					{ColumnIndex: 11, DataColumn: []byte{3, 4, 5}},
-					{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}}, // OK if duplicate
-					{ColumnIndex: 13, DataColumn: []byte{6, 7, 8}},
-				},
-				{2}: {
-					{ColumnIndex: 12, DataColumn: []byte{3, 4, 5}},
-					{ColumnIndex: 13, DataColumn: []byte{6, 7, 8}},
-				},
-			},
-		)
-
-		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
-		err := dataColumnStorage.Save(inputVerifiedRoDataColumnSidecars)
-		require.NoError(t, err)
-
-		_, inputVerifiedRoDataColumnSidecars = verification.CreateTestVerifiedRoDataColumnSidecars(
-			t,
-			verification.DataColumnsParamsByRoot{
-				{1}: {
-					{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}}, // OK if duplicate
-					{ColumnIndex: 15, DataColumn: []byte{2, 3, 4}},
-					{ColumnIndex: 1, DataColumn: []byte{2, 3, 4}},
-				},
-				{3}: {
-					{ColumnIndex: 6, DataColumn: []byte{3, 4, 5}},
-					{ColumnIndex: 2, DataColumn: []byte{6, 7, 8}},
-				},
-			},
-		)
-
-		err = dataColumnStorage.Save(inputVerifiedRoDataColumnSidecars)
-		require.NoError(t, err)
-
-		type fixture struct {
-			fileName         string
-			blockRoot        [fieldparams.RootLength]byte
-			expectedIndices  [mandatoryNumberOfColumns]byte
-			dataColumnParams []verification.DataColumnParams
-		}
-
-		fixtures := []fixture{
-			{
-				fileName:  "0/0/0x0100000000000000000000000000000000000000000000000000000000000000.sszs",
-				blockRoot: [fieldparams.RootLength]byte{1},
-				expectedIndices: [mandatoryNumberOfColumns]byte{
-					0, nonZeroOffset + 4, 0, 0, 0, 0, 0, 0,
-					0, 0, 0, nonZeroOffset + 1, nonZeroOffset, nonZeroOffset + 2, 0, nonZeroOffset + 3,
-					// The rest is filled with zeroes.
-				},
-				dataColumnParams: []verification.DataColumnParams{
-					{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}},
-					{ColumnIndex: 11, DataColumn: []byte{3, 4, 5}},
-					{ColumnIndex: 13, DataColumn: []byte{6, 7, 8}},
-					{ColumnIndex: 15, DataColumn: []byte{2, 3, 4}},
-					{ColumnIndex: 1, DataColumn: []byte{2, 3, 4}},
-				},
-			},
-			{
-				fileName:  "0/0/0x0200000000000000000000000000000000000000000000000000000000000000.sszs",
-				blockRoot: [fieldparams.RootLength]byte{2},
-				expectedIndices: [mandatoryNumberOfColumns]byte{
-					0, 0, 0, 0, 0, 0, 0, 0,
-					0, 0, 0, 0, nonZeroOffset, nonZeroOffset + 1, 0, 0,
-					// The rest is filled with zeroes.
-				},
-				dataColumnParams: []verification.DataColumnParams{
-					{ColumnIndex: 12, DataColumn: []byte{3, 4, 5}},
-					{ColumnIndex: 13, DataColumn: []byte{6, 7, 8}},
-				},
-			},
-			{
-				fileName:  "0/0/0x0300000000000000000000000000000000000000000000000000000000000000.sszs",
-				blockRoot: [fieldparams.RootLength]byte{3},
-				expectedIndices: [mandatoryNumberOfColumns]byte{
-					0, 0, nonZeroOffset + 1, 0, 0, 0, nonZeroOffset, 0,
-					// The rest is filled with zeroes.
-				},
-				dataColumnParams: []verification.DataColumnParams{
-					{ColumnIndex: 6, DataColumn: []byte{3, 4, 5}},
-					{ColumnIndex: 2, DataColumn: []byte{6, 7, 8}},
-				},
-			},
-		}
-
-		for _, fixture := range fixtures {
-			// Build expected data column sidecars.
-			_, expectedDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
-				t,
-				verification.DataColumnsParamsByRoot{fixture.blockRoot: fixture.dataColumnParams},
-			)
-
-			// Build expected bytes.
-			firstSszEncodedDataColumnSidecar, err := expectedDataColumnSidecars[0].MarshalSSZ()
-			require.NoError(t, err)
-
-			dataColumnSidecarsCount := len(expectedDataColumnSidecars)
-			sszEncodedDataColumnSidecarSize := len(firstSszEncodedDataColumnSidecar)
-
-			sszEncodedDataColumnSidecars := make([]byte, 0, dataColumnSidecarsCount*sszEncodedDataColumnSidecarSize)
-			sszEncodedDataColumnSidecars = append(sszEncodedDataColumnSidecars, firstSszEncodedDataColumnSidecar...)
-			for _, dataColumnSidecar := range expectedDataColumnSidecars[1:] {
-				sszEncodedDataColumnSidecar, err := dataColumnSidecar.MarshalSSZ()
-				require.NoError(t, err)
-				sszEncodedDataColumnSidecars = append(sszEncodedDataColumnSidecars, sszEncodedDataColumnSidecar...)
-			}
-
-			var encodedSszEncodedDataColumnSidecarSize [encodedSszEncodedDataColumnSidecarSizeSize]byte
-			binary.BigEndian.PutUint32(encodedSszEncodedDataColumnSidecarSize[:], uint32(sszEncodedDataColumnSidecarSize))
-
-			expectedBytes := make([]byte, 0, headerSize+dataColumnSidecarsCount*sszEncodedDataColumnSidecarSize)
-			expectedBytes = append(expectedBytes, []byte{0x01}...)
-			expectedBytes = append(expectedBytes, encodedSszEncodedDataColumnSidecarSize[:]...)
-			expectedBytes = append(expectedBytes, fixture.expectedIndices[:]...)
-			expectedBytes = append(expectedBytes, sszEncodedDataColumnSidecars...)
-
-			// Check the actual content of the file.
-			actualBytes, err := afero.ReadFile(dataColumnStorage.fs, fixture.fileName)
-			require.NoError(t, err)
-			require.DeepSSZEqual(t, expectedBytes, actualBytes)
-
-			// Check the summary.
-			indices := map[uint64]bool{}
-			for _, dataColumnParam := range fixture.dataColumnParams {
-				indices[dataColumnParam.ColumnIndex] = true
-			}
-
-			summary := dataColumnStorage.Summary(fixture.blockRoot)
-			for index := range uint64(mandatoryNumberOfColumns) {
-				require.Equal(t, indices[index], summary.HasIndex(index))
-			}
-
-			err = dataColumnStorage.Remove(fixture.blockRoot)
-			require.NoError(t, err)
-
-			summary = dataColumnStorage.Summary(fixture.blockRoot)
-			for index := range uint64(mandatoryNumberOfColumns) {
-				require.Equal(t, false, summary.HasIndex(index))
-			}
-
-			_, err = afero.ReadFile(dataColumnStorage.fs, fixture.fileName)
-			require.ErrorIs(t, err, os.ErrNotExist)
-		}
-	})
-}
-
-func TestGetDataColumnSidecars(t *testing.T) {
-	t.Run("not found", func(t *testing.T) {
-		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
-
-		verifiedRODataColumnSidecars, err := dataColumnStorage.Get([fieldparams.RootLength]byte{1}, []uint64{12, 13, 14})
-		require.NoError(t, err)
-		require.Equal(t, 0, len(verifiedRODataColumnSidecars))
-	})
-
-	t.Run("nominal", func(t *testing.T) {
-		_, expectedVerifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
-			t,
-			verification.DataColumnsParamsByRoot{
-				{1}: {
-					{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}},
-					{ColumnIndex: 14, DataColumn: []byte{2, 3, 4}},
-				},
-			},
-		)
-
-		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
-		err := dataColumnStorage.Save(expectedVerifiedRoDataColumnSidecars)
-		require.NoError(t, err)
-
-		verifiedRODataColumnSidecars, err := dataColumnStorage.Get([fieldparams.RootLength]byte{1}, nil)
-		require.NoError(t, err)
-		require.DeepSSZEqual(t, expectedVerifiedRoDataColumnSidecars, verifiedRODataColumnSidecars)
-
-		verifiedRODataColumnSidecars, err = dataColumnStorage.Get([fieldparams.RootLength]byte{1}, []uint64{12, 13, 14})
-		require.NoError(t, err)
-		require.DeepSSZEqual(t, expectedVerifiedRoDataColumnSidecars, verifiedRODataColumnSidecars)
-	})
-}
-
-func TestRemove(t *testing.T) {
-	t.Run("not found", func(t *testing.T) {
-		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
-		err := dataColumnStorage.Remove([fieldparams.RootLength]byte{1})
-		require.NoError(t, err)
-	})
-
-	t.Run("nominal", func(t *testing.T) {
-		_, inputVerifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
-			t,
-			verification.DataColumnsParamsByRoot{
-				{1}: {
-					{Slot: 32, ColumnIndex: 10, DataColumn: []byte{1, 2, 3}},
-					{Slot: 32, ColumnIndex: 11, DataColumn: []byte{2, 3, 4}},
-				},
-				{2}: {
-					{Slot: 33, ColumnIndex: 10, DataColumn: []byte{1, 2, 3}},
-					{Slot: 33, ColumnIndex: 11, DataColumn: []byte{2, 3, 4}},
-				},
-			},
-		)
-
-		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
-		err := dataColumnStorage.Save(inputVerifiedRoDataColumnSidecars)
-		require.NoError(t, err)
-
-		err = dataColumnStorage.Remove([fieldparams.RootLength]byte{1})
-		require.NoError(t, err)
-
-		summary := dataColumnStorage.Summary([fieldparams.RootLength]byte{1})
-		require.Equal(t, primitives.Epoch(0), summary.epoch)
-		require.Equal(t, uint64(0), summary.Count())
-
-		summary = dataColumnStorage.Summary([fieldparams.RootLength]byte{2})
-		require.Equal(t, primitives.Epoch(1), summary.epoch)
-		require.Equal(t, uint64(2), summary.Count())
-
-		actual, err := dataColumnStorage.Get([fieldparams.RootLength]byte{1}, nil)
-		require.NoError(t, err)
-		require.Equal(t, 0, len(actual))
-
-		actual, err = dataColumnStorage.Get([fieldparams.RootLength]byte{2}, nil)
-		require.NoError(t, err)
-		require.Equal(t, 2, len(actual))
-	})
-}
-
-func TestClear(t *testing.T) {
-	_, inputVerifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
-		t,
-		verification.DataColumnsParamsByRoot{
-			{1}: {{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}}},
-			{2}: {{ColumnIndex: 13, DataColumn: []byte{6, 7, 8}}},
-		},
-	)
-
-	_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
-	err := dataColumnStorage.Save(inputVerifiedRoDataColumnSidecars)
-	require.NoError(t, err)
-
-	filePaths := []string{
-		"0/0/0x0100000000000000000000000000000000000000000000000000000000000000.sszs",
-		"0/0/0x0200000000000000000000000000000000000000000000000000000000000000.sszs",
-	}
-
-	for _, filePath := range filePaths {
-		_, err = afero.ReadFile(dataColumnStorage.fs, filePath)
-		require.NoError(t, err)
-	}
-
-	err = dataColumnStorage.Clear()
-	require.NoError(t, err)
-
-	summary := dataColumnStorage.Summary([fieldparams.RootLength]byte{1})
-	for index := range uint64(mandatoryNumberOfColumns) {
-		require.Equal(t, false, summary.HasIndex(index))
-	}
-
-	for _, filePath := range filePaths {
-		_, err = afero.ReadFile(dataColumnStorage.fs, filePath)
-		require.ErrorIs(t, err, os.ErrNotExist)
-	}
-}
-
-func TestMetadata(t *testing.T) {
-	t.Run("wrong version", func(t *testing.T) {
-		_, verifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
-			t,
-			verification.DataColumnsParamsByRoot{
-				{1}: {{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}}},
-			},
-		)
-
-		// Save data columns into a file.
-		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
-		err := dataColumnStorage.Save(verifiedRoDataColumnSidecars)
-		require.NoError(t, err)
-
-		// Alter the version.
-		const filePath = "0/0/0x0100000000000000000000000000000000000000000000000000000000000000.sszs"
-		file, err := dataColumnStorage.fs.OpenFile(filePath, os.O_WRONLY, os.FileMode(0600))
-		require.NoError(t, err)
-
-		count, err := file.Write([]byte{42})
-		require.NoError(t, err)
-		require.Equal(t, 1, count)
-
-		// Try to read the metadata.
-		_, err = dataColumnStorage.metadata(file)
-		require.ErrorIs(t, err, errWrongVersion)
-
-		err = file.Close()
-		require.NoError(t, err)
-	})
-}
-
-func TestNewStorageIndices(t *testing.T) {
-	t.Run("wrong number of columns", func(t *testing.T) {
-		_, err := newStorageIndices(nil)
-		require.ErrorIs(t, err, errWrongNumberOfColumns)
-	})
-
-	t.Run("nominal", func(t *testing.T) {
-		var indices [mandatoryNumberOfColumns]byte
-		indices[0] = 1
-
-		storageIndices, err := newStorageIndices(indices[:])
-		require.NoError(t, err)
-		require.Equal(t, indices, storageIndices.indices)
-	})
-}
-
-func TestStorageIndicesGet(t *testing.T) {
-	t.Run("index too large", func(t *testing.T) {
-		var indices storageIndices
-		_, _, err := indices.get(1_000_000)
-		require.ErrorIs(t, errDataColumnIndexTooLarge, err)
-	})
-
-	t.Run("index not set", func(t *testing.T) {
-		const expected = false
-		var indices storageIndices
-		actual, _, err := indices.get(0)
-		require.NoError(t, err)
-		require.Equal(t, expected, actual)
-	})
-
-	t.Run("index set", func(t *testing.T) {
-		const (
-			expectedOk       = true
-			expectedPosition = int64(3)
-		)
-
-		indices := storageIndices{indices: [mandatoryNumberOfColumns]byte{0, 131}}
-		actualOk, actualPosition, err := indices.get(1)
-		require.NoError(t, err)
-		require.Equal(t, expectedOk, actualOk)
-		require.Equal(t, expectedPosition, actualPosition)
-	})
-}
-
-func TestStorageIndicesLen(t *testing.T) {
-	const expected = int64(2)
-	indices := storageIndices{count: 2}
-	actual := indices.len()
-	require.Equal(t, expected, actual)
-}
-
-func TestStorageIndicesAll(t *testing.T) {
-	expectedIndices := []uint64{1, 3}
-	indices := storageIndices{indices: [mandatoryNumberOfColumns]byte{0, 131, 0, 128}}
-	actualIndices := indices.all()
-	require.DeepEqual(t, expectedIndices, actualIndices)
-}
-
-func TestStorageIndicesSet(t *testing.T) {
-	t.Run("data column index too large", func(t *testing.T) {
-		var indices storageIndices
-		err := indices.set(1_000_000, 0)
-		require.ErrorIs(t, errDataColumnIndexTooLarge, err)
-	})
-
-	t.Run("position too large", func(t *testing.T) {
-		var indices storageIndices
-		err := indices.set(0, 255)
-		require.ErrorIs(t, errDataColumnIndexTooLarge, err)
-	})
-
-	t.Run("nominal", func(t *testing.T) {
-		expected := [mandatoryNumberOfColumns]byte{0, 0, 128, 0, 131}
-		var storageIndices storageIndices
-		require.Equal(t, int64(0), storageIndices.len())
-
-		err := storageIndices.set(2, 1)
-		require.NoError(t, err)
-		require.Equal(t, int64(1), storageIndices.len())
-
-		err = storageIndices.set(4, 3)
-		require.NoError(t, err)
-		require.Equal(t, int64(2), storageIndices.len())
-
-		err = storageIndices.set(2, 0)
-		require.NoError(t, err)
-		require.Equal(t, int64(2), storageIndices.len())
-
-		actual := storageIndices.indices
-		require.Equal(t, expected, actual)
-	})
-}
-
-func TestPrune(t *testing.T) {
-	t.Run(("nothing to prune"), func(t *testing.T) {
-		dir := t.TempDir()
-		dataColumnStorage, err := NewDataColumnStorage(context.Background(), WithDataColumnBasePath(dir))
-		require.NoError(t, err)
-
-		dataColumnStorage.prune()
-	})
-	t.Run("nominal", func(t *testing.T) {
-		var compareSlices = func(left, right []string) bool {
-			if len(left) != len(right) {
-				return false
-			}
-
-			leftMap := make(map[string]bool, len(left))
-			for _, leftItem := range left {
-				leftMap[leftItem] = true
-			}
-
-			for _, rightItem := range right {
-				if _, ok := leftMap[rightItem]; !ok {
-					return false
-				}
-			}
-
-			return true
-		}
-		_, verifiedRoDataColumnSidecars := verification.CreateTestVerifiedRoDataColumnSidecars(
-			t,
-			verification.DataColumnsParamsByRoot{
-				{0}: {
-					{Slot: 33, ColumnIndex: 2, DataColumn: []byte{1, 2, 3}}, // Period 0 - Epoch 1
-					{Slot: 33, ColumnIndex: 4, DataColumn: []byte{2, 3, 4}}, // Period 0 - Epoch 1
-				},
-				{1}: {
-					{Slot: 128_002, ColumnIndex: 2, DataColumn: []byte{1, 2, 3}}, // Period 0 - Epoch 4000
-					{Slot: 128_002, ColumnIndex: 4, DataColumn: []byte{2, 3, 4}}, // Period 0 - Epoch 4000
-				},
-				{2}: {
-					{Slot: 128_003, ColumnIndex: 1, DataColumn: []byte{1, 2, 3}}, // Period 0 - Epoch 4000
-					{Slot: 128_003, ColumnIndex: 3, DataColumn: []byte{2, 3, 4}}, // Period 0 - Epoch 4000
-				},
-				{3}: {
-					{Slot: 131_138, ColumnIndex: 2, DataColumn: []byte{1, 2, 3}}, // Period 1 - Epoch 4098
-					{Slot: 131_138, ColumnIndex: 3, DataColumn: []byte{1, 2, 3}}, // Period 1 - Epoch 4098
-				},
-				{4}: {
-					{Slot: 131_169, ColumnIndex: 2, DataColumn: []byte{1, 2, 3}}, // Period 1 - Epoch 4099
-					{Slot: 131_169, ColumnIndex: 3, DataColumn: []byte{1, 2, 3}}, // Period 1 - Epoch 4099
-				},
-				{5}: {
-					{Slot: 262_144, ColumnIndex: 2, DataColumn: []byte{1, 2, 3}}, // Period 2 - Epoch 8192
-					{Slot: 262_144, ColumnIndex: 3, DataColumn: []byte{1, 2, 3}}, // Period 2 - Epoch 8292
-				},
-			},
-		)
-
-		dir := t.TempDir()
-		dataColumnStorage, err := NewDataColumnStorage(context.Background(), WithDataColumnBasePath(dir), WithDataColumnRetentionEpochs(10_000))
-		require.NoError(t, err)
-
-		err = dataColumnStorage.Save(verifiedRoDataColumnSidecars)
-		require.NoError(t, err)
-
-		dirs, err := listDir(dataColumnStorage.fs, ".")
-		require.NoError(t, err)
-		require.Equal(t, true, compareSlices([]string{"0", "1", "2"}, dirs))
-
-		dirs, err = listDir(dataColumnStorage.fs, "0")
-		require.NoError(t, err)
-		require.Equal(t, true, compareSlices([]string{"1", "4000"}, dirs))
-
-		dirs, err = listDir(dataColumnStorage.fs, "1")
-		require.NoError(t, err)
-		require.Equal(t, true, compareSlices([]string{"4099", "4098"}, dirs))
-
-		dirs, err = listDir(dataColumnStorage.fs, "2")
-		require.NoError(t, err)
-		require.Equal(t, true, compareSlices([]string{"8192"}, dirs))
-
-		dirs, err = listDir(dataColumnStorage.fs, "0/1")
-		require.NoError(t, err)
-		require.Equal(t, true, compareSlices([]string{"0x0000000000000000000000000000000000000000000000000000000000000000.sszs"}, dirs))
-
-		dirs, err = listDir(dataColumnStorage.fs, "0/4000")
-		require.NoError(t, err)
-		require.Equal(t, true, compareSlices([]string{
-			"0x0200000000000000000000000000000000000000000000000000000000000000.sszs",
-			"0x0100000000000000000000000000000000000000000000000000000000000000.sszs",
-		}, dirs))
-
-		dirs, err = listDir(dataColumnStorage.fs, "1/4098")
-		require.NoError(t, err)
-		require.Equal(t, true, compareSlices([]string{"0x0300000000000000000000000000000000000000000000000000000000000000.sszs"}, dirs))
-
-		dirs, err = listDir(dataColumnStorage.fs, "1/4099")
-		require.NoError(t, err)
-		require.Equal(t, true, compareSlices([]string{"0x0400000000000000000000000000000000000000000000000000000000000000.sszs"}, dirs))
-
-		dirs, err = listDir(dataColumnStorage.fs, "2/8192")
-		require.NoError(t, err)
-		require.Equal(t, true, compareSlices([]string{"0x0500000000000000000000000000000000000000000000000000000000000000.sszs"}, dirs))
-
-		_, verifiedRoDataColumnSidecars = verification.CreateTestVerifiedRoDataColumnSidecars(
-			t,
-			verification.DataColumnsParamsByRoot{
-				{6}: {{Slot: 451_141, ColumnIndex: 2, DataColumn: []byte{1, 2, 3}}}, // Period 3 - Epoch 14_098
-			},
-		)
-
-		err = dataColumnStorage.Save(verifiedRoDataColumnSidecars)
-		require.NoError(t, err)
-
-		// dataColumnStorage.prune(14_098)
-		dataColumnStorage.prune()
-
-		dirs, err = listDir(dataColumnStorage.fs, ".")
-		require.NoError(t, err)
-		require.Equal(t, true, compareSlices([]string{"1", "2", "3"}, dirs))
-
-		dirs, err = listDir(dataColumnStorage.fs, "1")
-		require.NoError(t, err)
-		require.Equal(t, true, compareSlices([]string{"4099"}, dirs))
-
-		dirs, err = listDir(dataColumnStorage.fs, "2")
-		require.NoError(t, err)
-		require.Equal(t, true, compareSlices([]string{"8192"}, dirs))
-
-		dirs, err = listDir(dataColumnStorage.fs, "3")
-		require.NoError(t, err)
-		require.Equal(t, true, compareSlices([]string{"14098"}, dirs))
-
-		dirs, err = listDir(dataColumnStorage.fs, "1/4099")
-		require.NoError(t, err)
-		require.Equal(t, true, compareSlices([]string{"0x0400000000000000000000000000000000000000000000000000000000000000.sszs"}, dirs))
-
-		dirs, err = listDir(dataColumnStorage.fs, "2/8192")
-		require.NoError(t, err)
-		require.Equal(t, true, compareSlices([]string{"0x0500000000000000000000000000000000000000000000000000000000000000.sszs"}, dirs))
-
-		dirs, err = listDir(dataColumnStorage.fs, "3/14098")
-		require.NoError(t, err)
-		require.Equal(t, true, compareSlices([]string{"0x0600000000000000000000000000000000000000000000000000000000000000.sszs"}, dirs))
-	})
-}

beacon-chain/db/filesystem/doc.go

@@ -1,104 +0,0 @@
-package filesystem
-
-// nolint:dupword
-/*
-Data column sidecars storage documentation
-==========================================
-
-File organisation
------------------
-- The first byte represents the version of the file structure (up to 0xff = 255).
-  We set it to 0x01.
-  Note: This is not strictly needed, but it will help a lot if, in the future,
-        we want to modify the file structure.
-- The next 4 bytes represents the size of a SSZ encoded data column sidecar.
-  (See the `Computation of the maximum size of a DataColumnSidecar` section to a description
-  of how this value is computed).
-- The next 128 bytes represent the index in the file of a given column.
-  The first bit of each byte in the index is set to 0 if there is no data column,
-  and set to 1 if there is a data column.
-  The remaining 7 bits (from 0 to 127) represent the index of the data column.
-  This sentinel bit is needed to distinguish between the column with index 0 and no column.
-  Example: If the column with index 5 is in the 3th position in the file, then indices[5] = 0x80 + 0x03 = 0x83.
-- The rest of the file is a repeat of the SSZ encoded data columns sidecars.
-
-
-|------------------------------------------|------------------------------------------------------------------------------------|
-|                Byte offset               |                                     Description                                    |
-|------------------------------------------|------------------------------------------------------------------------------------|
-|                                        0 | version (1 byte) | sszEncodedDataColumnSidecarSize (4 bytes) | indices (128 bytes) |
-|133 +   0*sszEncodedDataColumnSidecarSize | sszEncodedDataColumnSidecar (sszEncodedDataColumnSidecarSize bytes)                |
-|133 +   1*sszEncodedDataColumnSidecarSize | sszEncodedDataColumnSidecar (sszEncodedDataColumnSidecarSize bytes)                |
-|133 +   2*sszEncodedDataColumnSidecarSize | sszEncodedDataColumnSidecar (sszEncodedDataColumnSidecarSize bytes)                |
-|                  ...                     |                                 ...                                                |
-|133 + 127*sszEncodedDataColumnSidecarSize | sszEncodedDataColumnSidecar (sszEncodedDataColumnSidecarSize bytes)                |
-|------------------------------------------|------------------------------------------------------------------------------------|
-
-Each file is named after the block root where the data columns were data columns are committed to.
-Example: `0x259c6d2f6a0bb75e2405cea7cb248e5663dc26b9404fd3bcd777afc20de91c1e.sszs`
-
-Database organisation
----------------------
-SSZ encoded data column sidecars are stored following the `by-epoch` layout.
-- The first layer is a directory corresponding to the `period`, which corresponds to the epoch divided by the 4096.
-- The second layer is a directory corresponding to the epoch.
-- Then all files are stored in the epoch directory.
-
-Example:
-data-columns
-├── 0
-│   ├── 3638
-│   │   ├── 0x259c6d2f6a0bb75e2405cea7cb248e5663dc26b9404fd3bcd777afc20de91c1e.sszs
-│   │   ├── 0x2a855b1f6e9a2f04f8383e336325bf7d5ba02d1eab3ef90ef183736f8c768533.sszs
-│   │   ├── ...
-│   │   ├── 0xeb78e2b2350a71c640f1e96fea9e42f38e65705ab7e6e100c8bc9c589f2c5f2b.sszs
-│   │   └── 0xeb7ee68da988fd20d773d45aad01dd62527734367a146e2b048715bd68a4e370.sszs
-│   └── 3639
-│       ├── 0x0fd231fe95e57936fa44f6c712c490b9e337a481b661dfd46768901e90444330.sszs
-│       ├── 0x1bf5edff6b6ba2b65b1db325ff3312bbb57da461ef2ae651bd741af851aada3a.sszs
-│       ├── ...
-│       ├── 0xa156a527e631f858fee79fab7ef1fde3f6117a2e1201d47c09fbab0c6780c937.sszs
-│       └── 0xcd80bc535ddc467dea1d19e0c39c1160875ccd1989061bcd8ce206e3c1261c87.sszs
-└── 1
-    ├── 4096
-    │   ├── 0x0d244009093e2bedb72eb265280290199e8c7bf1d90d7583c41af40d9f662269.sszs
-    │   ├── 0x11f420928d8de41c50e735caab0369996824a5299c5f054e097965855925697d.sszs
-    │   ├── ...
-    │   ├── 0xbe91fc782877ed400d95c02c61aebfdd592635d11f8e64c94b46abd84f45c967.sszs
-    │   └── 0xf246189f078f02d30173ff74605cf31c9e65b5e463275ebdbeb40476638135ff.sszs
-    └── 4097
-        ├── 0x454d000674793c479e90504c0fe9827b50bb176ae022dab4e37d6a21471ab570.sszs
-        ├── 0xac5eb7437d7190c48cfa863e3c45f96a7f8af371d47ac12ccda07129a06af763.sszs
-        ├── ...
-        ├── 0xb7df30561d9d92ab5fafdd96bca8b44526497c8debf0fc425c7a0770b2abeb83.sszs
-        └── 0xc1dd0b1ae847b6ec62303a36d08c6a4a2e9e3ec4be3ff70551972a0ee3de9c14.sszs
-
-Computation of the maximum size of a DataColumnSidecar
-------------------------------------------------------
-https://github.com/ethereum/consensus-specs/blob/dev/specs/fulu/das-core.md#datacolumnsidecar
-
-
-class DataColumnSidecar(Container):
-    index: ColumnIndex  # Index of column in extended matrix
-    column: List[Cell, MAX_BLOB_COMMITMENTS_PER_BLOCK]
-    kzg_commitments: List[KZGCommitment, MAX_BLOB_COMMITMENTS_PER_BLOCK]
-    kzg_proofs: List[KZGProof, MAX_BLOB_COMMITMENTS_PER_BLOCK]
-    signed_block_header: SignedBeaconBlockHeader
-    kzg_commitments_inclusion_proof: Vector[Bytes32, KZG_COMMITMENTS_INCLUSION_PROOF_DEPTH]
-
-
-- index: 2 bytes (ColumnIndex)
-- `column`: 4,096 (MAX_BLOB_COMMITMENTS_PER_BLOCK) * 64 (FIELD_ELEMENTS_PER_CELL) * 32 bytes (BYTES_PER_FIELD_ELEMENT) = 8,388,608 bytes
-- kzg_commitments: 4,096 (MAX_BLOB_COMMITMENTS_PER_BLOCK) * 48 bytes (KZGCommitment) = 196,608 bytes
-- kzg_proofs: 4,096 (MAX_BLOB_COMMITMENTS_PER_BLOCK) * 48 bytes (KZGProof) = 196,608 bytes
-- signed_block_header: 2 bytes (Slot) + 2 bytes (ValidatorIndex) + 3 * 2 bytes (Root) + 96 bytes (BLSSignature) = 106 bytes
-- kzg_commitments_inclusion_proof: 4 (KZG_COMMITMENTS_INCLUSION_PROOF_DEPTH) * 32 bytes = 128 bytes
-
-TOTAL: 8,782,060 bytes = 70,256,480 bits
-log(70,256,480) / log(2) ~= 26.07
-
-==> 32 bits (4 bytes) are enough to store the maximum size of a data column sidecar.
-
-The maximum size of an SSZ encoded data column can be 2**32 bits = 536,879,912 bytes,
-which left a room of 536,879,912 bytes - 8,782,060 bytes ~= 503 mega bytes to store the extra data needed by SSZ encoding (which is more than enough.)
-*/

beacon-chain/db/filesystem/iteration_test.go

@@ -1,6 +1,7 @@
 package filesystem
 
 import (
+	"bytes"
 	"fmt"
 	"io"
 	"math"
@@ -219,6 +220,28 @@ func TestListDir(t *testing.T) {
 	}
 }
 
+func TestSlotFromBlob(t *testing.T) {
+	cases := []struct {
+		slot primitives.Slot
+	}{
+		{slot: 0},
+		{slot: 2},
+		{slot: 1123581321},
+		{slot: math.MaxUint64},
+	}
+	for _, c := range cases {
+		t.Run(fmt.Sprintf("slot %d", c.slot), func(t *testing.T) {
+			_, sidecars := util.GenerateTestDenebBlockWithSidecar(t, [32]byte{}, c.slot, 1)
+			sc := sidecars[0]
+			enc, err := sc.MarshalSSZ()
+			require.NoError(t, err)
+			slot, err := slotFromBlob(bytes.NewReader(enc))
+			require.NoError(t, err)
+			require.Equal(t, c.slot, slot)
+		})
+	}
+}
+
 func TestIterationComplete(t *testing.T) {
 	targets := []migrationTestTarget{
 		{

beacon-chain/db/filesystem/layout_flat.go

@@ -1,11 +1,12 @@
 package filesystem
 
 import (
+	"encoding/binary"
+	"io"
 	"os"
 	"path"
 
 	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
-	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
 	"github.com/OffchainLabs/prysm/v6/time/slots"
 	"github.com/pkg/errors"
 	"github.com/spf13/afero"
@@ -114,25 +115,29 @@ func (l *flatLayout) pruneBefore(before primitives.Epoch) (*pruneSummary, error)
 
 // Read slot from marshaled BlobSidecar data in the given file. See slotFromBlob for details.
 func slotFromFile(name string, fs afero.Fs) (primitives.Slot, error) {
-	// Read whole file, try to unmarshal it as data columns sidecar, if it fails, try to unmarshal it as blob sidecar.
-	content, err := afero.ReadFile(fs, name)
+	f, err := fs.Open(name)
 	if err != nil {
 		return 0, err
 	}
+	defer func() {
+		if err := f.Close(); err != nil {
+			log.WithError(err).Errorf("Could not close blob file")
+		}
+	}()
+	return slotFromBlob(f)
+}
 
-	dataColumnSidecar := &ethpb.DataColumnSidecar{}
-	err = dataColumnSidecar.UnmarshalSSZ(content)
-	if err == nil {
-		return dataColumnSidecar.SignedBlockHeader.Header.Slot, nil
+// slotFromBlob reads the ssz data of a file at the specified offset (8 + 131072 + 48 + 48 = 131176 bytes),
+// which is calculated based on the size of the BlobSidecar struct and is based on the size of the fields
+// preceding the slot information within SignedBeaconBlockHeader.
+func slotFromBlob(at io.ReaderAt) (primitives.Slot, error) {
+	b := make([]byte, 8)
+	_, err := at.ReadAt(b, 131176)
+	if err != nil {
+		return 0, err
 	}
-
-	blobSidecar := &ethpb.BlobSidecar{}
-	err = blobSidecar.UnmarshalSSZ(content)
-	if err == nil {
-		return blobSidecar.SignedBlockHeader.Header.Slot, nil
-	}
-
-	return 0, errors.New("failed to unmarshal as blob sidecar or data column sidecar")
+	rawSlot := binary.LittleEndian.Uint64(b)
+	return primitives.Slot(rawSlot), nil
 }
 
 type flatSlotReader struct {

beacon-chain/db/filesystem/metrics.go

@@ -6,7 +6,6 @@ import (
 )
 
 var (
-	// Blobs
 	blobBuckets     = []float64{3, 5, 7, 9, 11, 13}
 	blobSaveLatency = promauto.NewHistogram(prometheus.HistogramOpts{
 		Name:    "blob_storage_save_latency",
@@ -34,29 +33,4 @@ var (
 		Name: "blob_disk_bytes",
 		Help: "Approximate number of bytes occupied by blobs in storage",
 	})
-
-	// Data columns
-	dataColumnBuckets     = []float64{3, 5, 7, 9, 11, 13}
-	dataColumnSaveLatency = promauto.NewHistogram(prometheus.HistogramOpts{
-		Name:    "data_column_storage_save_latency",
-		Help:    "Latency of DataColumnSidecar storage save operations in milliseconds",
-		Buckets: dataColumnBuckets,
-	})
-	dataColumnFetchLatency = promauto.NewHistogram(prometheus.HistogramOpts{
-		Name:    "data_column_storage_get_latency",
-		Help:    "Latency of DataColumnSidecar storage get operations in milliseconds",
-		Buckets: dataColumnBuckets,
-	})
-	dataColumnPrunedCounter = promauto.NewCounter(prometheus.CounterOpts{
-		Name: "data_column_pruned",
-		Help: "Number of DataColumnSidecar pruned.",
-	})
-	dataColumnWrittenCounter = promauto.NewCounter(prometheus.CounterOpts{
-		Name: "data_column_written",
-		Help: "Number of DataColumnSidecar written",
-	})
-	dataColumnDiskCount = promauto.NewGauge(prometheus.GaugeOpts{
-		Name: "data_column_disk_count",
-		Help: "Approximate number of data columns in storage",
-	})
 )

beacon-chain/db/filesystem/mock.go

@@ -1,19 +1,14 @@
 package filesystem
 
 import (
-	"context"
 	"testing"
 
-	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
 	"github.com/OffchainLabs/prysm/v6/config/params"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
 	"github.com/OffchainLabs/prysm/v6/time/slots"
 	"github.com/spf13/afero"
 )
 
-// Blobs
-// -----
-
 // NewEphemeralBlobStorage should only be used for tests.
 // The instance of BlobStorage returned is backed by an in-memory virtual filesystem,
 // improving test performance and simplifying cleanup.
@@ -80,52 +75,3 @@ func NewMockBlobStorageSummarizer(t *testing.T, set map[[32]byte][]int) BlobStor
 	}
 	return c
 }
-
-// Data columns
-// ------------
-
-// NewEphemeralDataColumnStorage should only be used for tests.
-// The instance of DataColumnStorage returned is backed by an in-memory virtual filesystem,
-// improving test performance and simplifying cleanup.
-func NewEphemeralDataColumnStorage(t testing.TB, opts ...DataColumnStorageOption) *DataColumnStorage {
-	return NewWarmedEphemeralDataColumnStorageUsingFs(t, afero.NewMemMapFs(), opts...)
-}
-
-// NewEphemeralDataColumnStorageAndFs can be used by tests that want access to the virtual filesystem
-// in order to interact with it outside the parameters of the DataColumnStorage API.
-func NewEphemeralDataColumnStorageAndFs(t testing.TB, opts ...DataColumnStorageOption) (afero.Fs, *DataColumnStorage) {
-	fs := afero.NewMemMapFs()
-	dcs := NewWarmedEphemeralDataColumnStorageUsingFs(t, fs, opts...)
-	return fs, dcs
-}
-
-func NewWarmedEphemeralDataColumnStorageUsingFs(t testing.TB, fs afero.Fs, opts ...DataColumnStorageOption) *DataColumnStorage {
-	bs := NewEphemeralDataColumnStorageUsingFs(t, fs, opts...)
-	bs.WarmCache()
-	return bs
-}
-
-func NewEphemeralDataColumnStorageUsingFs(t testing.TB, fs afero.Fs, opts ...DataColumnStorageOption) *DataColumnStorage {
-	opts = append(opts,
-		WithDataColumnRetentionEpochs(params.BeaconConfig().MinEpochsForBlobsSidecarsRequest),
-		WithDataColumnFs(fs),
-	)
-
-	bs, err := NewDataColumnStorage(context.Background(), opts...)
-	if err != nil {
-		t.Fatal(err)
-	}
-
-	return bs
-}
-
-func NewMockDataColumnStorageSummarizer(t *testing.T, set map[[fieldparams.RootLength]byte][]uint64) DataColumnStorageSummarizer {
-	c := newDataColumnStorageSummaryCache()
-	for root, indices := range set {
-		if err := c.set(DataColumnsIdent{Root: root, Epoch: 0, Indices: indices}); err != nil {
-			t.Fatal(err)
-		}
-	}
-
-	return c
-}

beacon-chain/db/iface/interface.go

@@ -41,6 +41,8 @@ type ReadOnlyDatabase interface {
 	StateSummary(ctx context.Context, blockRoot [32]byte) (*ethpb.StateSummary, error)
 	HasStateSummary(ctx context.Context, blockRoot [32]byte) bool
 	HighestSlotStatesBelow(ctx context.Context, slot primitives.Slot) ([]state.ReadOnlyBeaconState, error)
+	// StateDiff related methods.
+	StateDiff(ctx context.Context, slot primitives.Slot) (state.BeaconState, error)
 	// Checkpoint operations.
 	JustifiedCheckpoint(ctx context.Context) (*ethpb.Checkpoint, error)
 	FinalizedCheckpoint(ctx context.Context) (*ethpb.Checkpoint, error)
@@ -83,6 +85,8 @@ type NoHeadAccessDatabase interface {
 	DeleteStates(ctx context.Context, blockRoots [][32]byte) error
 	SaveStateSummary(ctx context.Context, summary *ethpb.StateSummary) error
 	SaveStateSummaries(ctx context.Context, summaries []*ethpb.StateSummary) error
+	// Statediff related methods.
+	SaveStateDiff(ctx context.Context, state state.ReadOnlyBeaconState) error
 	// Checkpoint operations.
 	SaveJustifiedCheckpoint(ctx context.Context, checkpoint *ethpb.Checkpoint) error
 	SaveFinalizedCheckpoint(ctx context.Context, checkpoint *ethpb.Checkpoint) error

beacon-chain/db/kv/BUILD.bazel

@@ -25,6 +25,7 @@ go_library(
         "migration_state_validators.go",
         "schema.go",
         "state.go",
+        "state_diff.go",
         "state_summary.go",
         "state_summary_cache.go",
         "utils.go",
@@ -44,6 +45,7 @@ go_library(
         "//config/fieldparams:go_default_library",
         "//config/params:go_default_library",
         "//consensus-types/blocks:go_default_library",
+        "//consensus-types/hdiff:go_default_library",
         "//consensus-types/interfaces:go_default_library",
         "//consensus-types/light-client:go_default_library",
         "//consensus-types/primitives:go_default_library",
@@ -59,7 +61,7 @@ go_library(
         "//runtime/version:go_default_library",
         "//time:go_default_library",
         "//time/slots:go_default_library",
-        "@com_github_dgraph_io_ristretto//:go_default_library",
+        "@com_github_dgraph_io_ristretto_v2//:go_default_library",
         "@com_github_ethereum_go_ethereum//common:go_default_library",
         "@com_github_ethereum_go_ethereum//common/hexutil:go_default_library",
         "@com_github_golang_snappy//:go_default_library",
@@ -94,6 +96,7 @@ go_test(
         "migration_archived_index_test.go",
         "migration_block_slot_index_test.go",
         "migration_state_validators_test.go",
+        "state_diff_test.go",
         "state_summary_test.go",
         "state_test.go",
         "utils_test.go",
@@ -116,6 +119,7 @@ go_test(
         "//consensus-types/light-client:go_default_library",
         "//consensus-types/primitives:go_default_library",
         "//encoding/bytesutil:go_default_library",
+        "//math:go_default_library",
         "//proto/dbval:go_default_library",
         "//proto/engine/v1:go_default_library",
         "//proto/prysm/v1alpha1:go_default_library",

beacon-chain/db/kv/blocks.go

@@ -40,7 +40,7 @@ func (s *Store) Block(ctx context.Context, blockRoot [32]byte) (interfaces.ReadO
 
 func (s *Store) getBlock(ctx context.Context, blockRoot [32]byte, tx *bolt.Tx) (interfaces.ReadOnlySignedBeaconBlock, error) {
 	if v, ok := s.blockCache.Get(string(blockRoot[:])); v != nil && ok {
-		return v.(interfaces.ReadOnlySignedBeaconBlock), nil
+		return v, nil
 	}
 	// This method allows the caller to pass in its tx if one is already open.
 	// Or if a nil value is used, a transaction will be managed intenally.

beacon-chain/db/kv/kv.go

@@ -13,8 +13,10 @@ import (
 	"github.com/OffchainLabs/prysm/v6/config/features"
 	"github.com/OffchainLabs/prysm/v6/config/params"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
 	"github.com/OffchainLabs/prysm/v6/io/file"
-	"github.com/dgraph-io/ristretto"
+	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
+	"github.com/dgraph-io/ristretto/v2"
 	"github.com/pkg/errors"
 	"github.com/prometheus/client_golang/prometheus"
 	"github.com/prometheus/client_golang/prometheus/promauto"
@@ -86,8 +88,8 @@ var blockedBuckets = [][]byte{
 type Store struct {
 	db                  *bolt.DB
 	databasePath        string
-	blockCache          *ristretto.Cache
-	validatorEntryCache *ristretto.Cache
+	blockCache          *ristretto.Cache[string, interfaces.ReadOnlySignedBeaconBlock]
+	validatorEntryCache *ristretto.Cache[[]byte, *ethpb.Validator]
 	stateSummaryCache   *stateSummaryCache
 	ctx                 context.Context
 }
@@ -110,6 +112,7 @@ var Buckets = [][]byte{
 	lightClientUpdatesBucket,
 	lightClientBootstrapBucket,
 	lightClientSyncCommitteeBucket,
+	stateDiffBucket,
 	// Indices buckets.
 	blockSlotIndicesBucket,
 	stateSlotIndicesBucket,
@@ -156,7 +159,7 @@ func NewKVStore(ctx context.Context, dirPath string, opts ...KVStoreOption) (*St
 		return nil, err
 	}
 	boltDB.AllocSize = boltAllocSize
-	blockCache, err := ristretto.NewCache(&ristretto.Config{
+	blockCache, err := ristretto.NewCache(&ristretto.Config[string, interfaces.ReadOnlySignedBeaconBlock]{
 		NumCounters: 1000,           // number of keys to track frequency of (1000).
 		MaxCost:     BlockCacheSize, // maximum cost of cache (1000 Blocks).
 		BufferItems: 64,             // number of keys per Get buffer.
@@ -165,7 +168,7 @@ func NewKVStore(ctx context.Context, dirPath string, opts ...KVStoreOption) (*St
 		return nil, err
 	}
 
-	validatorCache, err := ristretto.NewCache(&ristretto.Config{
+	validatorCache, err := ristretto.NewCache(&ristretto.Config[[]byte, *ethpb.Validator]{
 		NumCounters: NumOfValidatorEntries, // number of entries in cache (2 Million).
 		MaxCost:     ValidatorEntryMaxCost, // maximum size of the cache (64Mb)
 		BufferItems: 64,                    // number of keys per Get buffer.

beacon-chain/db/kv/schema.go

@@ -16,6 +16,7 @@ var (
 	stateValidatorsBucket = []byte("state-validators")
 	feeRecipientBucket    = []byte("fee-recipient")
 	registrationBucket    = []byte("registration")
+	stateDiffBucket       = []byte("state-diff")
 
 	// Light Client Updates Bucket
 	lightClientUpdatesBucket       = []byte("light-client-updates")

beacon-chain/db/kv/state.go

@@ -744,14 +744,9 @@ func (s *Store) validatorEntries(ctx context.Context, blockRoot [32]byte) ([]*et
 			// get the entry bytes from the cache or from the DB.
 			v, ok := s.validatorEntryCache.Get(key)
 			if ok {
-				valEntry, vType := v.(*ethpb.Validator)
-				if vType {
-					validatorEntries = append(validatorEntries, valEntry)
-					validatorEntryCacheHit.Inc()
-				} else {
-					// this should never happen, but anyway it's good to bail out if one happens.
-					return errors.New("validator cache does not have proper object type")
-				}
+				valEntry := v
+				validatorEntries = append(validatorEntries, valEntry)
+				validatorEntryCacheHit.Inc()
 			} else {
 				// not in cache, so get it from the DB, decode it and add to the entry list.
 				valEntryBytes := valBkt.Get(key)

beacon-chain/db/kv/state_diff.go

@@ -0,0 +1,272 @@
+package kv
+
+import (
+	"context"
+
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/state"
+	"github.com/OffchainLabs/prysm/v6/config/params"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/hdiff"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
+	"github.com/OffchainLabs/prysm/v6/math"
+	"github.com/OffchainLabs/prysm/v6/monitoring/tracing/trace"
+	"github.com/pkg/errors"
+	bolt "go.etcd.io/bbolt"
+)
+
+/*
+	We use a level-based approach to save state diffs. The levels are 0-6, where each level corresponds to an exponent of 2 (exponents[lvl]).
+	The data at level 0 is saved every 2**exponent[0] slots and always contains a full state snapshot that is used as a base for the delta saved at other levels.
+*/
+
+var (
+	anchorCache = make(map[int]state.ReadOnlyBeaconState, len(params.StateHierarchyExponents())) // cache full states at the last node at each level
+)
+
+// SaveStateDiff takes a state and decides between saving a full state snapshot or a diff.
+func (s *Store) SaveStateDiff(ctx context.Context, st state.ReadOnlyBeaconState) error {
+	ctx, span := trace.StartSpan(ctx, "BeaconDB.SaveStateDiff")
+	defer span.End()
+
+	slot := st.Slot()
+	offset, err := s.loadOrInitOffset(slot)
+	if err != nil {
+		return err
+	}
+	if uint64(slot) < offset {
+		return ErrSlotBeforeOffset
+	}
+
+	// Find the level to save the state.
+	lvl := computeLevel(offset, slot)
+	if lvl == -1 {
+		return nil
+	}
+
+	// Save full state if level is 0.
+	if lvl == 0 {
+		return s.saveFullSnapshot(lvl, st)
+	}
+
+	// Get anchor state to compute the diff from.
+	anchorState, err := s.getAnchorState(offset, lvl, slot)
+	if err != nil {
+		return err
+	}
+
+	err = s.saveHdiff(lvl, anchorState, st)
+	if err != nil {
+		return err
+	}
+
+	return nil
+}
+
+// StateDiff retrieves the full state for a given slot.
+func (s *Store) StateDiff(ctx context.Context, slot primitives.Slot) (state.BeaconState, error) {
+	offset, err := s.getOffset()
+	if err != nil {
+		return nil, err
+	}
+	if uint64(slot) < offset {
+		return nil, ErrSlotBeforeOffset
+	}
+
+	snapshot, diffChain, err := s.getBaseAndDiffChain(offset, slot)
+
+	// TODO: apply the diff chain to the snapshot and return the final state.
+
+	return nil, nil
+}
+
+// SaveHdiff computes the diff between the anchor state and the current state and saves it to the database.
+func (s *Store) saveHdiff(lvl int, anchor, st state.ReadOnlyBeaconState) error {
+	slot := uint64(st.Slot())
+	key := makeKey(lvl, slot)
+
+	// TODO: compute actual diff
+	diff := hdiff.HdiffSerialized{}
+
+	err := s.db.Update(func(tx *bolt.Tx) error {
+		bucket := tx.Bucket(stateDiffBucket)
+		if bucket == nil {
+			return bolt.ErrBucketNotFound
+		}
+		// TODO: save the diff bytes per field with suffix
+		buf := make([]byte, len(key)+len("_s"))
+		copy(buf, key)
+		copy(buf[len(key):], "_s")
+		if err := bucket.Put(buf, diff); err != nil {
+			return err
+		}
+		copy(buf[len(key):], "_v")
+		if err := bucket.Put(buf, diff); err != nil {
+			return err
+		}
+		copy(buf[len(key):], "_b")
+		if err := bucket.Put(buf, diff); err != nil {
+			return err
+		}
+		return nil
+	})
+	if err != nil {
+		return err
+	}
+
+	// Save the full state to the cache (if not the last level).
+	if lvl != len(params.StateHierarchyExponents())-1 {
+		anchorCache[lvl] = st
+	}
+
+	return nil
+}
+
+// SaveFullSnapshot saves the full level 0 state snapshot to the database.
+func (s *Store) saveFullSnapshot(lvl int, st state.ReadOnlyBeaconState) error {
+	slot := uint64(st.Slot())
+	key := makeKey(lvl, slot)
+	stateBytes, err := st.MarshalSSZ()
+	// add version key to value
+	enc, err := addKey(st.Version(), stateBytes)
+	if err != nil {
+		return err
+	}
+
+	err = s.db.Update(func(tx *bolt.Tx) error {
+		bucket := tx.Bucket(stateDiffBucket)
+		if bucket == nil {
+			return bolt.ErrBucketNotFound
+		}
+
+		if err := bucket.Put(key, enc); err != nil {
+			return err
+		}
+
+		return nil
+	})
+	if err != nil {
+		return err
+	}
+	// Save the full state to the cache.
+	anchorCache[lvl] = st
+	return nil
+}
+
+func addKey(v int, bytes []byte) ([]byte, error) {
+	key, err := keyForSnapshot(v)
+	if err != nil {
+		return nil, err
+	}
+	enc := make([]byte, len(key)+len(bytes))
+	copy(enc, key)
+	copy(enc[len(key):], bytes)
+	return enc, nil
+}
+
+func (s *Store) getBaseAndDiffChain(offset uint64, slot primitives.Slot) (state.BeaconState, []*hdiff.HdiffSerialized, error) {
+	rel := uint64(slot) - offset
+	lvl := computeLevel(offset, slot)
+	if lvl == -1 {
+		return nil, nil, errors.New("slot not in tree")
+	}
+
+	exponents := params.StateHierarchyExponents()
+
+	baseSpan := math.PowerOf2(exponents[0])
+	baseAnchorSlot := (rel / baseSpan * baseSpan) + offset
+
+	var diffChainIndices []uint64
+	for i := 1; i < lvl; i++ {
+		span := math.PowerOf2(exponents[i])
+		diffSlot := rel / span * span
+		if diffSlot == baseAnchorSlot {
+			continue
+		}
+		diffChainIndices = appendUnique(diffChainIndices, diffSlot+offset)
+	}
+
+	baseSnapshot, err := s.getFullSnapshot(lvl, baseAnchorSlot)
+	if err != nil {
+		return nil, nil, err
+	}
+
+	diffChain := make([]*hdiff.HdiffSerialized, len(diffChainIndices))
+	for _, diffSlot := range diffChainIndices {
+		diff, err := s.getDiff(computeLevel(offset, primitives.Slot(diffSlot)), diffSlot)
+		if err != nil {
+			return nil, nil, err
+		}
+		diffChain = append(diffChain, diff)
+	}
+
+	return baseSnapshot, diffChain, nil
+}
+
+func (s *Store) getDiff(lvl int, slot uint64) (*hdiff.HdiffSerialized, error) {
+	key := makeKey(lvl, slot)
+	var stateDiff []byte
+	var validatorDiff []byte
+	var balancesDiff []byte
+
+	err := s.db.View(func(tx *bolt.Tx) error {
+		bucket := tx.Bucket(stateDiffBucket)
+		if bucket == nil {
+			return bolt.ErrBucketNotFound
+		}
+		buf := make([]byte, len(key)+len("_s"))
+		copy(buf, key)
+		copy(buf[len(key):], "_s")
+		stateDiff = bucket.Get(buf)
+		if stateDiff == nil {
+			return errors.New("state diff not found")
+		}
+		copy(buf[len(key):], "_v")
+		validatorDiff = bucket.Get(buf)
+		if validatorDiff == nil {
+			return errors.New("validator diff not found")
+		}
+		copy(buf[len(key):], "_b")
+		balancesDiff = bucket.Get(buf)
+		if balancesDiff == nil {
+			return errors.New("balances diff not found")
+		}
+		return nil
+	})
+
+	if err != nil {
+		return nil, err
+	}
+
+	return &hdiff.HdiffSerialized{}, nil
+}
+
+func (s *Store) getFullSnapshot(lvl int, slot uint64) (state.BeaconState, error) {
+	key := makeKey(lvl, slot)
+	var enc []byte
+
+	err := s.db.View(func(tx *bolt.Tx) error {
+		bucket := tx.Bucket(stateDiffBucket)
+		if bucket == nil {
+			return bolt.ErrBucketNotFound
+		}
+		enc = bucket.Get(key)
+		if enc == nil {
+			return errors.New("state not found")
+		}
+		return nil
+	})
+
+	if err != nil {
+		return nil, err
+	}
+
+	return s.decodeStateSnapshot(enc)
+}
+
+func appendUnique(s []uint64, v uint64) []uint64 {
+	for _, x := range s {
+		if x == v {
+			return s
+		}
+	}
+	return append(s, v)
+}

beacon-chain/db/kv/state_diff_helpers.go

@@ -0,0 +1,167 @@
+package kv
+
+import (
+	"context"
+	"encoding/binary"
+	"errors"
+	"fmt"
+
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/state"
+	state_native "github.com/OffchainLabs/prysm/v6/beacon-chain/state/state-native"
+	"github.com/OffchainLabs/prysm/v6/config/params"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
+	"github.com/OffchainLabs/prysm/v6/math"
+	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
+	"github.com/OffchainLabs/prysm/v6/runtime/version"
+	"go.etcd.io/bbolt"
+)
+
+var (
+	offsetKey           = []byte("offset")
+	ErrSlotBeforeOffset = errors.New("slot is before root offset")
+)
+
+func makeKey(level int, slot uint64) []byte {
+	buf := make([]byte, 1+8)
+	buf[0] = byte(level)
+	binary.BigEndian.PutUint64(buf[1:], slot)
+	return buf
+}
+
+func (s *Store) getAnchorState(offset uint64, lvl int, slot primitives.Slot) (anchor state.ReadOnlyBeaconState, err error) {
+	if lvl == 0 {
+		return nil, errors.New("no anchor for level 0")
+	}
+
+	relSlot := uint64(slot) - offset
+	prevExp := params.StateHierarchyExponents()[lvl-1]
+	span := math.PowerOf2(prevExp)
+	anchorSlot := primitives.Slot((relSlot / span * span) + offset)
+
+	anchorLvl := computeLevel(offset, anchorSlot)
+	if anchorLvl == -1 {
+		return nil, errors.New("could not compute anchor level")
+	}
+
+	// Check if we have the anchor in cache.
+	anchor, ok := anchorCache[anchorLvl]
+	if ok {
+		return anchor, nil
+	}
+
+	// If not, load it from the database.
+	anchor, err = s.StateDiff(context.Background(), anchorSlot)
+	if err != nil {
+		return nil, err
+	}
+
+	// Save it in the cache.
+	anchorCache[anchorLvl] = anchor
+	return anchor, nil
+}
+
+// ComputeLevel computes the level in the diff tree. Returns -1 in case slot should not be in tree.
+func computeLevel(offset uint64, slot primitives.Slot) int {
+	rel := uint64(slot) - offset
+	for i, exp := range params.StateHierarchyExponents() {
+		span := math.PowerOf2(exp)
+		if rel%span == 0 {
+			return i
+		}
+	}
+	// If rel isn’t on any of the boundaries, we should ignore saving it.
+	return -1
+}
+
+func (s *Store) loadOrInitOffset(slot primitives.Slot) (offset uint64, err error) {
+	return offset, s.db.Update(func(tx *bbolt.Tx) error {
+		bucket := tx.Bucket(stateDiffBucket)
+		if bucket == nil {
+			return bbolt.ErrBucketNotFound
+		}
+
+		offsetBytes := bucket.Get(offsetKey)
+		if offsetBytes != nil {
+			offset = binary.BigEndian.Uint64(offsetBytes)
+			return nil
+		}
+
+		offset = uint64(slot)
+		offsetBytes = make([]byte, 8)
+		binary.BigEndian.PutUint64(offsetBytes, offset)
+		if err := bucket.Put(offsetKey, offsetBytes); err != nil {
+			return err
+		}
+		return nil
+	})
+}
+
+func (s *Store) getOffset() (offset uint64, err error) {
+	return offset, s.db.View(func(tx *bbolt.Tx) error {
+		bucket := tx.Bucket(stateDiffBucket)
+		if bucket == nil {
+			return bbolt.ErrBucketNotFound
+		}
+
+		offsetBytes := bucket.Get(offsetKey)
+		if offsetBytes != nil {
+			offset = binary.BigEndian.Uint64(offsetBytes)
+			return nil
+		}
+		return bbolt.ErrIncompatibleValue
+	})
+}
+
+func keyForSnapshot(v int) ([]byte, error) {
+	switch v {
+	case version.Electra:
+		return ElectraKey, nil
+	case version.Deneb:
+		return denebKey, nil
+	case version.Capella:
+		return capellaKey, nil
+	case version.Bellatrix:
+		return bellatrixKey, nil
+	case version.Altair:
+		return altairKey, nil
+	default:
+		return nil, fmt.Errorf("unsupported version %s", version.String(v))
+	}
+}
+
+func (s *Store) decodeStateSnapshot(enc []byte) (state.BeaconState, error) {
+	switch {
+	case HasElectraKey(enc):
+		var electraState ethpb.BeaconStateElectra
+		if err := electraState.UnmarshalSSZ(enc[len(ElectraKey):]); err != nil {
+			return nil, err
+		}
+		return state_native.InitializeFromProtoElectra(&electraState)
+	case hasDenebKey(enc):
+		var denebState ethpb.BeaconStateDeneb
+		if err := denebState.UnmarshalSSZ(enc[len(denebKey):]); err != nil {
+			return nil, err
+		}
+		return state_native.InitializeFromProtoDeneb(&denebState)
+	case hasCapellaKey(enc):
+		var capellaState ethpb.BeaconStateCapella
+		if err := capellaState.UnmarshalSSZ(enc[len(capellaKey):]); err != nil {
+			return nil, err
+		}
+		return state_native.InitializeFromProtoCapella(&capellaState)
+	case hasBellatrixKey(enc):
+		var bellatrixState ethpb.BeaconStateBellatrix
+		if err := bellatrixState.UnmarshalSSZ(enc[len(bellatrixKey):]); err != nil {
+			return nil, err
+		}
+		return state_native.InitializeFromProtoBellatrix(&bellatrixState)
+	case hasAltairKey(enc):
+		var altairState ethpb.BeaconStateAltair
+		if err := altairState.UnmarshalSSZ(enc[len(altairKey):]); err != nil {
+			return nil, err
+		}
+		return state_native.InitializeFromProtoAltair(&altairState)
+	default:
+		return nil, fmt.Errorf("unsupported encoding %x", enc)
+	}
+}

beacon-chain/db/kv/state_diff_test.go

@@ -0,0 +1,163 @@
+package kv
+
+import (
+	"testing"
+
+	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
+	"github.com/OffchainLabs/prysm/v6/math"
+	"github.com/OffchainLabs/prysm/v6/testing/require"
+	"github.com/OffchainLabs/prysm/v6/testing/util"
+	"go.etcd.io/bbolt"
+)
+
+func TestStateDiff_LoadOrInitOffset(t *testing.T) {
+	db := setupDB(t)
+
+	offset, err := loadOrInitOffset(db, 10)
+	require.NoError(t, err)
+	require.Equal(t, uint64(10), offset)
+
+	offset, err = loadOrInitOffset(db, 20)
+	require.NoError(t, err)
+	require.Equal(t, uint64(10), offset)
+
+	offset, err = loadOrInitOffset(db, 5)
+	require.NoError(t, err)
+	require.Equal(t, uint64(10), offset)
+}
+
+func TestStateDiff_ComputeLevel(t *testing.T) {
+	db := setupDB(t)
+
+	offset, err := loadOrInitOffset(db, 0)
+	require.NoError(t, err)
+	require.Equal(t, uint64(0), offset)
+
+	// 2 ** 21
+	lvl, shouldSave := computeLevel(math.PowerOf2(21))
+	require.Equal(t, 0, lvl)
+	require.Equal(t, true, shouldSave)
+
+	// 2 ** 21 * 3
+	lvl, shouldSave = computeLevel(math.PowerOf2(21) * 3)
+	require.Equal(t, 0, lvl)
+	require.Equal(t, true, shouldSave)
+
+	// 2 ** 18
+	lvl, shouldSave = computeLevel(math.PowerOf2(18))
+	require.Equal(t, 1, lvl)
+	require.Equal(t, true, shouldSave)
+
+	// 2 ** 18 * 3
+	lvl, shouldSave = computeLevel(math.PowerOf2(18) * 3)
+	require.Equal(t, 1, lvl)
+	require.Equal(t, true, shouldSave)
+
+	// 2 ** 16
+	lvl, shouldSave = computeLevel(math.PowerOf2(16))
+	require.Equal(t, 2, lvl)
+	require.Equal(t, true, shouldSave)
+
+	// 2 ** 16 * 3
+	lvl, shouldSave = computeLevel(math.PowerOf2(16) * 3)
+	require.Equal(t, 2, lvl)
+	require.Equal(t, true, shouldSave)
+
+	// 2 ** 13
+	lvl, shouldSave = computeLevel(math.PowerOf2(13))
+	require.Equal(t, 3, lvl)
+	require.Equal(t, true, shouldSave)
+
+	// 2 ** 13 * 3
+	lvl, shouldSave = computeLevel(math.PowerOf2(13) * 3)
+	require.Equal(t, 3, lvl)
+	require.Equal(t, true, shouldSave)
+
+	// 2 ** 11
+	lvl, shouldSave = computeLevel(math.PowerOf2(11))
+	require.Equal(t, 4, lvl)
+	require.Equal(t, true, shouldSave)
+
+	// 2 ** 11 * 3
+	lvl, shouldSave = computeLevel(math.PowerOf2(11) * 3)
+	require.Equal(t, 4, lvl)
+	require.Equal(t, true, shouldSave)
+
+	// 2 ** 9
+	lvl, shouldSave = computeLevel(math.PowerOf2(9))
+	require.Equal(t, 5, lvl)
+	require.Equal(t, true, shouldSave)
+
+	// 2 ** 9 * 3
+	lvl, shouldSave = computeLevel(math.PowerOf2(9) * 3)
+	require.Equal(t, 5, lvl)
+	require.Equal(t, true, shouldSave)
+
+	// 2 ** 5
+	lvl, shouldSave = computeLevel(math.PowerOf2(5))
+	require.Equal(t, 6, lvl)
+	require.Equal(t, true, shouldSave)
+
+	// 2 ** 5 * 3
+	lvl, shouldSave = computeLevel(math.PowerOf2(5) * 3)
+	require.Equal(t, 6, lvl)
+	require.Equal(t, true, shouldSave)
+
+	// 2 ** 7
+	lvl, shouldSave = computeLevel(math.PowerOf2(7))
+	require.Equal(t, 6, lvl)
+	require.Equal(t, true, shouldSave)
+
+	// 2 ** 5 + 1
+	lvl, shouldSave = computeLevel(math.PowerOf2(5) + 1)
+	require.Equal(t, false, shouldSave)
+
+	// 2 ** 5 + 16
+	lvl, shouldSave = computeLevel(math.PowerOf2(5) + 16)
+	require.Equal(t, false, shouldSave)
+
+	// 2 ** 5 + 32
+	lvl, shouldSave = computeLevel(math.PowerOf2(5) + 32)
+	require.Equal(t, true, shouldSave)
+	require.Equal(t, 6, lvl)
+
+}
+
+func TestStateDiff_SaveFullSnapshot(t *testing.T) {
+	db := setupDB(t)
+
+	// Set offset to zero
+	offset, err := loadOrInitOffset(db, 0)
+	require.NoError(t, err)
+	require.Equal(t, uint64(0), offset)
+
+	// Create state with slot 2**21 * 3
+	st, err := util.NewBeaconStateElectra()
+	require.NoError(t, err)
+	slot := primitives.Slot(math.PowerOf2(21))
+	err = st.SetSlot(slot)
+	require.NoError(t, err)
+	stssz, err := st.MarshalSSZ()
+	require.NoError(t, err)
+
+	err = db.SaveStateDiff(nil, st)
+	require.NoError(t, err)
+
+	err = db.db.View(func(tx *bbolt.Tx) error {
+		bucket := tx.Bucket(stateDiffBucket)
+		if bucket == nil {
+			return bbolt.ErrBucketNotFound
+		}
+		for i := 6; i >= 0; i-- {
+			s := bucket.Get(makeKey(i, uint64(slot)))
+			require.NotNil(t, s, "key not found")
+			if i > 0 {
+				require.DeepEqual(t, EmptyNodeMarker, s, "node not marked as empty")
+			} else {
+				require.DeepSSZEqual(t, stssz, s, "retrieved state does not match saved state")
+			}
+		}
+		return nil
+	})
+	require.NoError(t, err)
+}

beacon-chain/db/kv/state_test.go

@@ -321,15 +321,11 @@ func TestState_CanSaveRetrieveValidatorEntriesFromCache(t *testing.T) {
 		hash, hashErr := stateValidators[i].HashTreeRoot()
 		assert.NoError(t, hashErr)
 
-		data, ok := db.validatorEntryCache.Get(string(hash[:]))
+		data, ok := db.validatorEntryCache.Get(hash[:])
 		assert.Equal(t, true, ok)
 		require.NotNil(t, data)
 
-		valEntry, vType := data.(*ethpb.Validator)
-		assert.Equal(t, true, vType)
-		require.NotNil(t, valEntry)
-
-		require.DeepSSZEqual(t, stateValidators[i], valEntry, "validator entry is not matching")
+		require.DeepSSZEqual(t, stateValidators[i], data, "validator entry is not matching")
 	}
 
 	// check if all the validator entries are still intact in the validator entry bucket.
@@ -447,7 +443,7 @@ func TestState_DeleteState(t *testing.T) {
 		assert.NoError(t, hashErr)
 		v, found := db.validatorEntryCache.Get(hash[:])
 		require.Equal(t, false, found)
-		require.Equal(t, nil, v)
+		require.IsNil(t, v)
 	}
 
 	// check if the index of the first state is deleted.

beacon-chain/execution/BUILD.bazel

@@ -25,14 +25,12 @@ go_library(
         "//testing/spectest:__subpackages__",
     ],
     deps = [
-        "//beacon-chain/blockchain/kzg:go_default_library",
         "//beacon-chain/cache:go_default_library",
         "//beacon-chain/cache/depositsnapshot:go_default_library",
         "//beacon-chain/core/altair:go_default_library",
         "//beacon-chain/core/feed:go_default_library",
         "//beacon-chain/core/feed/state:go_default_library",
         "//beacon-chain/core/helpers:go_default_library",
-        "//beacon-chain/core/peerdas:go_default_library",
         "//beacon-chain/core/transition:go_default_library",
         "//beacon-chain/db:go_default_library",
         "//beacon-chain/execution/types:go_default_library",
@@ -99,7 +97,6 @@ go_test(
     embed = [":go_default_library"],
     deps = [
         "//async/event:go_default_library",
-        "//beacon-chain/blockchain/kzg:go_default_library",
         "//beacon-chain/cache/depositsnapshot:go_default_library",
         "//beacon-chain/core/feed:go_default_library",
         "//beacon-chain/core/feed/state:go_default_library",

beacon-chain/execution/engine_client.go

@@ -7,8 +7,6 @@ import (
 	"strings"
 	"time"
 
-	"github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg"
-	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/execution/types"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/verification"
 	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
@@ -46,16 +44,11 @@ var (
 		GetPayloadMethodV3,
 		GetPayloadBodiesByHashV1,
 		GetPayloadBodiesByRangeV1,
-		GetBlobsV1,
 	}
 	electraEngineEndpoints = []string{
 		NewPayloadMethodV4,
 		GetPayloadMethodV4,
 	}
-	fuluEngineEndpoints = []string{
-		GetPayloadMethodV5,
-		GetBlobsV2,
-	}
 )
 
 const (
@@ -80,8 +73,6 @@ const (
 	GetPayloadMethodV3 = "engine_getPayloadV3"
 	// GetPayloadMethodV4 is the get payload method added for electra
 	GetPayloadMethodV4 = "engine_getPayloadV4"
-	// GetPayloadMethodV5 is the get payload method added for fulu
-	GetPayloadMethodV5 = "engine_getPayloadV5"
 	// BlockByHashMethod request string for JSON-RPC.
 	BlockByHashMethod = "eth_getBlockByHash"
 	// BlockByNumberMethod request string for JSON-RPC.
@@ -94,15 +85,8 @@ const (
 	ExchangeCapabilities = "engine_exchangeCapabilities"
 	// GetBlobsV1 request string for JSON-RPC.
 	GetBlobsV1 = "engine_getBlobsV1"
-	// GetBlobsV2 request string for JSON-RPC.
-	GetBlobsV2 = "engine_getBlobsV2"
 	// Defines the seconds before timing out engine endpoints with non-block execution semantics.
-	// TODO: Remove temporarily needed hack since geth takes an input blobs txs with blobs proofs, and
-	// does the heavy lifting of building cells proofs, while normally this is done by the tx sender.
-	// This is a cool hack because it lets the CL to act as if the tx sender actually computed the cells proofs.
-	// The only counter part is the `engine_getPayloadv<x>` takes a lot of time.
-	// defaultEngineTimeout = time.Second
-	defaultEngineTimeout = 2 * time.Second
+	defaultEngineTimeout = time.Second
 )
 
 var errInvalidPayloadBodyResponse = errors.New("engine api payload body response is invalid")
@@ -124,7 +108,6 @@ type Reconstructor interface {
 		ctx context.Context, blindedBlocks []interfaces.ReadOnlySignedBeaconBlock,
 	) ([]interfaces.SignedBeaconBlock, error)
 	ReconstructBlobSidecars(ctx context.Context, block interfaces.ReadOnlySignedBeaconBlock, blockRoot [32]byte, hi func(uint64) bool) ([]blocks.VerifiedROBlob, error)
-	ReconstructDataColumnSidecars(ctx context.Context, block interfaces.ReadOnlySignedBeaconBlock, blockRoot [32]byte) ([]blocks.VerifiedRODataColumn, error)
 }
 
 // EngineCaller defines a client that can interact with an Ethereum
@@ -274,9 +257,6 @@ func (s *Service) ForkchoiceUpdated(
 
 func getPayloadMethodAndMessage(slot primitives.Slot) (string, proto.Message) {
 	pe := slots.ToEpoch(slot)
-	if pe >= params.BeaconConfig().FuluForkEpoch {
-		return GetPayloadMethodV5, &pb.ExecutionBundleFulu{}
-	}
 	if pe >= params.BeaconConfig().ElectraForkEpoch {
 		return GetPayloadMethodV4, &pb.ExecutionBundleElectra{}
 	}
@@ -309,7 +289,7 @@ func (s *Service) GetPayload(ctx context.Context, payloadId [8]byte, slot primit
 	}
 	res, err := blocks.NewGetPayloadResponse(result)
 	if err != nil {
-		return nil, errors.Wrap(err, "new get payload response")
+		return nil, err
 	}
 	return res, nil
 }
@@ -322,9 +302,6 @@ func (s *Service) ExchangeCapabilities(ctx context.Context) ([]string, error) {
 	if params.ElectraEnabled() {
 		supportedEngineEndpoints = append(supportedEngineEndpoints, electraEngineEndpoints...)
 	}
-	if params.FuluEnabled() {
-		supportedEngineEndpoints = append(supportedEngineEndpoints, fuluEngineEndpoints...)
-	}
 	var result []string
 	err := s.rpcClient.CallContext(ctx, &result, ExchangeCapabilities, supportedEngineEndpoints)
 	if err != nil {
@@ -518,10 +495,9 @@ func (s *Service) HeaderByNumber(ctx context.Context, number *big.Int) (*types.H
 func (s *Service) GetBlobs(ctx context.Context, versionedHashes []common.Hash) ([]*pb.BlobAndProof, error) {
 	ctx, span := trace.StartSpan(ctx, "powchain.engine-api-client.GetBlobs")
 	defer span.End()
-
 	// If the execution engine does not support `GetBlobsV1`, return early to prevent encountering an error later.
 	if !s.capabilityCache.has(GetBlobsV1) {
-		return nil, errors.New(fmt.Sprintf("%s is not supported", GetBlobsV1))
+		return nil, nil
 	}
 
 	result := make([]*pb.BlobAndProof, len(versionedHashes))
@@ -529,19 +505,6 @@ func (s *Service) GetBlobs(ctx context.Context, versionedHashes []common.Hash) (
 	return result, handleRPCError(err)
 }
 
-func (s *Service) GetBlobsV2(ctx context.Context, versionedHashes []common.Hash) ([]*pb.BlobAndProofV2, error) {
-	ctx, span := trace.StartSpan(ctx, "powchain.engine-api-client.GetBlobsV2")
-	defer span.End()
-
-	if !s.capabilityCache.has(GetBlobsV2) {
-		return nil, errors.New(fmt.Sprintf("%s is not supported", GetBlobsV2))
-	}
-
-	result := make([]*pb.BlobAndProofV2, len(versionedHashes))
-	err := s.rpcClient.CallContext(ctx, &result, GetBlobsV2, versionedHashes)
-	return result, handleRPCError(err)
-}
-
 // ReconstructFullBlock takes in a blinded beacon block and reconstructs
 // a beacon block with a full execution payload via the engine API.
 func (s *Service) ReconstructFullBlock(
@@ -652,85 +615,6 @@ func (s *Service) ReconstructBlobSidecars(ctx context.Context, block interfaces.
 	return verifiedBlobs, nil
 }
 
-// ReconstructDataColumnSidecars reconstructs the verified data column sidecars for a given beacon block.
-// It retrieves the KZG commitments from the block body, fetches the associated blobs and cell proofs from the EL,
-// and constructs the corresponding verified read-only data column sidecars.
-func (s *Service) ReconstructDataColumnSidecars(ctx context.Context, block interfaces.ReadOnlySignedBeaconBlock, blockRoot [32]byte) ([]blocks.VerifiedRODataColumn, error) {
-	blockBody := block.Block().Body()
-	kzgCommitments, err := blockBody.BlobKzgCommitments()
-	if err != nil {
-		return nil, wrapWithBlockRoot(err, blockRoot, "blob KZG commitments")
-	}
-
-	// Collect KZG hashes for all blobs
-	var kzgHashes []common.Hash
-	for _, commitment := range kzgCommitments {
-		kzgHashes = append(kzgHashes, primitives.ConvertKzgCommitmentToVersionedHash(commitment))
-	}
-
-	// Fetch all blobsAndCellsProofs from EL
-	blobAndProofV2s, err := s.GetBlobsV2(ctx, kzgHashes)
-	if err != nil {
-		return nil, wrapWithBlockRoot(err, blockRoot, "get blobs V2")
-	}
-
-	var cellsAndProofs []kzg.CellsAndProofs
-	for _, blobAndProof := range blobAndProofV2s {
-		if blobAndProof == nil {
-			return nil, wrapWithBlockRoot(errors.New("unable to reconstruct data column sidecars, did not get all blobs from EL"), blockRoot, "")
-		}
-
-		var blob kzg.Blob
-		copy(blob[:], blobAndProof.Blob)
-		cells, err := kzg.ComputeCells(&blob)
-		if err != nil {
-			return nil, wrapWithBlockRoot(err, blockRoot, "could not compute cells")
-		}
-
-		proofs := make([]kzg.Proof, len(blobAndProof.KzgProofs))
-		for i, proof := range blobAndProof.KzgProofs {
-			proofs[i] = kzg.Proof(proof)
-		}
-		cellsAndProofs = append(cellsAndProofs, kzg.CellsAndProofs{
-			Cells:  cells,
-			Proofs: proofs,
-		})
-	}
-
-	header, err := block.Header()
-	if err != nil {
-		return nil, wrapWithBlockRoot(err, blockRoot, "could not get header")
-	}
-
-	kzgCommitmentsInclusionProof, err := blocks.MerkleProofKZGCommitments(blockBody)
-	if err != nil {
-		return nil, wrapWithBlockRoot(err, blockRoot, "could not get Merkle proof for KZG commitments")
-	}
-
-	dataColumnSidecars, err := peerdas.DataColumnSidecarsForReconstruct(kzgCommitments, header, kzgCommitmentsInclusionProof, cellsAndProofs)
-	if err != nil {
-		return nil, wrapWithBlockRoot(err, blockRoot, "could not reconstruct data column sidecars")
-	}
-
-	verifiedRODataColumns := make([]blocks.VerifiedRODataColumn, len(dataColumnSidecars))
-	for i, dataColumnSidecar := range dataColumnSidecars {
-		roDataColumn, err := blocks.NewRODataColumnWithRoot(dataColumnSidecar, blockRoot)
-		if err != nil {
-			return nil, wrapWithBlockRoot(err, blockRoot, "new read-only data column with root")
-		}
-
-		// We trust the execution layer we are connected to, so we can upgrade the read only data column sidecar into a verified one.
-		verifiedRODataColumns[i] = blocks.NewVerifiedRODataColumn(roDataColumn)
-	}
-
-	log.WithFields(logrus.Fields{
-		"root": fmt.Sprintf("%#x", blockRoot),
-		"slot": block.Block().Slot(),
-	}).Debug("Data columns successfully reconstructed from EL")
-
-	return verifiedRODataColumns, nil
-}
-
 func fullPayloadFromPayloadBody(
 	header interfaces.ExecutionData, body *pb.ExecutionPayloadBody, bVersion int,
 ) (interfaces.ExecutionData, error) {
@@ -1018,8 +902,3 @@ func toBlockNumArg(number *big.Int) string {
 	}
 	return hexutil.EncodeBig(number)
 }
-
-// wrapWithBlockRoot returns a new error with the given block root.
-func wrapWithBlockRoot(err error, blockRoot [32]byte, message string) error {
-	return errors.Wrap(err, fmt.Sprintf("%s for block %#x", message, blockRoot))
-}

beacon-chain/execution/engine_client_test.go

@@ -13,7 +13,6 @@ import (
 	"strings"
 	"testing"
 
-	"github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/db/filesystem"
 	mocks "github.com/OffchainLabs/prysm/v6/beacon-chain/execution/testing"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/verification"
@@ -168,7 +167,6 @@ func TestClient_HTTP(t *testing.T) {
 	cfg.CapellaForkEpoch = 1
 	cfg.DenebForkEpoch = 2
 	cfg.ElectraForkEpoch = 3
-	cfg.FuluForkEpoch = 4
 	params.OverrideBeaconConfig(cfg)
 
 	t.Run(GetPayloadMethod, func(t *testing.T) {
@@ -319,11 +317,11 @@ func TestClient_HTTP(t *testing.T) {
 		require.DeepEqual(t, uint64(2), g)
 
 		commitments := [][]byte{bytesutil.PadTo([]byte("commitment1"), fieldparams.BLSPubkeyLength), bytesutil.PadTo([]byte("commitment2"), fieldparams.BLSPubkeyLength)}
-		require.DeepEqual(t, commitments, resp.BlobsBundler.GetKzgCommitments())
+		require.DeepEqual(t, commitments, resp.BlobsBundle.KzgCommitments)
 		proofs := [][]byte{bytesutil.PadTo([]byte("proof1"), fieldparams.BLSPubkeyLength), bytesutil.PadTo([]byte("proof2"), fieldparams.BLSPubkeyLength)}
-		require.DeepEqual(t, proofs, resp.BlobsBundler.GetProofs())
+		require.DeepEqual(t, proofs, resp.BlobsBundle.Proofs)
 		blobs := [][]byte{bytesutil.PadTo([]byte("a"), fieldparams.BlobLength), bytesutil.PadTo([]byte("b"), fieldparams.BlobLength)}
-		require.DeepEqual(t, blobs, resp.BlobsBundler.GetBlobs())
+		require.DeepEqual(t, blobs, resp.BlobsBundle.Blobs)
 	})
 	t.Run(GetPayloadMethodV4, func(t *testing.T) {
 		payloadId := [8]byte{1}
@@ -374,11 +372,11 @@ func TestClient_HTTP(t *testing.T) {
 		require.DeepEqual(t, uint64(2), g)
 
 		commitments := [][]byte{bytesutil.PadTo([]byte("commitment1"), fieldparams.BLSPubkeyLength), bytesutil.PadTo([]byte("commitment2"), fieldparams.BLSPubkeyLength)}
-		require.DeepEqual(t, commitments, resp.BlobsBundler.GetKzgCommitments())
+		require.DeepEqual(t, commitments, resp.BlobsBundle.KzgCommitments)
 		proofs := [][]byte{bytesutil.PadTo([]byte("proof1"), fieldparams.BLSPubkeyLength), bytesutil.PadTo([]byte("proof2"), fieldparams.BLSPubkeyLength)}
-		require.DeepEqual(t, proofs, resp.BlobsBundler.GetProofs())
+		require.DeepEqual(t, proofs, resp.BlobsBundle.Proofs)
 		blobs := [][]byte{bytesutil.PadTo([]byte("a"), fieldparams.BlobLength), bytesutil.PadTo([]byte("b"), fieldparams.BlobLength)}
-		require.DeepEqual(t, blobs, resp.BlobsBundler.GetBlobs())
+		require.DeepEqual(t, blobs, resp.BlobsBundle.Blobs)
 		requests := &pb.ExecutionRequests{
 			Deposits: []*pb.DepositRequest{
 				{
@@ -407,52 +405,7 @@ func TestClient_HTTP(t *testing.T) {
 
 		require.DeepEqual(t, requests, resp.ExecutionRequests)
 	})
-	t.Run(GetPayloadMethodV5, func(t *testing.T) {
-		payloadId := [8]byte{1}
-		want, ok := fix["ExecutionBundleFulu"].(*pb.GetPayloadV5ResponseJson)
-		require.Equal(t, true, ok)
-		srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
-			w.Header().Set("Content-Type", "application/json")
-			defer func() {
-				require.NoError(t, r.Body.Close())
-			}()
-			enc, err := io.ReadAll(r.Body)
-			require.NoError(t, err)
-			jsonRequestString := string(enc)
 
-			reqArg, err := json.Marshal(pb.PayloadIDBytes(payloadId))
-			require.NoError(t, err)
-
-			// We expect the JSON string RPC request contains the right arguments.
-			require.Equal(t, true, strings.Contains(
-				jsonRequestString, string(reqArg),
-			))
-			resp := map[string]interface{}{
-				"jsonrpc": "2.0",
-				"id":      1,
-				"result":  want,
-			}
-			err = json.NewEncoder(w).Encode(resp)
-			require.NoError(t, err)
-		}))
-		defer srv.Close()
-
-		rpcClient, err := rpc.DialHTTP(srv.URL)
-		require.NoError(t, err)
-		defer rpcClient.Close()
-
-		client := &Service{}
-		client.rpcClient = rpcClient
-
-		// We call the RPC method via HTTP and expect a proper result.
-		resp, err := client.GetPayload(ctx, payloadId, 4*params.BeaconConfig().SlotsPerEpoch)
-		require.NoError(t, err)
-		_, ok = resp.BlobsBundler.(*pb.BlobsBundleV2)
-		if !ok {
-			t.Logf("resp.BlobsBundler has unexpected type: %T", resp.BlobsBundler)
-		}
-		require.Equal(t, ok, true)
-	})
 	t.Run(ForkchoiceUpdatedMethod+" VALID status", func(t *testing.T) {
 		forkChoiceState := &pb.ForkchoiceState{
 			HeadBlockHash:      []byte("head"),
@@ -1586,7 +1539,6 @@ func fixtures() map[string]interface{} {
 		"ExecutionPayloadCapellaWithValue":  s.ExecutionPayloadWithValueCapella,
 		"ExecutionPayloadDenebWithValue":    s.ExecutionPayloadWithValueDeneb,
 		"ExecutionBundleElectra":            s.ExecutionBundleElectra,
-		"ExecutionBundleFulu":               s.ExecutionBundleFulu,
 		"ValidPayloadStatus":                s.ValidPayloadStatus,
 		"InvalidBlockHashStatus":            s.InvalidBlockHashStatus,
 		"AcceptedStatus":                    s.AcceptedStatus,
@@ -1822,36 +1774,6 @@ func fixturesStruct() *payloadFixtures {
 			append([]byte{pb.WithdrawalRequestType}, withdrawalRequestBytes...),
 			append([]byte{pb.ConsolidationRequestType}, consolidationRequestBytes...)},
 	}
-	executionBundleFixtureFulu := &pb.GetPayloadV5ResponseJson{
-		ShouldOverrideBuilder: true,
-		ExecutionPayload: &pb.ExecutionPayloadDenebJSON{
-			ParentHash:    &common.Hash{'a'},
-			FeeRecipient:  &common.Address{'b'},
-			StateRoot:     &common.Hash{'c'},
-			ReceiptsRoot:  &common.Hash{'d'},
-			LogsBloom:     &hexutil.Bytes{'e'},
-			PrevRandao:    &common.Hash{'f'},
-			BaseFeePerGas: "0x123",
-			BlockHash:     &common.Hash{'g'},
-			Transactions:  []hexutil.Bytes{{'h'}},
-			Withdrawals:   []*pb.Withdrawal{},
-			BlockNumber:   &hexUint,
-			GasLimit:      &hexUint,
-			GasUsed:       &hexUint,
-			Timestamp:     &hexUint,
-			BlobGasUsed:   &bgu,
-			ExcessBlobGas: &ebg,
-		},
-		BlockValue: "0x11fffffffff",
-		BlobsBundle: &pb.BlobBundleV2JSON{
-			Commitments: []hexutil.Bytes{[]byte("commitment1"), []byte("commitment2")},
-			Proofs:      []hexutil.Bytes{[]byte("proof1"), []byte("proof2")},
-			Blobs:       []hexutil.Bytes{{'a'}, {'b'}},
-		},
-		ExecutionRequests: []hexutil.Bytes{append([]byte{pb.DepositRequestType}, depositRequestBytes...),
-			append([]byte{pb.WithdrawalRequestType}, withdrawalRequestBytes...),
-			append([]byte{pb.ConsolidationRequestType}, consolidationRequestBytes...)},
-	}
 	parent := bytesutil.PadTo([]byte("parentHash"), fieldparams.RootLength)
 	sha3Uncles := bytesutil.PadTo([]byte("sha3Uncles"), fieldparams.RootLength)
 	miner := bytesutil.PadTo([]byte("miner"), fieldparams.FeeRecipientLength)
@@ -1946,7 +1868,6 @@ func fixturesStruct() *payloadFixtures {
 		ExecutionPayloadWithValueCapella:  executionPayloadWithValueFixtureCapella,
 		ExecutionPayloadWithValueDeneb:    executionPayloadWithValueFixtureDeneb,
 		ExecutionBundleElectra:            executionBundleFixtureElectra,
-		ExecutionBundleFulu:               executionBundleFixtureFulu,
 		ValidPayloadStatus:                validStatus,
 		InvalidBlockHashStatus:            inValidBlockHashStatus,
 		AcceptedStatus:                    acceptedStatus,
@@ -1971,7 +1892,6 @@ type payloadFixtures struct {
 	ExecutionPayloadWithValueCapella  *pb.GetPayloadV2ResponseJson
 	ExecutionPayloadWithValueDeneb    *pb.GetPayloadV3ResponseJson
 	ExecutionBundleElectra            *pb.GetPayloadV4ResponseJson
-	ExecutionBundleFulu               *pb.GetPayloadV5ResponseJson
 	ValidPayloadStatus                *pb.PayloadStatus
 	InvalidBlockHashStatus            *pb.PayloadStatus
 	AcceptedStatus                    *pb.PayloadStatus
@@ -2504,7 +2424,7 @@ func TestReconstructBlobSidecars(t *testing.T) {
 	t.Run("get-blobs end point is not supported", func(t *testing.T) {
 		hi := mockSummary(t, []bool{true, true, true, true, true, false})
 		verifiedBlobs, err := client.ReconstructBlobSidecars(ctx, sb, r, hi)
-		require.ErrorContains(t, "engine_getBlobsV1 is not supported", err)
+		require.NoError(t, err)
 		require.Equal(t, 0, len(verifiedBlobs))
 	})
 
@@ -2556,64 +2476,6 @@ func TestReconstructBlobSidecars(t *testing.T) {
 	})
 }
 
-func TestReconstructDataColumnSidecars(t *testing.T) {
-	// Start the trusted setup.
-	err := kzg.Start()
-	require.NoError(t, err)
-
-	// Setup right fork epoch
-	params.SetupTestConfigCleanup(t)
-	cfg := params.BeaconConfig().Copy()
-	cfg.CapellaForkEpoch = 1
-	cfg.DenebForkEpoch = 2
-	cfg.ElectraForkEpoch = 3
-	cfg.FuluForkEpoch = 4
-	params.OverrideBeaconConfig(cfg)
-
-	client := &Service{capabilityCache: &capabilityCache{}}
-	b := util.NewBeaconBlockFulu()
-	b.Block.Slot = 4 * params.BeaconConfig().SlotsPerEpoch
-	kzgCommitments := createRandomKzgCommitments(t, 6)
-	b.Block.Body.BlobKzgCommitments = kzgCommitments
-	r, err := b.Block.HashTreeRoot()
-	require.NoError(t, err)
-	sb, err := blocks.NewSignedBeaconBlock(b)
-	require.NoError(t, err)
-
-	ctx := context.Background()
-
-	t.Run("GetBlobsV2 is not supported", func(t *testing.T) {
-		_, err := client.ReconstructDataColumnSidecars(ctx, sb, r)
-		require.ErrorContains(t, "get blobs V2 for block", err)
-	})
-
-	t.Run("receiving all blobs", func(t *testing.T) {
-		blobMasks := []bool{true, true, true, true, true, true}
-		srv := createBlobServerV2(t, 6, blobMasks)
-		defer srv.Close()
-
-		rpcClient, client := setupRpcClientV2(t, srv.URL, client)
-		defer rpcClient.Close()
-
-		dataColumns, err := client.ReconstructDataColumnSidecars(ctx, sb, r)
-		require.NoError(t, err)
-		require.Equal(t, 128, len(dataColumns))
-	})
-
-	t.Run("missing some blobs", func(t *testing.T) {
-		blobMasks := []bool{false, true, true, true, true, true}
-		srv := createBlobServerV2(t, 6, blobMasks)
-		defer srv.Close()
-
-		rpcClient, client := setupRpcClientV2(t, srv.URL, client)
-		defer rpcClient.Close()
-
-		dataColumns, err := client.ReconstructDataColumnSidecars(ctx, sb, r)
-		require.ErrorContains(t, "unable to reconstruct data column sidecars, did not get all blobs from EL", err)
-		require.Equal(t, 0, len(dataColumns))
-	})
-}
-
 func createRandomKzgCommitments(t *testing.T, num int) [][]byte {
 	kzgCommitments := make([][]byte, num)
 	for i := range kzgCommitments {
@@ -2649,42 +2511,6 @@ func createBlobServer(t *testing.T, numBlobs int, callbackFuncs ...func()) *http
 	}))
 }
 
-func createBlobServerV2(t *testing.T, numBlobs int, blobMasks []bool) *httptest.Server {
-	return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
-		w.Header().Set("Content-Type", "application/json")
-		defer func() {
-			require.NoError(t, r.Body.Close())
-		}()
-
-		require.Equal(t, len(blobMasks), numBlobs)
-
-		blobAndCellProofs := make([]*pb.BlobAndProofV2Json, numBlobs)
-		for i := range blobAndCellProofs {
-			if !blobMasks[i] {
-				continue
-			}
-
-			blobAndCellProofs[i] = &pb.BlobAndProofV2Json{
-				Blob:      []byte("0xblob"),
-				KzgProofs: []hexutil.Bytes{},
-			}
-			for j := 0; j < int(params.BeaconConfig().NumberOfColumns); j++ {
-				cellProof := make([]byte, 48)
-				blobAndCellProofs[i].KzgProofs = append(blobAndCellProofs[i].KzgProofs, cellProof)
-			}
-		}
-
-		respJSON := map[string]interface{}{
-			"jsonrpc": "2.0",
-			"id":      1,
-			"result":  blobAndCellProofs,
-		}
-
-		err := json.NewEncoder(w).Encode(respJSON)
-		require.NoError(t, err)
-	}))
-}
-
 func setupRpcClient(t *testing.T, url string, client *Service) (*rpc.Client, *Service) {
 	rpcClient, err := rpc.DialHTTP(url)
 	require.NoError(t, err)
@@ -2696,12 +2522,6 @@ func setupRpcClient(t *testing.T, url string, client *Service) (*rpc.Client, *Se
 	return rpcClient, client
 }
 
-func setupRpcClientV2(t *testing.T, url string, client *Service) (*rpc.Client, *Service) {
-	rpcClient, client := setupRpcClient(t, url, client)
-	client.capabilityCache = &capabilityCache{capabilities: map[string]interface{}{GetBlobsV2: nil}}
-	return rpcClient, client
-}
-
 func testNewBlobVerifier() verification.NewBlobVerifier {
 	return func(b blocks.ROBlob, reqs []verification.Requirement) verification.BlobVerifier {
 		return &verification.MockBlobVerifier{

beacon-chain/execution/testing/mock_engine_client.go

@@ -38,8 +38,6 @@ type EngineClient struct {
 	ErrGetPayload               error
 	BlobSidecars                []blocks.VerifiedROBlob
 	ErrorBlobSidecars           error
-	DataColumnSidecars          []blocks.VerifiedRODataColumn
-	ErrorDataColumnSidecars     error
 }
 
 // NewPayload --
@@ -115,10 +113,6 @@ func (e *EngineClient) ReconstructBlobSidecars(context.Context, interfaces.ReadO
 	return e.BlobSidecars, e.ErrorBlobSidecars
 }
 
-func (e *EngineClient) ReconstructDataColumnSidecars(ctx context.Context, block interfaces.ReadOnlySignedBeaconBlock, blockRoot [32]byte) ([]blocks.VerifiedRODataColumn, error) {
-	return e.DataColumnSidecars, e.ErrorDataColumnSidecars
-}
-
 // GetTerminalBlockHash --
 func (e *EngineClient) GetTerminalBlockHash(ctx context.Context, transitionTime uint64) ([]byte, bool, error) {
 	ttd := new(big.Int)

beacon-chain/node/BUILD.bazel

@@ -23,7 +23,6 @@ go_library(
         "//beacon-chain/cache:go_default_library",
         "//beacon-chain/cache/depositsnapshot:go_default_library",
         "//beacon-chain/core/light-client:go_default_library",
-        "//beacon-chain/core/peerdas:go_default_library",
         "//beacon-chain/db:go_default_library",
         "//beacon-chain/db/filesystem:go_default_library",
         "//beacon-chain/db/kv:go_default_library",

beacon-chain/node/node.go

@@ -25,7 +25,6 @@ import (
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/cache"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/cache/depositsnapshot"
 	lightclient "github.com/OffchainLabs/prysm/v6/beacon-chain/core/light-client"
-	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/db"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/db/filesystem"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/db/kv"
@@ -87,47 +86,44 @@ type serviceFlagOpts struct {
 // full PoS node. It handles the lifecycle of the entire system and registers
 // services to a service registry.
 type BeaconNode struct {
-	cliCtx                   *cli.Context
-	ctx                      context.Context
-	cancel                   context.CancelFunc
-	services                 *runtime.ServiceRegistry
-	lock                     sync.RWMutex
-	stop                     chan struct{} // Channel to wait for termination notifications.
-	db                       db.Database
-	slasherDB                db.SlasherDatabase
-	attestationCache         *cache.AttestationCache
-	attestationPool          attestations.Pool
-	exitPool                 voluntaryexits.PoolManager
-	slashingsPool            slashings.PoolManager
-	syncCommitteePool        synccommittee.Pool
-	blsToExecPool            blstoexec.PoolManager
-	depositCache             cache.DepositCache
-	trackedValidatorsCache   *cache.TrackedValidatorsCache
-	payloadIDCache           *cache.PayloadIDCache
-	stateFeed                *event.Feed
-	blockFeed                *event.Feed
-	opFeed                   *event.Feed
-	stateGen                 *stategen.State
-	collector                *bcnodeCollector
-	slasherBlockHeadersFeed  *event.Feed
-	slasherAttestationsFeed  *event.Feed
-	finalizedStateAtStartUp  state.BeaconState
-	serviceFlagOpts          *serviceFlagOpts
-	GenesisInitializer       genesis.Initializer
-	CheckpointInitializer    checkpoint.Initializer
-	forkChoicer              forkchoice.ForkChoicer
-	clockWaiter              startup.ClockWaiter
-	BackfillOpts             []backfill.ServiceOption
-	initialSyncComplete      chan struct{}
-	BlobStorage              *filesystem.BlobStorage
-	BlobStorageOptions       []filesystem.BlobStorageOption
-	DataColumnStorage        *filesystem.DataColumnStorage
-	DataColumnStorageOptions []filesystem.DataColumnStorageOption
-	custodyInfo              *peerdas.CustodyInfo
-	verifyInitWaiter         *verification.InitializerWaiter
-	syncChecker              *initialsync.SyncChecker
-	slasherEnabled           bool
-	lcStore                  *lightclient.Store
+	cliCtx                  *cli.Context
+	ctx                     context.Context
+	cancel                  context.CancelFunc
+	services                *runtime.ServiceRegistry
+	lock                    sync.RWMutex
+	stop                    chan struct{} // Channel to wait for termination notifications.
+	db                      db.Database
+	slasherDB               db.SlasherDatabase
+	attestationCache        *cache.AttestationCache
+	attestationPool         attestations.Pool
+	exitPool                voluntaryexits.PoolManager
+	slashingsPool           slashings.PoolManager
+	syncCommitteePool       synccommittee.Pool
+	blsToExecPool           blstoexec.PoolManager
+	depositCache            cache.DepositCache
+	trackedValidatorsCache  *cache.TrackedValidatorsCache
+	payloadIDCache          *cache.PayloadIDCache
+	stateFeed               *event.Feed
+	blockFeed               *event.Feed
+	opFeed                  *event.Feed
+	stateGen                *stategen.State
+	collector               *bcnodeCollector
+	slasherBlockHeadersFeed *event.Feed
+	slasherAttestationsFeed *event.Feed
+	finalizedStateAtStartUp state.BeaconState
+	serviceFlagOpts         *serviceFlagOpts
+	GenesisInitializer      genesis.Initializer
+	CheckpointInitializer   checkpoint.Initializer
+	forkChoicer             forkchoice.ForkChoicer
+	clockWaiter             startup.ClockWaiter
+	BackfillOpts            []backfill.ServiceOption
+	initialSyncComplete     chan struct{}
+	BlobStorage             *filesystem.BlobStorage
+	BlobStorageOptions      []filesystem.BlobStorageOption
+	verifyInitWaiter        *verification.InitializerWaiter
+	syncChecker             *initialsync.SyncChecker
+	slasherEnabled          bool
+	lcStore                 *lightclient.Store
 }
 
 // New creates a new node instance, sets up configuration options, and registers
@@ -165,7 +161,6 @@ func New(cliCtx *cli.Context, cancel context.CancelFunc, opts ...Option) (*Beaco
 		serviceFlagOpts:         &serviceFlagOpts{},
 		initialSyncComplete:     make(chan struct{}),
 		syncChecker:             &initialsync.SyncChecker{},
-		custodyInfo:             &peerdas.CustodyInfo{},
 		slasherEnabled:          cliCtx.Bool(flags.SlasherFlag.Name),
 		lcStore:                 &lightclient.Store{},
 	}
@@ -195,15 +190,6 @@ func New(cliCtx *cli.Context, cancel context.CancelFunc, opts ...Option) (*Beaco
 		beacon.BlobStorage = blobs
 	}
 
-	if beacon.DataColumnStorage == nil {
-		dataColumnStorage, err := filesystem.NewDataColumnStorage(cliCtx.Context, beacon.DataColumnStorageOptions...)
-		if err != nil {
-			return nil, errors.Wrap(err, "new data column storage")
-		}
-
-		beacon.DataColumnStorage = dataColumnStorage
-	}
-
 	bfs, err := startBaseServices(cliCtx, beacon, depositAddress)
 	if err != nil {
 		return nil, errors.Wrap(err, "could not start modules")
@@ -294,9 +280,7 @@ func startBaseServices(cliCtx *cli.Context, beacon *BeaconNode, depositAddress s
 	if err := beacon.startDB(cliCtx, depositAddress); err != nil {
 		return nil, errors.Wrap(err, "could not start DB")
 	}
-
 	beacon.BlobStorage.WarmCache()
-	beacon.DataColumnStorage.WarmCache()
 
 	log.Debugln("Starting Slashing DB")
 	if err := beacon.startSlasherDB(cliCtx); err != nil {
@@ -507,10 +491,6 @@ func (b *BeaconNode) clearDB(clearDB, forceClearDB bool, d *kv.Store, dbPath str
 			return nil, errors.Wrap(err, "could not clear blob storage")
 		}
 
-		if err := b.DataColumnStorage.Clear(); err != nil {
-			return nil, errors.Wrap(err, "could not clear data column storage")
-		}
-
 		d, err = kv.NewKVStore(b.ctx, dbPath)
 		if err != nil {
 			return nil, errors.Wrap(err, "could not create new database")
@@ -717,7 +697,6 @@ func (b *BeaconNode) registerP2P(cliCtx *cli.Context) error {
 		StateNotifier:        b,
 		DB:                   b.db,
 		ClockWaiter:          b.clockWaiter,
-		CustodyInfo:          b.custodyInfo,
 	})
 	if err != nil {
 		return err
@@ -796,11 +775,9 @@ func (b *BeaconNode) registerBlockchainService(fc forkchoice.ForkChoicer, gs *st
 		blockchain.WithClockSynchronizer(gs),
 		blockchain.WithSyncComplete(syncComplete),
 		blockchain.WithBlobStorage(b.BlobStorage),
-		blockchain.WithDataColumnStorage(b.DataColumnStorage),
 		blockchain.WithTrackedValidatorsCache(b.trackedValidatorsCache),
 		blockchain.WithPayloadIDCache(b.payloadIDCache),
 		blockchain.WithSyncChecker(b.syncChecker),
-		blockchain.WithCustodyInfo(b.custodyInfo),
 		blockchain.WithSlasherEnabled(b.slasherEnabled),
 		blockchain.WithLightClientStore(b.lcStore),
 	)
@@ -885,12 +862,10 @@ func (b *BeaconNode) registerSyncService(initialSyncComplete chan struct{}, bFil
 		regularsync.WithInitialSyncComplete(initialSyncComplete),
 		regularsync.WithStateNotifier(b),
 		regularsync.WithBlobStorage(b.BlobStorage),
-		regularsync.WithDataColumnStorage(b.DataColumnStorage),
 		regularsync.WithVerifierWaiter(b.verifyInitWaiter),
 		regularsync.WithAvailableBlocker(bFillStore),
-		regularsync.WithTrackedValidatorsCache(b.trackedValidatorsCache),
-		regularsync.WithCustodyInfo(b.custodyInfo),
 		regularsync.WithSlasherEnabled(b.slasherEnabled),
+		regularsync.WithLightClientStore(b.lcStore),
 	)
 	return b.services.RegisterService(rs)
 }
@@ -914,8 +889,6 @@ func (b *BeaconNode) registerInitialSyncService(complete chan struct{}) error {
 		ClockWaiter:         b.clockWaiter,
 		InitialSyncComplete: complete,
 		BlobStorage:         b.BlobStorage,
-		DataColumnStorage:   b.DataColumnStorage,
-		CustodyInfo:         b.custodyInfo,
 	}, opts...)
 	return b.services.RegisterService(is)
 }
@@ -1010,7 +983,6 @@ func (b *BeaconNode) registerRPCService(router *http.ServeMux) error {
 		FinalizationFetcher:       chainService,
 		BlockReceiver:             chainService,
 		BlobReceiver:              chainService,
-		DataColumnReceiver:        chainService,
 		AttestationReceiver:       chainService,
 		GenesisTimeFetcher:        chainService,
 		GenesisFetcher:            chainService,
@@ -1038,7 +1010,6 @@ func (b *BeaconNode) registerRPCService(router *http.ServeMux) error {
 		Router:                    router,
 		ClockWaiter:               b.clockWaiter,
 		BlobStorage:               b.BlobStorage,
-		DataColumnStorage:         b.DataColumnStorage,
 		TrackedValidatorsCache:    b.trackedValidatorsCache,
 		PayloadIDCache:            b.payloadIDCache,
 		LCStore:                   b.lcStore,
@@ -1061,6 +1032,7 @@ func (b *BeaconNode) registerPrometheusService(_ *cli.Context) error {
 	}
 
 	service := prometheus.NewService(
+		b.cliCtx.Context,
 		fmt.Sprintf("%s:%d", b.cliCtx.String(cmd.MonitoringHostFlag.Name), b.cliCtx.Int(flags.MonitoringPortFlag.Name)),
 		b.services,
 		additionalHandlers...,
@@ -1179,7 +1151,6 @@ func (b *BeaconNode) registerPrunerService(cliCtx *cli.Context) error {
 
 func (b *BeaconNode) RegisterBackfillService(cliCtx *cli.Context, bfs *backfill.Store) error {
 	pa := peers.NewAssigner(b.fetchP2P().Peers(), b.forkChoicer)
-	// TODO: Add backfill for data column storage
 	bf, err := backfill.NewService(cliCtx.Context, bfs, b.BlobStorage, b.clockWaiter, b.fetchP2P(), pa, b.BackfillOpts...)
 	if err != nil {
 		return errors.Wrap(err, "error initializing backfill service")

beacon-chain/node/node_test.go

@@ -54,7 +54,7 @@ func TestNodeClose_OK(t *testing.T) {
 	cmd.ValidatorMonitorIndicesFlag.Value.SetInt(1)
 	ctx, cancel := newCliContextWithCancel(&app, set)
 
-	node, err := New(ctx, cancel, WithBlobStorage(filesystem.NewEphemeralBlobStorage(t)), WithDataColumnStorage(filesystem.NewEphemeralDataColumnStorage(t)))
+	node, err := New(ctx, cancel, WithBlobStorage(filesystem.NewEphemeralBlobStorage(t)))
 	require.NoError(t, err)
 
 	node.Close()
@@ -75,7 +75,7 @@ func TestNodeStart_Ok(t *testing.T) {
 	node, err := New(ctx, cancel, WithBlockchainFlagOptions([]blockchain.Option{}),
 		WithBuilderFlagOptions([]builder.Option{}),
 		WithExecutionChainOptions([]execution.Option{}),
-		WithBlobStorage(filesystem.NewEphemeralBlobStorage(t)), WithDataColumnStorage(filesystem.NewEphemeralDataColumnStorage(t)))
+		WithBlobStorage(filesystem.NewEphemeralBlobStorage(t)))
 	require.NoError(t, err)
 	node.services = &runtime.ServiceRegistry{}
 	go func() {
@@ -99,7 +99,7 @@ func TestNodeStart_SyncChecker(t *testing.T) {
 	node, err := New(ctx, cancel, WithBlockchainFlagOptions([]blockchain.Option{}),
 		WithBuilderFlagOptions([]builder.Option{}),
 		WithExecutionChainOptions([]execution.Option{}),
-		WithBlobStorage(filesystem.NewEphemeralBlobStorage(t)), WithDataColumnStorage(filesystem.NewEphemeralDataColumnStorage(t)))
+		WithBlobStorage(filesystem.NewEphemeralBlobStorage(t)))
 	require.NoError(t, err)
 	go func() {
 		node.Start()
@@ -130,7 +130,7 @@ func TestClearDB(t *testing.T) {
 	context, cancel := newCliContextWithCancel(&app, set)
 	options := []Option{
 		WithExecutionChainOptions([]execution.Option{execution.WithHttpEndpoint(endpoint)}),
-		WithBlobStorage(filesystem.NewEphemeralBlobStorage(t)), WithDataColumnStorage(filesystem.NewEphemeralDataColumnStorage(t)),
+		WithBlobStorage(filesystem.NewEphemeralBlobStorage(t)),
 	}
 	_, err = New(context, cancel, options...)
 	require.NoError(t, err)

beacon-chain/node/options.go

@@ -50,20 +50,3 @@ func WithBlobStorageOptions(opt ...filesystem.BlobStorageOption) Option {
 		return nil
 	}
 }
-
-// WithDataColumnStorage sets the DataColumnStorage backend for the BeaconNode
-func WithDataColumnStorage(bs *filesystem.DataColumnStorage) Option {
-	return func(bn *BeaconNode) error {
-		bn.DataColumnStorage = bs
-		return nil
-	}
-}
-
-// WithDataColumnStorageOptions appends 1 or more filesystem.DataColumnStorageOption on the beacon node,
-// to be used when initializing data column storage.
-func WithDataColumnStorageOptions(opt ...filesystem.DataColumnStorageOption) Option {
-	return func(bn *BeaconNode) error {
-		bn.DataColumnStorageOptions = append(bn.DataColumnStorageOptions, opt...)
-		return nil
-	}
-}