Sort by descending

* fix wording

* changelog

CHANGELOG.md

@@ -4,6 +4,54 @@ All notable changes to this project will be documented in this file.
 
 The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
 
+## [v6.0.1](https://github.com/prysmaticlabs/prysm/compare/v6.0.0...v6.0.1) - 2025-05-02
+
+This release fixes two bugs related to the `payload_attributes` [event stream](https://ethereum.github.io/beacon-APIs/#/Events/eventstream). If you are using or planning to use this endpoint, upgrading to version 6.0.1 is mandatory.  
+Also, a reminder: like other Beacon API endpoints, when a node is syncing, it may return historical data as `finalized` or `head`. Until the node is fully synced to the head of the chain, you should avoid using this data, depending on your application's needs.
+
+We currently recommend against using the `--enable-beacon-rest-api` flag on Mainnet. As you may have noticed, we put a deprecation notice on our gRPC code, in particular on gRPC-related flags. The reason for this is that we want to eventually remove gRPC and have REST HTTP as the standard way of communication between the validator client and the beacon node. That being said, the REST option is still unstable and thus marked as experimental in the flag's description (the flag is `--enable-beacon-rest-api`). Therefore we encourage everyone to keep using gRPC, which is currently the default. It is fine to test the REST option on testnets, but doing it on Mainnet can lead to missing attestations and even missing blocks.
+
+**Updating to v6.0.0 or later is required for Pectra Mainnet!**
+
+This patch release has a few important fixes from v6.0.0. Updating to this release is encouraged.
+
+### Added
+
+- `UpgradeToFulu` spectests. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15190)
+- PeerDAS related KZG wrappers. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15186)
+- Add light client p2p broadcaster functions. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15175)
+- Added immediate broadcasting of proposer slashings when equivocating blocks are detected during block processing. [[PR]](https://github.com/prysmaticlabs/prysm/pull/14693)
+- Added 2 new errors: `HeadStateErr` and `ErrCouldNotVerifyBlockHeader`. [[PR]](https://github.com/prysmaticlabs/prysm/pull/14693)
+- Implement pending consolidations Beacon API endpoint. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15219)
+- PeerDAS: Add needed proto files and corresponding generated code. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15187)
+- Add light client p2p validator and subscriber functions. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15214)
+
+### Changed
+
+- Refactored internal function `reValidateSubscriptions` to `pruneSubscriptions` in `beacon-chain/sync/subscriber.go` for improved clarity, addressing a TODO comment. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15160)
+- Updated geth to v1.15.9. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15216)
+- Removed the slot from `UpdateDuties`. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15223)
+- Update hoodie bootnodes. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15240)
+
+### Fixed
+
+- avoid nondeterministic default fork value when generate genesis. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15151)
+- `UpgradeToFulu`. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15190)
+- fixed underflow with balances in leaking edge case with expected withdrawals. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15191)
+- Fixes our generated ssz files to have the correct package name. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15199)
+- Fixes our blob sidecar by root request lists for electra. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15209)
+- Ensure that the `payload_attributes` event has a consistent view of the head state by passing the head block in the event and using stategen to retrieve the corresponding state. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15213)
+- Pass parent context to update duties when dependent roots change. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15221)
+- Process slots across epoch for payload attribute event. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15228)
+- extend the payload attribute computation deadline to the beginning of the proposal slot. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15230)
+
+### Security
+
+- Fix CVE-2025-22869. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15204)
+- Fix CVE-2025-22870. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15204)
+- Fix CVE-2025-22872. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15204)
+- Fix CVE-2025-30204. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15204)
+
 ## [v6.0.0](https://github.com/prysmaticlabs/prysm/compare/v5.3.2...v6.0.0) - 2025-04-21
 
 This release introduces Mainnet support for the upcoming Electra + Prague (Pectra) fork. The fork is scheduled for mainnet epoch 364032 (May 7, 2025, 10:05:11 UTC). You MUST update Prysm Beacon Node, Prysm Validator Client, and your execution layer client to the Pectra ready release prior to the fork to stay on the correct chain.

WORKSPACE

@@ -255,7 +255,7 @@ filegroup(
     url = "https://github.com/ethereum/EIPs/archive/5480440fe51742ed23342b68cf106cefd427e39d.tar.gz",
 )
 
-consensus_spec_version = "v1.5.0-beta.5"
+consensus_spec_version = "v1.6.0-alpha.0"
 
 bls_test_version = "v0.1.1"
 
@@ -271,7 +271,7 @@ filegroup(
     visibility = ["//visibility:public"],
 )
     """,
-    integrity = "sha256-H+Pt4z+HCVDnEBAv814yvsjR7f5l1IpumjFoTj2XnLE=",
+    integrity = "sha256-W7oKvoM0nAkyitykRxAw6kmCvjYC01IqiNJy0AmCnMM=",
     url = "https://github.com/ethereum/consensus-spec-tests/releases/download/%s/general.tar.gz" % consensus_spec_version,
 )
 
@@ -287,7 +287,7 @@ filegroup(
     visibility = ["//visibility:public"],
 )
     """,
-    integrity = "sha256-Dqiwf5BG7yYyURGf+i87AIdArAyztvcgjoi2kSxrGvo=",
+    integrity = "sha256-ig7/zxomjv6buBWMom4IxAJh3lFJ9+JnY44E7c8ZNP8=",
     url = "https://github.com/ethereum/consensus-spec-tests/releases/download/%s/minimal.tar.gz" % consensus_spec_version,
 )
 
@@ -303,7 +303,7 @@ filegroup(
     visibility = ["//visibility:public"],
 )
     """,
-    integrity = "sha256-xrmsFF243pzXHAjh1EQYKS9gtcwmtHK3wRZDSLlVVRk=",
+    integrity = "sha256-mjx+MkXtPhCNv4c4knLYLIkvIdpF7WTjx/ElvGPQzSo=",
     url = "https://github.com/ethereum/consensus-spec-tests/releases/download/%s/mainnet.tar.gz" % consensus_spec_version,
 )
 
@@ -318,7 +318,7 @@ filegroup(
     visibility = ["//visibility:public"],
 )
     """,
-    integrity = "sha256-c+gGapqifCvFtmtxfhOwieBDO2Syxp13GECWEpWM/Ho=",
+    integrity = "sha256-u0RkIZIeGttb3sInR31mO64aBSwxALqO5SYIPlqEvPo=",
     strip_prefix = "consensus-specs-" + consensus_spec_version[1:],
     url = "https://github.com/ethereum/consensus-specs/archive/refs/tags/%s.tar.gz" % consensus_spec_version,
 )

api/client/builder/client.go

@@ -241,7 +241,7 @@ func (c *Client) GetHeader(ctx context.Context, slot primitives.Slot, parentHash
 		return nil, errors.Wrap(err, "error getting header from builder server")
 	}
 
-	bid, err := c.parseHeaderResponse(data, header)
+	bid, err := c.parseHeaderResponse(data, header, slot)
 	if err != nil {
 		return nil, errors.Wrapf(
 			err,
@@ -254,7 +254,7 @@ func (c *Client) GetHeader(ctx context.Context, slot primitives.Slot, parentHash
 	return bid, nil
 }
 
-func (c *Client) parseHeaderResponse(data []byte, header http.Header) (SignedBid, error) {
+func (c *Client) parseHeaderResponse(data []byte, header http.Header, slot primitives.Slot) (SignedBid, error) {
 	var versionHeader string
 	if c.sszEnabled || header.Get(api.VersionHeader) != "" {
 		versionHeader = header.Get(api.VersionHeader)
@@ -276,7 +276,7 @@ func (c *Client) parseHeaderResponse(data []byte, header http.Header) (SignedBid
 	}
 
 	if ver >= version.Electra {
-		return c.parseHeaderElectra(data)
+		return c.parseHeaderElectra(data, slot)
 	}
 	if ver >= version.Deneb {
 		return c.parseHeaderDeneb(data)
@@ -291,7 +291,7 @@ func (c *Client) parseHeaderResponse(data []byte, header http.Header) (SignedBid
 	return nil, fmt.Errorf("unsupported header version %s", versionHeader)
 }
 
-func (c *Client) parseHeaderElectra(data []byte) (SignedBid, error) {
+func (c *Client) parseHeaderElectra(data []byte, slot primitives.Slot) (SignedBid, error) {
 	if c.sszEnabled {
 		sb := &ethpb.SignedBuilderBidElectra{}
 		if err := sb.UnmarshalSSZ(data); err != nil {
@@ -303,7 +303,7 @@ func (c *Client) parseHeaderElectra(data []byte) (SignedBid, error) {
 	if err := json.Unmarshal(data, hr); err != nil {
 		return nil, errors.Wrap(err, "could not unmarshal ExecHeaderResponseElectra JSON")
 	}
-	p, err := hr.ToProto()
+	p, err := hr.ToProto(slot)
 	if err != nil {
 		return nil, errors.Wrap(err, "could not convert ExecHeaderResponseElectra to proto")
 	}

api/client/builder/client_test.go

@@ -532,7 +532,7 @@ func TestClient_GetHeader(t *testing.T) {
 				require.Equal(t, expectedPath, r.URL.Path)
 				epr := &ExecHeaderResponseElectra{}
 				require.NoError(t, json.Unmarshal([]byte(testExampleHeaderResponseElectra), epr))
-				pro, err := epr.ToProto()
+				pro, err := epr.ToProto(100)
 				require.NoError(t, err)
 				ssz, err := pro.MarshalSSZ()
 				require.NoError(t, err)

api/client/builder/types.go

@@ -1284,8 +1284,8 @@ type ExecHeaderResponseElectra struct {
 }
 
 // ToProto creates a SignedBuilderBidElectra Proto from ExecHeaderResponseElectra.
-func (ehr *ExecHeaderResponseElectra) ToProto() (*eth.SignedBuilderBidElectra, error) {
-	bb, err := ehr.Data.Message.ToProto()
+func (ehr *ExecHeaderResponseElectra) ToProto(slot types.Slot) (*eth.SignedBuilderBidElectra, error) {
+	bb, err := ehr.Data.Message.ToProto(slot)
 	if err != nil {
 		return nil, err
 	}
@@ -1296,13 +1296,14 @@ func (ehr *ExecHeaderResponseElectra) ToProto() (*eth.SignedBuilderBidElectra, e
 }
 
 // ToProto creates a BuilderBidElectra Proto from BuilderBidElectra.
-func (bb *BuilderBidElectra) ToProto() (*eth.BuilderBidElectra, error) {
+func (bb *BuilderBidElectra) ToProto(slot types.Slot) (*eth.BuilderBidElectra, error) {
 	header, err := bb.Header.ToProto()
 	if err != nil {
 		return nil, err
 	}
-	if len(bb.BlobKzgCommitments) > params.BeaconConfig().MaxBlobsPerBlockByVersion(version.Electra) {
-		return nil, fmt.Errorf("blob commitment count %d exceeds the maximum %d", len(bb.BlobKzgCommitments), params.BeaconConfig().MaxBlobsPerBlockByVersion(version.Electra))
+	maxBlobsPerBlock := params.BeaconConfig().MaxBlobsPerBlock(slot)
+	if len(bb.BlobKzgCommitments) > maxBlobsPerBlock {
+		return nil, fmt.Errorf("blob commitment count %d exceeds the maximum %d", len(bb.BlobKzgCommitments), maxBlobsPerBlock)
 	}
 	kzgCommitments := make([][]byte, len(bb.BlobKzgCommitments))
 	for i, commit := range bb.BlobKzgCommitments {

api/client/event/event_stream.go

@@ -14,22 +14,10 @@ import (
 )
 
 const (
-	EventHead                        = "head"
-	EventBlock                       = "block"
-	EventAttestation                 = "attestation"
-	EventVoluntaryExit               = "voluntary_exit"
-	EventBlsToExecutionChange        = "bls_to_execution_change"
-	EventProposerSlashing            = "proposer_slashing"
-	EventAttesterSlashing            = "attester_slashing"
-	EventFinalizedCheckpoint         = "finalized_checkpoint"
-	EventChainReorg                  = "chain_reorg"
-	EventContributionAndProof        = "contribution_and_proof"
-	EventLightClientFinalityUpdate   = "light_client_finality_update"
-	EventLightClientOptimisticUpdate = "light_client_optimistic_update"
-	EventPayloadAttributes           = "payload_attributes"
-	EventBlobSidecar                 = "blob_sidecar"
-	EventError                       = "error"
-	EventConnectionError             = "connection_error"
+	EventHead = "head"
+
+	EventError           = "error"
+	EventConnectionError = "connection_error"
 )
 
 var (

api/server/structs/BUILD.bazel

@@ -57,6 +57,7 @@ go_test(
     deps = [
         "//proto/engine/v1:go_default_library",
         "//proto/prysm/v1alpha1:go_default_library",
+        "//testing/assert:go_default_library",
         "//testing/require:go_default_library",
         "@com_github_ethereum_go_ethereum//common:go_default_library",
         "@com_github_ethereum_go_ethereum//common/hexutil:go_default_library",

api/server/structs/conversions_state.go

@@ -26,10 +26,7 @@ func BeaconStateFromConsensus(st beaconState.BeaconState) (*BeaconState, error)
 	for i, r := range srcSr {
 		sr[i] = hexutil.Encode(r)
 	}
-	srcHr, err := st.HistoricalRoots()
-	if err != nil {
-		return nil, err
-	}
+	srcHr := st.HistoricalRoots()
 	hr := make([]string, len(srcHr))
 	for i, r := range srcHr {
 		hr[i] = hexutil.Encode(r)
@@ -116,10 +113,7 @@ func BeaconStateAltairFromConsensus(st beaconState.BeaconState) (*BeaconStateAlt
 	for i, r := range srcSr {
 		sr[i] = hexutil.Encode(r)
 	}
-	srcHr, err := st.HistoricalRoots()
-	if err != nil {
-		return nil, err
-	}
+	srcHr := st.HistoricalRoots()
 	hr := make([]string, len(srcHr))
 	for i, r := range srcHr {
 		hr[i] = hexutil.Encode(r)
@@ -225,10 +219,7 @@ func BeaconStateBellatrixFromConsensus(st beaconState.BeaconState) (*BeaconState
 	for i, r := range srcSr {
 		sr[i] = hexutil.Encode(r)
 	}
-	srcHr, err := st.HistoricalRoots()
-	if err != nil {
-		return nil, err
-	}
+	srcHr := st.HistoricalRoots()
 	hr := make([]string, len(srcHr))
 	for i, r := range srcHr {
 		hr[i] = hexutil.Encode(r)
@@ -347,10 +338,7 @@ func BeaconStateCapellaFromConsensus(st beaconState.BeaconState) (*BeaconStateCa
 	for i, r := range srcSr {
 		sr[i] = hexutil.Encode(r)
 	}
-	srcHr, err := st.HistoricalRoots()
-	if err != nil {
-		return nil, err
-	}
+	srcHr := st.HistoricalRoots()
 	hr := make([]string, len(srcHr))
 	for i, r := range srcHr {
 		hr[i] = hexutil.Encode(r)
@@ -488,10 +476,7 @@ func BeaconStateDenebFromConsensus(st beaconState.BeaconState) (*BeaconStateDene
 	for i, r := range srcSr {
 		sr[i] = hexutil.Encode(r)
 	}
-	srcHr, err := st.HistoricalRoots()
-	if err != nil {
-		return nil, err
-	}
+	srcHr := st.HistoricalRoots()
 	hr := make([]string, len(srcHr))
 	for i, r := range srcHr {
 		hr[i] = hexutil.Encode(r)
@@ -629,10 +614,7 @@ func BeaconStateElectraFromConsensus(st beaconState.BeaconState) (*BeaconStateEl
 	for i, r := range srcSr {
 		sr[i] = hexutil.Encode(r)
 	}
-	srcHr, err := st.HistoricalRoots()
-	if err != nil {
-		return nil, err
-	}
+	srcHr := st.HistoricalRoots()
 	hr := make([]string, len(srcHr))
 	for i, r := range srcHr {
 		hr[i] = hexutil.Encode(r)
@@ -815,10 +797,7 @@ func BeaconStateFuluFromConsensus(st beaconState.BeaconState) (*BeaconStateFulu,
 	for i, r := range srcSr {
 		sr[i] = hexutil.Encode(r)
 	}
-	srcHr, err := st.HistoricalRoots()
-	if err != nil {
-		return nil, err
-	}
+	srcHr := st.HistoricalRoots()
 	hr := make([]string, len(srcHr))
 	for i, r := range srcHr {
 		hr[i] = hexutil.Encode(r)

api/server/structs/conversions_test.go

@@ -4,7 +4,9 @@ import (
 	"testing"
 
 	eth "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
+	"github.com/OffchainLabs/prysm/v6/testing/assert"
 	"github.com/OffchainLabs/prysm/v6/testing/require"
+	"github.com/ethereum/go-ethereum/common/hexutil"
 )
 
 func TestDepositSnapshotFromConsensus(t *testing.T) {
@@ -102,12 +104,266 @@ func TestProposerSlashing_ToConsensus(t *testing.T) {
 	require.ErrorContains(t, errNilValue.Error(), err)
 }
 
+func TestProposerSlashing_FromConsensus(t *testing.T) {
+	input := []*eth.ProposerSlashing{
+		{
+			Header_1: &eth.SignedBeaconBlockHeader{
+				Header: &eth.BeaconBlockHeader{
+					Slot:          1,
+					ProposerIndex: 2,
+					ParentRoot:    []byte{3},
+					StateRoot:     []byte{4},
+					BodyRoot:      []byte{5},
+				},
+				Signature: []byte{6},
+			},
+			Header_2: &eth.SignedBeaconBlockHeader{
+				Header: &eth.BeaconBlockHeader{
+					Slot:          7,
+					ProposerIndex: 8,
+					ParentRoot:    []byte{9},
+					StateRoot:     []byte{10},
+					BodyRoot:      []byte{11},
+				},
+				Signature: []byte{12},
+			},
+		},
+		{
+			Header_1: &eth.SignedBeaconBlockHeader{
+				Header: &eth.BeaconBlockHeader{
+					Slot:          13,
+					ProposerIndex: 14,
+					ParentRoot:    []byte{15},
+					StateRoot:     []byte{16},
+					BodyRoot:      []byte{17},
+				},
+				Signature: []byte{18},
+			},
+			Header_2: &eth.SignedBeaconBlockHeader{
+				Header: &eth.BeaconBlockHeader{
+					Slot:          19,
+					ProposerIndex: 20,
+					ParentRoot:    []byte{21},
+					StateRoot:     []byte{22},
+					BodyRoot:      []byte{23},
+				},
+				Signature: []byte{24},
+			},
+		},
+	}
+
+	expectedResult := []*ProposerSlashing{
+		{
+			SignedHeader1: &SignedBeaconBlockHeader{
+				Message: &BeaconBlockHeader{
+					Slot:          "1",
+					ProposerIndex: "2",
+					ParentRoot:    hexutil.Encode([]byte{3}),
+					StateRoot:     hexutil.Encode([]byte{4}),
+					BodyRoot:      hexutil.Encode([]byte{5}),
+				},
+				Signature: hexutil.Encode([]byte{6}),
+			},
+			SignedHeader2: &SignedBeaconBlockHeader{
+				Message: &BeaconBlockHeader{
+					Slot:          "7",
+					ProposerIndex: "8",
+					ParentRoot:    hexutil.Encode([]byte{9}),
+					StateRoot:     hexutil.Encode([]byte{10}),
+					BodyRoot:      hexutil.Encode([]byte{11}),
+				},
+				Signature: hexutil.Encode([]byte{12}),
+			},
+		},
+		{
+			SignedHeader1: &SignedBeaconBlockHeader{
+				Message: &BeaconBlockHeader{
+					Slot:          "13",
+					ProposerIndex: "14",
+					ParentRoot:    hexutil.Encode([]byte{15}),
+					StateRoot:     hexutil.Encode([]byte{16}),
+					BodyRoot:      hexutil.Encode([]byte{17}),
+				},
+				Signature: hexutil.Encode([]byte{18}),
+			},
+			SignedHeader2: &SignedBeaconBlockHeader{
+				Message: &BeaconBlockHeader{
+					Slot:          "19",
+					ProposerIndex: "20",
+					ParentRoot:    hexutil.Encode([]byte{21}),
+					StateRoot:     hexutil.Encode([]byte{22}),
+					BodyRoot:      hexutil.Encode([]byte{23}),
+				},
+				Signature: hexutil.Encode([]byte{24}),
+			},
+		},
+	}
+
+	result := ProposerSlashingsFromConsensus(input)
+	assert.DeepEqual(t, expectedResult, result)
+}
+
 func TestAttesterSlashing_ToConsensus(t *testing.T) {
 	a := &AttesterSlashing{Attestation1: nil, Attestation2: nil}
 	_, err := a.ToConsensus()
 	require.ErrorContains(t, errNilValue.Error(), err)
 }
 
+func TestAttesterSlashing_FromConsensus(t *testing.T) {
+	input := []*eth.AttesterSlashing{
+		{
+			Attestation_1: &eth.IndexedAttestation{
+				AttestingIndices: []uint64{1, 2},
+				Data: &eth.AttestationData{
+					Slot:            3,
+					CommitteeIndex:  4,
+					BeaconBlockRoot: []byte{5},
+					Source: &eth.Checkpoint{
+						Epoch: 6,
+						Root:  []byte{7},
+					},
+					Target: &eth.Checkpoint{
+						Epoch: 8,
+						Root:  []byte{9},
+					},
+				},
+				Signature: []byte{10},
+			},
+			Attestation_2: &eth.IndexedAttestation{
+				AttestingIndices: []uint64{11, 12},
+				Data: &eth.AttestationData{
+					Slot:            13,
+					CommitteeIndex:  14,
+					BeaconBlockRoot: []byte{15},
+					Source: &eth.Checkpoint{
+						Epoch: 16,
+						Root:  []byte{17},
+					},
+					Target: &eth.Checkpoint{
+						Epoch: 18,
+						Root:  []byte{19},
+					},
+				},
+				Signature: []byte{20},
+			},
+		},
+		{
+			Attestation_1: &eth.IndexedAttestation{
+				AttestingIndices: []uint64{21, 22},
+				Data: &eth.AttestationData{
+					Slot:            23,
+					CommitteeIndex:  24,
+					BeaconBlockRoot: []byte{25},
+					Source: &eth.Checkpoint{
+						Epoch: 26,
+						Root:  []byte{27},
+					},
+					Target: &eth.Checkpoint{
+						Epoch: 28,
+						Root:  []byte{29},
+					},
+				},
+				Signature: []byte{30},
+			},
+			Attestation_2: &eth.IndexedAttestation{
+				AttestingIndices: []uint64{31, 32},
+				Data: &eth.AttestationData{
+					Slot:            33,
+					CommitteeIndex:  34,
+					BeaconBlockRoot: []byte{35},
+					Source: &eth.Checkpoint{
+						Epoch: 36,
+						Root:  []byte{37},
+					},
+					Target: &eth.Checkpoint{
+						Epoch: 38,
+						Root:  []byte{39},
+					},
+				},
+				Signature: []byte{40},
+			},
+		},
+	}
+
+	expectedResult := []*AttesterSlashing{
+		{
+			Attestation1: &IndexedAttestation{
+				AttestingIndices: []string{"1", "2"},
+				Data: &AttestationData{
+					Slot:            "3",
+					CommitteeIndex:  "4",
+					BeaconBlockRoot: hexutil.Encode([]byte{5}),
+					Source: &Checkpoint{
+						Epoch: "6",
+						Root:  hexutil.Encode([]byte{7}),
+					},
+					Target: &Checkpoint{
+						Epoch: "8",
+						Root:  hexutil.Encode([]byte{9}),
+					},
+				},
+				Signature: hexutil.Encode([]byte{10}),
+			},
+			Attestation2: &IndexedAttestation{
+				AttestingIndices: []string{"11", "12"},
+				Data: &AttestationData{
+					Slot:            "13",
+					CommitteeIndex:  "14",
+					BeaconBlockRoot: hexutil.Encode([]byte{15}),
+					Source: &Checkpoint{
+						Epoch: "16",
+						Root:  hexutil.Encode([]byte{17}),
+					},
+					Target: &Checkpoint{
+						Epoch: "18",
+						Root:  hexutil.Encode([]byte{19}),
+					},
+				},
+				Signature: hexutil.Encode([]byte{20}),
+			},
+		},
+		{
+			Attestation1: &IndexedAttestation{
+				AttestingIndices: []string{"21", "22"},
+				Data: &AttestationData{
+					Slot:            "23",
+					CommitteeIndex:  "24",
+					BeaconBlockRoot: hexutil.Encode([]byte{25}),
+					Source: &Checkpoint{
+						Epoch: "26",
+						Root:  hexutil.Encode([]byte{27}),
+					},
+					Target: &Checkpoint{
+						Epoch: "28",
+						Root:  hexutil.Encode([]byte{29}),
+					},
+				},
+				Signature: hexutil.Encode([]byte{30}),
+			},
+			Attestation2: &IndexedAttestation{
+				AttestingIndices: []string{"31", "32"},
+				Data: &AttestationData{
+					Slot:            "33",
+					CommitteeIndex:  "34",
+					BeaconBlockRoot: hexutil.Encode([]byte{35}),
+					Source: &Checkpoint{
+						Epoch: "36",
+						Root:  hexutil.Encode([]byte{37}),
+					},
+					Target: &Checkpoint{
+						Epoch: "38",
+						Root:  hexutil.Encode([]byte{39}),
+					},
+				},
+				Signature: hexutil.Encode([]byte{40}),
+			},
+		},
+	}
+
+	result := AttesterSlashingsFromConsensus(input)
+	assert.DeepEqual(t, expectedResult, result)
+}
+
 func TestIndexedAttestation_ToConsensus(t *testing.T) {
 	a := &IndexedAttestation{
 		AttestingIndices: []string{"1"},

api/server/structs/other.go

@@ -253,7 +253,7 @@ type PendingDeposit struct {
 }
 
 type PendingPartialWithdrawal struct {
-	Index             string `json:"index"`
+	Index             string `json:"validator_index"`
 	Amount            string `json:"amount"`
 	WithdrawableEpoch string `json:"withdrawable_epoch"`
 }

beacon-chain/blockchain/BUILD.bazel

@@ -163,6 +163,7 @@ go_test(
         "//beacon-chain/operations/slashings:go_default_library",
         "//beacon-chain/operations/voluntaryexits:go_default_library",
         "//beacon-chain/p2p:go_default_library",
+        "//beacon-chain/p2p/testing:go_default_library",
         "//beacon-chain/startup:go_default_library",
         "//beacon-chain/state:go_default_library",
         "//beacon-chain/state/state-native:go_default_library",

beacon-chain/blockchain/kzg/BUILD.bazel

@@ -29,8 +29,8 @@ go_test(
     embed = [":go_default_library"],
     deps = [
         "//consensus-types/blocks:go_default_library",
+        "//crypto/random:go_default_library",
         "//testing/require:go_default_library",
-        "//testing/util:go_default_library",
         "@com_github_crate_crypto_go_kzg_4844//:go_default_library",
     ],
 )

beacon-chain/blockchain/kzg/validation_test.go

@@ -4,8 +4,8 @@ import (
 	"testing"
 
 	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
+	"github.com/OffchainLabs/prysm/v6/crypto/random"
 	"github.com/OffchainLabs/prysm/v6/testing/require"
-	"github.com/OffchainLabs/prysm/v6/testing/util"
 	GoKZG "github.com/crate-crypto/go-kzg-4844"
 )
 
@@ -37,7 +37,7 @@ func TestBytesToAny(t *testing.T) {
 }
 
 func TestGenerateCommitmentAndProof(t *testing.T) {
-	blob := util.GetRandBlob(123)
+	blob := random.GetRandBlob(123)
 	commitment, proof, err := GenerateCommitmentAndProof(blob)
 	require.NoError(t, err)
 	expectedCommitment := GoKZG.KZGCommitment{180, 218, 156, 194, 59, 20, 10, 189, 186, 254, 132, 93, 7, 127, 104, 172, 238, 240, 237, 70, 83, 89, 1, 152, 99, 0, 165, 65, 143, 62, 20, 215, 230, 14, 205, 95, 28, 245, 54, 25, 160, 16, 178, 31, 232, 207, 38, 85}

beacon-chain/blockchain/process_block_helpers.go

@@ -309,6 +309,11 @@ func (s *Service) processLightClientFinalityUpdate(
 		Type: statefeed.LightClientFinalityUpdate,
 		Data: newUpdate,
 	})
+
+	if err = s.cfg.P2p.BroadcastLightClientFinalityUpdate(ctx, newUpdate); err != nil {
+		return errors.Wrap(err, "could not broadcast light client finality update")
+	}
+
 	return nil
 }
 
@@ -358,6 +363,10 @@ func (s *Service) processLightClientOptimisticUpdate(ctx context.Context, signed
 		Data: newUpdate,
 	})
 
+	if err = s.cfg.P2p.BroadcastLightClientOptimisticUpdate(ctx, newUpdate); err != nil {
+		return errors.Wrap(err, "could not broadcast light client optimistic update")
+	}
+
 	return nil
 }
 

beacon-chain/blockchain/process_block_test.go

@@ -24,6 +24,7 @@ import (
 	doublylinkedtree "github.com/OffchainLabs/prysm/v6/beacon-chain/forkchoice/doubly-linked-tree"
 	forkchoicetypes "github.com/OffchainLabs/prysm/v6/beacon-chain/forkchoice/types"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/operations/attestations/kv"
+	mockp2p "github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/testing"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/state"
 	"github.com/OffchainLabs/prysm/v6/config/features"
 	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
@@ -3309,6 +3310,7 @@ func TestProcessLightClientOptimisticUpdate(t *testing.T) {
 	params.OverrideBeaconConfig(beaconCfg)
 
 	s, tr := minimalTestService(t)
+	s.cfg.P2p = &mockp2p.FakeP2P{}
 	ctx := tr.ctx
 
 	testCases := []struct {
@@ -3444,6 +3446,7 @@ func TestProcessLightClientFinalityUpdate(t *testing.T) {
 	params.OverrideBeaconConfig(beaconCfg)
 
 	s, tr := minimalTestService(t)
+	s.cfg.P2p = &mockp2p.FakeP2P{}
 	ctx := tr.ctx
 
 	testCases := []struct {

beacon-chain/blockchain/setup_test.go

@@ -22,6 +22,7 @@ import (
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/startup"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/state/stategen"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
 	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
 	"github.com/OffchainLabs/prysm/v6/testing/require"
 	"google.golang.org/protobuf/proto"
@@ -66,6 +67,16 @@ func (mb *mockBroadcaster) BroadcastBlob(_ context.Context, _ uint64, _ *ethpb.B
 	return nil
 }
 
+func (mb *mockBroadcaster) BroadcastLightClientOptimisticUpdate(_ context.Context, _ interfaces.LightClientOptimisticUpdate) error {
+	mb.broadcastCalled = true
+	return nil
+}
+
+func (mb *mockBroadcaster) BroadcastLightClientFinalityUpdate(_ context.Context, _ interfaces.LightClientFinalityUpdate) error {
+	mb.broadcastCalled = true
+	return nil
+}
+
 func (mb *mockBroadcaster) BroadcastBLSChanges(_ context.Context, _ []*ethpb.SignedBLSToExecutionChange) {
 }
 

beacon-chain/cache/committee.go

@@ -14,6 +14,7 @@ import (
 	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
 	"github.com/OffchainLabs/prysm/v6/container/slice"
 	mathutil "github.com/OffchainLabs/prysm/v6/math"
+	"github.com/OffchainLabs/prysm/v6/monitoring/tracing/trace"
 	lru "github.com/hashicorp/golang-lru"
 	"github.com/prometheus/client_golang/prometheus"
 	"github.com/prometheus/client_golang/prometheus/promauto"
@@ -104,11 +105,16 @@ func (c *CommitteeCache) CompressCommitteeCache() {
 // Committee fetches the shuffled indices by slot and committee index. Every list of indices
 // represent one committee. Returns true if the list exists with slot and committee index. Otherwise returns false, nil.
 func (c *CommitteeCache) Committee(ctx context.Context, slot primitives.Slot, seed [32]byte, index primitives.CommitteeIndex) ([]primitives.ValidatorIndex, error) {
+	ctx, span := trace.StartSpan(ctx, "committeeCache.Committee")
+	defer span.End()
+	span.SetAttributes(trace.Int64Attribute("slot", int64(slot)), trace.Int64Attribute("index", int64(index))) // lint:ignore uintcast -- OK for tracing.
+
 	if err := c.checkInProgress(ctx, seed); err != nil {
 		return nil, err
 	}
 
 	obj, exists := c.CommitteeCache.Get(key(seed))
+	span.SetAttributes(trace.BoolAttribute("cache_hit", exists))
 	if exists {
 		CommitteeCacheHit.Inc()
 	} else {
@@ -157,11 +163,14 @@ func (c *CommitteeCache) AddCommitteeShuffledList(ctx context.Context, committee
 
 // ActiveIndices returns the active indices of a given seed stored in cache.
 func (c *CommitteeCache) ActiveIndices(ctx context.Context, seed [32]byte) ([]primitives.ValidatorIndex, error) {
+	ctx, span := trace.StartSpan(ctx, "committeeCache.ActiveIndices")
+	defer span.End()
+
 	if err := c.checkInProgress(ctx, seed); err != nil {
 		return nil, err
 	}
 	obj, exists := c.CommitteeCache.Get(key(seed))
-
+	span.SetAttributes(trace.BoolAttribute("cache_hit", exists))
 	if exists {
 		CommitteeCacheHit.Inc()
 	} else {

beacon-chain/core/altair/upgrade.go

@@ -67,10 +67,6 @@ func UpgradeToAltair(ctx context.Context, state state.BeaconState) (state.Beacon
 	epoch := time.CurrentEpoch(state)
 
 	numValidators := state.NumValidators()
-	hrs, err := state.HistoricalRoots()
-	if err != nil {
-		return nil, err
-	}
 	s := &ethpb.BeaconStateAltair{
 		GenesisTime:           state.GenesisTime(),
 		GenesisValidatorsRoot: state.GenesisValidatorsRoot(),
@@ -83,7 +79,7 @@ func UpgradeToAltair(ctx context.Context, state state.BeaconState) (state.Beacon
 		LatestBlockHeader:           state.LatestBlockHeader(),
 		BlockRoots:                  state.BlockRoots(),
 		StateRoots:                  state.StateRoots(),
-		HistoricalRoots:             hrs,
+		HistoricalRoots:             state.HistoricalRoots(),
 		Eth1Data:                    state.Eth1Data(),
 		Eth1DataVotes:               state.Eth1DataVotes(),
 		Eth1DepositIndex:            state.Eth1DepositIndex(),

beacon-chain/core/altair/upgrade_test.go

@@ -82,10 +82,8 @@ func TestUpgradeToAltair(t *testing.T) {
 	require.DeepSSZEqual(t, preForkState.LatestBlockHeader(), aState.LatestBlockHeader())
 	require.DeepSSZEqual(t, preForkState.BlockRoots(), aState.BlockRoots())
 	require.DeepSSZEqual(t, preForkState.StateRoots(), aState.StateRoots())
-	r1, err := preForkState.HistoricalRoots()
-	require.NoError(t, err)
-	r2, err := aState.HistoricalRoots()
-	require.NoError(t, err)
+	r1 := preForkState.HistoricalRoots()
+	r2 := aState.HistoricalRoots()
 	require.DeepSSZEqual(t, r1, r2)
 	require.DeepSSZEqual(t, preForkState.Eth1Data(), aState.Eth1Data())
 	require.DeepSSZEqual(t, preForkState.Eth1DataVotes(), aState.Eth1DataVotes())

beacon-chain/core/blocks/payload.go

@@ -12,7 +12,6 @@ import (
 	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
-	"github.com/OffchainLabs/prysm/v6/encoding/bytesutil"
 	"github.com/OffchainLabs/prysm/v6/runtime/version"
 	"github.com/OffchainLabs/prysm/v6/time/slots"
 	"github.com/pkg/errors"
@@ -240,16 +239,3 @@ func verifyBlobCommitmentCount(slot primitives.Slot, body interfaces.ReadOnlyBea
 	}
 	return nil
 }
-
-// GetBlockPayloadHash returns the hash of the execution payload of the block
-func GetBlockPayloadHash(blk interfaces.ReadOnlyBeaconBlock) ([32]byte, error) {
-	var payloadHash [32]byte
-	if IsPreBellatrixVersion(blk.Version()) {
-		return payloadHash, nil
-	}
-	payload, err := blk.Body().Execution()
-	if err != nil {
-		return payloadHash, err
-	}
-	return bytesutil.ToBytes32(payload.BlockHash()), nil
-}

beacon-chain/core/capella/upgrade.go

@@ -42,10 +42,6 @@ func UpgradeToCapella(state state.BeaconState) (state.BeaconState, error) {
 		return nil, err
 	}
 
-	hrs, err := state.HistoricalRoots()
-	if err != nil {
-		return nil, err
-	}
 	s := &ethpb.BeaconStateCapella{
 		GenesisTime:           state.GenesisTime(),
 		GenesisValidatorsRoot: state.GenesisValidatorsRoot(),
@@ -58,7 +54,7 @@ func UpgradeToCapella(state state.BeaconState) (state.BeaconState, error) {
 		LatestBlockHeader:           state.LatestBlockHeader(),
 		BlockRoots:                  state.BlockRoots(),
 		StateRoots:                  state.StateRoots(),
-		HistoricalRoots:             hrs,
+		HistoricalRoots:             state.HistoricalRoots(),
 		Eth1Data:                    state.Eth1Data(),
 		Eth1DataVotes:               state.Eth1DataVotes(),
 		Eth1DepositIndex:            state.Eth1DepositIndex(),

beacon-chain/core/deneb/upgrade.go

@@ -57,10 +57,6 @@ func UpgradeToDeneb(state state.BeaconState) (state.BeaconState, error) {
 	if err != nil {
 		return nil, err
 	}
-	historicalRoots, err := state.HistoricalRoots()
-	if err != nil {
-		return nil, err
-	}
 
 	s := &ethpb.BeaconStateDeneb{
 		GenesisTime:           state.GenesisTime(),
@@ -74,7 +70,7 @@ func UpgradeToDeneb(state state.BeaconState) (state.BeaconState, error) {
 		LatestBlockHeader:           state.LatestBlockHeader(),
 		BlockRoots:                  state.BlockRoots(),
 		StateRoots:                  state.StateRoots(),
-		HistoricalRoots:             historicalRoots,
+		HistoricalRoots:             state.HistoricalRoots(),
 		Eth1Data:                    state.Eth1Data(),
 		Eth1DataVotes:               state.Eth1DataVotes(),
 		Eth1DepositIndex:            state.Eth1DepositIndex(),

beacon-chain/core/deneb/upgrade_test.go

@@ -47,10 +47,8 @@ func TestUpgradeToDeneb(t *testing.T) {
 	require.NoError(t, err)
 	require.DeepSSZEqual(t, make([]uint64, numValidators), s)
 
-	hr1, err := preForkState.HistoricalRoots()
-	require.NoError(t, err)
-	hr2, err := mSt.HistoricalRoots()
-	require.NoError(t, err)
+	hr1 := preForkState.HistoricalRoots()
+	hr2 := mSt.HistoricalRoots()
 	require.DeepEqual(t, hr1, hr2)
 
 	f := mSt.Fork()

beacon-chain/core/electra/upgrade.go

@@ -170,10 +170,6 @@ func UpgradeToElectra(beaconState state.BeaconState) (state.BeaconState, error)
 	if err != nil {
 		return nil, err
 	}
-	historicalRoots, err := beaconState.HistoricalRoots()
-	if err != nil {
-		return nil, err
-	}
 	excessBlobGas, err := payloadHeader.ExcessBlobGas()
 	if err != nil {
 		return nil, err
@@ -223,7 +219,7 @@ func UpgradeToElectra(beaconState state.BeaconState) (state.BeaconState, error)
 		LatestBlockHeader:           beaconState.LatestBlockHeader(),
 		BlockRoots:                  beaconState.BlockRoots(),
 		StateRoots:                  beaconState.StateRoots(),
-		HistoricalRoots:             historicalRoots,
+		HistoricalRoots:             beaconState.HistoricalRoots(),
 		Eth1Data:                    beaconState.Eth1Data(),
 		Eth1DataVotes:               beaconState.Eth1DataVotes(),
 		Eth1DepositIndex:            beaconState.Eth1DepositIndex(),

beacon-chain/core/electra/upgrade_test.go

@@ -77,10 +77,8 @@ func TestUpgradeToElectra(t *testing.T) {
 	require.NoError(t, err)
 	require.DeepSSZEqual(t, make([]uint64, numValidators), s)
 
-	hr1, err := preForkState.HistoricalRoots()
-	require.NoError(t, err)
-	hr2, err := mSt.HistoricalRoots()
-	require.NoError(t, err)
+	hr1 := preForkState.HistoricalRoots()
+	hr2 := mSt.HistoricalRoots()
 	require.DeepEqual(t, hr1, hr2)
 
 	f := mSt.Fork()

beacon-chain/core/epoch/epoch_processing_test.go

@@ -148,8 +148,7 @@ func TestProcessFinalUpdates_CanProcess(t *testing.T) {
 	assert.DeepNotEqual(t, params.BeaconConfig().ZeroHash[:], mix, "latest RANDAO still zero hashes")
 
 	// Verify historical root accumulator was appended.
-	roots, err := newS.HistoricalRoots()
-	require.NoError(t, err)
+	roots := newS.HistoricalRoots()
 	assert.Equal(t, 1, len(roots), "Unexpected slashed balance")
 	currAtt, err := newS.CurrentEpochAttestations()
 	require.NoError(t, err)
@@ -379,8 +378,7 @@ func TestProcessHistoricalDataUpdate(t *testing.T) {
 				return st
 			},
 			verifier: func(st state.BeaconState) {
-				roots, err := st.HistoricalRoots()
-				require.NoError(t, err)
+				roots := st.HistoricalRoots()
 				require.Equal(t, 0, len(roots))
 			},
 		},
@@ -393,8 +391,7 @@ func TestProcessHistoricalDataUpdate(t *testing.T) {
 				return st
 			},
 			verifier: func(st state.BeaconState) {
-				roots, err := st.HistoricalRoots()
-				require.NoError(t, err)
+				roots := st.HistoricalRoots()
 				require.Equal(t, 1, len(roots))
 
 				b := &ethpb.HistoricalBatch{
@@ -431,8 +428,7 @@ func TestProcessHistoricalDataUpdate(t *testing.T) {
 					StateSummaryRoot: sr[:],
 				}
 				require.DeepEqual(t, b, summaries[0])
-				hrs, err := st.HistoricalRoots()
-				require.NoError(t, err)
+				hrs := st.HistoricalRoots()
 				require.DeepEqual(t, hrs, [][]byte{})
 			},
 		},

beacon-chain/core/execution/upgrade.go

@@ -35,10 +35,6 @@ func UpgradeToBellatrix(state state.BeaconState) (state.BeaconState, error) {
 		return nil, err
 	}
 
-	hrs, err := state.HistoricalRoots()
-	if err != nil {
-		return nil, err
-	}
 	s := &ethpb.BeaconStateBellatrix{
 		GenesisTime:           state.GenesisTime(),
 		GenesisValidatorsRoot: state.GenesisValidatorsRoot(),
@@ -51,7 +47,7 @@ func UpgradeToBellatrix(state state.BeaconState) (state.BeaconState, error) {
 		LatestBlockHeader:           state.LatestBlockHeader(),
 		BlockRoots:                  state.BlockRoots(),
 		StateRoots:                  state.StateRoots(),
-		HistoricalRoots:             hrs,
+		HistoricalRoots:             state.HistoricalRoots(),
 		Eth1Data:                    state.Eth1Data(),
 		Eth1DataVotes:               state.Eth1DataVotes(),
 		Eth1DepositIndex:            state.Eth1DepositIndex(),

beacon-chain/core/execution/upgrade_test.go

@@ -24,10 +24,8 @@ func TestUpgradeToBellatrix(t *testing.T) {
 	require.DeepSSZEqual(t, preForkState.LatestBlockHeader(), mSt.LatestBlockHeader())
 	require.DeepSSZEqual(t, preForkState.BlockRoots(), mSt.BlockRoots())
 	require.DeepSSZEqual(t, preForkState.StateRoots(), mSt.StateRoots())
-	r1, err := preForkState.HistoricalRoots()
-	require.NoError(t, err)
-	r2, err := mSt.HistoricalRoots()
-	require.NoError(t, err)
+	r1 := preForkState.HistoricalRoots()
+	r2 := mSt.HistoricalRoots()
 	require.DeepSSZEqual(t, r1, r2)
 	require.DeepSSZEqual(t, preForkState.Eth1Data(), mSt.Eth1Data())
 	require.DeepSSZEqual(t, preForkState.Eth1DataVotes(), mSt.Eth1DataVotes())

beacon-chain/core/fulu/upgrade.go

@@ -57,10 +57,6 @@ func UpgradeToFulu(beaconState state.BeaconState) (state.BeaconState, error) {
 	if err != nil {
 		return nil, err
 	}
-	historicalRoots, err := beaconState.HistoricalRoots()
-	if err != nil {
-		return nil, err
-	}
 	excessBlobGas, err := payloadHeader.ExcessBlobGas()
 	if err != nil {
 		return nil, err
@@ -118,7 +114,7 @@ func UpgradeToFulu(beaconState state.BeaconState) (state.BeaconState, error) {
 		LatestBlockHeader:           beaconState.LatestBlockHeader(),
 		BlockRoots:                  beaconState.BlockRoots(),
 		StateRoots:                  beaconState.StateRoots(),
-		HistoricalRoots:             historicalRoots,
+		HistoricalRoots:             beaconState.HistoricalRoots(),
 		Eth1Data:                    beaconState.Eth1Data(),
 		Eth1DataVotes:               beaconState.Eth1DataVotes(),
 		Eth1DepositIndex:            beaconState.Eth1DepositIndex(),

beacon-chain/core/fulu/upgrade_test.go

@@ -43,10 +43,8 @@ func TestUpgradeToFulu(t *testing.T) {
 	require.DeepSSZEqual(t, preForkState.BlockRoots(), mSt.BlockRoots())
 	require.DeepSSZEqual(t, preForkState.StateRoots(), mSt.StateRoots())
 
-	hr1, err := preForkState.HistoricalRoots()
-	require.NoError(t, err)
-	hr2, err := mSt.HistoricalRoots()
-	require.NoError(t, err)
+	hr1 := preForkState.HistoricalRoots()
+	hr2 := mSt.HistoricalRoots()
 	require.DeepEqual(t, hr1, hr2)
 
 	require.DeepSSZEqual(t, preForkState.Eth1Data(), mSt.Eth1Data())

beacon-chain/core/helpers/attestation.go

@@ -187,9 +187,7 @@ func ValidateAttestationTime(attSlot primitives.Slot, genesisTime time.Time, clo
 // VerifyCheckpointEpoch is within current epoch and previous epoch
 // with respect to current time. Returns true if it's within, false if it's not.
 func VerifyCheckpointEpoch(c *ethpb.Checkpoint, genesis time.Time) bool {
-	now := uint64(prysmTime.Now().Unix())
-	genesisTime := uint64(genesis.Unix())
-	currentSlot := primitives.Slot((now - genesisTime) / params.BeaconConfig().SecondsPerSlot)
+	currentSlot := slots.CurrentSlot(uint64(genesis.Unix()))
 	currentEpoch := slots.ToEpoch(currentSlot)
 
 	var prevEpoch primitives.Epoch

beacon-chain/core/helpers/beacon_committee.go

@@ -18,6 +18,7 @@ import (
 	"github.com/OffchainLabs/prysm/v6/crypto/hash"
 	"github.com/OffchainLabs/prysm/v6/encoding/bytesutil"
 	"github.com/OffchainLabs/prysm/v6/math"
+	"github.com/OffchainLabs/prysm/v6/monitoring/tracing/trace"
 	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
 	"github.com/OffchainLabs/prysm/v6/runtime/version"
 	"github.com/OffchainLabs/prysm/v6/time/slots"
@@ -118,6 +119,9 @@ func attestationCommittees(
 
 // BeaconCommittees returns the list of all beacon committees for a given state at a given slot.
 func BeaconCommittees(ctx context.Context, state state.ReadOnlyBeaconState, slot primitives.Slot) ([][]primitives.ValidatorIndex, error) {
+	ctx, span := trace.StartSpan(ctx, "helpers.BeaconCommittees")
+	defer span.End()
+
 	epoch := slots.ToEpoch(slot)
 	activeCount, err := ActiveValidatorCount(ctx, state, epoch)
 	if err != nil {
@@ -244,6 +248,9 @@ func BeaconCommittee(
 	slot primitives.Slot,
 	committeeIndex primitives.CommitteeIndex,
 ) ([]primitives.ValidatorIndex, error) {
+	ctx, span := trace.StartSpan(ctx, "helpers.BeaconCommittee")
+	defer span.End()
+
 	committee, err := committeeCache.Committee(ctx, slot, seed, committeeIndex)
 	if err != nil {
 		return nil, errors.Wrap(err, "could not interface with committee cache")
@@ -297,6 +304,9 @@ func verifyAssignmentEpoch(epoch primitives.Epoch, state state.BeaconState) erro
 // It verifies the validity of the epoch, then iterates through each slot in the epoch to determine the
 // proposer for that slot and assigns them accordingly.
 func ProposerAssignments(ctx context.Context, state state.BeaconState, epoch primitives.Epoch) (map[primitives.ValidatorIndex][]primitives.Slot, error) {
+	ctx, span := trace.StartSpan(ctx, "helpers.ProposerAssignments")
+	defer span.End()
+
 	// Verify if the epoch is valid for assignment based on the provided state.
 	if err := verifyAssignmentEpoch(epoch, state); err != nil {
 		return nil, err
@@ -345,6 +355,9 @@ func ProposerAssignments(ctx context.Context, state state.BeaconState, epoch pri
 // It retrieves active validator indices, determines the number of committees per slot, and computes
 // assignments for each validator based on their presence in the provided validators slice.
 func CommitteeAssignments(ctx context.Context, state state.BeaconState, epoch primitives.Epoch, validators []primitives.ValidatorIndex) (map[primitives.ValidatorIndex]*CommitteeAssignment, error) {
+	ctx, span := trace.StartSpan(ctx, "helpers.CommitteeAssignments")
+	defer span.End()
+
 	// Verify if the epoch is valid for assignment based on the provided state.
 	if err := verifyAssignmentEpoch(epoch, state); err != nil {
 		return nil, err
@@ -432,6 +445,9 @@ func CommitteeIndices(committeeBits bitfield.Bitfield) []primitives.CommitteeInd
 // UpdateCommitteeCache gets called at the beginning of every epoch to cache the committee shuffled indices
 // list with committee index and epoch number. It caches the shuffled indices for the input epoch.
 func UpdateCommitteeCache(ctx context.Context, state state.ReadOnlyBeaconState, e primitives.Epoch) error {
+	ctx, span := trace.StartSpan(ctx, "committeeCache.UpdateCommitteeCache")
+	defer span.End()
+
 	seed, err := Seed(state, e, params.BeaconConfig().DomainBeaconAttester)
 	if err != nil {
 		return err

beacon-chain/core/helpers/validators.go

@@ -102,6 +102,9 @@ func checkValidatorSlashable(activationEpoch, withdrawableEpoch primitives.Epoch
 //	  """
 //	  return [ValidatorIndex(i) for i, v in enumerate(state.validators) if is_active_validator(v, epoch)]
 func ActiveValidatorIndices(ctx context.Context, s state.ReadOnlyBeaconState, epoch primitives.Epoch) ([]primitives.ValidatorIndex, error) {
+	ctx, span := trace.StartSpan(ctx, "helpers.ActiveValidatorIndices")
+	defer span.End()
+
 	seed, err := Seed(s, epoch, params.BeaconConfig().DomainBeaconAttester)
 	if err != nil {
 		return nil, errors.Wrap(err, "could not get seed")

beacon-chain/core/peerdas/BUILD.bazel

@@ -0,0 +1,71 @@
+load("@prysm//tools/go:def.bzl", "go_library", "go_test")
+
+go_library(
+    name = "go_default_library",
+    srcs = [
+        "das_core.go",
+        "info.go",
+        "metrics.go",
+        "p2p_interface.go",
+        "reconstruction.go",
+        "util.go",
+        "validator.go",
+    ],
+    importpath = "github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas",
+    visibility = ["//visibility:public"],
+    deps = [
+        "//beacon-chain/blockchain/kzg:go_default_library",
+        "//beacon-chain/state:go_default_library",
+        "//cmd/beacon-chain/flags:go_default_library",
+        "//config/fieldparams:go_default_library",
+        "//config/params:go_default_library",
+        "//consensus-types/blocks:go_default_library",
+        "//consensus-types/interfaces:go_default_library",
+        "//consensus-types/primitives:go_default_library",
+        "//container/trie:go_default_library",
+        "//crypto/hash:go_default_library",
+        "//encoding/bytesutil:go_default_library",
+        "//proto/prysm/v1alpha1:go_default_library",
+        "//runtime/version:go_default_library",
+        "@com_github_ethereum_go_ethereum//p2p/enode:go_default_library",
+        "@com_github_ethereum_go_ethereum//p2p/enr:go_default_library",
+        "@com_github_hashicorp_golang_lru//:go_default_library",
+        "@com_github_holiman_uint256//:go_default_library",
+        "@com_github_pkg_errors//:go_default_library",
+        "@com_github_prometheus_client_golang//prometheus:go_default_library",
+        "@com_github_prometheus_client_golang//prometheus/promauto:go_default_library",
+        "@org_golang_x_sync//errgroup:go_default_library",
+    ],
+)
+
+go_test(
+    name = "go_default_test",
+    srcs = [
+        "das_core_test.go",
+        "info_test.go",
+        "p2p_interface_test.go",
+        "reconstruction_test.go",
+        "utils_test.go",
+        "validator_test.go",
+    ],
+    deps = [
+        ":go_default_library",
+        "//beacon-chain/blockchain/kzg:go_default_library",
+        "//beacon-chain/state/state-native:go_default_library",
+        "//cmd/beacon-chain/flags:go_default_library",
+        "//config/fieldparams:go_default_library",
+        "//config/params:go_default_library",
+        "//consensus-types/blocks:go_default_library",
+        "//consensus-types/primitives:go_default_library",
+        "//proto/engine/v1:go_default_library",
+        "//proto/prysm/v1alpha1:go_default_library",
+        "//testing/require:go_default_library",
+        "//testing/util:go_default_library",
+        "@com_github_consensys_gnark_crypto//ecc/bls12-381/fr:go_default_library",
+        "@com_github_crate_crypto_go_kzg_4844//:go_default_library",
+        "@com_github_ethereum_go_ethereum//p2p/enode:go_default_library",
+        "@com_github_ethereum_go_ethereum//p2p/enr:go_default_library",
+        "@com_github_pkg_errors//:go_default_library",
+        "@com_github_sirupsen_logrus//:go_default_library",
+    ],
+)

beacon-chain/core/peerdas/das_core.go

@@ -0,0 +1,402 @@
+package peerdas
+
+import (
+	"encoding/binary"
+	"math"
+	"slices"
+	"time"
+
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg"
+	"github.com/OffchainLabs/prysm/v6/config/params"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
+	"github.com/OffchainLabs/prysm/v6/crypto/hash"
+	"github.com/OffchainLabs/prysm/v6/encoding/bytesutil"
+	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
+	"github.com/ethereum/go-ethereum/p2p/enode"
+	"github.com/holiman/uint256"
+	"github.com/pkg/errors"
+)
+
+var (
+	// Custom errors
+	ErrCustodyGroupTooLarge           = errors.New("custody group too large")
+	ErrCustodyGroupCountTooLarge      = errors.New("custody group count too large")
+	ErrMismatchSize                   = errors.New("mismatch in the number of blob KZG commitments and cellsAndProofs")
+	errWrongComputedCustodyGroupCount = errors.New("wrong computed custody group count, should never happen")
+
+	// maxUint256 is the maximum value of an uint256.
+	maxUint256 = &uint256.Int{math.MaxUint64, math.MaxUint64, math.MaxUint64, math.MaxUint64}
+)
+
+type CustodyType int
+
+const (
+	Target CustodyType = iota
+	Actual
+)
+
+// CustodyGroups computes the custody groups the node should participate in for custody.
+// https://github.com/ethereum/consensus-specs/blob/v1.5.0-beta.5/specs/fulu/das-core.md#get_custody_groups
+func CustodyGroups(nodeId enode.ID, custodyGroupCount uint64) ([]uint64, error) {
+	numberOfCustodyGroup := params.BeaconConfig().NumberOfCustodyGroups
+
+	// Check if the custody group count is larger than the number of custody groups.
+	if custodyGroupCount > numberOfCustodyGroup {
+		return nil, ErrCustodyGroupCountTooLarge
+	}
+
+	// Shortcut if all custody groups are needed.
+	if custodyGroupCount == numberOfCustodyGroup {
+		custodyGroups := make([]uint64, 0, numberOfCustodyGroup)
+		for i := range numberOfCustodyGroup {
+			custodyGroups = append(custodyGroups, i)
+		}
+
+		return custodyGroups, nil
+	}
+
+	one := uint256.NewInt(1)
+
+	custodyGroupsMap := make(map[uint64]bool, custodyGroupCount)
+	custodyGroups := make([]uint64, 0, custodyGroupCount)
+	for currentId := new(uint256.Int).SetBytes(nodeId.Bytes()); uint64(len(custodyGroups)) < custodyGroupCount; {
+		// Convert to big endian bytes.
+		currentIdBytesBigEndian := currentId.Bytes32()
+
+		// Convert to little endian.
+		currentIdBytesLittleEndian := bytesutil.ReverseByteOrder(currentIdBytesBigEndian[:])
+
+		// Hash the result.
+		hashedCurrentId := hash.Hash(currentIdBytesLittleEndian)
+
+		// Get the custody group ID.
+		custodyGroup := binary.LittleEndian.Uint64(hashedCurrentId[:8]) % numberOfCustodyGroup
+
+		// Add the custody group to the map.
+		if !custodyGroupsMap[custodyGroup] {
+			custodyGroupsMap[custodyGroup] = true
+			custodyGroups = append(custodyGroups, custodyGroup)
+		}
+
+		if currentId.Cmp(maxUint256) == 0 {
+			// Overflow prevention.
+			currentId = uint256.NewInt(0)
+		} else {
+			// Increment the current ID.
+			currentId.Add(currentId, one)
+		}
+
+		// Sort the custody groups.
+		slices.Sort[[]uint64](custodyGroups)
+	}
+
+	// Final check.
+	if uint64(len(custodyGroups)) != custodyGroupCount {
+		return nil, errWrongComputedCustodyGroupCount
+	}
+
+	return custodyGroups, nil
+}
+
+// ComputeColumnsForCustodyGroup computes the columns for a given custody group.
+// https://github.com/ethereum/consensus-specs/blob/v1.5.0-beta.5/specs/fulu/das-core.md#compute_columns_for_custody_group
+func ComputeColumnsForCustodyGroup(custodyGroup uint64) ([]uint64, error) {
+	beaconConfig := params.BeaconConfig()
+	numberOfCustodyGroup := beaconConfig.NumberOfCustodyGroups
+
+	if custodyGroup >= numberOfCustodyGroup {
+		return nil, ErrCustodyGroupTooLarge
+	}
+
+	numberOfColumns := beaconConfig.NumberOfColumns
+
+	columnsPerGroup := numberOfColumns / numberOfCustodyGroup
+
+	columns := make([]uint64, 0, columnsPerGroup)
+	for i := range columnsPerGroup {
+		column := numberOfCustodyGroup*i + custodyGroup
+		columns = append(columns, column)
+	}
+
+	return columns, nil
+}
+
+// DataColumnSidecars computes the data column sidecars from the signed block, cells and cell proofs.
+// The returned value contains pointers to function parameters.
+// (If the caller alterates `cellsAndProofs` afterwards, the returned value will be modified as well.)
+// https://github.com/ethereum/consensus-specs/blob/v1.5.0-beta.3/specs/fulu/das-core.md#get_data_column_sidecars
+func DataColumnSidecars(signedBlock interfaces.ReadOnlySignedBeaconBlock, cellsAndProofs []kzg.CellsAndProofs) ([]*ethpb.DataColumnSidecar, error) {
+	if signedBlock == nil || signedBlock.IsNil() || len(cellsAndProofs) == 0 {
+		return nil, nil
+	}
+
+	block := signedBlock.Block()
+	blockBody := block.Body()
+	blobKzgCommitments, err := blockBody.BlobKzgCommitments()
+	if err != nil {
+		return nil, errors.Wrap(err, "blob KZG commitments")
+	}
+
+	if len(blobKzgCommitments) != len(cellsAndProofs) {
+		return nil, ErrMismatchSize
+	}
+
+	signedBlockHeader, err := signedBlock.Header()
+	if err != nil {
+		return nil, errors.Wrap(err, "signed block header")
+	}
+
+	kzgCommitmentsInclusionProof, err := blocks.MerkleProofKZGCommitments(blockBody)
+	if err != nil {
+		return nil, errors.Wrap(err, "merkle proof ZKG commitments")
+	}
+
+	dataColumnSidecars, err := DataColumnsSidecarsFromItems(signedBlockHeader, blobKzgCommitments, kzgCommitmentsInclusionProof, cellsAndProofs)
+	if err != nil {
+		return nil, errors.Wrap(err, "data column sidecars from items")
+	}
+
+	return dataColumnSidecars, nil
+}
+
+// DataColumnsSidecarsFromItems computes the data column sidecars from the signed block header, the blob KZG commiments,
+// the KZG commitment includion proofs and cells and cell proofs.
+// The returned value contains pointers to function parameters.
+// (If the caller alterates input parameters afterwards, the returned value will be modified as well.)
+func DataColumnsSidecarsFromItems(
+	signedBlockHeader *ethpb.SignedBeaconBlockHeader,
+	blobKzgCommitments [][]byte,
+	kzgCommitmentsInclusionProof [][]byte,
+	cellsAndProofs []kzg.CellsAndProofs,
+) ([]*ethpb.DataColumnSidecar, error) {
+	start := time.Now()
+	if len(blobKzgCommitments) != len(cellsAndProofs) {
+		return nil, ErrMismatchSize
+	}
+
+	numberOfColumns := params.BeaconConfig().NumberOfColumns
+
+	blobsCount := len(cellsAndProofs)
+	sidecars := make([]*ethpb.DataColumnSidecar, 0, numberOfColumns)
+	for columnIndex := range numberOfColumns {
+		column := make([]kzg.Cell, 0, blobsCount)
+		kzgProofOfColumn := make([]kzg.Proof, 0, blobsCount)
+
+		for rowIndex := range blobsCount {
+			cellsForRow := cellsAndProofs[rowIndex].Cells
+			proofsForRow := cellsAndProofs[rowIndex].Proofs
+
+			cell := cellsForRow[columnIndex]
+			column = append(column, cell)
+
+			kzgProof := proofsForRow[columnIndex]
+			kzgProofOfColumn = append(kzgProofOfColumn, kzgProof)
+		}
+
+		columnBytes := make([][]byte, 0, blobsCount)
+		for i := range column {
+			columnBytes = append(columnBytes, column[i][:])
+		}
+
+		kzgProofOfColumnBytes := make([][]byte, 0, blobsCount)
+		for _, kzgProof := range kzgProofOfColumn {
+			kzgProofOfColumnBytes = append(kzgProofOfColumnBytes, kzgProof[:])
+		}
+
+		sidecar := &ethpb.DataColumnSidecar{
+			Index:                        columnIndex,
+			Column:                       columnBytes,
+			KzgCommitments:               blobKzgCommitments,
+			KzgProofs:                    kzgProofOfColumnBytes,
+			SignedBlockHeader:            signedBlockHeader,
+			KzgCommitmentsInclusionProof: kzgCommitmentsInclusionProof,
+		}
+
+		sidecars = append(sidecars, sidecar)
+	}
+
+	dataColumnComputationTime.Observe(float64(time.Since(start).Milliseconds()))
+	return sidecars, nil
+}
+
+// ComputeCustodyGroupForColumn computes the custody group for a given column.
+// It is the reciprocal function of ComputeColumnsForCustodyGroup.
+func ComputeCustodyGroupForColumn(columnIndex uint64) (uint64, error) {
+	beaconConfig := params.BeaconConfig()
+	numberOfColumns := beaconConfig.NumberOfColumns
+	numberOfCustodyGroups := beaconConfig.NumberOfCustodyGroups
+
+	if columnIndex >= numberOfColumns {
+		return 0, ErrIndexTooLarge
+	}
+
+	return columnIndex % numberOfCustodyGroups, nil
+}
+
+// Blobs extract blobs from `dataColumnsSidecar`.
+// This can be seen as the reciprocal function of DataColumnSidecars.
+// `dataColumnsSidecar` needs to contain the datacolumns corresponding to the non-extended matrix,
+// else an error will be returned.
+// (`dataColumnsSidecar` can contain extra columns, but they will be ignored.)
+func Blobs(indices map[uint64]bool, dataColumnsSidecar []*ethpb.DataColumnSidecar) ([]*blocks.VerifiedROBlob, error) {
+	numberOfColumns := params.BeaconConfig().NumberOfColumns
+
+	// Compute the number of needed columns, including the number of columns is odd case.
+	neededColumnCount := (numberOfColumns + 1) / 2
+
+	// Check if all needed columns are present.
+	sliceIndexFromColumnIndex := make(map[uint64]int, len(dataColumnsSidecar))
+	for i := range dataColumnsSidecar {
+		dataColumnSideCar := dataColumnsSidecar[i]
+		index := dataColumnSideCar.Index
+
+		if index < neededColumnCount {
+			sliceIndexFromColumnIndex[index] = i
+		}
+	}
+
+	actualColumnCount := uint64(len(sliceIndexFromColumnIndex))
+
+	// Get missing columns.
+	if actualColumnCount < neededColumnCount {
+		var missingColumnsSlice []uint64
+
+		for i := range neededColumnCount {
+			if _, ok := sliceIndexFromColumnIndex[i]; !ok {
+				missingColumnsSlice = append(missingColumnsSlice, i)
+			}
+		}
+
+		slices.Sort[[]uint64](missingColumnsSlice)
+		return nil, errors.Errorf("some columns are missing: %v", missingColumnsSlice)
+	}
+
+	// It is safe to retrieve the first column since we already checked that `dataColumnsSidecar` is not empty.
+	firstDataColumnSidecar := dataColumnsSidecar[0]
+
+	blobCount := uint64(len(firstDataColumnSidecar.Column))
+
+	// Check all colums have te same length.
+	for i := range dataColumnsSidecar {
+		if uint64(len(dataColumnsSidecar[i].Column)) != blobCount {
+			return nil, errors.Errorf("mismatch in the length of the data columns, expected %d, got %d", blobCount, len(dataColumnsSidecar[i].Column))
+		}
+	}
+
+	// Reconstruct verified RO blobs from columns.
+	verifiedROBlobs := make([]*blocks.VerifiedROBlob, 0, blobCount)
+
+	// Populate and filter indices.
+	indicesSlice := populateAndFilterIndices(indices, blobCount)
+
+	for _, blobIndex := range indicesSlice {
+		var blob kzg.Blob
+
+		// Compute the content of the blob.
+		for columnIndex := range neededColumnCount {
+			sliceIndex, ok := sliceIndexFromColumnIndex[columnIndex]
+			if !ok {
+				return nil, errors.Errorf("missing column %d, this should never happen", columnIndex)
+			}
+
+			dataColumnSideCar := dataColumnsSidecar[sliceIndex]
+			cell := dataColumnSideCar.Column[blobIndex]
+
+			for i := range cell {
+				blob[columnIndex*kzg.BytesPerCell+uint64(i)] = cell[i]
+			}
+		}
+
+		// Retrieve the blob KZG commitment.
+		blobKZGCommitment := kzg.Commitment(firstDataColumnSidecar.KzgCommitments[blobIndex])
+
+		// Compute the blob KZG proof.
+		blobKzgProof, err := kzg.ComputeBlobKZGProof(&blob, blobKZGCommitment)
+		if err != nil {
+			return nil, errors.Wrap(err, "compute blob KZG proof")
+		}
+
+		blobSidecar := &ethpb.BlobSidecar{
+			Index:                    blobIndex,
+			Blob:                     blob[:],
+			KzgCommitment:            blobKZGCommitment[:],
+			KzgProof:                 blobKzgProof[:],
+			SignedBlockHeader:        firstDataColumnSidecar.SignedBlockHeader,
+			CommitmentInclusionProof: firstDataColumnSidecar.KzgCommitmentsInclusionProof,
+		}
+
+		roBlob, err := blocks.NewROBlob(blobSidecar)
+		if err != nil {
+			return nil, errors.Wrap(err, "new RO blob")
+		}
+
+		verifiedROBlob := blocks.NewVerifiedROBlob(roBlob)
+		verifiedROBlobs = append(verifiedROBlobs, &verifiedROBlob)
+	}
+
+	return verifiedROBlobs, nil
+}
+
+// CustodyGroupSamplingSize returns the number of custody groups the node should sample from.
+// https://github.com/ethereum/consensus-specs/blob/v1.5.0-beta.5/specs/fulu/das-core.md#custody-sampling
+func (custodyInfo *CustodyInfo) CustodyGroupSamplingSize(ct CustodyType) uint64 {
+	custodyGroupCount := custodyInfo.TargetGroupCount.Get()
+
+	if ct == Actual {
+		custodyGroupCount = custodyInfo.ActualGroupCount()
+	}
+
+	samplesPerSlot := params.BeaconConfig().SamplesPerSlot
+	return max(samplesPerSlot, custodyGroupCount)
+}
+
+// CustodyColumns computes the custody columns from the custody groups.
+func CustodyColumns(custodyGroups []uint64) (map[uint64]bool, error) {
+	numberOfCustodyGroups := params.BeaconConfig().NumberOfCustodyGroups
+
+	custodyGroupCount := len(custodyGroups)
+
+	// Compute the columns for each custody group.
+	columns := make(map[uint64]bool, custodyGroupCount)
+	for _, group := range custodyGroups {
+		if group >= numberOfCustodyGroups {
+			return nil, ErrCustodyGroupTooLarge
+		}
+
+		groupColumns, err := ComputeColumnsForCustodyGroup(group)
+		if err != nil {
+			return nil, errors.Wrap(err, "compute columns for custody group")
+		}
+
+		for _, column := range groupColumns {
+			columns[column] = true
+		}
+	}
+
+	return columns, nil
+}
+
+// populateAndFilterIndices returns a sorted slices of indices, setting all indices if none are provided,
+// and filtering out indices higher than the blob count.
+func populateAndFilterIndices(indices map[uint64]bool, blobCount uint64) []uint64 {
+	// If no indices are provided, provide all blobs.
+	if len(indices) == 0 {
+		for i := range blobCount {
+			indices[i] = true
+		}
+	}
+
+	// Filter blobs index higher than the blob count.
+	indicesSlice := make([]uint64, 0, len(indices))
+	for i := range indices {
+		if i < blobCount {
+			indicesSlice = append(indicesSlice, i)
+		}
+	}
+
+	// Sort the indices.
+	slices.Sort[[]uint64](indicesSlice)
+
+	return indicesSlice
+}

beacon-chain/core/peerdas/das_core_test.go

@@ -0,0 +1,311 @@
+package peerdas_test
+
+import (
+	"testing"
+
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg"
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
+	"github.com/OffchainLabs/prysm/v6/config/params"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
+	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
+	"github.com/OffchainLabs/prysm/v6/testing/require"
+	"github.com/OffchainLabs/prysm/v6/testing/util"
+	"github.com/ethereum/go-ethereum/p2p/enode"
+	"github.com/pkg/errors"
+)
+
+func TestCustodyGroups(t *testing.T) {
+	// The happy path is unit tested in spec tests.
+	numberOfCustodyGroup := params.BeaconConfig().NumberOfCustodyGroups
+	_, err := peerdas.CustodyGroups(enode.ID{}, numberOfCustodyGroup+1)
+	require.ErrorIs(t, err, peerdas.ErrCustodyGroupCountTooLarge)
+}
+
+func TestComputeColumnsForCustodyGroup(t *testing.T) {
+	// The happy path is unit tested in spec tests.
+	numberOfCustodyGroup := params.BeaconConfig().NumberOfCustodyGroups
+	_, err := peerdas.ComputeColumnsForCustodyGroup(numberOfCustodyGroup)
+	require.ErrorIs(t, err, peerdas.ErrCustodyGroupTooLarge)
+}
+
+func TestDataColumnSidecars(t *testing.T) {
+	t.Run("nil signed block", func(t *testing.T) {
+		var expected []*ethpb.DataColumnSidecar = nil
+		actual, err := peerdas.DataColumnSidecars(nil, []kzg.CellsAndProofs{})
+		require.NoError(t, err)
+
+		require.DeepSSZEqual(t, expected, actual)
+	})
+
+	t.Run("empty cells and proofs", func(t *testing.T) {
+		// Create a protobuf signed beacon block.
+		signedBeaconBlockPb := util.NewBeaconBlockDeneb()
+
+		// Create a signed beacon block from the protobuf.
+		signedBeaconBlock, err := blocks.NewSignedBeaconBlock(signedBeaconBlockPb)
+		require.NoError(t, err)
+
+		actual, err := peerdas.DataColumnSidecars(signedBeaconBlock, []kzg.CellsAndProofs{})
+		require.NoError(t, err)
+		require.IsNil(t, actual)
+	})
+
+	t.Run("sizes mismatch", func(t *testing.T) {
+		// Create a protobuf signed beacon block.
+		signedBeaconBlockPb := util.NewBeaconBlockDeneb()
+
+		// Create a signed beacon block from the protobuf.
+		signedBeaconBlock, err := blocks.NewSignedBeaconBlock(signedBeaconBlockPb)
+		require.NoError(t, err)
+
+		// Create cells and proofs.
+		cellsAndProofs := make([]kzg.CellsAndProofs, 1)
+
+		_, err = peerdas.DataColumnSidecars(signedBeaconBlock, cellsAndProofs)
+		require.ErrorIs(t, err, peerdas.ErrMismatchSize)
+	})
+}
+
+// --------------------------------------------------------------------------------------------------------------------------------------
+// DataColumnsSidecarsFromItems is tested as part of the DataColumnSidecars tests, in the TestDataColumnsSidecarsBlobsRoundtrip function.
+// --------------------------------------------------------------------------------------------------------------------------------------
+
+func TestComputeCustodyGroupForColumn(t *testing.T) {
+	params.SetupTestConfigCleanup(t)
+	config := params.BeaconConfig()
+	config.NumberOfColumns = 128
+	config.NumberOfCustodyGroups = 64
+	params.OverrideBeaconConfig(config)
+
+	t.Run("index too large", func(t *testing.T) {
+		_, err := peerdas.ComputeCustodyGroupForColumn(1_000_000)
+		require.ErrorIs(t, err, peerdas.ErrIndexTooLarge)
+	})
+
+	t.Run("nominal", func(t *testing.T) {
+		expected := uint64(2)
+		actual, err := peerdas.ComputeCustodyGroupForColumn(2)
+		require.NoError(t, err)
+		require.Equal(t, expected, actual)
+
+		expected = uint64(3)
+		actual, err = peerdas.ComputeCustodyGroupForColumn(3)
+		require.NoError(t, err)
+		require.Equal(t, expected, actual)
+
+		expected = uint64(2)
+		actual, err = peerdas.ComputeCustodyGroupForColumn(66)
+		require.NoError(t, err)
+		require.Equal(t, expected, actual)
+
+		expected = uint64(3)
+		actual, err = peerdas.ComputeCustodyGroupForColumn(67)
+		require.NoError(t, err)
+		require.Equal(t, expected, actual)
+	})
+}
+
+func TestBlobs(t *testing.T) {
+	blobsIndice := map[uint64]bool{}
+
+	numberOfColumns := params.BeaconConfig().NumberOfColumns
+
+	almostAllColumns := make([]*ethpb.DataColumnSidecar, 0, numberOfColumns/2)
+	for i := uint64(2); i < numberOfColumns/2+2; i++ {
+		almostAllColumns = append(almostAllColumns, &ethpb.DataColumnSidecar{
+			Index: i,
+		})
+	}
+
+	testCases := []struct {
+		name     string
+		input    []*ethpb.DataColumnSidecar
+		expected []*blocks.VerifiedROBlob
+		err      error
+	}{
+		{
+			name:     "empty input",
+			input:    []*ethpb.DataColumnSidecar{},
+			expected: nil,
+			err:      errors.New("some columns are missing: [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63]"),
+		},
+		{
+			name:     "missing columns",
+			input:    almostAllColumns,
+			expected: nil,
+			err:      errors.New("some columns are missing: [0 1]"),
+		},
+	}
+
+	for _, tc := range testCases {
+		t.Run(tc.name, func(t *testing.T) {
+			actual, err := peerdas.Blobs(blobsIndice, tc.input)
+			if tc.err != nil {
+				require.Equal(t, tc.err.Error(), err.Error())
+			} else {
+				require.NoError(t, err)
+			}
+			require.DeepSSZEqual(t, tc.expected, actual)
+		})
+	}
+}
+
+func TestDataColumnsSidecarsBlobsRoundtrip(t *testing.T) {
+	const blobCount = 5
+	blobsIndex := map[uint64]bool{}
+
+	// Start the trusted setup.
+	err := kzg.Start()
+	require.NoError(t, err)
+
+	// Create a protobuf signed beacon block.
+	signedBeaconBlockPb := util.NewBeaconBlockDeneb()
+
+	// Generate random blobs and their corresponding commitments and proofs.
+	blobs := make([]kzg.Blob, 0, blobCount)
+	blobKzgCommitments := make([]*kzg.Commitment, 0, blobCount)
+	blobKzgProofs := make([]*kzg.Proof, 0, blobCount)
+
+	for blobIndex := range blobCount {
+		// Create a random blob.
+		blob := getRandBlob(int64(blobIndex))
+		blobs = append(blobs, blob)
+
+		// Generate a blobKZGCommitment for the blob.
+		blobKZGCommitment, proof, err := generateCommitmentAndProof(&blob)
+		require.NoError(t, err)
+
+		blobKzgCommitments = append(blobKzgCommitments, blobKZGCommitment)
+		blobKzgProofs = append(blobKzgProofs, proof)
+	}
+
+	// Set the commitments into the block.
+	blobZkgCommitmentsBytes := make([][]byte, 0, blobCount)
+	for _, blobKZGCommitment := range blobKzgCommitments {
+		blobZkgCommitmentsBytes = append(blobZkgCommitmentsBytes, blobKZGCommitment[:])
+	}
+
+	signedBeaconBlockPb.Block.Body.BlobKzgCommitments = blobZkgCommitmentsBytes
+
+	// Generate verified RO blobs.
+	verifiedROBlobs := make([]*blocks.VerifiedROBlob, 0, blobCount)
+
+	// Create a signed beacon block from the protobuf.
+	signedBeaconBlock, err := blocks.NewSignedBeaconBlock(signedBeaconBlockPb)
+	require.NoError(t, err)
+
+	commitmentInclusionProof, err := blocks.MerkleProofKZGCommitments(signedBeaconBlock.Block().Body())
+	require.NoError(t, err)
+
+	for blobIndex := range blobCount {
+		blob := blobs[blobIndex]
+		blobKZGCommitment := blobKzgCommitments[blobIndex]
+		blobKzgProof := blobKzgProofs[blobIndex]
+
+		// Get the signed beacon block header.
+		signedBeaconBlockHeader, err := signedBeaconBlock.Header()
+		require.NoError(t, err)
+
+		blobSidecar := &ethpb.BlobSidecar{
+			Index:                    uint64(blobIndex),
+			Blob:                     blob[:],
+			KzgCommitment:            blobKZGCommitment[:],
+			KzgProof:                 blobKzgProof[:],
+			SignedBlockHeader:        signedBeaconBlockHeader,
+			CommitmentInclusionProof: commitmentInclusionProof,
+		}
+
+		roBlob, err := blocks.NewROBlob(blobSidecar)
+		require.NoError(t, err)
+
+		verifiedROBlob := blocks.NewVerifiedROBlob(roBlob)
+		verifiedROBlobs = append(verifiedROBlobs, &verifiedROBlob)
+	}
+
+	// Compute data columns sidecars from the signed beacon block and from the blobs.
+	cellsAndProofs := util.GenerateCellsAndProofs(t, blobs)
+	dataColumnsSidecar, err := peerdas.DataColumnSidecars(signedBeaconBlock, cellsAndProofs)
+	require.NoError(t, err)
+
+	// Compute the blobs from the data columns sidecar.
+	roundtripBlobs, err := peerdas.Blobs(blobsIndex, dataColumnsSidecar)
+	require.NoError(t, err)
+
+	// Check that the blobs are the same.
+	require.DeepSSZEqual(t, verifiedROBlobs, roundtripBlobs)
+}
+
+func TestCustodyGroupSamplingSize(t *testing.T) {
+	testCases := []struct {
+		name                         string
+		custodyType                  peerdas.CustodyType
+		validatorsCustodyRequirement uint64
+		toAdvertiseCustodyGroupCount uint64
+		expected                     uint64
+	}{
+		{
+			name:                         "target, lower than samples per slot",
+			custodyType:                  peerdas.Target,
+			validatorsCustodyRequirement: 2,
+			expected:                     8,
+		},
+		{
+			name:                         "target, higher than samples per slot",
+			custodyType:                  peerdas.Target,
+			validatorsCustodyRequirement: 100,
+			expected:                     100,
+		},
+		{
+			name:                         "actual, lower than samples per slot",
+			custodyType:                  peerdas.Actual,
+			validatorsCustodyRequirement: 3,
+			toAdvertiseCustodyGroupCount: 4,
+			expected:                     8,
+		},
+		{
+			name:                         "actual, higher than samples per slot",
+			custodyType:                  peerdas.Actual,
+			validatorsCustodyRequirement: 100,
+			toAdvertiseCustodyGroupCount: 101,
+			expected:                     100,
+		},
+	}
+
+	for _, tc := range testCases {
+		t.Run(tc.name, func(t *testing.T) {
+			// Create a custody info.
+			custodyInfo := peerdas.CustodyInfo{}
+
+			// Set the validators custody requirement for target custody group count.
+			custodyInfo.TargetGroupCount.SetValidatorsCustodyRequirement(tc.validatorsCustodyRequirement)
+
+			// Set the to advertise custody group count.
+			custodyInfo.ToAdvertiseGroupCount.Set(tc.toAdvertiseCustodyGroupCount)
+
+			// Compute the custody group sampling size.
+			actual := custodyInfo.CustodyGroupSamplingSize(tc.custodyType)
+
+			// Check the result.
+			require.Equal(t, tc.expected, actual)
+		})
+	}
+}
+
+func TestCustodyColumns(t *testing.T) {
+	t.Run("group too large", func(t *testing.T) {
+		_, err := peerdas.CustodyColumns([]uint64{1_000_000})
+		require.ErrorIs(t, err, peerdas.ErrCustodyGroupTooLarge)
+	})
+
+	t.Run("nominal", func(t *testing.T) {
+		input := []uint64{1, 2}
+		expected := map[uint64]bool{1: true, 2: true}
+
+		actual, err := peerdas.CustodyColumns(input)
+		require.NoError(t, err)
+		require.Equal(t, len(expected), len(actual))
+		for i := range actual {
+			require.Equal(t, expected[i], actual[i])
+		}
+	})
+}

beacon-chain/core/peerdas/info.go

@@ -0,0 +1,192 @@
+package peerdas
+
+import (
+	"encoding/binary"
+	"sync"
+
+	"github.com/OffchainLabs/prysm/v6/cmd/beacon-chain/flags"
+	"github.com/OffchainLabs/prysm/v6/config/params"
+	"github.com/ethereum/go-ethereum/p2p/enode"
+	lru "github.com/hashicorp/golang-lru"
+	"github.com/pkg/errors"
+)
+
+// info contains all useful peerDAS related information regarding a peer.
+type (
+	info struct {
+		CustodyGroups      map[uint64]bool
+		CustodyColumns     map[uint64]bool
+		DataColumnsSubnets map[uint64]bool
+	}
+
+	targetCustodyGroupCount struct {
+		mut                          sync.RWMutex
+		validatorsCustodyRequirement uint64
+	}
+
+	toAdverstiseCustodyGroupCount struct {
+		mut   sync.RWMutex
+		value uint64
+	}
+
+	CustodyInfo struct {
+		// Mut is a mutex to be used by caller to ensure neither
+		// TargetCustodyGroupCount nor ToAdvertiseCustodyGroupCount are being modified.
+		// (This is not necessary to use this mutex for any data protection.)
+		Mut sync.RWMutex
+
+		// TargetGroupCount represents the target number of custody groups we should custody
+		// regarding the validators we are tracking.
+		TargetGroupCount targetCustodyGroupCount
+
+		// ToAdvertiseGroupCount represents the number of custody groups to advertise to the network.
+		ToAdvertiseGroupCount toAdverstiseCustodyGroupCount
+	}
+)
+
+const (
+	nodeInfoCacheSize   = 200
+	nodeInfoCachKeySize = 32 + 8
+)
+
+var (
+	nodeInfoCacheMut sync.Mutex
+	nodeInfoCache    *lru.Cache
+)
+
+// Info returns the peerDAS information for a given nodeID and custodyGroupCount.
+// It returns a boolean indicating if the peer info was already in the cache and an error if any.
+func Info(nodeID enode.ID, custodyGroupCount uint64) (*info, bool, error) {
+	// Create a new cache if it doesn't exist.
+	if err := createInfoCacheIfNeeded(); err != nil {
+		return nil, false, errors.Wrap(err, "create cache if needed")
+	}
+
+	// Compute the key.
+	key := computeInfoCacheKey(nodeID, custodyGroupCount)
+
+	// If the value is already in the cache, return it.
+	if value, ok := nodeInfoCache.Get(key); ok {
+		peerInfo, ok := value.(*info)
+		if !ok {
+			return nil, false, errors.New("failed to cast peer info (should never happen)")
+		}
+
+		return peerInfo, true, nil
+	}
+
+	// The peer info is not in the cache, compute it.
+	// Compute custody groups.
+	custodyGroups, err := CustodyGroups(nodeID, custodyGroupCount)
+	if err != nil {
+		return nil, false, errors.Wrap(err, "custody groups")
+	}
+
+	// Compute custody columns.
+	custodyColumns, err := CustodyColumns(custodyGroups)
+	if err != nil {
+		return nil, false, errors.Wrap(err, "custody columns")
+	}
+
+	// Compute data columns subnets.
+	dataColumnsSubnets := DataColumnSubnets(custodyColumns)
+
+	// Convert the custody groups to a map.
+	custodyGroupsMap := make(map[uint64]bool, len(custodyGroups))
+	for _, group := range custodyGroups {
+		custodyGroupsMap[group] = true
+	}
+
+	result := &info{
+		CustodyGroups:      custodyGroupsMap,
+		CustodyColumns:     custodyColumns,
+		DataColumnsSubnets: dataColumnsSubnets,
+	}
+
+	// Add the result to the cache.
+	nodeInfoCache.Add(key, result)
+
+	return result, false, nil
+}
+
+// ActualGroupCount returns the actual custody group count.
+func (custodyInfo *CustodyInfo) ActualGroupCount() uint64 {
+	return min(custodyInfo.TargetGroupCount.Get(), custodyInfo.ToAdvertiseGroupCount.Get())
+}
+
+// CustodyGroupCount returns the number of groups we should participate in for custody.
+func (tcgc *targetCustodyGroupCount) Get() uint64 {
+	// If subscribed to all subnets, return the number of custody groups.
+	if flags.Get().SubscribeToAllSubnets {
+		return params.BeaconConfig().NumberOfCustodyGroups
+	}
+
+	tcgc.mut.RLock()
+	defer tcgc.mut.RUnlock()
+
+	// If no validators are tracked, return the default custody requirement.
+	if tcgc.validatorsCustodyRequirement == 0 {
+		return params.BeaconConfig().CustodyRequirement
+	}
+
+	// Return the validators custody requirement.
+	return tcgc.validatorsCustodyRequirement
+}
+
+// setValidatorsCustodyRequirement sets the validators custody requirement.
+func (tcgc *targetCustodyGroupCount) SetValidatorsCustodyRequirement(value uint64) {
+	tcgc.mut.Lock()
+	defer tcgc.mut.Unlock()
+
+	tcgc.validatorsCustodyRequirement = value
+}
+
+// Get returns the to advertise custody group count.
+func (tacgc *toAdverstiseCustodyGroupCount) Get() uint64 {
+	// If subscribed to all subnets, return the number of custody groups.
+	if flags.Get().SubscribeToAllSubnets {
+		return params.BeaconConfig().NumberOfCustodyGroups
+	}
+
+	custodyRequirement := params.BeaconConfig().CustodyRequirement
+
+	tacgc.mut.RLock()
+	defer tacgc.mut.RUnlock()
+
+	return max(tacgc.value, custodyRequirement)
+}
+
+// Set sets the to advertise custody group count.
+func (tacgc *toAdverstiseCustodyGroupCount) Set(value uint64) {
+	tacgc.mut.Lock()
+	defer tacgc.mut.Unlock()
+
+	tacgc.value = value
+}
+
+// createInfoCacheIfNeeded creates a new cache if it doesn't exist.
+func createInfoCacheIfNeeded() error {
+	nodeInfoCacheMut.Lock()
+	defer nodeInfoCacheMut.Unlock()
+
+	if nodeInfoCache == nil {
+		c, err := lru.New(nodeInfoCacheSize)
+		if err != nil {
+			return errors.Wrap(err, "lru new")
+		}
+
+		nodeInfoCache = c
+	}
+
+	return nil
+}
+
+// computeInfoCacheKey returns a unique key for a node and its custodyGroupCount.
+func computeInfoCacheKey(nodeID enode.ID, custodyGroupCount uint64) [nodeInfoCachKeySize]byte {
+	var key [nodeInfoCachKeySize]byte
+
+	copy(key[:32], nodeID[:])
+	binary.BigEndian.PutUint64(key[32:], custodyGroupCount)
+
+	return key
+}

beacon-chain/core/peerdas/info_test.go

@@ -0,0 +1,133 @@
+package peerdas_test
+
+import (
+	"testing"
+
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
+	"github.com/OffchainLabs/prysm/v6/cmd/beacon-chain/flags"
+	"github.com/OffchainLabs/prysm/v6/testing/require"
+	"github.com/ethereum/go-ethereum/p2p/enode"
+)
+
+func TestInfo(t *testing.T) {
+	nodeID := enode.ID{}
+	custodyGroupCount := uint64(7)
+
+	expectedCustodyGroup := map[uint64]bool{1: true, 17: true, 19: true, 42: true, 75: true, 87: true, 102: true}
+	expectedCustodyColumns := map[uint64]bool{1: true, 17: true, 19: true, 42: true, 75: true, 87: true, 102: true}
+	expectedDataColumnsSubnets := map[uint64]bool{1: true, 17: true, 19: true, 42: true, 75: true, 87: true, 102: true}
+
+	for _, cached := range []bool{false, true} {
+		actual, ok, err := peerdas.Info(nodeID, custodyGroupCount)
+		require.NoError(t, err)
+		require.Equal(t, cached, ok)
+		require.DeepEqual(t, expectedCustodyGroup, actual.CustodyGroups)
+		require.DeepEqual(t, expectedCustodyColumns, actual.CustodyColumns)
+		require.DeepEqual(t, expectedDataColumnsSubnets, actual.DataColumnsSubnets)
+	}
+}
+
+func TestTargetCustodyGroupCount(t *testing.T) {
+	testCases := []struct {
+		name                         string
+		subscribeToAllSubnets        bool
+		validatorsCustodyRequirement uint64
+		expected                     uint64
+	}{
+		{
+			name:                         "subscribed to all subnets",
+			subscribeToAllSubnets:        true,
+			validatorsCustodyRequirement: 100,
+			expected:                     128,
+		},
+		{
+			name:                         "no validators attached",
+			subscribeToAllSubnets:        false,
+			validatorsCustodyRequirement: 0,
+			expected:                     4,
+		},
+		{
+			name:                         "some validators attached",
+			subscribeToAllSubnets:        false,
+			validatorsCustodyRequirement: 100,
+			expected:                     100,
+		},
+	}
+
+	for _, tc := range testCases {
+		t.Run(tc.name, func(t *testing.T) {
+			// Subscribe to all subnets if needed.
+			if tc.subscribeToAllSubnets {
+				resetFlags := flags.Get()
+				gFlags := new(flags.GlobalFlags)
+				gFlags.SubscribeToAllSubnets = true
+				flags.Init(gFlags)
+				defer flags.Init(resetFlags)
+			}
+
+			var custodyInfo peerdas.CustodyInfo
+
+			// Set the validators custody requirement.
+			custodyInfo.TargetGroupCount.SetValidatorsCustodyRequirement(tc.validatorsCustodyRequirement)
+
+			// Get the target custody group count.
+			actual := custodyInfo.TargetGroupCount.Get()
+
+			// Compare the expected and actual values.
+			require.Equal(t, tc.expected, actual)
+		})
+	}
+}
+
+func TestToAdvertiseCustodyGroupCount(t *testing.T) {
+	testCases := []struct {
+		name                         string
+		subscribeToAllSubnets        bool
+		toAdvertiseCustodyGroupCount uint64
+		expected                     uint64
+	}{
+		{
+			name:                         "subscribed to all subnets",
+			subscribeToAllSubnets:        true,
+			toAdvertiseCustodyGroupCount: 100,
+			expected:                     128,
+		},
+		{
+			name:                         "higher than custody requirement",
+			subscribeToAllSubnets:        false,
+			toAdvertiseCustodyGroupCount: 100,
+			expected:                     100,
+		},
+		{
+			name:                         "lower than custody requirement",
+			subscribeToAllSubnets:        false,
+			toAdvertiseCustodyGroupCount: 1,
+			expected:                     4,
+		},
+	}
+
+	for _, tc := range testCases {
+		t.Run(tc.name, func(t *testing.T) {
+			// Subscribe to all subnets if needed.
+			if tc.subscribeToAllSubnets {
+				resetFlags := flags.Get()
+				gFlags := new(flags.GlobalFlags)
+				gFlags.SubscribeToAllSubnets = true
+				flags.Init(gFlags)
+				defer flags.Init(resetFlags)
+			}
+
+			// Create a custody info.
+			var custodyInfo peerdas.CustodyInfo
+
+			// Set the to advertise custody group count.
+			custodyInfo.ToAdvertiseGroupCount.Set(tc.toAdvertiseCustodyGroupCount)
+
+			// Get the to advertise custody group count.
+			actual := custodyInfo.ToAdvertiseGroupCount.Get()
+
+			// Compare the expected and actual values.
+			require.Equal(t, tc.expected, actual)
+		})
+	}
+}

beacon-chain/core/peerdas/metrics.go

@@ -0,0 +1,14 @@
+package peerdas
+
+import (
+	"github.com/prometheus/client_golang/prometheus"
+	"github.com/prometheus/client_golang/prometheus/promauto"
+)
+
+var dataColumnComputationTime = promauto.NewHistogram(
+	prometheus.HistogramOpts{
+		Name:    "beacon_data_column_sidecar_computation_milliseconds",
+		Help:    "Captures the time taken to compute data column sidecars from blobs.",
+		Buckets: []float64{100, 250, 500, 750, 1000, 1500, 2000, 4000, 8000, 12000, 16000},
+	},
+)

beacon-chain/core/peerdas/p2p_interface.go

@@ -0,0 +1,162 @@
+package peerdas
+
+import (
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg"
+	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
+	"github.com/OffchainLabs/prysm/v6/config/params"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
+	"github.com/OffchainLabs/prysm/v6/container/trie"
+	"github.com/ethereum/go-ethereum/p2p/enr"
+	"github.com/pkg/errors"
+)
+
+const (
+	CustodyGroupCountEnrKey = "cgc"
+	kzgPosition             = 11 // The index of the KZG commitment list in the Body
+)
+
+var (
+	ErrIndexTooLarge               = errors.New("column index is larger than the specified columns count")
+	ErrNoKzgCommitments            = errors.New("no KZG commitments found")
+	ErrMismatchLength              = errors.New("mismatch in the length of the column, commitments or proofs")
+	ErrInvalidKZGProof             = errors.New("invalid KZG proof")
+	ErrBadRootLength               = errors.New("bad root length")
+	ErrInvalidInclusionProof       = errors.New("invalid inclusion proof")
+	ErrRecordNil                   = errors.New("record is nil")
+	ErrNilBlockHeader              = errors.New("nil beacon block header")
+	ErrCannotLoadCustodyGroupCount = errors.New("cannot load the custody group count from peer")
+)
+
+// https://github.com/ethereum/consensus-specs/blob/v1.5.0-beta.5/specs/fulu/p2p-interface.md#custody-group-count
+type Cgc uint64
+
+func (Cgc) ENRKey() string { return CustodyGroupCountEnrKey }
+
+// VerifyDataColumnSidecar verifies if the data column sidecar is valid.
+// https://github.com/ethereum/consensus-specs/blob/v1.5.0-beta.5/specs/fulu/p2p-interface.md#verify_data_column_sidecar
+func VerifyDataColumnSidecar(sidecar blocks.RODataColumn) error {
+	// The sidecar index must be within the valid range.
+	numberOfColumns := params.BeaconConfig().NumberOfColumns
+	if sidecar.Index >= numberOfColumns {
+		return ErrIndexTooLarge
+	}
+
+	// A sidecar for zero blobs is invalid.
+	if len(sidecar.KzgCommitments) == 0 {
+		return ErrNoKzgCommitments
+	}
+
+	// The column length must be equal to the number of commitments/proofs.
+	if len(sidecar.Column) != len(sidecar.KzgCommitments) || len(sidecar.Column) != len(sidecar.KzgProofs) {
+		return ErrMismatchLength
+	}
+
+	return nil
+}
+
+// VerifyDataColumnsSidecarKZGProofs verifies if the KZG proofs are correct.
+// Note: We are slightly deviating from the specification here:
+// The specification verifies the KZG proofs for each sidecar separately,
+// while we are verifying all the KZG proofs from multiple sidecars in a batch.
+// This is done to improve performance since the internal KZG library is way more
+// efficient when verifying in batch.
+// https://github.com/ethereum/consensus-specs/blob/v1.5.0-beta.5/specs/fulu/p2p-interface.md#verify_data_column_sidecar_kzg_proofs
+func VerifyDataColumnsSidecarKZGProofs(sidecars []blocks.RODataColumn) error {
+	// Compute the total count.
+	count := 0
+	for _, sidecar := range sidecars {
+		count += len(sidecar.Column)
+	}
+
+	commitments := make([]kzg.Bytes48, 0, count)
+	indices := make([]uint64, 0, count)
+	cells := make([]kzg.Cell, 0, count)
+	proofs := make([]kzg.Bytes48, 0, count)
+
+	for _, sidecar := range sidecars {
+		for i := range sidecar.Column {
+			commitments = append(commitments, kzg.Bytes48(sidecar.KzgCommitments[i]))
+			indices = append(indices, sidecar.Index)
+			cells = append(cells, kzg.Cell(sidecar.Column[i]))
+			proofs = append(proofs, kzg.Bytes48(sidecar.KzgProofs[i]))
+		}
+	}
+
+	// Batch verify that the cells match the corresponding commitments and proofs.
+	verified, err := kzg.VerifyCellKZGProofBatch(commitments, indices, cells, proofs)
+	if err != nil {
+		return errors.Wrap(err, "verify cell KZG proof batch")
+	}
+
+	if !verified {
+		return ErrInvalidKZGProof
+	}
+
+	return nil
+}
+
+// VerifyDataColumnSidecarInclusionProof verifies if the given KZG commitments included in the given beacon block.
+// https://github.com/ethereum/consensus-specs/blob/v1.5.0-beta.5/specs/fulu/p2p-interface.md#verify_data_column_sidecar_inclusion_proof
+func VerifyDataColumnSidecarInclusionProof(sidecar blocks.RODataColumn) error {
+	if sidecar.SignedBlockHeader == nil || sidecar.SignedBlockHeader.Header == nil {
+		return ErrNilBlockHeader
+	}
+
+	root := sidecar.SignedBlockHeader.Header.BodyRoot
+	if len(root) != fieldparams.RootLength {
+		return ErrBadRootLength
+	}
+
+	leaves := blocks.LeavesFromCommitments(sidecar.KzgCommitments)
+
+	sparse, err := trie.GenerateTrieFromItems(leaves, fieldparams.LogMaxBlobCommitments)
+	if err != nil {
+		return errors.Wrap(err, "generate trie from items")
+	}
+
+	hashTreeRoot, err := sparse.HashTreeRoot()
+	if err != nil {
+		return errors.Wrap(err, "hash tree root")
+	}
+
+	verified := trie.VerifyMerkleProof(root, hashTreeRoot[:], kzgPosition, sidecar.KzgCommitmentsInclusionProof)
+	if !verified {
+		return ErrInvalidInclusionProof
+	}
+
+	return nil
+}
+
+// ComputeSubnetForDataColumnSidecar computes the subnet for a data column sidecar.
+// https://github.com/ethereum/consensus-specs/blob/v1.5.0-beta.5/specs/fulu/p2p-interface.md#compute_subnet_for_data_column_sidecar
+func ComputeSubnetForDataColumnSidecar(columnIndex uint64) uint64 {
+	dataColumnSidecarSubnetCount := params.BeaconConfig().DataColumnSidecarSubnetCount
+	return columnIndex % dataColumnSidecarSubnetCount
+}
+
+// DataColumnSubnets computes the subnets for the data columns.
+func DataColumnSubnets(dataColumns map[uint64]bool) map[uint64]bool {
+	subnets := make(map[uint64]bool, len(dataColumns))
+
+	for column := range dataColumns {
+		subnet := ComputeSubnetForDataColumnSidecar(column)
+		subnets[subnet] = true
+	}
+
+	return subnets
+}
+
+// CustodyGroupCountFromRecord extracts the custody group count from an ENR record.
+func CustodyGroupCountFromRecord(record *enr.Record) (uint64, error) {
+	if record == nil {
+		return 0, ErrRecordNil
+	}
+
+	// Load the `cgc`
+	var cgc Cgc
+	if err := record.Load(&cgc); err != nil {
+		return 0, ErrCannotLoadCustodyGroupCount
+	}
+
+	return uint64(cgc), nil
+}

beacon-chain/core/peerdas/p2p_interface_test.go

@@ -0,0 +1,398 @@
+package peerdas_test
+
+import (
+	"crypto/rand"
+	"fmt"
+	"testing"
+
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg"
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
+	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
+	"github.com/OffchainLabs/prysm/v6/config/params"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
+	enginev1 "github.com/OffchainLabs/prysm/v6/proto/engine/v1"
+	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
+	"github.com/OffchainLabs/prysm/v6/testing/require"
+	"github.com/OffchainLabs/prysm/v6/testing/util"
+	"github.com/ethereum/go-ethereum/p2p/enr"
+)
+
+func TestVerifyDataColumnSidecar(t *testing.T) {
+	t.Run("index too large", func(t *testing.T) {
+		roSidecar := createTestSidecar(t, 1_000_000, nil, nil, nil)
+		err := peerdas.VerifyDataColumnSidecar(roSidecar)
+		require.ErrorIs(t, err, peerdas.ErrIndexTooLarge)
+	})
+
+	t.Run("no commitments", func(t *testing.T) {
+		roSidecar := createTestSidecar(t, 0, nil, nil, nil)
+		err := peerdas.VerifyDataColumnSidecar(roSidecar)
+		require.ErrorIs(t, err, peerdas.ErrNoKzgCommitments)
+	})
+
+	t.Run("KZG commitments size mismatch", func(t *testing.T) {
+		kzgCommitments := make([][]byte, 1)
+		roSidecar := createTestSidecar(t, 0, nil, kzgCommitments, nil)
+		err := peerdas.VerifyDataColumnSidecar(roSidecar)
+		require.ErrorIs(t, err, peerdas.ErrMismatchLength)
+	})
+
+	t.Run("KZG proofs size mismatch", func(t *testing.T) {
+		column, kzgCommitments := make([][]byte, 1), make([][]byte, 1)
+		roSidecar := createTestSidecar(t, 0, column, kzgCommitments, nil)
+		err := peerdas.VerifyDataColumnSidecar(roSidecar)
+		require.ErrorIs(t, err, peerdas.ErrMismatchLength)
+	})
+
+	t.Run("nominal", func(t *testing.T) {
+		column, kzgCommitments, kzgProofs := make([][]byte, 1), make([][]byte, 1), make([][]byte, 1)
+		roSidecar := createTestSidecar(t, 0, column, kzgCommitments, kzgProofs)
+		err := peerdas.VerifyDataColumnSidecar(roSidecar)
+		require.NoError(t, err)
+	})
+}
+
+func TestVerifyDataColumnSidecarKZGProofs(t *testing.T) {
+	const (
+		blobCount = 6
+		seed      = 0
+	)
+	err := kzg.Start()
+	require.NoError(t, err)
+
+	t.Run("invalid proof", func(t *testing.T) {
+		sidecars := generateRandomSidecars(t, seed, blobCount)
+		sidecars[0].Column[0][0]++ // It is OK to overflow
+		roDataColumnSidecars := generateRODataColumnSidecars(t, sidecars)
+
+		err := peerdas.VerifyDataColumnsSidecarKZGProofs(roDataColumnSidecars)
+		require.ErrorIs(t, err, peerdas.ErrInvalidKZGProof)
+	})
+
+	t.Run("nominal", func(t *testing.T) {
+		sidecars := generateRandomSidecars(t, seed, blobCount)
+		roDataColumnSidecars := generateRODataColumnSidecars(t, sidecars)
+
+		err := peerdas.VerifyDataColumnsSidecarKZGProofs(roDataColumnSidecars)
+		require.NoError(t, err)
+	})
+}
+
+func Test_VerifyKZGInclusionProofColumn(t *testing.T) {
+	const (
+		blobCount   = 3
+		columnIndex = 0
+	)
+
+	// Generate random KZG commitments `blobCount` blobs.
+	kzgCommitments := make([][]byte, blobCount)
+
+	for i := 0; i < blobCount; i++ {
+		kzgCommitments[i] = make([]byte, 48)
+		_, err := rand.Read(kzgCommitments[i])
+		require.NoError(t, err)
+	}
+
+	pbBody := &ethpb.BeaconBlockBodyDeneb{
+		RandaoReveal: make([]byte, 96),
+		Eth1Data: &ethpb.Eth1Data{
+			DepositRoot: make([]byte, fieldparams.RootLength),
+			BlockHash:   make([]byte, fieldparams.RootLength),
+		},
+		Graffiti: make([]byte, 32),
+		SyncAggregate: &ethpb.SyncAggregate{
+			SyncCommitteeBits:      make([]byte, fieldparams.SyncAggregateSyncCommitteeBytesLength),
+			SyncCommitteeSignature: make([]byte, fieldparams.BLSSignatureLength),
+		},
+		ExecutionPayload: &enginev1.ExecutionPayloadDeneb{
+			ParentHash:    make([]byte, fieldparams.RootLength),
+			FeeRecipient:  make([]byte, 20),
+			StateRoot:     make([]byte, fieldparams.RootLength),
+			ReceiptsRoot:  make([]byte, fieldparams.RootLength),
+			LogsBloom:     make([]byte, 256),
+			PrevRandao:    make([]byte, fieldparams.RootLength),
+			BaseFeePerGas: make([]byte, fieldparams.RootLength),
+			BlockHash:     make([]byte, fieldparams.RootLength),
+			Transactions:  make([][]byte, 0),
+			ExtraData:     make([]byte, 0),
+		},
+		BlobKzgCommitments: kzgCommitments,
+	}
+
+	root, err := pbBody.HashTreeRoot()
+	require.NoError(t, err)
+
+	body, err := blocks.NewBeaconBlockBody(pbBody)
+	require.NoError(t, err)
+
+	kzgCommitmentsInclusionProof, err := blocks.MerkleProofKZGCommitments(body)
+	require.NoError(t, err)
+
+	testCases := []struct {
+		name              string
+		expectedError     error
+		dataColumnSidecar *ethpb.DataColumnSidecar
+	}{
+		{
+			name:              "nilSignedBlockHeader",
+			expectedError:     peerdas.ErrNilBlockHeader,
+			dataColumnSidecar: &ethpb.DataColumnSidecar{},
+		},
+		{
+			name:          "nilHeader",
+			expectedError: peerdas.ErrNilBlockHeader,
+			dataColumnSidecar: &ethpb.DataColumnSidecar{
+				SignedBlockHeader: &ethpb.SignedBeaconBlockHeader{},
+			},
+		},
+		{
+			name:          "invalidBodyRoot",
+			expectedError: peerdas.ErrBadRootLength,
+			dataColumnSidecar: &ethpb.DataColumnSidecar{
+				SignedBlockHeader: &ethpb.SignedBeaconBlockHeader{
+					Header: &ethpb.BeaconBlockHeader{},
+				},
+			},
+		},
+		{
+			name:          "unverifiedMerkleProof",
+			expectedError: peerdas.ErrInvalidInclusionProof,
+			dataColumnSidecar: &ethpb.DataColumnSidecar{
+				SignedBlockHeader: &ethpb.SignedBeaconBlockHeader{
+					Header: &ethpb.BeaconBlockHeader{
+						BodyRoot: make([]byte, 32),
+					},
+				},
+				KzgCommitments: kzgCommitments,
+			},
+		},
+		{
+			name:          "nominal",
+			expectedError: nil,
+			dataColumnSidecar: &ethpb.DataColumnSidecar{
+				KzgCommitments: kzgCommitments,
+				SignedBlockHeader: &ethpb.SignedBeaconBlockHeader{
+					Header: &ethpb.BeaconBlockHeader{
+						BodyRoot: root[:],
+					},
+				},
+				KzgCommitmentsInclusionProof: kzgCommitmentsInclusionProof,
+			},
+		},
+	}
+
+	for _, tc := range testCases {
+		t.Run(tc.name, func(t *testing.T) {
+			roDataColumn := blocks.RODataColumn{DataColumnSidecar: tc.dataColumnSidecar}
+			err = peerdas.VerifyDataColumnSidecarInclusionProof(roDataColumn)
+			if tc.expectedError == nil {
+				require.NoError(t, err)
+				return
+			}
+
+			require.ErrorIs(t, tc.expectedError, err)
+		})
+	}
+}
+
+func TestComputeSubnetForDataColumnSidecar(t *testing.T) {
+	params.SetupTestConfigCleanup(t)
+	config := params.BeaconConfig()
+	config.DataColumnSidecarSubnetCount = 128
+	params.OverrideBeaconConfig(config)
+
+	require.Equal(t, uint64(0), peerdas.ComputeSubnetForDataColumnSidecar(0))
+	require.Equal(t, uint64(1), peerdas.ComputeSubnetForDataColumnSidecar(1))
+	require.Equal(t, uint64(0), peerdas.ComputeSubnetForDataColumnSidecar(128))
+	require.Equal(t, uint64(1), peerdas.ComputeSubnetForDataColumnSidecar(129))
+}
+
+func TestDataColumnSubnets(t *testing.T) {
+	params.SetupTestConfigCleanup(t)
+	config := params.BeaconConfig()
+	config.DataColumnSidecarSubnetCount = 128
+	params.OverrideBeaconConfig(config)
+
+	input := map[uint64]bool{0: true, 1: true, 128: true, 129: true, 131: true}
+	expected := map[uint64]bool{0: true, 1: true, 3: true}
+	actual := peerdas.DataColumnSubnets(input)
+
+	require.Equal(t, len(expected), len(actual))
+	for k, v := range expected {
+		require.Equal(t, v, actual[k])
+	}
+}
+
+func TestCustodyGroupCountFromRecord(t *testing.T) {
+	t.Run("nil record", func(t *testing.T) {
+		_, err := peerdas.CustodyGroupCountFromRecord(nil)
+		require.ErrorIs(t, err, peerdas.ErrRecordNil)
+	})
+
+	t.Run("no cgc", func(t *testing.T) {
+		_, err := peerdas.CustodyGroupCountFromRecord(&enr.Record{})
+		require.ErrorIs(t, err, peerdas.ErrCannotLoadCustodyGroupCount)
+	})
+
+	t.Run("nominal", func(t *testing.T) {
+		const expected uint64 = 7
+
+		record := &enr.Record{}
+		record.Set(peerdas.Cgc(expected))
+
+		actual, err := peerdas.CustodyGroupCountFromRecord(record)
+		require.NoError(t, err)
+		require.Equal(t, expected, actual)
+	})
+}
+
+func BenchmarkVerifyDataColumnSidecarKZGProofs_SameCommitments_NoBatch(b *testing.B) {
+	const blobCount = 12
+	err := kzg.Start()
+	require.NoError(b, err)
+
+	b.StopTimer()
+	b.ResetTimer()
+	for i := range int64(b.N) {
+		// Generate new random sidecars to ensure the KZG backend does not cache anything.
+		sidecars := generateRandomSidecars(b, i, blobCount)
+		roDataColumnSidecars := generateRODataColumnSidecars(b, sidecars)
+
+		for _, sidecar := range roDataColumnSidecars {
+			sidecars := []blocks.RODataColumn{sidecar}
+			b.StartTimer()
+			err := peerdas.VerifyDataColumnsSidecarKZGProofs(sidecars)
+			b.StopTimer()
+			require.NoError(b, err)
+		}
+	}
+}
+
+func BenchmarkVerifyDataColumnSidecarKZGProofs_DiffCommitments_Batch(b *testing.B) {
+	const blobCount = 12
+	numberOfColumns := int64(params.BeaconConfig().NumberOfColumns)
+	err := kzg.Start()
+	require.NoError(b, err)
+
+	columnsCounts := []int64{1, 2, 4, 8, 16, 32, 64, 128}
+
+	for i, columnsCount := range columnsCounts {
+		b.Run(fmt.Sprintf("columnsCount_%d", columnsCount), func(b *testing.B) {
+			b.StopTimer()
+			b.ResetTimer()
+
+			for j := range int64(b.N) {
+				allSidecars := make([]*ethpb.DataColumnSidecar, 0, numberOfColumns)
+				for k := int64(0); k < numberOfColumns; k += columnsCount {
+					// Use different seeds to generate different blobs/commitments
+					seed := int64(b.N*i) + numberOfColumns*j + blobCount*k
+					sidecars := generateRandomSidecars(b, seed, blobCount)
+
+					// Pick sidecars.
+					allSidecars = append(allSidecars, sidecars[k:k+columnsCount]...)
+				}
+
+				roDataColumnSidecars := generateRODataColumnSidecars(b, allSidecars)
+
+				b.StartTimer()
+				err := peerdas.VerifyDataColumnsSidecarKZGProofs(roDataColumnSidecars)
+				b.StopTimer()
+				require.NoError(b, err)
+			}
+		})
+	}
+}
+
+func BenchmarkVerifyDataColumnSidecarKZGProofs_DiffCommitments_Batch4(b *testing.B) {
+	const (
+		blobCount = 12
+
+		// columnsCount*batchCount = 128
+		columnsCount = 4
+		batchCount   = 32
+	)
+
+	err := kzg.Start()
+	require.NoError(b, err)
+
+	b.StopTimer()
+	b.ResetTimer()
+
+	for i := range int64(b.N) {
+		allSidecars := make([][]blocks.RODataColumn, 0, batchCount)
+		for j := range int64(batchCount) {
+			// Use different seeds to generate different blobs/commitments
+			sidecars := generateRandomSidecars(b, int64(batchCount)*i+j*blobCount, blobCount)
+			roDataColumnSidecars := generateRODataColumnSidecars(b, sidecars[:columnsCount])
+			allSidecars = append(allSidecars, roDataColumnSidecars)
+		}
+
+		for _, sidecars := range allSidecars {
+			b.StartTimer()
+			err := peerdas.VerifyDataColumnsSidecarKZGProofs(sidecars)
+			b.StopTimer()
+			require.NoError(b, err)
+		}
+	}
+}
+
+func createTestSidecar(t *testing.T, index uint64, column, kzgCommitments, kzgProofs [][]byte) blocks.RODataColumn {
+	pbSignedBeaconBlock := util.NewBeaconBlockDeneb()
+	signedBeaconBlock, err := blocks.NewSignedBeaconBlock(pbSignedBeaconBlock)
+	require.NoError(t, err)
+
+	signedBlockHeader, err := signedBeaconBlock.Header()
+	require.NoError(t, err)
+
+	sidecar := &ethpb.DataColumnSidecar{
+		Index:             index,
+		Column:            column,
+		KzgCommitments:    kzgCommitments,
+		KzgProofs:         kzgProofs,
+		SignedBlockHeader: signedBlockHeader,
+	}
+
+	roSidecar, err := blocks.NewRODataColumn(sidecar)
+	require.NoError(t, err)
+
+	return roSidecar
+}
+
+func generateRandomSidecars(t testing.TB, seed, blobCount int64) []*ethpb.DataColumnSidecar {
+	dbBlock := util.NewBeaconBlockDeneb()
+
+	commitments := make([][]byte, 0, blobCount)
+	blobs := make([]kzg.Blob, 0, blobCount)
+
+	for i := range blobCount {
+		subSeed := seed + i
+		blob := getRandBlob(subSeed)
+		commitment, err := generateCommitment(&blob)
+		require.NoError(t, err)
+
+		commitments = append(commitments, commitment[:])
+		blobs = append(blobs, blob)
+	}
+
+	dbBlock.Block.Body.BlobKzgCommitments = commitments
+	sBlock, err := blocks.NewSignedBeaconBlock(dbBlock)
+	require.NoError(t, err)
+
+	cellsAndProofs := util.GenerateCellsAndProofs(t, blobs)
+	sidecars, err := peerdas.DataColumnSidecars(sBlock, cellsAndProofs)
+	require.NoError(t, err)
+
+	return sidecars
+}
+
+func generateRODataColumnSidecars(t testing.TB, sidecars []*ethpb.DataColumnSidecar) []blocks.RODataColumn {
+	roDataColumnSidecars := make([]blocks.RODataColumn, 0, len(sidecars))
+	for _, sidecar := range sidecars {
+		roCol, err := blocks.NewRODataColumn(sidecar)
+		require.NoError(t, err)
+
+		roDataColumnSidecars = append(roDataColumnSidecars, roCol)
+	}
+
+	return roDataColumnSidecars
+}

beacon-chain/core/peerdas/reconstruction.go

@@ -0,0 +1,76 @@
+package peerdas
+
+import (
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg"
+	"github.com/OffchainLabs/prysm/v6/config/params"
+	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
+	"github.com/pkg/errors"
+	"golang.org/x/sync/errgroup"
+)
+
+// CanSelfReconstruct returns true if the node can self-reconstruct all the data columns from its custody group count.
+func CanSelfReconstruct(custodyGroupCount uint64) bool {
+	total := params.BeaconConfig().NumberOfCustodyGroups
+	// If total is odd, then we need total / 2 + 1 columns to reconstruct.
+	// If total is even, then we need total / 2 columns to reconstruct.
+	return custodyGroupCount >= (total+1)/2
+}
+
+// RecoverCellsAndProofs recovers the cells and proofs from the data column sidecars.
+func RecoverCellsAndProofs(dataColumnSideCars []*ethpb.DataColumnSidecar) ([]kzg.CellsAndProofs, error) {
+	var wg errgroup.Group
+
+	dataColumnSideCarsCount := len(dataColumnSideCars)
+
+	if dataColumnSideCarsCount == 0 {
+		return nil, errors.New("no data column sidecars")
+	}
+
+	// Check if all columns have the same length.
+	blobCount := len(dataColumnSideCars[0].Column)
+	for _, sidecar := range dataColumnSideCars {
+		length := len(sidecar.Column)
+
+		if length != blobCount {
+			return nil, errors.New("columns do not have the same length")
+		}
+	}
+
+	// Recover cells and compute proofs in parallel.
+	recoveredCellsAndProofs := make([]kzg.CellsAndProofs, blobCount)
+
+	for blobIndex := 0; blobIndex < blobCount; blobIndex++ {
+		bIndex := blobIndex
+		wg.Go(func() error {
+			cellsIndices := make([]uint64, 0, dataColumnSideCarsCount)
+			cells := make([]kzg.Cell, 0, dataColumnSideCarsCount)
+
+			for _, sidecar := range dataColumnSideCars {
+				// Build the cell indices.
+				cellsIndices = append(cellsIndices, sidecar.Index)
+
+				// Get the cell.
+				column := sidecar.Column
+				cell := column[bIndex]
+
+				cells = append(cells, kzg.Cell(cell))
+			}
+
+			// Recover the cells and proofs for the corresponding blob
+			cellsAndProofs, err := kzg.RecoverCellsAndKZGProofs(cellsIndices, cells)
+
+			if err != nil {
+				return errors.Wrapf(err, "recover cells and KZG proofs for blob %d", bIndex)
+			}
+
+			recoveredCellsAndProofs[bIndex] = cellsAndProofs
+			return nil
+		})
+	}
+
+	if err := wg.Wait(); err != nil {
+		return nil, err
+	}
+
+	return recoveredCellsAndProofs, nil
+}

beacon-chain/core/peerdas/reconstruction_test.go

@@ -0,0 +1,57 @@
+package peerdas_test
+
+import (
+	"testing"
+
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
+	"github.com/OffchainLabs/prysm/v6/config/params"
+	"github.com/OffchainLabs/prysm/v6/testing/require"
+)
+
+func TestCanSelfReconstruct(t *testing.T) {
+	testCases := []struct {
+		name                       string
+		totalNumberOfCustodyGroups uint64
+		custodyNumberOfGroups      uint64
+		expected                   bool
+	}{
+		{
+			name:                       "totalNumberOfCustodyGroups=64, custodyNumberOfGroups=31",
+			totalNumberOfCustodyGroups: 64,
+			custodyNumberOfGroups:      31,
+			expected:                   false,
+		},
+		{
+			name:                       "totalNumberOfCustodyGroups=64, custodyNumberOfGroups=32",
+			totalNumberOfCustodyGroups: 64,
+			custodyNumberOfGroups:      32,
+			expected:                   true,
+		},
+		{
+			name:                       "totalNumberOfCustodyGroups=65, custodyNumberOfGroups=32",
+			totalNumberOfCustodyGroups: 65,
+			custodyNumberOfGroups:      32,
+			expected:                   false,
+		},
+		{
+			name:                       "totalNumberOfCustodyGroups=63, custodyNumberOfGroups=33",
+			totalNumberOfCustodyGroups: 65,
+			custodyNumberOfGroups:      33,
+			expected:                   true,
+		},
+	}
+
+	for _, tc := range testCases {
+		t.Run(tc.name, func(t *testing.T) {
+			// Set the total number of columns.
+			params.SetupTestConfigCleanup(t)
+			cfg := params.BeaconConfig().Copy()
+			cfg.NumberOfCustodyGroups = tc.totalNumberOfCustodyGroups
+			params.OverrideBeaconConfig(cfg)
+
+			// Check if reconstuction is possible.
+			actual := peerdas.CanSelfReconstruct(tc.custodyNumberOfGroups)
+			require.Equal(t, tc.expected, actual)
+		})
+	}
+}

beacon-chain/core/peerdas/util.go

@@ -0,0 +1,54 @@
+package peerdas
+
+import (
+	"fmt"
+
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg"
+	"github.com/OffchainLabs/prysm/v6/config/params"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
+	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
+	"github.com/OffchainLabs/prysm/v6/runtime/version"
+	"github.com/pkg/errors"
+)
+
+// ConstructDataColumnSidecars constructs data column sidecars from a block, blobs and their cell proofs.
+// This is a convenience method as blob and cell proofs are common inputs.
+func ConstructDataColumnSidecars(block interfaces.SignedBeaconBlock, blobs [][]byte, cellProofs [][]byte) ([]*ethpb.DataColumnSidecar, error) {
+	// Check if the block is at least a Fulu block.
+	if block.Version() < version.Fulu {
+		return nil, nil
+	}
+
+	numberOfColumns := params.BeaconConfig().NumberOfColumns
+	if uint64(len(blobs))*numberOfColumns != uint64(len(cellProofs)) {
+		return nil, fmt.Errorf("number of blobs and cell proofs do not match: %d * %d != %d", len(blobs), numberOfColumns, len(cellProofs))
+	}
+
+	cellsAndProofs := make([]kzg.CellsAndProofs, 0, len(blobs))
+
+	for i, blob := range blobs {
+		var b kzg.Blob
+		copy(b[:], blob)
+		cells, err := kzg.ComputeCells(&b)
+		if err != nil {
+			return nil, err
+		}
+
+		var proofs []kzg.Proof
+		for idx := uint64(i) * numberOfColumns; idx < (uint64(i)+1)*numberOfColumns; idx++ {
+			proofs = append(proofs, kzg.Proof(cellProofs[idx]))
+		}
+
+		cellsAndProofs = append(cellsAndProofs, kzg.CellsAndProofs{
+			Cells:  cells,
+			Proofs: proofs,
+		})
+	}
+
+	dataColumnSidecars, err := DataColumnSidecars(block, cellsAndProofs)
+	if err != nil {
+		return nil, errors.Wrap(err, "data column sidcars")
+	}
+
+	return dataColumnSidecars, nil
+}

beacon-chain/core/peerdas/utils_test.go

@@ -0,0 +1,69 @@
+package peerdas_test
+
+import (
+	"bytes"
+	"crypto/sha256"
+	"encoding/binary"
+
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg"
+	"github.com/consensys/gnark-crypto/ecc/bls12-381/fr"
+	GoKZG "github.com/crate-crypto/go-kzg-4844"
+	"github.com/pkg/errors"
+	"github.com/sirupsen/logrus"
+)
+
+func generateCommitment(blob *kzg.Blob) (*kzg.Commitment, error) {
+	commitment, err := kzg.BlobToKZGCommitment(blob)
+	if err != nil {
+		return nil, errors.Wrap(err, "blob to kzg commitment")
+	}
+
+	return &commitment, nil
+}
+
+func generateCommitmentAndProof(blob *kzg.Blob) (*kzg.Commitment, *kzg.Proof, error) {
+	commitment, err := kzg.BlobToKZGCommitment(blob)
+	if err != nil {
+		return nil, nil, err
+	}
+
+	proof, err := kzg.ComputeBlobKZGProof(blob, commitment)
+	if err != nil {
+		return nil, nil, err
+	}
+
+	return &commitment, &proof, err
+}
+
+// Returns a random blob using the passed seed as entropy
+func getRandBlob(seed int64) kzg.Blob {
+	var blob kzg.Blob
+	bytesPerBlob := GoKZG.ScalarsPerBlob * GoKZG.SerializedScalarSize
+	for i := 0; i < bytesPerBlob; i += GoKZG.SerializedScalarSize {
+		fieldElementBytes := getRandFieldElement(seed + int64(i))
+		copy(blob[i:i+GoKZG.SerializedScalarSize], fieldElementBytes[:])
+	}
+	return blob
+}
+
+// Returns a serialized random field element in big-endian
+func getRandFieldElement(seed int64) [32]byte {
+	bytes := deterministicRandomness(seed)
+	var r fr.Element
+	r.SetBytes(bytes[:])
+
+	return GoKZG.SerializeScalar(r)
+}
+
+func deterministicRandomness(seed int64) [32]byte {
+	// Converts an int64 to a byte slice
+	buf := new(bytes.Buffer)
+	err := binary.Write(buf, binary.BigEndian, seed)
+	if err != nil {
+		logrus.WithError(err).Error("Failed to write int64 to bytes buffer")
+		return [32]byte{}
+	}
+	bytes := buf.Bytes()
+
+	return sha256.Sum256(bytes)
+}

beacon-chain/core/peerdas/validator.go

@@ -0,0 +1,30 @@
+package peerdas
+
+import (
+	beaconState "github.com/OffchainLabs/prysm/v6/beacon-chain/state"
+	"github.com/OffchainLabs/prysm/v6/config/params"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
+	"github.com/pkg/errors"
+)
+
+// ValidatorsCustodyRequirement returns the number of custody groups regarding the validator indices attached to the beacon node.
+// https://github.com/ethereum/consensus-specs/blob/v1.5.0-beta.5/specs/fulu/validator.md#validator-custody
+func ValidatorsCustodyRequirement(state beaconState.ReadOnlyBeaconState, validatorsIndex map[primitives.ValidatorIndex]bool) (uint64, error) {
+	totalNodeBalance := uint64(0)
+	for index := range validatorsIndex {
+		validator, err := state.ValidatorAtIndexReadOnly(index)
+		if err != nil {
+			return 0, errors.Wrapf(err, "validator at index %v", index)
+		}
+
+		totalNodeBalance += validator.EffectiveBalance()
+	}
+
+	beaconConfig := params.BeaconConfig()
+	numberOfCustodyGroup := beaconConfig.NumberOfCustodyGroups
+	validatorCustodyRequirement := beaconConfig.ValidatorCustodyRequirement
+	balancePerAdditionalCustodyGroup := beaconConfig.BalancePerAdditionalCustodyGroup
+
+	count := totalNodeBalance / balancePerAdditionalCustodyGroup
+	return min(max(count, validatorCustodyRequirement), numberOfCustodyGroup), nil
+}

beacon-chain/core/peerdas/validator_test.go

@@ -0,0 +1,55 @@
+package peerdas_test
+
+import (
+	"testing"
+
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
+	state_native "github.com/OffchainLabs/prysm/v6/beacon-chain/state/state-native"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
+	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
+	"github.com/OffchainLabs/prysm/v6/testing/require"
+)
+
+func TestValidatorsCustodyRequirement(t *testing.T) {
+	testCases := []struct {
+		name     string
+		count    uint64
+		expected uint64
+	}{
+		{name: "0 validators", count: 0, expected: 8},
+		{name: "1 validator", count: 1, expected: 8},
+		{name: "8 validators", count: 8, expected: 8},
+		{name: "9 validators", count: 9, expected: 9},
+		{name: "100 validators", count: 100, expected: 100},
+		{name: "128 validators", count: 128, expected: 128},
+		{name: "129 validators", count: 129, expected: 128},
+		{name: "1000 validators", count: 1000, expected: 128},
+	}
+
+	const balance = uint64(32_000_000_000)
+
+	for _, tc := range testCases {
+		t.Run(tc.name, func(t *testing.T) {
+			validators := make([]*ethpb.Validator, 0, tc.count)
+			for range tc.count {
+				validator := &ethpb.Validator{
+					EffectiveBalance: balance,
+				}
+
+				validators = append(validators, validator)
+			}
+
+			validatorsIndex := make(map[primitives.ValidatorIndex]bool)
+			for i := range tc.count {
+				validatorsIndex[primitives.ValidatorIndex(i)] = true
+			}
+
+			beaconState, err := state_native.InitializeFromProtoFulu(&ethpb.BeaconStateElectra{Validators: validators})
+			require.NoError(t, err)
+
+			actual, err := peerdas.ValidatorsCustodyRequirement(beaconState, validatorsIndex)
+			require.NoError(t, err)
+			require.Equal(t, tc.expected, actual)
+		})
+	}
+}

beacon-chain/core/transition/BUILD.bazel

@@ -46,6 +46,7 @@ go_library(
         "//proto/engine/v1:go_default_library",
         "//proto/prysm/v1alpha1:go_default_library",
         "//runtime/version:go_default_library",
+        "//time/slots:go_default_library",
         "@com_github_pkg_errors//:go_default_library",
         "@com_github_prometheus_client_golang//prometheus:go_default_library",
         "@com_github_prometheus_client_golang//prometheus/promauto:go_default_library",

beacon-chain/core/transition/transition.go

@@ -27,7 +27,9 @@ import (
 	"github.com/OffchainLabs/prysm/v6/monitoring/tracing"
 	prysmTrace "github.com/OffchainLabs/prysm/v6/monitoring/tracing/trace"
 	"github.com/OffchainLabs/prysm/v6/runtime/version"
+	"github.com/OffchainLabs/prysm/v6/time/slots"
 	"github.com/pkg/errors"
+	"github.com/sirupsen/logrus"
 	"go.opentelemetry.io/otel/trace"
 )
 
@@ -291,6 +293,8 @@ func ProcessSlotsCore(ctx context.Context, span trace.Span, state state.BeaconSt
 			tracing.AnnotateError(span, err)
 			return nil, errors.Wrap(err, "failed to upgrade state")
 		}
+
+		logBlobLimitIncrease(state.Slot())
 	}
 	return state, nil
 }
@@ -507,3 +511,19 @@ func ProcessEpochPrecompute(ctx context.Context, state state.BeaconState) (state
 	}
 	return state, nil
 }
+
+func logBlobLimitIncrease(slot primitives.Slot) {
+	if !slots.IsEpochStart(slot) {
+		return
+	}
+
+	epoch := slots.ToEpoch(slot)
+	for _, entry := range params.BeaconConfig().BlobSchedule {
+		if entry.Epoch == epoch {
+			log.WithFields(logrus.Fields{
+				"epoch":     epoch,
+				"blobLimit": entry.MaxBlobsPerBlock,
+			}).Info("Blob limit updated")
+		}
+	}
+}

beacon-chain/db/filesystem/BUILD.bazel

@@ -5,6 +5,9 @@ go_library(
     srcs = [
         "blob.go",
         "cache.go",
+        "data_column.go",
+        "data_column_cache.go",
+        "doc.go",
         "iteration.go",
         "layout.go",
         "layout_by_epoch.go",
@@ -17,6 +20,8 @@ go_library(
     importpath = "github.com/OffchainLabs/prysm/v6/beacon-chain/db/filesystem",
     visibility = ["//visibility:public"],
     deps = [
+        "//async:go_default_library",
+        "//async/event:go_default_library",
         "//beacon-chain/db:go_default_library",
         "//beacon-chain/verification:go_default_library",
         "//config/fieldparams:go_default_library",
@@ -41,6 +46,8 @@ go_test(
     srcs = [
         "blob_test.go",
         "cache_test.go",
+        "data_column_cache_test.go",
+        "data_column_test.go",
         "iteration_test.go",
         "layout_test.go",
         "migration_test.go",
@@ -50,6 +57,7 @@ go_test(
     deps = [
         "//beacon-chain/db:go_default_library",
         "//beacon-chain/verification:go_default_library",
+        "//config/fieldparams:go_default_library",
         "//config/params:go_default_library",
         "//consensus-types/primitives:go_default_library",
         "//encoding/bytesutil:go_default_library",

beacon-chain/db/filesystem/data_column_cache.go

@@ -0,0 +1,220 @@
+package filesystem
+
+import (
+	"sync"
+
+	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
+	"github.com/OffchainLabs/prysm/v6/config/params"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
+	"github.com/pkg/errors"
+)
+
+var errDataColumnIndexOutOfBounds = errors.New("data column index too high")
+
+// DataColumnStorageSummary represents cached information about the DataColumnSidecars on disk for each root the cache knows about.
+type DataColumnStorageSummary struct {
+	epoch primitives.Epoch
+	mask  [fieldparams.NumberOfColumns]bool
+}
+
+// NewDataColumnStorageSummary creates a new DataColumnStorageSummary for a given epoch and mask.
+func NewDataColumnStorageSummary(epoch primitives.Epoch, mask [fieldparams.NumberOfColumns]bool) DataColumnStorageSummary {
+	return DataColumnStorageSummary{
+		epoch: epoch,
+		mask:  mask,
+	}
+}
+
+// HasIndex returns true if the DataColumnSidecar at the given index is available in the filesystem.
+func (s DataColumnStorageSummary) HasIndex(index uint64) bool {
+	if index >= uint64(fieldparams.NumberOfColumns) {
+		return false
+	}
+	return s.mask[index]
+}
+
+// Count returns the number of available data columns.
+func (s DataColumnStorageSummary) Count() uint64 {
+	count := uint64(0)
+
+	for _, available := range s.mask {
+		if available {
+			count++
+		}
+	}
+
+	return count
+}
+
+// AllAvailable returns true if we have all data columns for corresponding indices.
+func (s DataColumnStorageSummary) AllAvailable(indices map[uint64]bool) bool {
+	if len(indices) > len(s.mask) {
+		return false
+	}
+
+	for index := range indices {
+		if !s.mask[index] {
+			return false
+		}
+	}
+
+	return true
+}
+
+// DataColumnStorageSummarizer can be used to receive a summary of metadata about data columns on disk for a given root.
+// The DataColumnStorageSummary can be used to check which indices (if any) are available for a given block by root.
+type DataColumnStorageSummarizer interface {
+	Summary(root [fieldparams.RootLength]byte) DataColumnStorageSummary
+}
+
+type dataColumnStorageSummaryCache struct {
+	mu                 sync.RWMutex
+	dataColumnCount    float64
+	lowestCachedEpoch  primitives.Epoch
+	highestCachedEpoch primitives.Epoch
+	cache              map[[fieldparams.RootLength]byte]DataColumnStorageSummary
+}
+
+var _ DataColumnStorageSummarizer = &dataColumnStorageSummaryCache{}
+
+func newDataColumnStorageSummaryCache() *dataColumnStorageSummaryCache {
+	return &dataColumnStorageSummaryCache{
+		cache:             make(map[[fieldparams.RootLength]byte]DataColumnStorageSummary),
+		lowestCachedEpoch: params.BeaconConfig().FarFutureEpoch,
+	}
+}
+
+// Summary returns the DataColumnStorageSummary for `root`.
+// The DataColumnStorageSummary can be used to check for the presence of DataColumnSidecars based on Index.
+func (sc *dataColumnStorageSummaryCache) Summary(root [fieldparams.RootLength]byte) DataColumnStorageSummary {
+	sc.mu.RLock()
+	defer sc.mu.RUnlock()
+
+	return sc.cache[root]
+}
+
+func (sc *dataColumnStorageSummaryCache) HighestEpoch() primitives.Epoch {
+	sc.mu.RLock()
+	defer sc.mu.RUnlock()
+	return sc.highestCachedEpoch
+}
+
+// set updates the cache.
+func (sc *dataColumnStorageSummaryCache) set(dataColumnsIdent DataColumnsIdent) error {
+	numberOfColumns := params.BeaconConfig().NumberOfColumns
+
+	sc.mu.Lock()
+	defer sc.mu.Unlock()
+
+	summary := sc.cache[dataColumnsIdent.Root]
+	summary.epoch = dataColumnsIdent.Epoch
+
+	count := uint64(0)
+	for _, index := range dataColumnsIdent.Indices {
+		if index >= numberOfColumns {
+			return errDataColumnIndexOutOfBounds
+		}
+
+		if summary.mask[index] {
+			continue
+		}
+
+		count++
+
+		summary.mask[index] = true
+		sc.lowestCachedEpoch = min(sc.lowestCachedEpoch, dataColumnsIdent.Epoch)
+		sc.highestCachedEpoch = max(sc.highestCachedEpoch, dataColumnsIdent.Epoch)
+	}
+
+	sc.cache[dataColumnsIdent.Root] = summary
+
+	countFloat := float64(count)
+	sc.dataColumnCount += countFloat
+	dataColumnDiskCount.Set(sc.dataColumnCount)
+	dataColumnWrittenCounter.Add(countFloat)
+
+	return nil
+}
+
+// get returns the DataColumnStorageSummary for the given block root.
+// If the root is not in the cache, the second return value will be false.
+func (sc *dataColumnStorageSummaryCache) get(blockRoot [fieldparams.RootLength]byte) (DataColumnStorageSummary, bool) {
+	sc.mu.RLock()
+	defer sc.mu.RUnlock()
+
+	v, ok := sc.cache[blockRoot]
+	return v, ok
+}
+
+// evict removes the DataColumnStorageSummary for the given block root from the cache.
+func (s *dataColumnStorageSummaryCache) evict(blockRoot [fieldparams.RootLength]byte) int {
+	deleted := 0
+
+	s.mu.Lock()
+	defer s.mu.Unlock()
+
+	summary, ok := s.cache[blockRoot]
+	if !ok {
+		return 0
+	}
+
+	for i := range summary.mask {
+		if summary.mask[i] {
+			deleted += 1
+		}
+	}
+
+	delete(s.cache, blockRoot)
+	if deleted > 0 {
+		s.dataColumnCount -= float64(deleted)
+		dataColumnDiskCount.Set(s.dataColumnCount)
+	}
+
+	// The lowest and highest cached epoch may no longer be valid here,
+	// but is not worth the effort to recalculate.
+
+	return deleted
+}
+
+// pruneUpTo removes all entries from the cache up to the given target epoch included.
+func (sc *dataColumnStorageSummaryCache) pruneUpTo(targetEpoch primitives.Epoch) uint64 {
+	sc.mu.Lock()
+	defer sc.mu.Unlock()
+
+	prunedCount := uint64(0)
+	newLowestCachedEpoch := params.BeaconConfig().FarFutureEpoch
+	newHighestCachedEpoch := primitives.Epoch(0)
+
+	for blockRoot, summary := range sc.cache {
+		epoch := summary.epoch
+
+		if epoch > targetEpoch {
+			newLowestCachedEpoch = min(newLowestCachedEpoch, epoch)
+			newHighestCachedEpoch = max(newHighestCachedEpoch, epoch)
+		}
+
+		if epoch <= targetEpoch {
+			for i := range summary.mask {
+				if summary.mask[i] {
+					prunedCount += 1
+				}
+			}
+
+			delete(sc.cache, blockRoot)
+		}
+	}
+
+	if prunedCount > 0 {
+		sc.lowestCachedEpoch = newLowestCachedEpoch
+		sc.highestCachedEpoch = newHighestCachedEpoch
+		sc.dataColumnCount -= float64(prunedCount)
+		dataColumnDiskCount.Set(sc.dataColumnCount)
+	}
+
+	return prunedCount
+}
+
+// clear removes all entries from the cache.
+func (sc *dataColumnStorageSummaryCache) clear() uint64 {
+	return sc.pruneUpTo(params.BeaconConfig().FarFutureEpoch)
+}

beacon-chain/db/filesystem/data_column_cache_test.go

@@ -0,0 +1,235 @@
+package filesystem
+
+import (
+	"testing"
+
+	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
+	"github.com/OffchainLabs/prysm/v6/config/params"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
+	"github.com/OffchainLabs/prysm/v6/testing/require"
+)
+
+func TestHasIndex(t *testing.T) {
+	summary := NewDataColumnStorageSummary(0, [fieldparams.NumberOfColumns]bool{false, true})
+
+	hasIndex := summary.HasIndex(1_000_000)
+	require.Equal(t, false, hasIndex)
+
+	hasIndex = summary.HasIndex(0)
+	require.Equal(t, false, hasIndex)
+
+	hasIndex = summary.HasIndex(1)
+	require.Equal(t, true, hasIndex)
+}
+
+func TestCount(t *testing.T) {
+	summary := NewDataColumnStorageSummary(0, [fieldparams.NumberOfColumns]bool{false, true, false, true})
+
+	count := summary.Count()
+	require.Equal(t, uint64(2), count)
+}
+
+func TestAllAvailableDataColumns(t *testing.T) {
+	const count = uint64(1_000)
+
+	summary := NewDataColumnStorageSummary(0, [fieldparams.NumberOfColumns]bool{false, true, false, true})
+
+	indices := make(map[uint64]bool, count)
+	for i := range count {
+		indices[i] = true
+	}
+
+	allAvailable := summary.AllAvailable(indices)
+	require.Equal(t, false, allAvailable)
+
+	indices = map[uint64]bool{1: true, 2: true}
+	allAvailable = summary.AllAvailable(indices)
+	require.Equal(t, false, allAvailable)
+
+	indices = map[uint64]bool{1: true, 3: true}
+	allAvailable = summary.AllAvailable(indices)
+	require.Equal(t, true, allAvailable)
+}
+
+func TestSummary(t *testing.T) {
+	root := [fieldparams.RootLength]byte{}
+
+	summaryCache := newDataColumnStorageSummaryCache()
+	expected := NewDataColumnStorageSummary(0, [fieldparams.NumberOfColumns]bool{})
+	actual := summaryCache.Summary(root)
+	require.DeepEqual(t, expected, actual)
+
+	summaryCache = newDataColumnStorageSummaryCache()
+	expected = NewDataColumnStorageSummary(0, [fieldparams.NumberOfColumns]bool{true, false, true, false})
+	summaryCache.cache[root] = expected
+	actual = summaryCache.Summary(root)
+	require.DeepEqual(t, expected, actual)
+}
+
+func TestHighestEpoch(t *testing.T) {
+	root1 := [fieldparams.RootLength]byte{1}
+	root2 := [fieldparams.RootLength]byte{2}
+	root3 := [fieldparams.RootLength]byte{3}
+
+	summaryCache := newDataColumnStorageSummaryCache()
+	actual := summaryCache.HighestEpoch()
+	require.Equal(t, primitives.Epoch(0), actual)
+
+	err := summaryCache.set(DataColumnsIdent{Root: root1, Epoch: 42, Indices: []uint64{1, 3}})
+	require.NoError(t, err)
+	require.Equal(t, primitives.Epoch(42), summaryCache.HighestEpoch())
+
+	err = summaryCache.set(DataColumnsIdent{Root: root2, Epoch: 43, Indices: []uint64{1, 3}})
+	require.NoError(t, err)
+	require.Equal(t, primitives.Epoch(43), summaryCache.HighestEpoch())
+
+	err = summaryCache.set(DataColumnsIdent{Root: root3, Epoch: 40, Indices: []uint64{1, 3}})
+	require.NoError(t, err)
+	require.Equal(t, primitives.Epoch(43), summaryCache.HighestEpoch())
+}
+
+func TestSet(t *testing.T) {
+	t.Run("Index out of bounds", func(t *testing.T) {
+		summaryCache := newDataColumnStorageSummaryCache()
+		err := summaryCache.set(DataColumnsIdent{Indices: []uint64{1_000_000}})
+		require.ErrorIs(t, err, errDataColumnIndexOutOfBounds)
+		require.Equal(t, params.BeaconConfig().FarFutureEpoch, summaryCache.lowestCachedEpoch)
+		require.Equal(t, 0, len(summaryCache.cache))
+	})
+
+	t.Run("Nominal", func(t *testing.T) {
+		root1 := [fieldparams.RootLength]byte{1}
+		root2 := [fieldparams.RootLength]byte{2}
+
+		summaryCache := newDataColumnStorageSummaryCache()
+
+		err := summaryCache.set(DataColumnsIdent{Root: root1, Epoch: 42, Indices: []uint64{1, 3}})
+		require.NoError(t, err)
+		require.Equal(t, primitives.Epoch(42), summaryCache.lowestCachedEpoch)
+		require.Equal(t, 1, len(summaryCache.cache))
+		expected := DataColumnStorageSummary{epoch: 42, mask: [fieldparams.NumberOfColumns]bool{false, true, false, true}}
+		actual := summaryCache.cache[root1]
+		require.DeepEqual(t, expected, actual)
+
+		err = summaryCache.set(DataColumnsIdent{Root: root1, Epoch: 42, Indices: []uint64{0, 1}})
+		require.NoError(t, err)
+		require.Equal(t, primitives.Epoch(42), summaryCache.lowestCachedEpoch)
+		require.Equal(t, 1, len(summaryCache.cache))
+		expected = DataColumnStorageSummary{epoch: 42, mask: [fieldparams.NumberOfColumns]bool{true, true, false, true}}
+		actual = summaryCache.cache[root1]
+		require.DeepEqual(t, expected, actual)
+
+		err = summaryCache.set(DataColumnsIdent{Root: root2, Epoch: 43, Indices: []uint64{1}})
+		require.NoError(t, err)
+		require.Equal(t, primitives.Epoch(42), summaryCache.lowestCachedEpoch) // Epoch 42 is still the lowest
+		require.Equal(t, 2, len(summaryCache.cache))
+		expected = DataColumnStorageSummary{epoch: 43, mask: [fieldparams.NumberOfColumns]bool{false, true}}
+		actual = summaryCache.cache[root2]
+		require.DeepEqual(t, expected, actual)
+	})
+}
+
+func TestGet(t *testing.T) {
+	t.Run("Not in cache", func(t *testing.T) {
+		summaryCache := newDataColumnStorageSummaryCache()
+		root := [fieldparams.RootLength]byte{}
+		_, ok := summaryCache.get(root)
+		require.Equal(t, false, ok)
+	})
+
+	t.Run("In cache", func(t *testing.T) {
+		root := [fieldparams.RootLength]byte{}
+		summaryCache := newDataColumnStorageSummaryCache()
+		summaryCache.cache[root] = NewDataColumnStorageSummary(42, [fieldparams.NumberOfColumns]bool{true, false, true, false})
+		actual, ok := summaryCache.get(root)
+		require.Equal(t, true, ok)
+		expected := NewDataColumnStorageSummary(42, [fieldparams.NumberOfColumns]bool{true, false, true, false})
+		require.DeepEqual(t, expected, actual)
+	})
+}
+
+func TestEvict(t *testing.T) {
+	t.Run("No eviction", func(t *testing.T) {
+		root := [fieldparams.RootLength]byte{}
+		summaryCache := newDataColumnStorageSummaryCache()
+
+		evicted := summaryCache.evict(root)
+		require.Equal(t, 0, evicted)
+	})
+
+	t.Run("Eviction", func(t *testing.T) {
+		root1 := [fieldparams.RootLength]byte{1}
+		root2 := [fieldparams.RootLength]byte{2}
+		summaryCache := newDataColumnStorageSummaryCache()
+		summaryCache.cache[root1] = NewDataColumnStorageSummary(42, [fieldparams.NumberOfColumns]bool{true, false, true, false})
+		summaryCache.cache[root2] = NewDataColumnStorageSummary(43, [fieldparams.NumberOfColumns]bool{false, true, false, true})
+
+		evicted := summaryCache.evict(root1)
+		require.Equal(t, 2, evicted)
+		require.Equal(t, 1, len(summaryCache.cache))
+
+		_, ok := summaryCache.cache[root1]
+		require.Equal(t, false, ok)
+
+		_, ok = summaryCache.cache[root2]
+		require.Equal(t, true, ok)
+	})
+}
+
+func TestPruneUpTo(t *testing.T) {
+	t.Run("No pruning", func(t *testing.T) {
+		summaryCache := newDataColumnStorageSummaryCache()
+		err := summaryCache.set(DataColumnsIdent{Root: [fieldparams.RootLength]byte{1}, Epoch: 42, Indices: []uint64{1}})
+		require.NoError(t, err)
+
+		err = summaryCache.set(DataColumnsIdent{Root: [fieldparams.RootLength]byte{2}, Epoch: 43, Indices: []uint64{2, 4}})
+		require.NoError(t, err)
+
+		count := summaryCache.pruneUpTo(41)
+		require.Equal(t, uint64(0), count)
+		require.Equal(t, 2, len(summaryCache.cache))
+		require.Equal(t, primitives.Epoch(42), summaryCache.lowestCachedEpoch)
+	})
+
+	t.Run("Pruning", func(t *testing.T) {
+		summaryCache := newDataColumnStorageSummaryCache()
+		err := summaryCache.set(DataColumnsIdent{Root: [fieldparams.RootLength]byte{1}, Epoch: 42, Indices: []uint64{1}})
+		require.NoError(t, err)
+
+		err = summaryCache.set(DataColumnsIdent{Root: [fieldparams.RootLength]byte{2}, Epoch: 44, Indices: []uint64{2, 4}})
+		require.NoError(t, err)
+
+		err = summaryCache.set(DataColumnsIdent{Root: [fieldparams.RootLength]byte{3}, Epoch: 45, Indices: []uint64{2, 4}})
+		require.NoError(t, err)
+
+		count := summaryCache.pruneUpTo(42)
+		require.Equal(t, uint64(1), count)
+		require.Equal(t, 2, len(summaryCache.cache))
+		require.Equal(t, primitives.Epoch(44), summaryCache.lowestCachedEpoch)
+
+		count = summaryCache.pruneUpTo(45)
+		require.Equal(t, uint64(4), count)
+		require.Equal(t, 0, len(summaryCache.cache))
+		require.Equal(t, params.BeaconConfig().FarFutureEpoch, summaryCache.lowestCachedEpoch)
+		require.Equal(t, primitives.Epoch(0), summaryCache.highestCachedEpoch)
+
+	})
+
+	t.Run("Clear", func(t *testing.T) {
+		summaryCache := newDataColumnStorageSummaryCache()
+		err := summaryCache.set(DataColumnsIdent{Root: [fieldparams.RootLength]byte{1}, Epoch: 42, Indices: []uint64{1}})
+		require.NoError(t, err)
+
+		err = summaryCache.set(DataColumnsIdent{Root: [fieldparams.RootLength]byte{2}, Epoch: 44, Indices: []uint64{2, 4}})
+		require.NoError(t, err)
+
+		err = summaryCache.set(DataColumnsIdent{Root: [fieldparams.RootLength]byte{3}, Epoch: 45, Indices: []uint64{2, 4}})
+		require.NoError(t, err)
+
+		count := summaryCache.clear()
+		require.Equal(t, uint64(5), count)
+		require.Equal(t, 0, len(summaryCache.cache))
+		require.Equal(t, params.BeaconConfig().FarFutureEpoch, summaryCache.lowestCachedEpoch)
+		require.Equal(t, primitives.Epoch(0), summaryCache.highestCachedEpoch)
+	})
+}

beacon-chain/db/filesystem/data_column_test.go

@@ -0,0 +1,742 @@
+package filesystem
+
+import (
+	"context"
+	"encoding/binary"
+	"os"
+	"testing"
+
+	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
+	"github.com/OffchainLabs/prysm/v6/config/params"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
+	"github.com/OffchainLabs/prysm/v6/testing/require"
+	"github.com/OffchainLabs/prysm/v6/testing/util"
+	"github.com/spf13/afero"
+)
+
+func TestNewDataColumnStorage(t *testing.T) {
+	ctx := context.Background()
+
+	t.Run("No base path", func(t *testing.T) {
+		_, err := NewDataColumnStorage(ctx)
+		require.ErrorIs(t, err, errNoBasePath)
+	})
+
+	t.Run("Nominal", func(t *testing.T) {
+		dir := t.TempDir()
+
+		storage, err := NewDataColumnStorage(ctx, WithDataColumnBasePath(dir))
+		require.NoError(t, err)
+		require.Equal(t, dir, storage.base)
+	})
+}
+
+func TestWarmCache(t *testing.T) {
+	storage, err := NewDataColumnStorage(
+		context.Background(),
+		WithDataColumnBasePath(t.TempDir()),
+		WithDataColumnRetentionEpochs(10_000),
+	)
+	require.NoError(t, err)
+
+	_, verifiedRoDataColumnSidecars := util.CreateTestVerifiedRoDataColumnSidecars(
+		t,
+		util.DataColumnsParamsByRoot{
+			{0}: {
+				{Slot: 33, ColumnIndex: 2, DataColumn: []byte{1, 2, 3}}, // Period 0 - Epoch 1
+				{Slot: 33, ColumnIndex: 4, DataColumn: []byte{2, 3, 4}}, // Period 0 - Epoch 1
+			},
+			{1}: {
+				{Slot: 128_002, ColumnIndex: 2, DataColumn: []byte{1, 2, 3}}, // Period 0 - Epoch 4000
+				{Slot: 128_002, ColumnIndex: 4, DataColumn: []byte{2, 3, 4}}, // Period 0 - Epoch 4000
+			},
+			{2}: {
+				{Slot: 128_003, ColumnIndex: 1, DataColumn: []byte{1, 2, 3}}, // Period 0 - Epoch 4000
+				{Slot: 128_003, ColumnIndex: 3, DataColumn: []byte{2, 3, 4}}, // Period 0 - Epoch 4000
+			},
+			{3}: {
+				{Slot: 128_034, ColumnIndex: 2, DataColumn: []byte{1, 2, 3}}, // Period 0 - Epoch 4001
+				{Slot: 128_034, ColumnIndex: 4, DataColumn: []byte{2, 3, 4}}, // Period 0 - Epoch 4001
+			},
+			{4}: {
+				{Slot: 131_138, ColumnIndex: 2, DataColumn: []byte{1, 2, 3}}, // Period 1 - Epoch 4098
+			},
+			{5}: {
+				{Slot: 131_138, ColumnIndex: 1, DataColumn: []byte{1, 2, 3}}, // Period 1 - Epoch 4098
+			},
+			{6}: {
+				{Slot: 131_168, ColumnIndex: 0, DataColumn: []byte{1, 2, 3}}, // Period 1 - Epoch 4099
+			},
+		},
+	)
+
+	err = storage.Save(verifiedRoDataColumnSidecars)
+	require.NoError(t, err)
+
+	storage.retentionEpochs = 4_096
+
+	storage.WarmCache()
+	require.Equal(t, primitives.Epoch(4_000), storage.cache.lowestCachedEpoch)
+	require.Equal(t, 6, len(storage.cache.cache))
+
+	summary, ok := storage.cache.get([fieldparams.RootLength]byte{1})
+	require.Equal(t, true, ok)
+	require.DeepEqual(t, DataColumnStorageSummary{epoch: 4_000, mask: [fieldparams.NumberOfColumns]bool{false, false, true, false, true}}, summary)
+
+	summary, ok = storage.cache.get([fieldparams.RootLength]byte{2})
+	require.Equal(t, true, ok)
+	require.DeepEqual(t, DataColumnStorageSummary{epoch: 4_000, mask: [fieldparams.NumberOfColumns]bool{false, true, false, true}}, summary)
+
+	summary, ok = storage.cache.get([fieldparams.RootLength]byte{3})
+	require.Equal(t, true, ok)
+	require.DeepEqual(t, DataColumnStorageSummary{epoch: 4_001, mask: [fieldparams.NumberOfColumns]bool{false, false, true, false, true}}, summary)
+
+	summary, ok = storage.cache.get([fieldparams.RootLength]byte{4})
+	require.Equal(t, true, ok)
+	require.DeepEqual(t, DataColumnStorageSummary{epoch: 4_098, mask: [fieldparams.NumberOfColumns]bool{false, false, true}}, summary)
+
+	summary, ok = storage.cache.get([fieldparams.RootLength]byte{5})
+	require.Equal(t, true, ok)
+	require.DeepEqual(t, DataColumnStorageSummary{epoch: 4_098, mask: [fieldparams.NumberOfColumns]bool{false, true}}, summary)
+
+	summary, ok = storage.cache.get([fieldparams.RootLength]byte{6})
+	require.Equal(t, true, ok)
+	require.DeepEqual(t, DataColumnStorageSummary{epoch: 4_099, mask: [fieldparams.NumberOfColumns]bool{true}}, summary)
+}
+
+func TestSaveDataColumnsSidecars(t *testing.T) {
+	t.Run("wrong numbers of columns", func(t *testing.T) {
+		cfg := params.BeaconConfig().Copy()
+		cfg.NumberOfColumns = 0
+		params.OverrideBeaconConfig(cfg)
+		params.SetupTestConfigCleanup(t)
+
+		_, verifiedRoDataColumnSidecars := util.CreateTestVerifiedRoDataColumnSidecars(
+			t,
+			util.DataColumnsParamsByRoot{
+				{}: {{ColumnIndex: 12}, {ColumnIndex: 1_000_000}, {ColumnIndex: 48}},
+			},
+		)
+
+		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
+		err := dataColumnStorage.Save(verifiedRoDataColumnSidecars)
+		require.ErrorIs(t, err, errWrongNumberOfColumns)
+	})
+
+	t.Run("one of the column index is too large", func(t *testing.T) {
+		_, verifiedRoDataColumnSidecars := util.CreateTestVerifiedRoDataColumnSidecars(
+			t,
+			util.DataColumnsParamsByRoot{{}: {{ColumnIndex: 12}, {ColumnIndex: 1_000_000}, {ColumnIndex: 48}}},
+		)
+
+		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
+		err := dataColumnStorage.Save(verifiedRoDataColumnSidecars)
+		require.ErrorIs(t, err, errDataColumnIndexTooLarge)
+	})
+
+	t.Run("different slots", func(t *testing.T) {
+		_, verifiedRoDataColumnSidecars := util.CreateTestVerifiedRoDataColumnSidecars(
+			t,
+			util.DataColumnsParamsByRoot{
+				{}: {
+					{Slot: 1, ColumnIndex: 12, DataColumn: []byte{1, 2, 3}},
+					{Slot: 2, ColumnIndex: 12, DataColumn: []byte{1, 2, 3}},
+				},
+			},
+		)
+
+		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
+		err := dataColumnStorage.Save(verifiedRoDataColumnSidecars)
+		require.ErrorIs(t, err, errDataColumnSidecarsFromDifferentSlots)
+	})
+
+	t.Run("new file - no data columns to save", func(t *testing.T) {
+		_, verifiedRoDataColumnSidecars := util.CreateTestVerifiedRoDataColumnSidecars(
+			t,
+			util.DataColumnsParamsByRoot{{}: {}},
+		)
+
+		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
+		err := dataColumnStorage.Save(verifiedRoDataColumnSidecars)
+		require.NoError(t, err)
+	})
+
+	t.Run("new file - different data column size", func(t *testing.T) {
+		_, verifiedRoDataColumnSidecars := util.CreateTestVerifiedRoDataColumnSidecars(
+			t,
+			util.DataColumnsParamsByRoot{
+				{}: {
+					{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}},
+					{ColumnIndex: 11, DataColumn: []byte{1, 2, 3, 4}},
+				},
+			},
+		)
+
+		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
+		err := dataColumnStorage.Save(verifiedRoDataColumnSidecars)
+		require.ErrorIs(t, err, errWrongSszEncodedDataColumnSidecarSize)
+	})
+
+	t.Run("existing file - wrong incoming SSZ encoded size", func(t *testing.T) {
+		_, verifiedRoDataColumnSidecars := util.CreateTestVerifiedRoDataColumnSidecars(
+			t,
+			util.DataColumnsParamsByRoot{{1}: {{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}}}},
+		)
+
+		// Save data columns into a file.
+		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
+		err := dataColumnStorage.Save(verifiedRoDataColumnSidecars)
+		require.NoError(t, err)
+
+		// Build a data column sidecar for the same block but with a different
+		// column index and an different SSZ encoded size.
+		_, verifiedRoDataColumnSidecars = util.CreateTestVerifiedRoDataColumnSidecars(
+			t,
+			util.DataColumnsParamsByRoot{{1}: {{ColumnIndex: 13, DataColumn: []byte{1, 2, 3, 4}}}},
+		)
+
+		// Try to rewrite the file.
+		err = dataColumnStorage.Save(verifiedRoDataColumnSidecars)
+		require.ErrorIs(t, err, errWrongSszEncodedDataColumnSidecarSize)
+	})
+
+	t.Run("nominal", func(t *testing.T) {
+		_, inputVerifiedRoDataColumnSidecars := util.CreateTestVerifiedRoDataColumnSidecars(
+			t,
+			util.DataColumnsParamsByRoot{
+				{1}: {
+					{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}},
+					{ColumnIndex: 11, DataColumn: []byte{3, 4, 5}},
+					{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}}, // OK if duplicate
+					{ColumnIndex: 13, DataColumn: []byte{6, 7, 8}},
+				},
+				{2}: {
+					{ColumnIndex: 12, DataColumn: []byte{3, 4, 5}},
+					{ColumnIndex: 13, DataColumn: []byte{6, 7, 8}},
+				},
+			},
+		)
+
+		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
+		err := dataColumnStorage.Save(inputVerifiedRoDataColumnSidecars)
+		require.NoError(t, err)
+
+		_, inputVerifiedRoDataColumnSidecars = util.CreateTestVerifiedRoDataColumnSidecars(
+			t,
+			util.DataColumnsParamsByRoot{
+				{1}: {
+					{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}}, // OK if duplicate
+					{ColumnIndex: 15, DataColumn: []byte{2, 3, 4}},
+					{ColumnIndex: 1, DataColumn: []byte{2, 3, 4}},
+				},
+				{3}: {
+					{ColumnIndex: 6, DataColumn: []byte{3, 4, 5}},
+					{ColumnIndex: 2, DataColumn: []byte{6, 7, 8}},
+				},
+			},
+		)
+
+		err = dataColumnStorage.Save(inputVerifiedRoDataColumnSidecars)
+		require.NoError(t, err)
+
+		type fixture struct {
+			fileName         string
+			blockRoot        [fieldparams.RootLength]byte
+			expectedIndices  [mandatoryNumberOfColumns]byte
+			dataColumnParams []util.DataColumnParams
+		}
+
+		fixtures := []fixture{
+			{
+				fileName:  "0/0/0x0100000000000000000000000000000000000000000000000000000000000000.sszs",
+				blockRoot: [fieldparams.RootLength]byte{1},
+				expectedIndices: [mandatoryNumberOfColumns]byte{
+					0, nonZeroOffset + 4, 0, 0, 0, 0, 0, 0,
+					0, 0, 0, nonZeroOffset + 1, nonZeroOffset, nonZeroOffset + 2, 0, nonZeroOffset + 3,
+					// The rest is filled with zeroes.
+				},
+				dataColumnParams: []util.DataColumnParams{
+					{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}},
+					{ColumnIndex: 11, DataColumn: []byte{3, 4, 5}},
+					{ColumnIndex: 13, DataColumn: []byte{6, 7, 8}},
+					{ColumnIndex: 15, DataColumn: []byte{2, 3, 4}},
+					{ColumnIndex: 1, DataColumn: []byte{2, 3, 4}},
+				},
+			},
+			{
+				fileName:  "0/0/0x0200000000000000000000000000000000000000000000000000000000000000.sszs",
+				blockRoot: [fieldparams.RootLength]byte{2},
+				expectedIndices: [mandatoryNumberOfColumns]byte{
+					0, 0, 0, 0, 0, 0, 0, 0,
+					0, 0, 0, 0, nonZeroOffset, nonZeroOffset + 1, 0, 0,
+					// The rest is filled with zeroes.
+				},
+				dataColumnParams: []util.DataColumnParams{
+					{ColumnIndex: 12, DataColumn: []byte{3, 4, 5}},
+					{ColumnIndex: 13, DataColumn: []byte{6, 7, 8}},
+				},
+			},
+			{
+				fileName:  "0/0/0x0300000000000000000000000000000000000000000000000000000000000000.sszs",
+				blockRoot: [fieldparams.RootLength]byte{3},
+				expectedIndices: [mandatoryNumberOfColumns]byte{
+					0, 0, nonZeroOffset + 1, 0, 0, 0, nonZeroOffset, 0,
+					// The rest is filled with zeroes.
+				},
+				dataColumnParams: []util.DataColumnParams{
+					{ColumnIndex: 6, DataColumn: []byte{3, 4, 5}},
+					{ColumnIndex: 2, DataColumn: []byte{6, 7, 8}},
+				},
+			},
+		}
+
+		for _, fixture := range fixtures {
+			// Build expected data column sidecars.
+			_, expectedDataColumnSidecars := util.CreateTestVerifiedRoDataColumnSidecars(
+				t,
+				util.DataColumnsParamsByRoot{fixture.blockRoot: fixture.dataColumnParams},
+			)
+
+			// Build expected bytes.
+			firstSszEncodedDataColumnSidecar, err := expectedDataColumnSidecars[0].MarshalSSZ()
+			require.NoError(t, err)
+
+			dataColumnSidecarsCount := len(expectedDataColumnSidecars)
+			sszEncodedDataColumnSidecarSize := len(firstSszEncodedDataColumnSidecar)
+
+			sszEncodedDataColumnSidecars := make([]byte, 0, dataColumnSidecarsCount*sszEncodedDataColumnSidecarSize)
+			sszEncodedDataColumnSidecars = append(sszEncodedDataColumnSidecars, firstSszEncodedDataColumnSidecar...)
+			for _, dataColumnSidecar := range expectedDataColumnSidecars[1:] {
+				sszEncodedDataColumnSidecar, err := dataColumnSidecar.MarshalSSZ()
+				require.NoError(t, err)
+				sszEncodedDataColumnSidecars = append(sszEncodedDataColumnSidecars, sszEncodedDataColumnSidecar...)
+			}
+
+			var encodedSszEncodedDataColumnSidecarSize [sidecarByteLenSize]byte
+			binary.BigEndian.PutUint32(encodedSszEncodedDataColumnSidecarSize[:], uint32(sszEncodedDataColumnSidecarSize))
+
+			expectedBytes := make([]byte, 0, headerSize+dataColumnSidecarsCount*sszEncodedDataColumnSidecarSize)
+			expectedBytes = append(expectedBytes, []byte{0x01}...)
+			expectedBytes = append(expectedBytes, encodedSszEncodedDataColumnSidecarSize[:]...)
+			expectedBytes = append(expectedBytes, fixture.expectedIndices[:]...)
+			expectedBytes = append(expectedBytes, sszEncodedDataColumnSidecars...)
+
+			// Check the actual content of the file.
+			actualBytes, err := afero.ReadFile(dataColumnStorage.fs, fixture.fileName)
+			require.NoError(t, err)
+			require.DeepSSZEqual(t, expectedBytes, actualBytes)
+
+			// Check the summary.
+			indices := map[uint64]bool{}
+			for _, dataColumnParam := range fixture.dataColumnParams {
+				indices[dataColumnParam.ColumnIndex] = true
+			}
+
+			summary := dataColumnStorage.Summary(fixture.blockRoot)
+			for index := range uint64(mandatoryNumberOfColumns) {
+				require.Equal(t, indices[index], summary.HasIndex(index))
+			}
+
+			err = dataColumnStorage.Remove(fixture.blockRoot)
+			require.NoError(t, err)
+
+			summary = dataColumnStorage.Summary(fixture.blockRoot)
+			for index := range uint64(mandatoryNumberOfColumns) {
+				require.Equal(t, false, summary.HasIndex(index))
+			}
+
+			_, err = afero.ReadFile(dataColumnStorage.fs, fixture.fileName)
+			require.ErrorIs(t, err, os.ErrNotExist)
+		}
+	})
+}
+
+func TestGetDataColumnSidecars(t *testing.T) {
+	t.Run("not found", func(t *testing.T) {
+		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
+
+		verifiedRODataColumnSidecars, err := dataColumnStorage.Get([fieldparams.RootLength]byte{1}, []uint64{12, 13, 14})
+		require.NoError(t, err)
+		require.Equal(t, 0, len(verifiedRODataColumnSidecars))
+	})
+
+	t.Run("nominal", func(t *testing.T) {
+		_, expectedVerifiedRoDataColumnSidecars := util.CreateTestVerifiedRoDataColumnSidecars(
+			t,
+			util.DataColumnsParamsByRoot{
+				{1}: {
+					{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}},
+					{ColumnIndex: 14, DataColumn: []byte{2, 3, 4}},
+				},
+			},
+		)
+
+		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
+		err := dataColumnStorage.Save(expectedVerifiedRoDataColumnSidecars)
+		require.NoError(t, err)
+
+		verifiedRODataColumnSidecars, err := dataColumnStorage.Get([fieldparams.RootLength]byte{1}, nil)
+		require.NoError(t, err)
+		require.DeepSSZEqual(t, expectedVerifiedRoDataColumnSidecars, verifiedRODataColumnSidecars)
+
+		verifiedRODataColumnSidecars, err = dataColumnStorage.Get([fieldparams.RootLength]byte{1}, []uint64{12, 13, 14})
+		require.NoError(t, err)
+		require.DeepSSZEqual(t, expectedVerifiedRoDataColumnSidecars, verifiedRODataColumnSidecars)
+	})
+}
+
+func TestRemove(t *testing.T) {
+	t.Run("not found", func(t *testing.T) {
+		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
+		err := dataColumnStorage.Remove([fieldparams.RootLength]byte{1})
+		require.NoError(t, err)
+	})
+
+	t.Run("nominal", func(t *testing.T) {
+		_, inputVerifiedRoDataColumnSidecars := util.CreateTestVerifiedRoDataColumnSidecars(
+			t,
+			util.DataColumnsParamsByRoot{
+				{1}: {
+					{Slot: 32, ColumnIndex: 10, DataColumn: []byte{1, 2, 3}},
+					{Slot: 32, ColumnIndex: 11, DataColumn: []byte{2, 3, 4}},
+				},
+				{2}: {
+					{Slot: 33, ColumnIndex: 10, DataColumn: []byte{1, 2, 3}},
+					{Slot: 33, ColumnIndex: 11, DataColumn: []byte{2, 3, 4}},
+				},
+			},
+		)
+
+		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
+		err := dataColumnStorage.Save(inputVerifiedRoDataColumnSidecars)
+		require.NoError(t, err)
+
+		err = dataColumnStorage.Remove([fieldparams.RootLength]byte{1})
+		require.NoError(t, err)
+
+		summary := dataColumnStorage.Summary([fieldparams.RootLength]byte{1})
+		require.Equal(t, primitives.Epoch(0), summary.epoch)
+		require.Equal(t, uint64(0), summary.Count())
+
+		summary = dataColumnStorage.Summary([fieldparams.RootLength]byte{2})
+		require.Equal(t, primitives.Epoch(1), summary.epoch)
+		require.Equal(t, uint64(2), summary.Count())
+
+		actual, err := dataColumnStorage.Get([fieldparams.RootLength]byte{1}, nil)
+		require.NoError(t, err)
+		require.Equal(t, 0, len(actual))
+
+		actual, err = dataColumnStorage.Get([fieldparams.RootLength]byte{2}, nil)
+		require.NoError(t, err)
+		require.Equal(t, 2, len(actual))
+	})
+}
+
+func TestClear(t *testing.T) {
+	_, inputVerifiedRoDataColumnSidecars := util.CreateTestVerifiedRoDataColumnSidecars(
+		t,
+		util.DataColumnsParamsByRoot{
+			{1}: {{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}}},
+			{2}: {{ColumnIndex: 13, DataColumn: []byte{6, 7, 8}}},
+		},
+	)
+
+	_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
+	err := dataColumnStorage.Save(inputVerifiedRoDataColumnSidecars)
+	require.NoError(t, err)
+
+	filePaths := []string{
+		"0/0/0x0100000000000000000000000000000000000000000000000000000000000000.sszs",
+		"0/0/0x0200000000000000000000000000000000000000000000000000000000000000.sszs",
+	}
+
+	for _, filePath := range filePaths {
+		_, err = afero.ReadFile(dataColumnStorage.fs, filePath)
+		require.NoError(t, err)
+	}
+
+	err = dataColumnStorage.Clear()
+	require.NoError(t, err)
+
+	summary := dataColumnStorage.Summary([fieldparams.RootLength]byte{1})
+	for index := range uint64(mandatoryNumberOfColumns) {
+		require.Equal(t, false, summary.HasIndex(index))
+	}
+
+	for _, filePath := range filePaths {
+		_, err = afero.ReadFile(dataColumnStorage.fs, filePath)
+		require.ErrorIs(t, err, os.ErrNotExist)
+	}
+}
+
+func TestMetadata(t *testing.T) {
+	t.Run("wrong version", func(t *testing.T) {
+		_, verifiedRoDataColumnSidecars := util.CreateTestVerifiedRoDataColumnSidecars(
+			t,
+			util.DataColumnsParamsByRoot{
+				{1}: {{ColumnIndex: 12, DataColumn: []byte{1, 2, 3}}},
+			},
+		)
+
+		// Save data columns into a file.
+		_, dataColumnStorage := NewEphemeralDataColumnStorageAndFs(t)
+		err := dataColumnStorage.Save(verifiedRoDataColumnSidecars)
+		require.NoError(t, err)
+
+		// Alter the version.
+		const filePath = "0/0/0x0100000000000000000000000000000000000000000000000000000000000000.sszs"
+		file, err := dataColumnStorage.fs.OpenFile(filePath, os.O_WRONLY, os.FileMode(0600))
+		require.NoError(t, err)
+
+		count, err := file.Write([]byte{42})
+		require.NoError(t, err)
+		require.Equal(t, 1, count)
+
+		// Try to read the metadata.
+		_, err = dataColumnStorage.metadata(file)
+		require.ErrorIs(t, err, errWrongVersion)
+
+		err = file.Close()
+		require.NoError(t, err)
+	})
+}
+
+func TestNewStorageIndices(t *testing.T) {
+	t.Run("wrong number of columns", func(t *testing.T) {
+		_, err := newStorageIndices(nil)
+		require.ErrorIs(t, err, errWrongNumberOfColumns)
+	})
+
+	t.Run("nominal", func(t *testing.T) {
+		var indices [mandatoryNumberOfColumns]byte
+		indices[0] = 1
+
+		storageIndices, err := newStorageIndices(indices[:])
+		require.NoError(t, err)
+		require.Equal(t, indices, storageIndices.indices)
+	})
+}
+
+func TestStorageIndicesGet(t *testing.T) {
+	t.Run("index too large", func(t *testing.T) {
+		var indices storageIndices
+		_, _, err := indices.get(1_000_000)
+		require.ErrorIs(t, errDataColumnIndexTooLarge, err)
+	})
+
+	t.Run("index not set", func(t *testing.T) {
+		const expected = false
+		var indices storageIndices
+		actual, _, err := indices.get(0)
+		require.NoError(t, err)
+		require.Equal(t, expected, actual)
+	})
+
+	t.Run("index set", func(t *testing.T) {
+		const (
+			expectedOk       = true
+			expectedPosition = int64(3)
+		)
+
+		indices := storageIndices{indices: [mandatoryNumberOfColumns]byte{0, 131}}
+		actualOk, actualPosition, err := indices.get(1)
+		require.NoError(t, err)
+		require.Equal(t, expectedOk, actualOk)
+		require.Equal(t, expectedPosition, actualPosition)
+	})
+}
+
+func TestStorageIndicesLen(t *testing.T) {
+	const expected = int64(2)
+	indices := storageIndices{count: 2}
+	actual := indices.len()
+	require.Equal(t, expected, actual)
+}
+
+func TestStorageIndicesAll(t *testing.T) {
+	expectedIndices := []uint64{1, 3}
+	indices := storageIndices{indices: [mandatoryNumberOfColumns]byte{0, 131, 0, 128}}
+	actualIndices := indices.all()
+	require.DeepEqual(t, expectedIndices, actualIndices)
+}
+
+func TestStorageIndicesSet(t *testing.T) {
+	t.Run("data column index too large", func(t *testing.T) {
+		var indices storageIndices
+		err := indices.set(1_000_000, 0)
+		require.ErrorIs(t, errDataColumnIndexTooLarge, err)
+	})
+
+	t.Run("position too large", func(t *testing.T) {
+		var indices storageIndices
+		err := indices.set(0, 255)
+		require.ErrorIs(t, errDataColumnIndexTooLarge, err)
+	})
+
+	t.Run("nominal", func(t *testing.T) {
+		expected := [mandatoryNumberOfColumns]byte{0, 0, 128, 0, 131}
+		var storageIndices storageIndices
+		require.Equal(t, int64(0), storageIndices.len())
+
+		err := storageIndices.set(2, 1)
+		require.NoError(t, err)
+		require.Equal(t, int64(1), storageIndices.len())
+
+		err = storageIndices.set(4, 3)
+		require.NoError(t, err)
+		require.Equal(t, int64(2), storageIndices.len())
+
+		err = storageIndices.set(2, 0)
+		require.NoError(t, err)
+		require.Equal(t, int64(2), storageIndices.len())
+
+		actual := storageIndices.indices
+		require.Equal(t, expected, actual)
+	})
+}
+
+func TestPrune(t *testing.T) {
+	t.Run(("nothing to prune"), func(t *testing.T) {
+		dir := t.TempDir()
+		dataColumnStorage, err := NewDataColumnStorage(context.Background(), WithDataColumnBasePath(dir))
+		require.NoError(t, err)
+
+		dataColumnStorage.prune()
+	})
+	t.Run("nominal", func(t *testing.T) {
+		var compareSlices = func(left, right []string) bool {
+			if len(left) != len(right) {
+				return false
+			}
+
+			leftMap := make(map[string]bool, len(left))
+			for _, leftItem := range left {
+				leftMap[leftItem] = true
+			}
+
+			for _, rightItem := range right {
+				if _, ok := leftMap[rightItem]; !ok {
+					return false
+				}
+			}
+
+			return true
+		}
+		_, verifiedRoDataColumnSidecars := util.CreateTestVerifiedRoDataColumnSidecars(
+			t,
+			util.DataColumnsParamsByRoot{
+				{0}: {
+					{Slot: 33, ColumnIndex: 2, DataColumn: []byte{1, 2, 3}}, // Period 0 - Epoch 1
+					{Slot: 33, ColumnIndex: 4, DataColumn: []byte{2, 3, 4}}, // Period 0 - Epoch 1
+				},
+				{1}: {
+					{Slot: 128_002, ColumnIndex: 2, DataColumn: []byte{1, 2, 3}}, // Period 0 - Epoch 4000
+					{Slot: 128_002, ColumnIndex: 4, DataColumn: []byte{2, 3, 4}}, // Period 0 - Epoch 4000
+				},
+				{2}: {
+					{Slot: 128_003, ColumnIndex: 1, DataColumn: []byte{1, 2, 3}}, // Period 0 - Epoch 4000
+					{Slot: 128_003, ColumnIndex: 3, DataColumn: []byte{2, 3, 4}}, // Period 0 - Epoch 4000
+				},
+				{3}: {
+					{Slot: 131_138, ColumnIndex: 2, DataColumn: []byte{1, 2, 3}}, // Period 1 - Epoch 4098
+					{Slot: 131_138, ColumnIndex: 3, DataColumn: []byte{1, 2, 3}}, // Period 1 - Epoch 4098
+				},
+				{4}: {
+					{Slot: 131_169, ColumnIndex: 2, DataColumn: []byte{1, 2, 3}}, // Period 1 - Epoch 4099
+					{Slot: 131_169, ColumnIndex: 3, DataColumn: []byte{1, 2, 3}}, // Period 1 - Epoch 4099
+				},
+				{5}: {
+					{Slot: 262_144, ColumnIndex: 2, DataColumn: []byte{1, 2, 3}}, // Period 2 - Epoch 8192
+					{Slot: 262_144, ColumnIndex: 3, DataColumn: []byte{1, 2, 3}}, // Period 2 - Epoch 8292
+				},
+			},
+		)
+
+		dir := t.TempDir()
+		dataColumnStorage, err := NewDataColumnStorage(context.Background(), WithDataColumnBasePath(dir), WithDataColumnRetentionEpochs(10_000))
+		require.NoError(t, err)
+
+		err = dataColumnStorage.Save(verifiedRoDataColumnSidecars)
+		require.NoError(t, err)
+
+		dirs, err := listDir(dataColumnStorage.fs, ".")
+		require.NoError(t, err)
+		require.Equal(t, true, compareSlices([]string{"0", "1", "2"}, dirs))
+
+		dirs, err = listDir(dataColumnStorage.fs, "0")
+		require.NoError(t, err)
+		require.Equal(t, true, compareSlices([]string{"1", "4000"}, dirs))
+
+		dirs, err = listDir(dataColumnStorage.fs, "1")
+		require.NoError(t, err)
+		require.Equal(t, true, compareSlices([]string{"4099", "4098"}, dirs))
+
+		dirs, err = listDir(dataColumnStorage.fs, "2")
+		require.NoError(t, err)
+		require.Equal(t, true, compareSlices([]string{"8192"}, dirs))
+
+		dirs, err = listDir(dataColumnStorage.fs, "0/1")
+		require.NoError(t, err)
+		require.Equal(t, true, compareSlices([]string{"0x0000000000000000000000000000000000000000000000000000000000000000.sszs"}, dirs))
+
+		dirs, err = listDir(dataColumnStorage.fs, "0/4000")
+		require.NoError(t, err)
+		require.Equal(t, true, compareSlices([]string{
+			"0x0200000000000000000000000000000000000000000000000000000000000000.sszs",
+			"0x0100000000000000000000000000000000000000000000000000000000000000.sszs",
+		}, dirs))
+
+		dirs, err = listDir(dataColumnStorage.fs, "1/4098")
+		require.NoError(t, err)
+		require.Equal(t, true, compareSlices([]string{"0x0300000000000000000000000000000000000000000000000000000000000000.sszs"}, dirs))
+
+		dirs, err = listDir(dataColumnStorage.fs, "1/4099")
+		require.NoError(t, err)
+		require.Equal(t, true, compareSlices([]string{"0x0400000000000000000000000000000000000000000000000000000000000000.sszs"}, dirs))
+
+		dirs, err = listDir(dataColumnStorage.fs, "2/8192")
+		require.NoError(t, err)
+		require.Equal(t, true, compareSlices([]string{"0x0500000000000000000000000000000000000000000000000000000000000000.sszs"}, dirs))
+
+		_, verifiedRoDataColumnSidecars = util.CreateTestVerifiedRoDataColumnSidecars(
+			t,
+			util.DataColumnsParamsByRoot{
+				{6}: {{Slot: 451_141, ColumnIndex: 2, DataColumn: []byte{1, 2, 3}}}, // Period 3 - Epoch 14_098
+			},
+		)
+
+		err = dataColumnStorage.Save(verifiedRoDataColumnSidecars)
+		require.NoError(t, err)
+
+		// dataColumnStorage.prune(14_098)
+		dataColumnStorage.prune()
+
+		dirs, err = listDir(dataColumnStorage.fs, ".")
+		require.NoError(t, err)
+		require.Equal(t, true, compareSlices([]string{"1", "2", "3"}, dirs))
+
+		dirs, err = listDir(dataColumnStorage.fs, "1")
+		require.NoError(t, err)
+		require.Equal(t, true, compareSlices([]string{"4099"}, dirs))
+
+		dirs, err = listDir(dataColumnStorage.fs, "2")
+		require.NoError(t, err)
+		require.Equal(t, true, compareSlices([]string{"8192"}, dirs))
+
+		dirs, err = listDir(dataColumnStorage.fs, "3")
+		require.NoError(t, err)
+		require.Equal(t, true, compareSlices([]string{"14098"}, dirs))
+
+		dirs, err = listDir(dataColumnStorage.fs, "1/4099")
+		require.NoError(t, err)
+		require.Equal(t, true, compareSlices([]string{"0x0400000000000000000000000000000000000000000000000000000000000000.sszs"}, dirs))
+
+		dirs, err = listDir(dataColumnStorage.fs, "2/8192")
+		require.NoError(t, err)
+		require.Equal(t, true, compareSlices([]string{"0x0500000000000000000000000000000000000000000000000000000000000000.sszs"}, dirs))
+
+		dirs, err = listDir(dataColumnStorage.fs, "3/14098")
+		require.NoError(t, err)
+		require.Equal(t, true, compareSlices([]string{"0x0600000000000000000000000000000000000000000000000000000000000000.sszs"}, dirs))
+	})
+}

beacon-chain/db/filesystem/doc.go

@@ -0,0 +1,104 @@
+package filesystem
+
+// nolint:dupword
+/*
+Data column sidecars storage documentation
+==========================================
+
+File organisation
+-----------------
+- The first byte represents the version of the file structure (up to 0xff = 255).
+  We set it to 0x01.
+  Note: This is not strictly needed, but it will help a lot if, in the future,
+        we want to modify the file structure.
+- The next 4 bytes represents the size of a SSZ encoded data column sidecar.
+  (See the `Computation of the maximum size of a DataColumnSidecar` section to a description
+  of how this value is computed).
+- The next 128 bytes represent the index in the file of a given column.
+  The first bit of each byte in the index is set to 0 if there is no data column,
+  and set to 1 if there is a data column.
+  The remaining 7 bits (from 0 to 127) represent the index of the data column.
+  This sentinel bit is needed to distinguish between the column with index 0 and no column.
+  Example: If the column with index 5 is in the 3th position in the file, then indices[5] = 0x80 + 0x03 = 0x83.
+- The rest of the file is a repeat of the SSZ encoded data columns sidecars.
+
+
+|------------------------------------------|------------------------------------------------------------------------------------|
+|                Byte offset               |                                     Description                                    |
+|------------------------------------------|------------------------------------------------------------------------------------|
+|                                        0 | version (1 byte) | sszEncodedDataColumnSidecarSize (4 bytes) | indices (128 bytes) |
+|133 +   0*sszEncodedDataColumnSidecarSize | sszEncodedDataColumnSidecar (sszEncodedDataColumnSidecarSize bytes)                |
+|133 +   1*sszEncodedDataColumnSidecarSize | sszEncodedDataColumnSidecar (sszEncodedDataColumnSidecarSize bytes)                |
+|133 +   2*sszEncodedDataColumnSidecarSize | sszEncodedDataColumnSidecar (sszEncodedDataColumnSidecarSize bytes)                |
+|                  ...                     |                                 ...                                                |
+|133 + 127*sszEncodedDataColumnSidecarSize | sszEncodedDataColumnSidecar (sszEncodedDataColumnSidecarSize bytes)                |
+|------------------------------------------|------------------------------------------------------------------------------------|
+
+Each file is named after the block root where the data columns were data columns are committed to.
+Example: `0x259c6d2f6a0bb75e2405cea7cb248e5663dc26b9404fd3bcd777afc20de91c1e.sszs`
+
+Database organisation
+---------------------
+SSZ encoded data column sidecars are stored following the `by-epoch` layout.
+- The first layer is a directory corresponding to the `period`, which corresponds to the epoch divided by the 4096.
+- The second layer is a directory corresponding to the epoch.
+- Then all files are stored in the epoch directory.
+
+Example:
+data-columns
+├── 0
+│   ├── 3638
+│   │   ├── 0x259c6d2f6a0bb75e2405cea7cb248e5663dc26b9404fd3bcd777afc20de91c1e.sszs
+│   │   ├── 0x2a855b1f6e9a2f04f8383e336325bf7d5ba02d1eab3ef90ef183736f8c768533.sszs
+│   │   ├── ...
+│   │   ├── 0xeb78e2b2350a71c640f1e96fea9e42f38e65705ab7e6e100c8bc9c589f2c5f2b.sszs
+│   │   └── 0xeb7ee68da988fd20d773d45aad01dd62527734367a146e2b048715bd68a4e370.sszs
+│   └── 3639
+│       ├── 0x0fd231fe95e57936fa44f6c712c490b9e337a481b661dfd46768901e90444330.sszs
+│       ├── 0x1bf5edff6b6ba2b65b1db325ff3312bbb57da461ef2ae651bd741af851aada3a.sszs
+│       ├── ...
+│       ├── 0xa156a527e631f858fee79fab7ef1fde3f6117a2e1201d47c09fbab0c6780c937.sszs
+│       └── 0xcd80bc535ddc467dea1d19e0c39c1160875ccd1989061bcd8ce206e3c1261c87.sszs
+└── 1
+    ├── 4096
+    │   ├── 0x0d244009093e2bedb72eb265280290199e8c7bf1d90d7583c41af40d9f662269.sszs
+    │   ├── 0x11f420928d8de41c50e735caab0369996824a5299c5f054e097965855925697d.sszs
+    │   ├── ...
+    │   ├── 0xbe91fc782877ed400d95c02c61aebfdd592635d11f8e64c94b46abd84f45c967.sszs
+    │   └── 0xf246189f078f02d30173ff74605cf31c9e65b5e463275ebdbeb40476638135ff.sszs
+    └── 4097
+        ├── 0x454d000674793c479e90504c0fe9827b50bb176ae022dab4e37d6a21471ab570.sszs
+        ├── 0xac5eb7437d7190c48cfa863e3c45f96a7f8af371d47ac12ccda07129a06af763.sszs
+        ├── ...
+        ├── 0xb7df30561d9d92ab5fafdd96bca8b44526497c8debf0fc425c7a0770b2abeb83.sszs
+        └── 0xc1dd0b1ae847b6ec62303a36d08c6a4a2e9e3ec4be3ff70551972a0ee3de9c14.sszs
+
+Computation of the maximum size of a DataColumnSidecar
+------------------------------------------------------
+https://github.com/ethereum/consensus-specs/blob/dev/specs/fulu/das-core.md#datacolumnsidecar
+
+
+class DataColumnSidecar(Container):
+    index: ColumnIndex  # Index of column in extended matrix
+    column: List[Cell, MAX_BLOB_COMMITMENTS_PER_BLOCK]
+    kzg_commitments: List[KZGCommitment, MAX_BLOB_COMMITMENTS_PER_BLOCK]
+    kzg_proofs: List[KZGProof, MAX_BLOB_COMMITMENTS_PER_BLOCK]
+    signed_block_header: SignedBeaconBlockHeader
+    kzg_commitments_inclusion_proof: Vector[Bytes32, KZG_COMMITMENTS_INCLUSION_PROOF_DEPTH]
+
+
+- index: 2 bytes (ColumnIndex)
+- `column`: 4,096 (MAX_BLOB_COMMITMENTS_PER_BLOCK) * 64 (FIELD_ELEMENTS_PER_CELL) * 32 bytes (BYTES_PER_FIELD_ELEMENT) = 8,388,608 bytes
+- kzg_commitments: 4,096 (MAX_BLOB_COMMITMENTS_PER_BLOCK) * 48 bytes (KZGCommitment) = 196,608 bytes
+- kzg_proofs: 4,096 (MAX_BLOB_COMMITMENTS_PER_BLOCK) * 48 bytes (KZGProof) = 196,608 bytes
+- signed_block_header: 2 bytes (Slot) + 2 bytes (ValidatorIndex) + 3 * 2 bytes (Root) + 96 bytes (BLSSignature) = 106 bytes
+- kzg_commitments_inclusion_proof: 4 (KZG_COMMITMENTS_INCLUSION_PROOF_DEPTH) * 32 bytes = 128 bytes
+
+TOTAL: 8,782,060 bytes = 70,256,480 bits
+log(70,256,480) / log(2) ~= 26.07
+
+==> 32 bits (4 bytes) are enough to store the maximum size of a data column sidecar.
+
+The maximum size of an SSZ encoded data column can be 2**32 bits = 536,879,912 bytes,
+which left a room of 536,879,912 bytes - 8,782,060 bytes ~= 503 mega bytes to store the extra data needed by SSZ encoding (which is more than enough.)
+*/

beacon-chain/db/filesystem/metrics.go

@@ -6,6 +6,7 @@ import (
 )
 
 var (
+	// Blobs
 	blobBuckets     = []float64{3, 5, 7, 9, 11, 13}
 	blobSaveLatency = promauto.NewHistogram(prometheus.HistogramOpts{
 		Name:    "blob_storage_save_latency",
@@ -33,4 +34,29 @@ var (
 		Name: "blob_disk_bytes",
 		Help: "Approximate number of bytes occupied by blobs in storage",
 	})
+
+	// Data columns
+	dataColumnBuckets     = []float64{3, 5, 7, 9, 11, 13}
+	dataColumnSaveLatency = promauto.NewHistogram(prometheus.HistogramOpts{
+		Name:    "data_column_storage_save_latency",
+		Help:    "Latency of DataColumnSidecar storage save operations in milliseconds",
+		Buckets: dataColumnBuckets,
+	})
+	dataColumnFetchLatency = promauto.NewHistogram(prometheus.HistogramOpts{
+		Name:    "data_column_storage_get_latency",
+		Help:    "Latency of DataColumnSidecar storage get operations in milliseconds",
+		Buckets: dataColumnBuckets,
+	})
+	dataColumnPrunedCounter = promauto.NewCounter(prometheus.CounterOpts{
+		Name: "data_column_pruned",
+		Help: "Number of DataColumnSidecar pruned.",
+	})
+	dataColumnWrittenCounter = promauto.NewCounter(prometheus.CounterOpts{
+		Name: "data_column_written",
+		Help: "Number of DataColumnSidecar written",
+	})
+	dataColumnDiskCount = promauto.NewGauge(prometheus.GaugeOpts{
+		Name: "data_column_disk_count",
+		Help: "Approximate number of data columns in storage",
+	})
 )

beacon-chain/db/filesystem/mock.go

@@ -1,6 +1,7 @@
 package filesystem
 
 import (
+	"context"
 	"testing"
 
 	"github.com/OffchainLabs/prysm/v6/config/params"
@@ -9,6 +10,9 @@ import (
 	"github.com/spf13/afero"
 )
 
+// Blobs
+// -----
+
 // NewEphemeralBlobStorage should only be used for tests.
 // The instance of BlobStorage returned is backed by an in-memory virtual filesystem,
 // improving test performance and simplifying cleanup.
@@ -75,3 +79,41 @@ func NewMockBlobStorageSummarizer(t *testing.T, set map[[32]byte][]int) BlobStor
 	}
 	return c
 }
+
+// Data columns
+// ------------
+
+// NewEphemeralDataColumnStorage should only be used for tests.
+// The instance of DataColumnStorage returned is backed by an in-memory virtual filesystem,
+// improving test performance and simplifying cleanup.
+func NewEphemeralDataColumnStorage(t testing.TB, opts ...DataColumnStorageOption) *DataColumnStorage {
+	return NewWarmedEphemeralDataColumnStorageUsingFs(t, afero.NewMemMapFs(), opts...)
+}
+
+// NewEphemeralDataColumnStorageAndFs can be used by tests that want access to the virtual filesystem
+// in order to interact with it outside the parameters of the DataColumnStorage API.
+func NewEphemeralDataColumnStorageAndFs(t testing.TB, opts ...DataColumnStorageOption) (afero.Fs, *DataColumnStorage) {
+	fs := afero.NewMemMapFs()
+	dcs := NewWarmedEphemeralDataColumnStorageUsingFs(t, fs, opts...)
+	return fs, dcs
+}
+
+func NewWarmedEphemeralDataColumnStorageUsingFs(t testing.TB, fs afero.Fs, opts ...DataColumnStorageOption) *DataColumnStorage {
+	bs := NewEphemeralDataColumnStorageUsingFs(t, fs, opts...)
+	bs.WarmCache()
+	return bs
+}
+
+func NewEphemeralDataColumnStorageUsingFs(t testing.TB, fs afero.Fs, opts ...DataColumnStorageOption) *DataColumnStorage {
+	opts = append(opts,
+		WithDataColumnRetentionEpochs(params.BeaconConfig().MinEpochsForBlobsSidecarsRequest),
+		WithDataColumnFs(fs),
+	)
+
+	bs, err := NewDataColumnStorage(context.Background(), opts...)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	return bs
+}

beacon-chain/db/kv/BUILD.bazel

@@ -59,7 +59,7 @@ go_library(
         "//runtime/version:go_default_library",
         "//time:go_default_library",
         "//time/slots:go_default_library",
-        "@com_github_dgraph_io_ristretto//:go_default_library",
+        "@com_github_dgraph_io_ristretto_v2//:go_default_library",
         "@com_github_ethereum_go_ethereum//common:go_default_library",
         "@com_github_ethereum_go_ethereum//common/hexutil:go_default_library",
         "@com_github_golang_snappy//:go_default_library",

beacon-chain/db/kv/blocks.go

@@ -40,7 +40,7 @@ func (s *Store) Block(ctx context.Context, blockRoot [32]byte) (interfaces.ReadO
 
 func (s *Store) getBlock(ctx context.Context, blockRoot [32]byte, tx *bolt.Tx) (interfaces.ReadOnlySignedBeaconBlock, error) {
 	if v, ok := s.blockCache.Get(string(blockRoot[:])); v != nil && ok {
-		return v.(interfaces.ReadOnlySignedBeaconBlock), nil
+		return v, nil
 	}
 	// This method allows the caller to pass in its tx if one is already open.
 	// Or if a nil value is used, a transaction will be managed intenally.

beacon-chain/db/kv/error.go

@@ -16,9 +16,6 @@ var ErrNotFoundOriginBlockRoot = errors.Wrap(ErrNotFound, "OriginBlockRoot")
 // ErrNotFoundGenesisBlockRoot means no genesis block root was found, indicating the db was not initialized with genesis
 var ErrNotFoundGenesisBlockRoot = errors.Wrap(ErrNotFound, "OriginGenesisRoot")
 
-// ErrNotFoundBackfillBlockRoot is an error specifically for the origin block root getter
-var ErrNotFoundBackfillBlockRoot = errors.Wrap(ErrNotFound, "BackfillBlockRoot")
-
 // ErrNotFoundFeeRecipient is a not found error specifically for the fee recipient getter
 var ErrNotFoundFeeRecipient = errors.Wrap(ErrNotFound, "fee recipient")
 

beacon-chain/db/kv/kv.go

@@ -13,8 +13,10 @@ import (
 	"github.com/OffchainLabs/prysm/v6/config/features"
 	"github.com/OffchainLabs/prysm/v6/config/params"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
 	"github.com/OffchainLabs/prysm/v6/io/file"
-	"github.com/dgraph-io/ristretto"
+	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
+	"github.com/dgraph-io/ristretto/v2"
 	"github.com/pkg/errors"
 	"github.com/prometheus/client_golang/prometheus"
 	"github.com/prometheus/client_golang/prometheus/promauto"
@@ -86,8 +88,8 @@ var blockedBuckets = [][]byte{
 type Store struct {
 	db                  *bolt.DB
 	databasePath        string
-	blockCache          *ristretto.Cache
-	validatorEntryCache *ristretto.Cache
+	blockCache          *ristretto.Cache[string, interfaces.ReadOnlySignedBeaconBlock]
+	validatorEntryCache *ristretto.Cache[[]byte, *ethpb.Validator]
 	stateSummaryCache   *stateSummaryCache
 	ctx                 context.Context
 }
@@ -156,7 +158,7 @@ func NewKVStore(ctx context.Context, dirPath string, opts ...KVStoreOption) (*St
 		return nil, err
 	}
 	boltDB.AllocSize = boltAllocSize
-	blockCache, err := ristretto.NewCache(&ristretto.Config{
+	blockCache, err := ristretto.NewCache(&ristretto.Config[string, interfaces.ReadOnlySignedBeaconBlock]{
 		NumCounters: 1000,           // number of keys to track frequency of (1000).
 		MaxCost:     BlockCacheSize, // maximum cost of cache (1000 Blocks).
 		BufferItems: 64,             // number of keys per Get buffer.
@@ -165,7 +167,7 @@ func NewKVStore(ctx context.Context, dirPath string, opts ...KVStoreOption) (*St
 		return nil, err
 	}
 
-	validatorCache, err := ristretto.NewCache(&ristretto.Config{
+	validatorCache, err := ristretto.NewCache(&ristretto.Config[[]byte, *ethpb.Validator]{
 		NumCounters: NumOfValidatorEntries, // number of entries in cache (2 Million).
 		MaxCost:     ValidatorEntryMaxCost, // maximum size of the cache (64Mb)
 		BufferItems: 64,                    // number of keys per Get buffer.

beacon-chain/db/kv/state.go

@@ -744,14 +744,9 @@ func (s *Store) validatorEntries(ctx context.Context, blockRoot [32]byte) ([]*et
 			// get the entry bytes from the cache or from the DB.
 			v, ok := s.validatorEntryCache.Get(key)
 			if ok {
-				valEntry, vType := v.(*ethpb.Validator)
-				if vType {
-					validatorEntries = append(validatorEntries, valEntry)
-					validatorEntryCacheHit.Inc()
-				} else {
-					// this should never happen, but anyway it's good to bail out if one happens.
-					return errors.New("validator cache does not have proper object type")
-				}
+				valEntry := v
+				validatorEntries = append(validatorEntries, valEntry)
+				validatorEntryCacheHit.Inc()
 			} else {
 				// not in cache, so get it from the DB, decode it and add to the entry list.
 				valEntryBytes := valBkt.Get(key)

beacon-chain/db/kv/state_test.go

@@ -321,15 +321,11 @@ func TestState_CanSaveRetrieveValidatorEntriesFromCache(t *testing.T) {
 		hash, hashErr := stateValidators[i].HashTreeRoot()
 		assert.NoError(t, hashErr)
 
-		data, ok := db.validatorEntryCache.Get(string(hash[:]))
+		data, ok := db.validatorEntryCache.Get(hash[:])
 		assert.Equal(t, true, ok)
 		require.NotNil(t, data)
 
-		valEntry, vType := data.(*ethpb.Validator)
-		assert.Equal(t, true, vType)
-		require.NotNil(t, valEntry)
-
-		require.DeepSSZEqual(t, stateValidators[i], valEntry, "validator entry is not matching")
+		require.DeepSSZEqual(t, stateValidators[i], data, "validator entry is not matching")
 	}
 
 	// check if all the validator entries are still intact in the validator entry bucket.
@@ -447,7 +443,7 @@ func TestState_DeleteState(t *testing.T) {
 		assert.NoError(t, hashErr)
 		v, found := db.validatorEntryCache.Get(hash[:])
 		require.Equal(t, false, found)
-		require.Equal(t, nil, v)
+		require.IsNil(t, v)
 	}
 
 	// check if the index of the first state is deleted.

beacon-chain/execution/log_processing_test.go

@@ -479,6 +479,93 @@ func TestProcessETH2GenesisLog_CorrectNumOfDeposits(t *testing.T) {
 	hook.Reset()
 }
 
+func TestProcessLogs_DepositRequestsStarted(t *testing.T) {
+	params.SetupTestConfigCleanup(t)
+	hook := logTest.NewGlobal()
+	testAcc, err := mock.Setup()
+	require.NoError(t, err, "Unable to set up simulated backend")
+	kvStore := testDB.SetupDB(t)
+	depositCache, err := depositsnapshot.New()
+	require.NoError(t, err)
+	server, endpoint, err := mockExecution.SetupRPCServer()
+	require.NoError(t, err)
+	t.Cleanup(func() {
+		server.Stop()
+	})
+
+	web3Service, err := NewService(context.Background(),
+		WithHttpEndpoint(endpoint),
+		WithDepositContractAddress(testAcc.ContractAddr),
+		WithDatabase(kvStore),
+		WithDepositCache(depositCache),
+	)
+	require.NoError(t, err, "unable to setup web3 ETH1.0 chain service")
+	web3Service = setDefaultMocks(web3Service)
+	web3Service.depositContractCaller, err = contracts.NewDepositContractCaller(testAcc.ContractAddr, testAcc.Backend.Client())
+	require.NoError(t, err)
+	web3Service.rpcClient = &mockExecution.RPCClient{Backend: testAcc.Backend}
+	web3Service.httpLogger = testAcc.Backend.Client()
+	web3Service.latestEth1Data.LastRequestedBlock = 0
+	block, err := testAcc.Backend.Client().BlockByNumber(context.Background(), nil)
+	require.NoError(t, err)
+	web3Service.latestEth1Data.BlockHeight = block.NumberU64()
+	web3Service.latestEth1Data.BlockTime = block.Time()
+	bConfig := params.MinimalSpecConfig().Copy()
+	bConfig.MinGenesisTime = 0
+	bConfig.SecondsPerETH1Block = 1
+	params.OverrideBeaconConfig(bConfig)
+	nConfig := params.BeaconNetworkConfig()
+	nConfig.ContractDeploymentBlock = 0
+	params.OverrideBeaconNetworkConfig(nConfig)
+
+	testAcc.Backend.Commit()
+
+	totalNumOfDeposits := depositsReqForChainStart + 30
+
+	deposits, _, err := util.DeterministicDepositsAndKeys(uint64(totalNumOfDeposits))
+	require.NoError(t, err)
+	_, depositRoots, err := util.DeterministicDepositTrie(len(deposits))
+	require.NoError(t, err)
+	depositOffset := 5
+
+	// 64 Validators are used as size required for beacon-chain to start. This number
+	// is defined in the deposit contract as the number required for the testnet. The actual number
+	// is 2**14
+	for i := 0; i < totalNumOfDeposits; i++ {
+		data := deposits[i].Data
+		testAcc.TxOpts.Value = mock.Amount32Eth()
+		testAcc.TxOpts.GasLimit = 1000000
+		_, err = testAcc.Contract.Deposit(testAcc.TxOpts, data.PublicKey, data.WithdrawalCredentials, data.Signature, depositRoots[i])
+		require.NoError(t, err, "Could not deposit to deposit contract")
+		// pack 8 deposits into a block with an offset of
+		// 5
+		if (i+1)%8 == depositOffset {
+			testAcc.Backend.Commit()
+		}
+	}
+	// Forward the chain to account for the follow distance
+	for i := uint64(0); i < params.BeaconConfig().Eth1FollowDistance; i++ {
+		testAcc.Backend.Commit()
+	}
+	b, err := testAcc.Backend.Client().BlockByNumber(context.Background(), nil)
+	require.NoError(t, err)
+	web3Service.latestEth1Data.BlockHeight = b.NumberU64()
+	web3Service.latestEth1Data.BlockTime = b.Time()
+
+	// Set up our subscriber now to listen for the chain started event.
+	stateChannel := make(chan *feed.Event, 1)
+	stateSub := web3Service.cfg.stateNotifier.StateFeed().Subscribe(stateChannel)
+	defer stateSub.Unsubscribe()
+
+	web3Service.depositRequestsStarted = true
+	web3Service.initPOWService()
+	require.NoError(t, err)
+
+	require.Equal(t, int64(-1), web3Service.lastReceivedMerkleIndex, "Processed deposit logs even when requests are active")
+
+	hook.Reset()
+}
+
 func TestProcessETH2GenesisLog_LargePeriodOfNoLogs(t *testing.T) {
 	params.SetupTestConfigCleanup(t)
 	hook := logTest.NewGlobal()

beacon-chain/execution/service.go

@@ -16,6 +16,7 @@ import (
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/cache"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/cache/depositsnapshot"
 	statefeed "github.com/OffchainLabs/prysm/v6/beacon-chain/core/feed/state"
+	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/transition"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/db"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/execution/types"
@@ -141,6 +142,7 @@ type config struct {
 type Service struct {
 	connectedETH1           bool
 	isRunning               bool
+	depositRequestsStarted  bool
 	processingLock          sync.RWMutex
 	latestEth1DataLock      sync.RWMutex
 	cfg                     *config
@@ -205,7 +207,7 @@ func NewService(ctx context.Context, opts ...Option) (*Service, error) {
 			return nil, err
 		}
 	}
-
+	s.initDepositRequests()
 	eth1Data, err := s.validPowchainData(ctx)
 	if err != nil {
 		return nil, errors.Wrap(err, "unable to validate powchain data")
@@ -463,7 +465,9 @@ func safelyHandlePanic() {
 func (s *Service) handleETH1FollowDistance() {
 	defer safelyHandlePanic()
 	ctx := s.ctx
-
+	if s.depositRequestsStarted {
+		return
+	}
 	// use a 5 minutes timeout for block time, because the max mining time is 278 sec (block 7208027)
 	// (analyzed the time of the block from 2018-09-01 to 2019-02-13)
 	fiveMinutesTimeout := prysmTime.Now().Add(-5 * time.Minute)
@@ -530,25 +534,27 @@ func (s *Service) initPOWService() {
 			s.latestEth1Data.BlockTime = header.Time
 			s.latestEth1DataLock.Unlock()
 
-			if err := s.processPastLogs(ctx); err != nil {
-				err = errors.Wrap(err, "processPastLogs")
-				s.retryExecutionClientConnection(ctx, err)
-				errorLogger(
-					err,
-					"Unable to process past deposit contract logs, perhaps your execution client is not fully synced",
-				)
-				continue
-			}
-			// Cache eth1 headers from our voting period.
-			if err := s.cacheHeadersForEth1DataVote(ctx); err != nil {
-				err = errors.Wrap(err, "cacheHeadersForEth1DataVote")
-				s.retryExecutionClientConnection(ctx, err)
-				if errors.Is(err, errBlockTimeTooLate) {
-					log.WithError(err).Debug("Unable to cache headers for execution client votes")
-				} else {
-					errorLogger(err, "Unable to cache headers for execution client votes")
+			if !s.depositRequestsStarted {
+				if err := s.processPastLogs(ctx); err != nil {
+					err = errors.Wrap(err, "processPastLogs")
+					s.retryExecutionClientConnection(ctx, err)
+					errorLogger(
+						err,
+						"Unable to process past deposit contract logs, perhaps your execution client is not fully synced",
+					)
+					continue
+				}
+				// Cache eth1 headers from our voting period.
+				if err := s.cacheHeadersForEth1DataVote(ctx); err != nil {
+					err = errors.Wrap(err, "cacheHeadersForEth1DataVote")
+					s.retryExecutionClientConnection(ctx, err)
+					if errors.Is(err, errBlockTimeTooLate) {
+						log.WithError(err).Debug("Unable to cache headers for execution client votes")
+					} else {
+						errorLogger(err, "Unable to cache headers for execution client votes")
+					}
+					continue
 				}
-				continue
 			}
 			// Handle edge case with embedded genesis state by fetching genesis header to determine
 			// its height.
@@ -824,7 +830,7 @@ func (s *Service) validPowchainData(ctx context.Context) (*ethpb.ETH1ChainData,
 	if genState == nil || genState.IsNil() {
 		return eth1Data, nil
 	}
-	if eth1Data == nil || !eth1Data.ChainstartData.Chainstarted || !validateDepositContainers(eth1Data.DepositContainers) {
+	if s.depositRequestsStarted || eth1Data == nil || !eth1Data.ChainstartData.Chainstarted || !validateDepositContainers(eth1Data.DepositContainers) {
 		pbState, err := native.ProtobufBeaconStatePhase0(s.preGenesisState.ToProtoUnsafe())
 		if err != nil {
 			return nil, err
@@ -900,6 +906,15 @@ func (s *Service) removeStartupState() {
 	s.cfg.finalizedStateAtStartup = nil
 }
 
+func (s *Service) initDepositRequests() {
+	fState := s.cfg.finalizedStateAtStartup
+	isNil := fState == nil || fState.IsNil()
+	if isNil {
+		return
+	}
+	s.depositRequestsStarted = helpers.DepositRequestsStarted(fState)
+}
+
 func newBlobVerifierFromInitializer(ini *verification.Initializer) verification.NewBlobVerifier {
 	return func(b blocks.ROBlob, reqs []verification.Requirement) verification.BlobVerifier {
 		return ini.NewBlobVerifier(b, reqs)

beacon-chain/forkchoice/doubly-linked-tree/store.go

@@ -267,7 +267,7 @@ func (f *ForkChoice) HighestReceivedBlockSlot() primitives.Slot {
 	return f.store.highestReceivedNode.slot
 }
 
-// HighestReceivedBlockSlotDelay returns the number of slots that the highest
+// HighestReceivedBlockDelay returns the number of slots that the highest
 // received block was late when receiving it
 func (f *ForkChoice) HighestReceivedBlockDelay() primitives.Slot {
 	n := f.store.highestReceivedNode

beacon-chain/node/node.go

@@ -865,6 +865,7 @@ func (b *BeaconNode) registerSyncService(initialSyncComplete chan struct{}, bFil
 		regularsync.WithVerifierWaiter(b.verifyInitWaiter),
 		regularsync.WithAvailableBlocker(bFillStore),
 		regularsync.WithSlasherEnabled(b.slasherEnabled),
+		regularsync.WithLightClientStore(b.lcStore),
 	)
 	return b.services.RegisterService(rs)
 }
@@ -1031,6 +1032,7 @@ func (b *BeaconNode) registerPrometheusService(_ *cli.Context) error {
 	}
 
 	service := prometheus.NewService(
+		b.cliCtx.Context,
 		fmt.Sprintf("%s:%d", b.cliCtx.String(cmd.MonitoringHostFlag.Name), b.cliCtx.Int(flags.MonitoringPortFlag.Name)),
 		b.services,
 		additionalHandlers...,

beacon-chain/p2p/BUILD.bazel

@@ -55,6 +55,7 @@ go_library(
         "//beacon-chain/startup:go_default_library",
         "//cmd/beacon-chain/flags:go_default_library",
         "//config/features:go_default_library",
+        "//config/fieldparams:go_default_library",
         "//config/params:go_default_library",
         "//consensus-types/interfaces:go_default_library",
         "//consensus-types/primitives:go_default_library",

beacon-chain/p2p/broadcaster.go

@@ -285,13 +285,13 @@ func (s *Service) BroadcastLightClientOptimisticUpdate(ctx context.Context, upda
 		return err
 	}
 
-	// TODO: should we check if the update is too early or too late to broadcast?
-
 	if err := s.broadcastObject(ctx, update, lcOptimisticToTopic(forkDigest)); err != nil {
+		log.WithError(err).Debug("Failed to broadcast light client optimistic update")
 		err := errors.Wrap(err, "could not publish message")
 		tracing.AnnotateError(span, err)
 		return err
 	}
+	log.Debug("Successfully broadcast light client optimistic update")
 
 	return nil
 }
@@ -311,13 +311,13 @@ func (s *Service) BroadcastLightClientFinalityUpdate(ctx context.Context, update
 		return err
 	}
 
-	// TODO: should we check if the update is too early or too late to broadcast?
-
 	if err := s.broadcastObject(ctx, update, lcFinalityToTopic(forkDigest)); err != nil {
+		log.WithError(err).Debug("Failed to broadcast light client finality update")
 		err := errors.Wrap(err, "could not publish message")
 		tracing.AnnotateError(span, err)
 		return err
 	}
+	log.Debug("Successfully broadcast light client finality update")
 
 	return nil
 }

beacon-chain/p2p/gossip_scoring_params.go

@@ -44,6 +44,12 @@ const (
 	// blsToExecutionChangeWeight specifies the scoring weight that we apply to
 	// our bls to execution topic.
 	blsToExecutionChangeWeight = 0.05
+	// lightClientOptimisticUpdateWeight specifies the scoring weight that we apply to
+	// our light client optimistic update topic.
+	lightClientOptimisticUpdateWeight = 0.05
+	// lightClientFinalityUpdateWeight specifies the scoring weight that we apply to
+	// our light client finality update topic.
+	lightClientFinalityUpdateWeight = 0.05
 
 	// maxInMeshScore describes the max score a peer can attain from being in the mesh.
 	maxInMeshScore = 10
@@ -124,6 +130,10 @@ func (s *Service) topicScoreParams(topic string) (*pubsub.TopicScoreParams, erro
 	case strings.Contains(topic, GossipBlobSidecarMessage):
 		// TODO(Deneb): Using the default block scoring. But this should be updated.
 		return defaultBlockTopicParams(), nil
+	case strings.Contains(topic, GossipLightClientOptimisticUpdateMessage):
+		return defaultLightClientOptimisticUpdateTopicParams(), nil
+	case strings.Contains(topic, GossipLightClientFinalityUpdateMessage):
+		return defaultLightClientFinalityUpdateTopicParams(), nil
 	default:
 		return nil, errors.Errorf("unrecognized topic provided for parameter registration: %s", topic)
 	}
@@ -503,6 +513,50 @@ func defaultBlsToExecutionChangeTopicParams() *pubsub.TopicScoreParams {
 	}
 }
 
+func defaultLightClientOptimisticUpdateTopicParams() *pubsub.TopicScoreParams {
+	return &pubsub.TopicScoreParams{
+		TopicWeight:                     lightClientOptimisticUpdateWeight,
+		TimeInMeshWeight:                maxInMeshScore / inMeshCap(),
+		TimeInMeshQuantum:               inMeshTime(),
+		TimeInMeshCap:                   inMeshCap(),
+		FirstMessageDeliveriesWeight:    2,
+		FirstMessageDeliveriesDecay:     scoreDecay(oneHundredEpochs),
+		FirstMessageDeliveriesCap:       5,
+		MeshMessageDeliveriesWeight:     0,
+		MeshMessageDeliveriesDecay:      0,
+		MeshMessageDeliveriesCap:        0,
+		MeshMessageDeliveriesThreshold:  0,
+		MeshMessageDeliveriesWindow:     0,
+		MeshMessageDeliveriesActivation: 0,
+		MeshFailurePenaltyWeight:        0,
+		MeshFailurePenaltyDecay:         0,
+		InvalidMessageDeliveriesWeight:  -2000,
+		InvalidMessageDeliveriesDecay:   scoreDecay(invalidDecayPeriod),
+	}
+}
+
+func defaultLightClientFinalityUpdateTopicParams() *pubsub.TopicScoreParams {
+	return &pubsub.TopicScoreParams{
+		TopicWeight:                     lightClientFinalityUpdateWeight,
+		TimeInMeshWeight:                maxInMeshScore / inMeshCap(),
+		TimeInMeshQuantum:               inMeshTime(),
+		TimeInMeshCap:                   inMeshCap(),
+		FirstMessageDeliveriesWeight:    2,
+		FirstMessageDeliveriesDecay:     scoreDecay(oneHundredEpochs),
+		FirstMessageDeliveriesCap:       5,
+		MeshMessageDeliveriesWeight:     0,
+		MeshMessageDeliveriesDecay:      0,
+		MeshMessageDeliveriesCap:        0,
+		MeshMessageDeliveriesThreshold:  0,
+		MeshMessageDeliveriesWindow:     0,
+		MeshMessageDeliveriesActivation: 0,
+		MeshFailurePenaltyWeight:        0,
+		MeshFailurePenaltyDecay:         0,
+		InvalidMessageDeliveriesWeight:  -2000,
+		InvalidMessageDeliveriesDecay:   scoreDecay(invalidDecayPeriod),
+	}
+}
+
 func oneSlotDuration() time.Duration {
 	return time.Duration(params.BeaconConfig().SecondsPerSlot) * time.Second
 }

beacon-chain/p2p/gossip_scoring_params_test.go

@@ -75,4 +75,6 @@ func TestLoggingParameters(_ *testing.T) {
 	logGossipParameters("testing", defaultAttesterSlashingTopicParams())
 	logGossipParameters("testing", defaultProposerSlashingTopicParams())
 	logGossipParameters("testing", defaultVoluntaryExitTopicParams())
+	logGossipParameters("testing", defaultLightClientOptimisticUpdateTopicParams())
+	logGossipParameters("testing", defaultLightClientFinalityUpdateTopicParams())
 }

beacon-chain/p2p/gossip_topic_mappings_test.go

@@ -73,6 +73,12 @@ func TestGossipTopicMappings_CorrectType(t *testing.T) {
 	pMessage = GossipTopicMappings(AggregateAndProofSubnetTopicFormat, altairForkEpoch)
 	_, ok = pMessage.(*ethpb.SignedAggregateAttestationAndProof)
 	assert.Equal(t, true, ok)
+	pMessage = GossipTopicMappings(LightClientOptimisticUpdateTopicFormat, altairForkEpoch)
+	_, ok = pMessage.(*ethpb.LightClientOptimisticUpdateAltair)
+	assert.Equal(t, true, ok)
+	pMessage = GossipTopicMappings(LightClientFinalityUpdateTopicFormat, altairForkEpoch)
+	_, ok = pMessage.(*ethpb.LightClientFinalityUpdateAltair)
+	assert.Equal(t, true, ok)
 
 	// Bellatrix Fork
 	pMessage = GossipTopicMappings(BlockSubnetTopicFormat, bellatrixForkEpoch)
@@ -87,6 +93,12 @@ func TestGossipTopicMappings_CorrectType(t *testing.T) {
 	pMessage = GossipTopicMappings(AggregateAndProofSubnetTopicFormat, bellatrixForkEpoch)
 	_, ok = pMessage.(*ethpb.SignedAggregateAttestationAndProof)
 	assert.Equal(t, true, ok)
+	pMessage = GossipTopicMappings(LightClientOptimisticUpdateTopicFormat, bellatrixForkEpoch)
+	_, ok = pMessage.(*ethpb.LightClientOptimisticUpdateAltair)
+	assert.Equal(t, true, ok)
+	pMessage = GossipTopicMappings(LightClientFinalityUpdateTopicFormat, bellatrixForkEpoch)
+	_, ok = pMessage.(*ethpb.LightClientFinalityUpdateAltair)
+	assert.Equal(t, true, ok)
 
 	// Capella Fork
 	pMessage = GossipTopicMappings(BlockSubnetTopicFormat, capellaForkEpoch)
@@ -101,6 +113,12 @@ func TestGossipTopicMappings_CorrectType(t *testing.T) {
 	pMessage = GossipTopicMappings(AggregateAndProofSubnetTopicFormat, capellaForkEpoch)
 	_, ok = pMessage.(*ethpb.SignedAggregateAttestationAndProof)
 	assert.Equal(t, true, ok)
+	pMessage = GossipTopicMappings(LightClientOptimisticUpdateTopicFormat, capellaForkEpoch)
+	_, ok = pMessage.(*ethpb.LightClientOptimisticUpdateCapella)
+	assert.Equal(t, true, ok)
+	pMessage = GossipTopicMappings(LightClientFinalityUpdateTopicFormat, capellaForkEpoch)
+	_, ok = pMessage.(*ethpb.LightClientFinalityUpdateCapella)
+	assert.Equal(t, true, ok)
 
 	// Deneb Fork
 	pMessage = GossipTopicMappings(BlockSubnetTopicFormat, denebForkEpoch)
@@ -115,6 +133,12 @@ func TestGossipTopicMappings_CorrectType(t *testing.T) {
 	pMessage = GossipTopicMappings(AggregateAndProofSubnetTopicFormat, denebForkEpoch)
 	_, ok = pMessage.(*ethpb.SignedAggregateAttestationAndProof)
 	assert.Equal(t, true, ok)
+	pMessage = GossipTopicMappings(LightClientOptimisticUpdateTopicFormat, denebForkEpoch)
+	_, ok = pMessage.(*ethpb.LightClientOptimisticUpdateDeneb)
+	assert.Equal(t, true, ok)
+	pMessage = GossipTopicMappings(LightClientFinalityUpdateTopicFormat, denebForkEpoch)
+	_, ok = pMessage.(*ethpb.LightClientFinalityUpdateDeneb)
+	assert.Equal(t, true, ok)
 
 	// Electra Fork
 	pMessage = GossipTopicMappings(BlockSubnetTopicFormat, electraForkEpoch)
@@ -129,4 +153,10 @@ func TestGossipTopicMappings_CorrectType(t *testing.T) {
 	pMessage = GossipTopicMappings(AggregateAndProofSubnetTopicFormat, electraForkEpoch)
 	_, ok = pMessage.(*ethpb.SignedAggregateAttestationAndProofElectra)
 	assert.Equal(t, true, ok)
+	pMessage = GossipTopicMappings(LightClientOptimisticUpdateTopicFormat, electraForkEpoch)
+	_, ok = pMessage.(*ethpb.LightClientOptimisticUpdateDeneb)
+	assert.Equal(t, true, ok)
+	pMessage = GossipTopicMappings(LightClientFinalityUpdateTopicFormat, electraForkEpoch)
+	_, ok = pMessage.(*ethpb.LightClientFinalityUpdateElectra)
+	assert.Equal(t, true, ok)
 }

beacon-chain/p2p/interfaces.go

@@ -5,6 +5,7 @@ import (
 
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/encoder"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/peers"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
 	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
 	"github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1/metadata"
 	"github.com/ethereum/go-ethereum/p2p/enr"
@@ -36,6 +37,8 @@ type Broadcaster interface {
 	BroadcastAttestation(ctx context.Context, subnet uint64, att ethpb.Att) error
 	BroadcastSyncCommitteeMessage(ctx context.Context, subnet uint64, sMsg *ethpb.SyncCommitteeMessage) error
 	BroadcastBlob(ctx context.Context, subnet uint64, blob *ethpb.BlobSidecar) error
+	BroadcastLightClientOptimisticUpdate(ctx context.Context, update interfaces.LightClientOptimisticUpdate) error
+	BroadcastLightClientFinalityUpdate(ctx context.Context, update interfaces.LightClientFinalityUpdate) error
 }
 
 // SetStreamHandler configures p2p to handle streams of a certain topic ID.

beacon-chain/p2p/rpc_topic_mappings.go

@@ -4,6 +4,7 @@ import (
 	"reflect"
 
 	p2ptypes "github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/types"
+	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
 	"github.com/OffchainLabs/prysm/v6/config/params"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
 	pb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
@@ -43,6 +44,18 @@ const BlobSidecarsByRangeName = "/blob_sidecars_by_range"
 // BlobSidecarsByRootName is the name for the BlobSidecarsByRoot v1 message topic.
 const BlobSidecarsByRootName = "/blob_sidecars_by_root"
 
+// LightClientBootstrapName is the name for the LightClientBootstrap message topic,
+const LightClientBootstrapName = "/light_client_bootstrap"
+
+// LightClientUpdatesByRangeName is the name for the LightClientUpdatesByRange topic.
+const LightClientUpdatesByRangeName = "/light_client_updates_by_range"
+
+// LightClientFinalityUpdateName is the name for the LightClientFinalityUpdate topic.
+const LightClientFinalityUpdateName = "/light_client_finality_update"
+
+// LightClientOptimisticUpdateName is the name for the LightClientOptimisticUpdate topic.
+const LightClientOptimisticUpdateName = "/light_client_optimistic_update"
+
 const (
 	// V1 RPC Topics
 	// RPCStatusTopicV1 defines the v1 topic for the status rpc method.
@@ -66,6 +79,15 @@ const (
 	// /eth2/beacon_chain/req/blob_sidecars_by_root/1/
 	RPCBlobSidecarsByRootTopicV1 = protocolPrefix + BlobSidecarsByRootName + SchemaVersionV1
 
+	// RPCLightClientBootstrapTopicV1 is a topic for requesting a light client bootstrap.
+	RPCLightClientBootstrapTopicV1 = protocolPrefix + LightClientBootstrapName + SchemaVersionV1
+	// RPCLightClientUpdatesByRangeTopicV1 is a topic for requesting light client updates by range.
+	RPCLightClientUpdatesByRangeTopicV1 = protocolPrefix + LightClientUpdatesByRangeName + SchemaVersionV1
+	// RPCLightClientFinalityUpdateTopicV1 is a topic for requesting a light client finality update.
+	RPCLightClientFinalityUpdateTopicV1 = protocolPrefix + LightClientFinalityUpdateName + SchemaVersionV1
+	// RPCLightClientOptimisticUpdateTopicV1 is a topic for requesting a light client Optimistic update.
+	RPCLightClientOptimisticUpdateTopicV1 = protocolPrefix + LightClientOptimisticUpdateName + SchemaVersionV1
+
 	// V2 RPC Topics
 	// RPCBlocksByRangeTopicV2 defines v2 the topic for the blocks by range rpc method.
 	RPCBlocksByRangeTopicV2 = protocolPrefix + BeaconBlocksByRangeMessageName + SchemaVersionV2
@@ -101,6 +123,12 @@ var RPCTopicMappings = map[string]interface{}{
 	RPCBlobSidecarsByRangeTopicV1: new(pb.BlobSidecarsByRangeRequest),
 	// BlobSidecarsByRoot v1 Message
 	RPCBlobSidecarsByRootTopicV1: new(p2ptypes.BlobSidecarsByRootReq),
+
+	// Light client
+	RPCLightClientBootstrapTopicV1:        new([fieldparams.RootLength]byte),
+	RPCLightClientUpdatesByRangeTopicV1:   new(pb.LightClientUpdatesByRangeRequest),
+	RPCLightClientFinalityUpdateTopicV1:   new(interface{}),
+	RPCLightClientOptimisticUpdateTopicV1: new(interface{}),
 }
 
 // Maps all registered protocol prefixes.
@@ -111,14 +139,18 @@ var protocolMapping = map[string]bool{
 // Maps all the protocol message names for the different rpc
 // topics.
 var messageMapping = map[string]bool{
-	StatusMessageName:              true,
-	GoodbyeMessageName:             true,
-	BeaconBlocksByRangeMessageName: true,
-	BeaconBlocksByRootsMessageName: true,
-	PingMessageName:                true,
-	MetadataMessageName:            true,
-	BlobSidecarsByRangeName:        true,
-	BlobSidecarsByRootName:         true,
+	StatusMessageName:               true,
+	GoodbyeMessageName:              true,
+	BeaconBlocksByRangeMessageName:  true,
+	BeaconBlocksByRootsMessageName:  true,
+	PingMessageName:                 true,
+	MetadataMessageName:             true,
+	BlobSidecarsByRangeName:         true,
+	BlobSidecarsByRootName:          true,
+	LightClientBootstrapName:        true,
+	LightClientUpdatesByRangeName:   true,
+	LightClientFinalityUpdateName:   true,
+	LightClientOptimisticUpdateName: true,
 }
 
 // Maps all the RPC messages which are to updated in altair.

beacon-chain/p2p/testing/BUILD.bazel

@@ -20,6 +20,7 @@ go_library(
         "//beacon-chain/p2p/encoder:go_default_library",
         "//beacon-chain/p2p/peers:go_default_library",
         "//beacon-chain/p2p/peers/scorers:go_default_library",
+        "//consensus-types/interfaces:go_default_library",
         "//proto/prysm/v1alpha1:go_default_library",
         "//proto/prysm/v1alpha1/metadata:go_default_library",
         "//testing/require:go_default_library",

beacon-chain/p2p/testing/fuzz_p2p.go

@@ -5,6 +5,7 @@ import (
 
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/encoder"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/peers"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
 	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
 	"github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1/metadata"
 	"github.com/ethereum/go-ethereum/p2p/enr"
@@ -148,6 +149,16 @@ func (*FakeP2P) BroadcastBlob(_ context.Context, _ uint64, _ *ethpb.BlobSidecar)
 	return nil
 }
 
+// BroadcastLightClientOptimisticUpdate -- fake.
+func (*FakeP2P) BroadcastLightClientOptimisticUpdate(_ context.Context, _ interfaces.LightClientOptimisticUpdate) error {
+	return nil
+}
+
+// BroadcastLightClientFinalityUpdate -- fake.
+func (*FakeP2P) BroadcastLightClientFinalityUpdate(_ context.Context, _ interfaces.LightClientFinalityUpdate) error {
+	return nil
+}
+
 // InterceptPeerDial -- fake.
 func (*FakeP2P) InterceptPeerDial(peer.ID) (allow bool) {
 	return true

beacon-chain/p2p/testing/mock_broadcaster.go

@@ -5,6 +5,7 @@ import (
 	"sync"
 	"sync/atomic"
 
+	"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
 	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
 	"google.golang.org/protobuf/proto"
 )
@@ -48,6 +49,18 @@ func (m *MockBroadcaster) BroadcastBlob(context.Context, uint64, *ethpb.BlobSide
 	return nil
 }
 
+// BroadcastLightClientOptimisticUpdate records a broadcast occurred.
+func (m *MockBroadcaster) BroadcastLightClientOptimisticUpdate(_ context.Context, _ interfaces.LightClientOptimisticUpdate) error {
+	m.BroadcastCalled.Store(true)
+	return nil
+}
+
+// BroadcastLightClientFinalityUpdate records a broadcast occurred.
+func (m *MockBroadcaster) BroadcastLightClientFinalityUpdate(_ context.Context, _ interfaces.LightClientFinalityUpdate) error {
+	m.BroadcastCalled.Store(true)
+	return nil
+}
+
 // NumMessages returns the number of messages broadcasted.
 func (m *MockBroadcaster) NumMessages() int {
 	m.msgLock.Lock()

beacon-chain/p2p/testing/p2p.go

@@ -13,6 +13,7 @@ import (
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/encoder"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/peers"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/peers/scorers"
+	"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
 	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
 	"github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1/metadata"
 	"github.com/OffchainLabs/prysm/v6/testing/require"
@@ -207,6 +208,18 @@ func (p *TestP2P) BroadcastBlob(context.Context, uint64, *ethpb.BlobSidecar) err
 	return nil
 }
 
+// BroadcastLightClientOptimisticUpdate broadcasts an optimistic update for mock.
+func (p *TestP2P) BroadcastLightClientOptimisticUpdate(_ context.Context, _ interfaces.LightClientOptimisticUpdate) error {
+	p.BroadcastCalled.Store(true)
+	return nil
+}
+
+// BroadcastLightClientFinalityUpdate broadcasts a finality update for mock.
+func (p *TestP2P) BroadcastLightClientFinalityUpdate(_ context.Context, _ interfaces.LightClientFinalityUpdate) error {
+	p.BroadcastCalled.Store(true)
+	return nil
+}
+
 // SetStreamHandler for RPC.
 func (p *TestP2P) SetStreamHandler(topic string, handler network.StreamHandler) {
 	p.BHost.SetStreamHandler(protocol.ID(topic), handler)

beacon-chain/p2p/types/BUILD.bazel

@@ -21,6 +21,7 @@ go_library(
         "//config/params:go_default_library",
         "//consensus-types/blocks:go_default_library",
         "//consensus-types/interfaces:go_default_library",
+        "//consensus-types/light-client:go_default_library",
         "//consensus-types/primitives:go_default_library",
         "//consensus-types/wrapper:go_default_library",
         "//encoding/bytesutil:go_default_library",

beacon-chain/p2p/types/object_mapping.go

@@ -4,6 +4,7 @@ import (
 	"github.com/OffchainLabs/prysm/v6/config/params"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
+	lightclientConsensusTypes "github.com/OffchainLabs/prysm/v6/consensus-types/light-client"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/wrapper"
 	"github.com/OffchainLabs/prysm/v6/encoding/bytesutil"
 	enginev1 "github.com/OffchainLabs/prysm/v6/proto/engine/v1"
@@ -36,6 +37,12 @@ var (
 	// AttesterSlashingMap maps the fork-version to the underlying data type for that particular
 	// fork period.
 	AttesterSlashingMap map[[4]byte]func() (ethpb.AttSlashing, error)
+	// LightClientOptimisticUpdateMap maps the fork-version to the underlying data type for that
+	// particular fork period.
+	LightClientOptimisticUpdateMap map[[4]byte]func() (interfaces.LightClientOptimisticUpdate, error)
+	// LightClientFinalityUpdateMap maps the fork-version to the underlying data type for that
+	// particular fork period.
+	LightClientFinalityUpdateMap map[[4]byte]func() (interfaces.LightClientFinalityUpdate, error)
 )
 
 // InitializeDataMaps initializes all the relevant object maps. This function is called to
@@ -179,4 +186,42 @@ func InitializeDataMaps() {
 			return &ethpb.AttesterSlashingElectra{}, nil
 		},
 	}
+
+	// Reset our light client optimistic update map.
+	LightClientOptimisticUpdateMap = map[[4]byte]func() (interfaces.LightClientOptimisticUpdate, error){
+		bytesutil.ToBytes4(params.BeaconConfig().AltairForkVersion): func() (interfaces.LightClientOptimisticUpdate, error) {
+			return lightclientConsensusTypes.NewEmptyOptimisticUpdateAltair(), nil
+		},
+		bytesutil.ToBytes4(params.BeaconConfig().BellatrixForkVersion): func() (interfaces.LightClientOptimisticUpdate, error) {
+			return lightclientConsensusTypes.NewEmptyOptimisticUpdateAltair(), nil
+		},
+		bytesutil.ToBytes4(params.BeaconConfig().CapellaForkVersion): func() (interfaces.LightClientOptimisticUpdate, error) {
+			return lightclientConsensusTypes.NewEmptyOptimisticUpdateCapella(), nil
+		},
+		bytesutil.ToBytes4(params.BeaconConfig().DenebForkVersion): func() (interfaces.LightClientOptimisticUpdate, error) {
+			return lightclientConsensusTypes.NewEmptyOptimisticUpdateDeneb(), nil
+		},
+		bytesutil.ToBytes4(params.BeaconConfig().ElectraForkVersion): func() (interfaces.LightClientOptimisticUpdate, error) {
+			return lightclientConsensusTypes.NewEmptyOptimisticUpdateDeneb(), nil
+		},
+	}
+
+	// Reset our light client finality update map.
+	LightClientFinalityUpdateMap = map[[4]byte]func() (interfaces.LightClientFinalityUpdate, error){
+		bytesutil.ToBytes4(params.BeaconConfig().AltairForkVersion): func() (interfaces.LightClientFinalityUpdate, error) {
+			return lightclientConsensusTypes.NewEmptyFinalityUpdateAltair(), nil
+		},
+		bytesutil.ToBytes4(params.BeaconConfig().BellatrixForkVersion): func() (interfaces.LightClientFinalityUpdate, error) {
+			return lightclientConsensusTypes.NewEmptyFinalityUpdateAltair(), nil
+		},
+		bytesutil.ToBytes4(params.BeaconConfig().CapellaForkVersion): func() (interfaces.LightClientFinalityUpdate, error) {
+			return lightclientConsensusTypes.NewEmptyFinalityUpdateCapella(), nil
+		},
+		bytesutil.ToBytes4(params.BeaconConfig().DenebForkVersion): func() (interfaces.LightClientFinalityUpdate, error) {
+			return lightclientConsensusTypes.NewEmptyFinalityUpdateDeneb(), nil
+		},
+		bytesutil.ToBytes4(params.BeaconConfig().ElectraForkVersion): func() (interfaces.LightClientFinalityUpdate, error) {
+			return lightclientConsensusTypes.NewEmptyFinalityUpdateElectra(), nil
+		},
+	}
 }

beacon-chain/p2p/utils.go

@@ -77,8 +77,9 @@ func privKey(cfg *Config) (*ecdsa.PrivateKey, error) {
 		return nil, err
 	}
 
-	// If the StaticPeerID flag is not set and if peerDAS is not enabled, return the private key.
-	if !(cfg.StaticPeerID || params.PeerDASEnabled()) {
+	// If the StaticPeerID flag is not set or the Fulu epoch is not set, return the private key.
+	// Starting at Fulu, we don't want to generate a new key every time, to avoid custody columns changes.
+	if !(cfg.StaticPeerID || params.FuluEnabled()) {
 		return ecdsaprysm.ConvertFromInterfacePrivKey(priv)
 	}
 

beacon-chain/rpc/BUILD.bazel

@@ -66,7 +66,7 @@ go_library(
         "@com_github_prometheus_client_golang//prometheus/promauto:go_default_library",
         "@com_github_prometheus_client_golang//prometheus/promhttp:go_default_library",
         "@com_github_sirupsen_logrus//:go_default_library",
-        "@io_opencensus_go//plugin/ocgrpc:go_default_library",
+        "@io_opentelemetry_go_contrib_instrumentation_google_golang_org_grpc_otelgrpc//:go_default_library",
         "@org_golang_google_grpc//:go_default_library",
         "@org_golang_google_grpc//credentials:go_default_library",
         "@org_golang_google_grpc//peer:go_default_library",
@@ -87,6 +87,7 @@ go_test(
         "//beacon-chain/execution/testing:go_default_library",
         "//beacon-chain/startup:go_default_library",
         "//beacon-chain/sync/initial-sync/testing:go_default_library",
+        "//config/features:go_default_library",
         "//testing/assert:go_default_library",
         "//testing/require:go_default_library",
         "@com_github_sirupsen_logrus//:go_default_library",

beacon-chain/rpc/endpoints.go

@@ -22,6 +22,7 @@ import (
 	validatorv1alpha1 "github.com/OffchainLabs/prysm/v6/beacon-chain/rpc/prysm/v1alpha1/validator"
 	validatorprysm "github.com/OffchainLabs/prysm/v6/beacon-chain/rpc/prysm/validator"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/state/stategen"
+	"github.com/OffchainLabs/prysm/v6/config/features"
 	"github.com/prometheus/client_golang/prometheus"
 	"github.com/prometheus/client_golang/prometheus/promhttp"
 )
@@ -95,14 +96,19 @@ func (s *Service) endpoints(
 	endpoints = append(endpoints, s.nodeEndpoints()...)
 	endpoints = append(endpoints, s.beaconEndpoints(ch, stater, blocker, validatorServer, coreService)...)
 	endpoints = append(endpoints, s.configEndpoints()...)
-	endpoints = append(endpoints, s.lightClientEndpoints(blocker, stater)...)
 	endpoints = append(endpoints, s.eventsEndpoints()...)
 	endpoints = append(endpoints, s.prysmBeaconEndpoints(ch, stater, coreService)...)
 	endpoints = append(endpoints, s.prysmNodeEndpoints()...)
 	endpoints = append(endpoints, s.prysmValidatorEndpoints(stater, coreService)...)
+
+	if features.Get().EnableLightClient {
+		endpoints = append(endpoints, s.lightClientEndpoints(blocker, stater)...)
+	}
+
 	if enableDebug {
 		endpoints = append(endpoints, s.debugEndpoints(stater)...)
 	}
+
 	return endpoints
 }
 
@@ -900,7 +906,7 @@ func (s *Service) beaconEndpoints(
 			middleware: []middleware.Middleware{
 				middleware.AcceptHeaderHandler([]string{api.JsonMediaType}),
 			},
-			handler: server.GetPendingDeposits,
+			handler: server.GetPendingConsolidations,
 			methods: []string{http.MethodGet},
 		},
 		{
@@ -1253,7 +1259,7 @@ func (s *Service) prysmValidatorEndpoints(stater lookup.Stater, coreService *cor
 			methods: []string{http.MethodPost},
 		},
 		{
-			template: "/prysm/v1/validators/participation",
+			template: "/prysm/v1/validators/{state_id}/participation",
 			name:     namespace + ".GetParticipation",
 			middleware: []middleware.Middleware{
 				middleware.AcceptHeaderHandler([]string{api.JsonMediaType}),
@@ -1262,7 +1268,7 @@ func (s *Service) prysmValidatorEndpoints(stater lookup.Stater, coreService *cor
 			methods: []string{http.MethodGet},
 		},
 		{
-			template: "/prysm/v1/validators/active_set_changes",
+			template: "/prysm/v1/validators/{state_id}/active_set_changes",
 			name:     namespace + ".GetActiveSetChanges",
 			middleware: []middleware.Middleware{
 				middleware.AcceptHeaderHandler([]string{api.JsonMediaType}),

beacon-chain/rpc/endpoints_test.go

@@ -6,6 +6,7 @@ import (
 	"slices"
 	"testing"
 
+	"github.com/OffchainLabs/prysm/v6/config/features"
 	"github.com/OffchainLabs/prysm/v6/testing/assert"
 )
 
@@ -133,33 +134,62 @@ func Test_endpoints(t *testing.T) {
 	}
 
 	prysmValidatorRoutes := map[string][]string{
-		"/prysm/validators/performance":           {http.MethodPost},
-		"/prysm/v1/validators/performance":        {http.MethodPost},
-		"/prysm/v1/validators/participation":      {http.MethodGet},
-		"/prysm/v1/validators/active_set_changes": {http.MethodGet},
+		"/prysm/validators/performance":                      {http.MethodPost},
+		"/prysm/v1/validators/performance":                   {http.MethodPost},
+		"/prysm/v1/validators/{state_id}/participation":      {http.MethodGet},
+		"/prysm/v1/validators/{state_id}/active_set_changes": {http.MethodGet},
 	}
 
-	s := &Service{cfg: &Config{}}
-
-	endpoints := s.endpoints(true, nil, nil, nil, nil, nil, nil)
-	actualRoutes := make(map[string][]string, len(endpoints))
-	for _, e := range endpoints {
-		if _, ok := actualRoutes[e.template]; ok {
-			actualRoutes[e.template] = append(actualRoutes[e.template], e.methods...)
-		} else {
-			actualRoutes[e.template] = e.methods
-		}
-	}
-	expectedRoutes := make(map[string][]string)
-	for _, m := range []map[string][]string{
-		beaconRoutes, builderRoutes, configRoutes, debugRoutes, eventsRoutes,
-		nodeRoutes, validatorRoutes, rewardsRoutes, lightClientRoutes, blobRoutes,
-		prysmValidatorRoutes, prysmNodeRoutes, prysmBeaconRoutes,
-	} {
-		maps.Copy(expectedRoutes, m)
+	testCases := []struct {
+		name                     string
+		flag                     *features.Flags
+		additionalExpectedRoutes []map[string][]string
+	}{
+		{
+			name: "no flags",
+		},
+		{
+			name: "light client enabled",
+			flag: &features.Flags{
+				EnableLightClient: true,
+			},
+			additionalExpectedRoutes: []map[string][]string{
+				lightClientRoutes,
+			},
+		},
 	}
 
-	assert.Equal(t, true, maps.EqualFunc(expectedRoutes, actualRoutes, func(actualMethods []string, expectedMethods []string) bool {
-		return slices.Equal(expectedMethods, actualMethods)
-	}))
+	for _, tc := range testCases {
+		t.Run(tc.name, func(t *testing.T) {
+			resetFn := features.InitWithReset(tc.flag)
+			defer resetFn()
+
+			s := &Service{cfg: &Config{}}
+
+			endpoints := s.endpoints(true, nil, nil, nil, nil, nil, nil)
+			actualRoutes := make(map[string][]string, len(endpoints))
+			for _, e := range endpoints {
+				if _, ok := actualRoutes[e.template]; ok {
+					actualRoutes[e.template] = append(actualRoutes[e.template], e.methods...)
+				} else {
+					actualRoutes[e.template] = e.methods
+				}
+			}
+			expectedRoutes := make(map[string][]string)
+			for _, m := range []map[string][]string{
+				beaconRoutes, builderRoutes, configRoutes, debugRoutes, eventsRoutes,
+				nodeRoutes, validatorRoutes, rewardsRoutes, blobRoutes,
+				prysmValidatorRoutes, prysmNodeRoutes, prysmBeaconRoutes,
+			} {
+				maps.Copy(expectedRoutes, m)
+			}
+			for _, m := range tc.additionalExpectedRoutes {
+				maps.Copy(expectedRoutes, m)
+			}
+
+			assert.Equal(t, true, maps.EqualFunc(expectedRoutes, actualRoutes, func(actualMethods []string, expectedMethods []string) bool {
+				return slices.Equal(expectedMethods, actualMethods)
+			}))
+		})
+	}
 }

beacon-chain/rpc/eth/beacon/handlers_pool.go

@@ -751,9 +751,9 @@ func (s *Server) GetAttesterSlashingsV2(w http.ResponseWriter, r *http.Request)
 		httputil.HandleError(w, "Could not get head state: "+err.Error(), http.StatusInternalServerError)
 		return
 	}
-	var attStructs []interface{}
-	sourceSlashings := s.SlashingsPool.PendingAttesterSlashings(ctx, headState, true /* return unlimited slashings */)
 
+	sourceSlashings := s.SlashingsPool.PendingAttesterSlashings(ctx, headState, true /* return unlimited slashings */)
+	attStructs := make([]interface{}, 0, len(sourceSlashings))
 	for _, slashing := range sourceSlashings {
 		var attStruct interface{}
 		if v >= version.Electra && slashing.Version() >= version.Electra {

beacon-chain/rpc/eth/beacon/handlers_pool_test.go

@@ -1858,6 +1858,7 @@ func TestGetAttesterSlashings(t *testing.T) {
 			// Unmarshal resp.Data into a slice of slashings
 			var slashings []*structs.AttesterSlashingElectra
 			require.NoError(t, json.Unmarshal(resp.Data, &slashings))
+			require.NotNil(t, slashings)
 			require.Equal(t, 0, len(slashings))
 		})
 	})

beacon-chain/rpc/eth/blob/handlers_test.go

@@ -265,7 +265,7 @@ func TestBlobs(t *testing.T) {
 		require.Equal(t, false, resp.Finalized)
 	})
 	t.Run("blob index over max", func(t *testing.T) {
-		overLimit := params.BeaconConfig().MaxBlobsPerBlockByVersion(version.Deneb)
+		overLimit := maxBlobsPerBlockByVersion(version.Deneb)
 		u := fmt.Sprintf("http://foo.example/123?indices=%d", overLimit)
 		request := httptest.NewRequest("GET", u, nil)
 		writer := httptest.NewRecorder()
@@ -412,10 +412,14 @@ func TestBlobs_Electra(t *testing.T) {
 	cfg := params.BeaconConfig().Copy()
 	cfg.DenebForkEpoch = 0
 	cfg.ElectraForkEpoch = 1
+	cfg.BlobSchedule = []params.BlobScheduleEntry{
+		{Epoch: 0, MaxBlobsPerBlock: 6},
+		{Epoch: 1, MaxBlobsPerBlock: 9},
+	}
 	params.OverrideBeaconConfig(cfg)
 
 	db := testDB.SetupDB(t)
-	electraBlock, blobs := util.GenerateTestElectraBlockWithSidecar(t, [32]byte{}, 123, params.BeaconConfig().MaxBlobsPerBlockByVersion(version.Electra))
+	electraBlock, blobs := util.GenerateTestElectraBlockWithSidecar(t, [32]byte{}, 123, maxBlobsPerBlockByVersion(version.Electra))
 	require.NoError(t, db.SaveBlock(context.Background(), electraBlock))
 	bs := filesystem.NewEphemeralBlobStorage(t)
 	testSidecars := verification.FakeVerifySliceForTest(t, blobs)
@@ -451,7 +455,7 @@ func TestBlobs_Electra(t *testing.T) {
 		assert.Equal(t, http.StatusOK, writer.Code)
 		resp := &structs.SidecarsResponse{}
 		require.NoError(t, json.Unmarshal(writer.Body.Bytes(), resp))
-		require.Equal(t, params.BeaconConfig().MaxBlobsPerBlockByVersion(version.Electra), len(resp.Data))
+		require.Equal(t, maxBlobsPerBlockByVersion(version.Electra), len(resp.Data))
 		sidecar := resp.Data[0]
 		require.NotNil(t, sidecar)
 		assert.Equal(t, "0", sidecar.Index)
@@ -464,7 +468,7 @@ func TestBlobs_Electra(t *testing.T) {
 		require.Equal(t, false, resp.Finalized)
 	})
 	t.Run("requested blob index at max", func(t *testing.T) {
-		limit := params.BeaconConfig().MaxBlobsPerBlockByVersion(version.Electra) - 1
+		limit := maxBlobsPerBlockByVersion(version.Electra) - 1
 		u := fmt.Sprintf("http://foo.example/123?indices=%d", limit)
 		request := httptest.NewRequest("GET", u, nil)
 		writer := httptest.NewRecorder()
@@ -496,7 +500,7 @@ func TestBlobs_Electra(t *testing.T) {
 		require.Equal(t, false, resp.Finalized)
 	})
 	t.Run("blob index over max", func(t *testing.T) {
-		overLimit := params.BeaconConfig().MaxBlobsPerBlockByVersion(version.Electra)
+		overLimit := maxBlobsPerBlockByVersion(version.Electra)
 		u := fmt.Sprintf("http://foo.example/123?indices=%d", overLimit)
 		request := httptest.NewRequest("GET", u, nil)
 		writer := httptest.NewRecorder()
@@ -554,3 +558,11 @@ func Test_parseIndices(t *testing.T) {
 		})
 	}
 }
+
+func maxBlobsPerBlockByVersion(v int) int {
+	if v >= version.Electra {
+		return params.BeaconConfig().DeprecatedMaxBlobsPerBlockElectra
+	}
+
+	return params.BeaconConfig().DeprecatedMaxBlobsPerBlock
+}

beacon-chain/rpc/eth/config/handlers_test.go

@@ -200,7 +200,7 @@ func TestGetSpec(t *testing.T) {
 	data, ok := resp.Data.(map[string]interface{})
 	require.Equal(t, true, ok)
 
-	assert.Equal(t, 169, len(data))
+	assert.Equal(t, 175, len(data))
 	for k, v := range data {
 		t.Run(k, func(t *testing.T) {
 			switch k {
@@ -545,6 +545,18 @@ func TestGetSpec(t *testing.T) {
 				assert.Equal(t, "9", v)
 			case "MAX_REQUEST_BLOB_SIDECARS_ELECTRA":
 				assert.Equal(t, "1152", v)
+			case "NUMBER_OF_CUSTODY_GROUPS":
+				assert.Equal(t, "128", v)
+			case "BALANCE_PER_ADDITIONAL_CUSTODY_GROUP":
+				assert.Equal(t, "32000000000", v)
+			case "CUSTODY_REQUIREMENT":
+				assert.Equal(t, "4", v)
+			case "SAMPLES_PER_SLOT":
+				assert.Equal(t, "8", v)
+			case "VALIDATOR_CUSTODY_REQUIREMENT":
+				assert.Equal(t, "8", v)
+			case "MIN_EPOCHS_FOR_DATA_COLUMN_SIDECARS_REQUESTS":
+				assert.Equal(t, "4096", v)
 			case "MAX_BLOB_COMMITMENTS_PER_BLOCK":
 				assert.Equal(t, "95", v)
 			case "MAX_BYTES_PER_TRANSACTION":
@@ -559,6 +571,8 @@ func TestGetSpec(t *testing.T) {
 				assert.Equal(t, "100", v)
 			case "KZG_COMMITMENT_INCLUSION_PROOF_DEPTH":
 				assert.Equal(t, "101", v)
+			case "MAX_BLOBS_PER_BLOCK_FULU":
+				assert.Equal(t, "12", v)
 			case "BLOB_SIDECAR_SUBNET_COUNT":
 				assert.Equal(t, "102", v)
 			case "BLOB_SIDECAR_SUBNET_COUNT_ELECTRA":

beacon-chain/rpc/eth/light-client/BUILD.bazel

@@ -17,7 +17,6 @@ go_library(
         "//beacon-chain/db:go_default_library",
         "//beacon-chain/rpc/eth/shared:go_default_library",
         "//beacon-chain/rpc/lookup:go_default_library",
-        "//config/features:go_default_library",
         "//config/params:go_default_library",
         "//encoding/bytesutil:go_default_library",
         "//monitoring/tracing/trace:go_default_library",
@@ -44,7 +43,6 @@ go_test(
         "//beacon-chain/core/light-client:go_default_library",
         "//beacon-chain/db/testing:go_default_library",
         "//beacon-chain/state:go_default_library",
-        "//config/features:go_default_library",
         "//config/fieldparams:go_default_library",
         "//config/params:go_default_library",
         "//consensus-types/interfaces:go_default_library",

beacon-chain/rpc/eth/light-client/handlers.go

@@ -8,7 +8,6 @@ import (
 	"github.com/OffchainLabs/prysm/v6/api/server/structs"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/core/signing"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/rpc/eth/shared"
-	"github.com/OffchainLabs/prysm/v6/config/features"
 	"github.com/OffchainLabs/prysm/v6/config/params"
 	"github.com/OffchainLabs/prysm/v6/encoding/bytesutil"
 	"github.com/OffchainLabs/prysm/v6/monitoring/tracing/trace"
@@ -22,11 +21,6 @@ import (
 
 // GetLightClientBootstrap - implements https://github.com/ethereum/beacon-APIs/blob/263f4ed6c263c967f13279c7a9f5629b51c5fc55/apis/beacon/light_client/bootstrap.yaml
 func (s *Server) GetLightClientBootstrap(w http.ResponseWriter, req *http.Request) {
-	if !features.Get().EnableLightClient {
-		httputil.HandleError(w, "Light client feature flag is not enabled", http.StatusNotFound)
-		return
-	}
-
 	// Prepare
 	ctx, span := trace.StartSpan(req.Context(), "beacon.GetLightClientBootstrap")
 	defer span.End()
@@ -74,11 +68,6 @@ func (s *Server) GetLightClientBootstrap(w http.ResponseWriter, req *http.Reques
 
 // GetLightClientUpdatesByRange - implements https://github.com/ethereum/beacon-APIs/blob/263f4ed6c263c967f13279c7a9f5629b51c5fc55/apis/beacon/light_client/updates.yaml
 func (s *Server) GetLightClientUpdatesByRange(w http.ResponseWriter, req *http.Request) {
-	if !features.Get().EnableLightClient {
-		httputil.HandleError(w, "Light client feature flag is not enabled", http.StatusNotFound)
-		return
-	}
-
 	ctx, span := trace.StartSpan(req.Context(), "beacon.GetLightClientUpdatesByRange")
 	defer span.End()
 
@@ -187,11 +176,6 @@ func (s *Server) GetLightClientUpdatesByRange(w http.ResponseWriter, req *http.R
 
 // GetLightClientFinalityUpdate - implements https://github.com/ethereum/beacon-APIs/blob/263f4ed6c263c967f13279c7a9f5629b51c5fc55/apis/beacon/light_client/finality_update.yaml
 func (s *Server) GetLightClientFinalityUpdate(w http.ResponseWriter, req *http.Request) {
-	if !features.Get().EnableLightClient {
-		httputil.HandleError(w, "Light client feature flag is not enabled", http.StatusNotFound)
-		return
-	}
-
 	_, span := trace.StartSpan(req.Context(), "beacon.GetLightClientFinalityUpdate")
 	defer span.End()
 
@@ -225,11 +209,6 @@ func (s *Server) GetLightClientFinalityUpdate(w http.ResponseWriter, req *http.R
 
 // GetLightClientOptimisticUpdate - implements https://github.com/ethereum/beacon-APIs/blob/263f4ed6c263c967f13279c7a9f5629b51c5fc55/apis/beacon/light_client/optimistic_update.yaml
 func (s *Server) GetLightClientOptimisticUpdate(w http.ResponseWriter, req *http.Request) {
-	if !features.Get().EnableLightClient {
-		httputil.HandleError(w, "Light client feature flag is not enabled", http.StatusNotFound)
-		return
-	}
-
 	_, span := trace.StartSpan(req.Context(), "beacon.GetLightClientOptimisticUpdate")
 	defer span.End()
 

beacon-chain/rpc/eth/light-client/handlers_test.go

@@ -17,7 +17,6 @@ import (
 	lightclient "github.com/OffchainLabs/prysm/v6/beacon-chain/core/light-client"
 	dbtesting "github.com/OffchainLabs/prysm/v6/beacon-chain/db/testing"
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/state"
-	"github.com/OffchainLabs/prysm/v6/config/features"
 	fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
 	"github.com/OffchainLabs/prysm/v6/config/params"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
@@ -34,11 +33,6 @@ import (
 )
 
 func TestLightClientHandler_GetLightClientBootstrap(t *testing.T) {
-	resetFn := features.InitWithReset(&features.Flags{
-		EnableLightClient: true,
-	})
-	defer resetFn()
-
 	params.SetupTestConfigCleanup(t)
 	cfg := params.BeaconConfig()
 	cfg.AltairForkEpoch = 0
@@ -454,11 +448,6 @@ func TestLightClientHandler_GetLightClientBootstrap(t *testing.T) {
 }
 
 func TestLightClientHandler_GetLightClientByRange(t *testing.T) {
-	resetFn := features.InitWithReset(&features.Flags{
-		EnableLightClient: true,
-	})
-	defer resetFn()
-
 	helpers.ClearCache()
 	ctx := context.Background()
 
@@ -1501,10 +1490,6 @@ func TestLightClientHandler_GetLightClientByRange(t *testing.T) {
 }
 
 func TestLightClientHandler_GetLightClientFinalityUpdate(t *testing.T) {
-	resetFn := features.InitWithReset(&features.Flags{
-		EnableLightClient: true,
-	})
-	defer resetFn()
 	helpers.ClearCache()
 
 	t.Run("no update", func(t *testing.T) {
@@ -1589,10 +1574,6 @@ func TestLightClientHandler_GetLightClientFinalityUpdate(t *testing.T) {
 }
 
 func TestLightClientHandler_GetLightClientOptimisticUpdate(t *testing.T) {
-	resetFn := features.InitWithReset(&features.Flags{
-		EnableLightClient: true,
-	})
-	defer resetFn()
 	helpers.ClearCache()
 
 	t.Run("no update", func(t *testing.T) {

beacon-chain/rpc/eth/shared/testing/json.go

@@ -678,187 +678,6 @@ const (
   "signature": "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"
 }`
 
-	// BadBlindedBellatrixBlock contains wrong data to create a block that does not pass ToConsensus conversion
-	// "parent_root" length too short
-	// "block_hash" length too short
-	// "state_root" length too short
-	BadBlindedBellatrixBlock = `{
-  "message": {
-    "slot": "1",
-    "proposer_index": "1",
-    "parent_root": "0xcf8e0d4e95872", 
-    "state_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-    "body": {
-      "randao_reveal": "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505",
-      "eth1_data": {
-        "deposit_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-        "deposit_count": "1",
-        "block_hash": "0xcf8e0d4e95872" 
-      },
-      "graffiti": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-      "proposer_slashings": [
-        {
-          "signed_header_1": {
-            "message": {
-              "slot": "1",
-              "proposer_index": "1",
-              "parent_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-              "state_root": "0xcf8e0d4e9580f2", 
-              "body_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"
-            },
-            "signature": "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"
-          },
-          "signed_header_2": {
-            "message": {
-              "slot": "1",
-              "proposer_index": "1",
-              "parent_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-              "state_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-              "body_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"
-            },
-            "signature": "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"
-          }
-        }
-      ],
-      "attester_slashings": [
-        {
-          "attestation_1": {
-            "attesting_indices": [
-              "1"
-            ],
-            "signature": "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505",
-            "data": {
-              "slot": "1",
-              "index": "1",
-              "beacon_block_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-              "source": {
-                "epoch": "1",
-                "root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"
-              },
-              "target": {
-                "epoch": "1",
-                "root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"
-              }
-            }
-          },
-          "attestation_2": {
-            "attesting_indices": [
-              "1"
-            ],
-            "signature": "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505",
-            "data": {
-              "slot": "1",
-              "index": "1",
-              "beacon_block_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-              "source": {
-                "epoch": "1",
-                "root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"
-              },
-              "target": {
-                "epoch": "1",
-                "root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"
-              }
-            }
-          }
-        }
-      ],
-      "attestations": [
-        {
-          "aggregation_bits": "0xffffffffffffffffffffffffffffffffff3f",
-          "signature": "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505",
-          "data": {
-            "slot": "1",
-            "index": "1",
-            "beacon_block_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "source": {
-              "epoch": "1",
-              "root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"
-            },
-            "target": {
-              "epoch": "1",
-              "root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"
-            }
-          }
-        }
-      ],
-      "deposits": [
-        {
-          "proof": [
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-			"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"
-          ],
-          "data": {
-            "pubkey": "0x93247f2209abcacf57b75a51dafae777f9dd38bc7053d1af526f220a7489a6d3a2753e5f3e8b1cfe39b56f43611df74a",
-            "withdrawal_credentials": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "amount": "1",
-            "signature": "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"
-          }
-        }
-      ],
-      "voluntary_exits": [
-        {
-          "message": {
-            "epoch": "1",
-            "validator_index": "1"
-          },
-          "signature": "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"
-        }
-      ],
-      "sync_aggregate": {
-        "sync_committee_bits": "0x6451e9f951ebf05edc01de67e593484b672877054f055903ff0df1a1a945cf30ca26bb4d4b154f94a1bc776bcf5d0efb3603e1f9b8ee2499ccdcfe2a18cef458",
-        "sync_committee_signature": "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"
-      },
-      "execution_payload_header": {
-        "parent_hash": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-        "fee_recipient": "0xabcf8e0d4e9587369b2301d0790347320302cc09",
-        "state_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-        "receipts_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-        "logs_bloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
-        "prev_randao": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-        "block_number": "1",
-        "gas_limit": "1",
-        "gas_used": "1",
-        "timestamp": "1",
-        "extra_data": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-        "base_fee_per_gas": "1",
-        "block_hash": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-        "transactions_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"
-      }
-    }
-  },
-  "signature": "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"
-}`
 	CapellaBlock = `{
   "message": {
     "slot": "1",
@@ -1056,208 +875,7 @@ const (
   },
   "signature": "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"
 }`
-	// BadCapellaBlock contains wrong data to create a block that does not pass ToConsensus conversion
-	// "state_root" length too short
-	// "block_hash" length too short
-	// "graffiti" length too short
-	// "state_root" length too short
-	BadCapellaBlock = `{
-  "message": {
-    "slot": "1",
-    "proposer_index": "1",
-    "parent_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-    "state_root": "0xcf8e0d4e957e8208d920f2",
-    "body": {
-      "randao_reveal": "0x1b66ac1fb663c9baf888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505",
-      "eth1_data": {
-        "deposit_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-        "deposit_count": "1",
-        "block_hash": "0xcf8e0d4e95873691884560367e8208d920f2"
-      },
-      "graffiti": "0xcf8e0d4e9587369b230120f2",
-      "proposer_slashings": [
-        {
-          "signed_header_1": {
-            "message": {
-              "slot": "1",
-              "proposer_index": "1",
-              "parent_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-              "state_root": "0xcf8e0d4e9587369b2301d208d920f2",
-              "body_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"
-            },
-            "signature": "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"
-          },
-          "signed_header_2": {
-            "message": {
-              "slot": "1",
-              "proposer_index": "1",
-              "parent_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-              "state_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-              "body_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"
-            },
-            "signature": "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"
-          }
-        }
-      ],
-      "attester_slashings": [
-        {
-          "attestation_1": {
-            "attesting_indices": [
-              "1"
-            ],
-            "signature": "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505",
-            "data": {
-              "slot": "1",
-              "index": "1",
-              "beacon_block_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-              "source": {
-                "epoch": "1",
-                "root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"
-              },
-              "target": {
-                "epoch": "1",
-                "root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"
-              }
-            }
-          },
-          "attestation_2": {
-            "attesting_indices": [
-              "1"
-            ],
-            "signature": "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505",
-            "data": {
-              "slot": "1",
-              "index": "1",
-              "beacon_block_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-              "source": {
-                "epoch": "1",
-                "root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"
-              },
-              "target": {
-                "epoch": "1",
-                "root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"
-              }
-            }
-          }
-        }
-      ],
-      "attestations": [
-        {
-          "aggregation_bits": "0xffffffffffffffffffffffffffffffffff3f",
-          "signature": "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505",
-          "data": {
-            "slot": "1",
-            "index": "1",
-            "beacon_block_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "source": {
-              "epoch": "1",
-              "root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"
-            },
-            "target": {
-              "epoch": "1",
-              "root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"
-            }
-          }
-        }
-      ],
-      "deposits": [
-        {
-          "proof": [
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-			"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"
-          ],
-          "data": {
-            "pubkey": "0x93247f2209abcacf57b75a51dafae777f9dd38bc7053d1af526f220a7489a6d3a2753e5f3e8b1cfe39b56f43611df74a",
-            "withdrawal_credentials": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-            "amount": "1",
-            "signature": "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"
-          }
-        }
-      ],
-      "voluntary_exits": [
-        {
-          "message": {
-            "epoch": "1",
-            "validator_index": "1"
-          },
-          "signature": "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"
-        }
-      ],
-      "sync_aggregate": {
-        "sync_committee_bits": "0x6451e9f951ebf05edc01de67e593484b672877054f055903ff0df1a1a945cf30ca26bb4d4b154f94a1bc776bcf5d0efb3603e1f9b8ee2499ccdcfe2a18cef458",
-        "sync_committee_signature": "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"
-      },
-      "execution_payload": {
-        "parent_hash": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-        "fee_recipient": "0xabcf8e0d4e9587369b2301d0790347320302cc09",
-        "state_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-        "receipts_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-        "logs_bloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
-        "prev_randao": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-        "block_number": "1",
-        "gas_limit": "1",
-        "gas_used": "1",
-        "timestamp": "1",
-        "extra_data": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-        "base_fee_per_gas": "14074904626401341155369551180448584754667373453244490859944217516317499064576",
-        "block_hash": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
-        "transactions": [
-          "0x02f878831469668303f51d843b9ac9f9843b9aca0082520894c93269b73096998db66be0441e836d873535cb9c8894a19041886f000080c001a031cc29234036afbf9a1fb9476b463367cb1f957ac0b919b69bbc798436e604aaa018c4e9c3914eb27aadd0b91e10b18655739fcf8c1fc398763a9f1beecb8ddc86"
-        ],
-        "withdrawals": [
-          {
-            "index": "1",
-            "validator_index": "1",
-            "address": "0xabcf8e0d4e9587369b2301d0790347320302cc09",
-            "amount": "1"
-          }
-        ]
-      },
-      "bls_to_execution_changes": [
-        {
-          "message": {
-            "validator_index": "1",
-            "from_bls_pubkey": "0x93247f2209abcacf57b75a51dafae777f9dd38bc7053d1af526f220a7489a6d3a2753e5f3e8b1cfe39b56f43611df74a",
-            "to_execution_address": "0xabcf8e0d4e9587369b2301d0790347320302cc09"
-          },
-          "signature": "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"
-        }
-      ]
-    }
-  },
-  "signature": "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"
-}`
+
 	BlindedCapellaBlock = `{
   "message": {
     "slot": "1",

beacon-chain/rpc/prysm/v1alpha1/validator/BUILD.bazel

@@ -103,7 +103,6 @@ go_library(
         "@com_github_prysmaticlabs_fastssz//:go_default_library",
         "@com_github_prysmaticlabs_go_bitfield//:go_default_library",
         "@com_github_sirupsen_logrus//:go_default_library",
-        "@io_opencensus_go//trace:go_default_library",
         "@org_golang_google_grpc//codes:go_default_library",
         "@org_golang_google_grpc//status:go_default_library",
         "@org_golang_google_protobuf//proto:go_default_library",

beacon-chain/rpc/prysm/v1alpha1/validator/duties.go

@@ -9,6 +9,7 @@ import (
 	"github.com/OffchainLabs/prysm/v6/beacon-chain/rpc/core"
 	"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
 	"github.com/OffchainLabs/prysm/v6/encoding/bytesutil"
+	"github.com/OffchainLabs/prysm/v6/monitoring/tracing/trace"
 	ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
 	"github.com/OffchainLabs/prysm/v6/time/slots"
 	"google.golang.org/grpc/codes"
@@ -80,9 +81,11 @@ func (vs *Server) duties(ctx context.Context, req *ethpb.DutiesRequest) (*ethpb.
 		return nil, status.Errorf(codes.Internal, "Could not compute proposer slots: %v", err)
 	}
 
+	ctx, span := trace.StartSpan(ctx, "getDuties.BuildResponse")
+	defer span.End()
+
 	validatorAssignments := make([]*ethpb.DutiesResponse_Duty, 0, len(req.PublicKeys))
 	nextValidatorAssignments := make([]*ethpb.DutiesResponse_Duty, 0, len(req.PublicKeys))
-
 	for _, pubKey := range req.PublicKeys {
 		if ctx.Err() != nil {
 			return nil, status.Errorf(codes.Aborted, "Could not continue fetching assignments: %v", ctx.Err())

beacon-chain/rpc/service.go

@@ -47,7 +47,7 @@ import (
 	grpcprometheus "github.com/grpc-ecosystem/go-grpc-prometheus"
 	"github.com/pkg/errors"
 	"github.com/sirupsen/logrus"
-	"go.opencensus.io/plugin/ocgrpc"
+	"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"
 	"google.golang.org/grpc"
 	"google.golang.org/grpc/credentials"
 	"google.golang.org/grpc/peer"
@@ -146,7 +146,7 @@ func NewService(ctx context.Context, cfg *Config) *Service {
 	log.WithField("address", address).Info("gRPC server listening on port")
 
 	opts := []grpc.ServerOption{
-		grpc.StatsHandler(&ocgrpc.ServerHandler{}),
+		grpc.StatsHandler(otelgrpc.NewServerHandler()),
 		grpc.StreamInterceptor(middleware.ChainStreamServer(
 			recovery.StreamServerInterceptor(
 				recovery.WithRecoveryHandlerContext(tracing.RecoveryHandlerFunc),

beacon-chain/state/interfaces.go

@@ -68,7 +68,7 @@ type ReadOnlyBeaconState interface {
 	Slot() primitives.Slot
 	Fork() *ethpb.Fork
 	LatestBlockHeader() *ethpb.BeaconBlockHeader
-	HistoricalRoots() ([][]byte, error)
+	HistoricalRoots() [][]byte
 	HistoricalSummaries() ([]*ethpb.HistoricalSummary, error)
 	Slashings() []uint64
 	FieldReferencesCount() map[string]uint64

beacon-chain/state/state-native/getters_misc.go

@@ -73,15 +73,15 @@ func (b *BeaconState) forkVal() *ethpb.Fork {
 }
 
 // HistoricalRoots based on epochs stored in the beacon state.
-func (b *BeaconState) HistoricalRoots() ([][]byte, error) {
+func (b *BeaconState) HistoricalRoots() [][]byte {
 	if b.historicalRoots == nil {
-		return nil, nil
+		return nil
 	}
 
 	b.lock.RLock()
 	defer b.lock.RUnlock()
 
-	return b.historicalRoots.Slice(), nil
+	return b.historicalRoots.Slice()
 }
 
 // HistoricalSummaries of the beacon state.

beacon-chain/state/state-native/setters_misc.go

@@ -39,7 +39,7 @@ const (
 	// This specifies the limit till which we process all dirty indices for a certain field.
 	// If we have more dirty indices than the threshold, then we rebuild the whole trie. This
 	// comes due to the fact that O(alogn) > O(n) beyond a certain value of a.
-	indicesLimit = 8000
+	indicesLimit = 20000
 )
 
 // SetGenesisTime for the beacon state.

beacon-chain/state/state-native/state_test.go

@@ -400,11 +400,11 @@ func TestDuplicateDirtyIndices(t *testing.T) {
 	newState.dirtyIndices[types.Balances] = append(newState.dirtyIndices[types.Balances], []uint64{0, 1, 2, 3, 4}...)
 
 	// We would remove the duplicates and stay under the threshold
-	newState.addDirtyIndices(types.Balances, []uint64{9997, 9998})
+	newState.addDirtyIndices(types.Balances, []uint64{20997, 20998})
 	assert.Equal(t, false, newState.rebuildTrie[types.Balances])
 
 	// We would trigger above the threshold.
-	newState.addDirtyIndices(types.Balances, []uint64{10000, 10001, 10002, 10003})
+	newState.addDirtyIndices(types.Balances, []uint64{21000, 21001, 21002, 21003})
 	assert.Equal(t, true, newState.rebuildTrie[types.Balances])
 }
 

beacon-chain/sync/BUILD.bazel

@@ -26,6 +26,7 @@ go_library(
         "rpc_blob_sidecars_by_root.go",
         "rpc_chunked_response.go",
         "rpc_goodbye.go",
+        "rpc_light_client.go",
         "rpc_metadata.go",
         "rpc_ping.go",
         "rpc_send_request.go",
@@ -38,6 +39,7 @@ go_library(
         "subscriber_blob_sidecar.go",
         "subscriber_bls_to_execution_change.go",
         "subscriber_handlers.go",
+        "subscriber_light_client.go",
         "subscriber_sync_committee_message.go",
         "subscriber_sync_contribution_proof.go",
         "subscription_topic_handler.go",
@@ -47,6 +49,7 @@ go_library(
         "validate_beacon_blocks.go",
         "validate_blob.go",
         "validate_bls_to_execution_change.go",
+        "validate_light_client.go",
         "validate_proposer_slashing.go",
         "validate_sync_committee_message.go",
         "validate_sync_contribution_proof.go",
@@ -71,6 +74,7 @@ go_library(
         "//beacon-chain/core/feed/operation:go_default_library",
         "//beacon-chain/core/feed/state:go_default_library",
         "//beacon-chain/core/helpers:go_default_library",
+        "//beacon-chain/core/light-client:go_default_library",
         "//beacon-chain/core/signing:go_default_library",
         "//beacon-chain/core/transition:go_default_library",
         "//beacon-chain/core/transition/interop:go_default_library",
@@ -101,6 +105,7 @@ go_library(
         "//config/params:go_default_library",
         "//consensus-types/blocks:go_default_library",
         "//consensus-types/interfaces:go_default_library",
+        "//consensus-types/light-client:go_default_library",
         "//consensus-types/primitives:go_default_library",
         "//consensus-types/wrapper:go_default_library",
         "//container/leaky-bucket:go_default_library",
@@ -110,6 +115,7 @@ go_library(
         "//encoding/bytesutil:go_default_library",
         "//encoding/ssz/equality:go_default_library",
         "//io/file:go_default_library",
+        "//math:go_default_library",
         "//monitoring/tracing:go_default_library",
         "//monitoring/tracing/trace:go_default_library",
         "//network/forks:go_default_library",
@@ -164,6 +170,7 @@ go_test(
         "rpc_blob_sidecars_by_root_test.go",
         "rpc_goodbye_test.go",
         "rpc_handler_test.go",
+        "rpc_light_client_test.go",
         "rpc_metadata_test.go",
         "rpc_ping_test.go",
         "rpc_send_request_test.go",
@@ -182,6 +189,7 @@ go_test(
         "validate_beacon_blocks_test.go",
         "validate_blob_test.go",
         "validate_bls_to_execution_change_test.go",
+        "validate_light_client_test.go",
         "validate_proposer_slashing_test.go",
         "validate_sync_committee_message_test.go",
         "validate_sync_contribution_proof_test.go",
@@ -198,6 +206,7 @@ go_test(
         "//beacon-chain/core/feed:go_default_library",
         "//beacon-chain/core/feed/operation:go_default_library",
         "//beacon-chain/core/helpers:go_default_library",
+        "//beacon-chain/core/light-client:go_default_library",
         "//beacon-chain/core/signing:go_default_library",
         "//beacon-chain/core/time:go_default_library",
         "//beacon-chain/core/transition:go_default_library",
@@ -225,6 +234,7 @@ go_test(
         "//beacon-chain/verification:go_default_library",
         "//cache/lru:go_default_library",
         "//cmd/beacon-chain/flags:go_default_library",
+        "//config/features:go_default_library",
         "//config/fieldparams:go_default_library",
         "//config/params:go_default_library",
         "//consensus-types/blocks:go_default_library",