Compare commits

...

9 Commits

Author SHA1 Message Date
james-prysm
253f91930a changelog v6.1.3 (#15901)
* updating changelog

* adding changelog

* kasey's comment
2025-10-21 16:46:44 +00:00
terence
7c3e45637f Fix proposer to use advanced state for sync committee position calculation (#15905)
* Sync committee use correct state to calculate position

* Unit test
2025-10-21 15:29:46 +00:00
Manu NALEPA
96429c5089 updateCustodyInfoInDB: Use NumberOfCustodyGroups instead of NumberOfColumns. (#15908)
* `updateCustodyInfoInDB`: Add tests.

* `updateCustodyInfoInDB`: Use `NumberOfCustodyGroups` instead of `NumberOfColumns`.

* Add changelog.

* Fix Potuz's comment.
2025-10-21 14:37:04 +00:00
satushh
d613f3a262 Update Earliest available slot when pruning (#15694)
* Update Earliest available slot when pruning

* bazel run //:gazelle -- fix

* custodyUpdater interface to avoid import cycle

* bazel run //:gazelle -- fix

* simplify test

* separation of concerns

* debug log for updating eas

* UpdateEarliestAvailableSlot function in CustodyManager

* fix test

* UpdateEarliestAvailableSlot function for FakeP2P

* lint

* UpdateEarliestAvailableSlot instead of UpdateCustodyInfo + check for Fulu

* fix test and lint

* bugfix: enforce minimum retention period in pruner

* remove MinEpochsForBlockRequests function and use from config

* remove modifying earliest_available_slot after data column pruning

* correct earliestAvailableSlot validation: allow backfill decrease but prevent increase within MIN_EPOCHS_FOR_BLOCK_REQUESTS

* lint

* bazel run //:gazelle -- fix

* lint and remove unwanted debug logs

* Return a wrapped error, and let the caller decide what to do

* fix tests because updateEarliestSlot returns error now

* avoid re-doing computation in the test function

* lint and correct changelog

* custody updater should be a mandatory part of the pruner service

* ensure never increase eas if we are in the block requests window

* slot level granularity edge case

* update the value stored in the DB

* log tidy up

* use errNoCustodyInfo

* allow earliestAvailableSlot edit when custodyGroupCount doesnt change

* undo the minimal config change

* add context to CustodyGroupCount after merging from develop

* cosmetic change

* shift responsibility from caller to callee, protection for updateEarliestSlot. UpdateEarliestAvailableSlot returns cgc

* allow increase in earliestAvailableSlot only when custodyGroupCount also increases

* remove CustodyGroupCount as it is no longer needed as UpdateEarliestAvailableSlot returns cgc now

* proper place for log and name refactor

* test for Nil custody info

* allow decreasing earliest slot in DB (just like in memory)

* invert if statement to make more readable

* UpdateEarliestAvailableSlot for DB (equivalent of p2p's UpdateEarliestAvailableSlot) & undo changes made to UpdateCustodyInfo

* in UpdateEarliestAvailableSlot, no need to return unused values

* no need to log stored group count

* log.WithField instead of log.WithFields
2025-10-21 13:54:52 +00:00
MozirDmitriy
5751dbf134 kv: write recovered state summaries to stateSummaryBucket (#15896)
* kv: write recovered state summaries to stateSummaryBucket

* Create MozirDmitriy_fix_kv-recover-state-summurt-bucket.md

* add a test
2025-10-21 11:21:10 +00:00
Potuz
426fbcc3b0 Add state diff serialization (#15250)
* Add serialization code for state diffs

Adds serialization code for state diffs.
Adds code to create and apply state diffs
Adds fuzz tests and benchmarks for serialization/deserialization

Co-authored-by: Claude <noreply@anthropic.com>

* Add Fulu support

* Review #1

* gazelle

* Fix some fuzzers

* Failing cases from the fuzzers in consensus-types/hdiff

* Fix more fuzz tests

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* add comparison tests

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Use ConvertToElectra in UpgradeToElectra

* Add comments on constants

* Fix readEth1Data

* remove colons from error messages

* Add design doc

* Apply suggestions from code review

Bast

Co-authored-by: Bastin <43618253+Inspector-Butters@users.noreply.github.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Preston Van Loon <preston@pvl.dev>
Co-authored-by: Bastin <43618253+Inspector-Butters@users.noreply.github.com>
2025-10-20 21:52:32 +00:00
Manu NALEPA
a3baf98b05 VerifyDataColumnsSidecarKZGProofs: Check if sizes match. (#15892) 2025-10-20 17:06:13 +00:00
Jun Song
5a897dfa6b SSZ-QL: Add endpoints (BeaconState/BeaconBlock) (#15888)
* Move ssz_query objects into testing folder (ensuring test objects only used in test environment)

* Add containers for response

* Export sszInfo

* Add QueryBeaconState/Block

* Add comments and few refactor

* Fix merge conflict issues

* Return 500 when calculate offset fails

* Add test for QueryBeaconState

* Add test for QueryBeaconBlock

* Changelog :)

* Rename `QuerySSZRequest` to `SSZQueryRequest`

* Fix middleware hooks for RPC to accept JSON from client and return SSZ

* Convert to `SSZObject` directly from proto

* Move marshalling/calculating hash tree root part after `CalculateOffsetAndLength`

* Make nogo happy

* Add informing comment for using proto unsafe conversion

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2025-10-20 16:24:06 +00:00
Muzry
90190883bc Fixed metadata extraction on Windows by correctly splitting file paths (#15899)
* Fixed metadata extraction on Windows by correctly splitting file paths

* `TestExtractFileMetadata`: Refactor a bit.

---------

Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
2025-10-20 14:17:32 +00:00
117 changed files with 9090 additions and 711 deletions

View File

@@ -4,6 +4,44 @@ All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
## [v6.1.3](https://github.com/prysmaticlabs/prysm/compare/v6.1.2...v6.1.3) - 2025-10-20
This release has several important beacon API and p2p fixes.
### Added
- Add Grandine to P2P known agents. (Useful for metrics). [[PR]](https://github.com/prysmaticlabs/prysm/pull/15829)
- Delegate sszInfo HashTreeRoot to FastSSZ-generated implementations via SSZObject, enabling roots calculation for generated types while avoiding duplicate logic. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15805)
- SSZ-QL: Use `fastssz`'s `SizeSSZ` method for calculating the size of `Container` type. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15864)
- SSZ-QL: Access n-th element in `List`/`Vector`. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15767)
### Changed
- Do not verify block data when calculating rewards. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15819)
- Process pending attestations after pending blocks are cleared. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15824)
- updated web3signer to 25.9.1. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15832)
- Gracefully handle submit blind block returning 502 errors. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15848)
- Improve returning individual message errors from Beacon API. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15835)
- SSZ-QL: Clarify `Size` method with more sophisticated `SSZType`s. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15864)
### Fixed
- Use service context and continue on slasher attestation errors (#15803). [[PR]](https://github.com/prysmaticlabs/prysm/pull/15803)
- block event probably shouldn't be sent on certain block processing failures, now sends only on successing processing Block is NON-CANONICAL, Block IS CANONICAL but getFCUArgs FAILS, and Full success. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15814)
- Fixed web3signer e2e, issues caused due to a regression on old fork support. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15832)
- Do not mark blocks as invalid from ErrNotDescendantOfFinalized. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15846)
- Fixed [#15812](https://github.com/OffchainLabs/prysm/issues/15812): Gossip attestation validation incorrectly rejecting attestations that arrive before their referenced blocks. Previously, attestations were saved to the pending queue but immediately rejected by forkchoice validation, causing "not descendant of finalized checkpoint" errors. Now attestations for missing blocks return `ValidationIgnore` without error, allowing them to be properly processed when their blocks arrive. This eliminates false positive rejections and prevents potential incorrect peer downscoring during network congestion. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15840)
- Mark the block as invalid if it has an invalid signature. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15847)
- Display error messages from the server verbatim when they are not encoded as `application/json`. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15860)
- `HasAtLeastOneIndex`: Check the index is not too high. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15865)
- Fix `/eth/v1/beacon/blob_sidecars/` beacon API is the fulu fork epoch is set to the far future epoch. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15867)
- `dataColumnSidecarsByRangeRPCHandler`: Gracefully close the stream if no data to return. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15866)
- `VerifyDataColumnSidecar`: Check if there is no too many commitments. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15859)
- `WithDataColumnRetentionEpochs`: Use `dataColumnRetentionEpoch` instead of `blobColumnRetentionEpoch`. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15872)
- Mark epoch transition correctly on new head events. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15871)
- reject committee index >= committees_per_slot in unaggregated attestation validation. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15855)
- Decreased attestation gossip validation batch deadline to 5ms. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15882)
## [v6.1.2](https://github.com/prysmaticlabs/prysm/compare/v6.1.1...v6.1.2) - 2025-10-10
This release has several important fixes to improve Prysm's peering, stability, and attestation inclusion on mainnet and all testnets. All node operators are encouraged to update to this release as soon as practical for the best mainnet performance.
@@ -3759,4 +3797,4 @@ There are no security updates in this release.
# Older than v2.0.0
For changelog history for releases older than v2.0.0, please refer to https://github.com/prysmaticlabs/prysm/releases
For changelog history for releases older than v2.0.0, please refer to https://github.com/prysmaticlabs/prysm/releases

View File

@@ -296,3 +296,8 @@ type GetBlobsResponse struct {
Finalized bool `json:"finalized"`
Data []string `json:"data"` //blobs
}
type SSZQueryRequest struct {
Query string `json:"query"`
IncludeProof bool `json:"include_proof,omitempty"`
}

View File

@@ -173,6 +173,7 @@ go_test(
"//beacon-chain/state/state-native:go_default_library",
"//beacon-chain/state/stategen:go_default_library",
"//beacon-chain/verification:go_default_library",
"//cmd/beacon-chain/flags:go_default_library",
"//config/features:go_default_library",
"//config/fieldparams:go_default_library",
"//config/params:go_default_library",

View File

@@ -493,7 +493,7 @@ func (s *Service) updateCustodyInfoInDB(slot primitives.Slot) (primitives.Slot,
// Compute the custody group count.
custodyGroupCount := custodyRequirement
if isSubscribedToAllDataSubnets {
custodyGroupCount = beaconConfig.NumberOfColumns
custodyGroupCount = beaconConfig.NumberOfCustodyGroups
}
// Safely compute the fulu fork slot.

View File

@@ -23,9 +23,11 @@ import (
"github.com/OffchainLabs/prysm/v6/beacon-chain/startup"
state_native "github.com/OffchainLabs/prysm/v6/beacon-chain/state/state-native"
"github.com/OffchainLabs/prysm/v6/beacon-chain/state/stategen"
"github.com/OffchainLabs/prysm/v6/cmd/beacon-chain/flags"
"github.com/OffchainLabs/prysm/v6/config/features"
fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
"github.com/OffchainLabs/prysm/v6/config/params"
"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
consensusblocks "github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
@@ -596,3 +598,103 @@ func TestNotifyIndex(t *testing.T) {
t.Errorf("Notifier channel did not receive the index")
}
}
func TestUpdateCustodyInfoInDB(t *testing.T) {
const (
fuluForkEpoch = 10
custodyRequirement = uint64(4)
earliestStoredSlot = primitives.Slot(12)
numberOfCustodyGroups = uint64(64)
numberOfColumns = uint64(128)
)
params.SetupTestConfigCleanup(t)
cfg := params.BeaconConfig()
cfg.FuluForkEpoch = fuluForkEpoch
cfg.CustodyRequirement = custodyRequirement
cfg.NumberOfCustodyGroups = numberOfCustodyGroups
cfg.NumberOfColumns = numberOfColumns
params.OverrideBeaconConfig(cfg)
ctx := t.Context()
pbBlock := util.NewBeaconBlock()
pbBlock.Block.Slot = 12
signedBeaconBlock, err := blocks.NewSignedBeaconBlock(pbBlock)
require.NoError(t, err)
roBlock, err := blocks.NewROBlock(signedBeaconBlock)
require.NoError(t, err)
t.Run("CGC increases before fulu", func(t *testing.T) {
service, requirements := minimalTestService(t)
err = requirements.db.SaveBlock(ctx, roBlock)
require.NoError(t, err)
// Before Fulu
// -----------
actualEas, actualCgc, err := service.updateCustodyInfoInDB(15)
require.NoError(t, err)
require.Equal(t, earliestStoredSlot, actualEas)
require.Equal(t, custodyRequirement, actualCgc)
actualEas, actualCgc, err = service.updateCustodyInfoInDB(17)
require.NoError(t, err)
require.Equal(t, earliestStoredSlot, actualEas)
require.Equal(t, custodyRequirement, actualCgc)
resetFlags := flags.Get()
gFlags := new(flags.GlobalFlags)
gFlags.SubscribeAllDataSubnets = true
flags.Init(gFlags)
defer flags.Init(resetFlags)
actualEas, actualCgc, err = service.updateCustodyInfoInDB(19)
require.NoError(t, err)
require.Equal(t, earliestStoredSlot, actualEas)
require.Equal(t, numberOfCustodyGroups, actualCgc)
// After Fulu
// ----------
actualEas, actualCgc, err = service.updateCustodyInfoInDB(fuluForkEpoch*primitives.Slot(cfg.SlotsPerEpoch) + 1)
require.NoError(t, err)
require.Equal(t, earliestStoredSlot, actualEas)
require.Equal(t, numberOfCustodyGroups, actualCgc)
})
t.Run("CGC increases after fulu", func(t *testing.T) {
service, requirements := minimalTestService(t)
err = requirements.db.SaveBlock(ctx, roBlock)
require.NoError(t, err)
// Before Fulu
// -----------
actualEas, actualCgc, err := service.updateCustodyInfoInDB(15)
require.NoError(t, err)
require.Equal(t, earliestStoredSlot, actualEas)
require.Equal(t, custodyRequirement, actualCgc)
actualEas, actualCgc, err = service.updateCustodyInfoInDB(17)
require.NoError(t, err)
require.Equal(t, earliestStoredSlot, actualEas)
require.Equal(t, custodyRequirement, actualCgc)
// After Fulu
// ----------
resetFlags := flags.Get()
gFlags := new(flags.GlobalFlags)
gFlags.SubscribeAllDataSubnets = true
flags.Init(gFlags)
defer flags.Init(resetFlags)
slot := fuluForkEpoch*primitives.Slot(cfg.SlotsPerEpoch) + 1
actualEas, actualCgc, err = service.updateCustodyInfoInDB(slot)
require.NoError(t, err)
require.Equal(t, slot, actualEas)
require.Equal(t, numberOfCustodyGroups, actualCgc)
actualEas, actualCgc, err = service.updateCustodyInfoInDB(slot + 2)
require.NoError(t, err)
require.Equal(t, slot, actualEas)
require.Equal(t, numberOfCustodyGroups, actualCgc)
})
}

View File

@@ -130,6 +130,14 @@ func (dch *mockCustodyManager) UpdateCustodyInfo(earliestAvailableSlot primitive
return earliestAvailableSlot, custodyGroupCount, nil
}
func (dch *mockCustodyManager) UpdateEarliestAvailableSlot(earliestAvailableSlot primitives.Slot) error {
dch.mut.Lock()
defer dch.mut.Unlock()
dch.earliestAvailableSlot = earliestAvailableSlot
return nil
}
func (dch *mockCustodyManager) CustodyGroupCountFromPeer(peer.ID) uint64 {
return 0
}

View File

@@ -12,6 +12,46 @@ import (
"github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1/attestation"
)
// ConvertToAltair converts a Phase 0 beacon state to an Altair beacon state.
func ConvertToAltair(state state.BeaconState) (state.BeaconState, error) {
epoch := time.CurrentEpoch(state)
numValidators := state.NumValidators()
s := &ethpb.BeaconStateAltair{
GenesisTime: uint64(state.GenesisTime().Unix()),
GenesisValidatorsRoot: state.GenesisValidatorsRoot(),
Slot: state.Slot(),
Fork: &ethpb.Fork{
PreviousVersion: state.Fork().CurrentVersion,
CurrentVersion: params.BeaconConfig().AltairForkVersion,
Epoch: epoch,
},
LatestBlockHeader: state.LatestBlockHeader(),
BlockRoots: state.BlockRoots(),
StateRoots: state.StateRoots(),
HistoricalRoots: state.HistoricalRoots(),
Eth1Data: state.Eth1Data(),
Eth1DataVotes: state.Eth1DataVotes(),
Eth1DepositIndex: state.Eth1DepositIndex(),
Validators: state.Validators(),
Balances: state.Balances(),
RandaoMixes: state.RandaoMixes(),
Slashings: state.Slashings(),
PreviousEpochParticipation: make([]byte, numValidators),
CurrentEpochParticipation: make([]byte, numValidators),
JustificationBits: state.JustificationBits(),
PreviousJustifiedCheckpoint: state.PreviousJustifiedCheckpoint(),
CurrentJustifiedCheckpoint: state.CurrentJustifiedCheckpoint(),
FinalizedCheckpoint: state.FinalizedCheckpoint(),
InactivityScores: make([]uint64, numValidators),
}
newState, err := state_native.InitializeFromProtoUnsafeAltair(s)
if err != nil {
return nil, err
}
return newState, nil
}
// UpgradeToAltair updates input state to return the version Altair state.
//
// Spec code:
@@ -64,39 +104,7 @@ import (
// post.next_sync_committee = get_next_sync_committee(post)
// return post
func UpgradeToAltair(ctx context.Context, state state.BeaconState) (state.BeaconState, error) {
epoch := time.CurrentEpoch(state)
numValidators := state.NumValidators()
s := &ethpb.BeaconStateAltair{
GenesisTime: uint64(state.GenesisTime().Unix()),
GenesisValidatorsRoot: state.GenesisValidatorsRoot(),
Slot: state.Slot(),
Fork: &ethpb.Fork{
PreviousVersion: state.Fork().CurrentVersion,
CurrentVersion: params.BeaconConfig().AltairForkVersion,
Epoch: epoch,
},
LatestBlockHeader: state.LatestBlockHeader(),
BlockRoots: state.BlockRoots(),
StateRoots: state.StateRoots(),
HistoricalRoots: state.HistoricalRoots(),
Eth1Data: state.Eth1Data(),
Eth1DataVotes: state.Eth1DataVotes(),
Eth1DepositIndex: state.Eth1DepositIndex(),
Validators: state.Validators(),
Balances: state.Balances(),
RandaoMixes: state.RandaoMixes(),
Slashings: state.Slashings(),
PreviousEpochParticipation: make([]byte, numValidators),
CurrentEpochParticipation: make([]byte, numValidators),
JustificationBits: state.JustificationBits(),
PreviousJustifiedCheckpoint: state.PreviousJustifiedCheckpoint(),
CurrentJustifiedCheckpoint: state.CurrentJustifiedCheckpoint(),
FinalizedCheckpoint: state.FinalizedCheckpoint(),
InactivityScores: make([]uint64, numValidators),
}
newState, err := state_native.InitializeFromProtoUnsafeAltair(s)
newState, err := ConvertToAltair(state)
if err != nil {
return nil, err
}

View File

@@ -15,6 +15,129 @@ import (
"github.com/pkg/errors"
)
// ConvertToElectra converts a Deneb beacon state to an Electra beacon state. It does not perform any fork logic.
func ConvertToElectra(beaconState state.BeaconState) (state.BeaconState, error) {
currentSyncCommittee, err := beaconState.CurrentSyncCommittee()
if err != nil {
return nil, err
}
nextSyncCommittee, err := beaconState.NextSyncCommittee()
if err != nil {
return nil, err
}
prevEpochParticipation, err := beaconState.PreviousEpochParticipation()
if err != nil {
return nil, err
}
currentEpochParticipation, err := beaconState.CurrentEpochParticipation()
if err != nil {
return nil, err
}
inactivityScores, err := beaconState.InactivityScores()
if err != nil {
return nil, err
}
payloadHeader, err := beaconState.LatestExecutionPayloadHeader()
if err != nil {
return nil, err
}
txRoot, err := payloadHeader.TransactionsRoot()
if err != nil {
return nil, err
}
wdRoot, err := payloadHeader.WithdrawalsRoot()
if err != nil {
return nil, err
}
wi, err := beaconState.NextWithdrawalIndex()
if err != nil {
return nil, err
}
vi, err := beaconState.NextWithdrawalValidatorIndex()
if err != nil {
return nil, err
}
summaries, err := beaconState.HistoricalSummaries()
if err != nil {
return nil, err
}
excessBlobGas, err := payloadHeader.ExcessBlobGas()
if err != nil {
return nil, err
}
blobGasUsed, err := payloadHeader.BlobGasUsed()
if err != nil {
return nil, err
}
s := &ethpb.BeaconStateElectra{
GenesisTime: uint64(beaconState.GenesisTime().Unix()),
GenesisValidatorsRoot: beaconState.GenesisValidatorsRoot(),
Slot: beaconState.Slot(),
Fork: &ethpb.Fork{
PreviousVersion: beaconState.Fork().CurrentVersion,
CurrentVersion: params.BeaconConfig().ElectraForkVersion,
Epoch: time.CurrentEpoch(beaconState),
},
LatestBlockHeader: beaconState.LatestBlockHeader(),
BlockRoots: beaconState.BlockRoots(),
StateRoots: beaconState.StateRoots(),
HistoricalRoots: beaconState.HistoricalRoots(),
Eth1Data: beaconState.Eth1Data(),
Eth1DataVotes: beaconState.Eth1DataVotes(),
Eth1DepositIndex: beaconState.Eth1DepositIndex(),
Validators: beaconState.Validators(),
Balances: beaconState.Balances(),
RandaoMixes: beaconState.RandaoMixes(),
Slashings: beaconState.Slashings(),
PreviousEpochParticipation: prevEpochParticipation,
CurrentEpochParticipation: currentEpochParticipation,
JustificationBits: beaconState.JustificationBits(),
PreviousJustifiedCheckpoint: beaconState.PreviousJustifiedCheckpoint(),
CurrentJustifiedCheckpoint: beaconState.CurrentJustifiedCheckpoint(),
FinalizedCheckpoint: beaconState.FinalizedCheckpoint(),
InactivityScores: inactivityScores,
CurrentSyncCommittee: currentSyncCommittee,
NextSyncCommittee: nextSyncCommittee,
LatestExecutionPayloadHeader: &enginev1.ExecutionPayloadHeaderDeneb{
ParentHash: payloadHeader.ParentHash(),
FeeRecipient: payloadHeader.FeeRecipient(),
StateRoot: payloadHeader.StateRoot(),
ReceiptsRoot: payloadHeader.ReceiptsRoot(),
LogsBloom: payloadHeader.LogsBloom(),
PrevRandao: payloadHeader.PrevRandao(),
BlockNumber: payloadHeader.BlockNumber(),
GasLimit: payloadHeader.GasLimit(),
GasUsed: payloadHeader.GasUsed(),
Timestamp: payloadHeader.Timestamp(),
ExtraData: payloadHeader.ExtraData(),
BaseFeePerGas: payloadHeader.BaseFeePerGas(),
BlockHash: payloadHeader.BlockHash(),
TransactionsRoot: txRoot,
WithdrawalsRoot: wdRoot,
ExcessBlobGas: excessBlobGas,
BlobGasUsed: blobGasUsed,
},
NextWithdrawalIndex: wi,
NextWithdrawalValidatorIndex: vi,
HistoricalSummaries: summaries,
DepositRequestsStartIndex: params.BeaconConfig().UnsetDepositRequestsStartIndex,
DepositBalanceToConsume: 0,
EarliestConsolidationEpoch: helpers.ActivationExitEpoch(slots.ToEpoch(beaconState.Slot())),
PendingDeposits: make([]*ethpb.PendingDeposit, 0),
PendingPartialWithdrawals: make([]*ethpb.PendingPartialWithdrawal, 0),
PendingConsolidations: make([]*ethpb.PendingConsolidation, 0),
}
// need to cast the beaconState to use in helper functions
post, err := state_native.InitializeFromProtoUnsafeElectra(s)
if err != nil {
return nil, errors.Wrap(err, "failed to initialize post electra beaconState")
}
return post, nil
}
// UpgradeToElectra updates inputs a generic state to return the version Electra state.
//
// nolint:dupword
@@ -126,55 +249,7 @@ import (
//
// return post
func UpgradeToElectra(beaconState state.BeaconState) (state.BeaconState, error) {
currentSyncCommittee, err := beaconState.CurrentSyncCommittee()
if err != nil {
return nil, err
}
nextSyncCommittee, err := beaconState.NextSyncCommittee()
if err != nil {
return nil, err
}
prevEpochParticipation, err := beaconState.PreviousEpochParticipation()
if err != nil {
return nil, err
}
currentEpochParticipation, err := beaconState.CurrentEpochParticipation()
if err != nil {
return nil, err
}
inactivityScores, err := beaconState.InactivityScores()
if err != nil {
return nil, err
}
payloadHeader, err := beaconState.LatestExecutionPayloadHeader()
if err != nil {
return nil, err
}
txRoot, err := payloadHeader.TransactionsRoot()
if err != nil {
return nil, err
}
wdRoot, err := payloadHeader.WithdrawalsRoot()
if err != nil {
return nil, err
}
wi, err := beaconState.NextWithdrawalIndex()
if err != nil {
return nil, err
}
vi, err := beaconState.NextWithdrawalValidatorIndex()
if err != nil {
return nil, err
}
summaries, err := beaconState.HistoricalSummaries()
if err != nil {
return nil, err
}
excessBlobGas, err := payloadHeader.ExcessBlobGas()
if err != nil {
return nil, err
}
blobGasUsed, err := payloadHeader.BlobGasUsed()
s, err := ConvertToElectra(beaconState)
if err != nil {
return nil, err
}
@@ -206,97 +281,38 @@ func UpgradeToElectra(beaconState state.BeaconState) (state.BeaconState, error)
if err != nil {
return nil, errors.Wrap(err, "failed to get total active balance")
}
s := &ethpb.BeaconStateElectra{
GenesisTime: uint64(beaconState.GenesisTime().Unix()),
GenesisValidatorsRoot: beaconState.GenesisValidatorsRoot(),
Slot: beaconState.Slot(),
Fork: &ethpb.Fork{
PreviousVersion: beaconState.Fork().CurrentVersion,
CurrentVersion: params.BeaconConfig().ElectraForkVersion,
Epoch: time.CurrentEpoch(beaconState),
},
LatestBlockHeader: beaconState.LatestBlockHeader(),
BlockRoots: beaconState.BlockRoots(),
StateRoots: beaconState.StateRoots(),
HistoricalRoots: beaconState.HistoricalRoots(),
Eth1Data: beaconState.Eth1Data(),
Eth1DataVotes: beaconState.Eth1DataVotes(),
Eth1DepositIndex: beaconState.Eth1DepositIndex(),
Validators: beaconState.Validators(),
Balances: beaconState.Balances(),
RandaoMixes: beaconState.RandaoMixes(),
Slashings: beaconState.Slashings(),
PreviousEpochParticipation: prevEpochParticipation,
CurrentEpochParticipation: currentEpochParticipation,
JustificationBits: beaconState.JustificationBits(),
PreviousJustifiedCheckpoint: beaconState.PreviousJustifiedCheckpoint(),
CurrentJustifiedCheckpoint: beaconState.CurrentJustifiedCheckpoint(),
FinalizedCheckpoint: beaconState.FinalizedCheckpoint(),
InactivityScores: inactivityScores,
CurrentSyncCommittee: currentSyncCommittee,
NextSyncCommittee: nextSyncCommittee,
LatestExecutionPayloadHeader: &enginev1.ExecutionPayloadHeaderDeneb{
ParentHash: payloadHeader.ParentHash(),
FeeRecipient: payloadHeader.FeeRecipient(),
StateRoot: payloadHeader.StateRoot(),
ReceiptsRoot: payloadHeader.ReceiptsRoot(),
LogsBloom: payloadHeader.LogsBloom(),
PrevRandao: payloadHeader.PrevRandao(),
BlockNumber: payloadHeader.BlockNumber(),
GasLimit: payloadHeader.GasLimit(),
GasUsed: payloadHeader.GasUsed(),
Timestamp: payloadHeader.Timestamp(),
ExtraData: payloadHeader.ExtraData(),
BaseFeePerGas: payloadHeader.BaseFeePerGas(),
BlockHash: payloadHeader.BlockHash(),
TransactionsRoot: txRoot,
WithdrawalsRoot: wdRoot,
ExcessBlobGas: excessBlobGas,
BlobGasUsed: blobGasUsed,
},
NextWithdrawalIndex: wi,
NextWithdrawalValidatorIndex: vi,
HistoricalSummaries: summaries,
DepositRequestsStartIndex: params.BeaconConfig().UnsetDepositRequestsStartIndex,
DepositBalanceToConsume: 0,
ExitBalanceToConsume: helpers.ActivationExitChurnLimit(primitives.Gwei(tab)),
EarliestExitEpoch: earliestExitEpoch,
ConsolidationBalanceToConsume: helpers.ConsolidationChurnLimit(primitives.Gwei(tab)),
EarliestConsolidationEpoch: helpers.ActivationExitEpoch(slots.ToEpoch(beaconState.Slot())),
PendingDeposits: make([]*ethpb.PendingDeposit, 0),
PendingPartialWithdrawals: make([]*ethpb.PendingPartialWithdrawal, 0),
PendingConsolidations: make([]*ethpb.PendingConsolidation, 0),
if err := s.SetExitBalanceToConsume(helpers.ActivationExitChurnLimit(primitives.Gwei(tab))); err != nil {
return nil, errors.Wrap(err, "failed to set exit balance to consume")
}
if err := s.SetEarliestExitEpoch(earliestExitEpoch); err != nil {
return nil, errors.Wrap(err, "failed to set earliest exit epoch")
}
if err := s.SetConsolidationBalanceToConsume(helpers.ConsolidationChurnLimit(primitives.Gwei(tab))); err != nil {
return nil, errors.Wrap(err, "failed to set consolidation balance to consume")
}
// Sorting preActivationIndices based on a custom criteria
vals := s.Validators()
sort.Slice(preActivationIndices, func(i, j int) bool {
// Comparing based on ActivationEligibilityEpoch and then by index if the epochs are the same
if s.Validators[preActivationIndices[i]].ActivationEligibilityEpoch == s.Validators[preActivationIndices[j]].ActivationEligibilityEpoch {
if vals[preActivationIndices[i]].ActivationEligibilityEpoch == vals[preActivationIndices[j]].ActivationEligibilityEpoch {
return preActivationIndices[i] < preActivationIndices[j]
}
return s.Validators[preActivationIndices[i]].ActivationEligibilityEpoch < s.Validators[preActivationIndices[j]].ActivationEligibilityEpoch
return vals[preActivationIndices[i]].ActivationEligibilityEpoch < vals[preActivationIndices[j]].ActivationEligibilityEpoch
})
// need to cast the beaconState to use in helper functions
post, err := state_native.InitializeFromProtoUnsafeElectra(s)
if err != nil {
return nil, errors.Wrap(err, "failed to initialize post electra beaconState")
}
for _, index := range preActivationIndices {
if err := QueueEntireBalanceAndResetValidator(post, index); err != nil {
if err := QueueEntireBalanceAndResetValidator(s, index); err != nil {
return nil, errors.Wrap(err, "failed to queue entire balance and reset validator")
}
}
// Ensure early adopters of compounding credentials go through the activation churn
for _, index := range compoundWithdrawalIndices {
if err := QueueExcessActiveBalance(post, index); err != nil {
if err := QueueExcessActiveBalance(s, index); err != nil {
return nil, errors.Wrap(err, "failed to queue excess active balance")
}
}
return post, nil
return s, nil
}

View File

@@ -7,6 +7,7 @@ go_library(
visibility = [
"//beacon-chain:__subpackages__",
"//cmd/prysmctl/testnet:__pkg__",
"//consensus-types/hdiff:__subpackages__",
"//testing/spectest:__subpackages__",
"//validator/client:__pkg__",
],

View File

@@ -15,6 +15,7 @@ go_library(
"//beacon-chain/state:go_default_library",
"//beacon-chain/state/state-native:go_default_library",
"//config/params:go_default_library",
"//consensus-types/primitives:go_default_library",
"//monitoring/tracing/trace:go_default_library",
"//proto/engine/v1:go_default_library",
"//proto/prysm/v1alpha1:go_default_library",

View File

@@ -8,6 +8,7 @@ import (
"github.com/OffchainLabs/prysm/v6/beacon-chain/state"
state_native "github.com/OffchainLabs/prysm/v6/beacon-chain/state/state-native"
"github.com/OffchainLabs/prysm/v6/config/params"
"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
enginev1 "github.com/OffchainLabs/prysm/v6/proto/engine/v1"
ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
"github.com/OffchainLabs/prysm/v6/time/slots"
@@ -17,6 +18,25 @@ import (
// UpgradeToFulu updates inputs a generic state to return the version Fulu state.
// https://github.com/ethereum/consensus-specs/blob/master/specs/fulu/fork.md#upgrading-the-state
func UpgradeToFulu(ctx context.Context, beaconState state.BeaconState) (state.BeaconState, error) {
s, err := ConvertToFulu(beaconState)
if err != nil {
return nil, errors.Wrap(err, "could not convert to fulu")
}
proposerLookahead, err := helpers.InitializeProposerLookahead(ctx, beaconState, slots.ToEpoch(beaconState.Slot()))
if err != nil {
return nil, err
}
pl := make([]primitives.ValidatorIndex, len(proposerLookahead))
for i, v := range proposerLookahead {
pl[i] = primitives.ValidatorIndex(v)
}
if err := s.SetProposerLookahead(pl); err != nil {
return nil, errors.Wrap(err, "failed to set proposer lookahead")
}
return s, nil
}
func ConvertToFulu(beaconState state.BeaconState) (state.BeaconState, error) {
currentSyncCommittee, err := beaconState.CurrentSyncCommittee()
if err != nil {
return nil, err
@@ -105,11 +125,6 @@ func UpgradeToFulu(ctx context.Context, beaconState state.BeaconState) (state.Be
if err != nil {
return nil, err
}
proposerLookahead, err := helpers.InitializeProposerLookahead(ctx, beaconState, slots.ToEpoch(beaconState.Slot()))
if err != nil {
return nil, err
}
s := &ethpb.BeaconStateFulu{
GenesisTime: uint64(beaconState.GenesisTime().Unix()),
GenesisValidatorsRoot: beaconState.GenesisValidatorsRoot(),
@@ -171,14 +186,6 @@ func UpgradeToFulu(ctx context.Context, beaconState state.BeaconState) (state.Be
PendingDeposits: pendingDeposits,
PendingPartialWithdrawals: pendingPartialWithdrawals,
PendingConsolidations: pendingConsolidations,
ProposerLookahead: proposerLookahead,
}
// Need to cast the beaconState to use in helper functions
post, err := state_native.InitializeFromProtoUnsafeFulu(s)
if err != nil {
return nil, errors.Wrap(err, "failed to initialize post fulu beaconState")
}
return post, nil
return state_native.InitializeFromProtoUnsafeFulu(s)
}

View File

@@ -201,14 +201,3 @@ func ParseWeakSubjectivityInputString(wsCheckpointString string) (*v1alpha1.Chec
Root: bRoot,
}, nil
}
// MinEpochsForBlockRequests computes the number of epochs of block history that we need to maintain,
// relative to the current epoch, per the p2p specs. This is used to compute the slot where backfill is complete.
// value defined:
// https://github.com/ethereum/consensus-specs/blob/master/specs/phase0/p2p-interface.md#configuration
// MIN_VALIDATOR_WITHDRAWABILITY_DELAY + CHURN_LIMIT_QUOTIENT // 2 (= 33024, ~5 months)
// detailed rationale: https://github.com/ethereum/consensus-specs/blob/master/specs/phase0/p2p-interface.md#why-are-blocksbyrange-requests-only-required-to-be-served-for-the-latest-min_epochs_for_block_requests-epochs
func MinEpochsForBlockRequests() primitives.Epoch {
return params.BeaconConfig().MinValidatorWithdrawabilityDelay +
primitives.Epoch(params.BeaconConfig().ChurnLimitQuotient/2)
}

View File

@@ -286,20 +286,3 @@ func genState(t *testing.T, valCount, avgBalance uint64) state.BeaconState {
return beaconState
}
func TestMinEpochsForBlockRequests(t *testing.T) {
helpers.ClearCache()
params.SetActiveTestCleanup(t, params.MainnetConfig())
var expected primitives.Epoch = 33024
// expected value of 33024 via spec commentary:
// https://github.com/ethereum/consensus-specs/blob/master/specs/phase0/p2p-interface.md#why-are-blocksbyrange-requests-only-required-to-be-served-for-the-latest-min_epochs_for_block_requests-epochs
// MIN_EPOCHS_FOR_BLOCK_REQUESTS is calculated using the arithmetic from compute_weak_subjectivity_period found in the weak subjectivity guide. Specifically to find this max epoch range, we use the worst case event of a very large validator size (>= MIN_PER_EPOCH_CHURN_LIMIT * CHURN_LIMIT_QUOTIENT).
//
// MIN_EPOCHS_FOR_BLOCK_REQUESTS = (
// MIN_VALIDATOR_WITHDRAWABILITY_DELAY
// + MAX_SAFETY_DECAY * CHURN_LIMIT_QUOTIENT // (2 * 100)
// )
//
// Where MAX_SAFETY_DECAY = 100 and thus MIN_EPOCHS_FOR_BLOCK_REQUESTS = 33024 (~5 months).
require.Equal(t, expected, helpers.MinEpochsForBlockRequests())
}

View File

@@ -79,10 +79,30 @@ func VerifyDataColumnsSidecarKZGProofs(sidecars []blocks.RODataColumn) error {
for _, sidecar := range sidecars {
for i := range sidecar.Column {
commitments = append(commitments, kzg.Bytes48(sidecar.KzgCommitments[i]))
var (
commitment kzg.Bytes48
cell kzg.Cell
proof kzg.Bytes48
)
commitmentBytes := sidecar.KzgCommitments[i]
cellBytes := sidecar.Column[i]
proofBytes := sidecar.KzgProofs[i]
if len(commitmentBytes) != len(commitment) ||
len(cellBytes) != len(cell) ||
len(proofBytes) != len(proof) {
return ErrMismatchLength
}
copy(commitment[:], commitmentBytes)
copy(cell[:], cellBytes)
copy(proof[:], proofBytes)
commitments = append(commitments, commitment)
indices = append(indices, sidecar.Index)
cells = append(cells, kzg.Cell(sidecar.Column[i]))
proofs = append(proofs, kzg.Bytes48(sidecar.KzgProofs[i]))
cells = append(cells, cell)
proofs = append(proofs, proof)
}
}

View File

@@ -68,6 +68,14 @@ func TestVerifyDataColumnSidecarKZGProofs(t *testing.T) {
err := kzg.Start()
require.NoError(t, err)
t.Run("size mismatch", func(t *testing.T) {
sidecars := generateRandomSidecars(t, seed, blobCount)
sidecars[0].Column[0] = sidecars[0].Column[0][:len(sidecars[0].Column[0])-1] // Remove one byte to create size mismatch
err := peerdas.VerifyDataColumnsSidecarKZGProofs(sidecars)
require.ErrorIs(t, err, peerdas.ErrMismatchLength)
})
t.Run("invalid proof", func(t *testing.T) {
sidecars := generateRandomSidecars(t, seed, blobCount)
sidecars[0].Column[0][0]++ // It is OK to overflow

View File

@@ -200,6 +200,7 @@ func (dcs *DataColumnStorage) WarmCache() {
fileMetadata, err := extractFileMetadata(path)
if err != nil {
log.WithError(err).Error("Error encountered while extracting file metadata")
return nil
}
// Open the data column filesystem file.
@@ -988,8 +989,8 @@ func filePath(root [fieldparams.RootLength]byte, epoch primitives.Epoch) string
// extractFileMetadata extracts the metadata from a file path.
// If the path is not a leaf, it returns nil.
func extractFileMetadata(path string) (*fileMetadata, error) {
// Is this Windows friendly?
parts := strings.Split(path, "/")
// Use filepath.Separator to handle both Windows (\) and Unix (/) path separators
parts := strings.Split(path, string(filepath.Separator))
if len(parts) != 3 {
return nil, errors.Errorf("unexpected file %s", path)
}

View File

@@ -3,6 +3,7 @@ package filesystem
import (
"encoding/binary"
"os"
"path/filepath"
"testing"
fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
@@ -725,3 +726,37 @@ func TestPrune(t *testing.T) {
require.Equal(t, true, compareSlices([]string{"0x0de28a18cae63cbc6f0b20dc1afb0b1df38da40824a5f09f92d485ade04de97f.sszs"}, dirs))
})
}
func TestExtractFileMetadata(t *testing.T) {
t.Run("Unix", func(t *testing.T) {
// Test with Unix-style path separators (/)
path := "12/1234/0x8bb2f09de48c102635622dc27e6de03ae2b22639df7c33edbc8222b2ec423746.sszs"
metadata, err := extractFileMetadata(path)
if filepath.Separator == '/' {
// On Unix systems, this should succeed
require.NoError(t, err)
require.Equal(t, uint64(12), metadata.period)
require.Equal(t, primitives.Epoch(1234), metadata.epoch)
return
}
// On Windows systems, this should fail because it uses the wrong separator
require.NotNil(t, err)
})
t.Run("Windows", func(t *testing.T) {
// Test with Windows-style path separators (\)
path := "12\\1234\\0x8bb2f09de48c102635622dc27e6de03ae2b22639df7c33edbc8222b2ec423746.sszs"
metadata, err := extractFileMetadata(path)
if filepath.Separator == '\\' {
// On Windows systems, this should succeed
require.NoError(t, err)
require.Equal(t, uint64(12), metadata.period)
require.Equal(t, primitives.Epoch(1234), metadata.epoch)
return
}
// On Unix systems, this should fail because it uses the wrong separator
require.NotNil(t, err)
})
}

View File

@@ -129,6 +129,7 @@ type NoHeadAccessDatabase interface {
// Custody operations.
UpdateSubscribedToAllDataSubnets(ctx context.Context, subscribed bool) (bool, error)
UpdateCustodyInfo(ctx context.Context, earliestAvailableSlot primitives.Slot, custodyGroupCount uint64) (primitives.Slot, uint64, error)
UpdateEarliestAvailableSlot(ctx context.Context, earliestAvailableSlot primitives.Slot) error
// P2P Metadata operations.
SaveMetadataSeqNum(ctx context.Context, seqNum uint64) error

View File

@@ -132,6 +132,6 @@ func recoverStateSummary(ctx context.Context, tx *bolt.Tx, root []byte) error {
if err != nil {
return err
}
summaryBucket := tx.Bucket(stateBucket)
summaryBucket := tx.Bucket(stateSummaryBucket)
return summaryBucket.Put(root, summaryEnc)
}

View File

@@ -137,3 +137,32 @@ func TestStore_FinalizedCheckpoint_StateMustExist(t *testing.T) {
require.ErrorContains(t, errMissingStateForCheckpoint.Error(), db.SaveFinalizedCheckpoint(ctx, cp))
}
// Regression test: verify that saving a checkpoint triggers recovery which writes
// the state summary into the correct stateSummaryBucket so that HasStateSummary/StateSummary see it.
func TestRecoverStateSummary_WritesToStateSummaryBucket(t *testing.T) {
db := setupDB(t)
ctx := t.Context()
// Create a block without saving a state or summary, so recovery is needed.
blk := util.HydrateSignedBeaconBlock(&ethpb.SignedBeaconBlock{})
root, err := blk.Block.HashTreeRoot()
require.NoError(t, err)
wsb, err := blocks.NewSignedBeaconBlock(blk)
require.NoError(t, err)
require.NoError(t, db.SaveBlock(ctx, wsb))
// Precondition: summary not present yet.
require.Equal(t, false, db.HasStateSummary(ctx, root))
// Saving justified checkpoint should trigger recovery path calling recoverStateSummary.
cp := &ethpb.Checkpoint{Epoch: 2, Root: root[:]}
require.NoError(t, db.SaveJustifiedCheckpoint(ctx, cp))
// Postcondition: summary is visible via the public summary APIs (which read stateSummaryBucket).
require.Equal(t, true, db.HasStateSummary(ctx, root))
summary, err := db.StateSummary(ctx, root)
require.NoError(t, err)
require.NotNil(t, summary)
assert.DeepEqual(t, &ethpb.StateSummary{Slot: blk.Block.Slot, Root: root[:]}, summary)
}

View File

@@ -2,16 +2,19 @@ package kv
import (
"context"
"time"
"github.com/OffchainLabs/prysm/v6/config/params"
"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
"github.com/OffchainLabs/prysm/v6/encoding/bytesutil"
"github.com/OffchainLabs/prysm/v6/monitoring/tracing/trace"
"github.com/OffchainLabs/prysm/v6/time/slots"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
bolt "go.etcd.io/bbolt"
)
// UpdateCustodyInfo atomically updates the custody group count only it is greater than the stored one.
// UpdateCustodyInfo atomically updates the custody group count only if it is greater than the stored one.
// In this case, it also updates the earliest available slot with the provided value.
// It returns the (potentially updated) custody group count and earliest available slot.
func (s *Store) UpdateCustodyInfo(ctx context.Context, earliestAvailableSlot primitives.Slot, custodyGroupCount uint64) (primitives.Slot, uint64, error) {
@@ -70,6 +73,79 @@ func (s *Store) UpdateCustodyInfo(ctx context.Context, earliestAvailableSlot pri
return storedEarliestAvailableSlot, storedGroupCount, nil
}
// UpdateEarliestAvailableSlot updates the earliest available slot.
func (s *Store) UpdateEarliestAvailableSlot(ctx context.Context, earliestAvailableSlot primitives.Slot) error {
_, span := trace.StartSpan(ctx, "BeaconDB.UpdateEarliestAvailableSlot")
defer span.End()
storedEarliestAvailableSlot := primitives.Slot(0)
if err := s.db.Update(func(tx *bolt.Tx) error {
// Retrieve the custody bucket.
bucket, err := tx.CreateBucketIfNotExists(custodyBucket)
if err != nil {
return errors.Wrap(err, "create custody bucket")
}
// Retrieve the stored earliest available slot.
storedEarliestAvailableSlotBytes := bucket.Get(earliestAvailableSlotKey)
if len(storedEarliestAvailableSlotBytes) != 0 {
storedEarliestAvailableSlot = primitives.Slot(bytesutil.BytesToUint64BigEndian(storedEarliestAvailableSlotBytes))
}
// Allow decrease (for backfill scenarios)
if earliestAvailableSlot <= storedEarliestAvailableSlot {
storedEarliestAvailableSlot = earliestAvailableSlot
bytes := bytesutil.Uint64ToBytesBigEndian(uint64(earliestAvailableSlot))
if err := bucket.Put(earliestAvailableSlotKey, bytes); err != nil {
return errors.Wrap(err, "put earliest available slot")
}
return nil
}
// Prevent increase within the MIN_EPOCHS_FOR_BLOCK_REQUESTS period
// This ensures we don't voluntarily refuse to serve mandatory block data
genesisTime := time.Unix(int64(params.BeaconConfig().MinGenesisTime+params.BeaconConfig().GenesisDelay), 0)
currentSlot := slots.CurrentSlot(genesisTime)
currentEpoch := slots.ToEpoch(currentSlot)
minEpochsForBlocks := primitives.Epoch(params.BeaconConfig().MinEpochsForBlockRequests)
// Calculate the minimum required epoch (or 0 if we're early in the chain)
minRequiredEpoch := primitives.Epoch(0)
if currentEpoch > minEpochsForBlocks {
minRequiredEpoch = currentEpoch - minEpochsForBlocks
}
// Convert to slot to ensure we compare at slot-level granularity
minRequiredSlot, err := slots.EpochStart(minRequiredEpoch)
if err != nil {
return errors.Wrap(err, "calculate minimum required slot")
}
// Prevent any increase that would put earliest available slot beyond the minimum required slot
if earliestAvailableSlot > minRequiredSlot {
return errors.Errorf(
"cannot increase earliest available slot to %d (epoch %d) as it exceeds minimum required slot %d (epoch %d)",
earliestAvailableSlot, slots.ToEpoch(earliestAvailableSlot),
minRequiredSlot, minRequiredEpoch,
)
}
storedEarliestAvailableSlot = earliestAvailableSlot
bytes := bytesutil.Uint64ToBytesBigEndian(uint64(earliestAvailableSlot))
if err := bucket.Put(earliestAvailableSlotKey, bytes); err != nil {
return errors.Wrap(err, "put earliest available slot")
}
return nil
}); err != nil {
return err
}
log.WithField("earliestAvailableSlot", storedEarliestAvailableSlot).Debug("Updated earliest available slot")
return nil
}
// UpdateSubscribedToAllDataSubnets updates the "subscribed to all data subnets" status in the database
// only if `subscribed` is `true`.
// It returns the previous subscription status.

View File

@@ -3,10 +3,13 @@ package kv
import (
"context"
"testing"
"time"
"github.com/OffchainLabs/prysm/v6/config/params"
"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
"github.com/OffchainLabs/prysm/v6/encoding/bytesutil"
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/OffchainLabs/prysm/v6/time/slots"
bolt "go.etcd.io/bbolt"
)
@@ -132,6 +135,131 @@ func TestUpdateCustodyInfo(t *testing.T) {
})
}
func TestUpdateEarliestAvailableSlot(t *testing.T) {
ctx := t.Context()
t.Run("allow decreasing earliest slot (backfill scenario)", func(t *testing.T) {
const (
initialSlot = primitives.Slot(300)
initialCount = uint64(10)
earliestSlot = primitives.Slot(200) // Lower than initial (backfill discovered earlier blocks)
)
db := setupDB(t)
// Initialize custody info
_, _, err := db.UpdateCustodyInfo(ctx, initialSlot, initialCount)
require.NoError(t, err)
// Update with a lower slot (should update for backfill)
err = db.UpdateEarliestAvailableSlot(ctx, earliestSlot)
require.NoError(t, err)
storedSlot, storedCount := getCustodyInfoFromDB(t, db)
require.Equal(t, earliestSlot, storedSlot)
require.Equal(t, initialCount, storedCount)
})
t.Run("allow increasing slot within MIN_EPOCHS_FOR_BLOCK_REQUESTS (pruning scenario)", func(t *testing.T) {
db := setupDB(t)
// Calculate the current slot and minimum required slot based on actual current time
genesisTime := time.Unix(int64(params.BeaconConfig().MinGenesisTime+params.BeaconConfig().GenesisDelay), 0)
currentSlot := slots.CurrentSlot(genesisTime)
currentEpoch := slots.ToEpoch(currentSlot)
minEpochsForBlocks := primitives.Epoch(params.BeaconConfig().MinEpochsForBlockRequests)
var minRequiredEpoch primitives.Epoch
if currentEpoch > minEpochsForBlocks {
minRequiredEpoch = currentEpoch - minEpochsForBlocks
} else {
minRequiredEpoch = 0
}
minRequiredSlot, err := slots.EpochStart(minRequiredEpoch)
require.NoError(t, err)
// Initial setup: set earliest slot well before minRequiredSlot
const groupCount = uint64(5)
initialSlot := primitives.Slot(1000)
_, _, err = db.UpdateCustodyInfo(ctx, initialSlot, groupCount)
require.NoError(t, err)
// Try to increase to a slot that's still BEFORE minRequiredSlot (should succeed)
validSlot := minRequiredSlot - 100
err = db.UpdateEarliestAvailableSlot(ctx, validSlot)
require.NoError(t, err)
// Verify the database was updated
storedSlot, storedCount := getCustodyInfoFromDB(t, db)
require.Equal(t, validSlot, storedSlot)
require.Equal(t, groupCount, storedCount)
})
t.Run("prevent increasing slot beyond MIN_EPOCHS_FOR_BLOCK_REQUESTS", func(t *testing.T) {
db := setupDB(t)
// Calculate the current slot and minimum required slot based on actual current time
genesisTime := time.Unix(int64(params.BeaconConfig().MinGenesisTime+params.BeaconConfig().GenesisDelay), 0)
currentSlot := slots.CurrentSlot(genesisTime)
currentEpoch := slots.ToEpoch(currentSlot)
minEpochsForBlocks := primitives.Epoch(params.BeaconConfig().MinEpochsForBlockRequests)
var minRequiredEpoch primitives.Epoch
if currentEpoch > minEpochsForBlocks {
minRequiredEpoch = currentEpoch - minEpochsForBlocks
} else {
minRequiredEpoch = 0
}
minRequiredSlot, err := slots.EpochStart(minRequiredEpoch)
require.NoError(t, err)
// Initial setup: set a valid earliest slot (well before minRequiredSlot)
const initialCount = uint64(5)
initialSlot := primitives.Slot(1000)
_, _, err = db.UpdateCustodyInfo(ctx, initialSlot, initialCount)
require.NoError(t, err)
// Try to set earliest slot beyond the minimum required slot
invalidSlot := minRequiredSlot + 100
// This should fail
err = db.UpdateEarliestAvailableSlot(ctx, invalidSlot)
require.ErrorContains(t, "cannot increase earliest available slot", err)
require.ErrorContains(t, "exceeds minimum required slot", err)
// Verify the database wasn't updated
storedSlot, storedCount := getCustodyInfoFromDB(t, db)
require.Equal(t, initialSlot, storedSlot)
require.Equal(t, initialCount, storedCount)
})
t.Run("no change when slot equals current slot", func(t *testing.T) {
const (
initialSlot = primitives.Slot(100)
initialCount = uint64(5)
)
db := setupDB(t)
// Initialize custody info
_, _, err := db.UpdateCustodyInfo(ctx, initialSlot, initialCount)
require.NoError(t, err)
// Update with the same slot
err = db.UpdateEarliestAvailableSlot(ctx, initialSlot)
require.NoError(t, err)
storedSlot, storedCount := getCustodyInfoFromDB(t, db)
require.Equal(t, initialSlot, storedSlot)
require.Equal(t, initialCount, storedCount)
})
}
func TestUpdateSubscribedToAllDataSubnets(t *testing.T) {
ctx := context.Background()

View File

@@ -8,7 +8,6 @@ go_library(
"//beacon-chain:__subpackages__",
],
deps = [
"//beacon-chain/core/helpers:go_default_library",
"//beacon-chain/db:go_default_library",
"//beacon-chain/db/iface:go_default_library",
"//config/params:go_default_library",
@@ -29,6 +28,7 @@ go_test(
"//consensus-types/blocks:go_default_library",
"//consensus-types/primitives:go_default_library",
"//proto/prysm/v1alpha1:go_default_library",
"//testing/assert:go_default_library",
"//testing/require:go_default_library",
"//testing/util:go_default_library",
"//time/slots/testing:go_default_library",

View File

@@ -4,7 +4,6 @@ import (
"context"
"time"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
"github.com/OffchainLabs/prysm/v6/beacon-chain/db"
"github.com/OffchainLabs/prysm/v6/beacon-chain/db/iface"
"github.com/OffchainLabs/prysm/v6/config/params"
@@ -25,17 +24,24 @@ const (
defaultNumBatchesToPrune = 15
)
// custodyUpdater is a tiny interface that p2p service implements; kept here to avoid
// importing the p2p package and creating a cycle.
type custodyUpdater interface {
UpdateEarliestAvailableSlot(earliestAvailableSlot primitives.Slot) error
}
type ServiceOption func(*Service)
// WithRetentionPeriod allows the user to specify a different data retention period than the spec default.
// The retention period is specified in epochs, and must be >= MIN_EPOCHS_FOR_BLOCK_REQUESTS.
func WithRetentionPeriod(retentionEpochs primitives.Epoch) ServiceOption {
return func(s *Service) {
defaultRetentionEpochs := helpers.MinEpochsForBlockRequests() + 1
defaultRetentionEpochs := primitives.Epoch(params.BeaconConfig().MinEpochsForBlockRequests) + 1
if retentionEpochs < defaultRetentionEpochs {
log.WithField("userEpochs", retentionEpochs).
WithField("minRequired", defaultRetentionEpochs).
Warn("Retention period too low, using minimum required value")
Warn("Retention period too low, ignoring and using minimum required value")
retentionEpochs = defaultRetentionEpochs
}
s.ps = pruneStartSlotFunc(retentionEpochs)
@@ -58,17 +64,23 @@ type Service struct {
slotTicker slots.Ticker
backfillWaiter func() error
initSyncWaiter func() error
custody custodyUpdater
}
func New(ctx context.Context, db iface.Database, genesisTime time.Time, initSyncWaiter, backfillWaiter func() error, opts ...ServiceOption) (*Service, error) {
func New(ctx context.Context, db iface.Database, genesisTime time.Time, initSyncWaiter, backfillWaiter func() error, custody custodyUpdater, opts ...ServiceOption) (*Service, error) {
if custody == nil {
return nil, errors.New("custody updater is required for pruner but was not provided")
}
p := &Service{
ctx: ctx,
db: db,
ps: pruneStartSlotFunc(helpers.MinEpochsForBlockRequests() + 1), // Default retention epochs is MIN_EPOCHS_FOR_BLOCK_REQUESTS + 1 from the current slot.
ps: pruneStartSlotFunc(primitives.Epoch(params.BeaconConfig().MinEpochsForBlockRequests) + 1), // Default retention epochs is MIN_EPOCHS_FOR_BLOCK_REQUESTS + 1 from the current slot.
done: make(chan struct{}),
slotTicker: slots.NewSlotTicker(slots.UnsafeStartTime(genesisTime, 0), params.BeaconConfig().SecondsPerSlot),
initSyncWaiter: initSyncWaiter,
backfillWaiter: backfillWaiter,
custody: custody,
}
for _, o := range opts {
@@ -157,17 +169,45 @@ func (p *Service) prune(slot primitives.Slot) error {
return errors.Wrap(err, "failed to prune batches")
}
log.WithFields(logrus.Fields{
"prunedUpto": pruneUpto,
"duration": time.Since(tt),
"currentSlot": slot,
"batchSize": defaultPrunableBatchSize,
"numBatches": numBatches,
}).Debug("Successfully pruned chain data")
earliestAvailableSlot := pruneUpto + 1
// Update pruning checkpoint.
p.prunedUpto = pruneUpto
// Update the earliest available slot after pruning
if err := p.updateEarliestAvailableSlot(earliestAvailableSlot); err != nil {
return errors.Wrap(err, "update earliest available slot")
}
log.WithFields(logrus.Fields{
"prunedUpto": pruneUpto,
"earliestAvailableSlot": earliestAvailableSlot,
"duration": time.Since(tt),
"currentSlot": slot,
"batchSize": defaultPrunableBatchSize,
"numBatches": numBatches,
}).Debug("Successfully pruned chain data")
return nil
}
// updateEarliestAvailableSlot updates the earliest available slot via the injected custody updater
// and also persists it to the database.
func (p *Service) updateEarliestAvailableSlot(earliestAvailableSlot primitives.Slot) error {
if !params.FuluEnabled() {
return nil
}
// Update the p2p in-memory state
if err := p.custody.UpdateEarliestAvailableSlot(earliestAvailableSlot); err != nil {
return errors.Wrapf(err, "update earliest available slot after pruning to %d", earliestAvailableSlot)
}
// Persist to database to ensure it survives restarts
if err := p.db.UpdateEarliestAvailableSlot(p.ctx, earliestAvailableSlot); err != nil {
return errors.Wrapf(err, "update earliest available slot in database for slot %d", earliestAvailableSlot)
}
return nil
}

View File

@@ -2,6 +2,7 @@ package pruner
import (
"context"
"errors"
"testing"
"time"
@@ -15,6 +16,7 @@ import (
dbtest "github.com/OffchainLabs/prysm/v6/beacon-chain/db/testing"
"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
"github.com/OffchainLabs/prysm/v6/testing/assert"
"github.com/OffchainLabs/prysm/v6/testing/require"
logTest "github.com/sirupsen/logrus/hooks/test"
)
@@ -62,7 +64,9 @@ func TestPruner_PruningConditions(t *testing.T) {
if !tt.backfillCompleted {
backfillWaiter = waiter
}
p, err := New(ctx, beaconDB, time.Now(), initSyncWaiter, backfillWaiter, WithSlotTicker(slotTicker))
mockCustody := &mockCustodyUpdater{}
p, err := New(ctx, beaconDB, time.Now(), initSyncWaiter, backfillWaiter, mockCustody, WithSlotTicker(slotTicker))
require.NoError(t, err)
go p.Start()
@@ -97,12 +101,14 @@ func TestPruner_PruneSuccess(t *testing.T) {
retentionEpochs := primitives.Epoch(2)
slotTicker := &slottest.MockTicker{Channel: make(chan primitives.Slot)}
mockCustody := &mockCustodyUpdater{}
p, err := New(
ctx,
beaconDB,
time.Now(),
nil,
nil,
mockCustody,
WithSlotTicker(slotTicker),
)
require.NoError(t, err)
@@ -133,3 +139,242 @@ func TestPruner_PruneSuccess(t *testing.T) {
require.NoError(t, p.Stop())
}
// Mock custody updater for testing
type mockCustodyUpdater struct {
custodyGroupCount uint64
earliestAvailableSlot primitives.Slot
updateCallCount int
}
func (m *mockCustodyUpdater) UpdateEarliestAvailableSlot(earliestAvailableSlot primitives.Slot) error {
m.updateCallCount++
m.earliestAvailableSlot = earliestAvailableSlot
return nil
}
func TestPruner_UpdatesEarliestAvailableSlot(t *testing.T) {
params.SetupTestConfigCleanup(t)
config := params.BeaconConfig()
config.FuluForkEpoch = 0 // Enable Fulu from epoch 0
params.OverrideBeaconConfig(config)
logrus.SetLevel(logrus.DebugLevel)
hook := logTest.NewGlobal()
ctx, cancel := context.WithCancel(t.Context())
defer cancel()
beaconDB := dbtest.SetupDB(t)
retentionEpochs := primitives.Epoch(2)
slotTicker := &slottest.MockTicker{Channel: make(chan primitives.Slot)}
// Create mock custody updater
mockCustody := &mockCustodyUpdater{
custodyGroupCount: 4,
earliestAvailableSlot: 0,
}
// Create pruner with mock custody updater
p, err := New(
ctx,
beaconDB,
time.Now(),
nil,
nil,
mockCustody,
WithSlotTicker(slotTicker),
)
require.NoError(t, err)
p.ps = func(current primitives.Slot) primitives.Slot {
return current - primitives.Slot(retentionEpochs)*params.BeaconConfig().SlotsPerEpoch
}
// Save some blocks to be pruned
for i := primitives.Slot(1); i <= 32; i++ {
blk := util.NewBeaconBlock()
blk.Block.Slot = i
wsb, err := blocks.NewSignedBeaconBlock(blk)
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, wsb))
}
// Start pruner and trigger at slot 80 (middle of 3rd epoch)
go p.Start()
currentSlot := primitives.Slot(80)
slotTicker.Channel <- currentSlot
// Wait for pruning to complete
time.Sleep(100 * time.Millisecond)
// Check that UpdateEarliestAvailableSlot was called
assert.Equal(t, true, mockCustody.updateCallCount > 0, "UpdateEarliestAvailableSlot should have been called")
// The earliest available slot should be pruneUpto + 1
// pruneUpto = currentSlot - retentionEpochs*slotsPerEpoch = 80 - 2*32 = 16
// So earliest available slot should be 16 + 1 = 17
expectedEarliestSlot := primitives.Slot(17)
require.Equal(t, expectedEarliestSlot, mockCustody.earliestAvailableSlot, "Earliest available slot should be updated correctly")
require.Equal(t, uint64(4), mockCustody.custodyGroupCount, "Custody group count should be preserved")
// Verify that no error was logged
for _, entry := range hook.AllEntries() {
if entry.Level == logrus.ErrorLevel {
t.Errorf("Unexpected error log: %s", entry.Message)
}
}
require.NoError(t, p.Stop())
}
// Mock custody updater that returns an error for UpdateEarliestAvailableSlot
type mockCustodyUpdaterWithUpdateError struct {
updateCallCount int
}
func (m *mockCustodyUpdaterWithUpdateError) UpdateEarliestAvailableSlot(earliestAvailableSlot primitives.Slot) error {
m.updateCallCount++
return errors.New("failed to update earliest available slot")
}
func TestWithRetentionPeriod_EnforcesMinimum(t *testing.T) {
// Use minimal config for testing
params.SetupTestConfigCleanup(t)
config := params.MinimalSpecConfig()
params.OverrideBeaconConfig(config)
ctx := t.Context()
beaconDB := dbtest.SetupDB(t)
// Get the minimum required epochs (272 + 1 = 273 for minimal)
minRequiredEpochs := primitives.Epoch(params.BeaconConfig().MinEpochsForBlockRequests + 1)
// Use a slot that's guaranteed to be after the minimum retention period
currentSlot := primitives.Slot(minRequiredEpochs+100) * (params.BeaconConfig().SlotsPerEpoch)
tests := []struct {
name string
userRetentionEpochs primitives.Epoch
expectedPruneSlot primitives.Slot
description string
}{
{
name: "User value below minimum - should use minimum",
userRetentionEpochs: 2, // Way below minimum
expectedPruneSlot: currentSlot - primitives.Slot(minRequiredEpochs)*params.BeaconConfig().SlotsPerEpoch,
description: "Should use minimum when user value is too low",
},
{
name: "User value at minimum",
userRetentionEpochs: minRequiredEpochs,
expectedPruneSlot: currentSlot - primitives.Slot(minRequiredEpochs)*params.BeaconConfig().SlotsPerEpoch,
description: "Should use user value when at minimum",
},
{
name: "User value above minimum",
userRetentionEpochs: minRequiredEpochs + 10,
expectedPruneSlot: currentSlot - primitives.Slot(minRequiredEpochs+10)*params.BeaconConfig().SlotsPerEpoch,
description: "Should use user value when above minimum",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
hook := logTest.NewGlobal()
logrus.SetLevel(logrus.WarnLevel)
mockCustody := &mockCustodyUpdater{}
// Create pruner with retention period
p, err := New(
ctx,
beaconDB,
time.Now(),
nil,
nil,
mockCustody,
WithRetentionPeriod(tt.userRetentionEpochs),
)
require.NoError(t, err)
// Test the pruning calculation
pruneUptoSlot := p.ps(currentSlot)
// Verify the pruning slot
assert.Equal(t, tt.expectedPruneSlot, pruneUptoSlot, tt.description)
// Check if warning was logged when value was too low
if tt.userRetentionEpochs < minRequiredEpochs {
assert.LogsContain(t, hook, "Retention period too low, ignoring and using minimum required value")
}
})
}
}
func TestPruner_UpdateEarliestSlotError(t *testing.T) {
params.SetupTestConfigCleanup(t)
config := params.BeaconConfig()
config.FuluForkEpoch = 0 // Enable Fulu from epoch 0
params.OverrideBeaconConfig(config)
logrus.SetLevel(logrus.DebugLevel)
hook := logTest.NewGlobal()
ctx, cancel := context.WithCancel(t.Context())
defer cancel()
beaconDB := dbtest.SetupDB(t)
retentionEpochs := primitives.Epoch(2)
slotTicker := &slottest.MockTicker{Channel: make(chan primitives.Slot)}
// Create mock custody updater that returns an error for UpdateEarliestAvailableSlot
mockCustody := &mockCustodyUpdaterWithUpdateError{}
// Create pruner with mock custody updater
p, err := New(
ctx,
beaconDB,
time.Now(),
nil,
nil,
mockCustody,
WithSlotTicker(slotTicker),
)
require.NoError(t, err)
p.ps = func(current primitives.Slot) primitives.Slot {
return current - primitives.Slot(retentionEpochs)*params.BeaconConfig().SlotsPerEpoch
}
// Save some blocks to be pruned
for i := primitives.Slot(1); i <= 32; i++ {
blk := util.NewBeaconBlock()
blk.Block.Slot = i
wsb, err := blocks.NewSignedBeaconBlock(blk)
require.NoError(t, err)
require.NoError(t, beaconDB.SaveBlock(ctx, wsb))
}
// Start pruner and trigger at slot 80
go p.Start()
currentSlot := primitives.Slot(80)
slotTicker.Channel <- currentSlot
// Wait for pruning to complete
time.Sleep(100 * time.Millisecond)
// Should have called UpdateEarliestAvailableSlot
assert.Equal(t, 1, mockCustody.updateCallCount, "UpdateEarliestAvailableSlot should be called")
// Check that error was logged by the prune function
found := false
for _, entry := range hook.AllEntries() {
if entry.Level == logrus.ErrorLevel && entry.Message == "Failed to prune database" {
found = true
break
}
}
assert.Equal(t, true, found, "Should log error when UpdateEarliestAvailableSlot fails")
require.NoError(t, p.Stop())
}

View File

@@ -1108,6 +1108,7 @@ func (b *BeaconNode) registerPrunerService(cliCtx *cli.Context) error {
genesis,
initSyncWaiter(cliCtx.Context, b.initialSyncComplete),
backfillService.WaitForCompletion,
b.fetchP2P(),
opts...,
)
if err != nil {

View File

@@ -115,6 +115,57 @@ func (s *Service) UpdateCustodyInfo(earliestAvailableSlot primitives.Slot, custo
return earliestAvailableSlot, custodyGroupCount, nil
}
// UpdateEarliestAvailableSlot updates the earliest available slot.
//
// IMPORTANT: This function should only be called when Fulu is enabled. The caller is responsible
// for checking params.FuluEnabled() before calling this function.
func (s *Service) UpdateEarliestAvailableSlot(earliestAvailableSlot primitives.Slot) error {
s.custodyInfoLock.Lock()
defer s.custodyInfoLock.Unlock()
if s.custodyInfo == nil {
return errors.New("no custody info available")
}
currentSlot := slots.CurrentSlot(s.genesisTime)
currentEpoch := slots.ToEpoch(currentSlot)
// Allow decrease (for backfill scenarios)
if earliestAvailableSlot < s.custodyInfo.earliestAvailableSlot {
s.custodyInfo.earliestAvailableSlot = earliestAvailableSlot
return nil
}
// Prevent increase within the MIN_EPOCHS_FOR_BLOCK_REQUESTS period
// This ensures we don't voluntarily refuse to serve mandatory block data
// This check applies regardless of whether we're early or late in the chain
minEpochsForBlocks := primitives.Epoch(params.BeaconConfig().MinEpochsForBlockRequests)
// Calculate the minimum required epoch (or 0 if we're early in the chain)
minRequiredEpoch := primitives.Epoch(0)
if currentEpoch > minEpochsForBlocks {
minRequiredEpoch = currentEpoch - minEpochsForBlocks
}
// Convert to slot to ensure we compare at slot-level granularity, not epoch-level
// This prevents allowing increases to slots within minRequiredEpoch that are after its first slot
minRequiredSlot, err := slots.EpochStart(minRequiredEpoch)
if err != nil {
return errors.Wrap(err, "epoch start")
}
// Prevent any increase that would put earliest slot beyond the minimum required slot
if earliestAvailableSlot > s.custodyInfo.earliestAvailableSlot && earliestAvailableSlot > minRequiredSlot {
return errors.Errorf(
"cannot increase earliest available slot to %d (epoch %d) as it exceeds minimum required slot %d (epoch %d)",
earliestAvailableSlot, slots.ToEpoch(earliestAvailableSlot), minRequiredSlot, minRequiredEpoch,
)
}
s.custodyInfo.earliestAvailableSlot = earliestAvailableSlot
return nil
}
// CustodyGroupCountFromPeer retrieves custody group count from a peer.
// It first tries to get the custody group count from the peer's metadata,
// then falls back to the ENR value if the metadata is not available, then

View File

@@ -4,6 +4,7 @@ import (
"context"
"strings"
"testing"
"time"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/peers"
@@ -167,6 +168,148 @@ func TestUpdateCustodyInfo(t *testing.T) {
}
}
func TestUpdateEarliestAvailableSlot(t *testing.T) {
params.SetupTestConfigCleanup(t)
config := params.BeaconConfig()
config.FuluForkEpoch = 0 // Enable Fulu from epoch 0
params.OverrideBeaconConfig(config)
t.Run("Valid update", func(t *testing.T) {
const (
initialSlot primitives.Slot = 50
newSlot primitives.Slot = 100
groupCount uint64 = 5
)
// Set up a scenario where we're far enough in the chain that increasing to newSlot is valid
minEpochsForBlocks := primitives.Epoch(params.BeaconConfig().MinEpochsForBlockRequests)
currentEpoch := minEpochsForBlocks + 100 // Well beyond MIN_EPOCHS_FOR_BLOCK_REQUESTS
currentSlot := primitives.Slot(currentEpoch) * primitives.Slot(params.BeaconConfig().SlotsPerEpoch)
service := &Service{
// Set genesis time in the past so currentSlot is the "current" slot
genesisTime: time.Now().Add(-time.Duration(currentSlot) * time.Duration(params.BeaconConfig().SecondsPerSlot) * time.Second),
custodyInfo: &custodyInfo{
earliestAvailableSlot: initialSlot,
groupCount: groupCount,
},
}
err := service.UpdateEarliestAvailableSlot(newSlot)
require.NoError(t, err)
require.Equal(t, newSlot, service.custodyInfo.earliestAvailableSlot)
require.Equal(t, groupCount, service.custodyInfo.groupCount) // Should preserve group count
})
t.Run("Earlier slot - allowed for backfill", func(t *testing.T) {
const initialSlot primitives.Slot = 100
const earlierSlot primitives.Slot = 50
service := &Service{
genesisTime: time.Now(),
custodyInfo: &custodyInfo{
earliestAvailableSlot: initialSlot,
groupCount: 5,
},
}
err := service.UpdateEarliestAvailableSlot(earlierSlot)
require.NoError(t, err)
require.Equal(t, earlierSlot, service.custodyInfo.earliestAvailableSlot) // Should decrease for backfill
})
t.Run("Prevent increase within MIN_EPOCHS_FOR_BLOCK_REQUESTS - late in chain", func(t *testing.T) {
// Set current time far enough in the future to have a meaningful MIN_EPOCHS_FOR_BLOCK_REQUESTS period
minEpochsForBlocks := primitives.Epoch(params.BeaconConfig().MinEpochsForBlockRequests)
currentEpoch := minEpochsForBlocks + 100 // Well beyond the minimum
currentSlot := primitives.Slot(currentEpoch) * primitives.Slot(params.BeaconConfig().SlotsPerEpoch)
// Calculate the minimum allowed epoch
minRequiredEpoch := currentEpoch - minEpochsForBlocks
minRequiredSlot := primitives.Slot(minRequiredEpoch) * primitives.Slot(params.BeaconConfig().SlotsPerEpoch)
// Try to set earliest slot to a value within the MIN_EPOCHS_FOR_BLOCK_REQUESTS period (should fail)
attemptedSlot := minRequiredSlot + 1000 // Within the mandatory retention period
service := &Service{
genesisTime: time.Now().Add(-time.Duration(currentSlot) * time.Duration(params.BeaconConfig().SecondsPerSlot) * time.Second),
custodyInfo: &custodyInfo{
earliestAvailableSlot: minRequiredSlot - 100, // Current value is before the min required
groupCount: 5,
},
}
err := service.UpdateEarliestAvailableSlot(attemptedSlot)
require.NotNil(t, err)
require.Equal(t, true, strings.Contains(err.Error(), "cannot increase earliest available slot"))
})
t.Run("Prevent increase at epoch boundary - slot precision matters", func(t *testing.T) {
minEpochsForBlocks := primitives.Epoch(params.BeaconConfig().MinEpochsForBlockRequests)
currentEpoch := minEpochsForBlocks + 976 // Current epoch
currentSlot := primitives.Slot(currentEpoch) * primitives.Slot(params.BeaconConfig().SlotsPerEpoch)
minRequiredEpoch := currentEpoch - minEpochsForBlocks // = 976
storedEarliestSlot := primitives.Slot(minRequiredEpoch)*primitives.Slot(params.BeaconConfig().SlotsPerEpoch) - 232 // Before minRequired
// Try to set earliest to slot 8 of the minRequiredEpoch (should fail with slot comparison)
attemptedSlot := primitives.Slot(minRequiredEpoch)*primitives.Slot(params.BeaconConfig().SlotsPerEpoch) + 8
service := &Service{
genesisTime: time.Now().Add(-time.Duration(currentSlot) * time.Duration(params.BeaconConfig().SecondsPerSlot) * time.Second),
custodyInfo: &custodyInfo{
earliestAvailableSlot: storedEarliestSlot,
groupCount: 5,
},
}
err := service.UpdateEarliestAvailableSlot(attemptedSlot)
require.NotNil(t, err, "Should prevent increasing earliest slot beyond the minimum required SLOT (not just epoch)")
require.Equal(t, true, strings.Contains(err.Error(), "cannot increase earliest available slot"))
})
t.Run("Prevent increase within MIN_EPOCHS_FOR_BLOCK_REQUESTS - early in chain", func(t *testing.T) {
minEpochsForBlocks := primitives.Epoch(params.BeaconConfig().MinEpochsForBlockRequests)
currentEpoch := minEpochsForBlocks - 10 // Early in chain, BEFORE we have MIN_EPOCHS_FOR_BLOCK_REQUESTS of history
currentSlot := primitives.Slot(currentEpoch) * primitives.Slot(params.BeaconConfig().SlotsPerEpoch)
// Current earliest slot is at slot 100
currentEarliestSlot := primitives.Slot(100)
// Try to increase earliest slot to slot 1000 (which would be within the mandatory window from currentSlot)
attemptedSlot := primitives.Slot(1000)
service := &Service{
genesisTime: time.Now().Add(-time.Duration(currentSlot) * time.Duration(params.BeaconConfig().SecondsPerSlot) * time.Second),
custodyInfo: &custodyInfo{
earliestAvailableSlot: currentEarliestSlot,
groupCount: 5,
},
}
err := service.UpdateEarliestAvailableSlot(attemptedSlot)
require.NotNil(t, err, "Should prevent increasing earliest slot within the mandatory retention window, even early in chain")
require.Equal(t, true, strings.Contains(err.Error(), "cannot increase earliest available slot"))
})
t.Run("Nil custody info - should return error", func(t *testing.T) {
service := &Service{
genesisTime: time.Now(),
custodyInfo: nil, // No custody info set
}
err := service.UpdateEarliestAvailableSlot(100)
require.NotNil(t, err)
require.Equal(t, true, strings.Contains(err.Error(), "no custody info available"))
})
}
func TestCustodyGroupCountFromPeer(t *testing.T) {
const (
expectedENR uint64 = 7

View File

@@ -126,6 +126,7 @@ type (
EarliestAvailableSlot(ctx context.Context) (primitives.Slot, error)
CustodyGroupCount(ctx context.Context) (uint64, error)
UpdateCustodyInfo(earliestAvailableSlot primitives.Slot, custodyGroupCount uint64) (primitives.Slot, uint64, error)
UpdateEarliestAvailableSlot(earliestAvailableSlot primitives.Slot) error
CustodyGroupCountFromPeer(peer.ID) uint64
}
)

View File

@@ -213,6 +213,11 @@ func (s *FakeP2P) UpdateCustodyInfo(earliestAvailableSlot primitives.Slot, custo
return earliestAvailableSlot, custodyGroupCount, nil
}
// UpdateEarliestAvailableSlot -- fake.
func (*FakeP2P) UpdateEarliestAvailableSlot(earliestAvailableSlot primitives.Slot) error {
return nil
}
// CustodyGroupCountFromPeer -- fake.
func (*FakeP2P) CustodyGroupCountFromPeer(peer.ID) uint64 {
return 0

View File

@@ -499,6 +499,15 @@ func (s *TestP2P) UpdateCustodyInfo(earliestAvailableSlot primitives.Slot, custo
return s.earliestAvailableSlot, s.custodyGroupCount, nil
}
// UpdateEarliestAvailableSlot .
func (s *TestP2P) UpdateEarliestAvailableSlot(earliestAvailableSlot primitives.Slot) error {
s.custodyInfoMut.Lock()
defer s.custodyInfoMut.Unlock()
s.earliestAvailableSlot = earliestAvailableSlot
return nil
}
// CustodyGroupCountFromPeer retrieves custody group count from a peer.
// It first tries to get the custody group count from the peer's metadata,
// then falls back to the ENR value if the metadata is not available, then

View File

@@ -97,7 +97,7 @@ func (s *Service) endpoints(
endpoints = append(endpoints, s.beaconEndpoints(ch, stater, blocker, validatorServer, coreService)...)
endpoints = append(endpoints, s.configEndpoints()...)
endpoints = append(endpoints, s.eventsEndpoints()...)
endpoints = append(endpoints, s.prysmBeaconEndpoints(ch, stater, coreService)...)
endpoints = append(endpoints, s.prysmBeaconEndpoints(ch, stater, blocker, coreService)...)
endpoints = append(endpoints, s.prysmNodeEndpoints()...)
endpoints = append(endpoints, s.prysmValidatorEndpoints(stater, coreService)...)
@@ -1184,6 +1184,7 @@ func (s *Service) eventsEndpoints() []endpoint {
func (s *Service) prysmBeaconEndpoints(
ch *stategen.CanonicalHistory,
stater lookup.Stater,
blocker lookup.Blocker,
coreService *core.Service,
) []endpoint {
server := &beaconprysm.Server{
@@ -1194,6 +1195,7 @@ func (s *Service) prysmBeaconEndpoints(
CanonicalHistory: ch,
BeaconDB: s.cfg.BeaconDB,
Stater: stater,
Blocker: blocker,
ChainInfoFetcher: s.cfg.ChainInfoFetcher,
FinalizationFetcher: s.cfg.FinalizationFetcher,
CoreService: coreService,
@@ -1266,6 +1268,28 @@ func (s *Service) prysmBeaconEndpoints(
handler: server.PublishBlobs,
methods: []string{http.MethodPost},
},
{
template: "/prysm/v1/beacon/states/{state_id}/query",
name: namespace + ".QueryBeaconState",
middleware: []middleware.Middleware{
middleware.ContentTypeHandler([]string{api.JsonMediaType}),
middleware.AcceptHeaderHandler([]string{api.OctetStreamMediaType}),
middleware.AcceptEncodingHeaderHandler(),
},
handler: server.QueryBeaconState,
methods: []string{http.MethodPost},
},
{
template: "/prysm/v1/beacon/blocks/{block_id}/query",
name: namespace + ".QueryBeaconBlock",
middleware: []middleware.Middleware{
middleware.ContentTypeHandler([]string{api.JsonMediaType}),
middleware.AcceptHeaderHandler([]string{api.OctetStreamMediaType}),
middleware.AcceptEncodingHeaderHandler(),
},
handler: server.QueryBeaconBlock,
methods: []string{http.MethodPost},
},
}
}

View File

@@ -127,6 +127,8 @@ func Test_endpoints(t *testing.T) {
"/prysm/v1/beacon/states/{state_id}/validator_count": {http.MethodGet},
"/prysm/v1/beacon/chain_head": {http.MethodGet},
"/prysm/v1/beacon/blobs": {http.MethodPost},
"/prysm/v1/beacon/states/{state_id}/query": {http.MethodPost},
"/prysm/v1/beacon/blocks/{block_id}/query": {http.MethodPost},
}
prysmNodeRoutes := map[string][]string{

View File

@@ -5,11 +5,13 @@ go_library(
srcs = [
"handlers.go",
"server.go",
"ssz_query.go",
"validator_count.go",
],
importpath = "github.com/OffchainLabs/prysm/v6/beacon-chain/rpc/prysm/beacon",
visibility = ["//visibility:public"],
deps = [
"//api:go_default_library",
"//api/server/structs:go_default_library",
"//beacon-chain/blockchain:go_default_library",
"//beacon-chain/core/helpers:go_default_library",
@@ -27,10 +29,13 @@ go_library(
"//consensus-types/primitives:go_default_library",
"//consensus-types/validator:go_default_library",
"//encoding/bytesutil:go_default_library",
"//encoding/ssz/query:go_default_library",
"//monitoring/tracing/trace:go_default_library",
"//network/httputil:go_default_library",
"//proto/eth/v1:go_default_library",
"//proto/prysm/v1alpha1:go_default_library",
"//proto/ssz_query:go_default_library",
"//runtime/version:go_default_library",
"//time/slots:go_default_library",
"@com_github_ethereum_go_ethereum//common/hexutil:go_default_library",
"@com_github_pkg_errors//:go_default_library",
@@ -41,10 +46,12 @@ go_test(
name = "go_default_test",
srcs = [
"handlers_test.go",
"ssz_query_test.go",
"validator_count_test.go",
],
embed = [":go_default_library"],
deps = [
"//api:go_default_library",
"//api/server/structs:go_default_library",
"//beacon-chain/blockchain/testing:go_default_library",
"//beacon-chain/core/helpers:go_default_library",
@@ -63,10 +70,13 @@ go_test(
"//config/fieldparams:go_default_library",
"//config/params:go_default_library",
"//consensus-types/blocks:go_default_library",
"//consensus-types/interfaces:go_default_library",
"//consensus-types/primitives:go_default_library",
"//encoding/bytesutil:go_default_library",
"//network/httputil:go_default_library",
"//proto/prysm/v1alpha1:go_default_library",
"//proto/ssz_query:go_default_library",
"//runtime/version:go_default_library",
"//testing/assert:go_default_library",
"//testing/require:go_default_library",
"//testing/util:go_default_library",

View File

@@ -18,6 +18,7 @@ type Server struct {
CanonicalHistory *stategen.CanonicalHistory
BeaconDB beacondb.ReadOnlyDatabase
Stater lookup.Stater
Blocker lookup.Blocker
ChainInfoFetcher blockchain.ChainInfoFetcher
FinalizationFetcher blockchain.FinalizationFetcher
CoreService *core.Service

View File

@@ -0,0 +1,202 @@
package beacon
import (
"encoding/json"
"errors"
"io"
"net/http"
"github.com/OffchainLabs/prysm/v6/api"
"github.com/OffchainLabs/prysm/v6/api/server/structs"
"github.com/OffchainLabs/prysm/v6/beacon-chain/rpc/eth/shared"
"github.com/OffchainLabs/prysm/v6/beacon-chain/rpc/lookup"
"github.com/OffchainLabs/prysm/v6/encoding/ssz/query"
"github.com/OffchainLabs/prysm/v6/monitoring/tracing/trace"
"github.com/OffchainLabs/prysm/v6/network/httputil"
sszquerypb "github.com/OffchainLabs/prysm/v6/proto/ssz_query"
"github.com/OffchainLabs/prysm/v6/runtime/version"
)
// QueryBeaconState handles SSZ Query request for BeaconState.
// Returns as bytes serialized SSZQueryResponse.
func (s *Server) QueryBeaconState(w http.ResponseWriter, r *http.Request) {
ctx, span := trace.StartSpan(r.Context(), "beacon.QueryBeaconState")
defer span.End()
stateID := r.PathValue("state_id")
if stateID == "" {
httputil.HandleError(w, "state_id is required in URL params", http.StatusBadRequest)
return
}
// Validate path before lookup: it might be expensive.
var req structs.SSZQueryRequest
err := json.NewDecoder(r.Body).Decode(&req)
switch {
case errors.Is(err, io.EOF):
httputil.HandleError(w, "No data submitted", http.StatusBadRequest)
return
case err != nil:
httputil.HandleError(w, "Could not decode request body: "+err.Error(), http.StatusBadRequest)
return
}
if len(req.Query) == 0 {
httputil.HandleError(w, "Empty query submitted", http.StatusBadRequest)
return
}
path, err := query.ParsePath(req.Query)
if err != nil {
httputil.HandleError(w, "Could not parse path '"+req.Query+"': "+err.Error(), http.StatusBadRequest)
return
}
stateRoot, err := s.Stater.StateRoot(ctx, []byte(stateID))
if err != nil {
var rootNotFoundErr *lookup.StateRootNotFoundError
if errors.As(err, &rootNotFoundErr) {
httputil.HandleError(w, "State root not found: "+rootNotFoundErr.Error(), http.StatusNotFound)
return
}
httputil.HandleError(w, "Could not get state root: "+err.Error(), http.StatusInternalServerError)
return
}
st, err := s.Stater.State(ctx, []byte(stateID))
if err != nil {
shared.WriteStateFetchError(w, err)
return
}
// NOTE: Using unsafe conversion to proto is acceptable here,
// as we play with a copy of the state returned by Stater.
sszObject, ok := st.ToProtoUnsafe().(query.SSZObject)
if !ok {
httputil.HandleError(w, "Unsupported state version for querying: "+version.String(st.Version()), http.StatusBadRequest)
return
}
info, err := query.AnalyzeObject(sszObject)
if err != nil {
httputil.HandleError(w, "Could not analyze state object: "+err.Error(), http.StatusInternalServerError)
return
}
_, offset, length, err := query.CalculateOffsetAndLength(info, path)
if err != nil {
httputil.HandleError(w, "Could not calculate offset and length for path '"+req.Query+"': "+err.Error(), http.StatusInternalServerError)
return
}
encodedState, err := st.MarshalSSZ()
if err != nil {
httputil.HandleError(w, "Could not marshal state to SSZ: "+err.Error(), http.StatusInternalServerError)
return
}
response := &sszquerypb.SSZQueryResponse{
Root: stateRoot,
Result: encodedState[offset : offset+length],
}
responseSsz, err := response.MarshalSSZ()
if err != nil {
httputil.HandleError(w, "Could not marshal response to SSZ: "+err.Error(), http.StatusInternalServerError)
return
}
w.Header().Set(api.VersionHeader, version.String(st.Version()))
httputil.WriteSsz(w, responseSsz)
}
// QueryBeaconState handles SSZ Query request for BeaconState.
// Returns as bytes serialized SSZQueryResponse.
func (s *Server) QueryBeaconBlock(w http.ResponseWriter, r *http.Request) {
ctx, span := trace.StartSpan(r.Context(), "beacon.QueryBeaconBlock")
defer span.End()
blockId := r.PathValue("block_id")
if blockId == "" {
httputil.HandleError(w, "block_id is required in URL params", http.StatusBadRequest)
return
}
// Validate path before lookup: it might be expensive.
var req structs.SSZQueryRequest
err := json.NewDecoder(r.Body).Decode(&req)
switch {
case errors.Is(err, io.EOF):
httputil.HandleError(w, "No data submitted", http.StatusBadRequest)
return
case err != nil:
httputil.HandleError(w, "Could not decode request body: "+err.Error(), http.StatusBadRequest)
return
}
if len(req.Query) == 0 {
httputil.HandleError(w, "Empty query submitted", http.StatusBadRequest)
return
}
path, err := query.ParsePath(req.Query)
if err != nil {
httputil.HandleError(w, "Could not parse path '"+req.Query+"': "+err.Error(), http.StatusBadRequest)
return
}
signedBlock, err := s.Blocker.Block(ctx, []byte(blockId))
if !shared.WriteBlockFetchError(w, signedBlock, err) {
return
}
protoBlock, err := signedBlock.Block().Proto()
if err != nil {
httputil.HandleError(w, "Could not convert block to proto: "+err.Error(), http.StatusInternalServerError)
return
}
block, ok := protoBlock.(query.SSZObject)
if !ok {
httputil.HandleError(w, "Unsupported block version for querying: "+version.String(signedBlock.Version()), http.StatusBadRequest)
return
}
info, err := query.AnalyzeObject(block)
if err != nil {
httputil.HandleError(w, "Could not analyze block object: "+err.Error(), http.StatusInternalServerError)
return
}
_, offset, length, err := query.CalculateOffsetAndLength(info, path)
if err != nil {
httputil.HandleError(w, "Could not calculate offset and length for path '"+req.Query+"': "+err.Error(), http.StatusInternalServerError)
return
}
encodedBlock, err := signedBlock.Block().MarshalSSZ()
if err != nil {
httputil.HandleError(w, "Could not marshal block to SSZ: "+err.Error(), http.StatusInternalServerError)
return
}
blockRoot, err := block.HashTreeRoot()
if err != nil {
httputil.HandleError(w, "Could not compute block root: "+err.Error(), http.StatusInternalServerError)
return
}
response := &sszquerypb.SSZQueryResponse{
Root: blockRoot[:],
Result: encodedBlock[offset : offset+length],
}
responseSsz, err := response.MarshalSSZ()
if err != nil {
httputil.HandleError(w, "Could not marshal response to SSZ: "+err.Error(), http.StatusInternalServerError)
return
}
w.Header().Set(api.VersionHeader, version.String(signedBlock.Version()))
httputil.WriteSsz(w, responseSsz)
}

View File

@@ -0,0 +1,335 @@
package beacon
import (
"bytes"
"context"
"encoding/binary"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/OffchainLabs/prysm/v6/api"
"github.com/OffchainLabs/prysm/v6/api/server/structs"
chainMock "github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/testing"
"github.com/OffchainLabs/prysm/v6/beacon-chain/rpc/testutil"
"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
eth "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
sszquerypb "github.com/OffchainLabs/prysm/v6/proto/ssz_query"
"github.com/OffchainLabs/prysm/v6/runtime/version"
"github.com/OffchainLabs/prysm/v6/testing/assert"
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/OffchainLabs/prysm/v6/testing/util"
"github.com/ethereum/go-ethereum/common/hexutil"
"github.com/prysmaticlabs/go-bitfield"
)
func TestQueryBeaconState(t *testing.T) {
ctx := context.Background()
st, _ := util.DeterministicGenesisState(t, 16)
require.NoError(t, st.SetSlot(primitives.Slot(42)))
stateRoot, err := st.HashTreeRoot(ctx)
require.NoError(t, err)
require.NoError(t, st.UpdateBalancesAtIndex(0, 42000000000))
tests := []struct {
path string
expectedValue []byte
}{
{
path: ".slot",
expectedValue: func() []byte {
slot := st.Slot()
result, _ := slot.MarshalSSZ()
return result
}(),
},
{
path: ".latest_block_header",
expectedValue: func() []byte {
header := st.LatestBlockHeader()
result, _ := header.MarshalSSZ()
return result
}(),
},
{
path: ".validators",
expectedValue: func() []byte {
b := make([]byte, 0)
validators := st.Validators()
for _, v := range validators {
vBytes, _ := v.MarshalSSZ()
b = append(b, vBytes...)
}
return b
}(),
},
{
path: ".validators[0]",
expectedValue: func() []byte {
v, _ := st.ValidatorAtIndex(0)
result, _ := v.MarshalSSZ()
return result
}(),
},
{
path: ".validators[0].withdrawal_credentials",
expectedValue: func() []byte {
v, _ := st.ValidatorAtIndex(0)
return v.WithdrawalCredentials
}(),
},
{
path: ".validators[0].effective_balance",
expectedValue: func() []byte {
v, _ := st.ValidatorAtIndex(0)
b := make([]byte, 8)
binary.LittleEndian.PutUint64(b, uint64(v.EffectiveBalance))
return b
}(),
},
}
for _, tt := range tests {
t.Run(tt.path, func(t *testing.T) {
chainService := &chainMock.ChainService{Optimistic: false, FinalizedRoots: make(map[[32]byte]bool)}
s := &Server{
OptimisticModeFetcher: chainService,
FinalizationFetcher: chainService,
Stater: &testutil.MockStater{
BeaconStateRoot: stateRoot[:],
BeaconState: st,
},
}
requestBody := &structs.SSZQueryRequest{
Query: tt.path,
}
var buf bytes.Buffer
require.NoError(t, json.NewEncoder(&buf).Encode(requestBody))
request := httptest.NewRequest(http.MethodPost, "http://example.com/prysm/v1/beacon/states/{state_id}/query", &buf)
request.SetPathValue("state_id", "head")
writer := httptest.NewRecorder()
writer.Body = &bytes.Buffer{}
s.QueryBeaconState(writer, request)
require.Equal(t, http.StatusOK, writer.Code)
assert.Equal(t, version.String(version.Phase0), writer.Header().Get(api.VersionHeader))
expectedResponse := &sszquerypb.SSZQueryResponse{
Root: stateRoot[:],
Result: tt.expectedValue,
}
sszExpectedResponse, err := expectedResponse.MarshalSSZ()
require.NoError(t, err)
assert.DeepEqual(t, sszExpectedResponse, writer.Body.Bytes())
})
}
}
func TestQueryBeaconStateInvalidRequest(t *testing.T) {
ctx := context.Background()
st, _ := util.DeterministicGenesisState(t, 16)
require.NoError(t, st.SetSlot(primitives.Slot(42)))
stateRoot, err := st.HashTreeRoot(ctx)
require.NoError(t, err)
tests := []struct {
name string
stateId string
path string
code int
errorString string
}{
{
name: "empty query submitted",
stateId: "head",
path: "",
errorString: "Empty query submitted",
},
{
name: "invalid path",
stateId: "head",
path: ".invalid[]]",
errorString: "Could not parse path",
},
{
name: "non-existent field",
stateId: "head",
path: ".non_existent_field",
code: http.StatusInternalServerError,
errorString: "Could not calculate offset and length for path",
},
{
name: "empty state ID",
stateId: "",
path: "",
},
{
name: "far future slot",
stateId: "1000000000000",
path: "",
},
}
for _, tt := range tests {
t.Run(tt.path, func(t *testing.T) {
chainService := &chainMock.ChainService{Optimistic: false, FinalizedRoots: make(map[[32]byte]bool)}
s := &Server{
OptimisticModeFetcher: chainService,
FinalizationFetcher: chainService,
Stater: &testutil.MockStater{
BeaconStateRoot: stateRoot[:],
BeaconState: st,
},
}
requestBody := &structs.SSZQueryRequest{
Query: tt.path,
}
var buf bytes.Buffer
require.NoError(t, json.NewEncoder(&buf).Encode(requestBody))
request := httptest.NewRequest(http.MethodPost, "http://example.com/prysm/v1/beacon/states/{state_id}/query", &buf)
request.SetPathValue("state_id", tt.stateId)
writer := httptest.NewRecorder()
writer.Body = &bytes.Buffer{}
s.QueryBeaconState(writer, request)
if tt.code == 0 {
tt.code = http.StatusBadRequest
}
require.Equal(t, tt.code, writer.Code)
if tt.errorString != "" {
errorString := writer.Body.String()
require.Equal(t, true, strings.Contains(errorString, tt.errorString))
}
})
}
}
func TestQueryBeaconBlock(t *testing.T) {
randaoReveal, err := hexutil.Decode("0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505")
require.NoError(t, err)
root, err := hexutil.Decode("0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2")
require.NoError(t, err)
signature, err := hexutil.Decode("0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505")
require.NoError(t, err)
att := &eth.Attestation{
AggregationBits: bitfield.Bitlist{0x01},
Data: &eth.AttestationData{
Slot: 1,
CommitteeIndex: 1,
BeaconBlockRoot: root,
Source: &eth.Checkpoint{
Epoch: 1,
Root: root,
},
Target: &eth.Checkpoint{
Epoch: 1,
Root: root,
},
},
Signature: signature,
}
tests := []struct {
name string
path string
block interfaces.ReadOnlySignedBeaconBlock
expectedValue []byte
}{
{
name: "slot",
path: ".slot",
block: func() interfaces.ReadOnlySignedBeaconBlock {
b := util.NewBeaconBlock()
b.Block.Slot = 123
sb, err := blocks.NewSignedBeaconBlock(b)
require.NoError(t, err)
return sb
}(),
expectedValue: func() []byte {
b := make([]byte, 8)
binary.LittleEndian.PutUint64(b, 123)
return b
}(),
},
{
name: "randao_reveal",
path: ".body.randao_reveal",
block: func() interfaces.ReadOnlySignedBeaconBlock {
b := util.NewBeaconBlock()
b.Block.Body.RandaoReveal = randaoReveal
sb, err := blocks.NewSignedBeaconBlock(b)
require.NoError(t, err)
return sb
}(),
expectedValue: randaoReveal,
},
{
name: "attestations",
path: ".body.attestations",
block: func() interfaces.ReadOnlySignedBeaconBlock {
b := util.NewBeaconBlock()
b.Block.Body.Attestations = []*eth.Attestation{
att,
}
sb, err := blocks.NewSignedBeaconBlock(b)
require.NoError(t, err)
return sb
}(),
expectedValue: func() []byte {
b, err := att.MarshalSSZ()
require.NoError(t, err)
return b
}(),
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
mockBlockFetcher := &testutil.MockBlocker{BlockToReturn: tt.block}
mockChainService := &chainMock.ChainService{
FinalizedRoots: map[[32]byte]bool{},
}
s := &Server{
FinalizationFetcher: mockChainService,
Blocker: mockBlockFetcher,
}
requestBody := &structs.SSZQueryRequest{
Query: tt.path,
}
var buf bytes.Buffer
require.NoError(t, json.NewEncoder(&buf).Encode(requestBody))
request := httptest.NewRequest(http.MethodPost, "http://example.com/prysm/v1/beacon/blocks/{block_id}/query", &buf)
request.SetPathValue("block_id", "head")
writer := httptest.NewRecorder()
writer.Body = &bytes.Buffer{}
s.QueryBeaconBlock(writer, request)
require.Equal(t, http.StatusOK, writer.Code)
assert.Equal(t, version.String(version.Phase0), writer.Header().Get(api.VersionHeader))
blockRoot, err := tt.block.Block().HashTreeRoot()
require.NoError(t, err)
expectedResponse := &sszquerypb.SSZQueryResponse{
Root: blockRoot[:],
Result: tt.expectedValue,
}
sszExpectedResponse, err := expectedResponse.MarshalSSZ()
require.NoError(t, err)
assert.DeepEqual(t, sszExpectedResponse, writer.Body.Bytes())
})
}
}

View File

@@ -229,7 +229,7 @@ func (vs *Server) BuildBlockParallel(ctx context.Context, sBlk interfaces.Signed
sBlk.SetVoluntaryExits(vs.getExits(head, sBlk.Block().Slot()))
// Set sync aggregate. New in Altair.
vs.setSyncAggregate(ctx, sBlk)
vs.setSyncAggregate(ctx, sBlk, head)
// Set bls to execution change. New in Capella.
vs.setBlsToExecData(sBlk, head)

View File

@@ -5,6 +5,7 @@ import (
"context"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
"github.com/OffchainLabs/prysm/v6/beacon-chain/state"
"github.com/OffchainLabs/prysm/v6/config/params"
"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
@@ -20,12 +21,12 @@ import (
"github.com/prysmaticlabs/go-bitfield"
)
func (vs *Server) setSyncAggregate(ctx context.Context, blk interfaces.SignedBeaconBlock) {
func (vs *Server) setSyncAggregate(ctx context.Context, blk interfaces.SignedBeaconBlock, headState state.BeaconState) {
if blk.Version() < version.Altair {
return
}
syncAggregate, err := vs.getSyncAggregate(ctx, slots.PrevSlot(blk.Block().Slot()), blk.Block().ParentRoot())
syncAggregate, err := vs.getSyncAggregate(ctx, slots.PrevSlot(blk.Block().Slot()), blk.Block().ParentRoot(), headState)
if err != nil {
log.WithError(err).Error("Could not get sync aggregate")
emptySig := [96]byte{0xC0}
@@ -47,7 +48,7 @@ func (vs *Server) setSyncAggregate(ctx context.Context, blk interfaces.SignedBea
// getSyncAggregate retrieves the sync contributions from the pool to construct the sync aggregate object.
// The contributions are filtered based on matching of the input root and slot then profitability.
func (vs *Server) getSyncAggregate(ctx context.Context, slot primitives.Slot, root [32]byte) (*ethpb.SyncAggregate, error) {
func (vs *Server) getSyncAggregate(ctx context.Context, slot primitives.Slot, root [32]byte, headState state.BeaconState) (*ethpb.SyncAggregate, error) {
_, span := trace.StartSpan(ctx, "ProposerServer.getSyncAggregate")
defer span.End()
@@ -62,7 +63,7 @@ func (vs *Server) getSyncAggregate(ctx context.Context, slot primitives.Slot, ro
// Contributions have to match the input root
proposerContributions := proposerSyncContributions(poolContributions).filterByBlockRoot(root)
aggregatedContributions, err := vs.aggregatedSyncCommitteeMessages(ctx, slot, root, poolContributions)
aggregatedContributions, err := vs.aggregatedSyncCommitteeMessages(ctx, slot, root, poolContributions, headState)
if err != nil {
return nil, errors.Wrap(err, "could not get aggregated sync committee messages")
}
@@ -123,6 +124,7 @@ func (vs *Server) aggregatedSyncCommitteeMessages(
slot primitives.Slot,
root [32]byte,
poolContributions []*ethpb.SyncCommitteeContribution,
st state.BeaconState,
) ([]*ethpb.SyncCommitteeContribution, error) {
subcommitteeCount := params.BeaconConfig().SyncCommitteeSubnetCount
subcommitteeSize := params.BeaconConfig().SyncCommitteeSize / subcommitteeCount
@@ -146,10 +148,7 @@ func (vs *Server) aggregatedSyncCommitteeMessages(
messageSigs = append(messageSigs, msg.Signature)
}
}
st, err := vs.HeadFetcher.HeadState(ctx)
if err != nil {
return nil, errors.Wrap(err, "could not get head state")
}
positions, err := helpers.CurrentPeriodPositions(st, messageIndices)
if err != nil {
return nil, errors.Wrap(err, "could not get sync committee positions")

View File

@@ -9,6 +9,7 @@ import (
mockSync "github.com/OffchainLabs/prysm/v6/beacon-chain/sync/initial-sync/testing"
"github.com/OffchainLabs/prysm/v6/config/params"
"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
"github.com/OffchainLabs/prysm/v6/crypto/bls"
"github.com/OffchainLabs/prysm/v6/encoding/bytesutil"
ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
@@ -51,15 +52,15 @@ func TestProposer_GetSyncAggregate_OK(t *testing.T) {
require.NoError(t, proposerServer.SyncCommitteePool.SaveSyncCommitteeContribution(cont))
}
aggregate, err := proposerServer.getSyncAggregate(t.Context(), 1, bytesutil.ToBytes32(conts[0].BlockRoot))
aggregate, err := proposerServer.getSyncAggregate(t.Context(), 1, bytesutil.ToBytes32(conts[0].BlockRoot), st)
require.NoError(t, err)
require.DeepEqual(t, bitfield.Bitvector32{0xf, 0xf, 0xf, 0xf}, aggregate.SyncCommitteeBits)
aggregate, err = proposerServer.getSyncAggregate(t.Context(), 2, bytesutil.ToBytes32(conts[0].BlockRoot))
aggregate, err = proposerServer.getSyncAggregate(t.Context(), 2, bytesutil.ToBytes32(conts[0].BlockRoot), st)
require.NoError(t, err)
require.DeepEqual(t, bitfield.Bitvector32{0xaa, 0xaa, 0xaa, 0xaa}, aggregate.SyncCommitteeBits)
aggregate, err = proposerServer.getSyncAggregate(t.Context(), 3, bytesutil.ToBytes32(conts[0].BlockRoot))
aggregate, err = proposerServer.getSyncAggregate(t.Context(), 3, bytesutil.ToBytes32(conts[0].BlockRoot), st)
require.NoError(t, err)
require.DeepEqual(t, bitfield.NewBitvector32(), aggregate.SyncCommitteeBits)
}
@@ -68,7 +69,7 @@ func TestServer_SetSyncAggregate_EmptyCase(t *testing.T) {
b, err := blocks.NewSignedBeaconBlock(util.NewBeaconBlockAltair())
require.NoError(t, err)
s := &Server{} // Sever is not initialized with sync committee pool.
s.setSyncAggregate(t.Context(), b)
s.setSyncAggregate(t.Context(), b, nil)
agg, err := b.Block().Body().SyncAggregate()
require.NoError(t, err)
@@ -138,7 +139,7 @@ func TestProposer_GetSyncAggregate_IncludesSyncCommitteeMessages(t *testing.T) {
}
// The final sync aggregates must have indexes [0,1,2,3] set for both subcommittees
sa, err := proposerServer.getSyncAggregate(t.Context(), 1, r)
sa, err := proposerServer.getSyncAggregate(t.Context(), 1, r, st)
require.NoError(t, err)
assert.Equal(t, true, sa.SyncCommitteeBits.BitAt(0))
assert.Equal(t, true, sa.SyncCommitteeBits.BitAt(1))
@@ -194,8 +195,99 @@ func Test_aggregatedSyncCommitteeMessages_NoIntersectionWithPoolContributions(t
BlockRoot: r[:],
}
aggregated, err := proposerServer.aggregatedSyncCommitteeMessages(t.Context(), 1, r, []*ethpb.SyncCommitteeContribution{cont})
aggregated, err := proposerServer.aggregatedSyncCommitteeMessages(t.Context(), 1, r, []*ethpb.SyncCommitteeContribution{cont}, st)
require.NoError(t, err)
require.Equal(t, 1, len(aggregated))
assert.Equal(t, false, aggregated[0].AggregationBits.BitAt(3))
}
func TestGetSyncAggregate_CorrectStateAtSyncCommitteePeriodBoundary(t *testing.T) {
helpers.ClearCache()
syncPeriodBoundaryEpoch := primitives.Epoch(274176) // Real epoch from the bug report
slotsPerEpoch := params.BeaconConfig().SlotsPerEpoch
preEpochState, keys := util.DeterministicGenesisStateAltair(t, 100)
require.NoError(t, preEpochState.SetSlot(primitives.Slot(syncPeriodBoundaryEpoch)*slotsPerEpoch-1)) // Last slot of previous epoch
postEpochState := preEpochState.Copy()
require.NoError(t, postEpochState.SetSlot(primitives.Slot(syncPeriodBoundaryEpoch)*slotsPerEpoch+2)) // After 2 missed slots
oldCommittee := &ethpb.SyncCommittee{
Pubkeys: make([][]byte, params.BeaconConfig().SyncCommitteeSize),
}
newCommittee := &ethpb.SyncCommittee{
Pubkeys: make([][]byte, params.BeaconConfig().SyncCommitteeSize),
}
for i := 0; i < int(params.BeaconConfig().SyncCommitteeSize); i++ {
if i < len(keys) {
oldCommittee.Pubkeys[i] = keys[i%len(keys)].PublicKey().Marshal()
// Use different keys for new committee to simulate rotation
newCommittee.Pubkeys[i] = keys[(i+10)%len(keys)].PublicKey().Marshal()
}
}
require.NoError(t, preEpochState.SetCurrentSyncCommittee(oldCommittee))
require.NoError(t, postEpochState.SetCurrentSyncCommittee(newCommittee))
mockChainService := &chainmock.ChainService{
State: postEpochState,
}
proposerServer := &Server{
HeadFetcher: mockChainService,
SyncChecker: &mockSync.Sync{IsSyncing: false},
SyncCommitteePool: synccommittee.NewStore(),
}
slot := primitives.Slot(syncPeriodBoundaryEpoch)*slotsPerEpoch + 1 // First slot of new epoch
blockRoot := [32]byte{0x01, 0x02, 0x03}
msg1 := &ethpb.SyncCommitteeMessage{
Slot: slot,
BlockRoot: blockRoot[:],
ValidatorIndex: 0, // This validator is in position 0 of OLD committee
Signature: bls.NewAggregateSignature().Marshal(),
}
msg2 := &ethpb.SyncCommitteeMessage{
Slot: slot,
BlockRoot: blockRoot[:],
ValidatorIndex: 1, // This validator is in position 1 of OLD committee
Signature: bls.NewAggregateSignature().Marshal(),
}
require.NoError(t, proposerServer.SyncCommitteePool.SaveSyncCommitteeMessage(msg1))
require.NoError(t, proposerServer.SyncCommitteePool.SaveSyncCommitteeMessage(msg2))
aggregateWrongState, err := proposerServer.getSyncAggregate(t.Context(), slot, blockRoot, postEpochState)
require.NoError(t, err)
aggregateCorrectState, err := proposerServer.getSyncAggregate(t.Context(), slot, blockRoot, preEpochState)
require.NoError(t, err)
wrongStateBits := bitfield.Bitlist(aggregateWrongState.SyncCommitteeBits)
correctStateBits := bitfield.Bitlist(aggregateCorrectState.SyncCommitteeBits)
wrongStateHasValidators := false
correctStateHasValidators := false
for i := 0; i < len(wrongStateBits); i++ {
if wrongStateBits[i] != 0 {
wrongStateHasValidators = true
break
}
}
for i := 0; i < len(correctStateBits); i++ {
if correctStateBits[i] != 0 {
correctStateHasValidators = true
break
}
}
assert.Equal(t, true, correctStateHasValidators, "Correct state should include validators that sent messages")
assert.Equal(t, false, wrongStateHasValidators, "Wrong state should not find validators in incorrect sync committee")
t.Logf("Wrong state aggregate bits: %x (has validators: %v)", wrongStateBits, wrongStateHasValidators)
t.Logf("Correct state aggregate bits: %x (has validators: %v)", correctStateBits, correctStateHasValidators)
}

View File

@@ -266,6 +266,8 @@ type WriteOnlyEth1Data interface {
SetEth1DepositIndex(val uint64) error
ExitEpochAndUpdateChurn(exitBalance primitives.Gwei) (primitives.Epoch, error)
ExitEpochAndUpdateChurnForTotalBal(totalActiveBalance primitives.Gwei, exitBalance primitives.Gwei) (primitives.Epoch, error)
SetExitBalanceToConsume(val primitives.Gwei) error
SetEarliestExitEpoch(val primitives.Epoch) error
}
// WriteOnlyValidators defines a struct which only has write access to validators methods.
@@ -333,6 +335,7 @@ type WriteOnlyWithdrawals interface {
DequeuePendingPartialWithdrawals(num uint64) error
SetNextWithdrawalIndex(i uint64) error
SetNextWithdrawalValidatorIndex(i primitives.ValidatorIndex) error
SetPendingPartialWithdrawals(val []*ethpb.PendingPartialWithdrawal) error
}
type WriteOnlyConsolidations interface {

View File

@@ -91,3 +91,33 @@ func (b *BeaconState) exitEpochAndUpdateChurn(totalActiveBalance primitives.Gwei
return b.earliestExitEpoch, nil
}
// SetExitBalanceToConsume sets the exit balance to consume. This method mutates the state.
func (b *BeaconState) SetExitBalanceToConsume(exitBalanceToConsume primitives.Gwei) error {
if b.version < version.Electra {
return errNotSupported("SetExitBalanceToConsume", b.version)
}
b.lock.Lock()
defer b.lock.Unlock()
b.exitBalanceToConsume = exitBalanceToConsume
b.markFieldAsDirty(types.ExitBalanceToConsume)
return nil
}
// SetEarliestExitEpoch sets the earliest exit epoch. This method mutates the state.
func (b *BeaconState) SetEarliestExitEpoch(earliestExitEpoch primitives.Epoch) error {
if b.version < version.Electra {
return errNotSupported("SetEarliestExitEpoch", b.version)
}
b.lock.Lock()
defer b.lock.Unlock()
b.earliestExitEpoch = earliestExitEpoch
b.markFieldAsDirty(types.EarliestExitEpoch)
return nil
}

View File

@@ -100,3 +100,24 @@ func (b *BeaconState) DequeuePendingPartialWithdrawals(n uint64) error {
return nil
}
// SetPendingPartialWithdrawals sets the pending partial withdrawals. This method mutates the state.
func (b *BeaconState) SetPendingPartialWithdrawals(pendingPartialWithdrawals []*eth.PendingPartialWithdrawal) error {
if b.version < version.Electra {
return errNotSupported("SetPendingPartialWithdrawals", b.version)
}
b.lock.Lock()
defer b.lock.Unlock()
if pendingPartialWithdrawals == nil {
return errors.New("cannot set nil pending partial withdrawals")
}
b.sharedFieldReferences[types.PendingPartialWithdrawals].MinusRef()
b.sharedFieldReferences[types.PendingPartialWithdrawals] = stateutil.NewRef(1)
b.pendingPartialWithdrawals = pendingPartialWithdrawals
b.markFieldAsDirty(types.PendingPartialWithdrawals)
return nil
}

View File

@@ -650,6 +650,11 @@ func InitializeFromProtoUnsafeFulu(st *ethpb.BeaconStateFulu) (state.BeaconState
for i, v := range st.ProposerLookahead {
proposerLookahead[i] = primitives.ValidatorIndex(v)
}
// Proposer lookahead must be exactly 2 * SLOTS_PER_EPOCH in length. We fill in with zeroes instead of erroring out here
for i := len(proposerLookahead); i < 2*fieldparams.SlotsPerEpoch; i++ {
proposerLookahead = append(proposerLookahead, 0)
}
fieldCount := params.BeaconConfig().BeaconStateFuluFieldCount
b := &BeaconState{
version: version.Fulu,

View File

@@ -17,7 +17,6 @@ go_library(
importpath = "github.com/OffchainLabs/prysm/v6/beacon-chain/sync/backfill",
visibility = ["//visibility:public"],
deps = [
"//beacon-chain/core/helpers:go_default_library",
"//beacon-chain/core/signing:go_default_library",
"//beacon-chain/das:go_default_library",
"//beacon-chain/db:go_default_library",
@@ -61,7 +60,6 @@ go_test(
],
embed = [":go_default_library"],
deps = [
"//beacon-chain/core/helpers:go_default_library",
"//beacon-chain/core/signing:go_default_library",
"//beacon-chain/das:go_default_library",
"//beacon-chain/db:go_default_library",

View File

@@ -3,12 +3,12 @@ package backfill
import (
"context"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
"github.com/OffchainLabs/prysm/v6/beacon-chain/db/filesystem"
"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p"
"github.com/OffchainLabs/prysm/v6/beacon-chain/startup"
"github.com/OffchainLabs/prysm/v6/beacon-chain/sync"
"github.com/OffchainLabs/prysm/v6/beacon-chain/verification"
"github.com/OffchainLabs/prysm/v6/config/params"
"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
"github.com/OffchainLabs/prysm/v6/encoding/bytesutil"
@@ -348,7 +348,7 @@ func (*Service) Status() error {
// minimumBackfillSlot determines the lowest slot that backfill needs to download based on looking back
// MIN_EPOCHS_FOR_BLOCK_REQUESTS from the current slot.
func minimumBackfillSlot(current primitives.Slot) primitives.Slot {
oe := helpers.MinEpochsForBlockRequests()
oe := primitives.Epoch(params.BeaconConfig().MinEpochsForBlockRequests)
if oe > slots.MaxSafeEpoch() {
oe = slots.MaxSafeEpoch()
}

View File

@@ -5,7 +5,6 @@ import (
"testing"
"time"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
"github.com/OffchainLabs/prysm/v6/beacon-chain/db/filesystem"
p2ptest "github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/testing"
"github.com/OffchainLabs/prysm/v6/beacon-chain/startup"
@@ -84,7 +83,7 @@ func TestServiceInit(t *testing.T) {
}
func TestMinimumBackfillSlot(t *testing.T) {
oe := helpers.MinEpochsForBlockRequests()
oe := primitives.Epoch(params.BeaconConfig().MinEpochsForBlockRequests)
currSlot := (oe + 100).Mul(uint64(params.BeaconConfig().SlotsPerEpoch))
minSlot := minimumBackfillSlot(primitives.Slot(currSlot))
@@ -109,7 +108,7 @@ func testReadN(ctx context.Context, t *testing.T, c chan batch, n int, into []ba
}
func TestBackfillMinSlotDefault(t *testing.T) {
oe := helpers.MinEpochsForBlockRequests()
oe := primitives.Epoch(params.BeaconConfig().MinEpochsForBlockRequests)
current := primitives.Slot((oe + 100).Mul(uint64(params.BeaconConfig().SlotsPerEpoch)))
s := &Service{}
specMin := minimumBackfillSlot(current)

View File

@@ -1,3 +0,0 @@
### Fixed
- Use service context and continue on slasher attestation errors (#15803).

View File

@@ -0,0 +1,3 @@
### Fixed
- Fix recoverStateSummary to persist state summaries in stateSummaryBucket instead of stateBucket (#15896).

View File

@@ -1,3 +0,0 @@
### Added
- Delegate sszInfo HashTreeRoot to FastSSZ-generated implementations via SSZObject, enabling roots calculation for generated types while avoiding duplicate logic.

View File

@@ -1,3 +0,0 @@
### Ignored
- Small code changes for reusability and readability to processAggregate.

View File

@@ -1,3 +0,0 @@
### Fixed
- block event probably shouldn't be sent on certain block processing failures, now sends only on successing processing Block is NON-CANONICAL, Block IS CANONICAL but getFCUArgs FAILS, and Full success

View File

@@ -1,7 +0,0 @@
### Fixed
- Fixed web3signer e2e, issues caused due to a regression on old fork support
### Changed
- updated web3signer to 25.9.1

View File

@@ -0,0 +1,3 @@
### Ignored
- Changelog entries for v6.1.3 through v6.1.2

View File

@@ -1,2 +0,0 @@
### Fixed
- Decreased attestation gossip validation batch deadline to 5ms.

View File

@@ -1,2 +0,0 @@
### Ignored
- Data column sidecars fetch: Adjust log levels.

View File

@@ -1,2 +0,0 @@
### Fixed
- Fix `/eth/v1/beacon/blob_sidecars/` beacon API is the fulu fork epoch is set to the far future epoch.

View File

@@ -1,2 +0,0 @@
### Fixed
- `VerifyDataColumnSidecar`: Check if there is no too many commitments.

View File

@@ -1,2 +0,0 @@
### Fixed
- `WithDataColumnRetentionEpochs`: Use `dataColumnRetentionEpoch` instead of `blobColumnRetentionEpoch`.

View File

@@ -1,2 +0,0 @@
### Fixed
- `dataColumnSidecarsByRangeRPCHandler`: Gracefully close the stream if no data to return.

View File

@@ -1,2 +0,0 @@
### Added
- Add Grandine to P2P known agents. (Useful for metrics)

View File

@@ -1,2 +0,0 @@
### Fixed
- `HasAtLeastOneIndex`: Check the index is not too high.

View File

@@ -0,0 +1,2 @@
### Fixed
- `updateCustodyInfoInDB`: Use `NumberOfCustodyGroups` instead of `NumberOfColumns`.

View File

@@ -0,0 +1,2 @@
### Fixed
- `VerifyDataColumnsSidecarKZGProofs`: Check if sizes match.

View File

@@ -1,2 +0,0 @@
### Ignored
- Fix (unreleased) bug where the preallocated slice for KZG Proofs was 48x bigger than it needed to be.

View File

@@ -0,0 +1,2 @@
### Fixed
- Fixed metadata extraction on Windows by correctly splitting file paths

View File

@@ -1,3 +0,0 @@
### Fixed
- reject committee index >= committees_per_slot in unaggregated attestation validation

View File

@@ -1,3 +0,0 @@
### Fixed
- Mark epoch transition correctly on new head events

View File

@@ -0,0 +1,3 @@
### Added
- Add native state diff type and marshalling functions

View File

@@ -1,3 +0,0 @@
### Fixed
- Do not mark blocks as invalid from ErrNotDescendantOfFinalized

View File

@@ -1,3 +0,0 @@
### Fixed
- Mark the block as invalid if it has an invalid signature.

View File

@@ -1,3 +0,0 @@
### Ignored
- Remove redundant check for genesis root at startup.

View File

@@ -1,3 +0,0 @@
### Ignored
- Changelog entries for v6.1.2 through v6.0.5

View File

@@ -1,3 +0,0 @@
### Changed
- Improve returning individual message errors from Beacon API.

View File

@@ -1,3 +0,0 @@
### Fixed
- Display error messages from the server verbatim when they are not encoded as `application/json`.

View File

@@ -1,3 +0,0 @@
### Changed
- Do not verify block data when calculating rewards.

View File

@@ -1,3 +0,0 @@
### Fixed
- Fixed [#15812](https://github.com/OffchainLabs/prysm/issues/15812): Gossip attestation validation incorrectly rejecting attestations that arrive before their referenced blocks. Previously, attestations were saved to the pending queue but immediately rejected by forkchoice validation, causing "not descendant of finalized checkpoint" errors. Now attestations for missing blocks return `ValidationIgnore` without error, allowing them to be properly processed when their blocks arrive. This eliminates false positive rejections and prevents potential incorrect peer downscoring during network congestion.

View File

@@ -0,0 +1,3 @@
### Added
- Update the earliest available slot after pruning operations in beacon chain database pruner. This ensures the P2P layer accurately knows which historical data is available after pruning, preventing nodes from advertising or attempting to serve data that has been pruned.

View File

@@ -0,0 +1,3 @@
### Added
- SSZ-QL: Add endpoints for `BeaconState`/`BeaconBlock`.

View File

@@ -1,7 +0,0 @@
### Added
- SSZ-QL: Use `fastssz`'s `SizeSSZ` method for calculating the size of `Container` type.
### Changed
- SSZ-QL: Clarify `Size` method with more sophisticated `SSZType`s.

View File

@@ -1,3 +0,0 @@
### Added
- SSZ-QL: Access n-th element in `List`/`Vector`.

View File

@@ -0,0 +1,3 @@
### Fixed
- Sync committee uses correct state to calculate position

View File

@@ -1,3 +0,0 @@
### Changed
- Gracefully handle submit blind block returning 502 errors.

View File

@@ -1,3 +0,0 @@
### Changed
- Process pending attestations after pending blocks are cleared

View File

@@ -42,6 +42,12 @@ func NewWrappedExecutionData(v proto.Message) (interfaces.ExecutionData, error)
return WrappedExecutionPayloadDeneb(pbStruct.Payload)
case *enginev1.ExecutionBundleFulu:
return WrappedExecutionPayloadDeneb(pbStruct.Payload)
case *enginev1.ExecutionPayloadHeader:
return WrappedExecutionPayloadHeader(pbStruct)
case *enginev1.ExecutionPayloadHeaderCapella:
return WrappedExecutionPayloadHeaderCapella(pbStruct)
case *enginev1.ExecutionPayloadHeaderDeneb:
return WrappedExecutionPayloadHeaderDeneb(pbStruct)
default:
return nil, errors.Wrapf(ErrUnsupportedVersion, "type %T", pbStruct)
}

View File

@@ -0,0 +1,57 @@
load("@prysm//tools/go:def.bzl", "go_library", "go_test")
go_library(
name = "go_default_library",
srcs = ["state_diff.go"],
importpath = "github.com/OffchainLabs/prysm/v6/consensus-types/hdiff",
visibility = ["//visibility:public"],
deps = [
"//beacon-chain/core/altair:go_default_library",
"//beacon-chain/core/capella:go_default_library",
"//beacon-chain/core/deneb:go_default_library",
"//beacon-chain/core/electra:go_default_library",
"//beacon-chain/core/execution:go_default_library",
"//beacon-chain/core/fulu:go_default_library",
"//beacon-chain/state:go_default_library",
"//config/fieldparams:go_default_library",
"//consensus-types/blocks:go_default_library",
"//consensus-types/helpers:go_default_library",
"//consensus-types/interfaces:go_default_library",
"//consensus-types/primitives:go_default_library",
"//proto/engine/v1:go_default_library",
"//proto/prysm/v1alpha1:go_default_library",
"//runtime/version:go_default_library",
"@com_github_golang_snappy//:go_default_library",
"@com_github_pkg_errors//:go_default_library",
"@com_github_prysmaticlabs_fastssz//:go_default_library",
"@com_github_prysmaticlabs_go_bitfield//:go_default_library",
"@com_github_sirupsen_logrus//:go_default_library",
"@org_golang_google_protobuf//proto:go_default_library",
],
)
go_test(
name = "go_default_test",
srcs = [
"fuzz_test.go",
"property_test.go",
"security_test.go",
"state_diff_test.go",
],
data = glob(["testdata/**"]),
embed = [":go_default_library"],
deps = [
"//beacon-chain/core/transition:go_default_library",
"//beacon-chain/state:go_default_library",
"//beacon-chain/state/state-native:go_default_library",
"//config/fieldparams:go_default_library",
"//consensus-types/blocks:go_default_library",
"//consensus-types/primitives:go_default_library",
"//proto/prysm/v1alpha1:go_default_library",
"//runtime/version:go_default_library",
"//testing/require:go_default_library",
"//testing/util:go_default_library",
"@com_github_golang_snappy//:go_default_library",
"@com_github_pkg_errors//:go_default_library",
],
)

Binary file not shown.

After

Width:  |  Height:  |  Size: 383 KiB

View File

@@ -0,0 +1,636 @@
package hdiff
import (
"context"
"encoding/binary"
"strconv"
"strings"
"testing"
"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
"github.com/OffchainLabs/prysm/v6/testing/util"
)
const maxFuzzValidators = 10000
const maxFuzzStateDiffSize = 1000
const maxFuzzHistoricalRoots = 10000
const maxFuzzDecodedSize = maxFuzzStateDiffSize * 10
const maxFuzzScanRange = 200
const fuzzRootsLengthOffset = 16
const maxFuzzInputSize = 10
const oneEthInGwei = 1000000000
// FuzzNewHdiff tests parsing variations of realistic diffs
func FuzzNewHdiff(f *testing.F) {
// Add seed corpus with various valid diffs from realistic scenarios
sizes := []uint64{8, 16, 32}
for _, size := range sizes {
source, _ := util.DeterministicGenesisStateElectra(f, size)
// Create various realistic target states
scenarios := []string{"slot_change", "balance_change", "validator_change", "multiple_changes"}
for _, scenario := range scenarios {
target := source.Copy()
switch scenario {
case "slot_change":
_ = target.SetSlot(source.Slot() + 1)
case "balance_change":
balances := target.Balances()
if len(balances) > 0 {
balances[0] += 1000000000
_ = target.SetBalances(balances)
}
case "validator_change":
validators := target.Validators()
if len(validators) > 0 {
validators[0].EffectiveBalance += 1000000000
_ = target.SetValidators(validators)
}
case "multiple_changes":
_ = target.SetSlot(source.Slot() + 5)
balances := target.Balances()
validators := target.Validators()
if len(balances) > 0 && len(validators) > 0 {
balances[0] += 2000000000
validators[0].EffectiveBalance += 1000000000
_ = target.SetBalances(balances)
_ = target.SetValidators(validators)
}
}
validDiff, err := Diff(source, target)
if err == nil {
f.Add(validDiff.StateDiff, validDiff.ValidatorDiffs, validDiff.BalancesDiff)
}
}
}
f.Fuzz(func(t *testing.T, stateDiff, validatorDiffs, balancesDiff []byte) {
// Limit input sizes to reasonable bounds
if len(stateDiff) > 5000 || len(validatorDiffs) > 5000 || len(balancesDiff) > 5000 {
return
}
// Bound historical roots length in stateDiff (if it contains snappy-compressed data)
// The historicalRootsLength is read after snappy decompression, but we can still
// limit the compressed input size to prevent extreme decompression ratios
if len(stateDiff) > maxFuzzStateDiffSize {
// Limit stateDiff to prevent potential memory bombs from snappy decompression
stateDiff = stateDiff[:maxFuzzStateDiffSize]
}
// Bound validator count in validatorDiffs
if len(validatorDiffs) >= 8 {
count := binary.LittleEndian.Uint64(validatorDiffs[0:8])
if count >= maxFuzzValidators {
boundedCount := count % maxFuzzValidators
binary.LittleEndian.PutUint64(validatorDiffs[0:8], boundedCount)
}
}
// Bound balance count in balancesDiff
if len(balancesDiff) >= 8 {
count := binary.LittleEndian.Uint64(balancesDiff[0:8])
if count >= maxFuzzValidators {
boundedCount := count % maxFuzzValidators
binary.LittleEndian.PutUint64(balancesDiff[0:8], boundedCount)
}
}
input := HdiffBytes{
StateDiff: stateDiff,
ValidatorDiffs: validatorDiffs,
BalancesDiff: balancesDiff,
}
// Test parsing - should not panic even with corrupted but bounded data
_, err := newHdiff(input)
_ = err // Expected to fail with corrupted data
})
}
// FuzzNewStateDiff tests the newStateDiff function with valid random state diffs
func FuzzNewStateDiff(f *testing.F) {
f.Fuzz(func(t *testing.T, validatorCount uint8, slotDelta uint64, balanceData []byte, validatorData []byte) {
defer func() {
if r := recover(); r != nil {
t.Errorf("newStateDiff panicked: %v", r)
}
}()
// Bound validator count to reasonable range
validators := uint64(validatorCount%32 + 8) // 8-39 validators
if slotDelta > 100 {
slotDelta = slotDelta % 100
}
// Generate random source state
source, _ := util.DeterministicGenesisStateElectra(t, validators)
target := source.Copy()
// Apply random slot change
_ = target.SetSlot(source.Slot() + primitives.Slot(slotDelta))
// Apply random balance changes
if len(balanceData) >= 8 {
balances := target.Balances()
numChanges := int(binary.LittleEndian.Uint64(balanceData[:8])) % len(balances)
for i := 0; i < numChanges && i*8+8 < len(balanceData); i++ {
idx := i % len(balances)
delta := int64(binary.LittleEndian.Uint64(balanceData[i*8+8:(i+1)*8+8]))
// Keep delta reasonable
delta = delta % oneEthInGwei // Max 1 ETH change
if delta < 0 && uint64(-delta) > balances[idx] {
balances[idx] = 0
} else if delta < 0 {
balances[idx] -= uint64(-delta)
} else {
balances[idx] += uint64(delta)
}
}
_ = target.SetBalances(balances)
}
// Apply random validator changes
if len(validatorData) > 0 {
validators := target.Validators()
numChanges := int(validatorData[0]) % len(validators)
for i := 0; i < numChanges && i < len(validatorData)-1; i++ {
idx := i % len(validators)
if validatorData[i+1]%2 == 0 {
validators[idx].EffectiveBalance += oneEthInGwei // 1 ETH
}
}
_ = target.SetValidators(validators)
}
// Create diff between source and target
diff, err := Diff(source, target)
if err != nil {
return // Skip if diff creation fails
}
// Test newStateDiff with the valid serialized diff from StateDiff field
reconstructed, err := newStateDiff(diff.StateDiff)
if err != nil {
t.Errorf("newStateDiff failed on valid diff: %v", err)
return
}
// Basic validation that reconstruction worked
if reconstructed == nil {
t.Error("newStateDiff returned nil without error")
}
})
}
// FuzzNewValidatorDiffs tests validator diff deserialization with valid diffs
func FuzzNewValidatorDiffs(f *testing.F) {
f.Fuzz(func(t *testing.T, validatorCount uint8, changeData []byte) {
defer func() {
if r := recover(); r != nil {
t.Errorf("newValidatorDiffs panicked: %v", r)
}
}()
// Bound validator count to reasonable range
validators := uint64(validatorCount%16 + 4) // 4-19 validators
// Generate random source state
source, _ := util.DeterministicGenesisStateElectra(t, validators)
target := source.Copy()
// Apply random validator changes based on changeData
if len(changeData) > 0 {
vals := target.Validators()
numChanges := int(changeData[0]) % len(vals)
for i := 0; i < numChanges && i < len(changeData)-1; i++ {
idx := i % len(vals)
changeType := changeData[i+1] % 4
switch changeType {
case 0: // Change effective balance
vals[idx].EffectiveBalance += oneEthInGwei
case 1: // Toggle slashed status
vals[idx].Slashed = !vals[idx].Slashed
case 2: // Change activation epoch
vals[idx].ActivationEpoch++
case 3: // Change exit epoch
vals[idx].ExitEpoch++
}
}
_ = target.SetValidators(vals)
}
// Create diff between source and target
diff, err := Diff(source, target)
if err != nil {
return // Skip if diff creation fails
}
// Test newValidatorDiffs with the valid serialized diff
reconstructed, err := newValidatorDiffs(diff.ValidatorDiffs)
if err != nil {
t.Errorf("newValidatorDiffs failed on valid diff: %v", err)
return
}
// Basic validation that reconstruction worked
if reconstructed == nil {
t.Error("newValidatorDiffs returned nil without error")
}
})
}
// FuzzNewBalancesDiff tests balance diff deserialization with valid diffs
func FuzzNewBalancesDiff(f *testing.F) {
f.Fuzz(func(t *testing.T, balanceCount uint8, balanceData []byte) {
defer func() {
if r := recover(); r != nil {
t.Errorf("newBalancesDiff panicked: %v", r)
}
}()
// Bound balance count to reasonable range
numBalances := int(balanceCount%32 + 8) // 8-39 balances
// Generate simple source state
source, _ := util.DeterministicGenesisStateElectra(t, uint64(numBalances))
target := source.Copy()
// Apply random balance changes based on balanceData
if len(balanceData) >= 8 {
balances := target.Balances()
numChanges := int(binary.LittleEndian.Uint64(balanceData[:8])) % numBalances
for i := 0; i < numChanges && i*8+8 < len(balanceData); i++ {
idx := i % numBalances
delta := int64(binary.LittleEndian.Uint64(balanceData[i*8+8:(i+1)*8+8]))
// Keep delta reasonable
delta = delta % oneEthInGwei // Max 1 ETH change
if delta < 0 && uint64(-delta) > balances[idx] {
balances[idx] = 0
} else if delta < 0 {
balances[idx] -= uint64(-delta)
} else {
balances[idx] += uint64(delta)
}
}
_ = target.SetBalances(balances)
}
// Create diff between source and target to get BalancesDiff
diff, err := Diff(source, target)
if err != nil {
return // Skip if diff creation fails
}
// Test newBalancesDiff with the valid serialized diff
reconstructed, err := newBalancesDiff(diff.BalancesDiff)
if err != nil {
t.Errorf("newBalancesDiff failed on valid diff: %v", err)
return
}
// Basic validation that reconstruction worked
if reconstructed == nil {
t.Error("newBalancesDiff returned nil without error")
}
})
}
// FuzzApplyDiff tests applying variations of valid diffs
func FuzzApplyDiff(f *testing.F) {
// Test with realistic state variations, not random data
ctx := context.Background()
// Add seed corpus with various valid scenarios
sizes := []uint64{8, 16, 32, 64}
for _, size := range sizes {
source, _ := util.DeterministicGenesisStateElectra(f, size)
target := source.Copy()
// Different types of realistic changes
scenarios := []func(){
func() { _ = target.SetSlot(source.Slot() + 1) }, // Slot change
func() { // Balance change
balances := target.Balances()
if len(balances) > 0 {
balances[0] += 1000000000 // 1 ETH
_ = target.SetBalances(balances)
}
},
func() { // Validator change
validators := target.Validators()
if len(validators) > 0 {
validators[0].EffectiveBalance += 1000000000
_ = target.SetValidators(validators)
}
},
}
for _, scenario := range scenarios {
testTarget := source.Copy()
scenario()
validDiff, err := Diff(source, testTarget)
if err == nil {
f.Add(validDiff.StateDiff, validDiff.ValidatorDiffs, validDiff.BalancesDiff)
}
}
}
f.Fuzz(func(t *testing.T, stateDiff, validatorDiffs, balancesDiff []byte) {
// Only test with reasonable sized inputs
if len(stateDiff) > 10000 || len(validatorDiffs) > 10000 || len(balancesDiff) > 10000 {
return
}
// Bound historical roots length in stateDiff (same as FuzzNewHdiff)
if len(stateDiff) > maxFuzzStateDiffSize {
stateDiff = stateDiff[:maxFuzzStateDiffSize]
}
// Bound validator count in validatorDiffs
if len(validatorDiffs) >= 8 {
count := binary.LittleEndian.Uint64(validatorDiffs[0:8])
if count >= maxFuzzValidators {
boundedCount := count % maxFuzzValidators
binary.LittleEndian.PutUint64(validatorDiffs[0:8], boundedCount)
}
}
// Bound balance count in balancesDiff
if len(balancesDiff) >= 8 {
count := binary.LittleEndian.Uint64(balancesDiff[0:8])
if count >= maxFuzzValidators {
boundedCount := count % maxFuzzValidators
binary.LittleEndian.PutUint64(balancesDiff[0:8], boundedCount)
}
}
// Create fresh source state for each test
source, _ := util.DeterministicGenesisStateElectra(t, 8)
diff := HdiffBytes{
StateDiff: stateDiff,
ValidatorDiffs: validatorDiffs,
BalancesDiff: balancesDiff,
}
// Apply diff - errors are expected for fuzzed data
_, err := ApplyDiff(ctx, source, diff)
_ = err // Expected to fail with invalid data
})
}
// FuzzReadPendingAttestation tests the pending attestation deserialization
func FuzzReadPendingAttestation(f *testing.F) {
// Add edge cases - this function is particularly vulnerable
f.Add([]byte{})
f.Add([]byte{0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08}) // 8 bytes
f.Add(make([]byte, 200)) // Larger than expected
// Add a case with large reported length
largeLength := make([]byte, 8)
binary.LittleEndian.PutUint64(largeLength, 0xFFFFFFFF) // Large bits length
f.Add(largeLength)
f.Fuzz(func(t *testing.T, data []byte) {
defer func() {
if r := recover(); r != nil {
t.Errorf("readPendingAttestation panicked: %v", r)
}
}()
// Make a copy since the function modifies the slice
dataCopy := make([]byte, len(data))
copy(dataCopy, data)
// Bound the bits length by modifying the first 8 bytes if they exist
if len(dataCopy) >= 8 {
// Read the bits length and bound it to maxFuzzValidators
bitsLength := binary.LittleEndian.Uint64(dataCopy[0:8])
if bitsLength >= maxFuzzValidators {
boundedLength := bitsLength % maxFuzzValidators
binary.LittleEndian.PutUint64(dataCopy[0:8], boundedLength)
}
}
_, err := readPendingAttestation(&dataCopy)
_ = err
})
}
// FuzzKmpIndex tests the KMP algorithm implementation
func FuzzKmpIndex(f *testing.F) {
// Test with integer pointers to match the actual usage
f.Add("1,2,3", "4,5,6")
f.Add("1,2,3", "1,2,3")
f.Add("", "1,2,3")
f.Add("1,1,1", "2,2,2")
f.Fuzz(func(t *testing.T, sourceStr string, targetStr string) {
defer func() {
if r := recover(); r != nil {
t.Errorf("kmpIndex panicked: %v", r)
}
}()
// Parse comma-separated strings into int slices
var source, target []int
if sourceStr != "" {
for _, s := range strings.Split(sourceStr, ",") {
if val, err := strconv.Atoi(strings.TrimSpace(s)); err == nil {
source = append(source, val)
}
}
}
if targetStr != "" {
for _, s := range strings.Split(targetStr, ",") {
if val, err := strconv.Atoi(strings.TrimSpace(s)); err == nil {
target = append(target, val)
}
}
}
// Maintain the precondition: concatenate target with source
// This matches how kmpIndex is actually called in production
combined := make([]int, len(target)+len(source))
copy(combined, target)
copy(combined[len(target):], source)
// Convert to pointer slices as used in actual code
combinedPtrs := make([]*int, len(combined))
for i := range combined {
val := combined[i]
combinedPtrs[i] = &val
}
integerEquals := func(a, b *int) bool {
if a == nil && b == nil {
return true
}
if a == nil || b == nil {
return false
}
return *a == *b
}
result := kmpIndex(len(source), combinedPtrs, integerEquals)
// Basic sanity check: result should be in [0, len(source)]
if result < 0 || result > len(source) {
t.Errorf("kmpIndex returned invalid result: %d for source length=%d", result, len(source))
}
})
}
// FuzzComputeLPS tests the LPS computation for KMP
func FuzzComputeLPS(f *testing.F) {
// Add seed cases
f.Add("1,2,1")
f.Add("1,1,1")
f.Add("1,2,3,4")
f.Add("")
f.Fuzz(func(t *testing.T, patternStr string) {
defer func() {
if r := recover(); r != nil {
t.Errorf("computeLPS panicked: %v", r)
}
}()
// Parse comma-separated string into int slice
var pattern []int
if patternStr != "" {
for _, s := range strings.Split(patternStr, ",") {
if val, err := strconv.Atoi(strings.TrimSpace(s)); err == nil {
pattern = append(pattern, val)
}
}
}
// Convert to pointer slice
patternPtrs := make([]*int, len(pattern))
for i := range pattern {
val := pattern[i]
patternPtrs[i] = &val
}
integerEquals := func(a, b *int) bool {
if a == nil && b == nil {
return true
}
if a == nil || b == nil {
return false
}
return *a == *b
}
result := computeLPS(patternPtrs, integerEquals)
// Verify result length matches input
if len(result) != len(pattern) {
t.Errorf("computeLPS returned wrong length: got %d, expected %d", len(result), len(pattern))
}
// Verify all LPS values are non-negative and within bounds
for i, lps := range result {
if lps < 0 || lps > i {
t.Errorf("Invalid LPS value at index %d: %d", i, lps)
}
}
})
}
// FuzzDiffToBalances tests balance diff computation
func FuzzDiffToBalances(f *testing.F) {
f.Fuzz(func(t *testing.T, sourceData, targetData []byte) {
defer func() {
if r := recover(); r != nil {
t.Errorf("diffToBalances panicked: %v", r)
}
}()
// Convert byte data to balance arrays
var sourceBalances, targetBalances []uint64
// Parse source balances (8 bytes per uint64)
for i := 0; i+7 < len(sourceData) && len(sourceBalances) < 100; i += 8 {
balance := binary.LittleEndian.Uint64(sourceData[i : i+8])
sourceBalances = append(sourceBalances, balance)
}
// Parse target balances
for i := 0; i+7 < len(targetData) && len(targetBalances) < 100; i += 8 {
balance := binary.LittleEndian.Uint64(targetData[i : i+8])
targetBalances = append(targetBalances, balance)
}
// Create states with the provided balances
source, _ := util.DeterministicGenesisStateElectra(t, 1)
target, _ := util.DeterministicGenesisStateElectra(t, 1)
if len(sourceBalances) > 0 {
_ = source.SetBalances(sourceBalances)
}
if len(targetBalances) > 0 {
_ = target.SetBalances(targetBalances)
}
result, err := diffToBalances(source, target)
// If no error, verify result consistency
if err == nil && len(result) > 0 {
// Result length should match target length
if len(result) != len(target.Balances()) {
t.Errorf("diffToBalances result length mismatch: got %d, expected %d",
len(result), len(target.Balances()))
}
}
})
}
// FuzzValidatorsEqual tests validator comparison
func FuzzValidatorsEqual(f *testing.F) {
f.Fuzz(func(t *testing.T, data []byte) {
defer func() {
if r := recover(); r != nil {
t.Errorf("validatorsEqual panicked: %v", r)
}
}()
// Create two validators and fuzz their fields
if len(data) < 16 {
return
}
source, _ := util.DeterministicGenesisStateElectra(t, 2)
validators := source.Validators()
if len(validators) < 2 {
return
}
val1 := validators[0]
val2 := validators[1]
// Modify validator fields based on fuzz data
if len(data) > 0 && data[0]%2 == 0 {
val2.EffectiveBalance = val1.EffectiveBalance + uint64(data[0])
}
if len(data) > 1 && data[1]%2 == 0 {
val2.Slashed = !val1.Slashed
}
// Create ReadOnlyValidator wrappers if needed
// Since validatorsEqual expects ReadOnlyValidator interface,
// we'll skip this test for now as it requires state wrapper implementation
_ = val1
_ = val2
})
}

View File

@@ -0,0 +1,403 @@
package hdiff
import (
"encoding/binary"
"math"
"testing"
"time"
"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/OffchainLabs/prysm/v6/testing/util"
)
// maxSafeBalance ensures balances can be safely cast to int64 for diff computation
const maxSafeBalance = 1<<52 - 1
// PropertyTestRoundTrip verifies that diff->apply is idempotent with realistic data
func FuzzPropertyRoundTrip(f *testing.F) {
f.Fuzz(func(t *testing.T, slotDelta uint64, balanceData []byte, validatorData []byte) {
// Limit to realistic ranges
if slotDelta > 32 { // Max one epoch
slotDelta = slotDelta % 32
}
// Convert byte data to realistic deltas and changes
var balanceDeltas []int64
var validatorChanges []bool
// Parse balance deltas - limit to realistic amounts (8 bytes per int64)
for i := 0; i+7 < len(balanceData) && len(balanceDeltas) < 20; i += 8 {
delta := int64(binary.LittleEndian.Uint64(balanceData[i : i+8]))
// Keep deltas realistic (max 10 ETH change)
if delta > 10000000000 {
delta = delta % 10000000000
}
if delta < -10000000000 {
delta = -((-delta) % 10000000000)
}
balanceDeltas = append(balanceDeltas, delta)
}
// Parse validator changes (1 byte per bool) - limit to small number
for i := 0; i < len(validatorData) && len(validatorChanges) < 10; i++ {
validatorChanges = append(validatorChanges, validatorData[i]%2 == 0)
}
ctx := t.Context()
// Create source state with reasonable size
validatorCount := uint64(len(validatorChanges) + 8) // Minimum 8 validators
if validatorCount > 64 {
validatorCount = 64 // Cap at 64 for performance
}
source, _ := util.DeterministicGenesisStateElectra(t, validatorCount)
// Create target state with modifications
target := source.Copy()
// Apply slot change
_ = target.SetSlot(source.Slot() + primitives.Slot(slotDelta))
// Apply realistic balance changes
if len(balanceDeltas) > 0 {
balances := target.Balances()
for i, delta := range balanceDeltas {
if i >= len(balances) {
break
}
// Apply realistic balance changes with safe bounds
if delta < 0 {
if uint64(-delta) > balances[i] {
balances[i] = 0 // Can't go below 0
} else {
balances[i] -= uint64(-delta)
}
} else {
// Cap at reasonable maximum (1000 ETH)
maxBalance := uint64(1000000000000) // 1000 ETH in Gwei
if balances[i]+uint64(delta) > maxBalance {
balances[i] = maxBalance
} else {
balances[i] += uint64(delta)
}
}
}
_ = target.SetBalances(balances)
}
// Apply realistic validator changes
if len(validatorChanges) > 0 {
validators := target.Validators()
for i, shouldChange := range validatorChanges {
if i >= len(validators) {
break
}
if shouldChange {
// Make realistic changes - small effective balance adjustments
validators[i].EffectiveBalance += 1000000000 // 1 ETH
}
}
_ = target.SetValidators(validators)
}
// Create diff
diff, err := Diff(source, target)
if err != nil {
// If diff creation fails, that's acceptable for malformed inputs
return
}
// Apply diff
result, err := ApplyDiff(ctx, source, diff)
if err != nil {
// If diff application fails, that's acceptable
return
}
// Verify round-trip property: source + diff = target
require.Equal(t, target.Slot(), result.Slot())
// Verify balance consistency
targetBalances := target.Balances()
resultBalances := result.Balances()
require.Equal(t, len(targetBalances), len(resultBalances))
for i := range targetBalances {
require.Equal(t, targetBalances[i], resultBalances[i], "Balance mismatch at index %d", i)
}
// Verify validator consistency
targetVals := target.Validators()
resultVals := result.Validators()
require.Equal(t, len(targetVals), len(resultVals))
for i := range targetVals {
require.Equal(t, targetVals[i].Slashed, resultVals[i].Slashed, "Validator slashing mismatch at index %d", i)
require.Equal(t, targetVals[i].EffectiveBalance, resultVals[i].EffectiveBalance, "Validator balance mismatch at index %d", i)
}
})
}
// PropertyTestReasonablePerformance verifies operations complete quickly with realistic data
func FuzzPropertyResourceBounds(f *testing.F) {
f.Fuzz(func(t *testing.T, validatorCount uint8, slotDelta uint8, changeCount uint8) {
// Use realistic parameters
validators := uint64(validatorCount%64 + 8) // 8-71 validators
slots := uint64(slotDelta % 32) // 0-31 slots
changes := int(changeCount % 10) // 0-9 changes
// Create realistic states
source, _ := util.DeterministicGenesisStateElectra(t, validators)
target := source.Copy()
// Apply realistic changes
_ = target.SetSlot(source.Slot() + primitives.Slot(slots))
if changes > 0 {
validatorList := target.Validators()
for i := 0; i < changes && i < len(validatorList); i++ {
validatorList[i].EffectiveBalance += 1000000000 // 1 ETH
}
_ = target.SetValidators(validatorList)
}
// Operations should complete quickly
start := time.Now()
diff, err := Diff(source, target)
duration := time.Since(start)
if err == nil {
// Should be fast
require.Equal(t, true, duration < time.Second, "Diff creation too slow: %v", duration)
// Apply should also be fast
start = time.Now()
_, err = ApplyDiff(t.Context(), source, diff)
duration = time.Since(start)
if err == nil {
require.Equal(t, true, duration < time.Second, "Diff application too slow: %v", duration)
}
}
})
}
// PropertyTestDiffSize verifies that diffs are smaller than full states for typical cases
func FuzzPropertyDiffEfficiency(f *testing.F) {
f.Fuzz(func(t *testing.T, slotDelta uint64, numChanges uint8) {
if slotDelta > 100 {
slotDelta = slotDelta % 100
}
if numChanges > 10 {
numChanges = numChanges % 10
}
// Create states with small differences
source, _ := util.DeterministicGenesisStateElectra(t, 64)
target := source.Copy()
_ = target.SetSlot(source.Slot() + primitives.Slot(slotDelta))
// Make a few small changes
if numChanges > 0 {
validators := target.Validators()
for i := uint8(0); i < numChanges && int(i) < len(validators); i++ {
validators[i].EffectiveBalance += 1000
}
_ = target.SetValidators(validators)
}
// Create diff
diff, err := Diff(source, target)
if err != nil {
return
}
// For small changes, diff should be much smaller than full state
sourceSSZ, err := source.MarshalSSZ()
if err != nil {
return
}
diffSize := len(diff.StateDiff) + len(diff.ValidatorDiffs) + len(diff.BalancesDiff)
// Diff should be smaller than full state for small changes
if numChanges <= 5 && slotDelta <= 10 {
require.Equal(t, true, diffSize < len(sourceSSZ)/2,
"Diff size %d should be less than half of state size %d", diffSize, len(sourceSSZ))
}
})
}
// PropertyTestBalanceConservation verifies that balance operations don't create/destroy value unexpectedly
func FuzzPropertyBalanceConservation(f *testing.F) {
f.Fuzz(func(t *testing.T, balanceData []byte) {
// Convert byte data to balance changes, bounded to safe range
var balanceChanges []int64
for i := 0; i+7 < len(balanceData) && len(balanceChanges) < 50; i += 8 {
rawChange := int64(binary.LittleEndian.Uint64(balanceData[i : i+8]))
// Bound the change to ensure resulting balances stay within safe range
change := rawChange % (maxSafeBalance / 2) // Divide by 2 to allow for addition/subtraction
balanceChanges = append(balanceChanges, change)
}
source, _ := util.DeterministicGenesisStateElectra(t, uint64(len(balanceChanges)+10))
originalBalances := source.Balances()
// Ensure initial balances are within safe range for int64 casting
for i, balance := range originalBalances {
if balance > maxSafeBalance {
originalBalances[i] = balance % maxSafeBalance
}
}
_ = source.SetBalances(originalBalances)
// Calculate total before
var totalBefore uint64
for _, balance := range originalBalances {
totalBefore += balance
}
// Apply balance changes via diff system
target := source.Copy()
targetBalances := target.Balances()
var totalDelta int64
for i, delta := range balanceChanges {
if i >= len(targetBalances) {
break
}
// Prevent underflow
if delta < 0 && uint64(-delta) > targetBalances[i] {
totalDelta -= int64(targetBalances[i]) // Actually lost amount (negative)
targetBalances[i] = 0
} else if delta < 0 {
targetBalances[i] -= uint64(-delta)
totalDelta += delta
} else {
// Prevent overflow
if uint64(delta) > math.MaxUint64-targetBalances[i] {
gained := math.MaxUint64 - targetBalances[i]
totalDelta += int64(gained)
targetBalances[i] = math.MaxUint64
} else {
targetBalances[i] += uint64(delta)
totalDelta += delta
}
}
}
_ = target.SetBalances(targetBalances)
// Apply through diff system
diff, err := Diff(source, target)
if err != nil {
return
}
result, err := ApplyDiff(t.Context(), source, diff)
if err != nil {
return
}
// Calculate total after
resultBalances := result.Balances()
var totalAfter uint64
for _, balance := range resultBalances {
totalAfter += balance
}
// Verify conservation (accounting for intended changes)
expectedTotal := totalBefore
if totalDelta >= 0 {
expectedTotal += uint64(totalDelta)
} else {
if uint64(-totalDelta) <= expectedTotal {
expectedTotal -= uint64(-totalDelta)
} else {
expectedTotal = 0
}
}
require.Equal(t, expectedTotal, totalAfter,
"Balance conservation violated: before=%d, delta=%d, expected=%d, actual=%d",
totalBefore, totalDelta, expectedTotal, totalAfter)
})
}
// PropertyTestMonotonicSlot verifies slot only increases
func FuzzPropertyMonotonicSlot(f *testing.F) {
f.Fuzz(func(t *testing.T, slotDelta uint64) {
source, _ := util.DeterministicGenesisStateElectra(t, 16)
target := source.Copy()
targetSlot := source.Slot() + primitives.Slot(slotDelta)
_ = target.SetSlot(targetSlot)
diff, err := Diff(source, target)
if err != nil {
return
}
result, err := ApplyDiff(t.Context(), source, diff)
if err != nil {
return
}
// Slot should never decrease
require.Equal(t, true, result.Slot() >= source.Slot(),
"Slot decreased from %d to %d", source.Slot(), result.Slot())
// Slot should match target
require.Equal(t, targetSlot, result.Slot())
})
}
// PropertyTestValidatorIndexIntegrity verifies validator indices remain consistent
func FuzzPropertyValidatorIndices(f *testing.F) {
f.Fuzz(func(t *testing.T, changeData []byte) {
// Convert byte data to boolean changes
var changes []bool
for i := 0; i < len(changeData) && len(changes) < 20; i++ {
changes = append(changes, changeData[i]%2 == 0)
}
source, _ := util.DeterministicGenesisStateElectra(t, uint64(len(changes)+5))
target := source.Copy()
// Apply changes
validators := target.Validators()
for i, shouldChange := range changes {
if i >= len(validators) {
break
}
if shouldChange {
validators[i].EffectiveBalance += 1000
}
}
_ = target.SetValidators(validators)
diff, err := Diff(source, target)
if err != nil {
return
}
result, err := ApplyDiff(t.Context(), source, diff)
if err != nil {
return
}
// Validator count should not decrease
require.Equal(t, true, len(result.Validators()) >= len(source.Validators()),
"Validator count decreased from %d to %d", len(source.Validators()), len(result.Validators()))
// Public keys should be preserved for existing validators
sourceVals := source.Validators()
resultVals := result.Validators()
for i := range sourceVals {
if i < len(resultVals) {
require.DeepEqual(t, sourceVals[i].PublicKey, resultVals[i].PublicKey,
"Public key changed at validator index %d", i)
}
}
})
}

View File

@@ -0,0 +1,392 @@
package hdiff
import (
"fmt"
"sync"
"testing"
"time"
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/OffchainLabs/prysm/v6/testing/util"
)
// TestIntegerOverflowProtection tests protection against balance overflow attacks
func TestIntegerOverflowProtection(t *testing.T) {
source, _ := util.DeterministicGenesisStateElectra(t, 8)
// Test balance overflow in diffToBalances - use realistic values
t.Run("balance_diff_overflow", func(t *testing.T) {
target := source.Copy()
balances := target.Balances()
// Set high but realistic balance values (32 ETH in Gwei = 32e9)
balances[0] = 32000000000 // 32 ETH
balances[1] = 64000000000 // 64 ETH
_ = target.SetBalances(balances)
// This should work fine with realistic values
diffs, err := diffToBalances(source, target)
require.NoError(t, err)
// Verify the diffs are reasonable
require.Equal(t, true, len(diffs) > 0, "Should have balance diffs")
})
// Test reasonable balance changes
t.Run("realistic_balance_changes", func(t *testing.T) {
// Create realistic balance changes (slashing, rewards)
balancesDiff := []int64{1000000000, -500000000, 2000000000} // 1 ETH gain, 0.5 ETH loss, 2 ETH gain
// Apply to state with normal balances
testSource := source.Copy()
normalBalances := []uint64{32000000000, 32000000000, 32000000000} // 32 ETH each
_ = testSource.SetBalances(normalBalances)
// This should work fine
result, err := applyBalancesDiff(testSource, balancesDiff)
require.NoError(t, err)
resultBalances := result.Balances()
require.Equal(t, uint64(33000000000), resultBalances[0]) // 33 ETH
require.Equal(t, uint64(31500000000), resultBalances[1]) // 31.5 ETH
require.Equal(t, uint64(34000000000), resultBalances[2]) // 34 ETH
})
}
// TestReasonablePerformance tests that operations complete in reasonable time
func TestReasonablePerformance(t *testing.T) {
t.Run("large_state_performance", func(t *testing.T) {
// Test with a large but realistic validator set
source, _ := util.DeterministicGenesisStateElectra(t, 1000) // 1000 validators
target := source.Copy()
// Make realistic changes
_ = target.SetSlot(source.Slot() + 32) // One epoch
validators := target.Validators()
for i := 0; i < 100; i++ { // 10% of validators changed
validators[i].EffectiveBalance += 1000000000 // 1 ETH change
}
_ = target.SetValidators(validators)
// Should complete quickly
start := time.Now()
diff, err := Diff(source, target)
duration := time.Since(start)
require.NoError(t, err)
require.Equal(t, true, duration < time.Second, "Diff creation took too long: %v", duration)
require.Equal(t, true, len(diff.StateDiff) > 0, "Should have state diff")
})
t.Run("realistic_diff_application", func(t *testing.T) {
// Test applying diffs to large states
source, _ := util.DeterministicGenesisStateElectra(t, 500)
target := source.Copy()
_ = target.SetSlot(source.Slot() + 1)
// Create and apply diff
diff, err := Diff(source, target)
require.NoError(t, err)
start := time.Now()
result, err := ApplyDiff(t.Context(), source, diff)
duration := time.Since(start)
require.NoError(t, err)
require.Equal(t, target.Slot(), result.Slot())
require.Equal(t, true, duration < time.Second, "Diff application took too long: %v", duration)
})
}
// TestStateTransitionValidation tests realistic state transition scenarios
func TestStateTransitionValidation(t *testing.T) {
t.Run("validator_slashing_scenario", func(t *testing.T) {
source, _ := util.DeterministicGenesisStateElectra(t, 10)
target := source.Copy()
// Simulate validator slashing (realistic scenario)
validators := target.Validators()
validators[0].Slashed = true
validators[0].EffectiveBalance = 0 // Slashed validator loses balance
_ = target.SetValidators(validators)
// This should work fine
diff, err := Diff(source, target)
require.NoError(t, err)
result, err := ApplyDiff(t.Context(), source, diff)
require.NoError(t, err)
require.Equal(t, true, result.Validators()[0].Slashed)
require.Equal(t, uint64(0), result.Validators()[0].EffectiveBalance)
})
t.Run("epoch_transition_scenario", func(t *testing.T) {
source, _ := util.DeterministicGenesisStateElectra(t, 64)
target := source.Copy()
// Simulate epoch transition with multiple changes
_ = target.SetSlot(source.Slot() + 32) // One epoch
// Some validators get rewards, others get penalties
balances := target.Balances()
for i := 0; i < len(balances); i++ {
if i%2 == 0 {
balances[i] += 100000000 // 0.1 ETH reward
} else {
if balances[i] > 50000000 {
balances[i] -= 50000000 // 0.05 ETH penalty
}
}
}
_ = target.SetBalances(balances)
// This should work smoothly
diff, err := Diff(source, target)
require.NoError(t, err)
result, err := ApplyDiff(t.Context(), source, diff)
require.NoError(t, err)
require.Equal(t, target.Slot(), result.Slot())
})
t.Run("consistent_state_root", func(t *testing.T) {
// Test that diffs preserve state consistency
source, _ := util.DeterministicGenesisStateElectra(t, 32)
target := source.Copy()
// Make minimal changes
_ = target.SetSlot(source.Slot() + 1)
// Diff and apply should be consistent
diff, err := Diff(source, target)
require.NoError(t, err)
result, err := ApplyDiff(t.Context(), source, diff)
require.NoError(t, err)
// Result should match target
require.Equal(t, target.Slot(), result.Slot())
require.Equal(t, len(target.Validators()), len(result.Validators()))
require.Equal(t, len(target.Balances()), len(result.Balances()))
})
}
// TestSerializationRoundTrip tests serialization consistency
func TestSerializationRoundTrip(t *testing.T) {
t.Run("diff_serialization_consistency", func(t *testing.T) {
// Test that serialization and deserialization are consistent
source, _ := util.DeterministicGenesisStateElectra(t, 16)
target := source.Copy()
// Make changes
_ = target.SetSlot(source.Slot() + 5)
validators := target.Validators()
validators[0].EffectiveBalance += 1000000000
_ = target.SetValidators(validators)
// Create diff
diff1, err := Diff(source, target)
require.NoError(t, err)
// Deserialize and re-serialize
hdiff, err := newHdiff(diff1)
require.NoError(t, err)
diff2 := hdiff.serialize()
// Apply both diffs - should get same result
result1, err := ApplyDiff(t.Context(), source, diff1)
require.NoError(t, err)
result2, err := ApplyDiff(t.Context(), source, diff2)
require.NoError(t, err)
require.Equal(t, result1.Slot(), result2.Slot())
require.Equal(t, result1.Validators()[0].EffectiveBalance, result2.Validators()[0].EffectiveBalance)
})
t.Run("empty_diff_handling", func(t *testing.T) {
// Test that empty diffs are handled correctly
source, _ := util.DeterministicGenesisStateElectra(t, 8)
target := source.Copy() // No changes
// Should create minimal diff
diff, err := Diff(source, target)
require.NoError(t, err)
// Apply should work and return equivalent state
result, err := ApplyDiff(t.Context(), source, diff)
require.NoError(t, err)
require.Equal(t, source.Slot(), result.Slot())
require.Equal(t, len(source.Validators()), len(result.Validators()))
})
t.Run("compression_efficiency", func(t *testing.T) {
// Test that compression is working effectively
source, _ := util.DeterministicGenesisStateElectra(t, 100)
target := source.Copy()
// Make small changes
_ = target.SetSlot(source.Slot() + 1)
validators := target.Validators()
validators[0].EffectiveBalance += 1000000000
_ = target.SetValidators(validators)
// Create diff
diff, err := Diff(source, target)
require.NoError(t, err)
// Get full state size
fullStateSSZ, err := target.MarshalSSZ()
require.NoError(t, err)
// Diff should be much smaller than full state
diffSize := len(diff.StateDiff) + len(diff.ValidatorDiffs) + len(diff.BalancesDiff)
require.Equal(t, true, diffSize < len(fullStateSSZ)/2,
"Diff should be smaller than full state: diff=%d, full=%d", diffSize, len(fullStateSSZ))
})
}
// TestKMPSecurity tests the KMP algorithm for security issues
func TestKMPSecurity(t *testing.T) {
t.Run("nil_pointer_handling", func(t *testing.T) {
// Test with nil pointers in the pattern/text
pattern := []*int{nil, nil, nil}
text := []*int{nil, nil, nil, nil, nil}
equals := func(a, b *int) bool {
if a == nil && b == nil {
return true
}
if a == nil || b == nil {
return false
}
return *a == *b
}
// Should not panic - result can be any integer
result := kmpIndex(len(pattern), text, equals)
_ = result // Any result is valid, just ensure no panic
})
t.Run("empty_pattern_edge_case", func(t *testing.T) {
var pattern []*int
text := []*int{new(int), new(int)}
equals := func(a, b *int) bool { return a == b }
result := kmpIndex(0, text, equals)
require.Equal(t, 0, result, "Empty pattern should return 0")
_ = pattern // Silence unused variable warning
})
t.Run("realistic_pattern_performance", func(t *testing.T) {
// Test with realistic sizes to ensure good performance
realisticSize := 100 // More realistic for validator arrays
pattern := make([]*int, realisticSize)
text := make([]*int, realisticSize*2)
// Create realistic pattern
for i := range pattern {
val := i % 10 // More variation
pattern[i] = &val
}
for i := range text {
val := i % 10
text[i] = &val
}
equals := func(a, b *int) bool {
if a == nil && b == nil {
return true
}
if a == nil || b == nil {
return false
}
return *a == *b
}
start := time.Now()
result := kmpIndex(len(pattern), text, equals)
duration := time.Since(start)
// Should complete quickly with realistic inputs
require.Equal(t, true, duration < time.Second,
"KMP took too long: %v", duration)
_ = result // Any result is valid, just ensure performance is good
})
}
// TestConcurrencySafety tests thread safety of the hdiff operations
func TestConcurrencySafety(t *testing.T) {
t.Run("concurrent_diff_creation", func(t *testing.T) {
source, _ := util.DeterministicGenesisStateElectra(t, 32)
target := source.Copy()
_ = target.SetSlot(source.Slot() + 1)
const numGoroutines = 10
const iterations = 100
var wg sync.WaitGroup
errors := make(chan error, numGoroutines*iterations)
for i := 0; i < numGoroutines; i++ {
wg.Add(1)
go func(workerID int) {
defer wg.Done()
for j := 0; j < iterations; j++ {
_, err := Diff(source, target)
if err != nil {
errors <- fmt.Errorf("worker %d iteration %d: %v", workerID, j, err)
}
}
}(i)
}
wg.Wait()
close(errors)
// Check for any errors
for err := range errors {
t.Error(err)
}
})
t.Run("concurrent_diff_application", func(t *testing.T) {
ctx := t.Context()
source, _ := util.DeterministicGenesisStateElectra(t, 16)
target := source.Copy()
_ = target.SetSlot(source.Slot() + 5)
diff, err := Diff(source, target)
require.NoError(t, err)
const numGoroutines = 10
var wg sync.WaitGroup
errors := make(chan error, numGoroutines)
for i := 0; i < numGoroutines; i++ {
wg.Add(1)
go func(workerID int) {
defer wg.Done()
// Each goroutine needs its own copy of the source state
localSource := source.Copy()
_, err := ApplyDiff(ctx, localSource, diff)
if err != nil {
errors <- fmt.Errorf("worker %d: %v", workerID, err)
}
}(i)
}
wg.Wait()
close(errors)
// Check for any errors
for err := range errors {
t.Error(err)
}
})
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,399 @@
# State diffs in Prysm
The current document describes the implementation details and the design of hierarchical state diffs on Prysm. They follow the same design as [Lighthouse](https://github.com/dapplion/tree-states-review-guide/blob/main/persisted_hdiff.md) which in turn is an implementation of A. Nashatyrev's [design](https://hackmd.io/G82DNSdvR5Osw2kg565lBA).
Incremental state diffs can be used both for databases and memory representations of states. This document focuses on the state diffs necessary for the first usage. Prysm already handles memory deduplication of states with multi value slices, thus a diff mechanism would result in less impact.
## The basic design.
The idea is to diagram the cold-state database as a forest:
- Each tree in the forest is rooted by a full state snapshot, saved every λ_0 slots (think once a year).
- Each tree has the same height h. The root is unique and corresponds to the full snapshot, but on each level *1 ≤ i ≤ h*, there are β_i bifurcation nodes, which are stored every λ_i slots. Thus for example if we had *h = 2*, *λ_0 = 2^21*, *λ_1 = 2^18*, *λ_2 = 2^5*, we would have *β_1 = 7* and *β_2 = 8191* (notice that we subtract 1 since the first bifurcation node is just the state of the upper level). On the first level we would have 7 nodes written every ~36 days and on the second level we would have 8191 nodes written once every epoch.
- At each level *1 ≤ i ≤ h*, in the *β_i* nodes that are stored, instead of writing a full state snapshot, we store the diff between the state at that given slot and the state corresponding to the parent node in level *i-1*.
![database layout](./db_layout.png)
### Saving state diffs.
Let us assume that we have a running node that already has an hdiff compatible database. That is, some snapshot with a full state is saved at some slot `o` (for *offset*). Suppose that we have just updated finalization, thus we have some blocks that we may need to save a state diff (or even a snapshot) for. Suppose we try for a block with slot `c`. Then at each of the slots
o, o + λ_0, o + 2 λ_0, ..., o + k_0 λ_0
we have a full snapshot state saved. We assume that o + (k_0+1) λ_0 > c, so that our latest snapshot is in fact at slot o + k λ_0. Let us call this state *s_0*. At each of the slots
o + k_0 λ_0 + λ_1, o + k_0 λ_0 + 2 λ_1, ..., o + k_0 λ_0 + k_1 λ_1
we have stored a state diff between the state at that slot and *s_0*. We assume that
o + k_0 λ_0 + (k_1+1) λ_1 > c
so that the latest diff at level one is in fact at slot o + k_0 λ_0 + k_1 λ_1. Let us call the sate at that slot *s_1*. it is obtained by applying the state diff saved at that slot to the state *s_0*. Similarly at the next level, for each slot
o + k_0 λ_0 + k_1 λ_1 + λ_2, o + k_0 λ_0 + k_1 λ_1 + 2 λ_2, ..., o + k_0 λ_0 + k_1 λ_1 + k_2 λ_2
we have stored a state diff to the state *s_1*. We assume that
o + k_0 λ_0 + k_1 λ_1 + (k_2+1) λ_2 > c
so that the latest diff at level two is indeed at slot o + k_0 λ_0 + k_1 λ_1 + k_2 λ_2. Let us call the corresponding state *s_2*. It is obtained applying the last diff at level 2 to the state *s_1*, which in turn was obtained appplying a diff to the state *s_0*.
We continue until we have covered all of our levels up to level h. That is we have states *s_0*, *s_1*, ..., *s_{h}* and the last one is the state at slot
o + k_0 λ_0 + k_1 λ_1 + ... + k_h λ_h
So now we want to decide what do to with our state *t* at slot c. We act as follows. If o + k_0 λ_0 + k_1 λ_1 + ... + (k_h+1) λ_h > c. In this case we don't store anything. If on the other hand we have o + k_0 λ_0 + k_1 λ_1 + ... + (k_h+1) λ_h = c. In this case we will store either a state diff or an entire new snapshot. We proceed as follows.
If k_h < β_h, in this case we store a new state diff `Diff(s_{h-1},t)` at the slot c in level `h`.
If k_h = β_h, we check the next level. If k_{h-1} < β_{h-1}, then we store a new state diff `Diff(s_{h-2},t)` at level `h-1` at the slot `c`.
If k_{h-1} = β_{h-1} then we compare the next level: if k_{h-2} < β_{h-2}, then we store a new state diff `Diff(s_{h-3}, t)` at level `h-2` at the slot `c`.
We continue like this, if we reach the point in which all k_i = β_i for ì=1,...,h, then we store a new full snapshot with the state `t` at the slot `c`.
### Triggering storage
When we update finalization, we call `MigrateToCold`, this function, instead of calling the database to store a full state every few epochs (as we do today), will send the state `t` at slot `c` as in the previous section, to save the corresponding diff. The package that handles state saving internally is the `database` package. However, the function `MigrateToCold` is aware of the values of the offset *o* and the configuration constants λ_1, ..., λ_h so as to only send the states `t` for which `c` is of the form `o + k λ_h`.
### Database changes
The database exposes the following API to save states
```
SaveState(ctx context.Context, state state.ReadOnlyBeaconState, blockRoot [32]byte) error
```
This functions will change internally to save just the diff or a snapshot if appropriate. On the other hand, the following is the API to recover a state:
```go
HasState(ctx context.Context, blockRoot [32]byte) bool
State(ctx context.Context, blockRoot [32]byte) (state.BeaconState, error)
```
The first function can return true now in a static manner according to the slot of the corresponing `blockRoot`, it simply checks that it is of the form o + k λ_h. The second function can recover those states by applying the corresponding diffs.
Summarizing, the database has no changes in the exposed API, minimizing changes in the overal Prysm implementation, while the database internally changes the functions `State` and `SaveState` to use the `consensus-types/hdiff` package. This makes the serialization package fairly contained and only accessible from within the database package.
### Stategen changes
The `stategen` package is respondible for the migration to cold database, it exposes the function
```go
func (s *State) MigrateToCold(ctx context.Context, fRoot [32]byte) error {
```
that takes the finalized root and decides which states to save. This function is now changed to save only based on the slot of the state, for those slots that have the form o + k λ_h. A **warning** has to be said about missing blocks. Since the database will have to keep the state by slots now, a good approach in this function when there is a missing block at the corresponding slot, is to actually process the state to the right slot and save it already processed.
Another function that needs to change minimally is the function
```
func (s *State) StateByRoot(ctx context.Context, blockRoot [32]byte) (state.BeaconState, error)
```
That will get the ancestor from db simply by the slot rather than the root.
### Longer term changes
We could change the database API to include getters and setters by slot in the cold database, since anyway this will keep only canonical states this would make things easier at the stategen level.
### Configuration
We can make the constants h and λ_0, ... , λ_h user-configuratble. Thus, someone that is less storage constained and wants to run an archive RPC node, will set h higher and λ_h smaller (say 32 to save one diff every epoch), while a user that doesn't care about past states may even set `h=0` and not save anything.
### Database migration
There is no migration support expected.
### Startup from clean database
Starting up from a clean database and checkpoint sync will download the checkpoint state at slot o and set that slot as the offset in the database and save the first full snapshot with the checkpoint state.
Starting up from a clean database and from genesis will set o = 0 and start syncing from genesis as usual.
### Backfill
The following is added as an configurable option, pass the flag `--backfill-origin-state ssz`, in this case the node will download the state `ssz` and set as offset this state's slot. Will download the checkpoint state and start syncing forward as usual but will not call `MigrateToCold` until the backfill service is finished. In the background the node will download all blocks all the way up to the state ssz, then start forward syncing those blocks regenerating the finalized states and when they are of the form o + k λ_h. Once the forward syncing has caught up with the finalized checkpoint, we can start calling `MigrateToCold` again. This backfill mechanism is much faster than the current foward syncing to regenerate the states: we do not need to do any checks on the EL since the blocks are already finalized and trusted, the hashes are already confirmed.
### Database Prunning
Currently we have a flag `--pruner-retention-epochs` which will be deprecated. Instead, the pruning mechanism is simply the following, the user specifies how many snapshopts wants to keep (by default 0 means keep all snapshots). If the user say specifies `--pruner-retention-snapshots 1`, then the node will delete everything in the database everytime we save a new snapshot every λ_0 slots. So in particular, a user that wants to keep its database to a minimum, it will set h=0, λ_0 to a very large value, and pass 1 to this flag, thus the node will only keep one state at any time and will not update it.
## Implementation details.
This section contains actual implementation details of the feature. It will be populated as pull requests are being opened with the final details of the implementation. For a high level design document please refer to [this previous section](#the-basic-design).
### Serialization
The package `hdiff` located in `consensus-types/hdiff` is responsible for computing and applying state diffs between two different beacon states and serializing/deserializing them to/from a byte sequence.
#### Exported API
The only exported API consists of
```go
type HdiffBytes struct {
StateDiff []byte
ValidatorDiffs []byte
BalancesDiff []byte
}
func Diff(source, target state.ReadOnlyBeaconState) (HdiffBytes, error)
func ApplyDiff(ctx context.Context, source state.BeaconState, diff HdiffBytes) (state.BeaconState, error)
```
The structure `HdiffBytes` contains three different slices that can be handled independently by the caller (typically this will be database methods). These three slices are the serialized and Snappy compressed form of a state diff between two different states.
The function `Diff` takes two states and returns the serialized diff between them. The function `ApplyDiff` takes a state and a diff and returns the target state after having applied the diff to the source state.
#### The `hdiff` structure
When comparing a source state *s* and a target state *t*, before serializing, their difference is kept in a native structure `hdiff` which itself consist of three separate diffs.
```go
type hdiff struct {
stateDiff *stateDiff
validatorDiffs []validatorDiff
balancesDiff []int64
}
```
The `stateDiff` entry contains the bulk of the state diff, except the validator registry diff and the balance slice diff. These last two are separated to be able to store them separatedly. Often times, local RPC requests are for balances or validator status, and with the hierarchical strcutrure, we can reproduce them without regenerating the full state.
#### The `stateDiff` structure
This structure encodes the possible differences between two beacon states.
```go
type stateDiff struct {
targetVersion int
eth1VotesAppend bool
justificationBits byte
slot primitives.Slot
fork *ethpb.Fork
latestBlockHeader *ethpb.BeaconBlockHeader
blockRoots [fieldparams.BlockRootsLength][fieldparams.RootLength]byte
stateRoots [fieldparams.StateRootsLength][fieldparams.RootLength]byte
historicalRoots [][fieldparams.RootLength]byte
eth1Data *ethpb.Eth1Data
eth1DataVotes []*ethpb.Eth1Data
eth1DepositIndex uint64
randaoMixes [fieldparams.RandaoMixesLength][fieldparams.RootLength]byte
slashings [fieldparams.SlashingsLength]int64
previousEpochAttestations []*ethpb.PendingAttestation
currentEpochAttestations []*ethpb.PendingAttestation
previousJustifiedCheckpoint *ethpb.Checkpoint
currentJustifiedCheckpoint *ethpb.Checkpoint
finalizedCheckpoint *ethpb.Checkpoint
previousEpochParticipation []byte
currentEpochParticipation []byte
inactivityScores []uint64
currentSyncCommittee *ethpb.SyncCommittee
nextSyncCommittee *ethpb.SyncCommittee
executionPayloadHeader interfaces.ExecutionData
nextWithdrawalIndex uint64
nextWithdrawalValidatorIndex primitives.ValidatorIndex
historicalSummaries []*ethpb.HistoricalSummary
depositRequestsStartIndex uint64
depositBalanceToConsume primitives.Gwei
exitBalanceToConsume primitives.Gwei
earliestExitEpoch primitives.Epoch
consolidationBalanceToConsume primitives.Gwei
earliestConsolidationEpoch primitives.Epoch
pendingDepositIndex uint64
pendingPartialWithdrawalsIndex uint64
pendingConsolidationsIndex uint64
pendingDepositDiff []*ethpb.PendingDeposit
pendingPartialWithdrawalsDiff []*ethpb.PendingPartialWithdrawal
pendingConsolidationsDiffs []*ethpb.PendingConsolidation
proposerLookahead []uint64
}
```
This type is only used internally when serializing/deserializing and applying state diffs. We could in principle avoid double allocations and increase performance by avoiding entirely having a native type and working directly with the serialized bytes. The tradeoff is readability of the serialization functions.
#### The `validatorDiff` structure
This structure is similar to the `stateDiff` one, it is only used internally in the `hdiff` package in `consensus-types`
```go
type validatorDiff struct {
Slashed bool
index uint32
PublicKey []byte
WithdrawalCredentials []byte
EffectiveBalance uint64
ActivationEligibilityEpoch primitives.Epoch
ActivationEpoch primitives.Epoch
ExitEpoch primitives.Epoch
WithdrawableEpoch primitives.Epoch
}
```
#### The `balancesDiff` slice
Given a source state `s` and a target state `t` assumed to be newer than `s`, so that the length of `t.balances` is greater or equal than that of `s.balances`. Then the `balancesDiff` slice inside the `hdiff` structure is computed simply as the algebraic difference, it's *i-th* entry is given by `t.balances[i] - s.balances[i]` where the second term is considered as zero if `i ≥ len(s.balances)`.
#### Deserializing with `newHdiff`
The function
```go
func newHdiff(data HdiffBytes) (*hdiff, error)
```
takes a serialized diff and produces the native internal type `hdiff`. This function encodes the internal logic for deserialization. It internally calls the functions ` newStateDiff`, `newValidatorDiffs` and `newBalancesDiff` to obtain the three inner structures.
The main deserialization routines take the byte slices and they first decompress them with `snappy.Decode`. They create an empty `stateDiff`, `validatorDiff` or `balancesDiff` object `ret` and after that they pass a pointer to the decompressed byte slice `data` to helper functions `ret.readXXX(&data)` that populate each of the entries of `ret`. Here `XXX` corresponds to each of the entries in the beacon state, like `fork`, `slot`, etc. Each one of the helpers receives a pointer to the `data` slice that contains the byte slice of the diff that **is still yet to be deserialized**. The helper populates the corresponding entry in the hdiff structure and then modifies the `data` slice to drop the deserialized bytes. That is, each helper receives a slice that needs to be deserialized since its first byte.
The following list documents the method that is used for serialization/deserialization of each entry
##### Version
The version is stored as a little endian `uint64` in fixed 8 bytes of `data`. This version is the target version, that is, we override whatever the source state version is, with this target version.
##### Slot
The slot is treated exactly the same as the version entry.
##### Fork
The fork is deserialized as follows. If the first byte of `data` is zero (a constant called `nilMarker` in the package) then the fork pointer is `nil` in the `hdiff` struture. If the first byte of `data` is not zero then the remaining bytes deserialize to a full `Fork` object.
When applying the diff, if the fork pointer is `nil` then the source's Fork is not changed, while if it is not-nil, then the source's Fork is changed to whatever the `hdiff` pointer is.
##### Latest Block Header
The latest Block header is treated exactly like the Fork pointer.
##### Block Roots
The block roots slice is deserialized literally as a full slice of beacon block roots, this may seem like a large waste of memory and space since this slice is 8192 roots, each 32 bytes. However, the serialization process is as follows, if a blockroot has not changed between the source and the target state, we store a full zero root `0x00...`. For states that are *close by*, the block roots slice will not have changed much, this will produce a slice that is mostly zeroes, and these gets stored occupying minimal space with Snappy compression. When two states are more than 8192 slots appart, the target block roots slice will have to be saved in its entirety, which is what this method achieves.
We could get a little more performance here if instead of keeping a full zeroed out root in the internal `hdiff` structure, we stored an empty slice. But this way the check for lengths becomes slightly more complicated.
##### State Roots
The state roots slice is treated exactly like the block roots one.
##### Historical Roots
The historical roots slice diff is stored as follows, the first 8 bytes store a little endian `uint64` that determines the length of the slice. After this, the following bytes contain as many 32 byte roots as this length indicates. Again, as in the previous root slices, if the root is not to be changed from the source state, we store a zero root.
##### Eth1 Data
The Eth1 Data diff object is treated exactly like the fork object.
##### Eth1 Data Votes
The `stateDiff` structure has two fields related to Eth1 data votes. The boolean entry `eth1VotesAppend` and a slice `eth1DataVotes`. The boolean indicates if the slice is to be *appended* to the source target or if the eth1 data vote slice needs to be completely replaced with the slice in the diff.
Deserialization then goes as follows, if the first byte is `nilMarker` then `eth1VotesAppend` is set to `True`, and `False` otherwise. The following 8 bytes contain a `uint64` serialization of the length of the slice. The remaining bytes contain the serialized slice.
##### Eth1 Deposit Index
This field always overrides the source's value. It is stored as an 8 bytes serialized `uint64`.
##### Randao Mixes
This field is treated exactly like the block roots slice.
##### Slashings
The slashings slice is stored as the algebraic difference between the target and the source state `t.slashings - s.slashings`. Thus the data is read as a sequence of 8 bytes serialized little Endian `int64`. When applying this diff to a source state, we add this number to the source state's slashings. This way, numbers are kept small and they snappy compress better.
##### Pending Attestations
Pending attestations are only present in Phase 0 states. So the paths to deserialize them (both for *previous and current epoch attestations*) is only executed in case the target state is a Phase 0 state (notice that this implies that the source state must have been a Phase0 state as well).
For both of these slices we store first the length in the first 8 bytes. Then we loop over the remaining bytes deserializing each pending attestation. Each of them is of variable size and is deserialized as follows, the first 8 bytes contain the attestation aggregation bits length. The next bytes (how many is determined by the aggregation bits length) encode the aggregation bits. The next 128 bytes are the SSZ encoded attestation data. Finally the inclusion delay and the proposer index are serialized as 8 bytes `uint64`.
##### Previous and Current epoch participation
These slices are there post Altair. They are serialized as follows, the first 8 bytes contain the length, and the remaining bytes (indicated by the length) are just stored directly as a byte slice.
##### Justification Bits
These are stored as a single byte and they always override the value of the source state with this byte stored in the `hdiff` structure.
##### Finalized and Previous/Current justified Checkpoints
These are stored as SSZ serialized checkpoints.
##### Inactivity Scores
The first 8 bytes contain the little Endian encoded length, and the remaining bytes contain the `uint64` serialized slice.
##### Current and Next Sync committees
If the first byte is 0, then the sync committee is set to be nil (and therefore the source's sync committee is not changed). Otherwise the remaining bytes contain the SSZ serialized sync committee.
##### Execution Payload Header
This is serialized exactly like the sync committes. Notice that the implementation of `readExecutionPayloadHeader` is more involved because the SSZ serialization of the header depends on the state's version.
##### Withdrawal Indices
The fields `nextWithdrawalIndex` and `nextWithdrawalValidatorIndex` are treated just like the `Slot` field.
##### Historical Summaries
The first 8 bytes store the length of the list and the remaining bytes are stored as SSZ serializations of the summary entry. This slice is **appended** to the source state's historical summary state.
##### Electra requests indices
The fields `depositRequestsStartIndex`, `depositBalanceToConsume`, `exitBalanceToConsume`, `earliestExitEpoch`, `consolidationBalanceToConsume` and `earliestConsolidationEpoch` are stored like the `Slot` field.
##### Pending Deposits
The first 8 bytes store the `pendingDepositIndex`, the next 8 bytes store the length of the pending deposit diff slice. The remaining bytes store a slice of SSZ serialized `PendingDeposit` objects.
This diff slice is different than others, we store the extra index `pendingDepositIndex` in the `hdiff` structure that is used as follows. This index indicates how many pending deposits need to be dropped from the source state. The remaining slice is added to the end of the source state's pending deposits. The rationale for this serialization algorithm is that if taking the diff of two close enough states, the pending deposit queue may be very large. Between the source and the target, the first few deposits may have already been consumed, but the remaining large majority would still be there in the target. The target state may have some more extra deposits to be added in the end.
Similarly, when computing the diff between the source and the target state, we need to find the index of the first deposit in common. We use the [Knuth-Morris-Pratt](https://en.wikipedia.org/wiki/Knuth%E2%80%93Morris%E2%80%93Pratt_algorithm) algorith to find it.
Suppose that the source pending deposits are
```
[A, B, C, D, E, F, G, H]
```
And the target pending deposits are
```
[C, D, E, F, G, H, I, J, K]
```
Then we will store `pendingDepositIndex = 2` and the diff slice will be
```
[I, J, K]
```
##### Pending Partial Withdrawals
This field is treated exactly like the pending deposits.
##### Pending Consolidations
This field is treated exactly like the pending deposits.
##### Proposer Lookahead
The proposer lookahead is stored as the SSZ serialized version of the field. It always overrides the source's field.
#### Applying a diff
The exported function
```go
func ApplyDiff(ctx context.Context, source state.BeaconState, diff HdiffBytes) (state.BeaconState, error)
```
Takes care of applying the diff, it first calls `newHdiff` to convert the raw bytes in `diff` into an internal `hdiff` structure, and then it modifies the `source` state as explained above returning the modified state.
#### Computing a Diff
The exported function
```go
func Diff(source, target state.ReadOnlyBeaconState) (HdiffBytes, error)
```
Takes two states and returns the corresponding diff bytes. This function calls the function `diffInternal` which in turn calls `diffToState`, `diffToVals` and `diffToBalances` that each return the corresponding component of an internal `hdiff` structure. Then we call `serialize()` on the correponding `hdiff` structure. The function `serialize` constructs the `data` byte slice as described above in the [Deserialization](#deserialization) section and finally it calls `snappy.Encode()` on each of the three slices.

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,5 @@
go test fuzz v1
byte('\x00')
uint64(0)
[]byte("0")
[]byte("")

View File

@@ -0,0 +1,2 @@
go test fuzz v1
[]byte("0")

View File

@@ -0,0 +1,2 @@
go test fuzz v1
[]byte("0000000\xff")

View File

@@ -0,0 +1,16 @@
load("@prysm//tools/go:def.bzl", "go_library", "go_test")
go_library(
name = "go_default_library",
srcs = ["comparisons.go"],
importpath = "github.com/OffchainLabs/prysm/v6/consensus-types/helpers",
visibility = ["//visibility:public"],
deps = ["//proto/prysm/v1alpha1:go_default_library"],
)
go_test(
name = "go_default_test",
srcs = ["comparisons_test.go"],
embed = [":go_default_library"],
deps = ["//proto/prysm/v1alpha1:go_default_library"],
)

View File

@@ -0,0 +1,109 @@
package helpers
import (
"bytes"
ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
)
func ForksEqual(s, t *ethpb.Fork) bool {
if s == nil && t == nil {
return true
}
if s == nil || t == nil {
return false
}
if s.Epoch != t.Epoch {
return false
}
if !bytes.Equal(s.PreviousVersion, t.PreviousVersion) {
return false
}
return bytes.Equal(s.CurrentVersion, t.CurrentVersion)
}
func BlockHeadersEqual(s, t *ethpb.BeaconBlockHeader) bool {
if s == nil && t == nil {
return true
}
if s == nil || t == nil {
return false
}
if s.Slot != t.Slot {
return false
}
if s.ProposerIndex != t.ProposerIndex {
return false
}
if !bytes.Equal(s.ParentRoot, t.ParentRoot) {
return false
}
if !bytes.Equal(s.StateRoot, t.StateRoot) {
return false
}
return bytes.Equal(s.BodyRoot, t.BodyRoot)
}
func Eth1DataEqual(s, t *ethpb.Eth1Data) bool {
if s == nil && t == nil {
return true
}
if s == nil || t == nil {
return false
}
if !bytes.Equal(s.DepositRoot, t.DepositRoot) {
return false
}
if s.DepositCount != t.DepositCount {
return false
}
return bytes.Equal(s.BlockHash, t.BlockHash)
}
func PendingDepositsEqual(s, t *ethpb.PendingDeposit) bool {
if s == nil && t == nil {
return true
}
if s == nil || t == nil {
return false
}
if !bytes.Equal(s.PublicKey, t.PublicKey) {
return false
}
if !bytes.Equal(s.WithdrawalCredentials, t.WithdrawalCredentials) {
return false
}
if s.Amount != t.Amount {
return false
}
if !bytes.Equal(s.Signature, t.Signature) {
return false
}
return s.Slot == t.Slot
}
func PendingPartialWithdrawalsEqual(s, t *ethpb.PendingPartialWithdrawal) bool {
if s == nil && t == nil {
return true
}
if s == nil || t == nil {
return false
}
if s.Index != t.Index {
return false
}
if s.Amount != t.Amount {
return false
}
return s.WithdrawableEpoch == t.WithdrawableEpoch
}
func PendingConsolidationsEqual(s, t *ethpb.PendingConsolidation) bool {
if s == nil && t == nil {
return true
}
if s == nil || t == nil {
return false
}
return s.SourceIndex == t.SourceIndex && s.TargetIndex == t.TargetIndex
}

View File

@@ -0,0 +1,637 @@
package helpers
import (
"testing"
ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
)
func TestForksEqual(t *testing.T) {
tests := []struct {
name string
s *ethpb.Fork
t *ethpb.Fork
want bool
}{
{
name: "both nil",
s: nil,
t: nil,
want: true,
},
{
name: "first nil",
s: nil,
t: &ethpb.Fork{Epoch: 1},
want: false,
},
{
name: "second nil",
s: &ethpb.Fork{Epoch: 1},
t: nil,
want: false,
},
{
name: "equal forks",
s: &ethpb.Fork{
Epoch: 100,
PreviousVersion: []byte{1, 2, 3, 4},
CurrentVersion: []byte{5, 6, 7, 8},
},
t: &ethpb.Fork{
Epoch: 100,
PreviousVersion: []byte{1, 2, 3, 4},
CurrentVersion: []byte{5, 6, 7, 8},
},
want: true,
},
{
name: "different epoch",
s: &ethpb.Fork{
Epoch: 100,
PreviousVersion: []byte{1, 2, 3, 4},
CurrentVersion: []byte{5, 6, 7, 8},
},
t: &ethpb.Fork{
Epoch: 200,
PreviousVersion: []byte{1, 2, 3, 4},
CurrentVersion: []byte{5, 6, 7, 8},
},
want: false,
},
{
name: "different previous version",
s: &ethpb.Fork{
Epoch: 100,
PreviousVersion: []byte{1, 2, 3, 4},
CurrentVersion: []byte{5, 6, 7, 8},
},
t: &ethpb.Fork{
Epoch: 100,
PreviousVersion: []byte{9, 10, 11, 12},
CurrentVersion: []byte{5, 6, 7, 8},
},
want: false,
},
{
name: "different current version",
s: &ethpb.Fork{
Epoch: 100,
PreviousVersion: []byte{1, 2, 3, 4},
CurrentVersion: []byte{5, 6, 7, 8},
},
t: &ethpb.Fork{
Epoch: 100,
PreviousVersion: []byte{1, 2, 3, 4},
CurrentVersion: []byte{9, 10, 11, 12},
},
want: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := ForksEqual(tt.s, tt.t); got != tt.want {
t.Errorf("ForksEqual() = %v, want %v", got, tt.want)
}
})
}
}
func TestBlockHeadersEqual(t *testing.T) {
tests := []struct {
name string
s *ethpb.BeaconBlockHeader
t *ethpb.BeaconBlockHeader
want bool
}{
{
name: "both nil",
s: nil,
t: nil,
want: true,
},
{
name: "first nil",
s: nil,
t: &ethpb.BeaconBlockHeader{Slot: 1},
want: false,
},
{
name: "second nil",
s: &ethpb.BeaconBlockHeader{Slot: 1},
t: nil,
want: false,
},
{
name: "equal headers",
s: &ethpb.BeaconBlockHeader{
Slot: 100,
ProposerIndex: 50,
ParentRoot: []byte{1, 2, 3, 4},
StateRoot: []byte{5, 6, 7, 8},
BodyRoot: []byte{9, 10, 11, 12},
},
t: &ethpb.BeaconBlockHeader{
Slot: 100,
ProposerIndex: 50,
ParentRoot: []byte{1, 2, 3, 4},
StateRoot: []byte{5, 6, 7, 8},
BodyRoot: []byte{9, 10, 11, 12},
},
want: true,
},
{
name: "different slot",
s: &ethpb.BeaconBlockHeader{
Slot: 100,
ProposerIndex: 50,
ParentRoot: []byte{1, 2, 3, 4},
StateRoot: []byte{5, 6, 7, 8},
BodyRoot: []byte{9, 10, 11, 12},
},
t: &ethpb.BeaconBlockHeader{
Slot: 200,
ProposerIndex: 50,
ParentRoot: []byte{1, 2, 3, 4},
StateRoot: []byte{5, 6, 7, 8},
BodyRoot: []byte{9, 10, 11, 12},
},
want: false,
},
{
name: "different proposer index",
s: &ethpb.BeaconBlockHeader{
Slot: 100,
ProposerIndex: 50,
ParentRoot: []byte{1, 2, 3, 4},
StateRoot: []byte{5, 6, 7, 8},
BodyRoot: []byte{9, 10, 11, 12},
},
t: &ethpb.BeaconBlockHeader{
Slot: 100,
ProposerIndex: 75,
ParentRoot: []byte{1, 2, 3, 4},
StateRoot: []byte{5, 6, 7, 8},
BodyRoot: []byte{9, 10, 11, 12},
},
want: false,
},
{
name: "different parent root",
s: &ethpb.BeaconBlockHeader{
Slot: 100,
ProposerIndex: 50,
ParentRoot: []byte{1, 2, 3, 4},
StateRoot: []byte{5, 6, 7, 8},
BodyRoot: []byte{9, 10, 11, 12},
},
t: &ethpb.BeaconBlockHeader{
Slot: 100,
ProposerIndex: 50,
ParentRoot: []byte{13, 14, 15, 16},
StateRoot: []byte{5, 6, 7, 8},
BodyRoot: []byte{9, 10, 11, 12},
},
want: false,
},
{
name: "different state root",
s: &ethpb.BeaconBlockHeader{
Slot: 100,
ProposerIndex: 50,
ParentRoot: []byte{1, 2, 3, 4},
StateRoot: []byte{5, 6, 7, 8},
BodyRoot: []byte{9, 10, 11, 12},
},
t: &ethpb.BeaconBlockHeader{
Slot: 100,
ProposerIndex: 50,
ParentRoot: []byte{1, 2, 3, 4},
StateRoot: []byte{13, 14, 15, 16},
BodyRoot: []byte{9, 10, 11, 12},
},
want: false,
},
{
name: "different body root",
s: &ethpb.BeaconBlockHeader{
Slot: 100,
ProposerIndex: 50,
ParentRoot: []byte{1, 2, 3, 4},
StateRoot: []byte{5, 6, 7, 8},
BodyRoot: []byte{9, 10, 11, 12},
},
t: &ethpb.BeaconBlockHeader{
Slot: 100,
ProposerIndex: 50,
ParentRoot: []byte{1, 2, 3, 4},
StateRoot: []byte{5, 6, 7, 8},
BodyRoot: []byte{13, 14, 15, 16},
},
want: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := BlockHeadersEqual(tt.s, tt.t); got != tt.want {
t.Errorf("BlockHeadersEqual() = %v, want %v", got, tt.want)
}
})
}
}
func TestEth1DataEqual(t *testing.T) {
tests := []struct {
name string
s *ethpb.Eth1Data
t *ethpb.Eth1Data
want bool
}{
{
name: "both nil",
s: nil,
t: nil,
want: true,
},
{
name: "first nil",
s: nil,
t: &ethpb.Eth1Data{DepositCount: 1},
want: false,
},
{
name: "second nil",
s: &ethpb.Eth1Data{DepositCount: 1},
t: nil,
want: false,
},
{
name: "equal eth1 data",
s: &ethpb.Eth1Data{
DepositRoot: []byte{1, 2, 3, 4},
DepositCount: 100,
BlockHash: []byte{5, 6, 7, 8},
},
t: &ethpb.Eth1Data{
DepositRoot: []byte{1, 2, 3, 4},
DepositCount: 100,
BlockHash: []byte{5, 6, 7, 8},
},
want: true,
},
{
name: "different deposit root",
s: &ethpb.Eth1Data{
DepositRoot: []byte{1, 2, 3, 4},
DepositCount: 100,
BlockHash: []byte{5, 6, 7, 8},
},
t: &ethpb.Eth1Data{
DepositRoot: []byte{9, 10, 11, 12},
DepositCount: 100,
BlockHash: []byte{5, 6, 7, 8},
},
want: false,
},
{
name: "different deposit count",
s: &ethpb.Eth1Data{
DepositRoot: []byte{1, 2, 3, 4},
DepositCount: 100,
BlockHash: []byte{5, 6, 7, 8},
},
t: &ethpb.Eth1Data{
DepositRoot: []byte{1, 2, 3, 4},
DepositCount: 200,
BlockHash: []byte{5, 6, 7, 8},
},
want: false,
},
{
name: "different block hash",
s: &ethpb.Eth1Data{
DepositRoot: []byte{1, 2, 3, 4},
DepositCount: 100,
BlockHash: []byte{5, 6, 7, 8},
},
t: &ethpb.Eth1Data{
DepositRoot: []byte{1, 2, 3, 4},
DepositCount: 100,
BlockHash: []byte{9, 10, 11, 12},
},
want: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := Eth1DataEqual(tt.s, tt.t); got != tt.want {
t.Errorf("Eth1DataEqual() = %v, want %v", got, tt.want)
}
})
}
}
func TestPendingDepositsEqual(t *testing.T) {
tests := []struct {
name string
s *ethpb.PendingDeposit
t *ethpb.PendingDeposit
want bool
}{
{
name: "both nil",
s: nil,
t: nil,
want: true,
},
{
name: "first nil",
s: nil,
t: &ethpb.PendingDeposit{Amount: 1},
want: false,
},
{
name: "second nil",
s: &ethpb.PendingDeposit{Amount: 1},
t: nil,
want: false,
},
{
name: "equal pending deposits",
s: &ethpb.PendingDeposit{
PublicKey: []byte{1, 2, 3, 4},
WithdrawalCredentials: []byte{5, 6, 7, 8},
Amount: 32000000000,
Signature: []byte{9, 10, 11, 12},
Slot: 100,
},
t: &ethpb.PendingDeposit{
PublicKey: []byte{1, 2, 3, 4},
WithdrawalCredentials: []byte{5, 6, 7, 8},
Amount: 32000000000,
Signature: []byte{9, 10, 11, 12},
Slot: 100,
},
want: true,
},
{
name: "different public key",
s: &ethpb.PendingDeposit{
PublicKey: []byte{1, 2, 3, 4},
WithdrawalCredentials: []byte{5, 6, 7, 8},
Amount: 32000000000,
Signature: []byte{9, 10, 11, 12},
Slot: 100,
},
t: &ethpb.PendingDeposit{
PublicKey: []byte{13, 14, 15, 16},
WithdrawalCredentials: []byte{5, 6, 7, 8},
Amount: 32000000000,
Signature: []byte{9, 10, 11, 12},
Slot: 100,
},
want: false,
},
{
name: "different withdrawal credentials",
s: &ethpb.PendingDeposit{
PublicKey: []byte{1, 2, 3, 4},
WithdrawalCredentials: []byte{5, 6, 7, 8},
Amount: 32000000000,
Signature: []byte{9, 10, 11, 12},
Slot: 100,
},
t: &ethpb.PendingDeposit{
PublicKey: []byte{1, 2, 3, 4},
WithdrawalCredentials: []byte{13, 14, 15, 16},
Amount: 32000000000,
Signature: []byte{9, 10, 11, 12},
Slot: 100,
},
want: false,
},
{
name: "different amount",
s: &ethpb.PendingDeposit{
PublicKey: []byte{1, 2, 3, 4},
WithdrawalCredentials: []byte{5, 6, 7, 8},
Amount: 32000000000,
Signature: []byte{9, 10, 11, 12},
Slot: 100,
},
t: &ethpb.PendingDeposit{
PublicKey: []byte{1, 2, 3, 4},
WithdrawalCredentials: []byte{5, 6, 7, 8},
Amount: 16000000000,
Signature: []byte{9, 10, 11, 12},
Slot: 100,
},
want: false,
},
{
name: "different signature",
s: &ethpb.PendingDeposit{
PublicKey: []byte{1, 2, 3, 4},
WithdrawalCredentials: []byte{5, 6, 7, 8},
Amount: 32000000000,
Signature: []byte{9, 10, 11, 12},
Slot: 100,
},
t: &ethpb.PendingDeposit{
PublicKey: []byte{1, 2, 3, 4},
WithdrawalCredentials: []byte{5, 6, 7, 8},
Amount: 32000000000,
Signature: []byte{13, 14, 15, 16},
Slot: 100,
},
want: false,
},
{
name: "different slot",
s: &ethpb.PendingDeposit{
PublicKey: []byte{1, 2, 3, 4},
WithdrawalCredentials: []byte{5, 6, 7, 8},
Amount: 32000000000,
Signature: []byte{9, 10, 11, 12},
Slot: 100,
},
t: &ethpb.PendingDeposit{
PublicKey: []byte{1, 2, 3, 4},
WithdrawalCredentials: []byte{5, 6, 7, 8},
Amount: 32000000000,
Signature: []byte{9, 10, 11, 12},
Slot: 200,
},
want: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := PendingDepositsEqual(tt.s, tt.t); got != tt.want {
t.Errorf("PendingDepositsEqual() = %v, want %v", got, tt.want)
}
})
}
}
func TestPendingPartialWithdrawalsEqual(t *testing.T) {
tests := []struct {
name string
s *ethpb.PendingPartialWithdrawal
t *ethpb.PendingPartialWithdrawal
want bool
}{
{
name: "both nil",
s: nil,
t: nil,
want: true,
},
{
name: "first nil",
s: nil,
t: &ethpb.PendingPartialWithdrawal{Index: 1},
want: false,
},
{
name: "second nil",
s: &ethpb.PendingPartialWithdrawal{Index: 1},
t: nil,
want: false,
},
{
name: "equal pending partial withdrawals",
s: &ethpb.PendingPartialWithdrawal{
Index: 50,
Amount: 1000000000,
WithdrawableEpoch: 200,
},
t: &ethpb.PendingPartialWithdrawal{
Index: 50,
Amount: 1000000000,
WithdrawableEpoch: 200,
},
want: true,
},
{
name: "different index",
s: &ethpb.PendingPartialWithdrawal{
Index: 50,
Amount: 1000000000,
WithdrawableEpoch: 200,
},
t: &ethpb.PendingPartialWithdrawal{
Index: 75,
Amount: 1000000000,
WithdrawableEpoch: 200,
},
want: false,
},
{
name: "different amount",
s: &ethpb.PendingPartialWithdrawal{
Index: 50,
Amount: 1000000000,
WithdrawableEpoch: 200,
},
t: &ethpb.PendingPartialWithdrawal{
Index: 50,
Amount: 2000000000,
WithdrawableEpoch: 200,
},
want: false,
},
{
name: "different withdrawable epoch",
s: &ethpb.PendingPartialWithdrawal{
Index: 50,
Amount: 1000000000,
WithdrawableEpoch: 200,
},
t: &ethpb.PendingPartialWithdrawal{
Index: 50,
Amount: 1000000000,
WithdrawableEpoch: 300,
},
want: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := PendingPartialWithdrawalsEqual(tt.s, tt.t); got != tt.want {
t.Errorf("PendingPartialWithdrawalsEqual() = %v, want %v", got, tt.want)
}
})
}
}
func TestPendingConsolidationsEqual(t *testing.T) {
tests := []struct {
name string
s *ethpb.PendingConsolidation
t *ethpb.PendingConsolidation
want bool
}{
{
name: "both nil",
s: nil,
t: nil,
want: true,
},
{
name: "first nil",
s: nil,
t: &ethpb.PendingConsolidation{SourceIndex: 1},
want: false,
},
{
name: "second nil",
s: &ethpb.PendingConsolidation{SourceIndex: 1},
t: nil,
want: false,
},
{
name: "equal pending consolidations",
s: &ethpb.PendingConsolidation{
SourceIndex: 10,
TargetIndex: 20,
},
t: &ethpb.PendingConsolidation{
SourceIndex: 10,
TargetIndex: 20,
},
want: true,
},
{
name: "different source index",
s: &ethpb.PendingConsolidation{
SourceIndex: 10,
TargetIndex: 20,
},
t: &ethpb.PendingConsolidation{
SourceIndex: 15,
TargetIndex: 20,
},
want: false,
},
{
name: "different target index",
s: &ethpb.PendingConsolidation{
SourceIndex: 10,
TargetIndex: 20,
},
t: &ethpb.PendingConsolidation{
SourceIndex: 10,
TargetIndex: 25,
},
want: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := PendingConsolidationsEqual(tt.s, tt.t); got != tt.want {
t.Errorf("PendingConsolidationsEqual() = %v, want %v", got, tt.want)
}
})
}
}

View File

@@ -31,7 +31,7 @@ go_test(
deps = [
":go_default_library",
"//encoding/ssz/query/testutil:go_default_library",
"//proto/ssz_query:go_default_library",
"//proto/ssz_query/testing:go_default_library",
"//testing/require:go_default_library",
"@com_github_prysmaticlabs_go_bitfield//:go_default_library",
],

Some files were not shown because too many files have changed in this diff Show More