Compare commits

...

47 Commits

Author SHA1 Message Date
aarshkshah1992
08f038fe80 fix lint 2025-11-13 18:57:18 +04:00
aarshkshah1992
63279bcadf revert new line change 2025-11-13 17:06:47 +04:00
aarshkshah1992
61628efd44 run CI 2025-11-13 17:04:41 +04:00
aarshkshah1992
9b07f13cd3 tests for the crawler 2025-11-13 16:58:49 +04:00
aarshkshah1992
1397a79b4c changelog 2025-11-12 17:09:33 +04:00
aarshkshah1992
929115639d draft 2025-11-12 16:38:07 +04:00
aarshkshah1992
14dca40786 draft 2025-11-12 16:36:53 +04:00
aarshkshah1992
fa7596bceb first draft 2025-11-12 12:54:41 +04:00
aarshkshah1992
5161f087fc fix compilation 2025-11-06 19:38:59 +04:00
aarshkshah1992
71050ab076 fix bazel 2025-11-06 19:10:19 +04:00
aarshkshah1992
614367ddcf fix lint 2025-11-05 22:26:55 +04:00
aarshkshah1992
3f7371445b fix test in sync 2025-11-05 22:23:26 +04:00
aarshkshah1992
a15a1ade17 fix test 2025-11-05 21:31:04 +04:00
aarshkshah1992
798376b1d7 fix bazel 2025-11-05 21:21:51 +04:00
aarshkshah1992
93271050bf more tests 2025-11-05 21:17:03 +04:00
aarshkshah1992
8dfbabc691 fork watcher test works 2025-11-05 14:45:35 +04:00
aarshkshah1992
af2522e5f0 fix schedule 2025-11-05 09:49:27 +04:00
aarshkshah1992
452d42bd10 fix test in sync 2025-11-05 09:21:34 +04:00
aarshkshah1992
3e985377ce fix test 2025-11-05 09:05:57 +04:00
aarshkshah1992
ab2e836d3f fix test 2025-11-04 21:11:45 +04:00
Aarsh Shah
14158bea9c Merge branch 'develop' into feat/use-topic-abstraction-for-gossipsub-and-refactor-fork-watcher 2025-11-04 12:02:25 -05:00
Jun Song
165c4b0af1 Handle addition overflow in /eth/v1/beacon/rewards/attestations/{epoch} (#15970)
* Handle addition overflow in `/eth/v1/beacon/rewards/attestations/{epoch}`

* Changelog
2025-11-04 16:14:47 +00:00
aarshkshah1992
e14590636f bazel gazelle 2025-11-04 20:08:41 +04:00
aarshkshah1992
ce3660d2e7 finish draft 2025-11-04 20:08:11 +04:00
terence
d0f5253b8d Update go-bitfield from prysmaticlabs to OffchainLabs (#15968) 2025-11-03 23:59:03 +00:00
terence
1f926142b8 Update spec test to v1.6.0-beta.2 (#15960) 2025-11-03 17:54:02 +00:00
Manu NALEPA
d394f00e9f beacon_data_column_sidecar_gossip_verification_milliseconds: Adjust buckets. (#15964)
* `beacon_data_column_sidecar_gossip_verification_milliseconds`: Divide by 10.

* Fix Kasey's comment.
2025-11-03 17:00:37 +00:00
aarshkshah1992
7853cb9db0 changelog fragment 2025-11-03 19:14:46 +04:00
Muzry
11c6325b54 use filepath to perform path operations correctly on Windows (#15953) 2025-11-03 15:14:08 +00:00
aarshkshah1992
8cfeda1473 WIP gossipsub controller 2025-11-03 19:02:41 +04:00
Manu NALEPA
ec524ce99c RODataColumnsVerifier.ValidProposerSignature: Ensure the expensive signature verification is only performed once for concurrent requests for the same signature data. (#15954)
* `signatureData`: Add `string` function.

* `RODataColumnsVerifier.ValidProposerSignature`: Ensure the expensive signature verification is only performed once for concurrent requests for the same signature data.

Share flight group

* `parentState` ==> `state`.

* `RODataColumnsVerifier.SidecarProposerExpected: Ensure the expensive index computation is only performed once for concurrent requests.`

* Add `wrapAttestationError`

* Fix Kasey's comment.

* Fix Terence's comment.
2025-11-03 14:48:41 +00:00
Manu NALEPA
b2a9db0826 BeaconBlockContainerToSignedBeaconBlock: Add Fulu. (#15940) 2025-11-01 23:18:42 +00:00
Manu NALEPA
040661bd68 Update go-netroute to v0.4.0. (#15949) 2025-10-31 20:46:06 +00:00
fernantho
d3bd0eaa30 SSZ-QL: update "path parsing" data types (#15935)
* updated path processing data types, refactored ParsePath and fixed tests

* updated generalized index accordingly, changed input parameter path type from []PathElemen to Path

* updated query.go accordingly, changed input parameter path type from []PathElemen to Path

* added descriptive changelog

* Update encoding/ssz/query/path.go

Co-authored-by: Jun Song <87601811+syjn99@users.noreply.github.com>

* Added documentation for Path struct and renamed  to  for clarity

* Update encoding/ssz/query/path.go

Co-authored-by: Radosław Kapka <radoslaw.kapka@gmail.com>

* updated changelog to its correct type: Changed

* updated outdated comment in generalized_index.go and removed test in generalized_index_test.go as this one belongs in path_test.go

* Added validateRawPath with strict raw-path validation only - no raw-path fixing is added. Added test suite covering

* added extra tests for wrongly formated paths

---------

Co-authored-by: Jun Song <87601811+syjn99@users.noreply.github.com>
Co-authored-by: Radosław Kapka <radoslaw.kapka@gmail.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2025-10-31 17:37:59 +00:00
Preston Van Loon
577899bfec P2p active val count lock (#15955)
* Add a lock for p2p computation of active validator count and limit only to topics that need it.

* Changelog fragment

* Update gossip_scoring_params.go

Wrap errors
2025-10-31 15:25:18 +00:00
Muzry
374bae9c81 Fix incorrect version used when sending attestation version in Fulu (#15950)
* Fix incorrect version used when sending attestation version in Fulu

* update typo

* fix Eth-Consensus-Version in submit_signed_aggregate_proof.go
2025-10-31 13:17:44 +00:00
kasey
3e0492a636 also ignore errors from readdirnames (#15947)
* also ignore errors from readdirnames

* test case for empty blobs dir

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2025-10-30 19:02:25 +00:00
rocksload
5a1a5b5ae5 refactor: use slices.Contains to simplify code (#15646)
Signed-off-by: rocksload <rocksload@outlook.com>
2025-10-29 14:40:33 +00:00
james-prysm
dbb2f0b047 changelog (#15929) 2025-10-28 15:42:05 +00:00
Manu NALEPA
7b3c11c818 Do not serve sidecars if corresponding block is not available in the database (#15933)
* Implement `AvailableBlocks`.

* `blobSidecarByRootRPCHandler`: Do not serve a sidecar if the corresponding block is not available.

* `dataColumnSidecarByRootRPCHandler`: Do not do extra work if only needed for TRACE logging.

* `TestDataColumnSidecarsByRootRPCHandler`: Re-arrange (no functional change).

* `TestDataColumnSidecarsByRootRPCHandler`: Save blocks corresponding to sidecars into DB.

* `dataColumnSidecarByRootRPCHandler`: Do not serve a sidecar if the corresponding block is not available.

* Add changelog

* `TestDataColumnSidecarsByRootRPCHandler`: Use `assert` instead of `require` in goroutines.

https://github.com/stretchr/testify?tab=readme-ov-file#require-package
2025-10-28 15:39:35 +00:00
Manu NALEPA
c9b34d556d Update go-netroute to v0.3.0 (#15934) 2025-10-28 12:57:14 +00:00
fernantho
10a2f0687b SSZ-QL: calculate generalized indices for elements (#15873)
* added tests for calculating generalized indices

* added first version of GI calculation walking the specified path with no recursion. Extended test coverage for bitlist and bitvectors.
vectors need more testing

* refactored code. Detached PathElement processing, currently done at the beginning. Swap to regex to gain flexibility.

* added an updateRoot function with the GI formula. more refactoring

* added changelog

* replaced TODO tag

* udpated some comments

* simplified code - removed duplicated code in processingLengthField function

* run gazelle

* merging all input path processing into path.go

* reviewed Jun's feedback

* removed unnecessary idx pointer var + fixed error with length data type (uint64 instead of uint8)

* refactored path.go after merging path elements from generalized_indices.go

* re-computed GIs for tests as VariableTestContainer added a new field.

* added minor comment - rawPath MUST be snake case

removed extractFieldName func.

* fixed vector GI calculation - updated tests GIs

* removed updateRoot function in favor of inline code

* path input data enforced to be snake case

* added sanity checks for accessing outbound element indices - checked against vector.length/list.limit

* fixed issues triggered after merging develop

* Removed redundant comment

Co-authored-by: Jun Song <87601811+syjn99@users.noreply.github.com>

* removed unreachable condition as `strings.Split` always return a slice with length >= 1

If s does not contain sep and sep is not empty, Split returns a slice of
length 1 whose only element is s.

* added tests to cover edge cases + cleaned code (toLower is no longer needed in extractFieldName function

* added Jun's feedback + more testing

* postponed snake case conversion to do it on a per-element-basis. Added more testing focused mainly in snake case conversion

* addressed several Jun's comments.

* added sanity check to prevent length of a multi-dimensional array. added more tests with extended paths

* Update encoding/ssz/query/generalized_index.go

Co-authored-by: Radosław Kapka <radoslaw.kapka@gmail.com>

* Update encoding/ssz/query/generalized_index.go

Co-authored-by: Radosław Kapka <radoslaw.kapka@gmail.com>

* Update encoding/ssz/query/generalized_index.go

Co-authored-by: Radosław Kapka <radoslaw.kapka@gmail.com>

* placed constant bitsPerChunk in the right place. Exported BitsPerChunk and BytesPerChunk and updated code that use them

* added helpers for computing GI of each data type

* changed %q in favor of %s

* Update encoding/ssz/query/path.go

Co-authored-by: Jun Song <87601811+syjn99@users.noreply.github.com>

* removed the least restrictive condition isBasicType

* replaced length of containerInfo.order for containerInfo.fields for clarity

* removed outdated comment

* removed toSnakeCase conversion.

* moved isBasicType func to its natural place, SSZType

* cosmetic refactor

- renamed itemLengthFromInfo to itemLength (same name is in spec).
- arranged all SSZ helpers.

* cleaned tests

* renamed "root" to "index"

* removed unnecessary check for negative integers. Replaced %q for %s.

* refactored regex variables and prevented re-assignation

* added length regex explanation

* added more testing for stressing regex for path processing

* renamed currentIndex to parentIndex for clarity and documented the returns from calculate<Type>GeneralizedIndex functions

* Update encoding/ssz/query/generalized_index.go

Co-authored-by: Radosław Kapka <radoslaw.kapka@gmail.com>

* run gazelle

* fixed never asserted error. Updated error message

---------

Co-authored-by: Jun Song <87601811+syjn99@users.noreply.github.com>
Co-authored-by: Radosław Kapka <radoslaw.kapka@gmail.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2025-10-27 23:27:34 +00:00
Manu NALEPA
4fb75d6d0b Add some metrics improvements (#15922)
* Define TCP and QUIC as `InternetProtocol` (no functional change).

* Group types. (No functional changes)

* Rename variables and use range syntax.

* Add `p2pMaxPeers` and `p2pPeerCountDirectionType` metrics

* `p2p_subscribed_topic_peer_total`: Reset to avoid dangling values.

* `validateConfig`:
- Use `Warning` with fields instead of `Warnf`.
- Avoid to both modify in place the input value and return it.

* Add `p2p_minimum_peers_per_subnet` metric.

* `beaconConfig` => `cfg`.

https://github.com/OffchainLabs/prysm/pull/15880#discussion_r2436826215

* Add changelog

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2025-10-26 15:16:05 +00:00
terence
6d596edea2 Use SlotTicker instead of time.Ticker for attestation pool pruning (#15917)
* Use SlotTicker instead of time.Ticker for attestation pool pruning

* Offset one second before slot start
2025-10-24 15:35:26 +00:00
Bastin
9153c5a202 light client logging (#15927) 2025-10-24 14:42:27 +00:00
james-prysm
26ce94e224 removes misleading keymanager info log (#15926)
* simple change

* fixing test"
"
2025-10-24 14:28:30 +00:00
terence
255ea2fac1 Return optimistic response only when handling blinded blocks (#15925)
* Return optimistic response only when handling blinded blocks in proposer

* Remove blind condition
2025-10-24 03:37:32 +00:00
273 changed files with 5741 additions and 1155 deletions

View File

@@ -4,6 +4,29 @@ All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
## [v6.1.4](https://github.com/prysmaticlabs/prysm/compare/v6.1.3...v6.1.4) - 2025-10-24
This release includes a bug fix affecting block proposals in rare cases, along with an important update for Windows users running post-Fusaka fork.
### Added
- SSZ-QL: Add endpoints for `BeaconState`/`BeaconBlock`. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15888)
- Add native state diff type and marshalling functions. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15250)
- Update the earliest available slot after pruning operations in beacon chain database pruner. This ensures the P2P layer accurately knows which historical data is available after pruning, preventing nodes from advertising or attempting to serve data that has been pruned. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15694)
### Fixed
- Correctly advertise (in ENR and beacon API) attestation subnets when using `--subscribe-all-subnets`. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15880)
- `randomPeer`: Return if the context is cancelled when waiting for peers. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15876)
- Improve error message when the byte count read from disk when reading a data column sidecars is lower than expected. (Mostly, because the file is truncated.). [[PR]](https://github.com/prysmaticlabs/prysm/pull/15881)
- Delete the genesis state file when --clear-db / --force-clear-db is specified. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15883)
- Fix sync committee subscription to use subnet indices instead of committee indices. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15885)
- Fixed metadata extraction on Windows by correctly splitting file paths. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15899)
- `VerifyDataColumnsSidecarKZGProofs`: Check if sizes match. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15892)
- Fix recoverStateSummary to persist state summaries in stateSummaryBucket instead of stateBucket (#15896). [[PR]](https://github.com/prysmaticlabs/prysm/pull/15896)
- `updateCustodyInfoInDB`: Use `NumberOfCustodyGroups` instead of `NumberOfColumns`. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15908)
- Sync committee uses correct state to calculate position. [[PR]](https://github.com/prysmaticlabs/prysm/pull/15905)
## [v6.1.3](https://github.com/prysmaticlabs/prysm/compare/v6.1.2...v6.1.3) - 2025-10-20
This release has several important beacon API and p2p fixes.

View File

@@ -253,7 +253,7 @@ filegroup(
url = "https://github.com/ethereum/EIPs/archive/5480440fe51742ed23342b68cf106cefd427e39d.tar.gz",
)
consensus_spec_version = "v1.6.0-beta.1"
consensus_spec_version = "v1.6.0-beta.2"
load("@prysm//tools:download_spectests.bzl", "consensus_spec_tests")
@@ -278,7 +278,7 @@ filegroup(
visibility = ["//visibility:public"],
)
""",
integrity = "sha256-yrq3tdwPS8Ri+ueeLAHssIT3ssMrX7zvHiJ8Xf9GVYs=",
integrity = "sha256-MForEP9dTe0z3ZkTHjX4H6waSkSTghf3gQHPwrSCCro=",
strip_prefix = "consensus-specs-" + consensus_spec_version[1:],
url = "https://github.com/ethereum/consensus-specs/archive/refs/tags/%s.tar.gz" % consensus_spec_version,
)

View File

@@ -10,6 +10,7 @@ import (
"net/url"
"testing"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/api"
"github.com/OffchainLabs/prysm/v6/api/server/structs"
"github.com/OffchainLabs/prysm/v6/config/params"
@@ -22,7 +23,6 @@ import (
"github.com/OffchainLabs/prysm/v6/testing/assert"
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/OffchainLabs/prysm/v6/testing/util"
"github.com/prysmaticlabs/go-bitfield"
log "github.com/sirupsen/logrus"
)

View File

@@ -11,6 +11,7 @@ import (
"os"
"testing"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/api/server/structs"
fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
consensusblocks "github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
@@ -22,7 +23,6 @@ import (
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/ethereum/go-ethereum/common/hexutil"
"github.com/pkg/errors"
"github.com/prysmaticlabs/go-bitfield"
)
func ezDecode(t *testing.T, s string) []byte {

View File

@@ -5,6 +5,7 @@ import (
"fmt"
"time"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/blocks"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
@@ -29,7 +30,6 @@ import (
"github.com/OffchainLabs/prysm/v6/runtime/version"
"github.com/OffchainLabs/prysm/v6/time/slots"
"github.com/pkg/errors"
"github.com/prysmaticlabs/go-bitfield"
"github.com/sirupsen/logrus"
)

View File

@@ -9,6 +9,7 @@ import (
"testing"
"time"
"github.com/OffchainLabs/go-bitfield"
mock "github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/testing"
"github.com/OffchainLabs/prysm/v6/beacon-chain/cache"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/blocks"
@@ -49,7 +50,6 @@ import (
"github.com/ethereum/go-ethereum/common"
gethtypes "github.com/ethereum/go-ethereum/core/types"
"github.com/pkg/errors"
"github.com/prysmaticlabs/go-bitfield"
logTest "github.com/sirupsen/logrus/hooks/test"
)

View File

@@ -472,8 +472,8 @@ func (s *Service) removeStartupState() {
func (s *Service) updateCustodyInfoInDB(slot primitives.Slot) (primitives.Slot, uint64, error) {
isSubscribedToAllDataSubnets := flags.Get().SubscribeAllDataSubnets
beaconConfig := params.BeaconConfig()
custodyRequirement := beaconConfig.CustodyRequirement
cfg := params.BeaconConfig()
custodyRequirement := cfg.CustodyRequirement
// Check if the node was previously subscribed to all data subnets, and if so,
// store the new status accordingly.
@@ -493,7 +493,7 @@ func (s *Service) updateCustodyInfoInDB(slot primitives.Slot) (primitives.Slot,
// Compute the custody group count.
custodyGroupCount := custodyRequirement
if isSubscribedToAllDataSubnets {
custodyGroupCount = beaconConfig.NumberOfCustodyGroups
custodyGroupCount = cfg.NumberOfCustodyGroups
}
// Safely compute the fulu fork slot.
@@ -536,11 +536,11 @@ func spawnCountdownIfPreGenesis(ctx context.Context, genesisTime time.Time, db d
}
func fuluForkSlot() (primitives.Slot, error) {
beaconConfig := params.BeaconConfig()
cfg := params.BeaconConfig()
fuluForkEpoch := beaconConfig.FuluForkEpoch
if fuluForkEpoch == beaconConfig.FarFutureEpoch {
return beaconConfig.FarFutureSlot, nil
fuluForkEpoch := cfg.FuluForkEpoch
if fuluForkEpoch == cfg.FarFutureEpoch {
return cfg.FarFutureSlot, nil
}
forkFuluSlot, err := slots.EpochStart(fuluForkEpoch)

View File

@@ -472,6 +472,36 @@ func (s *ChainService) HasBlock(ctx context.Context, rt [32]byte) bool {
return s.InitSyncBlockRoots[rt]
}
func (s *ChainService) AvailableBlocks(ctx context.Context, blockRoots [][32]byte) map[[32]byte]bool {
if s.DB == nil {
return nil
}
count := len(blockRoots)
availableRoots := make(map[[32]byte]bool, count)
notInDBRoots := make([][32]byte, 0, count)
for _, root := range blockRoots {
if s.DB.HasBlock(ctx, root) {
availableRoots[root] = true
continue
}
notInDBRoots = append(notInDBRoots, root)
}
if s.InitSyncBlockRoots == nil {
return availableRoots
}
for _, root := range notInDBRoots {
if s.InitSyncBlockRoots[root] {
availableRoots[root] = true
}
}
return availableRoots
}
// RecentBlockSlot mocks the same method in the chain service.
func (s *ChainService) RecentBlockSlot([32]byte) (primitives.Slot, error) {
return s.BlockSlot, nil

View File

@@ -3,6 +3,7 @@ package blockchain
import (
"context"
"fmt"
"slices"
"github.com/OffchainLabs/prysm/v6/beacon-chain/db/filters"
"github.com/OffchainLabs/prysm/v6/config/params"
@@ -81,12 +82,10 @@ func (v *WeakSubjectivityVerifier) VerifyWeakSubjectivity(ctx context.Context, f
if err != nil {
return errors.Wrap(err, "error while retrieving block roots to verify weak subjectivity")
}
for _, root := range roots {
if v.root == root {
log.Info("Weak subjectivity check has passed!!")
v.verified = true
return nil
}
if slices.Contains(roots, v.root) {
log.Info("Weak subjectivity check has passed!!")
v.verified = true
return nil
}
return errors.Wrap(errWSBlockNotFoundInEpoch, fmt.Sprintf("root=%#x, epoch=%d", v.root, v.epoch))
}

View File

@@ -3,6 +3,7 @@ package cache
import (
"testing"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
"github.com/OffchainLabs/prysm/v6/crypto/bls/blst"
"github.com/OffchainLabs/prysm/v6/encoding/bytesutil"
@@ -10,7 +11,6 @@ import (
"github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1/attestation"
"github.com/OffchainLabs/prysm/v6/testing/assert"
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/prysmaticlabs/go-bitfield"
)
func TestAdd(t *testing.T) {

View File

@@ -4,6 +4,7 @@ import (
"fmt"
"testing"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/altair"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/signing"
@@ -22,7 +23,6 @@ import (
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/OffchainLabs/prysm/v6/testing/util"
gofuzz "github.com/google/gofuzz"
"github.com/prysmaticlabs/go-bitfield"
)
func TestProcessAttestations_InclusionDelayFailure(t *testing.T) {

View File

@@ -4,6 +4,7 @@ import (
"math"
"testing"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/altair"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/signing"
@@ -19,7 +20,6 @@ import (
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/OffchainLabs/prysm/v6/testing/util"
"github.com/OffchainLabs/prysm/v6/time/slots"
"github.com/prysmaticlabs/go-bitfield"
)
func TestProcessSyncCommittee_PerfectParticipation(t *testing.T) {

View File

@@ -3,6 +3,7 @@ package altair_test
import (
"testing"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/altair"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/time"
@@ -12,7 +13,6 @@ import (
"github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1/attestation"
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/OffchainLabs/prysm/v6/testing/util"
"github.com/prysmaticlabs/go-bitfield"
)
func TestTranslateParticipation(t *testing.T) {

View File

@@ -5,6 +5,7 @@ import (
"os"
"testing"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/blocks"
state_native "github.com/OffchainLabs/prysm/v6/beacon-chain/state/state-native"
"github.com/OffchainLabs/prysm/v6/config/params"
@@ -12,7 +13,6 @@ import (
"github.com/OffchainLabs/prysm/v6/testing/assert"
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/OffchainLabs/prysm/v6/testing/util"
"github.com/prysmaticlabs/go-bitfield"
)
// Beaconfuzz discovered an off by one issue where an attestation could be produced which would pass

View File

@@ -4,6 +4,7 @@ import (
"context"
"testing"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/blocks"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/signing"
@@ -20,7 +21,6 @@ import (
"github.com/OffchainLabs/prysm/v6/testing/assert"
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/OffchainLabs/prysm/v6/testing/util"
"github.com/prysmaticlabs/go-bitfield"
)
func TestProcessAggregatedAttestation_OverlappingBits(t *testing.T) {

View File

@@ -3,6 +3,7 @@ package precompute_test
import (
"testing"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/epoch/precompute"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
"github.com/OffchainLabs/prysm/v6/config/params"
@@ -12,7 +13,6 @@ import (
"github.com/OffchainLabs/prysm/v6/testing/assert"
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/OffchainLabs/prysm/v6/testing/util"
"github.com/prysmaticlabs/go-bitfield"
)
func TestUpdateValidator_Works(t *testing.T) {

View File

@@ -1,6 +1,7 @@
package precompute
import (
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/time"
"github.com/OffchainLabs/prysm/v6/beacon-chain/state"
@@ -8,7 +9,6 @@ import (
ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
"github.com/OffchainLabs/prysm/v6/time/slots"
"github.com/pkg/errors"
"github.com/prysmaticlabs/go-bitfield"
)
var errNilState = errors.New("nil state")

View File

@@ -3,6 +3,7 @@ package precompute_test
import (
"testing"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/altair"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/epoch/precompute"
state_native "github.com/OffchainLabs/prysm/v6/beacon-chain/state/state-native"
@@ -12,7 +13,6 @@ import (
ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
"github.com/OffchainLabs/prysm/v6/testing/assert"
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/prysmaticlabs/go-bitfield"
)
func TestProcessJustificationAndFinalizationPreCompute_ConsecutiveEpochs(t *testing.T) {

View File

@@ -3,6 +3,7 @@ package precompute
import (
"testing"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/time"
"github.com/OffchainLabs/prysm/v6/beacon-chain/state"
@@ -16,7 +17,6 @@ import (
"github.com/OffchainLabs/prysm/v6/testing/assert"
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/pkg/errors"
"github.com/prysmaticlabs/go-bitfield"
)
func TestProcessRewardsAndPenaltiesPrecompute(t *testing.T) {

View File

@@ -7,6 +7,7 @@ import (
"fmt"
"sort"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/beacon-chain/cache"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/time"
forkchoicetypes "github.com/OffchainLabs/prysm/v6/beacon-chain/forkchoice/types"
@@ -23,7 +24,6 @@ import (
"github.com/OffchainLabs/prysm/v6/runtime/version"
"github.com/OffchainLabs/prysm/v6/time/slots"
"github.com/pkg/errors"
"github.com/prysmaticlabs/go-bitfield"
)
var (

View File

@@ -5,6 +5,7 @@ import (
"strconv"
"testing"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/time"
state_native "github.com/OffchainLabs/prysm/v6/beacon-chain/state/state-native"
@@ -18,7 +19,6 @@ import (
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/OffchainLabs/prysm/v6/testing/util"
"github.com/OffchainLabs/prysm/v6/time/slots"
"github.com/prysmaticlabs/go-bitfield"
)
func TestComputeCommittee_WithoutCache(t *testing.T) {

View File

@@ -401,7 +401,7 @@ func ComputeProposerIndex(bState state.ReadOnlyBeaconState, activeIndices []prim
return 0, errors.New("empty active indices list")
}
hashFunc := hash.CustomSHA256Hasher()
beaconConfig := params.BeaconConfig()
cfg := params.BeaconConfig()
seedBuffer := make([]byte, len(seed)+8)
copy(seedBuffer, seed[:])
@@ -426,14 +426,14 @@ func ComputeProposerIndex(bState state.ReadOnlyBeaconState, activeIndices []prim
offset := (i % 16) * 2
randomValue := uint64(randomBytes[offset]) | uint64(randomBytes[offset+1])<<8
if effectiveBal*fieldparams.MaxRandomValueElectra >= beaconConfig.MaxEffectiveBalanceElectra*randomValue {
if effectiveBal*fieldparams.MaxRandomValueElectra >= cfg.MaxEffectiveBalanceElectra*randomValue {
return candidateIndex, nil
}
} else {
binary.LittleEndian.PutUint64(seedBuffer[len(seed):], i/32)
randomByte := hashFunc(seedBuffer)[i%32]
if effectiveBal*fieldparams.MaxRandomByte >= beaconConfig.MaxEffectiveBalance*uint64(randomByte) {
if effectiveBal*fieldparams.MaxRandomByte >= cfg.MaxEffectiveBalance*uint64(randomByte) {
return candidateIndex, nil
}
}

View File

@@ -285,4 +285,3 @@ func genState(t *testing.T, valCount, avgBalance uint64) state.BeaconState {
return beaconState
}

View File

@@ -89,14 +89,14 @@ func CustodyGroups(nodeId enode.ID, custodyGroupCount uint64) ([]uint64, error)
// ComputeColumnsForCustodyGroup computes the columns for a given custody group.
// https://github.com/ethereum/consensus-specs/blob/master/specs/fulu/das-core.md#compute_columns_for_custody_group
func ComputeColumnsForCustodyGroup(custodyGroup uint64) ([]uint64, error) {
beaconConfig := params.BeaconConfig()
numberOfCustodyGroups := beaconConfig.NumberOfCustodyGroups
cfg := params.BeaconConfig()
numberOfCustodyGroups := cfg.NumberOfCustodyGroups
if custodyGroup >= numberOfCustodyGroups {
return nil, ErrCustodyGroupTooLarge
}
numberOfColumns := beaconConfig.NumberOfColumns
numberOfColumns := cfg.NumberOfColumns
columnsPerGroup := numberOfColumns / numberOfCustodyGroups
@@ -112,9 +112,9 @@ func ComputeColumnsForCustodyGroup(custodyGroup uint64) ([]uint64, error) {
// ComputeCustodyGroupForColumn computes the custody group for a given column.
// It is the reciprocal function of ComputeColumnsForCustodyGroup.
func ComputeCustodyGroupForColumn(columnIndex uint64) (uint64, error) {
beaconConfig := params.BeaconConfig()
numberOfColumns := beaconConfig.NumberOfColumns
numberOfCustodyGroups := beaconConfig.NumberOfCustodyGroups
cfg := params.BeaconConfig()
numberOfColumns := cfg.NumberOfColumns
numberOfCustodyGroups := cfg.NumberOfCustodyGroups
if columnIndex >= numberOfColumns {
return 0, ErrIndexTooLarge

View File

@@ -84,10 +84,10 @@ func ValidatorsCustodyRequirement(state beaconState.ReadOnlyBeaconState, validat
totalNodeBalance += validator.EffectiveBalance()
}
beaconConfig := params.BeaconConfig()
numberOfCustodyGroups := beaconConfig.NumberOfCustodyGroups
validatorCustodyRequirement := beaconConfig.ValidatorCustodyRequirement
balancePerAdditionalCustodyGroup := beaconConfig.BalancePerAdditionalCustodyGroup
cfg := params.BeaconConfig()
numberOfCustodyGroups := cfg.NumberOfCustodyGroups
validatorCustodyRequirement := cfg.ValidatorCustodyRequirement
balancePerAdditionalCustodyGroup := cfg.BalancePerAdditionalCustodyGroup
count := totalNodeBalance / balancePerAdditionalCustodyGroup
return min(max(count, validatorCustodyRequirement), numberOfCustodyGroups), nil

View File

@@ -196,7 +196,7 @@ func TestAltairCompatible(t *testing.T) {
}
func TestCanUpgradeTo(t *testing.T) {
beaconConfig := params.BeaconConfig()
cfg := params.BeaconConfig()
outerTestCases := []struct {
name string
@@ -205,32 +205,32 @@ func TestCanUpgradeTo(t *testing.T) {
}{
{
name: "Altair",
forkEpoch: &beaconConfig.AltairForkEpoch,
forkEpoch: &cfg.AltairForkEpoch,
upgradeFunc: time.CanUpgradeToAltair,
},
{
name: "Bellatrix",
forkEpoch: &beaconConfig.BellatrixForkEpoch,
forkEpoch: &cfg.BellatrixForkEpoch,
upgradeFunc: time.CanUpgradeToBellatrix,
},
{
name: "Capella",
forkEpoch: &beaconConfig.CapellaForkEpoch,
forkEpoch: &cfg.CapellaForkEpoch,
upgradeFunc: time.CanUpgradeToCapella,
},
{
name: "Deneb",
forkEpoch: &beaconConfig.DenebForkEpoch,
forkEpoch: &cfg.DenebForkEpoch,
upgradeFunc: time.CanUpgradeToDeneb,
},
{
name: "Electra",
forkEpoch: &beaconConfig.ElectraForkEpoch,
forkEpoch: &cfg.ElectraForkEpoch,
upgradeFunc: time.CanUpgradeToElectra,
},
{
name: "Fulu",
forkEpoch: &beaconConfig.FuluForkEpoch,
forkEpoch: &cfg.FuluForkEpoch,
upgradeFunc: time.CanUpgradeToFulu,
},
}
@@ -238,7 +238,7 @@ func TestCanUpgradeTo(t *testing.T) {
for _, otc := range outerTestCases {
params.SetupTestConfigCleanup(t)
*otc.forkEpoch = 5
params.OverrideBeaconConfig(beaconConfig)
params.OverrideBeaconConfig(cfg)
innerTestCases := []struct {
name string

View File

@@ -4,6 +4,7 @@ import (
"math"
"testing"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/altair"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/signing"
@@ -20,7 +21,6 @@ import (
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/OffchainLabs/prysm/v6/testing/util"
"github.com/OffchainLabs/prysm/v6/time/slots"
"github.com/prysmaticlabs/go-bitfield"
)
func TestExecuteAltairStateTransitionNoVerify_FullProcess(t *testing.T) {

View File

@@ -4,6 +4,7 @@ import (
"math"
"testing"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/altair"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/signing"
@@ -22,7 +23,6 @@ import (
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/OffchainLabs/prysm/v6/testing/util"
"github.com/OffchainLabs/prysm/v6/time/slots"
"github.com/prysmaticlabs/go-bitfield"
)
func TestExecuteBellatrixStateTransitionNoVerify_FullProcess(t *testing.T) {

View File

@@ -4,6 +4,7 @@ import (
"fmt"
"testing"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/blocks"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/signing"
@@ -23,7 +24,6 @@ import (
"github.com/OffchainLabs/prysm/v6/testing/assert"
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/OffchainLabs/prysm/v6/testing/util"
"github.com/prysmaticlabs/go-bitfield"
)
func init() {

View File

@@ -28,6 +28,7 @@ type ReadOnlyDatabase interface {
BlocksBySlot(ctx context.Context, slot primitives.Slot) ([]interfaces.ReadOnlySignedBeaconBlock, error)
BlockRootsBySlot(ctx context.Context, slot primitives.Slot) (bool, [][32]byte, error)
HasBlock(ctx context.Context, blockRoot [32]byte) bool
AvailableBlocks(ctx context.Context, blockRoots [][32]byte) map[[32]byte]bool
GenesisBlock(ctx context.Context) (interfaces.ReadOnlySignedBeaconBlock, error)
GenesisBlockRoot(ctx context.Context) ([32]byte, error)
IsFinalizedBlock(ctx context.Context, blockRoot [32]byte) bool

View File

@@ -336,6 +336,42 @@ func (s *Store) HasBlock(ctx context.Context, blockRoot [32]byte) bool {
return exists
}
// AvailableBlocks returns a set of roots indicating which blocks corresponding to `blockRoots` are available in the storage.
func (s *Store) AvailableBlocks(ctx context.Context, blockRoots [][32]byte) map[[32]byte]bool {
_, span := trace.StartSpan(ctx, "BeaconDB.AvailableBlocks")
defer span.End()
count := len(blockRoots)
availableRoots := make(map[[32]byte]bool, count)
// First, check the cache for each block root.
notInCacheRoots := make([][32]byte, 0, count)
for _, root := range blockRoots {
if v, ok := s.blockCache.Get(string(root[:])); v != nil && ok {
availableRoots[root] = true
continue
}
notInCacheRoots = append(notInCacheRoots, root)
}
// Next, check the database for the remaining block roots.
if err := s.db.View(func(tx *bolt.Tx) error {
bkt := tx.Bucket(blocksBucket)
for _, root := range notInCacheRoots {
if bkt.Get(root[:]) != nil {
availableRoots[root] = true
}
}
return nil
}); err != nil {
panic(err) // lint:nopanic -- View never returns an error.
}
return availableRoots
}
// BlocksBySlot retrieves a list of beacon blocks and its respective roots by slot.
func (s *Store) BlocksBySlot(ctx context.Context, slot primitives.Slot) ([]interfaces.ReadOnlySignedBeaconBlock, error) {
ctx, span := trace.StartSpan(ctx, "BeaconDB.BlocksBySlot")

View File

@@ -656,6 +656,44 @@ func TestStore_BlocksCRUD_NoCache(t *testing.T) {
}
}
func TestAvailableBlocks(t *testing.T) {
ctx := t.Context()
db := setupDB(t)
b0, b1, b2 := util.NewBeaconBlock(), util.NewBeaconBlock(), util.NewBeaconBlock()
b0.Block.Slot, b1.Block.Slot, b2.Block.Slot = 10, 20, 30
sb0, err := blocks.NewSignedBeaconBlock(b0)
require.NoError(t, err)
r0, err := b0.Block.HashTreeRoot()
require.NoError(t, err)
// Save b0 but remove it from cache.
err = db.SaveBlock(ctx, sb0)
require.NoError(t, err)
db.blockCache.Del(string(r0[:]))
// b1 is not saved at all.
r1, err := b1.Block.HashTreeRoot()
require.NoError(t, err)
// Save b2 in cache and DB.
sb2, err := blocks.NewSignedBeaconBlock(b2)
require.NoError(t, err)
r2, err := b2.Block.HashTreeRoot()
require.NoError(t, err)
require.NoError(t, db.SaveBlock(ctx, sb2))
require.NoError(t, err)
expected := map[[32]byte]bool{r0: true, r2: true}
actual := db.AvailableBlocks(ctx, [][32]byte{r0, r1, r2})
require.Equal(t, len(expected), len(actual))
for i := range expected {
require.Equal(t, true, actual[i])
}
}
func TestStore_Blocks_FiltersCorrectly(t *testing.T) {
for _, tt := range blockTests {
t.Run(tt.name, func(t *testing.T) {

View File

@@ -6,6 +6,7 @@ go_library(
"cache.go",
"helpers.go",
"lightclient.go",
"log.go",
"store.go",
],
importpath = "github.com/OffchainLabs/prysm/v6/beacon-chain/light-client",

View File

@@ -0,0 +1,5 @@
package light_client
import "github.com/sirupsen/logrus"
var log = logrus.WithField("prefix", "light-client")

View File

@@ -14,7 +14,6 @@ import (
"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
"github.com/OffchainLabs/prysm/v6/time/slots"
"github.com/pkg/errors"
log "github.com/sirupsen/logrus"
)
var ErrLightClientBootstrapNotFound = errors.New("light client bootstrap not found")

View File

@@ -4,12 +4,12 @@ import (
"bytes"
"testing"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
"github.com/OffchainLabs/prysm/v6/encoding/bytesutil"
ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/OffchainLabs/prysm/v6/testing/util"
"github.com/prysmaticlabs/go-bitfield"
"github.com/sirupsen/logrus"
logTest "github.com/sirupsen/logrus/hooks/test"
)

View File

@@ -3,11 +3,11 @@ package monitor
import (
"testing"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/OffchainLabs/prysm/v6/testing/util"
"github.com/prysmaticlabs/go-bitfield"
logTest "github.com/sirupsen/logrus/hooks/test"
)

View File

@@ -12,6 +12,7 @@ import (
"os"
"os/signal"
"path/filepath"
"slices"
"strconv"
"strings"
"sync"
@@ -1135,10 +1136,8 @@ func (b *BeaconNode) registerLightClientStore() {
func hasNetworkFlag(cliCtx *cli.Context) bool {
for _, flag := range features.NetworkFlags {
for _, name := range flag.Names() {
if cliCtx.IsSet(name) {
return true
}
if slices.ContainsFunc(flag.Names(), cliCtx.IsSet) {
return true
}
}
return false

View File

@@ -4,6 +4,7 @@ import (
"sort"
"testing"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
"github.com/OffchainLabs/prysm/v6/crypto/bls"
ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
@@ -13,7 +14,6 @@ import (
"github.com/OffchainLabs/prysm/v6/testing/util"
c "github.com/patrickmn/go-cache"
"github.com/pkg/errors"
"github.com/prysmaticlabs/go-bitfield"
)
func TestKV_Aggregated_AggregateUnaggregatedAttestations(t *testing.T) {

View File

@@ -4,11 +4,11 @@ import (
"sort"
"testing"
"github.com/OffchainLabs/go-bitfield"
ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
"github.com/OffchainLabs/prysm/v6/testing/assert"
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/OffchainLabs/prysm/v6/testing/util"
"github.com/prysmaticlabs/go-bitfield"
)
func TestKV_BlockAttestation_CanSaveRetrieve(t *testing.T) {

View File

@@ -4,11 +4,11 @@ import (
"sort"
"testing"
"github.com/OffchainLabs/go-bitfield"
ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
"github.com/OffchainLabs/prysm/v6/testing/assert"
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/OffchainLabs/prysm/v6/testing/util"
"github.com/prysmaticlabs/go-bitfield"
)
func TestKV_Forkchoice_CanSaveRetrieve(t *testing.T) {

View File

@@ -1,11 +1,11 @@
package kv
import (
"github.com/OffchainLabs/go-bitfield"
ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
"github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1/attestation"
"github.com/patrickmn/go-cache"
"github.com/pkg/errors"
"github.com/prysmaticlabs/go-bitfield"
)
func (c *AttCaches) insertSeenBit(att ethpb.Att) error {

View File

@@ -3,11 +3,11 @@ package kv
import (
"testing"
"github.com/OffchainLabs/go-bitfield"
ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
"github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1/attestation"
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/OffchainLabs/prysm/v6/testing/util"
"github.com/prysmaticlabs/go-bitfield"
)
func TestAttCaches_hasSeenBit(t *testing.T) {

View File

@@ -5,6 +5,7 @@ import (
"sort"
"testing"
"github.com/OffchainLabs/go-bitfield"
fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
@@ -13,7 +14,6 @@ import (
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/OffchainLabs/prysm/v6/testing/util"
c "github.com/patrickmn/go-cache"
"github.com/prysmaticlabs/go-bitfield"
)
func TestKV_Unaggregated_UnaggregatedAttestations(t *testing.T) {

View File

@@ -5,6 +5,7 @@ import (
"context"
"time"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/config/features"
"github.com/OffchainLabs/prysm/v6/config/params"
"github.com/OffchainLabs/prysm/v6/monitoring/tracing/trace"
@@ -13,7 +14,6 @@ import (
attaggregation "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1/attestation/aggregation/attestations"
"github.com/OffchainLabs/prysm/v6/time/slots"
"github.com/pkg/errors"
"github.com/prysmaticlabs/go-bitfield"
)
// This prepares fork choice attestations by running batchForkChoiceAtts

View File

@@ -5,13 +5,13 @@ import (
"sort"
"testing"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/crypto/bls"
ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
attaggregation "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1/attestation/aggregation/attestations"
"github.com/OffchainLabs/prysm/v6/testing/assert"
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/OffchainLabs/prysm/v6/testing/util"
"github.com/prysmaticlabs/go-bitfield"
"google.golang.org/protobuf/proto"
)

View File

@@ -10,11 +10,13 @@ import (
// pruneExpired prunes attestations pool on every slot interval.
func (s *Service) pruneExpired() {
ticker := time.NewTicker(s.cfg.pruneInterval)
defer ticker.Stop()
secondsPerSlot := params.BeaconConfig().SecondsPerSlot
offset := time.Duration(secondsPerSlot-1) * time.Second
slotTicker := slots.NewSlotTickerWithOffset(s.genesisTime, offset, secondsPerSlot)
defer slotTicker.Done()
for {
select {
case <-ticker.C:
case <-slotTicker.C():
s.pruneExpiredAtts()
s.updateMetrics()
case <-s.ctx.Done():

View File

@@ -5,6 +5,7 @@ import (
"testing"
"time"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/async"
fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
"github.com/OffchainLabs/prysm/v6/config/params"
@@ -13,11 +14,12 @@ import (
"github.com/OffchainLabs/prysm/v6/testing/assert"
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/OffchainLabs/prysm/v6/testing/util"
"github.com/prysmaticlabs/go-bitfield"
)
func TestPruneExpired_Ticker(t *testing.T) {
ctx, cancel := context.WithTimeout(t.Context(), 3*time.Second)
// Need timeout longer than the offset (secondsPerSlot - 1) + some buffer
timeout := time.Duration(params.BeaconConfig().SecondsPerSlot+5) * time.Second
ctx, cancel := context.WithTimeout(t.Context(), timeout)
defer cancel()
s, err := NewService(ctx, &Config{

View File

@@ -15,6 +15,7 @@ go_library(
"fork_watcher.go",
"gossip_scoring_params.go",
"gossip_topic_mappings.go",
"gossipsub_peer_crawler.go",
"handshake.go",
"info.go",
"interfaces.go",
@@ -51,6 +52,7 @@ go_library(
"//beacon-chain/db:go_default_library",
"//beacon-chain/db/kv:go_default_library",
"//beacon-chain/p2p/encoder:go_default_library",
"//beacon-chain/p2p/gossipsubcrawler:go_default_library",
"//beacon-chain/p2p/peers:go_default_library",
"//beacon-chain/p2p/peers/peerdata:go_default_library",
"//beacon-chain/p2p/peers/scorers:go_default_library",
@@ -114,6 +116,7 @@ go_library(
"@com_github_prysmaticlabs_go_bitfield//:go_default_library",
"@com_github_sirupsen_logrus//:go_default_library",
"@org_golang_google_protobuf//proto:go_default_library",
"@org_golang_x_sync//semaphore:go_default_library",
],
)
@@ -129,6 +132,7 @@ go_test(
"fork_test.go",
"gossip_scoring_params_test.go",
"gossip_topic_mappings_test.go",
"gossipsub_peer_crawler_test.go",
"message_id_test.go",
"options_test.go",
"parameter_test.go",
@@ -154,6 +158,7 @@ go_test(
"//beacon-chain/core/signing:go_default_library",
"//beacon-chain/db/testing:go_default_library",
"//beacon-chain/p2p/encoder:go_default_library",
"//beacon-chain/p2p/gossipsubcrawler:go_default_library",
"//beacon-chain/p2p/peers:go_default_library",
"//beacon-chain/p2p/peers/peerdata:go_default_library",
"//beacon-chain/p2p/peers/scorers:go_default_library",
@@ -201,6 +206,7 @@ go_test(
"@com_github_pkg_errors//:go_default_library",
"@com_github_prysmaticlabs_go_bitfield//:go_default_library",
"@com_github_sirupsen_logrus//hooks/test:go_default_library",
"@com_github_stretchr_testify//require:go_default_library",
"@org_golang_google_protobuf//proto:go_default_library",
],
)

View File

@@ -131,7 +131,8 @@ func (s *Service) internalBroadcastAttestation(ctx context.Context, subnet uint6
s.subnetLocker(subnet).Lock()
defer s.subnetLocker(subnet).Unlock()
if err := s.FindAndDialPeersWithSubnets(ctx, AttestationSubnetTopicFormat, forkDigest, minimumPeersPerSubnetForBroadcast, map[uint64]bool{subnet: true}); err != nil {
builder := func(idx uint64) string { return attestationToTopic(idx, forkDigest) + s.Encoding().ProtocolSuffix() }
if err := s.FindAndDialPeersWithSubnets(ctx, builder, minimumPeersPerSubnetForBroadcast, map[uint64]bool{subnet: true}); err != nil {
return errors.Wrap(err, "find peers with subnets")
}
@@ -187,7 +188,8 @@ func (s *Service) broadcastSyncCommittee(ctx context.Context, subnet uint64, sMs
if err := func() error {
s.subnetLocker(wrappedSubIdx).Lock()
defer s.subnetLocker(wrappedSubIdx).Unlock()
if err := s.FindAndDialPeersWithSubnets(ctx, SyncCommitteeSubnetTopicFormat, forkDigest, minimumPeersPerSubnetForBroadcast, map[uint64]bool{subnet: true}); err != nil {
builder := func(idx uint64) string { return syncCommitteeToTopic(idx, forkDigest) + s.Encoding().ProtocolSuffix() }
if err := s.FindAndDialPeersWithSubnets(ctx, builder, minimumPeersPerSubnetForBroadcast, map[uint64]bool{subnet: true}); err != nil {
return errors.Wrap(err, "find peers with subnets")
}
@@ -252,7 +254,8 @@ func (s *Service) internalBroadcastBlob(ctx context.Context, subnet uint64, blob
s.subnetLocker(wrappedSubIdx).Lock()
defer s.subnetLocker(wrappedSubIdx).Unlock()
if err := s.FindAndDialPeersWithSubnets(ctx, BlobSubnetTopicFormat, forkDigest, minimumPeersPerSubnetForBroadcast, map[uint64]bool{subnet: true}); err != nil {
builder := func(idx uint64) string { return blobSubnetToTopic(idx, forkDigest) + s.Encoding().ProtocolSuffix() }
if err := s.FindAndDialPeersWithSubnets(ctx, builder, minimumPeersPerSubnetForBroadcast, map[uint64]bool{subnet: true}); err != nil {
return errors.Wrap(err, "find peers with subnets")
}
@@ -392,7 +395,10 @@ func (s *Service) broadcastDataColumnSidecars(ctx context.Context, forkDigest [f
wrappedSubIdx := subnet + dataColumnSubnetVal
// Find peers if needed.
if err := s.findPeersIfNeeded(ctx, wrappedSubIdx, DataColumnSubnetTopicFormat, forkDigest, subnet); err != nil {
builder := func(idx uint64) string {
return dataColumnSubnetToTopic(idx, forkDigest) + s.Encoding().ProtocolSuffix()
}
if err := s.findPeersIfNeeded(ctx, wrappedSubIdx, builder, subnet); err != nil {
tracing.AnnotateError(span, err)
log.WithError(err).Error("Cannot find peers if needed")
return
@@ -487,8 +493,7 @@ func (s *Service) broadcastDataColumnSidecars(ctx context.Context, forkDigest [f
func (s *Service) findPeersIfNeeded(
ctx context.Context,
wrappedSubIdx uint64,
topicFormat string,
forkDigest [fieldparams.VersionLength]byte,
fullTopicForSubnet func(uint64) string,
subnet uint64,
) error {
// Sending a data column sidecar to only one peer is not ideal,
@@ -497,7 +502,7 @@ func (s *Service) findPeersIfNeeded(
defer s.subnetLocker(wrappedSubIdx).Unlock()
// No peers found, attempt to find peers with this subnet.
if err := s.FindAndDialPeersWithSubnets(ctx, topicFormat, forkDigest, minimumPeersPerSubnetForBroadcast, map[uint64]bool{subnet: true}); err != nil {
if err := s.FindAndDialPeersWithSubnets(ctx, fullTopicForSubnet, minimumPeersPerSubnetForBroadcast, map[uint64]bool{subnet: true}); err != nil {
return errors.Wrap(err, "find peers with subnet")
}

View File

@@ -9,6 +9,7 @@ import (
"testing"
"time"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
@@ -31,7 +32,6 @@ import (
"github.com/OffchainLabs/prysm/v6/time/slots"
pubsub "github.com/libp2p/go-libp2p-pubsub"
"github.com/libp2p/go-libp2p/core/host"
"github.com/prysmaticlabs/go-bitfield"
"google.golang.org/protobuf/proto"
)

View File

@@ -7,6 +7,7 @@ import (
statefeed "github.com/OffchainLabs/prysm/v6/beacon-chain/core/feed/state"
"github.com/OffchainLabs/prysm/v6/beacon-chain/db"
"github.com/OffchainLabs/prysm/v6/beacon-chain/startup"
"github.com/sirupsen/logrus"
)
// This is the default queue size used if we have specified an invalid one.
@@ -63,12 +64,17 @@ func (cfg *Config) connManagerLowHigh() (int, int) {
return low, high
}
// validateConfig validates whether the values provided are accurate and will set
// the appropriate values for those that are invalid.
func validateConfig(cfg *Config) *Config {
if cfg.QueueSize == 0 {
log.Warnf("Invalid pubsub queue size of %d initialized, setting the quese size as %d instead", cfg.QueueSize, defaultPubsubQueueSize)
cfg.QueueSize = defaultPubsubQueueSize
// validateConfig validates whether the provided config has valid values and sets
// the invalid ones to default.
func validateConfig(cfg *Config) {
if cfg.QueueSize > 0 {
return
}
return cfg
log.WithFields(logrus.Fields{
"queueSize": cfg.QueueSize,
"default": defaultPubsubQueueSize,
}).Warning("Invalid pubsub queue size, setting the queue size to the default value")
cfg.QueueSize = defaultPubsubQueueSize
}

View File

@@ -259,11 +259,11 @@ func (s *Service) custodyGroupCountFromPeerENR(pid peer.ID) uint64 {
}
func fuluForkSlot() (primitives.Slot, error) {
beaconConfig := params.BeaconConfig()
cfg := params.BeaconConfig()
fuluForkEpoch := beaconConfig.FuluForkEpoch
if fuluForkEpoch == beaconConfig.FarFutureEpoch {
return beaconConfig.FarFutureSlot, nil
fuluForkEpoch := cfg.FuluForkEpoch
if fuluForkEpoch == cfg.FarFutureEpoch {
return cfg.FarFutureSlot, nil
}
forkFuluSlot, err := slots.EpochStart(fuluForkEpoch)

View File

@@ -9,6 +9,7 @@ import (
"sync"
"time"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/beacon-chain/cache"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
"github.com/OffchainLabs/prysm/v6/cmd/beacon-chain/flags"
@@ -24,7 +25,6 @@ import (
"github.com/libp2p/go-libp2p/core/peer"
ma "github.com/multiformats/go-multiaddr"
"github.com/pkg/errors"
"github.com/prysmaticlabs/go-bitfield"
"github.com/sirupsen/logrus"
)
@@ -554,6 +554,7 @@ func (s *Service) createListener(
Bootnodes: bootNodes,
PingInterval: s.cfg.PingInterval,
NoFindnodeLivenessCheck: s.cfg.DisableLivenessCheck,
V5RespTimeout: 300 * time.Millisecond,
}
listener, err := discover.ListenV5(conn, localNode, dv5Cfg)

View File

@@ -16,6 +16,7 @@ import (
"testing"
"time"
"github.com/OffchainLabs/go-bitfield"
mock "github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/testing"
"github.com/OffchainLabs/prysm/v6/beacon-chain/cache"
testDB "github.com/OffchainLabs/prysm/v6/beacon-chain/db/testing"
@@ -41,7 +42,6 @@ import (
"github.com/libp2p/go-libp2p/core/host"
"github.com/libp2p/go-libp2p/core/network"
"github.com/libp2p/go-libp2p/core/peer"
"github.com/prysmaticlabs/go-bitfield"
logTest "github.com/sirupsen/logrus/hooks/test"
)

View File

@@ -2,6 +2,7 @@ package p2p
import (
"context"
"fmt"
"math"
"net"
"reflect"
@@ -106,18 +107,26 @@ func peerScoringParams(colocationWhitelist []*net.IPNet) (*pubsub.PeerScoreParam
}
func (s *Service) topicScoreParams(topic string) (*pubsub.TopicScoreParams, error) {
activeValidators, err := s.retrieveActiveValidators()
if err != nil {
return nil, err
}
switch {
case strings.Contains(topic, GossipBlockMessage):
return defaultBlockTopicParams(), nil
case strings.Contains(topic, GossipAggregateAndProofMessage):
activeValidators, err := s.retrieveActiveValidators()
if err != nil {
return nil, fmt.Errorf("failed to compute active validator count for topic %s: %w", GossipAggregateAndProofMessage, err)
}
return defaultAggregateTopicParams(activeValidators), nil
case strings.Contains(topic, GossipAttestationMessage):
activeValidators, err := s.retrieveActiveValidators()
if err != nil {
return nil, fmt.Errorf("failed to compute active validator count for topic %s: %w", GossipAttestationMessage, err)
}
return defaultAggregateSubnetTopicParams(activeValidators), nil
case strings.Contains(topic, GossipSyncCommitteeMessage):
activeValidators, err := s.retrieveActiveValidators()
if err != nil {
return nil, fmt.Errorf("failed to compute active validator count for topic %s: %w", GossipSyncCommitteeMessage, err)
}
return defaultSyncSubnetTopicParams(activeValidators), nil
case strings.Contains(topic, GossipContributionAndProofMessage):
return defaultSyncContributionTopicParams(), nil
@@ -142,6 +151,8 @@ func (s *Service) topicScoreParams(topic string) (*pubsub.TopicScoreParams, erro
}
func (s *Service) retrieveActiveValidators() (uint64, error) {
s.activeValidatorCountLock.Lock()
defer s.activeValidatorCountLock.Unlock()
if s.activeValidatorCount != 0 {
return s.activeValidatorCount, nil
}

View File

@@ -0,0 +1,511 @@
package p2p
import (
"context"
"sort"
"sync"
"time"
"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/gossipsubcrawler"
"github.com/pkg/errors"
"golang.org/x/sync/semaphore"
"github.com/libp2p/go-libp2p/core/peer"
"github.com/ethereum/go-ethereum/p2p/enode"
)
type peerNode struct {
id enode.ID
isPinged bool
node *enode.Node
peerID peer.ID
topics map[gossipsubcrawler.Topic]struct{}
}
type crawledPeers struct {
g *GossipsubPeerCrawler
mu sync.RWMutex
byEnode map[enode.ID]*peerNode
byPeerId map[peer.ID]*peerNode
byTopic map[gossipsubcrawler.Topic]map[peer.ID]struct{}
}
func (cp *crawledPeers) updateStatusToPinged(enodeID enode.ID) {
cp.mu.Lock()
defer cp.mu.Unlock()
existingPNode, ok := cp.byEnode[enodeID]
if !ok {
return
}
// we only want to ping a node with a given NodeId once -> not on every sequence number change
// as ping is simply a test of a node being reachable and not fake
existingPNode.isPinged = true
}
func (cp *crawledPeers) removePeerOnPingFailure(enodeID enode.ID) {
cp.mu.Lock()
defer cp.mu.Unlock()
existingPNode, ok := cp.byEnode[enodeID]
if !ok {
return
}
// same idea as in "updateStatusToPinged" above.
// We don't want to test pings for every sequence number change for a given node as that
// can lead to an explosion in the number of pings the crawler needs to do.
// So, remove the peer when the first ping fails. If the node becomes reachable later,
// we will discover it during a re-crawl and ping it again to test for reachability.
// we're not blacklisting this peer anyways.
cp.updateTopicsUnlocked(existingPNode, nil)
}
func (cp *crawledPeers) updateCrawledIfNewer(node *enode.Node, topics []string) {
cp.mu.Lock()
enodeID := node.ID()
existingPNode, ok := cp.byEnode[enodeID]
if ok && existingPNode.node == nil {
log.WithField("enodeId", enodeID).Error("enode is nil for enodeId")
cp.mu.Unlock()
return
}
// we don't want to update enodes with a lower sequence number as they're stale records
if ok && existingPNode.node.Seq() >= node.Seq() {
cp.mu.Unlock()
return
}
if !ok {
// this is a new peer
peerID, err := enodeToPeerID(node)
if err != nil {
log.WithError(err).WithField("node", node.ID()).Debug("Failed to convert enode to peer ID")
cp.mu.Unlock()
return
}
existingPNode = &peerNode{
id: enodeID,
node: node,
peerID: peerID,
topics: make(map[gossipsubcrawler.Topic]struct{}),
}
cp.byEnode[enodeID] = existingPNode
cp.byPeerId[peerID] = existingPNode
} else {
existingPNode.node = node
}
cp.updateTopicsUnlocked(existingPNode, topics)
if existingPNode.isPinged || len(topics) == 0 {
cp.mu.Unlock()
return
}
cp.mu.Unlock()
select {
case cp.g.pingCh <- *node:
case <-cp.g.ctx.Done():
return
}
}
func (cp *crawledPeers) removeTopic(topic gossipsubcrawler.Topic) {
cp.mu.Lock()
defer cp.mu.Unlock()
// Get all peers subscribed to this topic
peers, ok := cp.byTopic[topic]
if !ok {
return // Topic doesn't exist
}
// Remove the topic from each peer's topic list
for peerID := range peers {
if pnode, exists := cp.byPeerId[peerID]; exists {
delete(pnode.topics, topic)
// remove the peer if it has no more topics left
if len(pnode.topics) == 0 {
cp.updateTopicsUnlocked(pnode, nil)
}
}
}
// Remove the topic from byTopic map
delete(cp.byTopic, topic)
}
func (cp *crawledPeers) removePeerId(peerID peer.ID) {
cp.mu.Lock()
defer cp.mu.Unlock()
pnode, exists := cp.byPeerId[peerID]
if !exists {
return
}
// Use updateTopicsUnlocked with empty topics to remove the peer
cp.updateTopicsUnlocked(pnode, nil)
}
func (cp *crawledPeers) removePeer(enodeID enode.ID) {
cp.mu.Lock()
defer cp.mu.Unlock()
pnode, exists := cp.byEnode[enodeID]
if !exists {
return
}
cp.updateTopicsUnlocked(pnode, nil)
}
// setting topics to empty will remove the peer completely.
func (cp *crawledPeers) updateTopicsUnlocked(pnode *peerNode, topics []string) {
// If topics is empty, remove the peer completely.
if len(topics) == 0 {
delete(cp.byPeerId, pnode.peerID)
delete(cp.byEnode, pnode.id)
for t := range pnode.topics {
if peers, ok := cp.byTopic[t]; ok {
delete(peers, pnode.peerID)
if len(peers) == 0 {
delete(cp.byTopic, t)
}
}
}
pnode.topics = nil // Clear topics to indicate removal.
return
}
newTopics := make(map[gossipsubcrawler.Topic]struct{})
for _, t := range topics {
newTopics[gossipsubcrawler.Topic(t)] = struct{}{}
}
// Remove old topics that are no longer present.
for oldTopic := range pnode.topics {
if _, exists := newTopics[oldTopic]; !exists {
if peers, ok := cp.byTopic[oldTopic]; ok {
delete(peers, pnode.peerID)
if len(peers) == 0 {
delete(cp.byTopic, oldTopic)
}
}
}
}
// Add new topics.
for newTopic := range newTopics {
if _, exists := pnode.topics[newTopic]; !exists {
if _, ok := cp.byTopic[newTopic]; !ok {
cp.byTopic[newTopic] = make(map[peer.ID]struct{})
}
cp.byTopic[newTopic][pnode.peerID] = struct{}{}
}
}
pnode.topics = newTopics
}
type GossipsubPeerCrawler struct {
ctx context.Context
cancel context.CancelFunc
crawlInterval time.Duration
crawlTimeout time.Duration
crawledPeers *crawledPeers
// Discovery interface for finding peers
dv5 ListenerRebooter
service *Service
topicExtractor gossipsubcrawler.TopicExtractor
peerFilter gossipsubcrawler.PeerFilterFunc
scorer PeerScoreFunc
maxConcurrentPings int
pingCh chan enode.Node
pingSemaphore *semaphore.Weighted
wg sync.WaitGroup
once sync.Once
}
// cleanupInterval controls how frequently we sweep crawled peers and prune
// those that are no longer useful.
const cleanupInterval = 5 * time.Minute
// PeerScoreFunc provides a way to calculate a score for a given peer ID.
// Higher scores should indicate better peers.
type PeerScoreFunc func(peer.ID) float64
func NewGossipsubPeerCrawler(
service *Service,
dv5 ListenerRebooter,
crawlTimeout time.Duration,
crawlInterval time.Duration,
maxConcurrentPings int,
peerFilter gossipsubcrawler.PeerFilterFunc,
scorer PeerScoreFunc,
) (*GossipsubPeerCrawler, error) {
if service == nil {
return nil, errors.New("service is nil")
}
if dv5 == nil {
return nil, errors.New("dv5 is nil")
}
if crawlTimeout <= 0 {
return nil, errors.New("crawl timeout must be greater than 0")
}
if crawlInterval <= 0 {
return nil, errors.New("crawl interval must be greater than 0")
}
if maxConcurrentPings <= 0 {
return nil, errors.New("max concurrent pings must be greater than 0")
}
if peerFilter == nil {
return nil, errors.New("peer filter is nil")
}
if scorer == nil {
return nil, errors.New("peer scorer is nil")
}
ctx, cancel := context.WithCancel(context.Background())
g := &GossipsubPeerCrawler{
ctx: ctx,
cancel: cancel,
crawlInterval: crawlInterval,
crawlTimeout: crawlTimeout,
service: service,
dv5: dv5,
maxConcurrentPings: maxConcurrentPings,
peerFilter: peerFilter,
scorer: scorer,
}
g.pingCh = make(chan enode.Node, 4*g.maxConcurrentPings)
g.pingSemaphore = semaphore.NewWeighted(int64(g.maxConcurrentPings))
g.crawledPeers = &crawledPeers{
g: g,
byEnode: make(map[enode.ID]*peerNode),
byPeerId: make(map[peer.ID]*peerNode),
byTopic: make(map[gossipsubcrawler.Topic]map[peer.ID]struct{}),
}
return g, nil
}
func (g *GossipsubPeerCrawler) PeersForTopic(topic gossipsubcrawler.Topic) []*enode.Node {
g.crawledPeers.mu.RLock()
defer g.crawledPeers.mu.RUnlock()
peerIDs, ok := g.crawledPeers.byTopic[topic]
if !ok {
return nil
}
var peerNodes []*peerNode
for peerID := range peerIDs {
peerNode, ok := g.crawledPeers.byPeerId[peerID]
if !ok {
continue
}
if peerNode.isPinged && g.peerFilter(peerNode.node) {
peerNodes = append(peerNodes, peerNode)
}
}
// Sort peerNodes in descending order of their scores.
sort.Slice(peerNodes, func(i, j int) bool {
scoreI := g.scorer(peerNodes[i].peerID)
scoreJ := g.scorer(peerNodes[j].peerID)
return scoreI > scoreJ
})
nodes := make([]*enode.Node, len(peerNodes))
for i, pn := range peerNodes {
nodes[i] = pn.node
}
return nodes
}
func (g *GossipsubPeerCrawler) RemovePeerId(peerID peer.ID) {
g.crawledPeers.removePeerId(peerID)
}
func (g *GossipsubPeerCrawler) RemoveTopic(topic gossipsubcrawler.Topic) {
g.crawledPeers.removeTopic(topic)
}
// Start runs the crawler's loops in the background.
func (g *GossipsubPeerCrawler) Start(te gossipsubcrawler.TopicExtractor) error {
if te == nil {
return errors.New("topic extractor is nil")
}
g.once.Do(func() {
g.topicExtractor = te
g.wg.Go(func() {
g.crawlLoop()
})
g.wg.Go(func() {
g.pingLoop()
})
g.wg.Go(func() {
g.cleanupLoop()
})
})
return nil
}
// Stop terminates the crawler.
func (g *GossipsubPeerCrawler) Stop() {
g.cancel()
g.wg.Wait()
}
func (g *GossipsubPeerCrawler) pingLoop() {
for {
select {
case node := <-g.pingCh:
if err := g.pingSemaphore.Acquire(g.ctx, 1); err != nil {
return // Context cancelled, exit loop.
}
go func(node *enode.Node) {
defer g.pingSemaphore.Release(1)
if err := g.dv5.Ping(node); err != nil {
g.crawledPeers.removePeerOnPingFailure(node.ID())
return
}
g.crawledPeers.updateStatusToPinged(node.ID())
}(&node)
case <-g.ctx.Done():
return
}
}
}
func (g *GossipsubPeerCrawler) crawlLoop() {
ticker := time.NewTicker(g.crawlInterval)
defer ticker.Stop()
g.crawl()
for {
select {
case <-ticker.C:
g.crawl()
case <-g.ctx.Done():
return
}
}
}
func (g *GossipsubPeerCrawler) crawl() {
ctx, cancel := context.WithTimeout(g.ctx, g.crawlTimeout)
defer cancel()
iterator := g.dv5.RandomNodes()
// Ensure iterator unblocks on context cancellation or timeout
go func() {
<-ctx.Done()
iterator.Close()
}()
for iterator.Next() {
if ctx.Err() != nil {
return
}
node := iterator.Node()
if node == nil {
continue
}
if !g.peerFilter(node) {
g.crawledPeers.removePeer(node.ID())
continue
}
topics, err := g.topicExtractor(ctx, node)
if err != nil {
log.WithError(err).WithField("node", node.ID()).Debug("Failed to extract topics, skipping")
continue
}
g.crawledPeers.updateCrawledIfNewer(node, topics)
}
}
// cleanupLoop periodically removes peers that the filter rejects or that
// have no topics of interest. It uses the same context lifecycle as other
// background loops.
func (g *GossipsubPeerCrawler) cleanupLoop() {
ticker := time.NewTicker(cleanupInterval)
defer ticker.Stop()
// Initial cleanup to catch any leftovers from startup state
g.cleanup()
for {
select {
case <-ticker.C:
g.cleanup()
case <-g.ctx.Done():
return
}
}
}
// cleanup scans the crawled peer set and removes entries that either fail
// the current peer filter or have no topics of interest remaining.
func (g *GossipsubPeerCrawler) cleanup() {
cp := g.crawledPeers
// Snapshot current peers to evaluate without holding the lock during
// filter and topic extraction.
cp.mu.RLock()
peers := make([]*peerNode, 0, len(cp.byPeerId))
for _, p := range cp.byPeerId {
peers = append(peers, p)
}
cp.mu.RUnlock()
for _, p := range peers {
p := p
// Remove peers that no longer pass the filter
if !g.peerFilter(p.node) {
cp.removePeer(p.id)
continue
}
// Re-extract topics; if the extractor errors or yields none, drop the peer.
topics, err := g.topicExtractor(g.ctx, p.node)
if err != nil || len(topics) == 0 {
cp.removePeer(p.id)
}
}
}
// enodeToPeerID converts an enode record to a peer ID.
func enodeToPeerID(n *enode.Node) (peer.ID, error) {
info, _, err := convertToAddrInfo(n)
if err != nil {
return "", err
}
if info == nil {
return "", errors.New("peer info is nil")
}
return info.ID, nil
}

View File

@@ -0,0 +1,781 @@
package p2p
import (
"context"
"net"
"testing"
"time"
"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/gossipsubcrawler"
p2ptest "github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/testing"
"github.com/ethereum/go-ethereum/p2p/enode"
"github.com/libp2p/go-libp2p/core/peer"
"github.com/stretchr/testify/require"
)
// Helpers for crawledPeers tests
func newTestCrawledPeers() *crawledPeers {
return &crawledPeers{
byEnode: make(map[enode.ID]*peerNode),
byPeerId: make(map[peer.ID]*peerNode),
byTopic: make(map[gossipsubcrawler.Topic]map[peer.ID]struct{}),
}
}
func addPeerWithTopics(t *testing.T, cp *crawledPeers, node *enode.Node, topics []string, pinged bool) *peerNode {
t.Helper()
pid, err := enodeToPeerID(node)
require.NoError(t, err)
p := &peerNode{
id: node.ID(),
isPinged: pinged,
node: node,
peerID: pid,
topics: make(map[gossipsubcrawler.Topic]struct{}),
}
cp.mu.Lock()
cp.byEnode[p.id] = p
cp.byPeerId[p.peerID] = p
cp.updateTopicsUnlocked(p, topics)
cp.mu.Unlock()
return p
}
func TestUpdateStatusToPinged(t *testing.T) {
localNode := createTestNodeRandom(t)
node1 := localNode.Node()
localNode2 := createTestNodeRandom(t)
node2 := localNode2.Node()
cases := []struct {
name string
prep func(*crawledPeers)
target *enode.Node
expectPinged map[enode.ID]bool
}{
{
name: "sets pinged for existing peer",
prep: func(cp *crawledPeers) {
addPeerWithTopics(t, cp, node1, []string{"a"}, false)
},
target: node1,
expectPinged: map[enode.ID]bool{
node1.ID(): true,
},
},
{
name: "idempotent when already pinged",
prep: func(cp *crawledPeers) {
addPeerWithTopics(t, cp, node1, []string{"a"}, true)
},
target: node1,
expectPinged: map[enode.ID]bool{
node1.ID(): true,
},
},
{
name: "no change when peer missing",
prep: func(cp *crawledPeers) {
addPeerWithTopics(t, cp, node1, []string{"a"}, false)
},
target: node2,
expectPinged: map[enode.ID]bool{
node1.ID(): false,
},
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
cp := newTestCrawledPeers()
tc.prep(cp)
cp.updateStatusToPinged(tc.target.ID())
cp.mu.RLock()
defer cp.mu.RUnlock()
for id, exp := range tc.expectPinged {
if p := cp.byEnode[id]; p != nil {
require.Equal(t, exp, p.isPinged)
}
}
})
}
}
func TestRemovePeerOnPingFailure(t *testing.T) {
localNode := createTestNodeRandom(t)
node1 := localNode.Node()
localNode2 := createTestNodeRandom(t)
node2 := localNode2.Node()
cases := []struct {
name string
prep func(*crawledPeers)
target *enode.Node
wantLeft int // peers left byPeerId
check func(*testing.T, *crawledPeers)
}{
{
name: "removes existing peer and prunes sole topic",
prep: func(cp *crawledPeers) {
addPeerWithTopics(t, cp, node1, []string{"t1"}, true)
},
target: node1,
wantLeft: 0,
check: func(t *testing.T, cp *crawledPeers) {
_, ok := cp.byTopic[gossipsubcrawler.Topic("t1")]
require.False(t, ok, "topic should be pruned when empty")
},
},
{
name: "removes only target peer, keeps topic for others",
prep: func(cp *crawledPeers) {
addPeerWithTopics(t, cp, node1, []string{"t1"}, true)
addPeerWithTopics(t, cp, node2, []string{"t1"}, true)
},
target: node1,
wantLeft: 1,
check: func(t *testing.T, cp *crawledPeers) {
peers := cp.byTopic[gossipsubcrawler.Topic("t1")]
require.Len(t, peers, 1)
},
},
{
name: "no-op when peer missing",
prep: func(cp *crawledPeers) {
addPeerWithTopics(t, cp, node1, []string{"t1"}, true)
},
target: node2,
wantLeft: 1,
check: func(t *testing.T, cp *crawledPeers) {},
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
cp := newTestCrawledPeers()
tc.prep(cp)
cp.removePeerOnPingFailure(tc.target.ID())
cp.mu.RLock()
defer cp.mu.RUnlock()
require.Len(t, cp.byPeerId, tc.wantLeft)
tc.check(t, cp)
})
}
}
func TestRemoveTopic(t *testing.T) {
localNode := createTestNodeRandom(t)
node1 := localNode.Node()
localNode2 := createTestNodeRandom(t)
node2 := localNode2.Node()
topic1 := gossipsubcrawler.Topic("t1")
topic2 := gossipsubcrawler.Topic("t2")
cases := []struct {
name string
prep func(*crawledPeers)
topic gossipsubcrawler.Topic
check func(*testing.T, *crawledPeers)
}{
{
name: "removes topic from all peers and index",
prep: func(cp *crawledPeers) {
addPeerWithTopics(t, cp, node1, []string{"t1", "t2"}, true)
addPeerWithTopics(t, cp, node2, []string{"t1"}, true)
},
topic: topic1,
check: func(t *testing.T, cp *crawledPeers) {
_, ok := cp.byTopic[topic1]
require.False(t, ok)
for _, p := range cp.byPeerId {
_, has := p.topics[topic1]
require.False(t, has)
}
// Ensure other topics remain
_, ok = cp.byTopic[topic2]
require.True(t, ok)
},
},
{
name: "no-op when topic missing",
prep: func(cp *crawledPeers) {
addPeerWithTopics(t, cp, node1, []string{"t2"}, true)
},
topic: topic1,
check: func(t *testing.T, cp *crawledPeers) {
_, ok := cp.byTopic[topic2]
require.True(t, ok)
},
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
cp := newTestCrawledPeers()
tc.prep(cp)
cp.removeTopic(tc.topic)
tc.check(t, cp)
})
}
}
func TestRemovePeer(t *testing.T) {
localNode := createTestNodeRandom(t)
node1 := localNode.Node()
localNode2 := createTestNodeRandom(t)
node2 := localNode2.Node()
cases := []struct {
name string
prep func(*crawledPeers)
target enode.ID
wantTopics int
}{
{
name: "removes existing peer and prunes empty topic",
prep: func(cp *crawledPeers) {
addPeerWithTopics(t, cp, node1, []string{"t1"}, true)
},
target: node1.ID(),
wantTopics: 0,
},
{
name: "removes only targeted peer; keeps topic for other",
prep: func(cp *crawledPeers) {
addPeerWithTopics(t, cp, node1, []string{"t1"}, true)
addPeerWithTopics(t, cp, node2, []string{"t1"}, true)
},
target: node1.ID(),
wantTopics: 1, // byTopic should still have t1 with one peer
},
{
name: "no-op when peer missing",
prep: func(cp *crawledPeers) {
addPeerWithTopics(t, cp, node1, []string{"t1"}, true)
},
target: node2.ID(),
wantTopics: 1,
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
cp := newTestCrawledPeers()
tc.prep(cp)
cp.removePeer(tc.target)
cp.mu.RLock()
defer cp.mu.RUnlock()
require.Len(t, cp.byTopic, tc.wantTopics)
})
}
}
func TestRemovePeerId(t *testing.T) {
localNode := createTestNodeRandom(t)
node1 := localNode.Node()
localNode2 := createTestNodeRandom(t)
node2 := localNode2.Node()
pid1, err := enodeToPeerID(node1)
require.NoError(t, err)
pid2, err := enodeToPeerID(node2)
require.NoError(t, err)
cases := []struct {
name string
prep func(*crawledPeers)
target peer.ID
wantTopics int
wantPeers int
}{
{
name: "removes existing peer by id and prunes topic",
prep: func(cp *crawledPeers) {
addPeerWithTopics(t, cp, node1, []string{"t1"}, true)
},
target: pid1,
wantTopics: 0,
wantPeers: 0,
},
{
name: "removes only targeted peer id; keeps topic for other",
prep: func(cp *crawledPeers) {
addPeerWithTopics(t, cp, node1, []string{"t1"}, true)
addPeerWithTopics(t, cp, node2, []string{"t1"}, true)
},
target: pid1,
wantTopics: 1,
wantPeers: 1,
},
{
name: "no-op when peer id missing",
prep: func(cp *crawledPeers) {
addPeerWithTopics(t, cp, node1, []string{"t1"}, true)
},
target: pid2,
wantTopics: 1,
wantPeers: 1,
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
cp := newTestCrawledPeers()
tc.prep(cp)
cp.removePeerId(tc.target)
cp.mu.RLock()
defer cp.mu.RUnlock()
require.Len(t, cp.byTopic, tc.wantTopics)
require.Len(t, cp.byPeerId, tc.wantPeers)
})
}
}
func TestUpdateCrawledIfNewer(t *testing.T) {
newCrawler := func() (*crawledPeers, *GossipsubPeerCrawler, func()) {
ctx, cancel := context.WithCancel(context.Background())
g := &GossipsubPeerCrawler{
ctx: ctx,
cancel: cancel,
pingCh: make(chan enode.Node, 8),
}
cp := newTestCrawledPeers()
cp.g = g
return cp, g, cancel
}
// Helper: non-blocking receive from ping channel
recvPing := func(ch <-chan enode.Node) (enode.Node, bool) {
select {
case n := <-ch:
return n, true
default:
return enode.Node{}, false
}
}
// Helper: local node that will cause enodeToPeerID to fail (no TCP/UDP multiaddrs)
newNodeNoPorts := func(t *testing.T) *enode.Node {
_, privKey := createAddrAndPrivKey(t)
db, err := enode.OpenDB("")
require.NoError(t, err)
t.Cleanup(func() { db.Close() })
ln := enode.NewLocalNode(db, privKey)
// Do not set TCP/UDP; keep only IP
ln.SetStaticIP(net.ParseIP("127.0.0.1"))
return ln.Node()
}
// Ensure both A nodes have the same enode.ID but differing seq
ln := createTestNodeRandom(t)
nodeA1 := ln.Node()
setNodeSeq(ln, nodeA1.Seq()+1)
nodeA2 := ln.Node()
tests := []struct {
name string
arrange func(*crawledPeers)
invokeNode *enode.Node
invokeTopics []string
assert func(*testing.T, *crawledPeers, <-chan enode.Node)
}{
{
name: "new peer with topics adds and pings once",
arrange: func(cp *crawledPeers) {},
invokeNode: nodeA1,
invokeTopics: []string{"a"},
assert: func(t *testing.T, cp *crawledPeers, ch <-chan enode.Node) {
cp.mu.RLock()
require.Len(t, cp.byEnode, 1)
require.Len(t, cp.byPeerId, 1)
require.Contains(t, cp.byTopic, gossipsubcrawler.Topic("a"))
cp.mu.RUnlock()
if n, ok := recvPing(ch); !ok || n.ID() != nodeA1.ID() {
t.Fatalf("expected one ping for nodeA1")
}
if _, ok := recvPing(ch); ok {
t.Fatalf("expected exactly one ping")
}
},
},
{
name: "new peer with empty topics is removed and not pinged",
arrange: func(cp *crawledPeers) {},
invokeNode: nodeA1,
invokeTopics: nil,
assert: func(t *testing.T, cp *crawledPeers, ch <-chan enode.Node) {
cp.mu.RLock()
require.Empty(t, cp.byEnode)
require.Empty(t, cp.byPeerId)
require.Empty(t, cp.byTopic)
cp.mu.RUnlock()
if _, ok := recvPing(ch); ok {
t.Fatalf("did not expect ping when topics empty")
}
},
},
{
name: "existing peer lower seq is ignored (no update, no ping)",
arrange: func(cp *crawledPeers) {
addPeerWithTopics(t, cp, nodeA2, []string{"x"}, false) // higher seq exists
},
invokeNode: nodeA1, // lower seq
invokeTopics: []string{"a", "b"},
assert: func(t *testing.T, cp *crawledPeers, ch <-chan enode.Node) {
cp.mu.RLock()
require.Contains(t, cp.byTopic, gossipsubcrawler.Topic("x"))
require.NotContains(t, cp.byTopic, gossipsubcrawler.Topic("a"))
cp.mu.RUnlock()
if _, ok := recvPing(ch); ok {
t.Fatalf("did not expect ping for lower/equal seq")
}
},
},
{
name: "existing peer equal seq is ignored (no update, no ping)",
arrange: func(cp *crawledPeers) {
addPeerWithTopics(t, cp, nodeA1, []string{"x"}, false)
},
invokeNode: nodeA1,
invokeTopics: []string{"a"},
assert: func(t *testing.T, cp *crawledPeers, ch <-chan enode.Node) {
cp.mu.RLock()
require.Contains(t, cp.byTopic, gossipsubcrawler.Topic("x"))
require.NotContains(t, cp.byTopic, gossipsubcrawler.Topic("a"))
cp.mu.RUnlock()
if _, ok := recvPing(ch); ok {
t.Fatalf("did not expect ping for equal seq")
}
},
},
{
name: "existing peer higher seq updates topics and pings if not pinged",
arrange: func(cp *crawledPeers) {
addPeerWithTopics(t, cp, nodeA1, []string{"x"}, false)
},
invokeNode: nodeA2,
invokeTopics: []string{"a"},
assert: func(t *testing.T, cp *crawledPeers, ch <-chan enode.Node) {
cp.mu.RLock()
require.NotContains(t, cp.byTopic, gossipsubcrawler.Topic("x"))
require.Contains(t, cp.byTopic, gossipsubcrawler.Topic("a"))
cp.mu.RUnlock()
if n, ok := recvPing(ch); !ok || n.ID() != nodeA2.ID() {
t.Fatalf("expected one ping for updated node")
}
},
},
{
name: "existing peer higher seq with already pinged does not ping",
arrange: func(cp *crawledPeers) {
p := addPeerWithTopics(t, cp, nodeA1, []string{"x"}, true)
// ensure pinged flag set
require.True(t, p.isPinged)
},
invokeNode: nodeA2,
invokeTopics: []string{"a"},
assert: func(t *testing.T, cp *crawledPeers, ch <-chan enode.Node) {
cp.mu.RLock()
require.Contains(t, cp.byTopic, gossipsubcrawler.Topic("a"))
cp.mu.RUnlock()
if _, ok := recvPing(ch); ok {
t.Fatalf("did not expect ping when already pinged")
}
},
},
{
name: "existing peer higher seq but empty topics removes peer and doesn't ping",
arrange: func(cp *crawledPeers) {
addPeerWithTopics(t, cp, nodeA1, []string{"x"}, false)
},
invokeNode: nodeA2,
invokeTopics: nil,
assert: func(t *testing.T, cp *crawledPeers, ch <-chan enode.Node) {
cp.mu.RLock()
require.Empty(t, cp.byEnode)
require.Empty(t, cp.byPeerId)
cp.mu.RUnlock()
if _, ok := recvPing(ch); ok {
t.Fatalf("did not expect ping when topics empty on update")
}
},
},
{
name: "corrupted existing entry with nil node is ignored (no change, no ping)",
arrange: func(cp *crawledPeers) {
pid, _ := enodeToPeerID(nodeA1)
cp.mu.Lock()
cp.byEnode[nodeA1.ID()] = &peerNode{id: nodeA1.ID(), node: nil, peerID: pid, topics: map[gossipsubcrawler.Topic]struct{}{gossipsubcrawler.Topic("x"): {}}}
cp.byPeerId[pid] = cp.byEnode[nodeA1.ID()]
cp.byTopic[gossipsubcrawler.Topic("x")] = map[peer.ID]struct{}{pid: {}}
cp.mu.Unlock()
},
invokeNode: nodeA2,
invokeTopics: []string{"a"},
assert: func(t *testing.T, cp *crawledPeers, ch <-chan enode.Node) {
cp.mu.RLock()
require.Contains(t, cp.byTopic, gossipsubcrawler.Topic("x"))
cp.mu.RUnlock()
if _, ok := recvPing(ch); ok {
t.Fatalf("did not expect ping for corrupted entry")
}
},
},
{
name: "new peer with no ports causes enodeToPeerID error; no add, no ping",
arrange: func(cp *crawledPeers) {},
invokeNode: newNodeNoPorts(t),
invokeTopics: []string{"a"},
assert: func(t *testing.T, cp *crawledPeers, ch <-chan enode.Node) {
cp.mu.RLock()
require.Empty(t, cp.byEnode)
require.Empty(t, cp.byPeerId)
require.Empty(t, cp.byTopic)
cp.mu.RUnlock()
if _, ok := recvPing(ch); ok {
t.Fatalf("did not expect ping when enodeToPeerID fails")
}
},
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
cp, g, cancel := newCrawler()
defer cancel()
tc.arrange(cp)
cp.updateCrawledIfNewer(tc.invokeNode, tc.invokeTopics)
tc.assert(t, cp, g.pingCh)
})
}
}
func TestPeersForTopic(t *testing.T) {
t.Parallel()
newCrawler := func(filter gossipsubcrawler.PeerFilterFunc) (*GossipsubPeerCrawler, *crawledPeers) {
g := &GossipsubPeerCrawler{
peerFilter: filter,
scorer: func(peer.ID) float64 { return 0 },
crawledPeers: newTestCrawledPeers(),
}
return g, g.crawledPeers
}
// Prepare nodes
ln1 := createTestNodeRandom(t)
ln2 := createTestNodeRandom(t)
ln3 := createTestNodeRandom(t)
n1, n2, n3 := ln1.Node(), ln2.Node(), ln3.Node()
topic := gossipsubcrawler.Topic("top")
cases := []struct {
name string
filter gossipsubcrawler.PeerFilterFunc
setup func(t *testing.T, g *GossipsubPeerCrawler, cp *crawledPeers)
wantIDs []enode.ID
}{
{
name: "no peers for topic returns empty",
filter: func(*enode.Node) bool { return true },
setup: func(t *testing.T, g *GossipsubPeerCrawler, cp *crawledPeers) {},
wantIDs: nil,
},
{
name: "excludes unpinged peers",
filter: func(*enode.Node) bool { return true },
setup: func(t *testing.T, g *GossipsubPeerCrawler, cp *crawledPeers) {
// Add one pinged and one not pinged on same topic
addPeerWithTopics(t, cp, n1, []string{string(topic)}, true)
addPeerWithTopics(t, cp, n2, []string{string(topic)}, false)
},
wantIDs: []enode.ID{n1.ID()},
},
{
name: "applies peer filter to exclude",
filter: func(n *enode.Node) bool { return n.ID() != n2.ID() },
setup: func(t *testing.T, g *GossipsubPeerCrawler, cp *crawledPeers) {
addPeerWithTopics(t, cp, n1, []string{string(topic)}, true)
addPeerWithTopics(t, cp, n2, []string{string(topic)}, true)
},
wantIDs: []enode.ID{n1.ID()},
},
{
name: "ignores dangling peerID in byTopic",
filter: func(*enode.Node) bool { return true },
setup: func(t *testing.T, g *GossipsubPeerCrawler, cp *crawledPeers) {
addPeerWithTopics(t, cp, n1, []string{string(topic)}, true)
// Add n2 then remove it from byPeerId to simulate dangling
p2 := addPeerWithTopics(t, cp, n2, []string{string(topic)}, true)
cp.mu.Lock()
delete(cp.byPeerId, p2.peerID)
cp.mu.Unlock()
},
wantIDs: []enode.ID{n1.ID()},
},
{
name: "sorted by score descending",
filter: func(*enode.Node) bool { return true },
setup: func(t *testing.T, g *GossipsubPeerCrawler, cp *crawledPeers) {
// Add three pinged peers
p1 := addPeerWithTopics(t, cp, n1, []string{string(topic)}, true)
p2 := addPeerWithTopics(t, cp, n2, []string{string(topic)}, true)
p3 := addPeerWithTopics(t, cp, n3, []string{string(topic)}, true)
// Provide a deterministic scoring function
scores := map[peer.ID]float64{
p1.peerID: 3.0,
p2.peerID: 2.0,
p3.peerID: 1.0,
}
g.scorer = func(id peer.ID) float64 { return scores[id] }
},
wantIDs: []enode.ID{n1.ID(), n2.ID(), n3.ID()},
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
g, cp := newCrawler(tc.filter)
cp.g = g
tc.setup(t, g, cp)
got := g.PeersForTopic(topic)
var gotIDs []enode.ID
for _, n := range got {
gotIDs = append(gotIDs, n.ID())
}
if tc.wantIDs == nil {
require.Empty(t, gotIDs)
return
}
require.Equal(t, tc.wantIDs, gotIDs)
})
}
}
func TestCrawler_AddsAndPingsPeer(t *testing.T) {
// Create a test node with valid ENR entries (IP/TCP/UDP)
localNode := createTestNodeRandom(t)
node := localNode.Node()
// Prepare a mock iterator returning our single node
iterator := p2ptest.NewMockIterator([]*enode.Node{node})
// Prepare a mock listener with successful Ping
mockListener := p2ptest.NewMockListener(localNode, iterator)
mockListener.PingFunc = func(*enode.Node) error { return nil }
// Inject a permissive peer filter
filter := gossipsubcrawler.PeerFilterFunc(func(n *enode.Node) bool { return true })
// Create crawler with small intervals
scorer := func(peer.ID) float64 { return 0 }
g, err := NewGossipsubPeerCrawler(&Service{}, mockListener, 2*time.Second, 10*time.Millisecond, 4, filter, scorer)
require.NoError(t, err)
// Assign a simple topic extractor
topic := "test/topic"
topicExtractor := func(ctx context.Context, n *enode.Node) ([]string, error) {
return []string{topic}, nil
}
// Run ping loop in background and perform a single crawl
require.NoError(t, g.Start(topicExtractor))
defer g.Stop()
// Verify that the peer has been indexed under the topic and marked as pinged
require.Eventually(t, func() bool {
g.crawledPeers.mu.RLock()
defer g.crawledPeers.mu.RUnlock()
peersByTopic := g.crawledPeers.byTopic[gossipsubcrawler.Topic(topic)]
if len(peersByTopic) == 0 {
return false
}
// Fetch the single peerNode and check status
for pid := range peersByTopic {
pn := g.crawledPeers.byPeerId[pid]
if pn == nil {
return false
}
return pn.isPinged
}
return false
}, 2*time.Second, 10*time.Millisecond)
}
func TestCrawler_SkipsPeer_WhenFilterRejects(t *testing.T) {
t.Parallel()
localNode := createTestNodeRandom(t)
node := localNode.Node()
iterator := p2ptest.NewMockIterator([]*enode.Node{node})
mockListener := p2ptest.NewMockListener(localNode, iterator)
mockListener.PingFunc = func(*enode.Node) error { return nil }
// Reject all peers via injected filter
filter := gossipsubcrawler.PeerFilterFunc(func(n *enode.Node) bool { return false })
scorer := func(peer.ID) float64 { return 0 }
g, err := NewGossipsubPeerCrawler(&Service{}, mockListener, 2*time.Second, 10*time.Millisecond, 2, filter, scorer)
if err != nil {
t.Fatalf("NewGossipsubPeerCrawler error: %v", err)
}
topic := "test/topic"
g.topicExtractor = func(ctx context.Context, n *enode.Node) ([]string, error) { return []string{topic}, nil }
g.crawl()
// Verify no peers are indexed, because filter rejected the node
g.crawledPeers.mu.RLock()
defer g.crawledPeers.mu.RUnlock()
if len(g.crawledPeers.byEnode) != 0 || len(g.crawledPeers.byPeerId) != 0 || len(g.crawledPeers.byTopic) != 0 {
t.Fatalf("expected no peers indexed, got byEnode=%d byPeerId=%d byTopic=%d",
len(g.crawledPeers.byEnode), len(g.crawledPeers.byPeerId), len(g.crawledPeers.byTopic))
}
}
func TestCrawler_RemoveTopic_RemovesTopicFromIndexes(t *testing.T) {
t.Parallel()
localNode := createTestNodeRandom(t)
node := localNode.Node()
iterator := p2ptest.NewMockIterator([]*enode.Node{node})
mockListener := p2ptest.NewMockListener(localNode, iterator)
mockListener.PingFunc = func(*enode.Node) error { return nil }
filter := gossipsubcrawler.PeerFilterFunc(func(n *enode.Node) bool { return true })
scorer := func(peer.ID) float64 { return 0 }
g, err := NewGossipsubPeerCrawler(&Service{}, mockListener, 2*time.Second, 10*time.Millisecond, 2, filter, scorer)
if err != nil {
t.Fatalf("NewGossipsubPeerCrawler error: %v", err)
}
topic1 := "test/topic1"
topic2 := "test/topic2"
g.topicExtractor = func(ctx context.Context, n *enode.Node) ([]string, error) { return []string{topic1, topic2}, nil }
// Single crawl to index topics
g.crawl()
// Remove one topic and assert it is pruned from all indexes
g.RemoveTopic(gossipsubcrawler.Topic(topic1))
g.crawledPeers.mu.RLock()
defer g.crawledPeers.mu.RUnlock()
if _, ok := g.crawledPeers.byTopic[gossipsubcrawler.Topic(topic1)]; ok {
t.Fatalf("expected topic1 to be removed from byTopic")
}
// Ensure peer still exists and retains topic2
for _, pn := range g.crawledPeers.byEnode {
if _, has1 := pn.topics[gossipsubcrawler.Topic(topic1)]; has1 {
t.Fatalf("expected topic1 to be removed from peer topics")
}
if _, has2 := pn.topics[gossipsubcrawler.Topic(topic2)]; !has2 {
t.Fatalf("expected topic2 to remain for peer")
}
}
}

View File

@@ -0,0 +1,14 @@
load("@prysm//tools/go:def.bzl", "go_library")
go_library(
name = "go_default_library",
srcs = ["interface.go"],
importpath = "github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/gossipsubcrawler",
visibility = [
"//visibility:public",
],
deps = [
"@com_github_ethereum_go_ethereum//p2p/enode:go_default_library",
"@com_github_libp2p_go_libp2p//core/peer:go_default_library",
],
)

View File

@@ -0,0 +1,26 @@
package gossipsubcrawler
import (
"context"
"github.com/ethereum/go-ethereum/p2p/enode"
"github.com/libp2p/go-libp2p/core/peer"
)
type Topic string
// TopicExtractor is a function that can determine the set of topics a current or potential peer
// is subscribed to based on key/value pairs from the ENR record.
type TopicExtractor func(ctx context.Context, node *enode.Node) ([]string, error)
// PeerFilterFunc defines the filtering interface used by the crawler to decide if a node
// is a valid candidate to index in the crawler.
type PeerFilterFunc func(*enode.Node) bool
type Crawler interface {
Start(te TopicExtractor) error
Stop()
RemovePeerId(peerID peer.ID)
RemoveTopic(topic Topic)
PeersForTopic(topic Topic) []*enode.Node
}

View File

@@ -225,6 +225,10 @@ func (s *Service) AddDisconnectionHandler(handler func(ctx context.Context, id p
return
}
if s.crawler != nil {
s.crawler.RemovePeerId(peerID)
}
priorState, err := s.peers.ConnectionState(peerID)
if err != nil {
// Can happen if the peer has already disconnected, so...

View File

@@ -4,8 +4,8 @@ import (
"context"
"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/encoder"
"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/gossipsubcrawler"
"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/peers"
fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
@@ -101,8 +101,9 @@ type (
NodeID() enode.ID
DiscoveryAddresses() ([]multiaddr.Multiaddr, error)
RefreshPersistentSubnets()
FindAndDialPeersWithSubnets(ctx context.Context, topicFormat string, digest [fieldparams.VersionLength]byte, minimumPeersPerSubnet int, subnets map[uint64]bool) error
FindAndDialPeersWithSubnets(ctx context.Context, fullTopicForSubnet func(uint64) string, minimumPeersPerSubnet int, subnets map[uint64]bool) error
AddPingMethod(reqFunc func(ctx context.Context, id peer.ID) error)
Crawler() gossipsubcrawler.Crawler
}
// Sender abstracts the sending functionality from libp2p.

View File

@@ -3,6 +3,7 @@ package p2p
import (
"strings"
"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/peers"
"github.com/libp2p/go-libp2p/core/peer"
"github.com/libp2p/go-libp2p/core/peerstore"
"github.com/prometheus/client_golang/prometheus"
@@ -26,12 +27,25 @@ var (
Help: "The number of peers in a given state.",
},
[]string{"state"})
p2pMaxPeers = promauto.NewGauge(prometheus.GaugeOpts{
Name: "p2p_max_peers",
Help: "The target maximum number of peers.",
})
p2pPeerCountDirectionType = promauto.NewGaugeVec(prometheus.GaugeOpts{
Name: "p2p_peer_count_direction_type",
Help: "The number of peers in a given direction and type.",
},
[]string{"direction", "type"})
connectedPeersCount = promauto.NewGaugeVec(prometheus.GaugeOpts{
Name: "connected_libp2p_peers",
Help: "Tracks the total number of connected libp2p peers by agent string",
},
[]string{"agent"},
)
minimumPeersPerSubnet = promauto.NewGauge(prometheus.GaugeOpts{
Name: "p2p_minimum_peers_per_subnet",
Help: "The minimum number of peers to connect to per subnet",
})
avgScoreConnectedClients = promauto.NewGaugeVec(prometheus.GaugeOpts{
Name: "connected_libp2p_peers_average_scores",
Help: "Tracks the overall p2p scores of connected libp2p peers by agent string",
@@ -174,18 +188,26 @@ var (
)
func (s *Service) updateMetrics() {
store := s.Host().Peerstore()
connectedPeers := s.peers.Connected()
p2pPeerCount.WithLabelValues("Connected").Set(float64(len(connectedPeers)))
p2pPeerCount.WithLabelValues("Disconnected").Set(float64(len(s.peers.Disconnected())))
p2pPeerCount.WithLabelValues("Connecting").Set(float64(len(s.peers.Connecting())))
p2pPeerCount.WithLabelValues("Disconnecting").Set(float64(len(s.peers.Disconnecting())))
p2pPeerCount.WithLabelValues("Bad").Set(float64(len(s.peers.Bad())))
store := s.Host().Peerstore()
numConnectedPeersByClient := make(map[string]float64)
upperTCP := strings.ToUpper(string(peers.TCP))
upperQUIC := strings.ToUpper(string(peers.QUIC))
p2pPeerCountDirectionType.WithLabelValues("inbound", upperTCP).Set(float64(len(s.peers.InboundConnectedWithProtocol(peers.TCP))))
p2pPeerCountDirectionType.WithLabelValues("inbound", upperQUIC).Set(float64(len(s.peers.InboundConnectedWithProtocol(peers.QUIC))))
p2pPeerCountDirectionType.WithLabelValues("outbound", upperTCP).Set(float64(len(s.peers.OutboundConnectedWithProtocol(peers.TCP))))
p2pPeerCountDirectionType.WithLabelValues("outbound", upperQUIC).Set(float64(len(s.peers.OutboundConnectedWithProtocol(peers.QUIC))))
connectedPeersCountByClient := make(map[string]float64)
peerScoresByClient := make(map[string][]float64)
for i := 0; i < len(connectedPeers); i++ {
p := connectedPeers[i]
for _, p := range connectedPeers {
pid, err := peer.Decode(p.String())
if err != nil {
log.WithError(err).Debug("Could not decode peer string")
@@ -193,16 +215,18 @@ func (s *Service) updateMetrics() {
}
foundName := agentFromPid(pid, store)
numConnectedPeersByClient[foundName] += 1
connectedPeersCountByClient[foundName] += 1
// Get peer scoring data.
overallScore := s.peers.Scorers().Score(pid)
peerScoresByClient[foundName] = append(peerScoresByClient[foundName], overallScore)
}
connectedPeersCount.Reset() // Clear out previous results.
for agent, total := range numConnectedPeersByClient {
for agent, total := range connectedPeersCountByClient {
connectedPeersCount.WithLabelValues(agent).Set(total)
}
avgScoreConnectedClients.Reset() // Clear out previous results.
for agent, scoringData := range peerScoresByClient {
avgScore := average(scoringData)

View File

@@ -3,7 +3,7 @@ package peers
import (
"testing"
"github.com/prysmaticlabs/go-bitfield"
"github.com/OffchainLabs/go-bitfield"
)
func Benchmark_retrieveIndicesFromBitfield(b *testing.B) {

View File

@@ -25,10 +25,12 @@ package peers
import (
"context"
"net"
"slices"
"sort"
"strings"
"time"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/peers/peerdata"
"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/peers/scorers"
"github.com/OffchainLabs/prysm/v6/config/features"
@@ -46,7 +48,6 @@ import (
ma "github.com/multiformats/go-multiaddr"
manet "github.com/multiformats/go-multiaddr/net"
"github.com/pkg/errors"
"github.com/prysmaticlabs/go-bitfield"
)
const (
@@ -81,29 +82,31 @@ const (
type InternetProtocol string
const (
TCP = "tcp"
QUIC = "quic"
TCP = InternetProtocol("tcp")
QUIC = InternetProtocol("quic")
)
// Status is the structure holding the peer status information.
type Status struct {
ctx context.Context
scorers *scorers.Service
store *peerdata.Store
ipTracker map[string]uint64
rand *rand.Rand
ipColocationWhitelist []*net.IPNet
}
type (
// Status is the structure holding the peer status information.
Status struct {
ctx context.Context
scorers *scorers.Service
store *peerdata.Store
ipTracker map[string]uint64
rand *rand.Rand
ipColocationWhitelist []*net.IPNet
}
// StatusConfig represents peer status service params.
type StatusConfig struct {
// PeerLimit specifies maximum amount of concurrent peers that are expected to be connect to the node.
PeerLimit int
// ScorerParams holds peer scorer configuration params.
ScorerParams *scorers.Config
// IPColocationWhitelist contains CIDR ranges that are exempt from IP colocation limits.
IPColocationWhitelist []*net.IPNet
}
// StatusConfig represents peer status service params.
StatusConfig struct {
// PeerLimit specifies maximum amount of concurrent peers that are expected to be connect to the node.
PeerLimit int
// ScorerParams holds peer scorer configuration params.
ScorerParams *scorers.Config
// IPColocationWhitelist contains CIDR ranges that are exempt from IP colocation limits.
IPColocationWhitelist []*net.IPNet
}
)
// NewStatus creates a new status entity.
func NewStatus(ctx context.Context, config *StatusConfig) *Status {
@@ -304,11 +307,8 @@ func (p *Status) SubscribedToSubnet(index uint64) []peer.ID {
connectedStatus := peerData.ConnState == Connecting || peerData.ConnState == Connected
if connectedStatus && peerData.MetaData != nil && !peerData.MetaData.IsNil() && peerData.MetaData.AttnetsBitfield() != nil {
indices := indicesFromBitfield(peerData.MetaData.AttnetsBitfield())
for _, idx := range indices {
if idx == index {
peers = append(peers, pid)
break
}
if slices.Contains(indices, index) {
peers = append(peers, pid)
}
}
}

View File

@@ -6,6 +6,7 @@ import (
"testing"
"time"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/peers"
"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/peers/peerdata"
"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/peers/scorers"
@@ -21,7 +22,6 @@ import (
"github.com/libp2p/go-libp2p/core/network"
"github.com/libp2p/go-libp2p/core/peer"
ma "github.com/multiformats/go-multiaddr"
"github.com/prysmaticlabs/go-bitfield"
)
func TestStatus(t *testing.T) {

View File

@@ -345,17 +345,17 @@ func TopicFromMessage(msg string, epoch primitives.Epoch) (string, error) {
return "", errors.Errorf("%s: %s", invalidRPCMessageType, msg)
}
beaconConfig := params.BeaconConfig()
cfg := params.BeaconConfig()
// Check if the message is to be updated in fulu.
if epoch >= beaconConfig.FuluForkEpoch {
if epoch >= cfg.FuluForkEpoch {
if version, ok := fuluMapping[msg]; ok {
return protocolPrefix + msg + version, nil
}
}
// Check if the message is to be updated in altair.
if epoch >= beaconConfig.AltairForkEpoch {
if epoch >= cfg.AltairForkEpoch {
if version, ok := altairMapping[msg]; ok {
return protocolPrefix + msg + version, nil
}

View File

@@ -11,9 +11,11 @@ import (
"github.com/OffchainLabs/prysm/v6/async"
"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/encoder"
"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/gossipsubcrawler"
"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/peers"
"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/peers/scorers"
"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/types"
"github.com/OffchainLabs/prysm/v6/cmd/beacon-chain/flags"
"github.com/OffchainLabs/prysm/v6/config/features"
"github.com/OffchainLabs/prysm/v6/config/params"
"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
@@ -60,39 +62,45 @@ var (
// for the current peer limit status for the time period
// defined below.
pollingPeriod = 6 * time.Second
crawlTimeout = 30 * time.Second
crawlInterval = 1 * time.Second
maxConcurrentDials = 256
)
// Service for managing peer to peer (p2p) networking.
type Service struct {
started bool
isPreGenesis bool
pingMethod func(ctx context.Context, id peer.ID) error
pingMethodLock sync.RWMutex
cancel context.CancelFunc
cfg *Config
peers *peers.Status
addrFilter *multiaddr.Filters
ipLimiter *leakybucket.Collector
privKey *ecdsa.PrivateKey
metaData metadata.Metadata
pubsub *pubsub.PubSub
joinedTopics map[string]*pubsub.Topic
joinedTopicsLock sync.RWMutex
subnetsLock map[uint64]*sync.RWMutex
subnetsLockLock sync.Mutex // Lock access to subnetsLock
initializationLock sync.Mutex
dv5Listener ListenerRebooter
startupErr error
ctx context.Context
host host.Host
genesisTime time.Time
genesisValidatorsRoot []byte
activeValidatorCount uint64
peerDisconnectionTime *cache.Cache
custodyInfo *custodyInfo
custodyInfoLock sync.RWMutex // Lock access to custodyInfo
custodyInfoSet chan struct{}
allForkDigests map[[4]byte]struct{}
started bool
isPreGenesis bool
pingMethod func(ctx context.Context, id peer.ID) error
pingMethodLock sync.RWMutex
cancel context.CancelFunc
cfg *Config
peers *peers.Status
addrFilter *multiaddr.Filters
ipLimiter *leakybucket.Collector
privKey *ecdsa.PrivateKey
metaData metadata.Metadata
pubsub *pubsub.PubSub
joinedTopics map[string]*pubsub.Topic
joinedTopicsLock sync.RWMutex
subnetsLock map[uint64]*sync.RWMutex
subnetsLockLock sync.Mutex // Lock access to subnetsLock
initializationLock sync.Mutex
dv5Listener ListenerRebooter
startupErr error
ctx context.Context
host host.Host
genesisTime time.Time
genesisValidatorsRoot []byte
activeValidatorCount uint64
activeValidatorCountLock sync.Mutex
peerDisconnectionTime *cache.Cache
custodyInfo *custodyInfo
custodyInfoLock sync.RWMutex // Lock access to custodyInfo
custodyInfoSet chan struct{}
allForkDigests map[[4]byte]struct{}
crawler gossipsubcrawler.Crawler
}
type custodyInfo struct {
@@ -106,12 +114,16 @@ func NewService(ctx context.Context, cfg *Config) (*Service, error) {
ctx, cancel := context.WithCancel(ctx)
_ = cancel // govet fix for lost cancel. Cancel is handled in service.Stop().
cfg = validateConfig(cfg)
validateConfig(cfg)
privKey, err := privKey(cfg)
if err != nil {
return nil, errors.Wrapf(err, "failed to generate p2p private key")
}
p2pMaxPeers.Set(float64(cfg.MaxPeers))
minimumPeersPerSubnet.Set(float64(flags.Get().MinimumPeersPerSubnet))
metaData, err := metaDataFromDB(ctx, cfg.DB)
if err != nil {
log.WithError(err).Error("Failed to create peer metadata")
@@ -235,6 +247,21 @@ func (s *Service) Start() {
s.dv5Listener = listener
go s.listenForNewNodes()
crawler, err := NewGossipsubPeerCrawler(
s,
s.dv5Listener,
crawlTimeout,
crawlInterval,
maxConcurrentDials,
gossipsubcrawler.PeerFilterFunc(s.filterPeer),
s.Peers().Scorers().Score,
)
if err != nil {
log.WithError(err).Fatal("Failed to create peer crawler")
s.startupErr = err
return
}
s.crawler = crawler
}
s.started = true
@@ -305,12 +332,19 @@ func (s *Service) Start() {
func (s *Service) Stop() error {
defer s.cancel()
s.started = false
if s.dv5Listener != nil {
s.dv5Listener.Close()
}
return nil
}
// Crawler returns the p2p service's peer crawler.
func (s *Service) Crawler() gossipsubcrawler.Crawler {
return s.crawler
}
// Status of the p2p service. Will return an error if the service is considered unhealthy to
// indicate that this node should not serve traffic until the issue has been resolved.
func (s *Service) Status() error {
@@ -551,3 +585,15 @@ func (s *Service) downscorePeer(peerID peer.ID, reason string) {
newScore := s.Peers().Scorers().BadResponsesScorer().Increment(peerID)
log.WithFields(logrus.Fields{"peerID": peerID, "reason": reason, "newScore": newScore}).Debug("Downscore peer")
}
func AttestationSubnets(nodeID enode.ID, node *enode.Node, record *enr.Record) (map[uint64]bool, error) {
return attestationSubnets(record)
}
func SyncSubnets(nodeID enode.ID, node *enode.Node, record *enr.Record) (map[uint64]bool, error) {
return syncSubnets(record)
}
func DataColumnSubnets(nodeID enode.ID, node *enode.Node, record *enr.Record) (map[uint64]bool, error) {
return dataColumnSubnets(nodeID, record)
}

View File

@@ -8,11 +8,12 @@ import (
"sync"
"time"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/beacon-chain/cache"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/gossipsubcrawler"
"github.com/OffchainLabs/prysm/v6/cmd/beacon-chain/flags"
fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
"github.com/OffchainLabs/prysm/v6/config/params"
"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
"github.com/OffchainLabs/prysm/v6/consensus-types/wrapper"
@@ -24,7 +25,6 @@ import (
"github.com/ethereum/go-ethereum/p2p/enr"
"github.com/holiman/uint256"
"github.com/pkg/errors"
"github.com/prysmaticlabs/go-bitfield"
"github.com/sirupsen/logrus"
)
@@ -84,8 +84,7 @@ func (s *Service) nodeFilter(topic string, indices map[uint64]int) (func(node *e
// In this case, the function returns an error.
func (s *Service) FindAndDialPeersWithSubnets(
ctx context.Context,
topicFormat string,
digest [fieldparams.VersionLength]byte,
fullTopicForSubnet func(uint64) string,
minimumPeersPerSubnet int,
subnets map[uint64]bool,
) error {
@@ -103,33 +102,34 @@ func (s *Service) FindAndDialPeersWithSubnets(
maxConcurrentDials = flags.Get().MaxConcurrentDials
}
defectiveSubnets := s.defectiveSubnets(topicFormat, digest, minimumPeersPerSubnet, subnets)
defectiveSubnets := s.defectiveSubnets(fullTopicForSubnet, minimumPeersPerSubnet, subnets)
for len(defectiveSubnets) > 0 {
defectiveSubnets = s.defectiveSubnets(fullTopicForSubnet, minimumPeersPerSubnet, subnets)
// Stop the search/dialing loop if the context is canceled.
if err := ctx.Err(); err != nil {
return err
}
peersToDial, err := func() ([]*enode.Node, error) {
ctx, cancel := context.WithTimeout(ctx, batchPeriod)
defer cancel()
peersToDial, err := s.findPeersWithSubnets(ctx, topicFormat, digest, minimumPeersPerSubnet, defectiveSubnets)
if err != nil && !errors.Is(err, context.DeadlineExceeded) {
return nil, errors.Wrap(err, "find peers with subnets")
var peersToDial []*enode.Node
for subnet := range defectiveSubnets {
topic := fullTopicForSubnet(subnet)
peersToDial = append(peersToDial, s.crawler.PeersForTopic(gossipsubcrawler.Topic(topic))...)
}
if len(peersToDial) > minimumPeersPerSubnet {
peersToDial = peersToDial[:minimumPeersPerSubnet]
}
if len(peersToDial) == 0 {
select {
case <-time.After(100 * time.Millisecond):
case <-ctx.Done():
return ctx.Err()
}
return peersToDial, nil
}()
if err != nil {
return err
continue
}
// Dial new peers in batches.
s.dialPeers(s.ctx, maxConcurrentDials, peersToDial)
defectiveSubnets = s.defectiveSubnets(topicFormat, digest, minimumPeersPerSubnet, subnets)
}
return nil
@@ -158,8 +158,7 @@ func updateDefectiveSubnets(
// It returns new peers found during the search.
func (s *Service) findPeersWithSubnets(
ctx context.Context,
topicFormat string,
digest [fieldparams.VersionLength]byte,
fullTopicForSubnet func(uint64) string,
minimumPeersPerSubnet int,
defectiveSubnetsOrigin map[uint64]int,
) ([]*enode.Node, error) {
@@ -181,7 +180,13 @@ func (s *Service) findPeersWithSubnets(
}()
// Retrieve the filter function that will be used to filter nodes based on the defective subnets.
filter, err := s.nodeFilter(topicFormat, defectiveSubnets)
// Use any subnet's full topic to infer the family type from the topic string.
var sampleTopic string
for k := range defectiveSubnets {
sampleTopic = fullTopicForSubnet(k)
break
}
filter, err := s.nodeFilter(sampleTopic, defectiveSubnets)
if err != nil {
return nil, errors.Wrap(err, "node filter")
}
@@ -225,8 +230,8 @@ func (s *Service) findPeersWithSubnets(
nodeSubnets, err := filter(node)
if err != nil {
log.WithError(err).WithFields(logrus.Fields{
"nodeID": node.ID(),
"topicFormat": topicFormat,
"nodeID": node.ID(),
"topic": sampleTopic,
}).Debug("Could not get needed subnets from peer")
continue
@@ -241,7 +246,7 @@ func (s *Service) findPeersWithSubnets(
nodeByNodeID[node.ID()] = node
updateDefectiveSubnets(nodeSubnets, defectiveSubnets)
filter, err = s.nodeFilter(topicFormat, defectiveSubnets)
filter, err = s.nodeFilter(sampleTopic, defectiveSubnets)
if err != nil {
return nil, errors.Wrap(err, "node filter")
}
@@ -258,14 +263,13 @@ func (s *Service) findPeersWithSubnets(
// defectiveSubnets returns a map of subnets that have fewer than the minimum peer count.
func (s *Service) defectiveSubnets(
topicFormat string,
digest [fieldparams.VersionLength]byte,
fullTopicForSubnet func(uint64) string,
minimumPeersPerSubnet int,
subnets map[uint64]bool,
) map[uint64]int {
missingCountPerSubnet := make(map[uint64]int, len(subnets))
for subnet := range subnets {
topic := fmt.Sprintf(topicFormat, digest, subnet) + s.Encoding().ProtocolSuffix()
topic := fullTopicForSubnet(subnet)
peers := s.pubsub.ListPeers(topic)
peerCount := len(peers)
if peerCount < minimumPeersPerSubnet {
@@ -514,18 +518,18 @@ func initializePersistentSubnets(id enode.ID, epoch primitives.Epoch) error {
//
// return [compute_subscribed_subnet(node_id, epoch, index) for index in range(SUBNETS_PER_NODE)]
func computeSubscribedSubnets(nodeID enode.ID, epoch primitives.Epoch) ([]uint64, error) {
beaconConfig := params.BeaconConfig()
cfg := params.BeaconConfig()
if flags.Get().SubscribeToAllSubnets {
subnets := make([]uint64, 0, beaconConfig.AttestationSubnetCount)
for i := range beaconConfig.AttestationSubnetCount {
subnets := make([]uint64, 0, cfg.AttestationSubnetCount)
for i := range cfg.AttestationSubnetCount {
subnets = append(subnets, i)
}
return subnets, nil
}
subnets := make([]uint64, 0, beaconConfig.SubnetsPerNode)
for i := range beaconConfig.SubnetsPerNode {
subnets := make([]uint64, 0, cfg.SubnetsPerNode)
for i := range cfg.SubnetsPerNode {
sub, err := computeSubscribedSubnet(nodeID, epoch, i)
if err != nil {
return nil, errors.Wrap(err, "compute subscribed subnet")

View File

@@ -7,6 +7,7 @@ import (
"testing"
"time"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/beacon-chain/cache"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
testDB "github.com/OffchainLabs/prysm/v6/beacon-chain/db/testing"
@@ -23,7 +24,6 @@ import (
"github.com/ethereum/go-ethereum/p2p/enr"
"github.com/libp2p/go-libp2p/core/crypto"
"github.com/libp2p/go-libp2p/core/network"
"github.com/prysmaticlabs/go-bitfield"
)
func TestStartDiscV5_FindAndDialPeersWithSubnet(t *testing.T) {
@@ -114,8 +114,23 @@ func TestStartDiscV5_FindAndDialPeersWithSubnet(t *testing.T) {
require.NoError(t, err)
require.Equal(t, true, nodeForkDigest == bootNodeForkDigest, "fork digest of the node doesn't match the boot node")
// Start the service.
service.Start()
// Start the service.
service.Start()
// start the crawler with a topic extractor that maps ENR attestation subnets
// to full attestation topics for the current fork digest and encoding.
_ = service.Crawler().Start(func(ctx context.Context, node *enode.Node) ([]string, error) {
subs, err := attestationSubnets(node.Record())
if err != nil {
return nil, err
}
var topics []string
for subnet := range subs {
t := fmt.Sprintf(AttestationSubnetTopicFormat, bootNodeForkDigest, subnet) + service.Encoding().ProtocolSuffix()
topics = append(topics, t)
}
return topics, nil
})
// Set the ENR `attnets`, used by Prysm to filter peers by subnet.
bitV := bitfield.NewBitvector64()
@@ -161,23 +176,40 @@ func TestStartDiscV5_FindAndDialPeersWithSubnet(t *testing.T) {
service.genesisValidatorsRoot = params.BeaconConfig().GenesisValidatorsRoot[:]
service.custodyInfo = &custodyInfo{}
service.Start()
service.Start()
// start the crawler with a topic extractor that maps ENR attestation subnets
// to full attestation topics for the current fork digest and encoding.
_ = service.Crawler().Start(func(ctx context.Context, node *enode.Node) ([]string, error) {
subs, err := attestationSubnets(node.Record())
if err != nil {
return nil, err
}
var topics []string
for subnet := range subs {
t := fmt.Sprintf(AttestationSubnetTopicFormat, bootNodeForkDigest, subnet) + service.Encoding().ProtocolSuffix()
topics = append(topics, t)
}
return topics, nil
})
defer func() {
err := service.Stop()
require.NoError(t, err)
}()
subnets := map[uint64]bool{1: true, 2: true, 3: true}
defectiveSubnets := service.defectiveSubnets(AttestationSubnetTopicFormat, bootNodeForkDigest, minimumPeersPerSubnet, subnets)
builder := func(idx uint64) string {
return fmt.Sprintf(AttestationSubnetTopicFormat, bootNodeForkDigest, idx) + service.Encoding().ProtocolSuffix()
}
defectiveSubnets := service.defectiveSubnets(builder, minimumPeersPerSubnet, subnets)
require.Equal(t, subnetCount, len(defectiveSubnets))
ctxWithTimeOut, cancel := context.WithTimeout(ctx, 5*time.Second)
ctxWithTimeOut, cancel := context.WithTimeout(ctx, 15*time.Second)
defer cancel()
err = service.FindAndDialPeersWithSubnets(ctxWithTimeOut, AttestationSubnetTopicFormat, bootNodeForkDigest, minimumPeersPerSubnet, subnets)
err = service.FindAndDialPeersWithSubnets(ctxWithTimeOut, builder, minimumPeersPerSubnet, subnets)
require.NoError(t, err)
defectiveSubnets = service.defectiveSubnets(AttestationSubnetTopicFormat, bootNodeForkDigest, minimumPeersPerSubnet, subnets)
defectiveSubnets = service.defectiveSubnets(builder, minimumPeersPerSubnet, subnets)
require.Equal(t, 0, len(defectiveSubnets))
}
@@ -524,12 +556,12 @@ func TestSubnetComputation(t *testing.T) {
require.NoError(t, err)
localNode := enode.NewLocalNode(db, convertedKey)
beaconConfig := params.BeaconConfig()
cfg := params.BeaconConfig()
t.Run("standard", func(t *testing.T) {
retrievedSubnets, err := computeSubscribedSubnets(localNode.ID(), 1000)
require.NoError(t, err)
require.Equal(t, beaconConfig.SubnetsPerNode, uint64(len(retrievedSubnets)))
require.Equal(t, cfg.SubnetsPerNode, uint64(len(retrievedSubnets)))
require.Equal(t, retrievedSubnets[0]+1, retrievedSubnets[1])
})
@@ -541,8 +573,8 @@ func TestSubnetComputation(t *testing.T) {
retrievedSubnets, err := computeSubscribedSubnets(localNode.ID(), 1000)
require.NoError(t, err)
require.Equal(t, beaconConfig.AttestationSubnetCount, uint64(len(retrievedSubnets)))
for i := range beaconConfig.AttestationSubnetCount {
require.Equal(t, cfg.AttestationSubnetCount, uint64(len(retrievedSubnets)))
for i := range cfg.AttestationSubnetCount {
require.Equal(t, i, retrievedSubnets[i])
}
})
@@ -762,10 +794,12 @@ func TestFindPeersWithSubnets_NodeDeduplication(t *testing.T) {
ctxWithTimeout, cancel := context.WithTimeout(ctx, 100*time.Millisecond)
defer cancel()
builder := func(idx uint64) string {
return fmt.Sprintf(AttestationSubnetTopicFormat, digest, idx) + s.Encoding().ProtocolSuffix()
}
result, err := s.findPeersWithSubnets(
ctxWithTimeout,
AttestationSubnetTopicFormat,
digest,
builder,
1,
tt.defectiveSubnets,
)
@@ -982,10 +1016,12 @@ func TestFindPeersWithSubnets_FilterPeerRemoval(t *testing.T) {
ctxWithTimeout, cancel := context.WithTimeout(ctx, 100*time.Millisecond)
defer cancel()
builder := func(idx uint64) string {
return fmt.Sprintf(AttestationSubnetTopicFormat, digest, idx) + s.Encoding().ProtocolSuffix()
}
result, err := s.findPeersWithSubnets(
ctxWithTimeout,
AttestationSubnetTopicFormat,
digest,
builder,
1,
tt.defectiveSubnets,
)
@@ -1105,10 +1141,12 @@ func TestFindPeersWithSubnets_received_bad_existing_node(t *testing.T) {
ctxWithTimeout, cancel := context.WithTimeout(ctx, 1*time.Second)
defer cancel()
builder := func(idx uint64) string {
return fmt.Sprintf(AttestationSubnetTopicFormat, digest, idx) + service.Encoding().ProtocolSuffix()
}
result, err := service.findPeersWithSubnets(
ctxWithTimeout,
AttestationSubnetTopicFormat,
digest,
builder,
1,
map[uint64]int{1: 2}, // Need 2 peers for subnet 1
)

View File

@@ -21,9 +21,9 @@ go_library(
deps = [
"//beacon-chain/core/peerdas:go_default_library",
"//beacon-chain/p2p/encoder:go_default_library",
"//beacon-chain/p2p/gossipsubcrawler:go_default_library",
"//beacon-chain/p2p/peers:go_default_library",
"//beacon-chain/p2p/peers/scorers:go_default_library",
"//config/fieldparams:go_default_library",
"//config/params:go_default_library",
"//consensus-types/blocks:go_default_library",
"//consensus-types/interfaces:go_default_library",

View File

@@ -4,8 +4,8 @@ import (
"context"
"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/encoder"
"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/gossipsubcrawler"
"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/peers"
fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
@@ -41,6 +41,11 @@ func (*FakeP2P) AddConnectionHandler(_, _ func(ctx context.Context, id peer.ID)
}
// Crawler -- fake.
func (*FakeP2P) Crawler() gossipsubcrawler.Crawler {
return &MockCrawler{}
}
// AddDisconnectionHandler -- fake.
func (*FakeP2P) AddDisconnectionHandler(_ func(ctx context.Context, id peer.ID) error) {
}
@@ -71,7 +76,7 @@ func (*FakeP2P) DiscoveryAddresses() ([]multiaddr.Multiaddr, error) {
}
// FindAndDialPeersWithSubnets mocks the p2p func.
func (*FakeP2P) FindAndDialPeersWithSubnets(ctx context.Context, topicFormat string, digest [fieldparams.VersionLength]byte, minimumPeersPerSubnet int, subnets map[uint64]bool) error {
func (*FakeP2P) FindAndDialPeersWithSubnets(ctx context.Context, fullTopicForSubnet func(uint64) string, minimumPeersPerSubnet int, subnets map[uint64]bool) error {
return nil
}

View File

@@ -4,7 +4,7 @@ import (
"context"
"errors"
fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/gossipsubcrawler"
"github.com/ethereum/go-ethereum/p2p/enode"
"github.com/ethereum/go-ethereum/p2p/enr"
"github.com/libp2p/go-libp2p/core/host"
@@ -58,9 +58,14 @@ func (m *MockPeerManager) DiscoveryAddresses() ([]multiaddr.Multiaddr, error) {
func (*MockPeerManager) RefreshPersistentSubnets() {}
// FindAndDialPeersWithSubnet .
func (*MockPeerManager) FindAndDialPeersWithSubnets(ctx context.Context, topicFormat string, digest [fieldparams.VersionLength]byte, minimumPeersPerSubnet int, subnets map[uint64]bool) error {
func (*MockPeerManager) FindAndDialPeersWithSubnets(ctx context.Context, fullTopicForSubnet func(uint64) string, minimumPeersPerSubnet int, subnets map[uint64]bool) error {
return nil
}
// AddPingMethod .
func (*MockPeerManager) AddPingMethod(_ func(ctx context.Context, id peer.ID) error) {}
// Crawler.
func (*MockPeerManager) Crawler() gossipsubcrawler.Crawler {
return nil
}

View File

@@ -13,9 +13,9 @@ import (
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas"
"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/encoder"
"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/gossipsubcrawler"
"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/peers"
"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p/peers/scorers"
fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
"github.com/OffchainLabs/prysm/v6/config/params"
"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
@@ -183,11 +183,7 @@ func (p *TestP2P) ReceivePubSub(topic string, msg proto.Message) {
if _, err := p.Encoding().EncodeGossip(buf, castedMsg); err != nil {
p.t.Fatalf("Failed to encode message: %v", err)
}
digest, err := p.ForkDigest()
if err != nil {
p.t.Fatal(err)
}
topicHandle, err := ps.Join(fmt.Sprintf(topic, digest) + p.Encoding().ProtocolSuffix())
topicHandle, err := ps.Join(topic)
if err != nil {
p.t.Fatal(err)
}
@@ -279,6 +275,9 @@ func (p *TestP2P) SubscribeToTopic(topic string, opts ...pubsub.SubOpt) (*pubsub
// LeaveTopic closes topic and removes corresponding handler from list of joined topics.
// This method will return error if there are outstanding event handlers or subscriptions.
func (p *TestP2P) LeaveTopic(topic string) error {
p.mu.Lock()
defer p.mu.Unlock()
if t, ok := p.joinedTopics[topic]; ok {
if err := t.Close(); err != nil {
return err
@@ -420,7 +419,7 @@ func (p *TestP2P) Peers() *peers.Status {
}
// FindAndDialPeersWithSubnets mocks the p2p func.
func (*TestP2P) FindAndDialPeersWithSubnets(ctx context.Context, topicFormat string, digest [fieldparams.VersionLength]byte, minimumPeersPerSubnet int, subnets map[uint64]bool) error {
func (*TestP2P) FindAndDialPeersWithSubnets(ctx context.Context, fullTopicForSubnet func(uint64) string, minimumPeersPerSubnet int, subnets map[uint64]bool) error {
return nil
}
@@ -558,3 +557,35 @@ func (s *TestP2P) custodyGroupCountFromPeerENR(pid peer.ID) uint64 {
return custodyGroupCount
}
// MockCrawler is a minimal mock implementation of PeerCrawler for testing
type MockCrawler struct{}
// Start does nothing as this is a mock
func (m *MockCrawler) Start(gossipsubcrawler.TopicExtractor) error {
return nil
}
// Stop does nothing as this is a mock
func (m *MockCrawler) Stop() {}
// SetTopicExtractor does nothing as this is a mock
func (m *MockCrawler) SetTopicExtractor(extractor func(context.Context, *enode.Node) ([]string, error)) error {
return nil
}
// RemoveTopic does nothing as this is a mock
func (m *MockCrawler) RemoveTopic(topic gossipsubcrawler.Topic) {}
// RemovePeerID does nothing as this is a mock
func (m *MockCrawler) RemovePeerId(pid peer.ID) {}
// PeersForTopic returns empty list as this is a mock
func (m *MockCrawler) PeersForTopic(topic gossipsubcrawler.Topic) []*enode.Node {
return []*enode.Node{}
}
// Crawler returns a mock crawler implementation for testing.
func (*TestP2P) Crawler() gossipsubcrawler.Crawler {
return &MockCrawler{}
}

View File

@@ -13,6 +13,7 @@ import (
"path"
"time"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/beacon-chain/db"
"github.com/OffchainLabs/prysm/v6/beacon-chain/db/kv"
"github.com/OffchainLabs/prysm/v6/config/params"
@@ -28,7 +29,6 @@ import (
"github.com/libp2p/go-libp2p/core/crypto"
"github.com/libp2p/go-libp2p/core/peer"
"github.com/pkg/errors"
"github.com/prysmaticlabs/go-bitfield"
"github.com/sirupsen/logrus"
)

View File

@@ -10,6 +10,7 @@ import (
"testing"
"time"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/api"
"github.com/OffchainLabs/prysm/v6/api/server"
"github.com/OffchainLabs/prysm/v6/api/server/structs"
@@ -42,7 +43,6 @@ import (
"github.com/OffchainLabs/prysm/v6/time/slots"
"github.com/ethereum/go-ethereum/common/hexutil"
"github.com/pkg/errors"
"github.com/prysmaticlabs/go-bitfield"
)
func TestListAttestations(t *testing.T) {

View File

@@ -12,6 +12,7 @@ import (
"testing"
"time"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/api"
"github.com/OffchainLabs/prysm/v6/api/server/structs"
"github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg"
@@ -44,7 +45,6 @@ import (
"github.com/ethereum/go-ethereum/common/hexutil"
"github.com/pkg/errors"
ssz "github.com/prysmaticlabs/fastssz"
"github.com/prysmaticlabs/go-bitfield"
logTest "github.com/sirupsen/logrus/hooks/test"
"github.com/stretchr/testify/mock"
"go.uber.org/mock/gomock"

View File

@@ -9,6 +9,7 @@ import (
"runtime"
"testing"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/api/server/structs"
mock "github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/testing"
"github.com/OffchainLabs/prysm/v6/beacon-chain/p2p"
@@ -28,7 +29,6 @@ import (
"github.com/ethereum/go-ethereum/p2p/enr"
"github.com/libp2p/go-libp2p/core/peer"
ma "github.com/multiformats/go-multiaddr"
"github.com/prysmaticlabs/go-bitfield"
)
type dummyIdentity enode.ID

View File

@@ -4,6 +4,7 @@ import (
"encoding/json"
"fmt"
"net/http"
"slices"
"strconv"
"strings"
@@ -208,8 +209,13 @@ func (s *Server) attRewardsState(w http.ResponseWriter, r *http.Request) (state.
httputil.HandleError(w, "Attestation rewards are not supported for Phase 0", http.StatusNotFound)
return nil, false
}
currentEpoch := uint64(slots.ToEpoch(s.TimeFetcher.CurrentSlot()))
if requestedEpoch+1 >= currentEpoch {
currentEpoch := slots.ToEpoch(s.TimeFetcher.CurrentSlot())
bufferedEpoch, err := primitives.Epoch(requestedEpoch).SafeAdd(1)
if err != nil {
httputil.HandleError(w, "Could not increment epoch: "+err.Error(), http.StatusNotFound)
return nil, false
}
if bufferedEpoch >= currentEpoch {
httputil.HandleError(w,
"Attestation rewards are available after two epoch transitions to ensure all attestations have a chance of inclusion",
http.StatusNotFound)
@@ -388,12 +394,9 @@ func syncRewardsVals(
scIndices := make([]primitives.ValidatorIndex, 0, len(allScIndices))
scVals := make([]*precompute.Validator, 0, len(allScIndices))
for _, valIdx := range valIndices {
for _, scIdx := range allScIndices {
if valIdx == scIdx {
scVals = append(scVals, allVals[valIdx])
scIndices = append(scIndices, valIdx)
break
}
if slices.Contains(allScIndices, valIdx) {
scVals = append(scVals, allVals[valIdx])
scIndices = append(scIndices, valIdx)
}
}

View File

@@ -4,12 +4,14 @@ import (
"bytes"
"encoding/json"
"fmt"
"math"
"net/http"
"net/http/httptest"
"strconv"
"strings"
"testing"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/api/server/structs"
mock "github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/testing"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/altair"
@@ -34,7 +36,6 @@ import (
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/OffchainLabs/prysm/v6/testing/util"
"github.com/pkg/errors"
"github.com/prysmaticlabs/go-bitfield"
)
func BlockRewardTestSetup(t *testing.T, ver int) (state.BeaconState, interfaces.SignedBeaconBlock, error) {
@@ -747,6 +748,18 @@ func TestAttestationRewards(t *testing.T) {
assert.Equal(t, http.StatusNotFound, e.Code)
assert.Equal(t, "Attestation rewards are available after two epoch transitions to ensure all attestations have a chance of inclusion", e.Message)
})
t.Run("epoch overflow", func(t *testing.T) {
url := "http://only.the.epoch.number.at.the.end.is.important/" + strconv.FormatUint(math.MaxUint64, 10)
request := httptest.NewRequest("POST", url, nil)
writer := httptest.NewRecorder()
writer.Body = &bytes.Buffer{}
s.AttestationRewards(writer, request)
assert.Equal(t, http.StatusNotFound, writer.Code)
e := &httputil.DefaultJsonError{}
require.NoError(t, json.Unmarshal(writer.Body.Bytes(), e))
assert.Equal(t, http.StatusNotFound, e.Code)
})
}
func TestSyncCommiteeRewards(t *testing.T) {

View File

@@ -11,6 +11,7 @@ import (
"testing"
"time"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/api"
"github.com/OffchainLabs/prysm/v6/api/server/structs"
mockChain "github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/testing"
@@ -43,7 +44,6 @@ import (
"github.com/OffchainLabs/prysm/v6/time/slots"
"github.com/ethereum/go-ethereum/common/hexutil"
"github.com/pkg/errors"
"github.com/prysmaticlabs/go-bitfield"
logTest "github.com/sirupsen/logrus/hooks/test"
)

View File

@@ -10,6 +10,7 @@ import (
"net/http/httptest"
"testing"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/api/server/structs"
chainMock "github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/testing"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
@@ -33,7 +34,6 @@ import (
"github.com/OffchainLabs/prysm/v6/testing/util"
"github.com/OffchainLabs/prysm/v6/time/slots"
"github.com/ethereum/go-ethereum/common/hexutil"
"github.com/prysmaticlabs/go-bitfield"
)
func individualVotesHelper(t *testing.T, request *structs.GetIndividualVotesRequest, s *Server) (string, *structs.GetIndividualVotesResponse) {

View File

@@ -10,6 +10,7 @@ import (
"strings"
"testing"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/api"
"github.com/OffchainLabs/prysm/v6/api/server/structs"
chainMock "github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/testing"
@@ -24,7 +25,6 @@ import (
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/OffchainLabs/prysm/v6/testing/util"
"github.com/ethereum/go-ethereum/common/hexutil"
"github.com/prysmaticlabs/go-bitfield"
)
func TestQueryBeaconState(t *testing.T) {

View File

@@ -7,6 +7,7 @@ import (
"testing"
"time"
"github.com/OffchainLabs/go-bitfield"
chainMock "github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/testing"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
dbTest "github.com/OffchainLabs/prysm/v6/beacon-chain/db/testing"
@@ -27,7 +28,6 @@ import (
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/OffchainLabs/prysm/v6/testing/util"
"github.com/OffchainLabs/prysm/v6/time/slots"
"github.com/prysmaticlabs/go-bitfield"
"google.golang.org/protobuf/proto"
)

View File

@@ -8,6 +8,7 @@ import (
"testing"
"time"
"github.com/OffchainLabs/go-bitfield"
mock "github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/testing"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/altair"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/epoch/precompute"
@@ -37,7 +38,6 @@ import (
"github.com/OffchainLabs/prysm/v6/testing/util"
prysmTime "github.com/OffchainLabs/prysm/v6/time"
"github.com/OffchainLabs/prysm/v6/time/slots"
"github.com/prysmaticlabs/go-bitfield"
"google.golang.org/protobuf/proto"
"google.golang.org/protobuf/types/known/emptypb"
)

View File

@@ -4,6 +4,7 @@ import (
"testing"
"time"
"github.com/OffchainLabs/go-bitfield"
mock "github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/testing"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
dbTest "github.com/OffchainLabs/prysm/v6/beacon-chain/db/testing"
@@ -15,7 +16,6 @@ import (
"github.com/OffchainLabs/prysm/v6/testing/assert"
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/OffchainLabs/prysm/v6/testing/util"
"github.com/prysmaticlabs/go-bitfield"
)
func TestServer_GetBlock(t *testing.T) {

View File

@@ -6,6 +6,7 @@ import (
"testing"
"time"
"github.com/OffchainLabs/go-bitfield"
mock "github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/testing"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/signing"
@@ -25,7 +26,6 @@ import (
"github.com/OffchainLabs/prysm/v6/testing/assert"
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/OffchainLabs/prysm/v6/testing/util"
"github.com/prysmaticlabs/go-bitfield"
)
func TestSubmitAggregateAndProof_Syncing(t *testing.T) {

View File

@@ -312,14 +312,14 @@ func (vs *Server) ProposeBeaconBlock(ctx context.Context, req *ethpb.GenericSign
rob, err := blocks.NewROBlockWithRoot(block, root)
if block.IsBlinded() {
block, blobSidecars, err = vs.handleBlindedBlock(ctx, block)
if errors.Is(err, builderapi.ErrBadGateway) {
log.WithError(err).Info("Optimistically proposed block - builder relay temporarily unavailable, block may arrive over P2P")
return &ethpb.ProposeResponse{BlockRoot: root[:]}, nil
}
} else if block.Version() >= version.Deneb {
blobSidecars, dataColumnSidecars, err = vs.handleUnblindedBlock(rob, req)
}
if err != nil {
if errors.Is(err, builderapi.ErrBadGateway) && block.IsBlinded() {
log.WithError(err).Info("Optimistically proposed block - builder relay temporarily unavailable, block may arrive over P2P")
return &ethpb.ProposeResponse{BlockRoot: root[:]}, nil
}
return nil, status.Errorf(codes.Internal, "%s: %v", "handle block failed", err)
}

View File

@@ -4,6 +4,7 @@ import (
"bytes"
"context"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
"github.com/OffchainLabs/prysm/v6/beacon-chain/state"
"github.com/OffchainLabs/prysm/v6/config/params"
@@ -18,7 +19,6 @@ import (
"github.com/OffchainLabs/prysm/v6/runtime/version"
"github.com/OffchainLabs/prysm/v6/time/slots"
"github.com/pkg/errors"
"github.com/prysmaticlabs/go-bitfield"
)
func (vs *Server) setSyncAggregate(ctx context.Context, blk interfaces.SignedBeaconBlock, headState state.BeaconState) {

View File

@@ -3,6 +3,7 @@ package validator
import (
"testing"
"github.com/OffchainLabs/go-bitfield"
chainmock "github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/testing"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
"github.com/OffchainLabs/prysm/v6/beacon-chain/operations/synccommittee"
@@ -16,7 +17,6 @@ import (
"github.com/OffchainLabs/prysm/v6/testing/assert"
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/OffchainLabs/prysm/v6/testing/util"
"github.com/prysmaticlabs/go-bitfield"
)
func TestProposer_GetSyncAggregate_OK(t *testing.T) {

View File

@@ -8,6 +8,7 @@ import (
"slices"
"sort"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/blocks"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/electra"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
@@ -23,7 +24,6 @@ import (
"github.com/OffchainLabs/prysm/v6/runtime/version"
"github.com/OffchainLabs/prysm/v6/time/slots"
"github.com/pkg/errors"
"github.com/prysmaticlabs/go-bitfield"
"github.com/sirupsen/logrus"
)

View File

@@ -4,10 +4,10 @@ import (
"cmp"
"slices"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
"github.com/OffchainLabs/prysm/v6/crypto/bls"
ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/go-bitfield"
)
// computeOnChainAggregate constructs a final aggregate form a list of network aggregates with equal attestation data.

View File

@@ -4,6 +4,7 @@ import (
"reflect"
"testing"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/config/params"
"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
"github.com/OffchainLabs/prysm/v6/crypto/bls/blst"
@@ -11,7 +12,6 @@ import (
ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
"github.com/OffchainLabs/prysm/v6/testing/assert"
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/prysmaticlabs/go-bitfield"
)
func Test_computeOnChainAggregate(t *testing.T) {

View File

@@ -7,6 +7,7 @@ import (
"strconv"
"testing"
"github.com/OffchainLabs/go-bitfield"
chainMock "github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/testing"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/electra"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/helpers"
@@ -21,7 +22,6 @@ import (
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/OffchainLabs/prysm/v6/testing/util"
"github.com/OffchainLabs/prysm/v6/time/slots"
"github.com/prysmaticlabs/go-bitfield"
)
func TestProposer_ProposerAtts_committeeAwareSort(t *testing.T) {

View File

@@ -6,6 +6,7 @@ import (
"testing"
"time"
"github.com/OffchainLabs/go-bitfield"
builderapi "github.com/OffchainLabs/prysm/v6/api/client/builder"
"github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg"
mock "github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/testing"
@@ -52,7 +53,6 @@ import (
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/common/hexutil"
"github.com/pkg/errors"
"github.com/prysmaticlabs/go-bitfield"
logTest "github.com/sirupsen/logrus/hooks/test"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"

View File

@@ -4,11 +4,11 @@ import (
"fmt"
"testing"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/config/params"
ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
aggtesting "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1/attestation/aggregation/testing"
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/prysmaticlabs/go-bitfield"
)
func BenchmarkProposerAtts_sortByProfitability(b *testing.B) {

View File

@@ -11,6 +11,7 @@ import (
"testing"
"time"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/api/server/structs"
mock "github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/testing"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/altair"
@@ -37,7 +38,6 @@ import (
prysmTime "github.com/OffchainLabs/prysm/v6/time"
"github.com/OffchainLabs/prysm/v6/time/slots"
"github.com/ethereum/go-ethereum/common/hexutil"
"github.com/prysmaticlabs/go-bitfield"
)
func addDefaultReplayerBuilder(s *Server, h stategen.HistoryAccessor) {

View File

@@ -9,6 +9,7 @@ import (
"testing"
"time"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/api/server/structs"
mock "github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/testing"
"github.com/OffchainLabs/prysm/v6/beacon-chain/core/epoch/precompute"
@@ -23,7 +24,6 @@ import (
"github.com/OffchainLabs/prysm/v6/runtime/version"
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/OffchainLabs/prysm/v6/testing/util"
"github.com/prysmaticlabs/go-bitfield"
)
func TestServer_GetValidatorPerformance(t *testing.T) {

View File

@@ -8,6 +8,7 @@ import (
"encoding/json"
"time"
"github.com/OffchainLabs/go-bitfield"
customtypes "github.com/OffchainLabs/prysm/v6/beacon-chain/state/state-native/custom-types"
fieldparams "github.com/OffchainLabs/prysm/v6/config/fieldparams"
"github.com/OffchainLabs/prysm/v6/consensus-types/interfaces"
@@ -15,7 +16,6 @@ import (
"github.com/OffchainLabs/prysm/v6/crypto/bls"
enginev1 "github.com/OffchainLabs/prysm/v6/proto/engine/v1"
ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/go-bitfield"
)
// BeaconState has read and write access to beacon state methods.

View File

@@ -4,6 +4,7 @@ import (
"encoding/json"
"sync"
"github.com/OffchainLabs/go-bitfield"
"github.com/OffchainLabs/prysm/v6/beacon-chain/state/fieldtrie"
customtypes "github.com/OffchainLabs/prysm/v6/beacon-chain/state/state-native/custom-types"
"github.com/OffchainLabs/prysm/v6/beacon-chain/state/state-native/types"
@@ -11,7 +12,6 @@ import (
"github.com/OffchainLabs/prysm/v6/consensus-types/primitives"
enginev1 "github.com/OffchainLabs/prysm/v6/proto/engine/v1"
ethpb "github.com/OffchainLabs/prysm/v6/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/go-bitfield"
)
// BeaconState defines a struct containing utilities for the Ethereum Beacon Chain state, defining

Some files were not shown because too many files have changed in this diff Show More