This PR adds gloas specific time basis points for validator duties
(attestation, aggregation, and sync contribution) and wires the
validator client to use them once `GloasForkEpoch` is active. For now,
`GloasForkEpoch` is set to far-future so behavior is unchanged until the
fork epoch is configured
Note to the rev: today sync message duty uses `waitOneThirdOrValidBlock`
which shares with attestation dead line, that's why sync message bps is
not used
Initialize state-diff database on new node startup
When starting a node with --enable-state-diff feature flag, the database
is now initialized appropriately based on the startup scenario:
- New database + checkpoint sync: Set offset to checkpoint state slot,
save checkpoint state as full snapshot
- New database + genesis sync: Set offset to 0, save genesis state as
full snapshot
- Existing database without state-diff: Log warning that feature is
ignored due to old schema
- Existing database with state-diff: Return error (restarts will be
handled in a future PR)
Review after #16049 has merged
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Bastin <43618253+Inspector-Butters@users.noreply.github.com>
<!-- Thanks for sending a PR! Before submitting:
1. If this is your first PR, check out our contribution guide here
https://docs.prylabs.network/docs/contribute/contribution-guidelines
You will then need to sign our Contributor License Agreement (CLA),
which will show up as a comment from a bot in this pull request after
you open it. We cannot review code without a signed CLA.
2. Please file an associated tracking issue if this pull request is
non-trivial and requires context for our team to understand. All
features and most bug fixes should have
an associated issue with a design discussed and decided upon. Small bug
fixes and documentation improvements don't need issues.
3. New features and bug fixes must have tests. Documentation may need to
be updated. If you're unsure what to update, send the PR, and we'll
discuss
in review.
4. Note that PRs updating dependencies and new Go versions are not
accepted.
Please file an issue instead.
5. A changelog entry is required for user facing issues.
-->
**What type of PR is this?**
Bug fix
**What does this PR do? Why is it needed?**
This PR fixes a typo which resulted in a wrong variable name to be
returned on the Beacon API `/eth/v1/config/spec` endpoint:
```
curl http://127.0.0.1:49183/eth/v1/config/spec
{"data":{"AGGREGRATE_DUE_BPS":"6667", [...]
```
I discovered the discrepancy while testing the change to these "BPS"
values in the Vero VC which checks spec values against the ones it ships
with.
**Which issues(s) does this PR fix?**
N/A
**Other notes for review**
**Acknowledgements**
- [x] I have read
[CONTRIBUTING.md](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md).
- [x] I have included a uniquely named [changelog fragment
file](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md#maintaining-changelogmd).
- [x] I have added a description with sufficient context for reviewers
to understand this PR.
- [x] I have tested that my changes work as expected and I added a
testing plan to the PR description (if applicable).
---------
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
<!-- Thanks for sending a PR! Before submitting:
1. If this is your first PR, check out our contribution guide here
https://docs.prylabs.network/docs/contribute/contribution-guidelines
You will then need to sign our Contributor License Agreement (CLA),
which will show up as a comment from a bot in this pull request after
you open it. We cannot review code without a signed CLA.
2. Please file an associated tracking issue if this pull request is
non-trivial and requires context for our team to understand. All
features and most bug fixes should have
an associated issue with a design discussed and decided upon. Small bug
fixes and documentation improvements don't need issues.
3. New features and bug fixes must have tests. Documentation may need to
be updated. If you're unsure what to update, send the PR, and we'll
discuss
in review.
4. Note that PRs updating dependencies and new Go versions are not
accepted.
Please file an issue instead.
5. A changelog entry is required for user facing issues.
-->
**What type of PR is this?**
Feature
**What does this PR do? Why is it needed?**
- upgrading go ethereum to 1.16.7
- enabling fulu e2e
- added new e2e field params and build option
- removes github.com/MariusVanDerWijden/FuzzyVM
v0.0.0-20240516070431-7828990cad7d and
github.com/MariusVanDerWijden/tx-fuzz v1.4.0
what changed
- e2e config on slots per epoch increased to match minimum 6->8 a 33%
increase in run time ( this is needed because field params only have
minimum presets and proposer look ahead feature uses it, so if it
doesn't match it fails)
- reduce presubmit epochs from 18 -> 10 and only run for electra -> fulu
- moves bellatrix -> fulu post merge test to post submit
**Which issues(s) does this PR fix?**
Fixes #
**Other notes for review**
**Acknowledgements**
- [x] I have read
[CONTRIBUTING.md](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md).
- [x] I have included a uniquely named [changelog fragment
file](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md#maintaining-changelogmd).
- [x] I have added a description to this PR with sufficient context for
reviewers to understand this PR.
---------
Co-authored-by: Preston Van Loon <preston@pvl.dev>
Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Adds a feature flag `--enable-proposer-preprocessing` to verify
individual signatures in the block right before proposing to fallback to
empty fields in case of failure.
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Gate the withdrawal sweep optimization (using min of validator count and
MaxValidatorsPerWithdrawalsSweep) behind a hidden feature flag that
defaults to false. Enable the flag for spectests to match consensus
spec.
The backported changes were from
[4788](https://github.com/ethereum/consensus-specs/pull/4788)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
This pr implements the Gloas builder registry and related beacon state
fields per the spec, including proto/SSZ updates and state-native wiring
for builders, payload availability, pending payments/withdrawals, and
expected withdrawals. This aligns BeaconState with the Gloas container
changes and adds supporting hashing/copy helpers.
Spec ref:
https://github.com/ethereum/consensus-specs/blob/master/specs/gloas/beacon-chain.md
#### This PR sets the foundation for the new logging features.
---
The goal of this big PR is the following:
1. Adding a log.go file to every package:
[_commit_](54f6396d4c)
- Writing a bash script that adds the log.go file to every package that
imports logrus, except the excluded packages, configured at the top of
the bash script.
- the log.go file creates a log variable and sets a field called
`package` to the full path of that package.
- I have tried to fix every error/problem that came from mass generation
of this file. (duplicate declarations, different prefix names, etc...)
- some packages had the log.go file from before, and had some helper
functions in there as well. I've moved all of them to a `log_helpers.go`
file within each package.
2. Create a CI rule which verifies that:
[_commit_](b799c3a0ef)
- every package which imports logrus, also has a log.go file, except the
excluded packages.
- the `package` field of each log.go variable, has the correct path. (to
detect when we move a package or change it's name)
- I pushed a commit with a manually changed log.go file to trigger the
ci check failure and it worked.
3. Alter the logging system to read the prefix from this `package` field
for every log while outputing:
[_commit_](b0c7f1146c)
- some packages have/want/need a different log prefix than their package
name (like `kv`). This can be solved by keeping a map of package paths
to prefix names somewhere.
---
**Some notes:**
- Please review everything carefully.
- I created the `prefixReplacement` map and populated the data that I
deemed necessary. Please check it and complain if something doesn't make
sense or is missing. I attached at the bottom, the list of all the
packages that used to use a different name than their package name as
their prefix.
- I have chosen to mark some packages to be excluded from this whole
process. They will either not log anything, or log without a prefix, or
log using their previously defined prefix. See the list of exclusions in
the bottom.
- I fixed all the tests that failed because of this change. These were
failing because they were expecting the old prefix to be in the
generated logs. I have changed those to expect the new `package` field
instead. This might not be a great solution. Ideally we might want to
remove this from the tests so they only test for relevant fields in the
logs. but this is a problem for another day.
- Please run the node with this config, and mention if you see something
weird in the logs. (use different verbosities)
- The CI workflow uses a script that basically runs the
`hack/gen-logs.sh` and checks that the git diff is zero. that script is
`hack/check-logs.sh`. This means that if one runs this script locally,
it will not actually _check_ anything, rather than just regenerate the
log.go files and fix any mistake. This might be confusing. Please
suggest solutions if you think it's a problem.
---
**A list of packages that used a different prefix than their package
names for their logs:**
- beacon-chain/cache/depositsnapshot/ package depositsnapshot, prefix
"cache"
- beacon-chain/core/transition/log.go — package transition, prefix
"state"
- beacon-chain/db/kv/log.go — package kv, prefix "db"
- beacon-chain/db/slasherkv/log.go — package slasherkv, prefix
"slasherdb"
- beacon-chain/db/pruner/pruner.go — package pruner, prefix "db-pruner"
- beacon-chain/light-client/log.go — package light_client, prefix
"light-client"
- beacon-chain/operations/attestations/log.go — package attestations,
prefix "pool/attestations"
- beacon-chain/operations/slashings/log.go — package slashings, prefix
"pool/slashings"
- beacon-chain/rpc/core/log.go — package core, prefix "rpc/core"
- beacon-chain/rpc/eth/beacon/log.go — package beacon, prefix
"rpc/beaconv1"
- beacon-chain/rpc/eth/validator/log.go — package validator, prefix
"beacon-api"
- beacon-chain/rpc/prysm/v1alpha1/beacon/log.go — package beacon, prefix
"rpc"
- beacon-chain/rpc/prysm/v1alpha1/validator/log.go — package validator,
prefix "rpc/validator"
- beacon-chain/state/stategen/log.go — package stategen, prefix
"state-gen"
- beacon-chain/sync/checkpoint/log.go — package checkpoint, prefix
"checkpoint-sync"
- beacon-chain/sync/initial-sync/log.go — package initialsync, prefix
"initial-sync"
- cmd/prysmctl/p2p/log.go — package p2p, prefix "prysmctl-p2p"
- config/features/log.go -- package features, prefix "flags"
- io/file/log.go — package file, prefix "fileutil"
- proto/prysm/v1alpha1/log.go — package eth, prefix "protobuf"
- validator/client/beacon-api/log.go — package beacon_api, prefix
"beacon-api"
- validator/db/kv/log.go — package kv, prefix "db"
- validator/db/filesystem/db.go — package filesystem, prefix "db"
- validator/keymanager/derived/log.go — package derived, prefix
"derived-keymanager"
- validator/keymanager/local/log.go — package local, prefix
"local-keymanager"
- validator/keymanager/remote-web3signer/log.go — package
remote_web3signer, prefix "remote-keymanager"
- validator/keymanager/remote-web3signer/internal/log.go — package
internal, prefix "remote-web3signer-
internal"
- beacon-chain/forkchoice/doubly... prefix is
"forkchoice-doublylinkedtree"
**List of excluded directories (their subdirectories are also
excluded):**
```
EXCLUDED_PATH_PREFIXES=(
"testing"
"validator/client/testutil"
"beacon-chain/p2p/testing"
"beacon-chain/rpc/eth/config"
"beacon-chain/rpc/prysm/v1alpha1/debug"
"tools"
"runtime"
"monitoring"
"io"
"cmd"
".well-known"
"changelog"
"hack"
"specrefs"
"third_party"
"bazel-out"
"bazel-bin"
"bazel-prysm"
"bazel-testlogs"
"build"
".github"
".jj"
".idea"
".vscode"
)
```
This PR introduces flag `--ignore-unviable-attestations` (replaces and
deprecates `--disable-last-epoch-targets`) to drop attestations whose
target state is not viable; default remains to process them unless
explicitly enabled.
**What type of PR is this?**
Other
**What does this PR do? Why is it needed?**
This pull request removes `NUMBER_OF_COLUMNS` and
`MAX_CELLS_IN_EXTENDED_MATRIX` configuration.
**Other notes for review**
Please read commit by commit, with commit messages.
**Acknowledgements**
- [x] I have read
[CONTRIBUTING.md](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md).
- [x] I have included a uniquely named [changelog fragment
file](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md#maintaining-changelogmd).
- [x] I have added a description to this PR with sufficient context for
reviewers to understand this PR.
**What type of PR is this?**
Other
**What does this PR do? Why is it needed?**
- Added log prefix to the `genesis` package.
- Added log prefix to the `params` package.
- `WithGenesisValidatorsRoot`: Use camelCase for log field param.
- Move `Origin checkpoint found in db` log from WARN to INFO, since it
is the expected behaviour.
**Other notes for review**
Please read commit by commit
**Acknowledgements**
- [x] I have read
[CONTRIBUTING.md](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md).
- [x] I have included a uniquely named [changelog fragment
file](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md#maintaining-changelogmd).
- [x] I have added a description to this PR with sufficient context for
reviewers to understand this PR.
<!-- Thanks for sending a PR! Before submitting:
1. If this is your first PR, check out our contribution guide here
https://docs.prylabs.network/docs/contribute/contribution-guidelines
You will then need to sign our Contributor License Agreement (CLA),
which will show up as a comment from a bot in this pull request after
you open it. We cannot review code without a signed CLA.
2. Please file an associated tracking issue if this pull request is
non-trivial and requires context for our team to understand. All
features and most bug fixes should have
an associated issue with a design discussed and decided upon. Small bug
fixes and documentation improvements don't need issues.
3. New features and bug fixes must have tests. Documentation may need to
be updated. If you're unsure what to update, send the PR, and we'll
discuss
in review.
4. Note that PRs updating dependencies and new Go versions are not
accepted.
Please file an issue instead.
5. A changelog entry is required for user facing issues.
-->
**What type of PR is this?**
Bug fix
**What does this PR do? Why is it needed?**
Allows for starting e2e tests from electra or a specific fork of
interest again. doesn't fix missing execution requests tests, nishant
reverted it.
**Which issues(s) does this PR fix?**
Fixes #
**Other notes for review**
**Acknowledgements**
- [x] I have read
[CONTRIBUTING.md](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md).
- [x] I have included a uniquely named [changelog fragment
file](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md#maintaining-changelogmd).
- [x] I have added a description to this PR with sufficient context for
reviewers to understand this PR.
---------
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
**What type of PR is this?**
Feature
**What does this PR do? Why is it needed?**
This PR integrates the state diff path into the `State()` function from
`db/kv`, which allows reading of states using the state diff db, when
the `EnableStateDiff` flag is enabled.
**Notes for reviewers:**
Files `kv/state_diff_test.go` and `config/features/config.go` only
contain renamings:
- `kv/state_diff_test.go`: rename `setDefaultExponents()` to
`setDefaultStateDiffExponents()` to be less vague.
- `config/features/config.go`: rename `enableStateDiff` to
`EnableStateDiff` to make it public.
* wip
* fixing tests
* adding script to update workspace for eth clients
* updating test sepc to 1.6.0 and fixing broadcaster test
* fix specrefs
* more ethspecify fixes
* still trying to fix ethspecify
* fixing attestation tests
* fixing sha for consensus specs
* removing script for now until i have something more standard
* fixing more p2p tests
* fixing discovery tests
* attempting to fix discovery test flakeyness
* attempting to fix port binding issue
* more attempts to fix flakey tests
* Revert "more attempts to fix flakey tests"
This reverts commit 25e8183703.
* Revert "attempting to fix port binding issue"
This reverts commit 583df8000d.
* Revert "attempting to fix discovery test flakeyness"
This reverts commit 3c76525870.
* Revert "fixing discovery tests"
This reverts commit 8c701bf3b9.
* Revert "fixing more p2p tests"
This reverts commit 140d5db203.
* Revert "fixing attestation tests"
This reverts commit 26ded244cb.
* fixing attestation tests
* fixing more p2p tests
* fixing discovery tests
* attempting to fix discovery test flakeyness
* attempting to fix port binding issue
* more attempts to fix flakey tests
* changelog
* fixing import
* adding some missing dependencies, but TestService_BroadcastAttestationWithDiscoveryAttempts is still failing
* attempting to fix test
* reverting test as it migrated to other pr
* reverting test
* fixing test from merge
* Fix `TestService_BroadcastAttestationWithDiscoveryAttempts`.
* Fix again `TestService_Start_OnlyStartsOnce`.
* fixing TestListenForNewNodes
* removing manual set of fulu epoch
* missed a few
* fixing subnet test
* Update beacon-chain/rpc/eth/config/handlers_test.go
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* removing a few more missed spots of reverting fulu epoch setting
* updating test name based on feedback
* fixing rest apis, they actually need the setting of the epoch due to the guard
---------
Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* Only use head if it's compatible with target
* Allow blocks from the previous epoch to be viable for checkpoints
* Add feature flag to make it configurable
* fix tests
* @satushh's review
* Manu's nit
* Use fields in logs
* exclude unscheduled forks from the schedule
* update tests that relied on fulu being far future
* don't mess with the version->epoch mapping
* LastFork cheats the tests by filtering by far_future_epoch
---------
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
* Update consensus spec to v1.6.0-alpha.4 and implement data column support for forkchoice spectests
* Apply suggestion from @prestonvanloon
Co-Authored-By: Preston Van Loon <pvanloon@offchainlabs.com>
---------
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* initialize genesis data asap at node start
* add genesis validation tests with embedded state verification
* Add test for hardcoded mainnet genesis validator root and time from init() function
* Add test for UnmarshalState in encoding/ssz/detect/configfork.go
* Add tests for genesis.Initialize
* Move genesis/embedded to genesis/internal/embedded
* Gazelle / BUILD fix
* James feedback
* Fix lint
* Revert lock
---------
Co-authored-by: Kasey <kasey@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@pvl.dev>
* Add entry for sequence number in chain-metadata bucket & Basic getter/setter
* Mark p2p-metadata flag as deprecated
* Fix metaDataFromConfig: use DB instead to get seqnum
* Save sequence number after updating the metadata
* Fix beacon-chain/p2p unit tests: add DB in config
* Add changelog
* Add ReadOnlyDatabaseWithSeqNum
* Code suggestion from Manu
* Remove seqnum getter at interface
---------
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
* removing ssz-only flag
* gaz
* reverting other uses of sszonly
* gaz
* adding kasey and radek's suggestions
* update changelog
* adding test
* radek advice with new headers and tests
* adding logs and fixing comments
* adding logs and fixing comments
* gaz
* Update validator/client/beacon-api/rest_handler_client.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update api/apiutil/header.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update api/apiutil/header.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* radek's comments
* adding another failing case based on radek's suggestion
* another unit test
---------
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Subnets subscription: Avoid dynamic subscribing blocking in case not enough peers per subnets are found.
* `subscribeWithParameters`: Use struct to avoid too many function parameters (no functional changes).
* Optimise subnets search.
Currently, when we are looking for peers in let's say data column sidecars subnets 3, 6 and 7, we first look for peers in subnet 3.
If, during the crawling, we meet some peers with subnet 6, we discard them (because we are exclusively looking for peers with subnet 3).
When we are happy, we start again with peers with subnet 6.
This commit optimizes that by looking for peers with satisfy our constraints in one look.
* Fix James' comment.
* Fix James' comment.
* Fix James' comment.
* Fix James' commnet.
* Fix James' comment.
* Fix James' comment.
* Fix James's comment.
* Simplify following James' comment.
* Fix James' comment.
* Update beacon-chain/sync/rpc_goodbye.go
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* Update config/params/config.go
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* Update beacon-chain/sync/subscriber.go
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* Fix Preston's comment.
* Fix Preston's comment.
* `TestService_BroadcastDataColumn`: Re-add sleep 50 ms.
* Fix Preston's comment.
* Update beacon-chain/p2p/subnets.go
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
---------
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
