This PR moves kzg commitments to bid. The rationale behind is captured
in this [issue](https://github.com/ethereum/consensus-specs/issues/4870)
* Moves blob KZG commitments to the earlier point where builder intent
is known
* Removes duplicated commitments from data column sidecars which saves
descent b/w per slot
* Enables nodes to start fetching blobs via getBlobs as soon as the bid
is received
* Slightly increases bid size and may add minor bidding channel latency
but the tradeoff favors lower network load and simpler DA handling
This pr implements the Gloas builder registry and related beacon state
fields per the spec, including proto/SSZ updates and state-native wiring
for builders, payload availability, pending payments/withdrawals, and
expected withdrawals. This aligns BeaconState with the Gloas container
changes and adds supporting hashing/copy helpers.
Spec ref:
https://github.com/ethereum/consensus-specs/blob/master/specs/gloas/beacon-chain.md
#### This PR sets the foundation for the new logging features.
---
The goal of this big PR is the following:
1. Adding a log.go file to every package:
[_commit_](54f6396d4c)
- Writing a bash script that adds the log.go file to every package that
imports logrus, except the excluded packages, configured at the top of
the bash script.
- the log.go file creates a log variable and sets a field called
`package` to the full path of that package.
- I have tried to fix every error/problem that came from mass generation
of this file. (duplicate declarations, different prefix names, etc...)
- some packages had the log.go file from before, and had some helper
functions in there as well. I've moved all of them to a `log_helpers.go`
file within each package.
2. Create a CI rule which verifies that:
[_commit_](b799c3a0ef)
- every package which imports logrus, also has a log.go file, except the
excluded packages.
- the `package` field of each log.go variable, has the correct path. (to
detect when we move a package or change it's name)
- I pushed a commit with a manually changed log.go file to trigger the
ci check failure and it worked.
3. Alter the logging system to read the prefix from this `package` field
for every log while outputing:
[_commit_](b0c7f1146c)
- some packages have/want/need a different log prefix than their package
name (like `kv`). This can be solved by keeping a map of package paths
to prefix names somewhere.
---
**Some notes:**
- Please review everything carefully.
- I created the `prefixReplacement` map and populated the data that I
deemed necessary. Please check it and complain if something doesn't make
sense or is missing. I attached at the bottom, the list of all the
packages that used to use a different name than their package name as
their prefix.
- I have chosen to mark some packages to be excluded from this whole
process. They will either not log anything, or log without a prefix, or
log using their previously defined prefix. See the list of exclusions in
the bottom.
- I fixed all the tests that failed because of this change. These were
failing because they were expecting the old prefix to be in the
generated logs. I have changed those to expect the new `package` field
instead. This might not be a great solution. Ideally we might want to
remove this from the tests so they only test for relevant fields in the
logs. but this is a problem for another day.
- Please run the node with this config, and mention if you see something
weird in the logs. (use different verbosities)
- The CI workflow uses a script that basically runs the
`hack/gen-logs.sh` and checks that the git diff is zero. that script is
`hack/check-logs.sh`. This means that if one runs this script locally,
it will not actually _check_ anything, rather than just regenerate the
log.go files and fix any mistake. This might be confusing. Please
suggest solutions if you think it's a problem.
---
**A list of packages that used a different prefix than their package
names for their logs:**
- beacon-chain/cache/depositsnapshot/ package depositsnapshot, prefix
"cache"
- beacon-chain/core/transition/log.go — package transition, prefix
"state"
- beacon-chain/db/kv/log.go — package kv, prefix "db"
- beacon-chain/db/slasherkv/log.go — package slasherkv, prefix
"slasherdb"
- beacon-chain/db/pruner/pruner.go — package pruner, prefix "db-pruner"
- beacon-chain/light-client/log.go — package light_client, prefix
"light-client"
- beacon-chain/operations/attestations/log.go — package attestations,
prefix "pool/attestations"
- beacon-chain/operations/slashings/log.go — package slashings, prefix
"pool/slashings"
- beacon-chain/rpc/core/log.go — package core, prefix "rpc/core"
- beacon-chain/rpc/eth/beacon/log.go — package beacon, prefix
"rpc/beaconv1"
- beacon-chain/rpc/eth/validator/log.go — package validator, prefix
"beacon-api"
- beacon-chain/rpc/prysm/v1alpha1/beacon/log.go — package beacon, prefix
"rpc"
- beacon-chain/rpc/prysm/v1alpha1/validator/log.go — package validator,
prefix "rpc/validator"
- beacon-chain/state/stategen/log.go — package stategen, prefix
"state-gen"
- beacon-chain/sync/checkpoint/log.go — package checkpoint, prefix
"checkpoint-sync"
- beacon-chain/sync/initial-sync/log.go — package initialsync, prefix
"initial-sync"
- cmd/prysmctl/p2p/log.go — package p2p, prefix "prysmctl-p2p"
- config/features/log.go -- package features, prefix "flags"
- io/file/log.go — package file, prefix "fileutil"
- proto/prysm/v1alpha1/log.go — package eth, prefix "protobuf"
- validator/client/beacon-api/log.go — package beacon_api, prefix
"beacon-api"
- validator/db/kv/log.go — package kv, prefix "db"
- validator/db/filesystem/db.go — package filesystem, prefix "db"
- validator/keymanager/derived/log.go — package derived, prefix
"derived-keymanager"
- validator/keymanager/local/log.go — package local, prefix
"local-keymanager"
- validator/keymanager/remote-web3signer/log.go — package
remote_web3signer, prefix "remote-keymanager"
- validator/keymanager/remote-web3signer/internal/log.go — package
internal, prefix "remote-web3signer-
internal"
- beacon-chain/forkchoice/doubly... prefix is
"forkchoice-doublylinkedtree"
**List of excluded directories (their subdirectories are also
excluded):**
```
EXCLUDED_PATH_PREFIXES=(
"testing"
"validator/client/testutil"
"beacon-chain/p2p/testing"
"beacon-chain/rpc/eth/config"
"beacon-chain/rpc/prysm/v1alpha1/debug"
"tools"
"runtime"
"monitoring"
"io"
"cmd"
".well-known"
"changelog"
"hack"
"specrefs"
"third_party"
"bazel-out"
"bazel-bin"
"bazel-prysm"
"bazel-testlogs"
"build"
".github"
".jj"
".idea"
".vscode"
)
```
* Add serialization code for state diffs
Adds serialization code for state diffs.
Adds code to create and apply state diffs
Adds fuzz tests and benchmarks for serialization/deserialization
Co-authored-by: Claude <noreply@anthropic.com>
* Add Fulu support
* Review #1
* gazelle
* Fix some fuzzers
* Failing cases from the fuzzers in consensus-types/hdiff
* Fix more fuzz tests
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* add comparison tests
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Use ConvertToElectra in UpgradeToElectra
* Add comments on constants
* Fix readEth1Data
* remove colons from error messages
* Add design doc
* Apply suggestions from code review
Bast
Co-authored-by: Bastin <43618253+Inspector-Butters@users.noreply.github.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Preston Van Loon <preston@pvl.dev>
Co-authored-by: Bastin <43618253+Inspector-Butters@users.noreply.github.com>
* create lc cache to track branches
* save lc stuff
* remove finalized data from LC cache on finalization
* read lc stuff
* edit tests
* changelog
* linter
* address commments
* address commments 2
* address commments 3
* address commments 4
* lint
* address commments 5 x_x
* set beacon lcStore to mimick registrable services
* clean up the error propagation
* pass the state to saveLCBootstrap since it's not saved in db yet
* Convert genesis times from seconds to time.Time
* Fixing failed forkchoice tests in a new commit so it doesn't get worse
Fixing failed spectest tests in a new commit so it doesn't get worse
Fixing forkchoice tests, then spectests
* Fixing forkchoice tests, then spectests. Now asking for help...
* Fix TestForkChoice_GetProposerHead
* Fix broken build
* Resolve TODO(preston) items
* Changelog fragment
* Resolve TODO(preston) items again
* Resolve lint issues
* Use consistant field names for sinceSlotStart (no spaces)
* Manu's feedback
* Renamed StartTime -> UnsafeStartTime, marked as deprecated because it doesn't handle overflow scenarios.
Renamed SlotTime -> StartTime
Renamed SlotAt -> At
Handled the error in cases where StartTime was used.
@james-prysm feedback
* Revert beacon-chain/blockchain/receive_block_test.go from 1b7844de
* Fixing issues after rebase
* Accepted suggestions from @potuz
* Remove CanonicalHeadSlot from merge conflicts
---------
Co-authored-by: potuz <potuz@prysmaticlabs.com>
* remove usages of params from proto packages
* make it harder to mess up the order of request limit args
* remove errant edit (Terence review)
* fix missed updates after sig change
---------
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
* PeerDAS: Implement the validation pipeline for data column sidecars received via gossip
* Fix Terence's comment
* Fix Terence's comment.
* Fix Terence's comment.
* include block in attr event and use stategen
* use no-copy state cache for proposal in same epoch
* only advance to the start of epoch
---------
Co-authored-by: Kasey <kasey@users.noreply.github.com>
* Migrate Prysm repo to Offchain Labs organization ahead of Pectra upgrade v6
* Replace prysmaticlabs with OffchainLabs on general markdowns
* Update mock
* Gazelle and add mock.go to excluded generated mock file
* Implement static analysis to prevent panics
* Add nopanic to nogo
* Fix violations and add exclusions
Fix violations and add exclusions for all
* Changelog fragment
* Use pass.Report instead of pass.Reportf
* Remove strings.ToLower for checking init method name
* Add exclusion for herumi init
* Move api/client/beacon template function to init and its own file
* Fix nopanic testcase
* change field IDs in `AggregateAttestationAndProofElectra`
* fix typo in `validator.proto`
* correct slashing indices length and shashings length
* check length in indexed attestation's `ToConsensus` method
* use `fieldparams.BLSSignatureLength`
* Add length checks for execution request
* fix typo in `beacon_state.proto`
* fix typo in `ssz_proto_library.bzl`
* fix error messages about incorrect types in block factory
* add Electra case to `BeaconBlockContainerToSignedBeaconBlock`
* move PeerDAS config items to PeerDAS section
* remove redundant params
* rename `PendingDepositLimit` to `PendingDepositsLimit`
* improve requests unmarshaling errors
* rename `get_validator_max_effective_balance` to `get_max_effective_balance`
* fix typo in `consolidations.go`
* rename `index` to `validator_index` in `PendingPartialWithdrawal`
* rename `randomByte` to `randomBytes` in `validators.go`
* fix for version in a comment in `validator.go`
* changelog <3
* Revert "rename `index` to `validator_index` in `PendingPartialWithdrawal`"
This reverts commit 87e4da0ea2.
* Prepare for future fork boilerplate.
* Implement the Fulu fork boilerplate.
* `Upgraded state to <fork> log`: Move from debug to info.
Rationale:
This log is the only one notifying the user a new fork happened.
A new fork is always a little bit stressful for a node operator.
Having at least one log indicating the client switched fork is something useful.
* Update testing/util/helpers.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Fix Radek's comment.
* Fix Radek's comment.
* Update beacon-chain/state/state-native/state_trie.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update beacon-chain/state/state-native/state_trie.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Fix Radek's comment.
* Fix Radek's comment.
* Fix Radek's comment.
* Remove Electra struct type aliasing.
---------
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
