**What type of PR is this?**
Feature + Bugfix
**What does this PR do? Why is it needed?**
Starting at Fusaka, the beacon node can pull blobs with the
`engine_getBlobsV2` API from the execution layer.
This reduces by a lot the burden on the beacon node. However, the beacon
node should be able to work 100% correctly without this execution layer
help.
This PR introduces the `--disable-get-blobs-v2` flag to simulate a 0%
success rate of this engine API.
This PR also fixes:
- https://github.com/OffchainLabs/prysm/issues/16171
Please read commit by commit with commit messages.
**How to test it:**
For the `--disable-get-blobs-v2` part:
Run the beacon node with the `--disable-get-blobs-v2` flag in DEBUG
mode.
For every block with commitments, the following log should be displayed:
```
[2025-12-19 15:36:25.49] DEBUG sync: No data column sidecars constructed from the execution client ...
```
And the following log should **never** be displayed:
```
[2026-01-05 10:19:00.55] DEBUG sync: Constructed data column sidecars from the execution client count=...
```
For the #16171 part:
- No ERROR log showed in the linked issue should never be displayed.
**Acknowledgements**
- [x] I have read
[CONTRIBUTING.md](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md).
- [x] I have included a uniquely named [changelog fragment
file](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md#maintaining-changelogmd).
- [x] I have added a description with sufficient context for reviewers
to understand this PR.
- [x] I have tested that my changes work as expected and I added a
testing plan to the PR description (if applicable).
## Review after #16059
**What type of PR is this?**
Feature
**What does this PR do?**
This PR introduces logrus writer hooks into the logging of prysm.
when log-format is text:
- set the default logrus output to be `io.Discard`
- create a writer hook for terminal, with formatting and coloring
enabled.
- create a separate writer hook for log-file (if enabled), without
coloring.
This immediately allows for having formatted/colored terminal logs,
while keeping the log-file clean.
**What type of PR is this?**
Other
**What does this PR do? Why is it needed?**
- Added log prefix to the `genesis` package.
- Added log prefix to the `params` package.
- `WithGenesisValidatorsRoot`: Use camelCase for log field param.
- Move `Origin checkpoint found in db` log from WARN to INFO, since it
is the expected behaviour.
**Other notes for review**
Please read commit by commit
**Acknowledgements**
- [x] I have read
[CONTRIBUTING.md](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md).
- [x] I have included a uniquely named [changelog fragment
file](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md#maintaining-changelogmd).
- [x] I have added a description to this PR with sufficient context for
reviewers to understand this PR.
<!-- Thanks for sending a PR! Before submitting:
1. If this is your first PR, check out our contribution guide here
https://docs.prylabs.network/docs/contribute/contribution-guidelines
You will then need to sign our Contributor License Agreement (CLA),
which will show up as a comment from a bot in this pull request after
you open it. We cannot review code without a signed CLA.
2. Please file an associated tracking issue if this pull request is
non-trivial and requires context for our team to understand. All
features and most bug fixes should have
an associated issue with a design discussed and decided upon. Small bug
fixes and documentation improvements don't need issues.
3. New features and bug fixes must have tests. Documentation may need to
be updated. If you're unsure what to update, send the PR, and we'll
discuss
in review.
4. Note that PRs updating dependencies and new Go versions are not
accepted.
Please file an issue instead.
5. A changelog entry is required for user facing issues.
-->
**What type of PR is this?**
Feature
**What does this PR do? Why is it needed?**
| Feature | Semi-Supernode | Supernode |
| ----------------------- | ------------------------- |
------------------------ |
| **Custody Groups** | 64 | 128 |
| **Data Columns** | 64 | 128 |
| **Storage** | ~50% | ~100% |
| **Blob Reconstruction** | Yes (via Reed-Solomon) | No reconstruction
needed |
| **Flag** | `--semi-supernode` | `--supernode` |
| **Can serve all blobs** | Yes (with reconstruction) | Yes (directly) |
**note** if your validator total effective balance results in more
custody than the semi-supernode it will override those those
requirements.
cgc=64 from @nalepae
Pro:
- We are useful to the network
- Less disconnection likelihood
- Straight forward to implement
Con:
- We cannot revert to a full node
- We have to serve incoming RPC requests corresponding to 64 columns
Tested the following using this kurtosis setup
```
participants:
# Super-nodes
- el_type: geth
el_image: ethpandaops/geth:master
cl_type: prysm
vc_image: gcr.io/offchainlabs/prysm/validator:latest
cl_image: gcr.io/offchainlabs/prysm/beacon-chain:latest
count: 2
cl_extra_params:
- --supernode
vc_extra_params:
- --verbosity=debug
# Full-nodes
- el_type: geth
el_image: ethpandaops/geth:master
cl_type: prysm
vc_image: gcr.io/offchainlabs/prysm/validator:latest
cl_image: gcr.io/offchainlabs/prysm/beacon-chain:latest
count: 2
validator_count: 1
cl_extra_params:
- --semi-supernode
vc_extra_params:
- --verbosity=debug
additional_services:
- dora
- spamoor
spamoor_params:
image: ethpandaops/spamoor:master
max_mem: 4000
spammers:
- scenario: eoatx
config:
throughput: 200
- scenario: blobs
config:
throughput: 20
network_params:
fulu_fork_epoch: 0
withdrawal_type: "0x02"
preset: mainnet
global_log_level: debug
```
```
curl -H "Accept: application/json" http://127.0.0.1:32961/eth/v1/node/identity
{"data":{"peer_id":"16Uiu2HAm7xzhnGwea8gkcxRSC6fzUkvryP6d9HdWNkoeTkj6RSqw","enr":"enr:-Ni4QIH5u2NQz17_pTe9DcCfUyG8TidDJJjIeBpJRRm4ACQzGBpCJdyUP9eGZzwwZ2HS1TnB9ACxFMQ5LP5njnMDLm-GAZqZEXjih2F0dG5ldHOIAAAAAAAwAACDY2djQIRldGgykLZy_whwAAA4__________-CaWSCdjSCaXCErBAAE4NuZmSEAAAAAIRxdWljgjLIiXNlY3AyNTZrMaECulJrXpSOBmCsQWcGYzQsst7r3-Owlc9iZbEcJTDkB6qIc3luY25ldHMFg3RjcIIyyIN1ZHCCLuA","p2p_addresses":["/ip4/172.16.0.19/tcp/13000/p2p/16Uiu2HAm7xzhnGwea8gkcxRSC6fzUkvryP6d9HdWNkoeTkj6RSqw","/ip4/172.16.0.19/udp/13000/quic-v1/p2p/16Uiu2HAm7xzhnGwea8gkcxRSC6fzUkvryP6d9HdWNkoeTkj6RSqw"],"discovery_addresses":["/ip4/172.16.0.19/udp/12000/p2p/16Uiu2HAm7xzhnGwea8gkcxRSC6fzUkvryP6d9HdWNkoeTkj6RSqw"],"metadata":{"seq_number":"3","attnets":"0x0000000000300000","syncnets":"0x05","custody_group_count":"64"}}}
```
```
curl -s http://127.0.0.1:32961/eth/v1/debug/beacon/data_column_sidecars/head | jq '.data | length'
64
```
```
curl -X 'GET' \
'http://127.0.0.1:32961/eth/v1/beacon/blobs/head' \
-H 'accept: application/json'
```
**Which issues(s) does this PR fix?**
Fixes #
**Other notes for review**
**Acknowledgements**
- [x] I have read [CONTRIBUTING.md](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md).
- [x] I have included a uniquely named [changelog fragment file](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md#maintaining-changelogmd).
- [x] I have added a description to this PR with sufficient context for reviewers to understand this PR.
---------
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
Co-authored-by: james-prysm <jhe@offchainlabs.com>
Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
This change ensures FlagOptions in cmd/beacon-chain/execution/options.go
appends only one endpoint option depending on whether a JWT secret is
present. Previously the code always appended WithHttpEndpoint and then
conditionally appended WithHttpEndpointAndJWTSecret which overwrote the
first option, adding unnecessary allocations and cognitive overhead.
Since WithHttpEndpointAndJWTSecret fully configures the endpoint,
including URL and Bearer auth needed by the Engine API, the initial
WithHttpEndpoint is redundant when a JWT is supplied. The refactor
preserves behavior while simplifying option composition and avoiding
redundant state churn.
---------
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
<!-- Thanks for sending a PR! Before submitting:
1. If this is your first PR, check out our contribution guide here
https://docs.prylabs.network/docs/contribute/contribution-guidelines
You will then need to sign our Contributor License Agreement (CLA),
which will show up as a comment from a bot in this pull request after
you open it. We cannot review code without a signed CLA.
2. Please file an associated tracking issue if this pull request is
non-trivial and requires context for our team to understand. All
features and most bug fixes should have
an associated issue with a design discussed and decided upon. Small bug
fixes and documentation improvements don't need issues.
3. New features and bug fixes must have tests. Documentation may need to
be updated. If you're unsure what to update, send the PR, and we'll
discuss
in review.
4. Note that PRs updating dependencies and new Go versions are not
accepted.
Please file an issue instead.
5. A changelog entry is required for user facing issues.
-->
**What type of PR is this?**
Bug fix
**What does this PR do? Why is it needed?**
Previously, JWT secrets longer than 256 bits could cause client
compatibility issues. For example, Prysm would accept longer secrets
while Geth strictly requires exactly 32 bytes, causing Geth startup
failures when using the same secret file.
This change enforces the Engine API specification requirement that JWT
secrets must be exactly 256 bits (32 bytes), ensuring consistent
behavior across different client implementations.
**Which issues(s) does this PR fix?**
Fixes #
**Other notes for review**
**Acknowledgements**
- [x] I have read
[CONTRIBUTING.md](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md).
- [x] I have included a uniquely named [changelog fragment
file](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md#maintaining-changelogmd).
- [x] I have added a description to this PR with sufficient context for
reviewers to understand this PR.
---------
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
* Set default value of `--blob-batch-limit` to 384.
So, using default values, `--blob-batch-limit * --blob-batch-limit-burst-factor = 384*3 = MAX_REQUEST_BLOB_SIDECARS = 1152.`
* `blobSidecarByRootRPCHandler`: Add rate limiting.
Bacause now the rate limiter validation is done before the request validation,
adapt `TestBlobsByRootValidation` consequently and add new specific tests for
`validateBlobByRootRequest` to cover the now untested case.
* Set default value of `--data-column-batch-limit-burst-factor` to 4.
So, using default values, `--data-column-batch-limit * --data-column-batch-limit-burst-factor = 4096*2 = MAX_REQUEST_DATA_COLUMN_SIDECARS_ELECTRA = 16384`.
* `validateDataColumnsByRootRequest`: Take a count instead of idents.
* `dataColumnSidecarByRootRPCHandler`: Add rate limiting.
* default new blob storage layouts to by-epoch
also, do not log migration message until we see a directory that needs to be migrated
Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
* manu feedback
---------
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
* remove "experimental" from backfill flag name
* backwards-compatible alias
* changelog
---------
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
* Add flag for colocation whitelisting. --p2p-ip-colocation-whitelist
This change updates the peer IP colocation checking to respect the
configured CIDR whitelist (--p2p-ip-colocation-whitelist flag).
Changes:
- Added IPColocationWhitelist field to peers.StatusConfig
- Added ipColocationWhitelist field to Status struct to store parsed IPNets
- Parse CIDR strings into net.IPNet in NewStatus constructor
- Updated isfromBadIP method to skip colocation limits for whitelisted IPs
- Pass IPColocationWhitelist from Service config when creating Status
The IP colocation whitelist allows operators to exempt specific IP ranges
from the colocation limit, useful for deployments with known trusted
address ranges or legitimate node clustering.
Only check if an IP is in the whitelist when the colocation limit
is actually exceeded, rather than checking for every IP. This is
more efficient and matches the intended behavior.
* Changelog fragment
* Apply suggestion from @nalepae
Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
* Apply suggestion from @nalepae
Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
* @kasey feedback: Move IP colocation parsing to the node construction
---------
Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
* `startBaseServices`: Warm data column storage cache.
* `TestFindPeers_NodeDeduplication`: Use `t.context`.
* `BUILD.bazel`: Moge `# gazelle.ignore` at the top of the file.
Rationale: This directive is applied to the whole file, regardless its position in the file.
* Improve `TestConstructGenericBeaconBlock`: Courtesy of Terence
* Add `TestDataColumnStoragePath_FlagSpecified`.
* `appFlags`: Move `flags.SubscribeAllDataSubnets` (cosmetic).
* `appFlags`: Add `storage.DataColumnStoragePathFlag`.
* Add changelog.
* initialize genesis data asap at node start
* add genesis validation tests with embedded state verification
* Add test for hardcoded mainnet genesis validator root and time from init() function
* Add test for UnmarshalState in encoding/ssz/detect/configfork.go
* Add tests for genesis.Initialize
* Move genesis/embedded to genesis/internal/embedded
* Gazelle / BUILD fix
* James feedback
* Fix lint
* Revert lock
---------
Co-authored-by: Kasey <kasey@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@pvl.dev>
* Add entry for sequence number in chain-metadata bucket & Basic getter/setter
* Mark p2p-metadata flag as deprecated
* Fix metaDataFromConfig: use DB instead to get seqnum
* Save sequence number after updating the metadata
* Fix beacon-chain/p2p unit tests: add DB in config
* Add changelog
* Add ReadOnlyDatabaseWithSeqNum
* Code suggestion from Manu
* Remove seqnum getter at interface
---------
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
* Add log capitalization analyzer and apply fixes across codebase
Implements a new nogo analyzer to enforce proper log message capitalization and applies the fixes to all affected log statements throughout the beacon chain, validator, and supporting components.
Co-Authored-By: Claude <noreply@anthropic.com>
* Radek's feedback
---------
Co-authored-by: Claude <noreply@anthropic.com>
* `dataColumnsRPCMinValidSlot`: Add new test case.
* peerDAS: Implement `dataColumnSidecarByRangeRPCHandler`.
* `rateLimitingAmount`: Define out of the function.
* Fix James' comment.
* Fix James' comment.
* Migrate Prysm repo to Offchain Labs organization ahead of Pectra upgrade v6
* Replace prysmaticlabs with OffchainLabs on general markdowns
* Update mock
* Gazelle and add mock.go to excluded generated mock file
* Implement static analysis to prevent panics
* Add nopanic to nogo
* Fix violations and add exclusions
Fix violations and add exclusions for all
* Changelog fragment
* Use pass.Report instead of pass.Reportf
* Remove strings.ToLower for checking init method name
* Add exclusion for herumi init
* Move api/client/beacon template function to init and its own file
* Fix nopanic testcase
* Beacon flags: Comment on deprecated section
* Beacon flags: Organize flags relevant to logging, comment on logging section
* Beacon flags: Organize flags relevant to p2p, comment on p2p section
* Beacon flags: Introduce db flag section, organize flags relevant to db, comment on db section
* Beacon flags: Introduce builder flag section, organize flags relevant to builder, comment on builder section
* Beacon flags: Introduce sync flag section, organize flags relevant to sync, comment on sync section
* Beacon flags: Introduce execution layer flag section, organize flags relevant to execution layer, comment on execution layer section
* Beacon flags: Introduce monitoring flag section, organize flags relevant to monitoring, comment on monitoring section
* Beacon flags: Organizing remaining flags in cmd and beacon-chain sections
* Beacon flags: Introduce slasher flag section, organize flags relevant to slasher, comment on slasher section
* Move slasher flag from features to the slasher section
* Changelog fragment
* Beacon flags: Reorganize sections
* Move MaxGoroutines to debug section
* organize blob directories by period and epoch
* changelog
* remove Indices and replace with Summary
* old PR feedback
* log to advise about the speed of blob migration
* rename level->layer (hoping term is more clear)
* assert path in tests for increased legibility
* lint
* lint
* remove test covering a newly impossible error
* improve feedback from flag validation failure
* Try to clean dangling dirs epoch->flat migration
* lint
* Preston feedback
* try all layouts and short-circuit if base not found
---------
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
* updating the goethereum dependency
* fixing dependencies
* reverting workspace
* more fixes, work in progress
* trying with upgraded geth version
* fixing deprecated functions except for the time related ones on eth1 distance due to time issues
* fixing time issues
* gaz
* fixing test and upgrading some dependencies and reverting others
* Disable cgo in hid, delete old vendored usb library
* changelog
* rolling back dependencies
* fixing go mod tidy
* Geth v1.13.6
* fix tests
* Add ping interval, set to 500ms for tests. This didnt work
* Update to v1.14.8
* Spread it out to different bootnodes
* Fix it
* Remove Memsize
* Update all out of date dependencies
* Fix geth body change
* Fix Test
* Fix Build
* Fix Tests
* Fix Tests Again
* Fix Tests Again
* Fix Tests
* Fix Test
* Copy USB Package for HID
* Push it up
* Finally fix all tests with felix's changes
* updating geth dependency
* Update go-ethereum to v1.14.11
* fixing import
* reverting blob change
* fixing Implicit memory aliasing in for loop.
* WIP changes
* wip getting a little further on e2e runs
* getting a little further
* getting a little further
* setting everything to capella
* more partial fixes
* more fixes but still WIP
* fixing access list transactions"
* some cleanup
* making configs dynamic
* reverting time
* skip lower bound in builder
* updating to geth v1.14.12
* fixing verify blob to pointer
* go mod tidy
* fixing linting
* missed removing another terminal difficulty item
* another missed update
* updating more dependencies to fix cicd
* fixing holiman dependency update
* downgrading geth to 1.14.11 due to p2p loop issue
* reverting builder middleware caused by downgrade
* fixing more rollback issues
* upgrading back to 1.14.12 after discussing with preston
* mod tidy
* gofmt
* partial review feedback
* trying to start e2e from bellatrix instead
* reverting some changes
---------
Co-authored-by: Preston Van Loon <preston@pvl.dev>
Co-authored-by: nisdas <nishdas93@gmail.com>
* wip passing e2e
* reverting temp comment
* remove unneeded comments
* fixing merge errors
* fixing more bugs from merge
* fixing test
* WIP moving code around and fixing tests
* unused linting
* gaz
* temp removing these tests as we need placeholder/wrapper APIs for them with the removal of the gateway
* attempting to remove dependencies to gRPC gateway , 1 mroe left in deps.bzl
* renaming flags and other gateway services to http
* goimport
* fixing deepsource
* git mv
* Update validator/package/validator.yaml
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update validator/package/validator.yaml
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update cmd/beacon-chain/flags/base.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update cmd/beacon-chain/flags/base.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update cmd/beacon-chain/flags/base.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* addressing feedback
* missed lint
* renaming import
* reversal based on feedback
* fixing web ui registration
* don't require mux handler
* gaz
* removing gRPC service from validator completely, merged with http service, renames are a work in progress
* updating go.sum
* linting
* trailing white space
* realized there was more cleanup i could do with code reuse
* adding wrapper for routes
* reverting version
* fixing dependencies from merging develop
* gaz
* fixing unit test
* fixing dependencies
* reverting unit test
* fixing conflict
* updating change log
* Update log.go
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* gaz
* Update api/server/httprest/server.go
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* addressing some feedback
* forgot to remove deprecated flag in usage
* gofmt
* fixing test
* fixing deepsource issue
* moving deprecated flag and adding timeout handler
* missed removal of a flag
* fixing test:
* Update CHANGELOG.md
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* addressing feedback
* updating comments based on feedback
* removing unused field for now, we can add it back in if we need to use the option
* removing unused struct
* changing api-timeout flag based on feedback
---------
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
