**Motivation**
During development need to run the `lodestar` from different
directories, current binary wrapper does not allow that. With this
change you can now run lodestar from any directory by just running
`/path/to/lodestar_repo/lodestar`
**Description**
- Add the bash source script directory support
**Steps to test or reproduce**
- Run it locally
**Motivation**
- when we end request to peers, they may respond with rate limit error
and we don't track it
- this is a prerequisite for rate limit prevention that I'll implement
later. We want to make sure `REQUEST_RATE_LIMITED` does not happen in
that branch
**Description**
- fix the malformed extracted error message, see the unit test
- throw `REQUEST_RATE_LIMITED` if message contains "rate limit", see
rate limited error messages of all clients
[here](https://github.com/ChainSafe/lodestar/issues/8065#issuecomment-3157266196)
- new metric to track out going error by reason
Closes#8065Closes#8110
**Test on dev nodes**
<img width="1506" height="358" alt="Screenshot 2025-08-06 at 15 52 46"
src="https://github.com/user-attachments/assets/9c8f2d4a-ef34-458d-81a9-85880b1624ac"
/>
---------
Co-authored-by: Tuyen Nguyen <twoeths@users.noreply.github.com>
**Motivation**
- https://github.com/ChainSafe/lodestar/issues/8097
- supersedes https://github.com/ChainSafe/lodestar/pull/8112
**Description**
- remove metrics from `CustodyConfig`. Instead, do metrics in
`BeaconChain` (set on initialization, set on update)
- add `initialCustodyGroupCount` opt that gets set at the CLI layer by
"doing the right thing" - comparing opts.supernode, the stored enr
value, and the config value.
- thread `initialCustodyGroupCount` thru the application
- update all `CustodyConfig` construction sites
**Motivation**
- Closes https://github.com/ChainSafe/lodestar/issues/8093
**Description**
- update validator custody group count (cgc) in the following cases:
- during registration if number of attached validators is increased,
this ensures we update during (or before) genesis
- `onForkChoiceFinalized`to account for effective balance changes of
attached validators
- use finalized state instead of head state as per
[spec](01454a2afc/specs/fulu/validator.md (L115))
Bumps [tmp](https://github.com/raszi/node-tmp) from 0.2.1 to 0.2.4.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/raszi/node-tmp/blob/master/CHANGELOG.md">tmp's
changelog</a>.</em></p>
<blockquote>
<h2>v0.2.2 (2024-02-28)</h2>
<h4>🐛 Bug Fix</h4>
<ul>
<li><a
href="https://redirect.github.com/raszi/node-tmp/pull/278">#278</a>
Closes <a
href="https://redirect.github.com/raszi/node-tmp/issues/268">#268</a>:
Revert "fix <a
href="https://redirect.github.com/raszi/node-tmp/issues/246">#246</a>:
remove any double quotes or single quotes… (<a
href="https://github.com/mbargiel"><code>@mbargiel</code></a>)</li>
</ul>
<h4>📝 Documentation</h4>
<ul>
<li><a
href="https://redirect.github.com/raszi/node-tmp/pull/279">#279</a>
Closes <a
href="https://redirect.github.com/raszi/node-tmp/issues/266">#266</a>:
move paragraph on graceful cleanup to the head of the documentation (<a
href="https://github.com/silkentrance"><code>@silkentrance</code></a>)</li>
</ul>
<h4>Committers: 5</h4>
<ul>
<li>Carsten Klein (<a
href="https://github.com/silkentrance"><code>@silkentrance</code></a>)</li>
<li>Dave Nicolson (<a
href="https://github.com/dnicolson"><code>@dnicolson</code></a>)</li>
<li>KARASZI István (<a
href="https://github.com/raszi"><code>@raszi</code></a>)</li>
<li>Maxime Bargiel (<a
href="https://github.com/mbargiel"><code>@mbargiel</code></a>)</li>
<li><a
href="https://github.com/robertoaceves"><code>@robertoaceves</code></a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="08fa3abac3"><code>08fa3ab</code></a>
Update version</li>
<li><a
href="1cf4ec5418"><code>1cf4ec5</code></a>
Merge commit from fork</li>
<li><a
href="188b25e529"><code>188b25e</code></a>
Fix GHSA-52f5-9888-hmc6</li>
<li><a
href="73b9fe45bb"><code>73b9fe4</code></a>
Add test case for GHSA-52f5-9888-hmc6</li>
<li><a
href="b8e2f29a75"><code>b8e2f29</code></a>
Remove broken tests</li>
<li><a
href="2892a027b4"><code>2892a02</code></a>
Remove outdated URL</li>
<li><a
href="f592318246"><code>f592318</code></a>
Reformat package.json</li>
<li><a
href="995ac8cc45"><code>995ac8c</code></a>
Merge pull request <a
href="https://redirect.github.com/raszi/node-tmp/issues/301">#301</a>
from raszi/dependabot/npm_and_yarn/braces-3.0.3</li>
<li><a
href="caa758d7b5"><code>caa758d</code></a>
Bump braces from 3.0.2 to 3.0.3</li>
<li><a
href="5f0b2525ed"><code>5f0b252</code></a>
Merge pull request <a
href="https://redirect.github.com/raszi/node-tmp/issues/297">#297</a>
from raszi/feat/release-v0.2.3</li>
<li>Additional commits viewable in <a
href="https://github.com/raszi/node-tmp/compare/v0.2.1...v0.2.4">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/ChainSafe/lodestar/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
**Motivation**
Some logs in the PeerDAS branch are inappropriate.
**Description**
This PR changes inappropriate PeerDAS log levels from `warn` and `error`
to `debug` level.
Closes#8107
**Steps to test or reproduce**
Run beacon node locally using `./scripts/dev/node*.sh` with --logLevel
`info` and `debug` and check the logs.
---------
Co-authored-by: Cayman <caymannava@gmail.com>
**Motivation**
~~as found in #8082, there is cyclic dependency between
`afterProcessEpoch` and `upgradeState*` like `upgradeStateToElectra`~~
this PR improves a couple of things:
- to prepare for the future: future contributor can just decide where to
put logic at and it's straightforward to them. Cyclic dependency does
not happen here because the logic of computing proposers is unchanged
for fulu, but it may happen for other fields in the future
- at fork boundary, (epoch 0 of fulu), we implicitly run pre-fork logic,
only run post-fork logic from epoch 1 and it's not great. In epoch
transition we should make everything explicit to make it easier to
maintain
- performance: we compute proposers for current epoch and next epoch
twice at epoch 0 of fulu
**Description**
- implement finalProcessEpoch(), this is to compute epoch data that
depends on `upgradeState*`
- call this after `upgradeState*()`
---------
Co-authored-by: Tuyen Nguyen <twoeths@users.noreply.github.com>
Co-authored-by: Nico Flaig <nflaig@protonmail.com>
**Motivation**
During sync we might not yet be in electra, hence `pendingDeposits` is
undefined and we mistakenly use clock slot to determine if we should
collect electra metrics but it should be based on state slot.
```
$ curl localhost:8008/metrics
TypeError: Cannot read properties of undefined (reading 'length')
at BeaconChain.onScrapeMetrics (file:///usr/app/packages/beacon-node/src/chain/chain.ts:1123:68)
at file:///usr/app/packages/beacon-node/src/chain/chain.ts:395:47
at GaugeExtra.collect (file:///usr/app/packages/beacon-node/src/metrics/utils/gauge.ts:19:7)
at GaugeExtra.get (/usr/app/node_modules/prom-client/lib/gauge.js:110:19)
at RegistryMetricCreator.getMetricsAsString (/usr/app/node_modules/prom-client/lib/registry.js:35:21)
at /usr/app/node_modules/prom-client/lib/registry.js:91:16
at Array.map (<anonymous>)
at RegistryMetricCreator.metrics (/usr/app/node_modules/prom-client/lib/registry.js:87:45)
at Server.onRequest (file:///usr/app/packages/beacon-node/src/metrics/server/http.ts:47:64)
```
**Description**
Use state slot instead of clock slot before gathering electra metrcis
This removes `NODE_CUSTODY_REQUIREMENT` config value as it doesn't seem
to be part of the spec. It seems like the purpose of this config right
now is to be able to pass down supernode requirements (ie. setting
requirement to 128) and potentially allow to override the custody
requirement for testing but this is a misuse how config should be
handled.
As discussed with @matthewkeil we don't need to manually set node
custody requirement or disable validator custody anymore and rather just
rely on `--supernode` flag to custody all groups or dynamic validator
custody.
**Motivation**
Fixes#8040
During fusaka-devnet-2 an issue was identified. There is a bug in Nimbus
where they disconnect us for bad protocol negotiation from lack of
StatusV2 support on their end. We get disconnected and then we
immediately redial the same peer. See issue #8040.
Adds idea of a "cool down" period. Implementation builds on how we do
normal score decay. Set the wait time for decay to the desired cool down
time.
Discovery will check for this before attempting to dial a peer, even
upon rediscovery.
**Motivation**
- https://github.com/ethereum/beacon-APIs/pull/546
**Description**
Add `GET /eth/v1/beacon/blobs/{block_id}` endpoint to retrieve blobs
- pre-fulu we can just return data stored in db
- post-fulu we need to reconstruct the blobs from stored data column
sidecars
**Motivation**
Based on https://github.com/ChainSafe/lodestar/pull/8079 our proposer
boost reorg implementation works as expected. To get more test coverage
and observe it on actual networks it makes sense to enable by default.
**Description**
Enable proposer boost reorg by default
**Motivation**
There was an [issue on fusaka-devnet-3 builder
flow](https://discord.com/channels/595666850260713488/1399324773758140426)
due to failed validator registrations. The problem was that we already
sent validator registrations pre-genesis and unless the registration
data (fee recipient and gas limit) changes we will keep sending the
cached registration data (as per
https://github.com/ChainSafe/lodestar/pull/4447) which reuses the
previous timestamp. Why this is problematic is because mev-relay will
reject those registrations if [timestamp is before genesis
time](344d5b67d5/services/api/service.go (L2903))
so it will keep rejecting them until we restart the validator client or
change registration data to clear the cache.
**Description**
Don't send validator registrations pre-genesis, ie. only start sending
and caching them once we are in epoch 0. This matches the behavior of
Prysm (see https://github.com/OffchainLabs/prysm/pull/12847) which has
been there for a while. The only downside I can see is that in epoch 0
we might not be able to produce builder blocks in the first few slots
but that is really only relevant for devnets or local kurtosis runs but
either way since mev-relay rejects the registrations anyways there is
not much we can do in that case.
**Motivation**
Right now we always compute shuffling during epoch transition when
processing proposer lookahead (`processProposerLookahead()`) post fulu
but it should already be in cache most of the time.
**Description**
Query shuffling from cache to compute proposer lookahead
**Motivation**
There was a regression in
https://github.com/ChainSafe/lodestar/pull/7810 which caused us to not
detect weak head when predicting proposer head.
**Description**
This PR addresses the issue to correctly detect weak head when
predicting proposer head and updates few logs / misc changes while
reviewing the code.
I tested this on Kurtosis by artificially delaying some blocks. Let's
look at the example of proposer for slot 54 which re-orged out the late
block from slot 53.
The block was received after 4 seconds into the slot via gossip
```
Received gossip block slot=53, root=0x77cc…92e1, currentSlot=53, peerId=16Uiu2HAmQ8AL3EKjxSs7V6AG1CA4xMXvWXnVEZ6jybSShxCX48Fy, delaySec=4.1579999923706055, pending=null, haveBlobs=0, expectedBlobs=0, recvToValLatency=0.003000020980834961
```
We correctly determined during block import that it's a weak block and
skipped fcu call
```
Block is weak. Should override forkchoice update blockRoot=0x77cc3e39bc0c843f25c9556141f30e16901516a49dfb956fdc68e9c6f98592e1, slot=53
Weak block detected. Skip fcu call in importBlock blockRoot=0x77cc3e39bc0c843f25c9556141f30e16901516a49dfb956fdc68e9c6f98592e1, slot=53
```
When preparing next slot, we predicted that next block will be built on
parent (at start of next slot)
```
Current head is weak. Predicting next block to be built on parent of head. slot=53, weakHead=0x77cc3e39bc0c843f25c9556141f30e16901516a49dfb956fdc68e9c6f98592e1, proposerHead=0xfe8dfdd3ebc9715bbf9fac23d14024cf9dc29bfe78d352a9ea7eeea5f8cd25f1
```
Which turned out to be the case and we re-orged out the weak head
```
Performing single-slot reorg to remove current weak head slot=54, proposerHead=0xfe8dfdd3ebc9715bbf9fac23d14024cf9dc29bfe78d352a9ea7eeea5f8cd25f1, weakHead=0x77cc3e39bc0c843f25c9556141f30e16901516a49dfb956fdc68e9c6f98592e1
```
The node (and other nodes) detected and accepted the re-org as part of
canonical chain
```
Chain reorg depth=2, epoch=1, slot=54, newHeadBlock=0x39be0f98e895e91bf245535becb2ef8d631abe9be6602cce87473d94719099d0, oldHeadBlock=0x77cc3e39bc0c843f25c9556141f30e16901516a49dfb956fdc68e9c6f98592e1, newHeadState=0x5afc8b88b1b805797ea1a37b86f956fb5560bf8bbfe3ed4a449f4cae1851cb6d, oldHeadState=0x2d79bf2fb26881b4efa41bedeae8181fb7f430ddaec839175baeb26a3250045c, executionOptimistic=false
```
<img width="726" height="503" alt="image"
src="https://github.com/user-attachments/assets/7298fe6f-cb4c-4de7-b5ce-f322bcfc7ca0"
/>
As per ACDC 160, we need `submitBlindedBlockV2` which has empty response
to cope with increasing size of blob bundles.
This PR adds `submitBlindedBlockV2`. Will call this endpoint and not
publish blinded blocks as they are published by builder post-fulu.
Spec reference: https://github.com/ethereum/builder-specs/pull/123
---------
Co-authored-by: Nico Flaig <nflaig@protonmail.com>
**Motivation**
Noticed https://github.com/ChainSafe/lodestar/pull/7841 breaks kurtosis
runs as we check against mainnet genesis state root as it's the
`defaultNetwork`.
**Description**
Only check genesis state root if network is specified, since this isn't
really relevant for mainnet anyways and just new networks that are
launched it doesn't really make a difference as in those cases we
explicitly need to set `--network` and genesis state root check will be
done.
