Reverts https://github.com/ChainSafe/lodestar/pull/7448 to unhide prune
history option. I've been running this on mainnet for a while and it's
seems pretty stable with no noticable impact on performance. The feature
is already widely used even though it was hidden so might as well show
it in our docs.
Metrics from my node
<img width="1888" height="340" alt="image"
src="https://github.com/user-attachments/assets/223b7e6b-101e-4b4f-b06a-6d74f830bf96"
/>
I did a few tweaks to how we query keys but nothing really improved the
fetch keys duration.
Caveat still remains that it's quite slow on first startup if the
previous db is large but that's a one time operation.
Closes https://github.com/ChainSafe/lodestar/issues/7556
**Motivation**
Make sure all specs passes.
**Description**
- Fix the broken spec tests
- Add condition to fix the types used for deserialization.
Closes#7839
**Steps to test or reproduce**
Run all tests
---------
Co-authored-by: Cayman <caymannava@gmail.com>
**Motivation**
Make the types exports consistent for all packages.
All modern runtimes support [conditional
exports](https://nodejs.org/api/packages.html#conditional-exports) and
there are caveats when we have both conditional exports and normal
exports present in a package.json. This PR tend to make all exports
follow same consistent and modern pattern.
**Description**
- We were using subpath exports for some packages and module exports for
other
- Keep all the types export consistent as subpath exports.
- Remove "types" and "exports` directive from package.json
- Remove `typesVersions`, this is useful only if we have different
version of types for different versions of Typescript. Or having
different types files for different file paths.
**Steps to test or reproduce**
- Run all CI
**Motivation**
This was brought up in the Fusaka bug bounty, when we receive a already
known data column sidecar with the same block header and column index,
the gossip message is accepted and rebroadcast without any additional
verification.
This allows a malicious peer to send a data column sidecar with the same
block header and column index but an invalid block header signature
which we would accept and rebroadcast and get downscored by our peers.
**Description**
Ignore already known data column sidecars (based on block header and
column index). We could also consider to run
`validateGossipDataColumnSidecar` on those sidecars to penalize the node
sending us the data but it's just additional work for us and
`GossipAction.IGNORE` seems sufficient.
**Motivation**
- this task runs once per epoch so there should be no issue being more
verbose on the log
Co-authored-by: Tuyen Nguyen <twoeths@users.noreply.github.com>
**Motivation**
- https://github.com/ChainSafe/lodestar/issues/8526
**Description**
- In #8526 it was discovered that the leveldb controller did not respect
`reverse`
- Visual review of the code shows it doesn't respect `reverse` nor
`limit`
- Add support for both `reverse` and `limit`
This simplifies the calculation of `startEpoch` in head chain range
sync.
We check the remote finalized epoch against ours here
d9cc6b90f7/packages/beacon-node/src/sync/utils/remoteSyncType.ts (L33)
This means `remote.finalizedEpoch == local.finalizedEpoch` in this case
and local head epoch should always be >= finalized epoch which means we
can simplify use epoch of `local.headSlot` here.
cc @twoeths
**Motivation**
See https://github.com/ethereum/consensus-specs/pull/4650
**Description**
Ensure data column sidecars respect blob limit by checking that
`kzgCommitments.length` of each data column sidecar does not exceed
`getMaxBlobsPerBlock(epoch)`.
**Motivation**
- investigate and maintain the performance of
`processFinalizedCheckpoint()`
- this is part of #8526
**Description**
- track duration of `processFinalizedCheckpoint()` by tasks, the result
on a hoodi node, it shows that `FrequencyStateArchiveStrategy` takes the
most time
<img width="941" height="297" alt="Screenshot 2025-10-14 at 13 45 38"
src="https://github.com/user-attachments/assets/ef440399-538b-4a4a-a63c-e775745b25e6"
/>
- track different steps of `FrequencyStateArchiveStrategy`, the result
shows that the mainthread is blocked by different db queries cc
@wemeetagain
<img width="1291" height="657" alt="Screenshot 2025-10-14 at 13 46 36"
src="https://github.com/user-attachments/assets/3b19f008-c7d8-49a4-9dc5-e68b1a5ba2a5"
/>
part of #8526
---------
Co-authored-by: Tuyen Nguyen <twoeths@users.noreply.github.com>
**Motivation**
If we don't have the proposer index in cache when having to build a
block, we create a default entry
[here](d9cc6b90f7/packages/beacon-node/src/chain/produceBlock/produceBlockBody.ts (L196)).
This shouldn't happen in normal circumstances as proposers are
registered beforehand, however if you produce a block each slot for
testing purposes this affects the custody of the node as it will have up
to 32 validators in proposer cache (assuming a block each slot) and
since we never reduce the cgc it will stay at that value.
Logs from Ethpandaops from a node without attached validators but that's
producing a block each slot
```
Oct-14 09:12:00.005[chain] verbose: Updated target custody group count finalizedEpoch=272653, validatorCount=32, targetCustodyGroupCount=33
```
```
Oct-14 09:12:00.008[network] debug: Updated cgc field in ENR custodyGroupCount=33
```
**Description**
Do not create default cache entry for unknown proposers by using normal
`Map` and just fall back to `suggestedFeeRecipient` if there isn't any
value. The behavior from a caller perspective stays the same but we no
longer create a proposer cache entry for unknown proposers.
**Motivation**
Voluntary exit validation previously returned only a boolean, which
gives a vague error codes and may cause harder debugging.
This PR aims to improve debuggability by providing clearer error message
and feedback during validator exits.
**Description**
<!-- A clear and concise general description of the changes of this PR
commits -->
This PR introduces the VoluntaryExitValidity enum to provide granular
reasons for voluntary exit validation failures.
It refactors processVoluntaryExit and getVoluntaryExitValidity to return
specific validity states, rather than a simple boolean.
Beacon node validation logic now maps these validity results to error
codes (VoluntaryExitErrorCode) for clearer gossip and API handling.
This improves debuggability and aligns exit validation with consensus
spec requirements.
Closes#6330
---------
Co-authored-by: Nico Flaig <nflaig@protonmail.com>
**Motivation**
- got "heap size limit too low" warn in our Bun instance but has no idea
what's the exact value of it
**Description**
- include `heapSizeLimit` in the log
---------
Co-authored-by: Tuyen Nguyen <twoeths@users.noreply.github.com>
**Motivation**
To make transition to `getBlobs` api smoother and make Lodestar work
with v0.29.0 release of checkpointz
https://github.com/ethpandaops/checkpointz/issues/215, we should be
serving blob sidecars even post fulu from `getBlobSidecars` api.
**Description**
Instead of throwing an error, we do the following now post fulu
- fetch data column sidecars for block from db (if we custody at least
64 columns)
- reconstruct blobs from data column sidecars
- reconstruct blob sidecars from blobs (recompute kzg proof and
inclusion proof)
**Motivation**
- Review of #8468 metrics
- In https://github.com/ChainSafe/lodestar/pull/8449, use of
`datastore-level` was unilaterally removed in favor of the bun-supported
`datastore-fs`
- This caused a regression
**Description**
- use `datastore-level` by default, only use `datastore-fs` in bun
---------
Co-authored-by: Nico Flaig <nflaig@protonmail.com>
**Motivation**
Recently, we have been experiencing many instances of AI generated slop
submitted by first time contributors and a policy rework is needed to
ensure our time is respected whilst allowing an external contributor
freely contribute to further enhancing this project. This proposal
ensures that new contributors self-disclose any use of AI technologies
as part of their submission to minimize time waste and allow issues to
be resolved with thorough understanding of the output at hand.
**Description**
This PR:
- Adds an AI Assistance Notice to contribution.md
- Moves important context for first time contributors to the top of the
document
- Corrects minor grammar
Additional credit to @nflaig and [Ghostly's
policy](https://github.com/ghostty-org/ghostty/blob/main/CONTRIBUTING.md#ai-assistance-notice)
to approach minimizing this problem.
---------
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
**Motivation**
Using `*` as a CLI flag value is not bash-friendly
**Description**
Add backward-compatible support for the bash-friendly value `all` as an
alternative to `*` for the following CLI flags:
- `--rest.namespace` - now accepts both `*` and `all` to enable all API
namespaces
- `--rest.cors` - now accepts both `*` and `all` to allow all origins
- `--keymanager.cors` - now accepts both `*` and `all` to allow all
origins
Closes#5963
**Motivation**
Since https://github.com/ChainSafe/lodestar/pull/7927 we started to use
parent slot (the slot of parent block) to get sync aggregate for block
but this can cause `BLOCK_ERROR_INVALID_SIGNATURE` during fork
transition as might include sync aggregates into our block that are
older than block slot - 1 which is used to compute domain during
signature verification.
8961b06c11/packages/state-transition/src/block/processSyncCommittee.ts (L81)
This means we will reject our own block due to invalid signature if
there are missed slots during the epoch transition.
> we currently hard code parent slot as producedSlot - 1 which is
incorrect in reorg scenario
This was the motivation from the previous change but this doesn't seem
right, sync committee messages are produced at latest 4 seconds into the
slot so if we re-org the block due to it being weak/late then sync
committee votes will also be for parent block root of that block and not
the block itself.
So irrespective of block being reorged or not we should always include
sync committee messages from previous slot.
**Description**
Use previous slot to get sync aggregate for block to avoid
`BLOCK_ERROR_INVALID_SIGNATURE` in case there are missed slots.
Local docker builds (with existing build artifacts) no longer work since
https://github.com/ChainSafe/lodestar/pull/8481 because the file
`packages/cli/tsconfig.build.tsbuildinfo` is copied to docker build
which causes typescript to skip the build, but we are not copying the
`lib` which means the whole build is missing (and not rebuilt). The
tsbuildinfo is usually part of `lib` which is why this was no issue
before.
I believe this happens because `"rootDir": "src"` was added to tsconfig,
there might be a different way to fix this but in general we don't want
to copy tsbuildinfo files into docker builds.
Update gloas types in `packages/types/src/types.ts` to use the
containers defined in #8464 and fix all type error that come along with
it
---------
Co-authored-by: Nico Flaig <nflaig@protonmail.com>
- Replace TODO comment with detailed usage instructions
- Add step-by-step guide for local monitoring setup
- Include port information and service URLs
- Explain host.docker.internal configuration"
This avoids an issue we had previously where configs that extend mainnet
preset would inherit wrong parameters. This just ensures this cannot
happen in case we those will be forked after mainnet.
similar to https://github.com/ChainSafe/lodestar/pull/7401
Follow up on https://github.com/ChainSafe/lodestar/pull/8091 to clean up
remaining usage of `SECONDS_PER_SLOT`.
---------
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
**Motivation**
- when we remove an eagerly persisted block input, it's very normal that
we did not have any columns persisted
- but right now it threw this error as shown in #8457
```
Sep-23 19:16:55.002[chain] �[33mwarn�[39m: Error pruning eagerly imported block inputs, DB may grow in size if this error happens frequently slot=30 - Invalid dataColumnSidecars=0 for custody expected custodyColumnsLen=128
Error: Invalid dataColumnSidecars=0 for custody expected custodyColumnsLen=128
```
**Description**
- do not throw error in this case
part of #8457
Co-authored-by: Tuyen Nguyen <twoeths@users.noreply.github.com>
**Motivation**
- when investigating #8457 I found that we did not set `hashAllColumn()`
correctly at the 1st time
**Description**
- correct `hasAllData` when creating IBlockInput from column
- nice to have: in `verifyBlocksDataAvailability`, only wait for all
data if not has all data. This is to make it consistent to
`writeBlockInputToDb()` where the error did not happen, see
https://github.com/ChainSafe/lodestar/issues/8457#issuecomment-3337347372
Co-authored-by: Tuyen Nguyen <twoeths@users.noreply.github.com>
**Motivation**
- #7280
**Description**
- update `@types/node` to the latest node v22 version
- add `@lodestar/bun` dependency using git+https (for now! so that we
don't have to republish a bazillion times)
- keep in mind that this requires a manual build/rebuild of the so (via
`cd node_modules/@lodestar/bun && zig build`)
- Add a bun-specific `LevelDbController` which uses
[`@lodestar/bun`](https://github.com/ChainSafe/lodestar-bun)
- Only expose the leveldb controller via a subpath export (and for
tests, as a custom import, like in #8320 )
- add bun bun global override types
**Motivation**
- we see different slots for the same root in `BlockInputSync` logs,
this causes confusion
- this was added last week in #8416
**Description**
- the [root, slot] is not trusted in network processor because messages
could be originated from an out-of-sync node
- we actually don't need the slot in BlockInputSync, it's just a nice to
have but caused confusion. So I reverted that part: do not emit slot to
`BlockInputSync` from network processor
- log the full root hex
Closes#8465
---------
Co-authored-by: Tuyen Nguyen <twoeths@users.noreply.github.com>
**Motivation**
Add `src` directory to package sources along with the `lib` files to
have working source map.
**Description**
- Add `src` to the files.
**Steps to test or reproduce**
- Run all tests
