Compare commits

..

33 Commits

Author SHA1 Message Date
rakita
0f09a92acc fix: update gas_used/gas_refunded to ResultGas API 2026-02-12 16:47:57 +01:00
rakita
df8ba50cb7 chore: update revm/inspectors/alloy-evm patches and fix Bytecode flatten 2026-02-12 16:37:56 +01:00
rakita
61341a1342 Merge remote-tracking branch 'origin/main' into staging
# Conflicts:
#	Cargo.lock
#	crates/optimism/evm/src/lib.rs
2026-02-12 16:32:00 +01:00
rakita
3d02124e50 Merge origin/main into staging 2026-02-03 13:24:01 +01:00
rakita
953a1b3399 chore: update revm, revm-inspectors, alloy-evm to staging
- revm: 6aa06829d2caa2aa38606ed22b83354a7a7ff98e
- revm-inspectors: cc4f62d8c107b3d2dc42a220690a079cdfc8dfb1
- alloy-evm: 80e6a436ecade2b9bd950fd70bd0d6a45ef0a43e
2026-02-03 13:22:25 +01:00
rakita
a364989c61 Merge origin/main into staging 2026-02-03 13:14:17 +01:00
rakita
f4075f5926 Merge remote staging 2026-01-28 19:12:11 +01:00
rakita
211d3d2924 fix: adapt to revm hasher type changes 2026-01-28 19:11:48 +01:00
rakita
dbc5313a2c Merge origin/main into staging - update patches 2026-01-28 19:09:24 +01:00
rakita
345fc9cfd2 Merge branch 'main' into staging 2026-01-26 22:35:00 +01:00
rakita
33d61c30cb chore: update revm, revm-inspectors, alloy-evm to staging
Updates dependencies:
- revm to 300efbf3e391e1796f5210cd4506508e385a55d2
- revm-inspectors to 9464524e7d983c8601ba7af3048cd60575dad6bd
- alloy-evm to 1c8ff5a179e760517016ab4e4d5af1c4b5923594

Fixes:
- Add slot_num field to BlockEnv for EIP-7843 SLOTNUM opcode
- Update BlockHashCache API: use lowest() instead of keys()
2026-01-26 21:32:08 +01:00
DaniPopes
beb0c5e527 chore: reduce number of nightly builds (#21446) 2026-01-26 21:30:39 +01:00
DaniPopes
73c39279b1 chore: remove unused docker from makefile (#21445) 2026-01-26 21:30:39 +01:00
ethfanWilliam
2fd0a703e2 fix(stages): retain RocksDB TempDir in TestStageDB to prevent premature deletion (#21444) 2026-01-26 21:30:39 +01:00
Dan Cline
7dcd77de95 fix(pruner): prune account and storage changeset static files (#21346) 2026-01-26 21:30:39 +01:00
Dan Cline
da6e6afe78 chore(metrics): add a gas_last metric similar to new_payload_last (#21437) 2026-01-26 21:30:39 +01:00
Brian Picciano
04d9a33c68 refactor(trie): always use ParallelSparseTrie, deprecate config flags (#21435) 2026-01-26 21:30:39 +01:00
Arsenii Kulikov
be1f657b3c perf: use shared channel for prewarm workers (#21429) 2026-01-26 21:30:39 +01:00
Rez
cab4cbf0ea feat: configurable EVM execution limits (#21088)
Co-authored-by: Arsenii Kulikov <klkvrr@gmail.com>
2026-01-26 21:30:39 +01:00
figtracer
29f51abb22 feat(rpc): add transaction hash caching to EthStateCache (#21180)
Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>
Co-authored-by: Amp <amp@ampcode.com>
2026-01-26 21:30:39 +01:00
Matthias Seitz
dd0ee0709c fix(rpc): add block timestamp validation in eth_simulateV1 (#21397)
Co-authored-by: Amp <amp@ampcode.com>
2026-01-26 21:30:39 +01:00
Matthias Seitz
dcd9e50663 fix(rpc): use correct error codes for eth_simulateV1 reverts and halts (#21412)
Co-authored-by: Amp <amp@ampcode.com>
2026-01-26 21:30:39 +01:00
Matthias Seitz
59d11378b9 fix(rpc): add block number validation in eth_simulateV1 (#21396)
Co-authored-by: Amp <amp@ampcode.com>
2026-01-26 21:30:39 +01:00
Matthias Seitz
70ceb755ad fix(rpc): populate block_hash in eth_simulateV1 logs (#21413)
Co-authored-by: Amp <amp@ampcode.com>
2026-01-26 21:30:39 +01:00
Matthias Seitz
a49991c766 feat(cli): make stopping on invalid block the default for reth import (#21403)
Co-authored-by: Amp <amp@ampcode.com>
2026-01-26 21:30:39 +01:00
Matthias Seitz
0a77c2aae1 feat(rpc): implement movePrecompileToAddress for eth_simulateV1 (#21414)
Co-authored-by: Amp <amp@ampcode.com>
2026-01-26 21:30:39 +01:00
Matthias Seitz
93adbf82a0 fix(rpc): set prevrandao to zero for eth_simulateV1 simulated blocks (#21399)
Co-authored-by: Amp <amp@ampcode.com>
2026-01-26 21:30:39 +01:00
Matthias Seitz
ce08b6f265 fix(rpc): cap simulate_v1 default gas limit to RPC gas cap (#21402)
Co-authored-by: Amp <amp@ampcode.com>
2026-01-26 21:30:39 +01:00
Matthias Seitz
b03a704a1e fix(engine): only warn for critical capability mismatches (#21398)
Co-authored-by: Amp <amp@ampcode.com>
2026-01-26 21:30:39 +01:00
Andrey Kolishchak
a51e03fce6 fix(net): FetchFullBlockRangeFuture can get stuck forever after partial body fetch + error (#21411)
Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>
Co-authored-by: Amp <amp@ampcode.com>
2026-01-26 21:30:39 +01:00
rakita
57d7c98f66 chore: merge main and update alloy-evm staging patch 2026-01-26 12:39:58 +01:00
rakita
5d9a43f2d4 Merge remote-tracking branch 'origin/main' into staging 2026-01-26 12:36:44 +01:00
rakita
defd0e8e5c Bump revm to staging and fix breaking changes
- Patch revm and all sub-crates to staging commit 0dc217a9
- Patch revm-inspectors to staging commit fccc4ac5
- Patch alloy-evm to staging commit 625ccc0f
- Add slot_num field to BlockEnv initializers
- Update BlockHashCache usage (no longer has keys method)
2026-01-26 02:41:26 +01:00
748 changed files with 25604 additions and 46021 deletions

View File

@@ -1,10 +0,0 @@
---
reth-engine-primitives: patch
reth-engine-tree: patch
reth-node-core: patch
reth-trie-parallel: minor
---
Removed legacy proof calculation system and V2-specific configuration flags.
Removed the legacy (non-V2) proof calculation code paths, simplified multiproof task architecture by removing the dual-mode system, and cleaned up V2-specific CLI flags (`--engine.disable-proof-v2`, `--engine.disable-trie-cache`) that are no longer needed. The codebase now exclusively uses V2 proofs with the sparse trie cache.

View File

@@ -1,5 +0,0 @@
---
reth-trie-sparse: patch
---
Refactored sparse trie node state tracking to use RLP nodes instead of hashes. Replaced `Option<B256>` hash fields with `SparseNodeState` enum that tracks either dirty nodes or cached RLP nodes with optional database storage flags. Added debug assertions to validate leaf path lengths and improved pruning logic to use node paths directly instead of path-hash tuples.

View File

@@ -1,5 +0,0 @@
---
reth-trie-sparse: patch
---
Fixed a bug in `merge_subtrie_updates` where source insertions did not cancel destination removals (and vice versa), causing inconsistent trie updates accumulated across multiple `root()` calls without intermediate `take_updates()`. Added a test covering the cross-cancellation behavior.

View File

@@ -1,5 +0,0 @@
---
reth-transaction-pool: minor
---
Added support for optional custom stateless and stateful validation hooks in `EthTransactionValidator` via `set_additional_stateless_validation` and `set_additional_stateful_validation` methods. Also implemented a manual `Debug` impl to handle the non-`Debug` function pointer fields.

View File

@@ -1,5 +0,0 @@
---
reth-tasks: patch
---
Added panic handler to all rayon thread pools that logs panics via `tracing::error` instead of aborting the process.

View File

@@ -1,5 +0,0 @@
---
reth-network-types: patch
---
Increased default maximum concurrent outbound dials from 15 to 30.

View File

@@ -1,5 +0,0 @@
---
reth-trie-sparse: patch
---
Added recording of `SetRoot` operation in `ParallelSparseTrie::set_root` when the `trie-debug` feature is enabled.

View File

@@ -1,6 +0,0 @@
---
reth-rpc-convert: minor
reth-storage-rpc-provider: minor
---
Replaced the separate `TryFromBlockResponse`, `TryFromReceiptResponse`, and `TryFromTransactionResponse` traits with a unified `RpcResponseConverter` trait and default `EthRpcConverter` implementation. Removed the `op-alloy-network` dependency and refactored `RpcBlockchainProvider` to store a dynamic converter instance instead of relying on per-type trait bounds.

View File

@@ -1,5 +0,0 @@
---
reth-payload-builder: minor
---
Added observability metrics for payload resolve latency and new payload job creation latency to the payload builder service.

View File

@@ -1,5 +0,0 @@
---
reth-transaction-pool: minor
---
Added `consensus_ref` method to `PoolTransaction` trait for borrowing consensus transactions without cloning.

View File

@@ -1,4 +0,0 @@
---
---
Improved nightly Docker build failure Slack notification with more detailed formatting and context.

View File

@@ -1,10 +0,0 @@
---
reth-chain-state: minor
reth-engine-primitives: minor
reth-engine-tree: minor
reth-node-core: minor
reth-node-events: minor
reth: patch
---
Added configurable slow block logging (`--engine.slow-block-threshold`) that emits a structured `warn!` log with detailed timing, state-operation counts, and cache hit-rate metrics for blocks whose total processing time exceeds the threshold. Introduced `ExecutionTimingStats`, `CacheStats`, `StateProviderStats`, and `SlowBlockInfo` types to carry execution statistics from block validation through persistence, and refactored `PersistenceResult` to carry commit duration alongside the last persisted block.

View File

@@ -1,5 +0,0 @@
---
reth-rpc-eth-types: patch
---
Updated `eth_simulateV1` revert error code from `-32000` to `3` to be consistent with `eth_call`, per [execution-apis#748](https://github.com/ethereum/execution-apis/pull/748).

View File

@@ -1,6 +0,0 @@
---
reth-trie-db: minor
reth-engine-tree: minor
---
Added `PendingChangeset` and `PendingChangesetGuard` to `ChangesetCache` so concurrent readers wait for an in-progress computation instead of falling back to the expensive DB-based path. The guard automatically cancels the pending entry on drop (e.g. task panic), ensuring waiters always make progress.

View File

@@ -1,5 +0,0 @@
---
reth-engine-tree: patch
---
Added sub-phase timing histograms to the sparse trie event loop, tracking channel wait, proof coalescing, multiproof reveal, and trie update durations separately.

View File

@@ -1,4 +0,0 @@
---
---
Improved documentation overview page with better structure and clarity.

View File

@@ -1,5 +0,0 @@
---
reth-node-events: patch
---
Updated consensus engine log message to be more accurate about received updates.

View File

@@ -1,8 +0,0 @@
---
reth-engine-primitives: minor
reth-engine-tree: minor
reth-node-core: minor
reth-trie-parallel: minor
---
Added `--engine.proof-jitter` CLI option behind the `trie-debug` feature flag. When set, each proof worker sleeps for a random duration up to the specified value before starting proof computation, useful for stress-testing timing-sensitive proof logic.

View File

@@ -1,6 +0,0 @@
---
reth-rpc-eth-api: minor
reth-rpc-server-types: minor
---
Added `eth_getStorageValues` RPC method for batch storage slot retrieval across multiple addresses.

View File

@@ -1,5 +0,0 @@
---
reth-rpc-convert: patch
---
Updated `alloy-evm` dependency to git revision `9bc2dba` and adapted `TxEnvConverter` impl to the updated `TryIntoTxEnv` trait signature that now includes a `Spec` generic parameter.

View File

@@ -1,6 +0,0 @@
---
reth-trie: patch
reth-trie-sparse: patch
---
Refactored test harness for sparse trie tests by extracting `TrieTestHarness` into a shared `reth-trie` test utility, replacing duplicated inline harness code across multiple test modules. Updated `proof_v2` return type to include an optional root hash, and converted `original_root` and `storage` from public fields to accessor methods.

View File

@@ -1,5 +0,0 @@
---
reth-trie: patch
---
Removed the local `increment_and_strip_trailing_zeros` function and `PATH_ALL_ZEROS` static in `proof_v2`, replacing them with the equivalent `Nibbles::next_without_prefix` and `Nibbles::is_zeroes` builtins. Also replaced manual `.get()` calls on `state_mask`/`hash_mask` with direct field access and switched to `Nibbles::unpack_array` over the unsafe `unpack_unchecked`.

View File

@@ -1,9 +0,0 @@
---
reth-network-api: minor
reth-network-types: minor
reth-network: minor
reth-node-core: minor
reth: minor
---
Added optional ENR fork ID enforcement to filter out peers from incompatible networks during peer discovery, controlled by the `--enforce-enr-fork-id` CLI flag.

View File

@@ -1,5 +0,0 @@
---
reth-engine-tree: patch
---
Fixed `compare_trie_updates` to return `bool` indicating whether differences were found, and updated the caller to properly use the return value instead of treating all successful comparisons as having no differences.

View File

@@ -1,7 +0,0 @@
---
reth-cli-commands: minor
reth-node-core: minor
reth: patch
---
Made v2 storage the default for all new databases, deprecating the `--storage.v2` flag to a hidden no-op kept for backwards compatibility. Updated CLI reference docs to remove the now-hidden flag from all command help pages.

View File

@@ -1,5 +0,0 @@
---
reth-node-core: minor
---
Added `with_dev_block_time` helper method to `NodeConfig` for configuring dev miner block production interval.

View File

@@ -1,5 +0,0 @@
---
reth-db-api: patch
---
Changed `StoredNibblesSubKey` encoding to use a stack-allocated `[u8; 65]` array instead of a heap-allocated `Vec<u8>`, avoiding unnecessary heap allocation.

View File

@@ -1,7 +0,0 @@
---
reth-engine-tree: patch
reth-trie-sparse: patch
reth-tasks: patch
---
Offloaded deallocation of expensive proof node buffers to a persistent background thread (`Runtime::spawn_drop`) to avoid blocking state root computation or lock-holding code.

View File

@@ -1,5 +0,0 @@
---
reth-storage-api: patch
---
Added `Arc` to `auto_impl` derive for storage-api traits to support automatic `Arc` wrapper implementations.

View File

@@ -1,8 +0,0 @@
---
reth: patch
reth-engine-tree: patch
reth-node-builder: patch
reth-trie-sparse: minor
---
Added `trie-debug` feature for recording sparse trie mutations to aid in debugging state root mismatches.

View File

@@ -1,6 +0,0 @@
---
reth-trie-sparse: patch
reth-engine-tree: patch
---
Removed the `skip_proof_node_filtering` flag, `revealed_account_paths`/`revealed_paths` tracking, and the `filter_revealed_v2_proof_nodes` function from the sparse trie implementation. Also removed the corresponding skipped-nodes metrics, simplifying the proof node reveal path to always pass nodes directly to the sparse trie without pre-filtering.

View File

@@ -1,5 +0,0 @@
---
reth-trie-sparse: minor
---
Added a comprehensive generic `SparseTrie` test suite covering `set_root`, `reveal_nodes`, `update_leaves`, `root`, `take_updates`, `commit_updates`, `prune`, `wipe`/`clear`, `get_leaf_value`, `find_leaf`, `size_hint`, and integration lifecycle scenarios. Tests are stamped out for all concrete `SparseTrie` implementations via a macro.

View File

@@ -1,5 +0,0 @@
---
reth-provider: patch
---
Fixed sender pruning during block reorg to skip when sender_recovery is fully pruned, preventing a fatal crash when no sender data exists in static files.

View File

@@ -1,7 +0,0 @@
---
reth-network-types: minor
reth-network: minor
reth-node-core: patch
---
Added `PersistedPeerInfo` struct to persist richer peer metadata (kind, fork ID, reputation) to disk. Updated `PeersConfig::with_basic_nodes_from_file` to support both the new `PersistedPeerInfo` format and the legacy `Vec<NodeRecord>` format with automatic conversion, and updated `write_peers_to_file` to exclude backed-off and banned peers.

View File

@@ -1,5 +0,0 @@
---
reth-cli-commands: minor
---
Added `reth_version` field to `SnapshotManifest` to record the Reth version that produced a snapshot. The field is optional and populated automatically during manifest generation.

View File

@@ -1,5 +0,0 @@
---
reth-network: minor
---
Added `fork_id` as a tiebreaker in peer selection when reputations are equal, preferring peers with a discovered `fork_id` as it indicates fork compatibility. Added a test to verify the tiebreaker behavior.

View File

@@ -1,5 +0,0 @@
---
reth-trie-sparse: patch
---
Fixed a bug where trie nodes could appear in both `updated_nodes` and `removed_nodes` simultaneously by removing entries from `removed_nodes` when a node is inserted as updated.

View File

@@ -1,8 +0,0 @@
---
reth-trie-common: minor
reth-trie: minor
reth-trie-parallel: minor
reth-engine-tree: patch
---
Moved `ProofV2Target`, `MultiProofTargetsV2`, and `ChunkedMultiProofTargetsV2` from `reth-trie-parallel::targets_v2` into a new `reth-trie-common::target_v2` module, making these types available at a lower level without pulling in the full parallel trie crate. Added a `multiproof_v2` method to `Proof` in `reth-trie` that generates a state multiproof using the V2 proof calculator with synchronous account value encoding.

View File

@@ -1,14 +0,0 @@
---
reth-engine-tree: patch
reth-evm-ethereum: patch
reth-evm: patch
reth-primitives-traits: patch
reth-revm: patch
reth-rpc-eth-api: patch
reth-rpc-eth-types: patch
reth-provider: patch
example-custom-evm: patch
example-precompile-cache: patch
---
Bumped revm to v35.0.0, revm-inspectors to 0.35.0, and alloy-evm to 0.29.0. Updated call sites throughout the codebase to align with the new APIs, including `ExecutionResult` field changes (`gas_used``gas.used()`/`gas.final_refunded()`), removal of `.without_state_clear()`, updated `EthPrecompiles::new(spec)` constructor, and updated `block_hashes.lowest()` access.

View File

@@ -1,9 +0,0 @@
---
reth-trie-sparse: minor
reth-engine-primitives: minor
reth-engine-tree: minor
reth-node-core: minor
reth-trie-common: patch
---
Added an arena-based sparse trie implementation (`ArenaParallelSparseTrie`) using `slotmap` arena allocation for node storage, enabling parallel subtrie mutation without per-node hashing overhead. Added `ConfigurableSparseTrie` enum to switch between the arena and hash-map implementations, and a `--engine.enable-arena-sparse-trie` CLI flag to opt in at runtime.

View File

@@ -1,6 +0,0 @@
---
reth-engine-local: minor
reth-node-builder: minor
---
Added trigger-based `MiningMode` variant that allows blocks to be built on-demand via custom streams, and exposed `with_mining_mode` method on `DebugNodeLauncherFuture` to override default mining configuration.

View File

@@ -1,5 +0,0 @@
---
reth-transaction-pool: patch
---
Fixed a bug where transactions from the same sender were added to the pending subpool out of nonce order. Ensured `process_updates` runs before `add_new_transaction` so that lower-nonce promotions are enqueued before the newly inserted higher-nonce transaction, preserving correct ordering for live `BestTransactions` iterators.

View File

@@ -1,7 +0,0 @@
---
reth-trie: major
reth-trie-db: major
reth-provider: minor
---
Added `MaskedTrieCursorFactory` and `MaskedTrieCursor` to handle prefix-set-based hash invalidation at the cursor layer, replacing the `DatabaseTrieWitness` trait abstraction. Removed `with_prefix_sets_mut` from `TrieWitness` and deleted `DatabaseTrieWitness` — callers should now wrap their cursor factory with `MaskedTrieCursorFactory` to apply prefix sets during witness/proof computation.

View File

@@ -1,5 +0,0 @@
---
reth-trie: patch
---
Fixed a potential panic in `ProofCalculator` by clearing internal computation state (`branch_stack`, `child_stack`, `branch_path`, etc.) after errors, preventing stale state from causing `usize` underflow panics when the calculator is reused. Added a test verifying correct behavior after simulated mid-computation errors.

View File

@@ -1,6 +1,6 @@
[profile.default]
retries = { backoff = "exponential", count = 2, delay = "2s", jitter = true }
slow-timeout = { period = "30s", terminate-after = 2 }
slow-timeout = { period = "30s", terminate-after = 4 }
[[profile.default.overrides]]
filter = "test(general_state_tests)"

View File

@@ -20,11 +20,6 @@
# include dist directory, where the reth binary is located after compilation
!/dist
# include PGO build helper used by Dockerfile.depot
!/.github
!/.github/scripts
!/.github/scripts/build_pgo_bolt.sh
# include licenses
!LICENSE-*

4
.github/CODEOWNERS vendored
View File

@@ -1,7 +1,7 @@
* @gakonst
crates/chain-state/ @fgimenez @mattsse
crates/chainspec/ @Rjected @joshieDo @mattsse
crates/cli/ @mattsse @Rjected
crates/cli/ @mattsse
crates/config/ @shekhirin @mattsse @Rjected
crates/consensus/ @mattsse @Rjected
crates/e2e-test-utils/ @mattsse @Rjected @klkvr @fgimenez
@@ -38,7 +38,7 @@ crates/storage/libmdbx-rs/ @shekhirin
crates/storage/nippy-jar/ @joshieDo @shekhirin
crates/storage/provider/ @joshieDo @shekhirin @yongkangc
crates/storage/storage-api/ @joshieDo
crates/tasks/ @mattsse @DaniPopes
crates/tasks/ @mattsse
crates/tokio-util/ @mattsse
crates/tracing/ @mattsse @shekhirin
crates/tracing-otlp/ @mattsse @Rjected

View File

@@ -1,276 +0,0 @@
#!/usr/bin/env python3
"""
Prometheus metrics proxy that fetches from a local reth node and
re-exposes with additional benchmark labels.
Reads labels from a JSON file (updated by local-reth-bench.sh between runs)
and injects them into every Prometheus metric line.
Returns empty 200 when reth is not running (clean Grafana gaps).
"""
import argparse
import ipaddress
import json
import subprocess
import sys
import time
from http.server import HTTPServer, BaseHTTPRequestHandler
from urllib.request import urlopen
from urllib.error import URLError
def read_labels(path):
try:
with open(path) as f:
return json.load(f)
except (FileNotFoundError, json.JSONDecodeError):
return {}
def inject_labels(metrics_bytes, label_str, label_names):
"""Inject labels into Prometheus text format.
Operates on bytes and uses simple string ops instead of regex
for speed on large payloads (reth exposes thousands of metrics).
Skips injecting into lines that already contain any of the label names
to avoid duplicate labels (which Prometheus rejects).
"""
if not label_str:
return metrics_bytes
label_bytes = label_str.encode("utf-8")
# Pre-encode label names for fast duplicate detection
label_name_bytes = [n.encode("utf-8") for n in label_names]
out = []
for line in metrics_bytes.split(b"\n"):
# Skip comments and blank lines
if line.startswith(b"#") or not line:
out.append(line)
continue
brace = line.find(b"{")
space = line.find(b" ")
if space == -1:
# Malformed, pass through
out.append(line)
elif brace != -1 and brace < space:
# Has labels: metric{existing="val"} 123
close = line.find(b"}", brace)
if close == -1:
out.append(line)
continue
# Filter out labels that already exist in this line
existing = line[brace + 1:close]
inject = label_bytes
if existing:
for name in label_name_bytes:
if name + b"=" in existing:
# Rebuild inject string excluding this label
inject = _remove_label(inject, name)
if not inject:
out.append(line)
continue
if close == brace + 1:
# Empty braces: metric{} 123
out.append(line[:close] + inject + line[close:])
else:
out.append(line[:close] + b"," + inject + line[close:])
else:
# No labels: metric 123
out.append(line[:space] + b"{" + label_bytes + b"}" + line[space:])
return b"\n".join(out)
def _remove_label(label_bytes, name):
"""Remove a single label (name=\"...\") from a comma-separated label string."""
parts = []
for part in label_bytes.split(b","):
if not part.startswith(name + b"="):
parts.append(part)
return b",".join(parts)
def build_label_str(labels):
"""Pre-format the label injection string: key1="val1",key2="val2" """
if not labels:
return ""
return ",".join(f'{k}="{v}"' for k, v in sorted(labels.items()))
def build_elapsed_gauge(labels):
"""Build a bench_elapsed_seconds gauge from run_start_epoch in labels."""
start = labels.get("run_start_epoch")
if not start:
return b""
try:
elapsed = time.time() - float(start)
except (ValueError, TypeError):
return b""
# Build labels excluding internal keys
display = {k: v for k, v in labels.items()
if k not in ("run_start_epoch", "reference_epoch")}
lstr = build_label_str(display)
return (
f"# HELP bench_elapsed_seconds Seconds since benchmark run started\n"
f"# TYPE bench_elapsed_seconds gauge\n"
f"bench_elapsed_seconds{{{lstr}}} {elapsed:.1f}\n"
).encode("utf-8")
def compute_timestamp_ms(labels):
"""Compute a synthetic timestamp so all runs share a common time origin.
Returns the timestamp in milliseconds, or None if not enough info.
Uses: reference_epoch + (now - run_start_epoch) → all runs overlay at
the same Grafana time range.
"""
ref = labels.get("reference_epoch")
start = labels.get("run_start_epoch")
if not ref or not start:
return None
try:
elapsed = time.time() - float(start)
return int((float(ref) + elapsed) * 1000)
except (ValueError, TypeError):
return None
def inject_timestamps(metrics_bytes, timestamp_ms):
"""Append a Prometheus timestamp (ms) to every data line.
Prometheus text format: metric{labels} value [timestamp_ms]
Adding timestamps causes Prometheus to store all runs' samples
at the same relative time, enabling natural overlay in Grafana.
"""
if timestamp_ms is None:
return metrics_bytes
ts = str(timestamp_ms).encode("utf-8")
out = []
for line in metrics_bytes.split(b"\n"):
if line.startswith(b"#") or not line:
out.append(line)
else:
out.append(line + b" " + ts)
return b"\n".join(out)
class MetricsHandler(BaseHTTPRequestHandler):
# Use HTTP/1.1 so Content-Length is respected and Prometheus
# doesn't have to rely on connection close to detect end of body.
protocol_version = "HTTP/1.1"
def do_GET(self):
src = self.client_address[0]
try:
resp = urlopen(self.server.upstream, timeout=2)
metrics = resp.read()
except (URLError, ConnectionError, OSError):
# reth not running — return empty 200
self._send(b"")
#print(f" scrape from {src}: empty (reth not running)", flush=True)
return
all_labels = read_labels(self.server.labels_file)
# Internal keys — not injected as Prometheus labels
internal = ("run_start_epoch", "reference_epoch")
labels = {k: v for k, v in all_labels.items() if k not in internal}
label_str = build_label_str(labels)
label_names = sorted(labels.keys())
t0 = time.monotonic()
result = inject_labels(metrics, label_str, label_names)
result += build_elapsed_gauge(all_labels)
ts_ms = compute_timestamp_ms(all_labels)
result = inject_timestamps(result, ts_ms)
dt = time.monotonic() - t0
self._send(result)
print(f" scrape from {src}: {len(metrics)} -> {len(result)} bytes, "
f"inject {dt*1000:.1f}ms", flush=True)
def _send(self, body):
self.send_response(200)
self.send_header("Content-Type", "text/plain; version=0.0.4")
self.send_header("Content-Length", str(len(body)))
self.send_header("Connection", "close")
self.end_headers()
if body:
self.wfile.write(body)
def log_message(self, format, *args):
pass # suppress per-request logging
def resolve_bind_address(subnet_cidr):
"""Find the local IP address that belongs to the given subnet.
Uses ``ip -j addr show`` to enumerate interfaces and returns the first
address that falls within *subnet_cidr* (e.g. ``10.10.0.0/24``).
"""
network = ipaddress.ip_network(subnet_cidr, strict=False)
try:
result = subprocess.run(
["ip", "-j", "addr", "show"],
capture_output=True, text=True, check=True,
)
interfaces = json.loads(result.stdout)
except (subprocess.CalledProcessError, FileNotFoundError, json.JSONDecodeError) as exc:
print(f"Error: cannot enumerate interfaces: {exc}", file=sys.stderr)
sys.exit(1)
for iface in interfaces:
for addr_info in iface.get("addr_info", []):
try:
addr = ipaddress.ip_address(addr_info["local"])
except (KeyError, ValueError):
continue
if addr in network:
return str(addr)
print(f"Error: no interface address found in subnet {subnet_cidr}", file=sys.stderr)
sys.exit(1)
def main():
parser = argparse.ArgumentParser(description="Prometheus metrics proxy with label injection")
parser.add_argument("--labels", default="/tmp/bench-metrics-labels.json",
help="Path to JSON file with labels to inject (default: /tmp/bench-metrics-labels.json)")
parser.add_argument("--upstream", default="http://127.0.0.1:9100/",
help="Upstream reth metrics URL (default: http://127.0.0.1:9100/)")
bind_group = parser.add_mutually_exclusive_group()
bind_group.add_argument("--bind", default=None,
help="Address to bind the proxy (default: 0.0.0.0)")
bind_group.add_argument("--subnet", default=None,
help="Auto-detect bind address from a local interface in this subnet (e.g. 10.10.0.0/24)")
parser.add_argument("--port", type=int, default=9090,
help="Port to bind the proxy (default: 9090)")
args = parser.parse_args()
if args.subnet:
bind_addr = resolve_bind_address(args.subnet)
elif args.bind:
bind_addr = args.bind
else:
bind_addr = "0.0.0.0"
server = HTTPServer((bind_addr, args.port), MetricsHandler)
server.upstream = args.upstream
server.labels_file = args.labels
print(f"bench-metrics-proxy listening on {bind_addr}:{args.port}")
print(f" upstream: {args.upstream}")
print(f" labels: {args.labels}")
sys.stdout.flush()
server.serve_forever()
if __name__ == "__main__":
main()

View File

@@ -1,131 +0,0 @@
#!/usr/bin/env bash
#
# Builds (or fetches from cache) reth binaries for benchmarking.
#
# Usage: bench-reth-build.sh <baseline|feature> <source-dir> <commit> [branch-sha]
#
# baseline — build/fetch the baseline binary at <commit> (merge-base)
# source-dir must be checked out at <commit>
# feature — build/fetch the candidate binary + reth-bench at <commit>
# source-dir must be checked out at <commit>
# optional branch-sha is the PR head commit for cache key
#
# Outputs:
# baseline: <source-dir>/target/profiling/reth
# feature: <source-dir>/target/profiling/reth, reth-bench installed to cargo bin
#
# Required: mc (MinIO client) with a configured alias
set -euo pipefail
MC="mc"
MODE="$1"
SOURCE_DIR="$2"
COMMIT="$3"
# Tracy support: when BENCH_TRACY is "on" or "full", add Tracy cargo features
# and frame pointers for accurate stack traces.
EXTRA_FEATURES=""
EXTRA_RUSTFLAGS=""
if [ "${BENCH_TRACY:-off}" != "off" ]; then
EXTRA_FEATURES="tracy,tracy-client/ondemand"
EXTRA_RUSTFLAGS=" -C force-frame-pointers=yes"
fi
# Cache suffix: hash of features+rustflags so different build configs get separate cache entries
if [ -n "$EXTRA_FEATURES" ] || [ -n "$EXTRA_RUSTFLAGS" ]; then
BUILD_SUFFIX="-$(echo "${EXTRA_FEATURES}${EXTRA_RUSTFLAGS}" | sha256sum | cut -c1-12)"
else
BUILD_SUFFIX=""
fi
# Verify a cached reth binary was built from the expected commit.
# `reth --version` outputs "Commit SHA: <full-sha>" on its own line.
verify_binary() {
local binary="$1" expected_commit="$2"
local version binary_sha
version=$("$binary" --version 2>/dev/null) || return 1
binary_sha=$(echo "$version" | sed -n 's/^Commit SHA: *//p')
if [ -z "$binary_sha" ]; then
echo "Warning: could not extract commit SHA from version output"
return 1
fi
if [ "$binary_sha" = "$expected_commit" ]; then
return 0
fi
echo "Cache mismatch: binary built from ${binary_sha} but expected ${expected_commit}"
return 1
}
case "$MODE" in
baseline|main)
BUCKET="minio/reth-binaries/${COMMIT}${BUILD_SUFFIX}"
mkdir -p "${SOURCE_DIR}/target/profiling"
CACHE_VALID=false
if $MC stat "${BUCKET}/reth" &>/dev/null; then
echo "Cache hit for baseline (${COMMIT}), downloading binary..."
$MC cp "${BUCKET}/reth" "${SOURCE_DIR}/target/profiling/reth"
chmod +x "${SOURCE_DIR}/target/profiling/reth"
if verify_binary "${SOURCE_DIR}/target/profiling/reth" "${COMMIT}"; then
CACHE_VALID=true
else
echo "Cached baseline binary is stale, rebuilding..."
fi
fi
if [ "$CACHE_VALID" = false ]; then
echo "Building baseline (${COMMIT}) from source..."
cd "${SOURCE_DIR}"
FEATURES_ARG=""
WORKSPACE_ARG=""
if [ -n "$EXTRA_FEATURES" ]; then
# --workspace is needed for cross-package feature syntax (tracy-client/ondemand)
FEATURES_ARG="--features ${EXTRA_FEATURES}"
WORKSPACE_ARG="--workspace"
fi
# shellcheck disable=SC2086
RUSTFLAGS="-C target-cpu=native${EXTRA_RUSTFLAGS}" \
cargo build --profile profiling --bin reth $WORKSPACE_ARG $FEATURES_ARG
$MC cp target/profiling/reth "${BUCKET}/reth"
fi
;;
feature|branch)
BRANCH_SHA="${4:-$COMMIT}"
BUCKET="minio/reth-binaries/${BRANCH_SHA}${BUILD_SUFFIX}"
CACHE_VALID=false
if $MC stat "${BUCKET}/reth" &>/dev/null && $MC stat "${BUCKET}/reth-bench" &>/dev/null; then
echo "Cache hit for ${BRANCH_SHA}, downloading binaries..."
mkdir -p "${SOURCE_DIR}/target/profiling"
$MC cp "${BUCKET}/reth" "${SOURCE_DIR}/target/profiling/reth"
$MC cp "${BUCKET}/reth-bench" /home/ubuntu/.cargo/bin/reth-bench
chmod +x "${SOURCE_DIR}/target/profiling/reth" /home/ubuntu/.cargo/bin/reth-bench
if verify_binary "${SOURCE_DIR}/target/profiling/reth" "${COMMIT}"; then
CACHE_VALID=true
else
echo "Cached feature binary is stale, rebuilding..."
fi
fi
if [ "$CACHE_VALID" = false ]; then
echo "Building feature (${COMMIT}) from source..."
cd "${SOURCE_DIR}"
rustup show active-toolchain || rustup default stable
if [ -n "$EXTRA_FEATURES" ]; then
# Can't use `make profiling` when adding features; build explicitly
# --workspace is needed for cross-package feature syntax (tracy-client/ondemand)
RUSTFLAGS="-C target-cpu=native${EXTRA_RUSTFLAGS}" \
cargo build --profile profiling --workspace --bin reth --features "${EXTRA_FEATURES}"
else
make profiling
fi
make install-reth-bench
$MC cp target/profiling/reth "${BUCKET}/reth"
$MC cp "$(which reth-bench)" "${BUCKET}/reth-bench"
fi
;;
*)
echo "Usage: $0 <baseline|feature> <source-dir> <commit> [branch-sha]"
exit 1
;;
esac

View File

@@ -1,260 +0,0 @@
#!/usr/bin/env python3
"""Generate benchmark charts from reth-bench CSV output.
Usage:
bench-engine-charts.py <combined_csv> --output-dir <dir> [--baseline <baseline_csv>]
Generates three PNG charts:
1. newPayload latency + Ggas/s per block (+ latency diff when baseline present)
2. Wait breakdown (persistence, execution cache, sparse trie) per block
3. Scatter plot of gas used vs latency
When --baseline is provided, charts overlay both datasets for comparison.
"""
import argparse
import csv
import sys
from pathlib import Path
import numpy as np
try:
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
except ImportError:
print("matplotlib is required: pip install matplotlib", file=sys.stderr)
sys.exit(1)
GIGAGAS = 1_000_000_000
def parse_combined_csv(path: str) -> list[dict]:
rows = []
with open(path) as f:
reader = csv.DictReader(f)
for row in reader:
rows.append(
{
"block_number": int(row["block_number"]),
"gas_used": int(row["gas_used"]),
"new_payload_latency_us": int(row["new_payload_latency"]),
"persistence_wait_us": int(row["persistence_wait"])
if row.get("persistence_wait")
else None,
"execution_cache_wait_us": int(row.get("execution_cache_wait", 0)),
"sparse_trie_wait_us": int(row.get("sparse_trie_wait", 0)),
}
)
return rows
def plot_latency_and_throughput(
feature: list[dict], baseline: list[dict] | None, out: Path,
baseline_name: str = "baseline", feature_name: str = "feature",
):
num_plots = 3 if baseline else 2
fig, axes = plt.subplots(num_plots, 1, figsize=(12, 4 * num_plots), sharex=True)
ax1, ax2 = axes[0], axes[1]
feat_x = [r["block_number"] for r in feature]
feat_lat = [r["new_payload_latency_us"] / 1_000 for r in feature]
feat_ggas = []
for r in feature:
lat_s = r["new_payload_latency_us"] / 1_000_000
feat_ggas.append(r["gas_used"] / lat_s / GIGAGAS if lat_s > 0 else 0)
if baseline:
base_x = [r["block_number"] for r in baseline]
base_lat = [r["new_payload_latency_us"] / 1_000 for r in baseline]
base_ggas = []
for r in baseline:
lat_s = r["new_payload_latency_us"] / 1_000_000
base_ggas.append(r["gas_used"] / lat_s / GIGAGAS if lat_s > 0 else 0)
l, = ax1.plot(base_x, base_lat, linewidth=0.8, label=baseline_name, alpha=0.7)
ax1.axhline(np.median(base_lat), color=l.get_color(), linestyle="--", linewidth=1, alpha=0.7, label=f"{baseline_name} median")
l, = ax2.plot(base_x, base_ggas, linewidth=0.8, label=baseline_name, alpha=0.7)
ax2.axhline(np.median(base_ggas), color=l.get_color(), linestyle="--", linewidth=1, alpha=0.7, label=f"{baseline_name} median")
l, = ax1.plot(feat_x, feat_lat, linewidth=0.8, label=feature_name)
ax1.axhline(np.median(feat_lat), color=l.get_color(), linestyle="--", linewidth=1, label=f"{feature_name} median")
ax1.set_ylabel("Latency (ms)")
ax1.set_title("newPayload Latency per Block")
ax1.grid(True, alpha=0.3)
ax1.legend()
l, = ax2.plot(feat_x, feat_ggas, linewidth=0.8, label=feature_name)
ax2.axhline(np.median(feat_ggas), color=l.get_color(), linestyle="--", linewidth=1, label=f"{feature_name} median")
ax2.set_ylabel("Ggas/s")
ax2.set_title("Execution Throughput per Block")
ax2.grid(True, alpha=0.3)
ax2.legend()
if baseline:
ax3 = axes[2]
base_by_block = {r["block_number"]: r["new_payload_latency_us"] for r in baseline}
blocks, diffs = [], []
for r in feature:
bn = r["block_number"]
if bn in base_by_block and base_by_block[bn] > 0:
pct = (r["new_payload_latency_us"] - base_by_block[bn]) / base_by_block[bn] * 100
blocks.append(bn)
diffs.append(pct)
if blocks:
colors = ["green" if d <= 0 else "red" for d in diffs]
ax3.bar(blocks, diffs, width=1.0, color=colors, alpha=0.7, edgecolor="none")
ax3.axhline(0, color="black", linewidth=0.5)
ax3.set_ylabel("Δ Latency (%)")
ax3.set_title("Per-Block newPayload Latency Change (feature vs baseline)")
ax3.grid(True, alpha=0.3, axis="y")
axes[-1].set_xlabel("Block Number")
fig.tight_layout()
fig.savefig(out, dpi=150)
plt.close(fig)
def plot_wait_breakdown(
feature: list[dict], baseline: list[dict] | None, out: Path,
baseline_name: str = "baseline", feature_name: str = "feature",
):
series = [
("Persistence Wait", "persistence_wait_us"),
("State Cache Wait", "execution_cache_wait_us"),
("Trie Cache Wait", "sparse_trie_wait_us"),
]
fig, axes = plt.subplots(len(series), 1, figsize=(12, 3 * len(series)), sharex=True)
for ax, (label, key) in zip(axes, series):
if baseline:
bx = [r["block_number"] for r in baseline if r[key] is not None]
by = [r[key] / 1_000 for r in baseline if r[key] is not None]
if bx:
ax.plot(bx, by, linewidth=0.8, label=baseline_name, alpha=0.7)
fx = [r["block_number"] for r in feature if r[key] is not None]
fy = [r[key] / 1_000 for r in feature if r[key] is not None]
if fx:
ax.plot(fx, fy, linewidth=0.8, label=feature_name)
ax.set_ylabel("ms")
ax.set_title(label)
ax.grid(True, alpha=0.3)
if baseline:
ax.legend()
axes[-1].set_xlabel("Block Number")
fig.suptitle("Wait Time Breakdown per Block", fontsize=14, y=1.01)
fig.tight_layout()
fig.savefig(out, dpi=150, bbox_inches="tight")
plt.close(fig)
def _add_regression(ax, x, y, color, label):
"""Add a linear regression line to the axes."""
if len(x) < 2:
return
xa, ya = np.array(x), np.array(y)
m, b = np.polyfit(xa, ya, 1)
x_range = np.linspace(xa.min(), xa.max(), 100)
ax.plot(x_range, m * x_range + b, color=color, linewidth=1.5, alpha=0.8,
label=label)
def plot_gas_vs_latency(
feature: list[dict], baseline: list[dict] | None, out: Path,
baseline_name: str = "baseline", feature_name: str = "feature",
):
fig, ax = plt.subplots(figsize=(8, 6))
if baseline:
bgas = [r["gas_used"] / 1_000_000 for r in baseline]
blat = [r["new_payload_latency_us"] / 1_000 for r in baseline]
ax.scatter(bgas, blat, s=8, alpha=0.5)
_add_regression(ax, bgas, blat, "tab:blue", baseline_name)
fgas = [r["gas_used"] / 1_000_000 for r in feature]
flat = [r["new_payload_latency_us"] / 1_000 for r in feature]
ax.scatter(fgas, flat, s=8, alpha=0.6)
_add_regression(ax, fgas, flat, "tab:orange", feature_name)
ax.set_xlabel("Gas Used (Mgas)")
ax.set_ylabel("newPayload Latency (ms)")
ax.set_title("Gas Used vs Latency")
ax.grid(True, alpha=0.3)
ax.legend()
fig.tight_layout()
fig.savefig(out, dpi=150)
plt.close(fig)
def merge_csvs(paths: list[str]) -> list[dict]:
"""Parse and merge multiple CSVs, averaging values for duplicate blocks."""
by_block: dict[int, list[dict]] = {}
for path in paths:
for row in parse_combined_csv(path):
by_block.setdefault(row["block_number"], []).append(row)
merged = []
for bn in sorted(by_block):
rows = by_block[bn]
if len(rows) == 1:
merged.append(rows[0])
else:
avg = {"block_number": bn}
for key in ("gas_used", "new_payload_latency_us"):
avg[key] = int(sum(r[key] for r in rows) / len(rows))
for key in ("persistence_wait_us", "execution_cache_wait_us", "sparse_trie_wait_us"):
vals = [r[key] for r in rows if r[key] is not None]
avg[key] = int(sum(vals) / len(vals)) if vals else None
merged.append(avg)
return merged
def main():
parser = argparse.ArgumentParser(description="Generate benchmark charts")
parser.add_argument(
"--feature", nargs="+", required=True,
help="Path(s) to feature combined_latency.csv",
)
parser.add_argument(
"--output-dir", required=True, help="Output directory for PNG charts"
)
parser.add_argument(
"--baseline", nargs="+", help="Path(s) to baseline combined_latency.csv"
)
parser.add_argument("--baseline-name", default="baseline", help="Label for baseline")
parser.add_argument("--feature-name", "--branch-name", default="feature", help="Label for feature")
args = parser.parse_args()
feature = merge_csvs(args.feature)
if not feature:
print("No results found in feature CSV(s)", file=sys.stderr)
sys.exit(1)
baseline = None
if args.baseline:
baseline = merge_csvs(args.baseline)
if not baseline:
print(
"Warning: no results in baseline CSV(s), skipping comparison",
file=sys.stderr,
)
baseline = None
out_dir = Path(args.output_dir)
out_dir.mkdir(parents=True, exist_ok=True)
bname = args.baseline_name
fname = args.feature_name
plot_latency_and_throughput(feature, baseline, out_dir / "latency_throughput.png", bname, fname)
plot_wait_breakdown(feature, baseline, out_dir / "wait_breakdown.png", bname, fname)
plot_gas_vs_latency(feature, baseline, out_dir / "gas_vs_latency.png", bname, fname)
print(f"Charts written to {out_dir}")
if __name__ == "__main__":
main()

View File

@@ -1,581 +0,0 @@
#!/usr/bin/env bash
#
# local-reth-bench.sh — Run the reth Engine API benchmark locally.
#
# Replicates the CI bench.yml workflow (build, snapshot, system tuning,
# interleaved B-F-F-B execution, summary, charts) without any GitHub
# Actions glue (no PR comments, no artifact upload, no Slack).
#
# Usage:
# local-reth-bench.sh <baseline-ref> <feature-ref> [options]
#
# Options:
# --blocks N Number of blocks to benchmark (default: 500)
# --warmup N Number of warmup blocks (default: 100)
# --cores N Limit reth to N CPU cores, 0 = all available (default: 0)
# --samply Enable samply profiling
# --tracy MODE Tracy profiling: off, on, full (default: off)
# --tracy-filter F Tracy tracing filter (default: debug)
# --no-tune Skip system tuning (useful on dev machines / macOS)
#
# Requires: the reth repo at RETH_REPO (default: ~/reth)
#
# Dependencies (install before first run):
# mc (MinIO client), schelk, cpupower, taskset, stdbuf, python3, curl,
# make, uv, pzstd, jq, Rust toolchain (cargo/rustup)
#
# The script delegates to the existing bench-reth-*.sh scripts in the reth
# repo for the actual build, snapshot, and run steps.
set -euo pipefail
# ── PATH ──────────────────────────────────────────────────────────────
# Ensure cargo and user-local bins (mc, uv) are visible
export PATH="$HOME/.local/bin:$HOME/.cargo/bin:$PATH"
# ── Defaults ──────────────────────────────────────────────────────────
RETH_REPO="${RETH_REPO:-$HOME/reth}"
BLOCKS=500
WARMUP=100
CORES=0
SAMPLY=false
TRACY="off"
TRACY_FILTER="debug"
TUNE=true
BASELINE_REF=""
FEATURE_REF=""
# ── Parse arguments ──────────────────────────────────────────────────
usage() {
cat <<EOF
Usage: $(basename "$0") <baseline-ref> <feature-ref> [options]
Options:
--blocks N Number of blocks to benchmark (default: 500)
--warmup N Number of warmup blocks (default: 100)
--cores N Limit reth to N CPU cores (default: 0 = all)
--samply Enable samply profiling
--tracy MODE Tracy profiling: off, on, full (default: off)
on = tracing only (lower overhead)
full = tracing + CPU sampling (higher overhead)
--tracy-filter F Tracy tracing filter (default: debug)
--no-tune Skip system tuning
EOF
exit 1
}
while [[ $# -gt 0 ]]; do
case "$1" in
--blocks) BLOCKS="$2"; shift 2 ;;
--warmup) WARMUP="$2"; shift 2 ;;
--cores) CORES="$2"; shift 2 ;;
--samply) SAMPLY=true; shift ;;
--tracy) TRACY="$2"; shift 2 ;;
--tracy-filter) TRACY_FILTER="$2"; shift 2 ;;
--no-tune) TUNE=false; shift ;;
--help|-h) usage ;;
-*) echo "Unknown option: $1"; usage ;;
*)
if [ -z "$BASELINE_REF" ]; then
BASELINE_REF="$1"
elif [ -z "$FEATURE_REF" ]; then
FEATURE_REF="$1"
else
echo "Unexpected argument: $1"; usage
fi
shift
;;
esac
done
if [ -z "$BASELINE_REF" ] || [ -z "$FEATURE_REF" ]; then
echo "Error: both <baseline-ref> and <feature-ref> are required."
usage
fi
# Validate --tracy value
case "$TRACY" in
off|on|full) ;;
*) echo "Error: --tracy must be off, on, or full (got: $TRACY)"; usage ;;
esac
# Samply + tracy=full are mutually exclusive (both use perf sampling)
if [ "$SAMPLY" = "true" ] && [ "$TRACY" = "full" ]; then
echo "Warning: samply and tracy=full both use perf sampling; downgrading tracy to 'on'."
TRACY="on"
fi
# ── Check dependencies ───────────────────────────────────────────────
missing=()
for cmd in mc schelk cpupower taskset stdbuf python3 curl make uv pzstd jq cargo; do
command -v "$cmd" &>/dev/null || missing+=("$cmd")
done
if [ ${#missing[@]} -gt 0 ]; then
echo "Error: missing required tools: ${missing[*]}"
echo "See the CI 'Install dependencies' step in .github/workflows/bench.yml for install instructions."
exit 1
fi
if [ "$TRACY" != "off" ]; then
if ! command -v tracy-capture &>/dev/null; then
echo "Error: tracy-capture is required for --tracy $TRACY"
exit 1
fi
fi
# Ensure tools that run via sudo are in a sudo-visible path.
# The bench scripts use `sudo schelk` / `sudo samply` but cargo installs
# them to ~/.cargo/bin which sudo's secure_path doesn't include.
for cmd in schelk samply; do
if command -v "$cmd" &>/dev/null && ! sudo sh -c "command -v $cmd" &>/dev/null; then
echo "Installing $cmd to /usr/local/bin (needed for sudo)..."
sudo install "$(command -v "$cmd")" /usr/local/bin/
fi
done
if [ ! -d "$RETH_REPO/.git" ]; then
echo "Error: RETH_REPO=$RETH_REPO is not a git repository."
echo "Set RETH_REPO or clone reth to ~/reth"
exit 1
fi
# ── Resolve paths ────────────────────────────────────────────────────
SELF_DIR="$(cd "$(dirname "$0")" && pwd)"
SCRIPTS_DIR="${RETH_REPO}/.github/scripts"
BENCH_WORK_DIR="${RETH_REPO}/../bench-work-$(date +%Y%m%d-%H%M%S)"
BASELINE_SRC="${RETH_REPO}/../reth-baseline"
FEATURE_SRC="${RETH_REPO}/../reth-feature"
mkdir -p "$BENCH_WORK_DIR"
BENCH_WORK_DIR="$(cd "$BENCH_WORK_DIR" && pwd)"
# ── Global cleanup trap (restores system tuning on any exit) ─────────
TUNING_APPLIED=false
CSTATE_PID=
METRICS_PROXY_PID=
cleanup_global() {
[ -n "$METRICS_PROXY_PID" ] && kill "$METRICS_PROXY_PID" 2>/dev/null || true
if [ "$TUNING_APPLIED" = true ]; then
echo
echo "▸ Restoring system settings..."
[ -n "$CSTATE_PID" ] && kill "$CSTATE_PID" 2>/dev/null || true
sudo systemctl start irqbalance cron atd 2>/dev/null || true
echo " System settings restored."
fi
}
trap cleanup_global EXIT
echo "═══════════════════════════════════════════════════════════"
echo " reth local benchmark"
echo "═══════════════════════════════════════════════════════════"
echo " Baseline ref : $BASELINE_REF"
echo " Feature ref : $FEATURE_REF"
echo " Blocks : $BLOCKS"
echo " Warmup : $WARMUP"
echo " Cores : $CORES"
echo " Samply : $SAMPLY"
echo " Tracy : $TRACY"
echo " Tracy filter : $TRACY_FILTER"
echo " System tune : $TUNE"
echo " Work dir : $BENCH_WORK_DIR"
echo " Reth repo : $RETH_REPO"
echo "═══════════════════════════════════════════════════════════"
echo
# Enable sccache if available (matches CI's RUSTC_WRAPPER=sccache)
if command -v sccache &>/dev/null; then
export RUSTC_WRAPPER="sccache"
fi
# Export env vars expected by the bench-reth-*.sh scripts
export BENCH_BLOCKS="$BLOCKS"
export BENCH_WARMUP_BLOCKS="$WARMUP"
export BENCH_CORES="$CORES"
export BENCH_SAMPLY="$SAMPLY"
export BENCH_TRACY="$TRACY"
export BENCH_TRACY_FILTER="$TRACY_FILTER"
export BENCH_WORK_DIR
export SCHELK_MOUNT="${SCHELK_MOUNT:-/reth-bench}"
export BENCH_RPC_URL="${BENCH_RPC_URL:-https://ethereum.reth.rs/rpc}"
export BENCH_METRICS_ADDR="127.0.0.1:9100"
# ── Step 1: Resolve refs to full SHAs ────────────────────────────────
echo "▸ Resolving git refs..."
cd "$RETH_REPO"
resolve_ref() {
local ref="$1"
git fetch origin "$ref" --quiet 2>/dev/null || true
git rev-parse "$ref" 2>/dev/null \
|| git rev-parse "origin/$ref" 2>/dev/null \
|| { echo "Error: cannot resolve ref '$ref'"; exit 1; }
}
BASELINE_SHA="$(resolve_ref "$BASELINE_REF")"
FEATURE_SHA="$(resolve_ref "$FEATURE_REF")"
echo " Baseline SHA : $BASELINE_SHA"
echo " Feature SHA : $FEATURE_SHA"
echo
# ── Step 2: Prepare source directories ───────────────────────────────
echo "▸ Preparing source directories..."
prepare_source() {
local src_dir="$1" ref="$2"
if [ -d "$src_dir" ]; then
git -C "$src_dir" fetch origin "$ref" 2>/dev/null || true
else
git clone --recurse-submodules "$RETH_REPO" "$src_dir"
fi
git -C "$src_dir" checkout "$ref" --force
git -C "$src_dir" submodule update --init --recursive
}
prepare_source "$BASELINE_SRC" "$BASELINE_SHA"
prepare_source "$FEATURE_SRC" "$FEATURE_SHA"
BASELINE_SRC="$(cd "$BASELINE_SRC" && pwd)"
FEATURE_SRC="$(cd "$FEATURE_SRC" && pwd)"
echo " Baseline src : $BASELINE_SRC"
echo " Feature src : $FEATURE_SRC"
echo
# ── Step 3: Check / download snapshot ────────────────────────────────
echo "▸ Checking snapshot..."
cd "$RETH_REPO"
SNAPSHOT_NEEDED=false
if ! "${SCRIPTS_DIR}/bench-reth-snapshot.sh" --check; then
SNAPSHOT_NEEDED=true
echo " Snapshot needs update."
else
echo " Snapshot is up-to-date."
fi
echo
# ── Step 4: Build binaries (+ snapshot download) in parallel ─────────
echo "▸ Building binaries (parallel)..."
cd "$RETH_REPO"
FAIL=0
"${SCRIPTS_DIR}/bench-reth-build.sh" baseline "$BASELINE_SRC" "$BASELINE_SHA" &
PID_BASELINE=$!
"${SCRIPTS_DIR}/bench-reth-build.sh" feature "$FEATURE_SRC" "$FEATURE_SHA" &
PID_FEATURE=$!
PID_SNAPSHOT=
if [ "$SNAPSHOT_NEEDED" = "true" ]; then
echo " Also downloading snapshot in parallel..."
"${SCRIPTS_DIR}/bench-reth-snapshot.sh" &
PID_SNAPSHOT=$!
fi
wait $PID_BASELINE || FAIL=1
wait $PID_FEATURE || FAIL=1
[ -n "$PID_SNAPSHOT" ] && { wait $PID_SNAPSHOT || FAIL=1; }
if [ $FAIL -ne 0 ]; then
echo "Error: one or more parallel tasks failed (builds / snapshot)"
exit 1
fi
echo " Binaries built successfully."
echo
# ── Step 5: System tuning (optional) ────────────────────────────────
if [ "$TUNE" = "true" ]; then
echo "▸ Applying system tuning..."
sudo cpupower frequency-set -g performance 2>/dev/null || true
# Disable turbo boost (Intel + AMD)
echo 1 | sudo tee /sys/devices/system/cpu/intel_pstate/no_turbo 2>/dev/null || true
echo 0 | sudo tee /sys/devices/system/cpu/cpufreq/boost 2>/dev/null || true
sudo swapoff -a 2>/dev/null || true
echo 0 | sudo tee /proc/sys/kernel/randomize_va_space 2>/dev/null || true
# Disable SMT (hyperthreading)
for cpu in /sys/devices/system/cpu/cpu*/topology/thread_siblings_list; do
[ -f "$cpu" ] || continue
first=$(cut -d, -f1 < "$cpu" | cut -d- -f1)
current=$(echo "$cpu" | grep -o 'cpu[0-9]*' | grep -o '[0-9]*')
if [ "$current" != "$first" ]; then
echo 0 | sudo tee "/sys/devices/system/cpu/cpu${current}/online" 2>/dev/null || true
fi
done
echo " Online CPUs: $(nproc)"
# Disable transparent huge pages
for p in /sys/kernel/mm/transparent_hugepage /sys/kernel/mm/transparent_hugepages; do
if [ -d "$p" ]; then
echo never | sudo tee "$p/enabled" 2>/dev/null || true
echo never | sudo tee "$p/defrag" 2>/dev/null || true
break
fi
done
# Prevent deep C-states
sudo sh -c 'exec 3<>/dev/cpu_dma_latency; echo -ne "\x00\x00\x00\x00" >&3; sleep infinity' &
CSTATE_PID=$!
# Pin IRQs to core 0
for irq in /proc/irq/*/smp_affinity_list; do
echo 0 | sudo tee "$irq" 2>/dev/null || true
done
# Stop noisy background services
sudo systemctl stop irqbalance cron atd unattended-upgrades snapd 2>/dev/null || true
TUNING_APPLIED=true
# Log environment for reproducibility (matches CI)
echo " === Benchmark environment ==="
echo " Kernel : $(uname -r)"
lscpu | grep -E 'Model name|CPU\(s\)|MHz|NUMA' | sed 's/^/ /'
echo " Governor : $(cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor 2>/dev/null || echo unknown)"
echo " Freq : $(cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq 2>/dev/null || echo unknown)"
echo " THP : $(cat /sys/kernel/mm/transparent_hugepage/enabled 2>/dev/null || cat /sys/kernel/mm/transparent_hugepages/enabled 2>/dev/null || echo unknown)"
free -h | sed 's/^/ /'
echo " System tuning applied."
echo
fi
# ── Step 5b: Tracefs mount (tracy=full only) ─────────────────────────
if [ "$TRACY" = "full" ] && [ "$(uname)" = "Linux" ]; then
echo "▸ Mounting tracefs for Tracy full mode..."
sudo mount -t tracefs tracefs /sys/kernel/tracing -o mode=755 2>/dev/null || true
fi
# ── Tracy upload & viewer helpers ────────────────────────────────────
TRACY_VIEWER_BASE="${TRACY_VIEWER_BASE:-}"
tracy_viewer_url() {
local profile_url="$1"
if [ -z "$TRACY_VIEWER_BASE" ]; then
echo ""
return
fi
local encoded
encoded=$(python3 -c "import urllib.parse, sys; print(urllib.parse.quote(sys.argv[1], safe=''))" "$profile_url")
echo "${TRACY_VIEWER_BASE}?profile_url=${encoded}"
}
upload_tracy() {
local label="$1" output_dir="$2" sha="$3"
local tracy_file="$output_dir/tracy-profile.tracy"
if [ ! -f "$tracy_file" ]; then
echo " Tracy: no profile found, skipping upload."
return
fi
local timestamp short_sha remote_name bucket mc_alias
timestamp=$(date +%Y%m%d-%H%M%S)
short_sha="${sha:0:7}"
remote_name="${label}-${short_sha}-${timestamp}.tracy"
bucket="${TRACY_BUCKET:-tracy-profiles}"
mc_alias="${MC_ALIAS:-minio}"
local minio_base="${TRACY_MINIO_URL:-http://minio.minio.svc.cluster.local:9000}"
echo " Tracy: uploading profile..."
if mc cp "$tracy_file" "${mc_alias}/${bucket}/${remote_name}"; then
local url="${minio_base}/${bucket}/${remote_name}"
echo "$url" > "$output_dir/tracy_url.txt"
local viewer
viewer=$(tracy_viewer_url "$url")
if [ -n "$viewer" ]; then
echo "$viewer" > "$output_dir/tracy_viewer_url.txt"
echo " Tracy: uploaded → $viewer"
else
echo " Tracy: uploaded → $url"
fi
else
echo " Tracy: upload failed (non-fatal)."
fi
# Delete large profile to free disk
rm -f "$tracy_file"
}
# ── Step 6: Pre-flight cleanup ───────────────────────────────────────
echo "▸ Pre-flight cleanup..."
pkill -f bench-metrics-proxy 2>/dev/null || true
sudo pkill -9 reth 2>/dev/null || true
sleep 1
if mountpoint -q "$SCHELK_MOUNT" 2>/dev/null; then
sudo umount -l "$SCHELK_MOUNT" 2>/dev/null || true
sudo schelk recover -y 2>/dev/null || true
fi
echo
# ── Step 7: Interleaved benchmark runs (B-F-F-B) ────────────────────
# This ordering reduces systematic bias from thermal drift and cache warming.
BASELINE_BIN="${BASELINE_SRC}/target/profiling/reth"
FEATURE_BIN="${FEATURE_SRC}/target/profiling/reth"
# Start metrics proxy (reth → label injection → Prometheus)
LABELS_FILE="/tmp/bench-metrics-labels.json"
echo '{}' > "$LABELS_FILE"
METRICS_SUBNET="${METRICS_SUBNET:-10.10.0.0/24}"
METRICS_PORT="${METRICS_PORT:-9090}"
python3 "${SELF_DIR}/bench-metrics-proxy.py" \
--labels "$LABELS_FILE" \
--upstream "http://${BENCH_METRICS_ADDR}/" \
--subnet "$METRICS_SUBNET" \
--port "$METRICS_PORT" &
METRICS_PROXY_PID=$!
echo "▸ Metrics proxy started (PID $METRICS_PROXY_PID) on subnet ${METRICS_SUBNET}, port ${METRICS_PORT}"
# Unique benchmark ID: local-<timestamp> for local runs, ci-<run_id> for CI
BENCH_ID="local-$(basename "$BENCH_WORK_DIR" | sed 's/bench-work-//')"
# Reference epoch: shared time origin so all runs overlay in Grafana.
# The proxy maps each run's elapsed time onto this common origin.
BENCH_REFERENCE_EPOCH=$(date +%s)
write_labels() {
local run_label="$1" run_type="$2" ref="$3" sha="$4"
LAST_RUN_START=$(date +%s)
cat > "$LABELS_FILE" <<-EOF
{"benchmark_run":"${run_label}","run_type":"${run_type}","git_ref":"${ref}","bench_sha":"${sha}","benchmark_id":"${BENCH_ID}","run_start_epoch":"${LAST_RUN_START}","reference_epoch":"${BENCH_REFERENCE_EPOCH}"}
EOF
}
run_bench() {
local label="$1" binary="$2" output_dir="$3"
echo "▸ Running benchmark: ${label}..."
cd "$RETH_REPO"
if command -v taskset &>/dev/null; then
taskset -c 0 "${SCRIPTS_DIR}/bench-reth-run.sh" "$label" "$binary" "$output_dir"
else
"${SCRIPTS_DIR}/bench-reth-run.sh" "$label" "$binary" "$output_dir"
fi
echo "${label} complete."
echo
}
write_labels "baseline-1" "baseline" "$BASELINE_REF" "$BASELINE_SHA"
run_bench "baseline-1" "$BASELINE_BIN" "$BENCH_WORK_DIR/baseline-1"
write_labels "feature-1" "feature" "$FEATURE_REF" "$FEATURE_SHA"
run_bench "feature-1" "$FEATURE_BIN" "$BENCH_WORK_DIR/feature-1"
write_labels "feature-2" "feature" "$FEATURE_REF" "$FEATURE_SHA"
run_bench "feature-2" "$FEATURE_BIN" "$BENCH_WORK_DIR/feature-2"
write_labels "baseline-2" "baseline" "$BASELINE_REF" "$BASELINE_SHA"
run_bench "baseline-2" "$BASELINE_BIN" "$BENCH_WORK_DIR/baseline-2"
# ── Compute Grafana URL ──────────────────────────────────────────────
GRAFANA_BASE_URL="https://tempoxyz.grafana.net/d/reth-bench-ghr/reth-bench-ghr"
GRAFANA_DATASOURCE="ef57fux92e9z4e"
LAST_RUN_DURATION=$(( $(date +%s) - LAST_RUN_START ))
FROM_MS=$(( BENCH_REFERENCE_EPOCH * 1000 ))
TO_MS=$(( (BENCH_REFERENCE_EPOCH + LAST_RUN_DURATION) * 1000 ))
GRAFANA_URL="${GRAFANA_BASE_URL}?orgId=1&from=${FROM_MS}&to=${TO_MS}&timezone=browser&var-datasource=${GRAFANA_DATASOURCE}&var-job=reth-bench&var-benchmark_id=${BENCH_ID}&var-benchmark_run=\$__all"
# ── Step 8: Scan logs for errors ─────────────────────────────────────
echo "▸ Scanning logs for errors..."
ERRORS_FILE="$BENCH_WORK_DIR/errors.md"
found_errors=false
for run_dir in baseline-1 feature-1 feature-2 baseline-2; do
LOG="$BENCH_WORK_DIR/$run_dir/node.log"
[ -f "$LOG" ] || continue
panics=$(grep -c -E 'panicked at' "$LOG" 2>/dev/null || true)
errors=$(grep -c ' ERROR ' "$LOG" 2>/dev/null || true)
if [ "$panics" -gt 0 ] || [ "$errors" -gt 0 ]; then
if [ "$found_errors" = false ]; then
printf '### ⚠️ Node Errors\n\n' >> "$ERRORS_FILE"
found_errors=true
fi
printf '<details><summary><b>%s</b>: %d panic(s), %d error(s)</summary>\n\n' \
"$run_dir" "$panics" "$errors" >> "$ERRORS_FILE"
if [ "$panics" -gt 0 ]; then
printf '**Panics:**\n```\n' >> "$ERRORS_FILE"
grep -E 'panicked at' "$LOG" | head -10 >> "$ERRORS_FILE"
printf '```\n' >> "$ERRORS_FILE"
fi
if [ "$errors" -gt 0 ]; then
printf '**Errors (first 20):**\n```\n' >> "$ERRORS_FILE"
grep ' ERROR ' "$LOG" | head -20 >> "$ERRORS_FILE"
printf '```\n' >> "$ERRORS_FILE"
fi
printf '\n</details>\n\n' >> "$ERRORS_FILE"
fi
done
if [ "$found_errors" = true ]; then
echo " ⚠ Errors found — see $ERRORS_FILE"
else
echo " No errors found."
fi
echo
# ── Step 9: Parse results ───────────────────────────────────────────
echo "▸ Parsing results..."
cd "$RETH_REPO"
SUMMARY_ARGS=(
--output-summary "$BENCH_WORK_DIR/summary.json"
--output-markdown "$BENCH_WORK_DIR/comment.md"
--repo "paradigmxyz/reth"
--baseline-ref "$BASELINE_SHA"
--baseline-name "$BASELINE_REF"
--feature-name "$FEATURE_REF"
--feature-ref "$FEATURE_SHA"
--baseline-csv "$BENCH_WORK_DIR/baseline-1/combined_latency.csv" "$BENCH_WORK_DIR/baseline-2/combined_latency.csv"
--feature-csv "$BENCH_WORK_DIR/feature-1/combined_latency.csv" "$BENCH_WORK_DIR/feature-2/combined_latency.csv"
--gas-csv "$BENCH_WORK_DIR/feature-1/total_gas.csv"
--grafana-url "$GRAFANA_URL"
)
python3 "${SCRIPTS_DIR}/bench-reth-summary.py" "${SUMMARY_ARGS[@]}"
echo
# ── Step 10: Generate charts ─────────────────────────────────────────
echo "▸ Generating charts..."
CHART_ARGS=(
--output-dir "$BENCH_WORK_DIR/charts"
--feature "$BENCH_WORK_DIR/feature-1/combined_latency.csv" "$BENCH_WORK_DIR/feature-2/combined_latency.csv"
--baseline "$BENCH_WORK_DIR/baseline-1/combined_latency.csv" "$BENCH_WORK_DIR/baseline-2/combined_latency.csv"
--baseline-name "$BASELINE_REF"
--feature-name "$FEATURE_REF"
)
if python3 -c "import matplotlib" 2>/dev/null; then
python3 "${SCRIPTS_DIR}/bench-reth-charts.py" "${CHART_ARGS[@]}"
elif command -v uv &>/dev/null; then
uv run --with matplotlib python3 "${SCRIPTS_DIR}/bench-reth-charts.py" "${CHART_ARGS[@]}"
else
echo " Warning: matplotlib not available, skipping chart generation."
fi
echo
# ── Step 11: Upload Tracy profiles ────────────────────────────────────
if [ "$TRACY" != "off" ]; then
echo "▸ Uploading Tracy profiles..."
upload_tracy "baseline-1" "$BENCH_WORK_DIR/baseline-1" "$BASELINE_SHA"
upload_tracy "feature-1" "$BENCH_WORK_DIR/feature-1" "$FEATURE_SHA"
upload_tracy "feature-2" "$BENCH_WORK_DIR/feature-2" "$FEATURE_SHA"
upload_tracy "baseline-2" "$BENCH_WORK_DIR/baseline-2" "$BASELINE_SHA"
echo
fi
# ── Done (system restore happens via EXIT trap) ─────────────────────
echo "═══════════════════════════════════════════════════════════"
echo " Benchmark complete!"
echo "═══════════════════════════════════════════════════════════"
echo " Results : $BENCH_WORK_DIR/summary.json"
echo " Markdown : $BENCH_WORK_DIR/comment.md"
echo " Charts : $BENCH_WORK_DIR/charts/"
if [ -f "$ERRORS_FILE" ]; then
echo " Errors : $ERRORS_FILE"
fi
echo " Grafana : $GRAFANA_URL"
if [ "$TRACY" != "off" ]; then
echo " ─── Tracy Profiles ───"
for run_dir in baseline-1 feature-1 feature-2 baseline-2; do
url_file="$BENCH_WORK_DIR/$run_dir/tracy_viewer_url.txt"
if [ -f "$url_file" ]; then
echo " $run_dir : $(cat "$url_file")"
fi
done
fi
echo "═══════════════════════════════════════════════════════════"

View File

@@ -1,250 +0,0 @@
#!/usr/bin/env bash
#
# Runs a single reth-bench cycle: mount snapshot → start node → warmup →
# benchmark → stop node → recover snapshot.
#
# Usage: bench-reth-run.sh <label> <binary> <output-dir>
#
# Required env: SCHELK_MOUNT, BENCH_RPC_URL, BENCH_BLOCKS, BENCH_WARMUP_BLOCKS
# Optional env: BENCH_BIG_BLOCKS (true/false), BENCH_WORK_DIR (for big blocks path)
# BENCH_RETH_NEW_PAYLOAD (true/false, default true)
# BENCH_WAIT_TIME (duration like 500ms, default empty)
# BENCH_BASELINE_ARGS (extra reth node args for baseline runs)
# BENCH_FEATURE_ARGS (extra reth node args for feature runs)
set -euo pipefail
LABEL="$1"
BINARY="$2"
OUTPUT_DIR="$3"
DATADIR="$SCHELK_MOUNT/datadir"
mkdir -p "$OUTPUT_DIR"
LOG="${OUTPUT_DIR}/node.log"
cleanup() {
kill "$TAIL_PID" 2>/dev/null || true
# Stop tracy-capture first (SIGINT makes it disconnect and flush to disk)
# Must happen before killing reth, otherwise reth keeps streaming data.
if [ -n "${TRACY_PID:-}" ] && kill -0 "$TRACY_PID" 2>/dev/null; then
echo "Stopping tracy-capture..."
kill -INT "$TRACY_PID" 2>/dev/null || true
for i in $(seq 1 30); do
kill -0 "$TRACY_PID" 2>/dev/null || break
if [ $((i % 10)) -eq 0 ]; then
echo "Waiting for tracy-capture to finish writing... (${i}s)"
fi
sleep 1
done
if kill -0 "$TRACY_PID" 2>/dev/null; then
echo "tracy-capture still running after 30s, killing..."
kill -9 "$TRACY_PID" 2>/dev/null || true
fi
wait "$TRACY_PID" 2>/dev/null || true
fi
if [ -n "${RETH_PID:-}" ] && sudo kill -0 "$RETH_PID" 2>/dev/null; then
if [ "${BENCH_SAMPLY:-false}" = "true" ]; then
# Send SIGINT to the inner reth process by exact name (not -f which
# would also match samply's cmdline containing "reth"). Samply will
# capture reth's exit and save the profile.
sudo pkill -INT -x reth 2>/dev/null || true
# Wait for samply to finish writing the profile and exit
for i in $(seq 1 120); do
sudo pgrep -x samply > /dev/null 2>&1 || break
if [ $((i % 10)) -eq 0 ]; then
echo "Waiting for samply to finish writing profile... (${i}s)"
fi
sleep 1
done
if sudo pgrep -x samply > /dev/null 2>&1; then
echo "Samply still running after 120s, sending SIGTERM..."
sudo pkill -x samply 2>/dev/null || true
fi
else
sudo kill "$RETH_PID"
for i in $(seq 1 30); do
sudo kill -0 "$RETH_PID" 2>/dev/null || break
sleep 1
done
fi
sudo kill -9 "$RETH_PID" 2>/dev/null || true
sleep 1
fi
# Fix ownership of reth-created files (reth runs as root)
sudo chown -R "$(id -un):$(id -gn)" "$OUTPUT_DIR" 2>/dev/null || true
if mountpoint -q "$SCHELK_MOUNT"; then
sudo umount -l "$SCHELK_MOUNT" || true
sudo schelk recover -y || true
fi
}
TAIL_PID=
TRACY_PID=
trap cleanup EXIT
# Clean up stale schelk state from a previous cancelled run.
# If schelk thinks it's still mounted (e.g. a cancelled run skipped cleanup),
# recover first to reset state.
sudo schelk recover -y -k || true
# Mount
sudo schelk mount -y
sync
sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'
echo "=== Cache state after drop ==="
free -h
grep Cached /proc/meminfo
# Start reth
# CPU layout: core 0 = OS/IRQs/reth-bench/aux, cores 1+ = reth node
RETH_BENCH="$(which reth-bench)"
ONLINE=$(nproc --all)
MAX_RETH=$(( ONLINE - 1 ))
if [ "${BENCH_CORES:-0}" -gt 0 ] && [ "$BENCH_CORES" -lt "$MAX_RETH" ]; then
MAX_RETH=$BENCH_CORES
fi
RETH_CPUS="1-${MAX_RETH}"
BIG_BLOCKS="${BENCH_BIG_BLOCKS:-false}"
RETH_ARGS=(
node
--datadir "$DATADIR"
--log.file.directory "$OUTPUT_DIR/reth-logs"
--engine.accept-execution-requests-hash
--http
--http.port 8545
--ws
--ws.api all
--authrpc.port 8551
--disable-discovery
--no-persist-peers
)
# Big blocks mode requires the testing API and skip-invalid-transactions
if [ "$BIG_BLOCKS" = "true" ]; then
RETH_ARGS+=(--http.api eth,net,web3,reth,testing --testing.skip-invalid-transactions)
fi
# Append per-label extra node args (baseline or feature)
EXTRA_NODE_ARGS=""
case "$LABEL" in
baseline*) EXTRA_NODE_ARGS="${BENCH_BASELINE_ARGS:-}" ;;
feature*) EXTRA_NODE_ARGS="${BENCH_FEATURE_ARGS:-}" ;;
esac
if [ -n "$EXTRA_NODE_ARGS" ]; then
# Word-split the string into individual args
# shellcheck disable=SC2206
RETH_ARGS+=($EXTRA_NODE_ARGS)
fi
if [ -n "${BENCH_METRICS_ADDR:-}" ]; then
RETH_ARGS+=(--metrics "$BENCH_METRICS_ADDR")
fi
# Tracy profiling: add --log.tracy flags and set environment
if [ "${BENCH_TRACY:-off}" != "off" ]; then
RETH_ARGS+=(--log.tracy --log.tracy.filter "${BENCH_TRACY_FILTER:-debug}")
if [ "${BENCH_TRACY}" = "on" ]; then
export TRACY_NO_SYS_TRACE=1
elif [ "${BENCH_TRACY}" = "full" ]; then
export TRACY_SAMPLING_HZ="${BENCH_TRACY_SAMPLING_HZ:-1}"
fi
fi
if [ "${BENCH_SAMPLY:-false}" = "true" ]; then
RETH_ARGS+=(--log.samply)
SAMPLY="$(which samply)"
sudo taskset -c "$RETH_CPUS" nice -n -20 \
"$SAMPLY" record --save-only --presymbolicate --rate 10000 \
--output "$OUTPUT_DIR/samply-profile.json.gz" \
-- "$BINARY" "${RETH_ARGS[@]}" \
> "$LOG" 2>&1 &
else
sudo taskset -c "$RETH_CPUS" nice -n -20 "$BINARY" "${RETH_ARGS[@]}" \
> "$LOG" 2>&1 &
fi
RETH_PID=$!
stdbuf -oL tail -f "$LOG" | sed -u "s/^/[reth] /" &
TAIL_PID=$!
for i in $(seq 1 60); do
if curl -sf http://127.0.0.1:8545 -X POST \
-H 'Content-Type: application/json' \
-d '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}' \
> /dev/null 2>&1; then
echo "reth (${LABEL}) is ready after ${i}s"
break
fi
if [ "$i" -eq 60 ]; then
echo "::error::reth (${LABEL}) failed to start within 60s"
cat "$LOG"
exit 1
fi
sleep 1
done
# Run reth-bench with high priority but as the current user so output
# files are not root-owned (avoids EACCES on next checkout).
BENCH_NICE="sudo nice -n -20 sudo -u $(id -un)"
# Build optional flags
EXTRA_BENCH_ARGS=()
if [ "${BENCH_RETH_NEW_PAYLOAD:-true}" != "false" ]; then
EXTRA_BENCH_ARGS+=(--reth-new-payload)
fi
if [ -n "${BENCH_WAIT_TIME:-}" ]; then
EXTRA_BENCH_ARGS+=(--wait-time "$BENCH_WAIT_TIME")
fi
if [ "$BIG_BLOCKS" = "true" ]; then
# Big blocks mode: replay pre-generated payloads with gas ramp
BIG_BLOCKS_DIR="${BENCH_WORK_DIR}/big-blocks"
# Count gas ramp blocks for reporting
GAS_RAMP_COUNT=$(find "$BIG_BLOCKS_DIR/gas-ramp-dir" -name '*.json' | wc -l)
echo "$GAS_RAMP_COUNT" > "$OUTPUT_DIR/gas_ramp_blocks.txt"
echo "Gas ramp blocks: $GAS_RAMP_COUNT"
# Start tracy-capture so profile only covers the benchmark
if [ "${BENCH_TRACY:-off}" != "off" ]; then
echo "Starting tracy-capture..."
tracy-capture -f -o "$OUTPUT_DIR/tracy-profile.tracy" &
TRACY_PID=$!
sleep 0.5 # give tracy-capture time to connect
fi
echo "Running big blocks benchmark (replay-payloads)..."
$BENCH_NICE "$RETH_BENCH" replay-payloads \
"${EXTRA_BENCH_ARGS[@]}" \
--gas-ramp-dir "$BIG_BLOCKS_DIR/gas-ramp-dir" \
--payload-dir "$BIG_BLOCKS_DIR/payloads" \
--engine-rpc-url http://127.0.0.1:8551 \
--jwt-secret "$DATADIR/jwt.hex" \
--output "$OUTPUT_DIR" 2>&1 | sed -u "s/^/[bench] /"
else
# Standard mode: warmup + new-payload-fcu
# Warmup
$BENCH_NICE "$RETH_BENCH" new-payload-fcu \
--rpc-url "$BENCH_RPC_URL" \
--engine-rpc-url http://127.0.0.1:8551 \
--jwt-secret "$DATADIR/jwt.hex" \
--advance "${BENCH_WARMUP_BLOCKS:-50}" \
"${EXTRA_BENCH_ARGS[@]}" 2>&1 | sed -u "s/^/[bench] /"
# Start tracy-capture after warmup so profile only covers the benchmark
if [ "${BENCH_TRACY:-off}" != "off" ]; then
echo "Starting tracy-capture..."
tracy-capture -f -o "$OUTPUT_DIR/tracy-profile.tracy" &
TRACY_PID=$!
sleep 0.5 # give tracy-capture time to connect
fi
# Benchmark
$BENCH_NICE "$RETH_BENCH" new-payload-fcu \
--rpc-url "$BENCH_RPC_URL" \
--engine-rpc-url http://127.0.0.1:8551 \
--jwt-secret "$DATADIR/jwt.hex" \
--advance "$BENCH_BLOCKS" \
"${EXTRA_BENCH_ARGS[@]}" \
--output "$OUTPUT_DIR" 2>&1 | sed -u "s/^/[bench] /"
fi
# cleanup runs via trap

View File

@@ -1,127 +0,0 @@
#!/usr/bin/env bash
#
# Downloads the latest nightly snapshot into the schelk volume with
# progress reporting to the GitHub PR comment.
#
# Skips the download if the local ETag marker matches the remote one.
#
# Usage: bench-reth-snapshot.sh [--check]
# --check Only check if a download is needed; exits 0 if up-to-date, 1 if not.
#
# Required env:
# SCHELK_MOUNT schelk mount point (e.g. /reth-bench)
# GITHUB_TOKEN token for GitHub API calls (only for download)
# BENCH_COMMENT_ID PR comment ID to update (optional)
# BENCH_REPO owner/repo (e.g. paradigmxyz/reth)
# BENCH_JOB_URL link to the Actions job
# BENCH_ACTOR user who triggered the benchmark
# BENCH_CONFIG config summary line
set -euo pipefail
BUCKET="minio/reth-snapshots/reth-1-minimal-nightly-previous.tar.zst"
DATADIR="$SCHELK_MOUNT/datadir"
ETAG_FILE="$HOME/.reth-bench-snapshot-etag"
# Get remote metadata via JSON for reliable parsing
MC_STAT=$(mc stat --json "$BUCKET" 2>/dev/null || true)
REMOTE_ETAG=$(echo "$MC_STAT" | jq -r '.etag // empty')
if [ -z "$REMOTE_ETAG" ]; then
echo "::warning::Failed to get ETag from mc stat, will re-download"
REMOTE_ETAG="unknown-$(date +%s)"
fi
LOCAL_ETAG=""
[ -f "$ETAG_FILE" ] && LOCAL_ETAG=$(cat "$ETAG_FILE")
if [ "$REMOTE_ETAG" = "$LOCAL_ETAG" ]; then
echo "Snapshot is up-to-date (ETag: ${REMOTE_ETAG})"
if [ "${1:-}" = "--check" ]; then
exit 0
fi
exit 0
fi
echo "Snapshot needs update (local: ${LOCAL_ETAG:-<none>}, remote: ${REMOTE_ETAG})"
if [ "${1:-}" = "--check" ]; then
exit 1
fi
# Get compressed size for progress tracking
TOTAL_BYTES=$(echo "$MC_STAT" | jq -r '.size // empty')
if [ -z "$TOTAL_BYTES" ] || [ "$TOTAL_BYTES" = "0" ]; then
echo "::error::Failed to get snapshot size from mc stat"
exit 1
fi
echo "Snapshot size: $TOTAL_BYTES bytes ($(numfmt --to=iec "$TOTAL_BYTES"))"
# Prepare mount
mountpoint -q "$SCHELK_MOUNT" && sudo schelk recover -y || true
sudo schelk mount -y
sudo rm -rf "$DATADIR"
sudo mkdir -p "$DATADIR"
update_comment() {
local pct="$1"
[ -z "${BENCH_COMMENT_ID:-}" ] && return 0
local status="Building binaries & downloading snapshot… ${pct}%"
local body
body="$(printf 'cc @%s\n\n🚀 Benchmark started! [View job](%s)\n\n⏳ **Status:** %s\n\n%s' \
"$BENCH_ACTOR" "$BENCH_JOB_URL" "$status" "$BENCH_CONFIG")"
curl -sf -X PATCH \
-H "Authorization: token ${GITHUB_TOKEN}" \
-H "Accept: application/vnd.github.v3+json" \
"https://api.github.com/repos/${BENCH_REPO}/issues/comments/${BENCH_COMMENT_ID}" \
-d "$(jq -nc --arg body "$body" '{body: $body}')" \
> /dev/null 2>&1 || true
}
# Track compressed bytes flowing through the pipe
DL_BYTES_FILE=$(mktemp)
echo 0 > "$DL_BYTES_FILE"
# Start progress reporter in background
(
while true; do
sleep 10
CURRENT=$(cat "$DL_BYTES_FILE" 2>/dev/null || echo 0)
if [ "$TOTAL_BYTES" -gt 0 ]; then
PCT=$(( CURRENT * 100 / TOTAL_BYTES ))
[ "$PCT" -gt 100 ] && PCT=100
echo "Snapshot download: $(numfmt --to=iec "$CURRENT") / $(numfmt --to=iec "$TOTAL_BYTES") (${PCT}%)"
update_comment "$PCT"
fi
done
) &
PROGRESS_PID=$!
trap 'kill $PROGRESS_PID 2>/dev/null || true; rm -f "$DL_BYTES_FILE"' EXIT
# Download and extract; python byte counter tracks compressed bytes received
mc cat "$BUCKET" | python3 -c "
import sys
count = 0
while True:
data = sys.stdin.buffer.read(1048576)
if not data:
break
count += len(data)
sys.stdout.buffer.write(data)
with open('$DL_BYTES_FILE', 'w') as f:
f.write(str(count))
" | pzstd -d -p 6 | sudo tar -xf - -C "$DATADIR"
# Stop progress reporter
kill $PROGRESS_PID 2>/dev/null || true
wait $PROGRESS_PID 2>/dev/null || true
update_comment "100"
echo "Snapshot download complete"
# Promote the new snapshot to become the schelk baseline (virgin volume).
# This copies changed blocks from scratch → virgin so that future
# `schelk recover` calls restore to this new state.
sync
sudo schelk promote -y
# Save ETag marker
echo "$REMOTE_ETAG" > "$ETAG_FILE"
echo "Snapshot promoted to schelk baseline (ETag: ${REMOTE_ETAG})"

View File

@@ -1,592 +0,0 @@
#!/usr/bin/env python3
"""Parse reth-bench CSV output and generate a summary JSON + markdown comparison.
Usage:
bench-reth-summary.py <combined_csv> <gas_csv> \
--output-summary <summary.json> \
--output-markdown <comment.md> \
--baseline-csv <baseline_combined.csv> \
[--repo <owner/repo>] \
[--baseline-ref <sha>] \
[--feature-name <name>] \
[--feature-sha <sha>]
Generates a paired statistical comparison between baseline and feature.
Matches blocks by number and computes per-block diffs to cancel out gas
variance. Fails if baseline or feature CSV is missing or empty.
"""
import argparse
import csv
import json
import math
import random
import sys
GIGAGAS = 1_000_000_000
T_CRITICAL = 1.96 # two-tailed 95% confidence
BOOTSTRAP_ITERATIONS = 10_000
def _opt_int(row: dict, key: str) -> int | None:
"""Return int value for a CSV field, or None if missing/empty."""
v = row.get(key)
if v is None or v == "":
return None
return int(v)
def parse_combined_csv(path: str) -> list[dict]:
"""Parse combined_latency.csv into a list of per-block dicts."""
rows = []
with open(path) as f:
reader = csv.DictReader(f)
for row in reader:
rows.append(
{
"block_number": int(row["block_number"]),
"gas_used": int(row["gas_used"]),
"gas_limit": int(row["gas_limit"]),
"transaction_count": int(row["transaction_count"]),
"new_payload_latency_us": int(row["new_payload_latency"]),
"fcu_latency_us": int(row["fcu_latency"]),
"total_latency_us": int(row["total_latency"]),
"persistence_wait_us": _opt_int(row, "persistence_wait"),
"execution_cache_wait_us": _opt_int(row, "execution_cache_wait"),
"sparse_trie_wait_us": _opt_int(row, "sparse_trie_wait"),
}
)
return rows
def parse_gas_csv(path: str) -> list[dict]:
"""Parse total_gas.csv into a list of per-block dicts."""
rows = []
with open(path) as f:
reader = csv.DictReader(f)
for row in reader:
rows.append(
{
"block_number": int(row["block_number"]),
"gas_used": int(row["gas_used"]),
"time_us": int(row["time"]),
}
)
return rows
def stddev(values: list[float], mean: float) -> float:
if len(values) < 2:
return 0.0
return math.sqrt(sum((v - mean) ** 2 for v in values) / (len(values) - 1))
def percentile(sorted_vals: list[float], pct: int) -> float:
if not sorted_vals:
return 0.0
idx = int(len(sorted_vals) * pct / 100)
idx = min(idx, len(sorted_vals) - 1)
return sorted_vals[idx]
def compute_stats(combined: list[dict]) -> dict:
"""Compute per-run statistics from parsed CSV data."""
n = len(combined)
if n == 0:
return {}
latencies_ms = [r["new_payload_latency_us"] / 1_000 for r in combined]
sorted_lat = sorted(latencies_ms)
mean_lat = sum(latencies_ms) / n
std_lat = stddev(latencies_ms, mean_lat)
mgas_s_values = []
for r in combined:
lat_s = r["new_payload_latency_us"] / 1_000_000
if lat_s > 0:
mgas_s_values.append(r["gas_used"] / lat_s / 1_000_000)
mean_mgas_s = sum(mgas_s_values) / len(mgas_s_values) if mgas_s_values else 0
total_latencies_ms = [r["total_latency_us"] / 1_000 for r in combined]
wall_clock_s = sum(total_latencies_ms) / 1_000
mean_total_lat_ms = sum(total_latencies_ms) / n
return {
"n": n,
"mean_ms": mean_lat,
"stddev_ms": std_lat,
"p50_ms": percentile(sorted_lat, 50),
"p90_ms": percentile(sorted_lat, 90),
"p99_ms": percentile(sorted_lat, 99),
"mean_mgas_s": mean_mgas_s,
"wall_clock_s": wall_clock_s,
"mean_total_lat_ms": mean_total_lat_ms,
}
def compute_wait_stats(combined: list[dict], field: str) -> dict:
"""Compute mean/p50/p95 for a wait time field (in ms)."""
values_ms = []
for r in combined:
v = r.get(field)
if v is not None:
values_ms.append(v / 1_000)
if not values_ms:
return {}
n = len(values_ms)
mean_val = sum(values_ms) / n
sorted_vals = sorted(values_ms)
return {
"mean_ms": mean_val,
"p50_ms": percentile(sorted_vals, 50),
"p95_ms": percentile(sorted_vals, 95),
}
def _paired_data(
baseline: list[dict], feature: list[dict]
) -> tuple[list[tuple[float, float]], list[float], list[float], list[float]]:
"""Match blocks and return paired latencies and per-block diffs.
Returns:
pairs: list of (baseline_ms, feature_ms) tuples
lat_diffs_ms: list of feature baseline latency diffs in ms
mgas_diffs: list of feature baseline Mgas/s diffs
total_lat_diffs_ms: list of feature baseline total latency diffs in ms
"""
baseline_by_block = {r["block_number"]: r for r in baseline}
feature_by_block = {r["block_number"]: r for r in feature}
common_blocks = sorted(set(baseline_by_block) & set(feature_by_block))
pairs = []
lat_diffs_ms = []
mgas_diffs = []
total_lat_diffs_ms = []
for bn in common_blocks:
b = baseline_by_block[bn]
f = feature_by_block[bn]
b_ms = b["new_payload_latency_us"] / 1_000
f_ms = f["new_payload_latency_us"] / 1_000
pairs.append((b_ms, f_ms))
lat_diffs_ms.append(f_ms - b_ms)
b_lat_s = b["new_payload_latency_us"] / 1_000_000
f_lat_s = f["new_payload_latency_us"] / 1_000_000
if b_lat_s > 0 and f_lat_s > 0:
mgas_diffs.append(
f["gas_used"] / f_lat_s / 1_000_000
- b["gas_used"] / b_lat_s / 1_000_000
)
total_lat_diffs_ms.append(
f["total_latency_us"] / 1_000 - b["total_latency_us"] / 1_000
)
return pairs, lat_diffs_ms, mgas_diffs, total_lat_diffs_ms
def compute_paired_stats(
baseline_runs: list[list[dict]],
feature_runs: list[list[dict]],
) -> dict:
"""Compute paired statistics between baseline and feature runs.
Each pair (baseline_runs[i], feature_runs[i]) produces per-block diffs.
All diffs are pooled for the final CI.
"""
all_pairs = []
all_lat_diffs = []
all_mgas_diffs = []
all_total_lat_diffs = []
blocks_per_pair = []
for baseline, feature in zip(baseline_runs, feature_runs):
pairs, lat_diffs, mgas_diffs, total_lat_diffs = _paired_data(baseline, feature)
all_pairs.extend(pairs)
all_lat_diffs.extend(lat_diffs)
all_mgas_diffs.extend(mgas_diffs)
all_total_lat_diffs.extend(total_lat_diffs)
blocks_per_pair.append(len(pairs))
if not all_lat_diffs:
return {}
n = len(all_lat_diffs)
mean_diff = sum(all_lat_diffs) / n
std_diff = stddev(all_lat_diffs, mean_diff)
se = std_diff / math.sqrt(n) if n > 0 else 0.0
ci = T_CRITICAL * se
# Bootstrap CI on difference-of-percentiles (resample paired blocks)
base_lats = sorted([p[0] for p in all_pairs])
feature_lats = sorted([p[1] for p in all_pairs])
p50_diff = percentile(feature_lats, 50) - percentile(base_lats, 50)
p90_diff = percentile(feature_lats, 90) - percentile(base_lats, 90)
p99_diff = percentile(feature_lats, 99) - percentile(base_lats, 99)
rng = random.Random(42)
p50_boot, p90_boot, p99_boot = [], [], []
for _ in range(BOOTSTRAP_ITERATIONS):
sample = rng.choices(all_pairs, k=n)
b_sorted = sorted(p[0] for p in sample)
f_sorted = sorted(p[1] for p in sample)
p50_boot.append(percentile(f_sorted, 50) - percentile(b_sorted, 50))
p90_boot.append(percentile(f_sorted, 90) - percentile(b_sorted, 90))
p99_boot.append(percentile(f_sorted, 99) - percentile(b_sorted, 99))
p50_boot.sort()
p90_boot.sort()
p99_boot.sort()
lo = int(BOOTSTRAP_ITERATIONS * 0.025)
hi = int(BOOTSTRAP_ITERATIONS * 0.975)
mean_mgas_diff = sum(all_mgas_diffs) / len(all_mgas_diffs) if all_mgas_diffs else 0.0
std_mgas_diff = stddev(all_mgas_diffs, mean_mgas_diff) if len(all_mgas_diffs) > 1 else 0.0
mgas_se = std_mgas_diff / math.sqrt(len(all_mgas_diffs)) if all_mgas_diffs else 0.0
mgas_ci = T_CRITICAL * mgas_se
mean_total_diff = sum(all_total_lat_diffs) / len(all_total_lat_diffs) if all_total_lat_diffs else 0.0
std_total_diff = stddev(all_total_lat_diffs, mean_total_diff) if len(all_total_lat_diffs) > 1 else 0.0
total_se = std_total_diff / math.sqrt(len(all_total_lat_diffs)) if all_total_lat_diffs else 0.0
wall_clock_ci_ms = T_CRITICAL * total_se
return {
"n": n,
"mean_diff_ms": mean_diff,
"ci_ms": ci,
"p50_diff_ms": p50_diff,
"p50_ci_ms": (p50_boot[hi] - p50_boot[lo]) / 2,
"p90_diff_ms": p90_diff,
"p90_ci_ms": (p90_boot[hi] - p90_boot[lo]) / 2,
"p99_diff_ms": p99_diff,
"p99_ci_ms": (p99_boot[hi] - p99_boot[lo]) / 2,
"mean_mgas_diff": mean_mgas_diff,
"mgas_ci": mgas_ci,
"wall_clock_ci_ms": wall_clock_ci_ms,
"blocks": max(blocks_per_pair),
}
def format_duration(seconds: float) -> str:
if seconds >= 60:
return f"{seconds / 60:.1f}min"
return f"{seconds}s"
def format_gas(gas: int) -> str:
if gas >= GIGAGAS:
return f"{gas / GIGAGAS:.1f}G"
if gas >= 1_000_000:
return f"{gas / 1_000_000:.1f}M"
return f"{gas:,}"
def fmt_ms(v: float) -> str:
return f"{v:.2f}ms"
def fmt_mgas(v: float) -> str:
return f"{v:.2f}"
def fmt_s(v: float) -> str:
return f"{v:.2f}s"
def significance(pct: float, ci_pct: float, lower_is_better: bool) -> str:
"""Return significance label: 'good', 'bad', or 'neutral'."""
significant = abs(pct) > ci_pct
if not significant:
return "neutral"
elif (pct < 0) == lower_is_better:
return "good"
else:
return "bad"
def change_str(pct: float, ci_pct: float, lower_is_better: bool) -> str:
"""Format change% with paired CI significance.
Significant if the CI doesn't cross zero (i.e. |pct| > ci_pct).
"""
sig = significance(pct, ci_pct, lower_is_better)
emoji = {"good": "", "bad": "", "neutral": ""}[sig]
return f"{pct:+.2f}% {emoji}{ci_pct:.2f}%)"
def compute_changes(
baseline_stats: dict, feature_stats: dict, paired_stats: dict
) -> dict:
"""Pre-compute change percentages and significance for each metric."""
def pct(base: float, feat: float) -> float:
return (feat - base) / base * 100.0 if base > 0 else 0.0
def ci_pct(ci_ms: float, base_ms: float) -> float:
return ci_ms / base_ms * 100.0 if base_ms > 0 else 0.0
metrics = [
("mean", "mean_ms", "ci_ms", "mean_ms", True),
("p50", "p50_ms", "p50_ci_ms", "p50_ms", True),
("p90", "p90_ms", "p90_ci_ms", "p90_ms", True),
("p99", "p99_ms", "p99_ci_ms", "p99_ms", True),
("mgas_s", "mean_mgas_s", "mgas_ci", "mean_mgas_s", False),
("wall_clock", "wall_clock_s", "wall_clock_ci_ms", "mean_total_lat_ms", True),
]
changes = {}
for name, stat_key, ci_key, base_key, lower_is_better in metrics:
p = pct(baseline_stats[stat_key], feature_stats[stat_key])
c = ci_pct(paired_stats[ci_key], baseline_stats[base_key])
changes[name] = {
"pct": round(p, 4),
"ci_pct": round(c, 4),
"sig": significance(p, c, lower_is_better),
}
return changes
def generate_comparison_table(
run1: dict,
run2: dict,
paired: dict,
repo: str,
baseline_ref: str,
baseline_name: str,
feature_name: str,
feature_sha: str,
big_blocks: bool = False,
) -> str:
"""Generate a markdown comparison table between baseline and feature."""
n = paired["blocks"]
def pct(base: float, feat: float) -> float:
return (feat - base) / base * 100.0 if base > 0 else 0.0
mean_pct = pct(run1["mean_ms"], run2["mean_ms"])
gas_pct = pct(run1["mean_mgas_s"], run2["mean_mgas_s"])
wall_pct = pct(run1["wall_clock_s"], run2["wall_clock_s"])
p50_pct = pct(run1["p50_ms"], run2["p50_ms"])
p90_pct = pct(run1["p90_ms"], run2["p90_ms"])
p99_pct = pct(run1["p99_ms"], run2["p99_ms"])
# Bootstrap CIs as % of baseline percentile
p50_ci_pct = paired["p50_ci_ms"] / run1["p50_ms"] * 100.0 if run1["p50_ms"] > 0 else 0.0
p90_ci_pct = paired["p90_ci_ms"] / run1["p90_ms"] * 100.0 if run1["p90_ms"] > 0 else 0.0
p99_ci_pct = paired["p99_ci_ms"] / run1["p99_ms"] * 100.0 if run1["p99_ms"] > 0 else 0.0
# CI as a percentage of baseline mean
lat_ci_pct = paired["ci_ms"] / run1["mean_ms"] * 100.0 if run1["mean_ms"] > 0 else 0.0
mgas_ci_pct = paired["mgas_ci"] / run1["mean_mgas_s"] * 100.0 if run1["mean_mgas_s"] > 0 else 0.0
wall_ci_pct = paired["wall_clock_ci_ms"] / run1["mean_total_lat_ms"] * 100.0 if run1["mean_total_lat_ms"] > 0 else 0.0
base_url = f"https://github.com/{repo}/commit"
baseline_label = f"[`{baseline_name}`]({base_url}/{baseline_ref})"
feature_label = f"[`{feature_name}`]({base_url}/{feature_sha})"
lines = [
f"| Metric | {baseline_label} | {feature_label} | Change |",
"|--------|------|--------|--------|",
f"| Mean | {fmt_ms(run1['mean_ms'])} | {fmt_ms(run2['mean_ms'])} | {change_str(mean_pct, lat_ci_pct, lower_is_better=True)} |",
f"| StdDev | {fmt_ms(run1['stddev_ms'])} | {fmt_ms(run2['stddev_ms'])} | |",
f"| P50 | {fmt_ms(run1['p50_ms'])} | {fmt_ms(run2['p50_ms'])} | {change_str(p50_pct, p50_ci_pct, lower_is_better=True)} |",
f"| P90 | {fmt_ms(run1['p90_ms'])} | {fmt_ms(run2['p90_ms'])} | {change_str(p90_pct, p90_ci_pct, lower_is_better=True)} |",
f"| P99 | {fmt_ms(run1['p99_ms'])} | {fmt_ms(run2['p99_ms'])} | {change_str(p99_pct, p99_ci_pct, lower_is_better=True)} |",
f"| Mgas/s | {fmt_mgas(run1['mean_mgas_s'])} | {fmt_mgas(run2['mean_mgas_s'])} | {change_str(gas_pct, mgas_ci_pct, lower_is_better=False)} |",
f"| Wall Clock | {fmt_s(run1['wall_clock_s'])} | {fmt_s(run2['wall_clock_s'])} | {change_str(wall_pct, wall_ci_pct, lower_is_better=True)} |",
"",
f"*{n} {'big blocks' if big_blocks else 'blocks'}*",
]
return "\n".join(lines)
def generate_wait_time_table(
title: str,
baseline_stats: dict,
feature_stats: dict,
baseline_label: str,
feature_label: str,
) -> str:
"""Generate a markdown table for a wait time metric."""
if not baseline_stats or not feature_stats:
return ""
lines = [
f"### {title}",
"",
f"| Metric | {baseline_label} | {feature_label} |",
"|--------|------|--------|",
f"| Mean | {fmt_ms(baseline_stats['mean_ms'])} | {fmt_ms(feature_stats['mean_ms'])} |",
f"| P50 | {fmt_ms(baseline_stats['p50_ms'])} | {fmt_ms(feature_stats['p50_ms'])} |",
f"| P95 | {fmt_ms(baseline_stats['p95_ms'])} | {fmt_ms(feature_stats['p95_ms'])} |",
]
return "\n".join(lines)
def generate_markdown(
summary: dict, comparison_table: str,
wait_time_tables: list[str] | None = None,
behind_baseline: int = 0, repo: str = "", baseline_ref: str = "", baseline_name: str = "",
grafana_url: str | None = None,
) -> str:
"""Generate a markdown comment body."""
lines = ["## Benchmark Results", ""]
if behind_baseline > 0:
s = "s" if behind_baseline > 1 else ""
diff_link = f"https://github.com/{repo}/compare/{baseline_ref[:12]}...{baseline_name}"
lines.append(f"> ⚠️ Feature is [**{behind_baseline} commit{s} behind `{baseline_name}`**]({diff_link}). Consider rebasing for accurate results.")
lines.append("")
lines.append(comparison_table)
if wait_time_tables:
lines.append("")
lines.append("<details>")
lines.append("<summary>Wait Time Breakdown</summary>")
lines.append("")
for table in wait_time_tables:
if table:
lines.append(table)
lines.append("")
lines.append("</details>")
if grafana_url:
lines.append("")
lines.append(f"**[Grafana Dashboard]({grafana_url})**")
return "\n".join(lines)
def main():
parser = argparse.ArgumentParser(description="Parse reth-bench ABBA results")
parser.add_argument(
"--baseline-csv", nargs="+", required=True,
help="Baseline combined_latency.csv files (A1, A2)",
)
parser.add_argument(
"--feature-csv", "--branch-csv", nargs="+", required=True,
help="Feature combined_latency.csv files (B1, B2)",
)
parser.add_argument("--gas-csv", required=True, help="Path to total_gas.csv")
parser.add_argument(
"--output-summary", required=True, help="Output JSON summary path"
)
parser.add_argument("--output-markdown", required=True, help="Output markdown path")
parser.add_argument(
"--repo", default="paradigmxyz/reth", help="GitHub repo (owner/name)"
)
parser.add_argument("--baseline-ref", default=None, help="Baseline commit SHA")
parser.add_argument("--baseline-name", default=None, help="Baseline display name")
parser.add_argument("--feature-name", "--branch-name", default=None, help="Feature branch name")
parser.add_argument("--feature-ref", "--branch-sha", "--feature-sha", default=None, help="Feature commit SHA")
parser.add_argument("--behind-baseline", "--behind-main", type=int, default=0, help="Commits behind baseline")
parser.add_argument("--big-blocks", action="store_true", default=False, help="Big blocks mode")
parser.add_argument("--gas-ramp-blocks", type=int, default=0, help="Number of gas ramp blocks (big blocks mode)")
parser.add_argument("--grafana-url", default=None, help="Grafana dashboard URL for this benchmark run")
args = parser.parse_args()
if len(args.baseline_csv) != len(args.feature_csv):
print("Must provide equal number of baseline and feature CSVs", file=sys.stderr)
sys.exit(1)
baseline_runs = []
feature_runs = []
for path in args.baseline_csv:
data = parse_combined_csv(path)
if not data:
print(f"No results in {path}", file=sys.stderr)
sys.exit(1)
baseline_runs.append(data)
for path in args.feature_csv:
data = parse_combined_csv(path)
if not data:
print(f"No results in {path}", file=sys.stderr)
sys.exit(1)
feature_runs.append(data)
gas = parse_gas_csv(args.gas_csv)
all_baseline = [r for run in baseline_runs for r in run]
all_feature = [r for run in feature_runs for r in run]
baseline_stats = compute_stats(all_baseline)
feature_stats = compute_stats(all_feature)
paired_stats = compute_paired_stats(baseline_runs, feature_runs)
if not paired_stats:
print("No common blocks between baseline and feature runs", file=sys.stderr)
sys.exit(1)
baseline_ref = args.baseline_ref or "main"
baseline_name = args.baseline_name or "baseline"
feature_name = args.feature_name or "feature"
feature_sha = args.feature_ref or "unknown"
comparison_table = generate_comparison_table(
baseline_stats,
feature_stats,
paired_stats,
repo=args.repo,
baseline_ref=baseline_ref,
baseline_name=baseline_name,
feature_name=feature_name,
feature_sha=feature_sha,
big_blocks=args.big_blocks,
)
print(f"Generated comparison ({paired_stats['n']} paired blocks, "
f"mean diff {paired_stats['mean_diff_ms']:+.3f}ms ± {paired_stats['ci_ms']:.3f}ms)")
base_url = f"https://github.com/{args.repo}/commit"
baseline_label = f"[`{baseline_name}`]({base_url}/{baseline_ref})"
feature_label = f"[`{feature_name}`]({base_url}/{feature_sha})"
wait_fields = [
("persistence_wait_us", "Persistence Wait"),
("sparse_trie_wait_us", "Trie Cache Update Wait"),
("execution_cache_wait_us", "Execution Cache Update Wait"),
]
wait_time_tables = []
wait_time_data = {}
for field, title in wait_fields:
b_stats = compute_wait_stats(all_baseline, field)
f_stats = compute_wait_stats(all_feature, field)
if b_stats and f_stats:
wait_time_data[field] = {
"title": title,
"baseline": b_stats,
"feature": f_stats,
}
table = generate_wait_time_table(title, b_stats, f_stats, baseline_label, feature_label)
if table:
wait_time_tables.append(table)
summary = {
"blocks": paired_stats["blocks"],
"big_blocks": args.big_blocks,
"gas_ramp_blocks": args.gas_ramp_blocks,
"baseline": {
"name": baseline_name,
"ref": baseline_ref,
"stats": baseline_stats,
},
"feature": {
"name": feature_name,
"ref": feature_sha,
"stats": feature_stats,
},
"paired": paired_stats,
"changes": compute_changes(baseline_stats, feature_stats, paired_stats),
"wait_times": wait_time_data,
}
with open(args.output_summary, "w") as f:
json.dump(summary, f, indent=2)
print(f"Summary written to {args.output_summary}")
markdown = generate_markdown(
summary, comparison_table,
wait_time_tables=wait_time_tables,
behind_baseline=args.behind_baseline,
repo=args.repo,
baseline_ref=baseline_ref,
baseline_name=baseline_name,
grafana_url=args.grafana_url,
)
with open(args.output_markdown, "w") as f:
f.write(markdown)
print(f"Markdown written to {args.output_markdown}")
if __name__ == "__main__":
main()

View File

@@ -1,350 +0,0 @@
// Sends Slack notifications for reth-bench results.
//
// Reads from environment:
// SLACK_BENCH_BOT_TOKEN Slack Bot User OAuth Token (xoxb-...)
// SLACK_BENCH_CHANNEL Public channel ID for significant improvements
// BENCH_WORK_DIR Directory containing summary.json
// BENCH_PR PR number (may be empty)
// BENCH_ACTOR GitHub user who triggered the bench
// BENCH_JOB_URL URL to the Actions job page
// BENCH_SAMPLY 'true' if samply profiling was enabled
//
// Usage from actions/github-script:
// const notify = require('./.github/scripts/bench-slack-notify.js');
// await notify.success({ core, context });
// await notify.failure({ core, context, failedStep: '...' });
const fs = require('fs');
const path = require('path');
const SLACK_API = 'https://slack.com/api/chat.postMessage';
function loadSlackUsers(repoRoot) {
try {
const raw = fs.readFileSync(path.join(repoRoot, '.github', 'scripts', 'bench-slack-users.json'), 'utf8');
const data = JSON.parse(raw);
// Filter out non-user-ID entries (like _comment)
const users = {};
for (const [k, v] of Object.entries(data)) {
if (!k.startsWith('_') && typeof v === 'string' && v.startsWith('U')) {
users[k] = v;
}
}
return users;
} catch {
return {};
}
}
async function postToSlack(token, channel, blocks, text, core, threadTs) {
const payload = { channel, blocks, text, unfurl_links: false };
if (threadTs) payload.thread_ts = threadTs;
const resp = await fetch(SLACK_API, {
method: 'POST',
headers: {
'Authorization': `Bearer ${token}`,
'Content-Type': 'application/json',
},
body: JSON.stringify(payload),
});
const data = await resp.json();
if (!data.ok) {
core.warning(`Slack API error (channel ${channel}): ${JSON.stringify(data)}`);
}
return data;
}
function cell(text) {
const s = String(text);
return { type: 'raw_text', text: s || ' ' };
}
function buildSuccessBlocks({ summary, prNumber, actor, actorSlackId, jobUrl, repo, samplyUrls }) {
const b = summary.baseline.stats;
const f = summary.feature.stats;
const c = summary.changes;
const sigEmoji = { good: '\u2705', bad: '\u274c', neutral: '\u26aa' };
function fmtMs(v) { return v.toFixed(2) + 'ms'; }
function fmtMgas(v) { return v.toFixed(2); }
function fmtS(v) { return v.toFixed(2) + 's'; }
function fmtChange(ch) {
if (!ch.pct && !ch.ci_pct) return ' ';
const pctStr = `${ch.pct >= 0 ? '+' : ''}${ch.pct.toFixed(2)}%`;
const ciStr = ch.ci_pct ? ` (\u00b1${ch.ci_pct.toFixed(2)}%)` : '';
return `${pctStr}${ciStr} ${sigEmoji[ch.sig]}`;
}
// Overall result for header
const vals = Object.values(c);
const hasBad = vals.some(v => v.sig === 'bad');
const hasGood = vals.some(v => v.sig === 'good');
let headerEmoji, headerResult;
if (hasBad && hasGood) {
headerEmoji = ':warning:';
headerResult = 'Mixed Results';
} else if (hasBad) {
headerEmoji = ':x:';
headerResult = 'Regression';
} else if (hasGood) {
headerEmoji = ':white_check_mark:';
headerResult = 'Improvement';
} else {
headerEmoji = ':white_circle:';
headerResult = 'No Difference';
}
const prUrl = prNumber ? `https://github.com/${repo}/pull/${prNumber}` : '';
const commitUrl = `https://github.com/${repo}/commit`;
const baselineLink = `<${commitUrl}/${summary.baseline.ref}|${summary.baseline.name}>`;
const featureLink = `<${commitUrl}/${summary.feature.ref}|${summary.feature.name}>`;
// Meta line
const metaParts = [];
if (prNumber) metaParts.push(`*<${prUrl}|PR #${prNumber}>*`);
metaParts.push(`triggered by ${actorSlackId ? `<@${actorSlackId}>` : `@${actor}`}`);
// Baseline/feature lines with samply profile links
let baselineLine = `*Baseline:* ${baselineLink}`;
const bl1 = samplyUrls['baseline-1'];
const bl2 = samplyUrls['baseline-2'];
if (bl1) baselineLine += ` | <${bl1}|Samply 1>`;
if (bl2) baselineLine += ` | <${bl2}|Samply 2>`;
let featureLine = `*Feature:* ${featureLink}`;
const fl1 = samplyUrls['feature-1'];
const fl2 = samplyUrls['feature-2'];
if (fl1) featureLine += ` | <${fl1}|Samply 1>`;
if (fl2) featureLine += ` | <${fl2}|Samply 2>`;
const cores = process.env.BENCH_CORES || '0';
const countsParts = [];
if (summary.big_blocks) {
const gasRamp = summary.gas_ramp_blocks || 0;
if (gasRamp > 0) countsParts.push(`*Gas Ramp:* ${gasRamp}`);
countsParts.push(`*Big Blocks:* ${summary.blocks}`);
} else {
const warmup = summary.warmup_blocks || process.env.BENCH_WARMUP_BLOCKS || '';
if (warmup) countsParts.push(`*Warmup:* ${warmup}`);
countsParts.push(`*Blocks:* ${summary.blocks}`);
}
if (cores !== '0') countsParts.push(`*Cores:* ${cores}`);
const countsLine = countsParts.join(' | ');
const sectionText = [metaParts.join(' | '), '', baselineLine, featureLine, countsLine].join('\n');
// Action buttons
const diffUrl = `https://github.com/${repo}/compare/${summary.baseline.ref}...${summary.feature.ref}`;
const buttons = [
{
type: 'button',
text: { type: 'plain_text', text: 'CI :github:', emoji: true },
url: jobUrl,
action_id: 'ci_button',
},
{
type: 'button',
text: { type: 'plain_text', text: 'Diff :github:', emoji: true },
url: diffUrl,
action_id: 'diff_button',
},
];
const blocks = [
{
type: 'header',
text: { type: 'plain_text', text: `${headerEmoji} ${headerResult}`, emoji: true },
},
{
type: 'section',
text: { type: 'mrkdwn', text: sectionText },
},
{
type: 'table',
column_settings: [
{ align: 'left' },
{ align: 'right' },
{ align: 'right' },
{ align: 'right' },
],
rows: [
[cell('Metric'), cell('Baseline'), cell('Feature'), cell('Change')],
[cell('Mean'), cell(fmtMs(b.mean_ms)), cell(fmtMs(f.mean_ms)), cell(fmtChange(c.mean))],
[cell('StdDev'), cell(fmtMs(b.stddev_ms)), cell(fmtMs(f.stddev_ms)), cell(' ')],
[cell('P50'), cell(fmtMs(b.p50_ms)), cell(fmtMs(f.p50_ms)), cell(fmtChange(c.p50))],
[cell('P90'), cell(fmtMs(b.p90_ms)), cell(fmtMs(f.p90_ms)), cell(fmtChange(c.p90))],
[cell('P99'), cell(fmtMs(b.p99_ms)), cell(fmtMs(f.p99_ms)), cell(fmtChange(c.p99))],
[cell('Mgas/s'), cell(fmtMgas(b.mean_mgas_s)), cell(fmtMgas(f.mean_mgas_s)), cell(fmtChange(c.mgas_s))],
[cell('Wall Clock'), cell(fmtS(b.wall_clock_s)), cell(fmtS(f.wall_clock_s)), cell(fmtChange(c.wall_clock))],
],
},
{
type: 'actions',
elements: buttons,
},
];
// Wait times as a separate table block (sent as threaded reply due to Slack one-table limit)
const threadBlocks = [];
const waitTimes = summary.wait_times || {};
const waitKeys = Object.keys(waitTimes);
if (waitKeys.length > 0) {
const waitRows = [
[cell('Wait Time'), cell('Baseline'), cell('Feature')],
];
for (const key of waitKeys) {
const wt = waitTimes[key];
waitRows.push([cell(wt.title), cell(fmtMs(wt.baseline.mean_ms)), cell(fmtMs(wt.feature.mean_ms))]);
}
threadBlocks.push({
type: 'table',
column_settings: [
{ align: 'left' },
{ align: 'right' },
{ align: 'right' },
],
rows: waitRows,
});
}
return { blocks, threadBlocks };
}
function buildFailureBlocks({ prNumber, actor, actorSlackId, jobUrl, repo, failedStep }) {
const prUrl = prNumber ? `https://github.com/${repo}/pull/${prNumber}` : '';
const actorMention = actorSlackId ? `<@${actorSlackId}>` : `@${actor}`;
const parts = [
prNumber ? `*<${prUrl}|PR #${prNumber}>*` : '',
`by ${actorMention}`,
`failed while *${failedStep}*`,
].filter(Boolean);
const buttons = [
{
type: 'button',
text: { type: 'plain_text', text: 'CI :github:', emoji: true },
url: jobUrl,
action_id: 'ci_button',
},
];
return [
{
type: 'header',
text: { type: 'plain_text', text: ':rotating_light: Bench Failed', emoji: true },
},
{
type: 'section',
text: { type: 'mrkdwn', text: parts.join(' | ') },
},
{
type: 'actions',
elements: buttons,
},
];
}
async function success({ core, context }) {
const token = process.env.SLACK_BENCH_BOT_TOKEN;
if (!token) {
core.info('SLACK_BENCH_BOT_TOKEN not set, skipping Slack notification');
return;
}
let summary;
try {
summary = JSON.parse(fs.readFileSync(process.env.BENCH_WORK_DIR + '/summary.json', 'utf8'));
} catch (e) {
core.warning('Could not read summary.json for Slack notification');
return;
}
const repo = `${context.repo.owner}/${context.repo.repo}`;
const prNumber = process.env.BENCH_PR;
const actor = process.env.BENCH_ACTOR;
const jobUrl = process.env.BENCH_JOB_URL ||
`${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;
// Load samply profile URLs (files exist when samply profiling was enabled)
const samplyUrls = {};
for (const run of ['baseline-1', 'baseline-2', 'feature-1', 'feature-2']) {
try {
const url = fs.readFileSync(
path.join(process.env.BENCH_WORK_DIR, run, 'samply-profile-url.txt'), 'utf8'
).trim();
if (url) samplyUrls[run] = url;
} catch {}
}
const slackUsers = loadSlackUsers(process.env.GITHUB_WORKSPACE || '.');
const actorSlackId = slackUsers[actor];
const { blocks, threadBlocks } = buildSuccessBlocks({ summary, prNumber, actor, actorSlackId, jobUrl, repo, samplyUrls });
const text = `Bench: ${summary.baseline.name} vs ${summary.feature.name}`;
async function sendWithThread(ch) {
const res = await postToSlack(token, ch, blocks, text, core);
if (res.ok && res.ts && threadBlocks.length > 0) {
for (const tb of threadBlocks) {
await postToSlack(token, ch, [tb], 'Wait time breakdown', core, res.ts);
}
}
}
// Post to public channel if any metric shows significant improvement or regression
const channel = process.env.SLACK_BENCH_CHANNEL;
let postedToChannel = false;
if (channel) {
const changes = summary.changes || {};
const hasImprovement = Object.values(changes).some(c => c.sig === 'good');
if (hasImprovement) {
await sendWithThread(channel);
postedToChannel = true;
} else {
core.info('No significant improvement, skipping public channel notification');
}
}
// DM the actor only when results were not posted to the public channel
if (!postedToChannel) {
if (actorSlackId) {
await sendWithThread(actorSlackId);
} else {
core.info(`No Slack user mapping for GitHub user '${actor}', skipping DM`);
}
} else {
core.info(`Results posted to channel, skipping DM to ${actor}`);
}
}
async function failure({ core, context, failedStep }) {
const token = process.env.SLACK_BENCH_BOT_TOKEN;
if (!token) {
core.info('SLACK_BENCH_BOT_TOKEN not set, skipping Slack notification');
return;
}
const repo = `${context.repo.owner}/${context.repo.repo}`;
const prNumber = process.env.BENCH_PR;
const actor = process.env.BENCH_ACTOR;
const jobUrl = process.env.BENCH_JOB_URL ||
`${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;
const slackUsers = loadSlackUsers(process.env.GITHUB_WORKSPACE || '.');
const actorSlackId = slackUsers[actor];
const blocks = buildFailureBlocks({ prNumber, actor, actorSlackId, jobUrl, repo, failedStep });
const text = `Bench failed while ${failedStep}`;
// Always DM the actor
if (actorSlackId) {
await postToSlack(token, actorSlackId, blocks, text, core);
} else {
core.info(`No Slack user mapping for GitHub user '${actor}', skipping DM`);
}
// Only DM for failures, don't post to public channel
}
module.exports = { success, failure };

View File

@@ -1,24 +0,0 @@
{
"_comment": "Maps GitHub usernames to Slack user IDs. Find yours: Slack profile > ··· > Copy member ID.",
"shekhirin": "U09FAL2UMLJ",
"mattsse": "U09FQNPMRT3",
"klkvr": "U09FAK95FC2",
"joshieDo": "U09LHN6GYAU",
"mediocregopher": "U09FF75KMQU",
"yongkangc": "U09FB0ECTD4",
"gakonst": "U092SEPDM40",
"Rjected": "U09F6SCKRGT",
"DaniPopes": "U09FAT8EK2A",
"emmajam": "U0A34UN92HW",
"onbjerg": "U09FB0UK5AA",
"fgimenez": "U09G3GP7CSU",
"rakita": "U09FB3Z2M7Y",
"jxom": "U09F72MG083",
"tmm": "U0AD0U8E88N",
"pepyakin": "U0A7HKMGEHJ",
"grandizzy": "U09F8DBDDRT",
"SuperFluffy": "U095BKHB2Q4",
"kamsz": "U0A2563UBRD",
"zerosnacks": "U09FARPMN74",
"samczsun": "U096R14E4H3"
}

View File

@@ -1,27 +0,0 @@
// Updates the reth-bench PR comment with current status.
//
// Reads from environment:
// BENCH_COMMENT_ID GitHub comment ID to update
// BENCH_JOB_URL URL to the Actions job page
// BENCH_CONFIG Config line (blocks, warmup, refs)
// BENCH_ACTOR User who triggered the benchmark
//
// Usage from actions/github-script:
// const s = require('./.github/scripts/bench-update-status.js');
// await s({github, context, status: 'Building baseline binary...'});
function buildBody(status) {
return `cc @${process.env.BENCH_ACTOR}\n\n🚀 Benchmark started! [View job](${process.env.BENCH_JOB_URL})\n\n⏳ **Status:** ${status}\n\n${process.env.BENCH_CONFIG}`;
}
async function updateStatus({ github, context, status }) {
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: parseInt(process.env.BENCH_COMMENT_ID),
body: buildBody(status),
});
}
updateStatus.buildBody = buildBody;
module.exports = updateStatus;

View File

@@ -1,414 +0,0 @@
#!/usr/bin/env bash
#
# Full PGO+BOLT optimized build for reth using real reth-bench workloads.
#
# Phases:
# 1. Build PGO-instrumented reth, run reth-bench → collect PGO profiles
# 2. Build BOLT-instrumented reth (with PGO), run reth-bench → collect BOLT profiles
# 3. Build final PGO+BOLT optimized binary
#
# Required environment variables:
# DATADIR - Path to reth datadir (must already contain chain data)
# RPC_URL - Source RPC URL for reth-bench to fetch payloads from
#
# Optional environment variables:
# PGO_BLOCKS - Number of blocks for PGO profiling (default: 20)
# BOLT_BLOCKS - Number of blocks for BOLT profiling (default: 20)
# SKIP_BOLT - Temporarily skip BOLT phases (default: false)
# STRIP_SYMBOLS - Strip debug symbols from output binary (default: true)
# COLLECT_PGO_ONLY - Stop after producing merged.profdata (default: false)
# PGO_PROFDATA - Path to pre-collected merged.profdata (optional)
# PROFILE - Cargo profile (default: maxperf-symbols)
# FEATURES - Cargo features (default: jemalloc,asm-keccak,min-debug-logs)
# TARGET - Target triple (default: auto-detected)
# EXTRA_RUSTFLAGS - Additional RUSTFLAGS (e.g. -C target-cpu=x86-64-v3)
#
# Output:
# target/$PROFILE_DIR/reth — final optimized binary
set -euo pipefail
gha_section_start() {
local title="$1"
if [ -n "${GITHUB_ACTIONS:-}" ]; then
echo "::group::$title"
else
echo ""
echo "=== $title ==="
fi
}
gha_section_end() {
if [ -n "${GITHUB_ACTIONS:-}" ]; then
echo "::endgroup::"
fi
}
cd "$(dirname "$0")/../.."
# ── Configuration ──────────────────────────────────────────────────────────────
PGO_BLOCKS="${PGO_BLOCKS:-20}"
BOLT_BLOCKS="${BOLT_BLOCKS:-20}"
SKIP_BOLT="${SKIP_BOLT:-false}"
STRIP_SYMBOLS="${STRIP_SYMBOLS:-true}"
COLLECT_PGO_ONLY="${COLLECT_PGO_ONLY:-false}"
PROFILE="${PROFILE:-maxperf-symbols}"
FEATURES="${FEATURES:-jemalloc,asm-keccak,min-debug-logs}"
TARGET="${TARGET:-$(rustc -Vv | grep host | cut -d' ' -f2)}"
BASE_RUSTFLAGS="${RUSTFLAGS:-}"
EXTRA_RUSTFLAGS="${EXTRA_RUSTFLAGS:-}"
COMBINED_RUSTFLAGS="$BASE_RUSTFLAGS $EXTRA_RUSTFLAGS"
PGO_PROFDATA="${PGO_PROFDATA:-}"
DATADIR="${DATADIR:-}"
RPC_URL="${RPC_URL:-}"
SKIP_BOLT_BOOL=false
if [[ "${SKIP_BOLT,,}" == "true" || "$SKIP_BOLT" == "1" ]]; then
SKIP_BOLT_BOOL=true
fi
STRIP_SYMBOLS_BOOL=false
if [[ "${STRIP_SYMBOLS,,}" == "true" || "$STRIP_SYMBOLS" == "1" ]]; then
STRIP_SYMBOLS_BOOL=true
fi
COLLECT_PGO_ONLY_BOOL=false
if [[ "${COLLECT_PGO_ONLY,,}" == "true" || "$COLLECT_PGO_ONLY" == "1" ]]; then
COLLECT_PGO_ONLY_BOOL=true
fi
USE_PRECOLLECTED_PGO=false
if [ -n "$PGO_PROFDATA" ]; then
if [ ! -f "$PGO_PROFDATA" ]; then
echo "error: PGO_PROFDATA points to a missing file: $PGO_PROFDATA"
exit 1
fi
USE_PRECOLLECTED_PGO=true
fi
NEEDS_BENCH_WORKLOAD=true
if [ "$USE_PRECOLLECTED_PGO" = true ] && [ "$SKIP_BOLT_BOOL" = true ]; then
NEEDS_BENCH_WORKLOAD=false
fi
if [ "$NEEDS_BENCH_WORKLOAD" = true ]; then
: "${DATADIR:?DATADIR must be set to the reth data directory}"
: "${RPC_URL:?RPC_URL must be set}"
fi
if [[ "$PROFILE" == dev ]]; then
PROFILE_DIR=debug
else
PROFILE_DIR=$PROFILE
fi
MANIFEST_PATH="bin/reth"
LLVM_VERSION=$(rustc -Vv | grep -oP 'LLVM version: \K\d+')
PGO_DIR="$PWD/target/pgo-profiles"
BOLT_DIR="$PWD/target/bolt-profiles"
CARGO_ARGS=(--profile "$PROFILE" --features "$FEATURES" --manifest-path "$MANIFEST_PATH/Cargo.toml" --bin "reth" --locked)
# Enable debug symbols for BOLT (requires symbols to reorder code).
# Strip them at the end.
PROFILE_UPPER=$(echo "$PROFILE" | tr '[:lower:]-' '[:upper:]_')
export "CARGO_PROFILE_${PROFILE_UPPER}_STRIP=debuginfo"
gha_section_start "Full PGO+BOLT Build"
echo "Binary: reth"
echo "Manifest: $MANIFEST_PATH"
echo "Target: $TARGET"
echo "Profile: $PROFILE"
echo "Features: $FEATURES"
echo "LLVM: $LLVM_VERSION"
echo "PGO blocks: $PGO_BLOCKS"
echo "BOLT blocks: $BOLT_BLOCKS"
echo "Skip BOLT: $SKIP_BOLT"
echo "Strip symbols: $STRIP_SYMBOLS"
echo "Collect only: $COLLECT_PGO_ONLY"
echo "PGO profdata: ${PGO_PROFDATA:-<collect with reth-bench>}"
echo "RUSTFLAGS: ${BASE_RUSTFLAGS:-<unset>}"
echo "EXTRA_RUSTFLAGS: ${EXTRA_RUSTFLAGS:-<unset>}"
if [ "$NEEDS_BENCH_WORKLOAD" = true ]; then
echo "Datadir: $DATADIR"
echo "RPC URL: $RPC_URL"
else
echo "Datadir: <not required>"
echo "RPC URL: <not required>"
fi
gha_section_end
# ── Prerequisites ──────────────────────────────────────────────────────────────
gha_section_start "Installing prerequisites"
rustup component add llvm-tools-preview
LLVM_PROFDATA=$(find "$(rustc --print sysroot)" -name llvm-profdata -type f | head -1)
if [ -z "$LLVM_PROFDATA" ]; then
echo "error: llvm-profdata not found"
exit 1
fi
install_bolt() {
if command -v llvm-bolt &>/dev/null; then
echo "BOLT already installed"
return
fi
echo "Installing BOLT from apt.llvm.org..."
wget -qO- https://apt.llvm.org/llvm-snapshot.gpg.key | sudo tee /etc/apt/trusted.gpg.d/apt.llvm.org.asc >/dev/null
CODENAME=$(lsb_release -cs)
echo "deb http://apt.llvm.org/$CODENAME/ llvm-toolchain-$CODENAME-$LLVM_VERSION main" | sudo tee /etc/apt/sources.list.d/llvm.list >/dev/null
sudo apt-get update -qq
sudo apt-get install -y -qq "bolt-$LLVM_VERSION"
sudo ln -sf "/usr/bin/llvm-bolt-$LLVM_VERSION" /usr/local/bin/llvm-bolt
sudo ln -sf "/usr/bin/merge-fdata-$LLVM_VERSION" /usr/local/bin/merge-fdata
}
if [ "$SKIP_BOLT_BOOL" = true ]; then
echo "Skipping BOLT installation (SKIP_BOLT=$SKIP_BOLT)"
else
install_bolt
fi
gha_section_end
if [ "$NEEDS_BENCH_WORKLOAD" = true ]; then
# Build reth-bench once (non-instrumented) — reused for both phases.
gha_section_start "Building reth-bench"
RUSTFLAGS="$COMBINED_RUSTFLAGS" \
cargo build --profile "$PROFILE" --features "$FEATURES" \
--manifest-path bin/reth-bench/Cargo.toml --bin reth-bench --locked
RETH_BENCH_BIN="$(find target -name reth-bench -type f -executable | head -1)"
echo "reth-bench: $RETH_BENCH_BIN"
gha_section_end
else
gha_section_start "Building reth-bench"
echo "Skipping reth-bench build (pre-collected PGO with SKIP_BOLT=true)"
gha_section_end
fi
# ── Helpers ────────────────────────────────────────────────────────────────────
RETH_PID=
cleanup() {
if [ -n "${RETH_PID:-}" ] && kill -0 "$RETH_PID" 2>/dev/null; then
echo "Stopping reth (pid $RETH_PID)..."
sudo kill "$RETH_PID" 2>/dev/null || true
for i in $(seq 1 60); do
sudo kill -0 "$RETH_PID" 2>/dev/null || break
if [ $((i % 10)) -eq 0 ]; then
echo " waiting... (${i}s)"
fi
sleep 1
done
sudo kill -9 "$RETH_PID" 2>/dev/null || true
fi
}
trap cleanup EXIT
# Start reth, wait for RPC, run reth-bench, then stop reth.
# Arguments: $1 = reth binary path, $2 = number of blocks, $3 = log label
run_bench_workload() {
local reth_bin="$1" blocks="$2" label="$3"
local http_port=8545 authrpc_port=8551
echo "--- Starting reth ($label) ---"
sudo "$reth_bin" node \
--datadir "$DATADIR" \
--log.file.directory "/tmp/reth-${label}-logs" \
--engine.accept-execution-requests-hash \
--http --http.port "$http_port" \
--authrpc.port "$authrpc_port" \
--disable-discovery --no-persist-peers \
> "/tmp/reth-${label}.log" 2>&1 &
RETH_PID=$!
echo "Waiting for reth RPC..."
for i in $(seq 1 120); do
if curl -sf "http://127.0.0.1:$http_port" -X POST \
-H 'Content-Type: application/json' \
-d '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}' \
> /dev/null 2>&1; then
echo "reth is ready after ${i}s"
break
fi
if [ "$i" -eq 120 ]; then
echo "error: reth failed to start within 120s"
cat "/tmp/reth-${label}.log"
exit 1
fi
sleep 1
done
echo "Running reth-bench ($blocks blocks)..."
"$RETH_BENCH_BIN" new-payload-fcu \
--rpc-url "$RPC_URL" \
--engine-rpc-url "http://127.0.0.1:$authrpc_port" \
--jwt-secret "$DATADIR/jwt.hex" \
--advance "$blocks" \
--reth-new-payload 2>&1 | sed -u "s/^/[$label] /"
echo "Stopping reth ($label)..."
sudo kill "$RETH_PID" 2>/dev/null || true
for i in $(seq 1 60); do
sudo kill -0 "$RETH_PID" 2>/dev/null || break
sleep 1
done
sudo kill -9 "$RETH_PID" 2>/dev/null || true
RETH_PID=
}
publish_binary() {
local source_bin="$1"
for out in "target/$TARGET/$PROFILE_DIR" "target/$PROFILE_DIR"; do
local destination="$out/reth"
mkdir -p "$out"
# Skip copying when source and destination resolve to the same inode.
if [ -e "$destination" ] && [ "$source_bin" -ef "$destination" ]; then
continue
fi
cp "$source_bin" "$destination"
done
}
if [ "$USE_PRECOLLECTED_PGO" = true ]; then
gha_section_start "Phase 1: Using Pre-Collected PGO Profile"
rm -rf "$PGO_DIR"
mkdir -p "$PGO_DIR"
cp "$PGO_PROFDATA" "$PGO_DIR/merged.profdata"
echo "Using pre-collected profile: $PGO_PROFDATA"
echo "PGO profile: $PGO_DIR/merged.profdata ($(ls -lh "$PGO_DIR/merged.profdata" | awk '{print $5}'))"
gha_section_end
else
# ── Phase 1: PGO profile collection ───────────────────────────────────────
gha_section_start "Phase 1: PGO Profile Collection"
rm -rf "$PGO_DIR"
mkdir -p "$PGO_DIR"
echo "Building PGO-instrumented binary..."
RUSTFLAGS="-Cprofile-generate=$PGO_DIR -Crelocation-model=pic $COMBINED_RUSTFLAGS" \
cargo build "${CARGO_ARGS[@]}" --target "$TARGET"
PGO_RETH_BIN="$PWD/target/$TARGET/$PROFILE_DIR/reth"
echo "Instrumented binary: $PGO_RETH_BIN ($(ls -lh "$PGO_RETH_BIN" | awk '{print $5}'))"
run_bench_workload "$PGO_RETH_BIN" "$PGO_BLOCKS" "pgo"
# Fix ownership if reth ran as root.
sudo chown -R "$(id -un):$(id -gn)" "$PGO_DIR" 2>/dev/null || true
# Merge PGO profiles.
echo "Merging PGO profiles..."
PROFRAW_COUNT=$(find "$PGO_DIR" -name '*.profraw' | wc -l)
echo "Found $PROFRAW_COUNT .profraw files"
if [ "$PROFRAW_COUNT" -eq 0 ]; then
echo "error: no .profraw files — instrumented binary did not produce profiles"
exit 1
fi
"$LLVM_PROFDATA" merge -o "$PGO_DIR/merged.profdata" "$PGO_DIR"/*.profraw
echo "PGO profile: $PGO_DIR/merged.profdata ($(ls -lh "$PGO_DIR/merged.profdata" | awk '{print $5}'))"
gha_section_end
fi
if [ "$COLLECT_PGO_ONLY_BOOL" = true ]; then
gha_section_start "PGO Collection Complete"
echo "COLLECT_PGO_ONLY=true, skipping PGO/BOLT optimized binary build"
echo "Profile: $PGO_DIR/merged.profdata"
gha_section_end
exit 0
fi
if [ "$SKIP_BOLT_BOOL" = true ]; then
gha_section_start "BOLT Phase Skipped"
echo "SKIP_BOLT=$SKIP_BOLT, building PGO-only binary"
echo "Building PGO-optimized binary..."
RUSTFLAGS="-Cprofile-use=$PGO_DIR/merged.profdata $COMBINED_RUSTFLAGS" \
cargo build "${CARGO_ARGS[@]}" --target "$TARGET"
BUILT_BIN="$PWD/target/$TARGET/$PROFILE_DIR/reth"
if [ "$STRIP_SYMBOLS_BOOL" = true ]; then
echo "Stripping debug symbols..."
strip "$BUILT_BIN"
else
echo "Skipping strip (STRIP_SYMBOLS=$STRIP_SYMBOLS)"
fi
publish_binary "$BUILT_BIN"
gha_section_end
else
# ── Phase 2: BOLT profile collection (with PGO) ──────────────────────────
gha_section_start "Phase 2: BOLT Profile Collection (with PGO)"
rm -rf "$BOLT_DIR"
mkdir -p "$BOLT_DIR"
echo "Building BOLT-instrumented binary with PGO..."
# --emit-relocs preserves relocation entries in the binary, required by llvm-bolt -instrument
RUSTFLAGS="-Cprofile-use=$PGO_DIR/merged.profdata -Clink-arg=-Wl,--emit-relocs $COMBINED_RUSTFLAGS" \
cargo build "${CARGO_ARGS[@]}" --target "$TARGET"
# Instrument with BOLT
BUILT_BIN="$PWD/target/$TARGET/$PROFILE_DIR/reth"
BOLT_INSTRUMENTED_BIN="$BUILT_BIN-bolt-instrumented"
echo "Instrumenting binary with BOLT..."
# --skip-funcs: skip compiler-generated drop_in_place functions that BOLT can't handle
# as split functions in relocation mode (triggered by --emit-relocs)
llvm-bolt "$BUILT_BIN" \
-instrument \
--instrumentation-file-append-pid \
--instrumentation-file="$BOLT_DIR/prof" \
--skip-funcs='.*drop_in_place.*' \
-o "$BOLT_INSTRUMENTED_BIN"
echo "BOLT-instrumented binary: $BOLT_INSTRUMENTED_BIN ($(ls -lh "$BOLT_INSTRUMENTED_BIN" | awk '{print $5}'))"
run_bench_workload "$BOLT_INSTRUMENTED_BIN" "$BOLT_BLOCKS" "bolt"
# Fix ownership for BOLT profiles
sudo chown -R "$(id -un):$(id -gn)" "$BOLT_DIR" 2>/dev/null || true
# Merge BOLT profiles
echo "Merging BOLT profiles..."
FDATA_COUNT=$(find "$BOLT_DIR" -name '*.fdata' | wc -l)
echo "Found $FDATA_COUNT .fdata files"
if [ "$FDATA_COUNT" -eq 0 ]; then
echo "error: no .fdata files — BOLT-instrumented binary did not produce profiles"
exit 1
fi
merge-fdata "$BOLT_DIR"/*.fdata > "$BOLT_DIR/merged.fdata"
echo "BOLT profile: $BOLT_DIR/merged.fdata ($(ls -lh "$BOLT_DIR/merged.fdata" | awk '{print $5}'))"
gha_section_end
# ── Phase 3: Final optimized build ───────────────────────────────────────
gha_section_start "Phase 3: Final PGO+BOLT Optimized Build"
echo "Building PGO-optimized binary..."
# --emit-relocs preserves relocation entries in the binary, required by llvm-bolt for code reordering
RUSTFLAGS="-Cprofile-use=$PGO_DIR/merged.profdata -Clink-arg=-Wl,--emit-relocs $COMBINED_RUSTFLAGS" \
cargo build "${CARGO_ARGS[@]}" --target "$TARGET"
BUILT_BIN="$PWD/target/$TARGET/$PROFILE_DIR/reth"
OPTIMIZED_BIN="$BUILT_BIN-bolt-optimized"
echo "Optimizing with BOLT..."
llvm-bolt "$BUILT_BIN" \
-o "$OPTIMIZED_BIN" \
--data "$BOLT_DIR/merged.fdata" \
-reorder-blocks=ext-tsp \
-reorder-functions=cdsort \
-split-functions \
-split-all-cold \
-dyno-stats \
-icf=1 \
-use-gnu-stack \
--skip-funcs='.*drop_in_place.*'
if [ "$STRIP_SYMBOLS_BOOL" = true ]; then
echo "Stripping debug symbols..."
strip "$OPTIMIZED_BIN"
else
echo "Skipping strip (STRIP_SYMBOLS=$STRIP_SYMBOLS)"
fi
publish_binary "$OPTIMIZED_BIN"
gha_section_end
fi
gha_section_start "Build Complete"
ls -lh "target/$PROFILE_DIR/reth"
echo "Output: target/$PROFILE_DIR/reth"
gha_section_end

View File

@@ -16,11 +16,28 @@ rpc-compat:
# syncing mode, the test expects syncing to be false on start
- eth_syncing/check-syncing (reth)
engine-withdrawals: [ ]
# no fix due to https://github.com/paradigmxyz/reth/issues/8732
engine-withdrawals:
- Withdrawals Fork On Genesis (Paris) (reth)
- Withdrawals Fork on Block 1 (Paris) (reth)
- Withdrawals Fork on Block 2 (Paris) (reth)
- Withdrawals Fork on Block 3 (Paris) (reth)
- Withdraw to a single account (Paris) (reth)
- Withdraw to two accounts (Paris) (reth)
- Withdraw many accounts (Paris) (reth)
- Withdraw zero amount (Paris) (reth)
- Empty Withdrawals (Paris) (reth)
- Corrupted Block Hash Payload (INVALID) (Paris) (reth)
- Withdrawals Fork on Canonical Block 8 / Side Block 7 - 10 Block Re-Org (Paris) (reth)
engine-api: [ ]
engine-cancun: [ ]
# no fix due to https://github.com/paradigmxyz/reth/issues/8732
engine-cancun:
- Invalid PayloadAttributes, Missing BeaconRoot, Syncing=True (Cancun) (reth)
# the test fails with older versions of the code for which it passed before, probably related to changes
# in hive or its dependencies
- Blob Transaction Ordering, Multiple Clients (Cancun) (reth)
sync: [ ]
@@ -32,7 +49,7 @@ engine-auth: [ ]
# The test artificially creates an empty account with storage, then tests EIP-7610's behavior.
# On mainnet, ~25 such accounts exist as contract addresses (derived from keccak(prefix, caller,
# nonce/salt), not from public keys). No private key exists for contract addresses. To trigger
# this with EIP-7702, you'd need to recover a private key from one of the already deployed contract addresses - mathematically impossible.
# this with EIP-7702, you'd need to recover a private key from one of the already deployed contract addresses - mathematically impossible.
#
# tests/paris/eip7610_create_collision/test_initcollision.py::test_init_collision_*
# Requires hash collision on create2 address to target already deployed accounts with storage.
@@ -42,6 +59,10 @@ engine-auth: [ ]
#
# System contract tests (already fixed and deployed):
#
# tests/prague/eip6110_deposits/test_modified_contract.py::test_invalid_layout and test_invalid_log_length
# System contract is already fixed and deployed; tests cover scenarios where contract is
# malformed which can't happen retroactively. No point in adding checks.
#
# tests/prague/eip7002_el_triggerable_withdrawals/test_contract_deployment.py::test_system_contract_deployment
# tests/prague/eip7251_consolidations/test_contract_deployment.py::test_system_contract_deployment
# Post-fork system contract deployment tests. Should fix for spec compliance but not realistic
@@ -50,8 +71,32 @@ eels/consume-engine:
- tests/prague/eip7702_set_code_tx/test_set_code_txs.py::test_set_code_to_non_empty_storage[fork_Prague-blockchain_test_engine-zero_nonce]-reth
- tests/prague/eip7251_consolidations/test_contract_deployment.py::test_system_contract_deployment[fork_CancunToPragueAtTime15k-blockchain_test_engine-deploy_after_fork-nonzero_balance]-reth
- tests/prague/eip7251_consolidations/test_contract_deployment.py::test_system_contract_deployment[fork_CancunToPragueAtTime15k-blockchain_test_engine-deploy_after_fork-zero_balance]-reth
- tests/prague/eip6110_deposits/test_modified_contract.py::test_invalid_layout[fork_Prague-blockchain_test_engine-log_argument_amount_offset-value_zero]-reth
- tests/prague/eip6110_deposits/test_modified_contract.py::test_invalid_layout[fork_Prague-blockchain_test_engine-log_argument_amount_size-value_zero]-reth
- tests/prague/eip6110_deposits/test_modified_contract.py::test_invalid_layout[fork_Prague-blockchain_test_engine-log_argument_index_offset-value_zero]-reth
- tests/prague/eip6110_deposits/test_modified_contract.py::test_invalid_layout[fork_Prague-blockchain_test_engine-log_argument_index_size-value_zero]-reth
- tests/prague/eip6110_deposits/test_modified_contract.py::test_invalid_layout[fork_Prague-blockchain_test_engine-log_argument_pubkey_offset-value_zero]-reth
- tests/prague/eip6110_deposits/test_modified_contract.py::test_invalid_layout[fork_Prague-blockchain_test_engine-log_argument_pubkey_size-value_zero]-reth
- tests/prague/eip6110_deposits/test_modified_contract.py::test_invalid_layout[fork_Prague-blockchain_test_engine-log_argument_signature_offset-value_zero]-reth
- tests/prague/eip6110_deposits/test_modified_contract.py::test_invalid_layout[fork_Prague-blockchain_test_engine-log_argument_signature_size-value_zero]-reth
- tests/prague/eip6110_deposits/test_modified_contract.py::test_invalid_layout[fork_Prague-blockchain_test_engine-log_argument_withdrawal_credentials_offset-value_zero]-reth
- tests/prague/eip6110_deposits/test_modified_contract.py::test_invalid_layout[fork_Prague-blockchain_test_engine-log_argument_withdrawal_credentials_size-value_zero]-reth
- tests/prague/eip7002_el_triggerable_withdrawals/test_contract_deployment.py::test_system_contract_deployment[fork_CancunToPragueAtTime15k-blockchain_test_engine-deploy_after_fork-nonzero_balance]-reth
- tests/prague/eip7002_el_triggerable_withdrawals/test_contract_deployment.py::test_system_contract_deployment[fork_CancunToPragueAtTime15k-blockchain_test_engine-deploy_after_fork-zero_balance]-reth
- tests/prague/eip6110_deposits/test_modified_contract.py::test_invalid_log_length[fork_Prague-blockchain_test_engine-slice_bytes_False]-reth
- tests/prague/eip6110_deposits/test_modified_contract.py::test_invalid_log_length[fork_Prague-blockchain_test_engine-slice_bytes_True]-reth
- tests/prague/eip6110_deposits/test_modified_contract.py::test_invalid_layout[fork_Osaka-blockchain_test_engine-log_argument_amount_offset-value_zero]-reth
- tests/prague/eip6110_deposits/test_modified_contract.py::test_invalid_layout[fork_Osaka-blockchain_test_engine-log_argument_amount_size-value_zero]-reth
- tests/prague/eip6110_deposits/test_modified_contract.py::test_invalid_layout[fork_Osaka-blockchain_test_engine-log_argument_index_offset-value_zero]-reth
- tests/prague/eip6110_deposits/test_modified_contract.py::test_invalid_layout[fork_Osaka-blockchain_test_engine-log_argument_index_size-value_zero]-reth
- tests/prague/eip6110_deposits/test_modified_contract.py::test_invalid_layout[fork_Osaka-blockchain_test_engine-log_argument_pubkey_offset-value_zero]-reth
- tests/prague/eip6110_deposits/test_modified_contract.py::test_invalid_layout[fork_Osaka-blockchain_test_engine-log_argument_pubkey_size-value_zero]-reth
- tests/prague/eip6110_deposits/test_modified_contract.py::test_invalid_layout[fork_Osaka-blockchain_test_engine-log_argument_signature_offset-value_zero]-reth
- tests/prague/eip6110_deposits/test_modified_contract.py::test_invalid_layout[fork_Osaka-blockchain_test_engine-log_argument_signature_size-value_zero]-reth
- tests/prague/eip6110_deposits/test_modified_contract.py::test_invalid_layout[fork_Osaka-blockchain_test_engine-log_argument_withdrawal_credentials_offset-value_zero]-reth
- tests/prague/eip6110_deposits/test_modified_contract.py::test_invalid_layout[fork_Osaka-blockchain_test_engine-log_argument_withdrawal_credentials_size-value_zero]-reth
- tests/prague/eip6110_deposits/test_modified_contract.py::test_invalid_log_length[fork_Osaka-blockchain_test_engine-slice_bytes_False]-reth
- tests/prague/eip6110_deposits/test_modified_contract.py::test_invalid_log_length[fork_Osaka-blockchain_test_engine-slice_bytes_True]-reth
- tests/paris/eip7610_create_collision/test_initcollision.py::test_init_collision_create_tx[fork_Osaka-tx_type_0-blockchain_test_engine_from_state_test-non-empty-balance-revert-initcode]-reth
- tests/paris/eip7610_create_collision/test_initcollision.py::test_init_collision_create_tx[fork_Prague-tx_type_0-blockchain_test_engine_from_state_test-non-empty-balance-correct-initcode]-reth
- tests/paris/eip7610_create_collision/test_initcollision.py::test_init_collision_create_tx[fork_Paris-tx_type_1-blockchain_test_engine_from_state_test-non-empty-balance-correct-initcode]-reth

View File

@@ -11,16 +11,17 @@
#
# When a test should no longer be ignored, remove it from this list.
# flaky
engine-withdrawals:
- Withdrawals Fork on Block 1 - 8 Block Re-Org NewPayload (Paris) (reth)
- Withdrawals Fork on Block 8 - 10 Block Re-Org NewPayload (Paris) (reth)
- Withdrawals Fork on Canonical Block 8 / Side Block 7 - 10 Block Re-Org (Paris) (reth)
- Sync after 128 blocks - Withdrawals on Block 2 - Multiple Withdrawal Accounts (Paris) (reth)
engine-cancun:
- Transaction Re-Org, New Payload on Revert Back (Cancun) (reth)
- Transaction Re-Org, Re-Org to Different Block (Cancun) (reth)
- Transaction Re-Org, Re-Org Out (Cancun) (reth)
- Invalid Missing Ancestor ReOrg, StateRoot, EmptyTxs=False, Invalid P9 (Cancun) (reth)
# Hive test infra bug: geth sidecar switched to PathScheme for state storage, which has
# strict trie integrity requirements incompatible with inserting intentionally invalid blocks.
# Affects all clients, not just reth. Tracked: https://github.com/ethereum/hive/issues/1382
- Invalid Missing Ancestor Syncing ReOrg, Timestamp, EmptyTxs=False, CanonicalReOrg=False, Invalid P8 (Cancun) (reth)
- Invalid Missing Ancestor Syncing ReOrg, Timestamp, EmptyTxs=False, CanonicalReOrg=True, Invalid P8 (Cancun) (reth)
- Multiple New Payloads Extending Canonical Chain, Wait for Canonical Payload (Cancun) (reth)
engine-api:
- Transaction Re-Org, Re-Org Out (Paris) (reth)

View File

@@ -6,14 +6,8 @@ cd hivetests/
sim="${1}"
limit="${2}"
# Use lower parallelism for eels tests to avoid OOM-killing the runner
parallelism=16
if [[ "${sim}" == *"eels"* ]]; then
parallelism=4
fi
run_hive() {
hive --sim "${sim}" --sim.limit "${limit}" --sim.parallelism "${parallelism}" --client reth 2>&1 | tee /tmp/log || true
hive --sim "${sim}" --sim.limit "${limit}" --sim.parallelism 16 --client reth 2>&1 | tee /tmp/log || true
}
check_log() {

File diff suppressed because it is too large Load Diff

View File

@@ -15,7 +15,7 @@ env:
jobs:
build:
runs-on: ${{ github.repository == 'paradigmxyz/reth' && 'depot-ubuntu-latest-8' || 'ubuntu-latest' }}
runs-on: depot-ubuntu-latest-8
timeout-minutes: 90
steps:
- name: Checkout

View File

@@ -25,7 +25,7 @@ env:
jobs:
check:
name: Check compilation with patched alloy
runs-on: ${{ github.repository == 'paradigmxyz/reth' && 'depot-ubuntu-latest-16' || 'ubuntu-latest' }}
runs-on: depot-ubuntu-latest-16
timeout-minutes: 60
steps:
- uses: actions/checkout@v4

View File

@@ -18,7 +18,7 @@ env:
name: compact-codec
jobs:
compact-codec:
runs-on: ${{ github.repository == 'paradigmxyz/reth' && 'depot-ubuntu-latest' || 'ubuntu-latest' }}
runs-on: depot-ubuntu-latest
strategy:
matrix:
bin:

View File

@@ -15,6 +15,7 @@ on:
jobs:
build:
if: github.repository == 'paradigmxyz/reth'
timeout-minutes: 45
runs-on: ubuntu-latest
permissions:
@@ -30,22 +31,10 @@ jobs:
echo "sha=${{ github.sha }}" >> "$GITHUB_OUTPUT"
echo "describe=$(git describe --always --tags)" >> "$GITHUB_OUTPUT"
- name: Detect fork
id: fork
run: |
if [ "${{ github.event_name }}" = "pull_request" ] && [ "${{ github.event.pull_request.head.repo.full_name }}" != "${{ github.repository }}" ]; then
echo "is_fork=true" >> "$GITHUB_OUTPUT"
else
echo "is_fork=false" >> "$GITHUB_OUTPUT"
fi
# Depot build (upstream only)
- name: Set up Depot CLI
if: steps.fork.outputs.is_fork == 'false'
uses: depot/setup-action@v1
- name: Build reth image (Depot)
if: steps.fork.outputs.is_fork == 'false'
- name: Build reth image
uses: depot/bake-action@v1
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
@@ -57,26 +46,8 @@ jobs:
targets: ${{ inputs.hive_target }}
push: false
# Docker build (forks)
- name: Set up Docker Buildx
if: steps.fork.outputs.is_fork == 'true'
uses: docker/setup-buildx-action@v3
- name: Build reth image (Docker)
if: steps.fork.outputs.is_fork == 'true'
uses: docker/bake-action@v6
env:
VERGEN_GIT_SHA: ${{ steps.git.outputs.sha }}
VERGEN_GIT_DESCRIBE: ${{ steps.git.outputs.describe }}
with:
files: docker-bake.hcl
targets: ${{ inputs.hive_target }}
push: false
set: |
*.dockerfile=Dockerfile
- name: Upload reth image
uses: actions/upload-artifact@v7
uses: actions/upload-artifact@v6
with:
name: ${{ inputs.artifact_name }}
path: ./artifacts

View File

@@ -28,25 +28,12 @@ on:
required: false
type: boolean
default: false
pgo_blocks:
description: "Number of blocks to execute for PGO profiling"
required: false
type: string
default: "20"
jobs:
collect-pgo-profile:
if: github.repository == 'paradigmxyz/reth'
uses: ./.github/workflows/pgo-profile.yml
with:
pgo_blocks: ${{ github.event_name == 'workflow_dispatch' && inputs.pgo_blocks || '20' }}
secrets: inherit
build:
if: github.repository == 'paradigmxyz/reth'
name: Build Docker images
runs-on: ubuntu-24.04
needs: collect-pgo-profile
permissions:
packages: write
contents: read
@@ -58,7 +45,7 @@ jobs:
uses: depot/setup-action@v1
- name: Log in to GHCR
uses: docker/login-action@v4
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
@@ -71,23 +58,6 @@ jobs:
echo "describe=$(git describe --always --tags)" >> "$GITHUB_OUTPUT"
echo "dirty=false" >> "$GITHUB_OUTPUT"
- name: Download pre-collected PGO profile
uses: actions/download-artifact@v7
with:
name: pgo-profdata
path: dist
- name: Configure PGO build args
id: pgo
run: |
if [ ! -f dist/merged.profdata ]; then
echo "::error::Expected dist/merged.profdata from collect-pgo-profile job"
exit 1
fi
echo "use_pgo_bolt=true" >> "$GITHUB_OUTPUT"
echo "pgo_profdata=dist/merged.profdata" >> "$GITHUB_OUTPUT"
echo "Using pre-collected PGO profile from collect-pgo-profile job"
- name: Determine build parameters
id: params
run: |
@@ -100,27 +70,18 @@ jobs:
# Add 'latest' tag for non-RC releases
if [[ ! "$VERSION" =~ -rc ]]; then
echo "ethereum_tags=${REGISTRY}/reth:${VERSION},${REGISTRY}/reth:latest" >> "$GITHUB_OUTPUT"
{
echo "ethereum_set<<EOF"
echo "ethereum.tags=${REGISTRY}/reth:${VERSION}"
echo "ethereum.tags=${REGISTRY}/reth:latest"
echo "EOF"
} >> "$GITHUB_OUTPUT"
else
echo "ethereum_tags=${REGISTRY}/reth:${VERSION}" >> "$GITHUB_OUTPUT"
echo "ethereum_set=ethereum.tags=${REGISTRY}/reth:${VERSION}" >> "$GITHUB_OUTPUT"
fi
elif [[ "${{ github.event_name }}" == "schedule" ]] || [[ "${{ inputs.build_type }}" == "nightly" ]]; then
echo "targets=nightly" >> "$GITHUB_OUTPUT"
echo "ethereum_tags=${REGISTRY}/reth:nightly" >> "$GITHUB_OUTPUT"
echo "ethereum_set=ethereum.tags=${REGISTRY}/reth:nightly" >> "$GITHUB_OUTPUT"
else
# git-sha build
echo "targets=ethereum" >> "$GITHUB_OUTPUT"
echo "ethereum_tags=${REGISTRY}/reth:${{ github.sha }}" >> "$GITHUB_OUTPUT"
echo "ethereum_set=ethereum.tags=${REGISTRY}/reth:${{ github.sha }}" >> "$GITHUB_OUTPUT"
fi
- name: Build and push images
@@ -136,10 +97,7 @@ jobs:
targets: ${{ steps.params.outputs.targets }}
push: ${{ !(github.event_name == 'workflow_dispatch' && inputs.dry_run) }}
set: |
${{ steps.params.outputs.ethereum_set }}
*.args.USE_PGO_BOLT=${{ steps.pgo.outputs.use_pgo_bolt }}
*.args.PGO_PROFDATA=${{ steps.pgo.outputs.pgo_profdata }}
*.args.STRIP_SYMBOLS=false
ethereum.tags=${{ steps.params.outputs.ethereum_tags }}
- name: Verify image architectures
env:
@@ -159,18 +117,6 @@ jobs:
- name: Slack Webhook Action
uses: rtCamp/action-slack-notify@v2
env:
SLACK_COLOR: danger
SLACK_ICON_EMOJI: ":rotating_light:"
SLACK_USERNAME: "GitHub Actions"
SLACK_TITLE: ":rotating_light: Nightly Docker Build Failed"
SLACK_MESSAGE: |
The scheduled nightly Docker build failed.
*Commit:* `${{ github.sha }}`
*Branch:* `${{ github.ref_name }}`
*Run:* <https://github.com/paradigmxyz/reth/actions/runs/${{ github.run_id }}|View logs>
*Action required:* Re-run the workflow or investigate the build failure.
SLACK_FOOTER: "paradigmxyz/reth · docker.yml"
MSG_MINIMAL: true
SLACK_COLOR: ${{ job.status }}
SLACK_MESSAGE: "Failed run: https://github.com/paradigmxyz/reth/actions/runs/${{ github.run_id }}"
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK_URL }}

View File

@@ -20,7 +20,7 @@ concurrency:
jobs:
test:
name: e2e-testsuite
runs-on: ${{ github.repository == 'paradigmxyz/reth' && 'depot-ubuntu-latest-4' || 'ubuntu-latest' }}
runs-on: depot-ubuntu-latest-4
env:
RUST_BACKTRACE: 1
timeout-minutes: 90
@@ -47,7 +47,7 @@ jobs:
rocksdb:
name: e2e-rocksdb
runs-on: ${{ github.repository == 'paradigmxyz/reth' && 'depot-ubuntu-latest-4' || 'ubuntu-latest' }}
runs-on: depot-ubuntu-latest-4
env:
RUST_BACKTRACE: 1
timeout-minutes: 60

View File

@@ -32,7 +32,7 @@ jobs:
prepare-hive:
if: github.repository == 'paradigmxyz/reth'
timeout-minutes: 45
runs-on: ${{ github.repository == 'paradigmxyz/reth' && 'depot-ubuntu-latest-4' || 'ubuntu-latest' }}
runs-on: depot-ubuntu-latest-4
steps:
- uses: actions/checkout@v6
- name: Checkout hive tests
@@ -75,7 +75,7 @@ jobs:
chmod +x hive
- name: Upload hive assets
uses: actions/upload-artifact@v7
uses: actions/upload-artifact@v6
with:
name: hive_assets
path: ./hive_assets
@@ -188,8 +188,7 @@ jobs:
- build-reth-edge
- prepare-hive
name: ${{ matrix.storage }} / ${{ matrix.scenario.sim }}${{ matrix.scenario.limit && format(' - {0}', matrix.scenario.limit) }}
# Use larger runners for eels tests to avoid OOM runner crashes
runs-on: ${{ github.repository == 'paradigmxyz/reth' && (contains(matrix.scenario.sim, 'eels') && 'depot-ubuntu-latest-8' || 'depot-ubuntu-latest-4') || 'ubuntu-latest' }}
runs-on: depot-ubuntu-latest-4
permissions:
issues: write
steps:
@@ -198,13 +197,13 @@ jobs:
fetch-depth: 0
- name: Download hive assets
uses: actions/download-artifact@v8
uses: actions/download-artifact@v7
with:
name: hive_assets
path: /tmp
- name: Download reth image
uses: actions/download-artifact@v8
uses: actions/download-artifact@v7
with:
name: reth-${{ matrix.storage }}
path: /tmp

View File

@@ -24,7 +24,7 @@ jobs:
test:
name: test / ${{ matrix.network }} / ${{ matrix.storage }}
if: github.event_name != 'schedule'
runs-on: ${{ github.repository == 'paradigmxyz/reth' && 'depot-ubuntu-latest-4' || 'ubuntu-latest' }}
runs-on: depot-ubuntu-latest-4
env:
RUST_BACKTRACE: 1
strategy:

View File

@@ -31,7 +31,7 @@ jobs:
strategy:
fail-fast: false
name: run kurtosis
runs-on: ${{ github.repository == 'paradigmxyz/reth' && 'depot-ubuntu-latest' || 'ubuntu-latest' }}
runs-on: depot-ubuntu-latest
needs:
- build-reth
steps:
@@ -40,7 +40,7 @@ jobs:
fetch-depth: 0
- name: Download reth image
uses: actions/download-artifact@v8
uses: actions/download-artifact@v7
with:
name: artifacts
path: /tmp

View File

@@ -13,7 +13,7 @@ env:
jobs:
clippy-binaries:
name: clippy binaries / ${{ matrix.type }}
runs-on: ${{ github.repository == 'paradigmxyz/reth' && 'depot-ubuntu-latest' || 'ubuntu-latest' }}
runs-on: depot-ubuntu-latest
timeout-minutes: 30
strategy:
matrix:
@@ -42,7 +42,7 @@ jobs:
clippy:
name: clippy
runs-on: ${{ github.repository == 'paradigmxyz/reth' && 'depot-ubuntu-latest' || 'ubuntu-latest' }}
runs-on: depot-ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v6
@@ -59,7 +59,7 @@ jobs:
RUSTFLAGS: -D warnings
wasm:
runs-on: ${{ github.repository == 'paradigmxyz/reth' && 'depot-ubuntu-latest' || 'ubuntu-latest' }}
runs-on: depot-ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v6
@@ -79,7 +79,7 @@ jobs:
.github/scripts/check_wasm.sh
riscv:
runs-on: ${{ github.repository == 'paradigmxyz/reth' && 'depot-ubuntu-latest' || 'ubuntu-latest' }}
runs-on: depot-ubuntu-latest
timeout-minutes: 60
steps:
- uses: actions/checkout@v6
@@ -98,7 +98,7 @@ jobs:
crate-checks:
name: crate-checks (${{ matrix.partition }}/${{ matrix.total_partitions }})
runs-on: ${{ github.repository == 'paradigmxyz/reth' && 'depot-ubuntu-latest-4' || 'ubuntu-latest' }}
runs-on: depot-ubuntu-latest-4
strategy:
matrix:
partition: [1, 2, 3]
@@ -117,14 +117,14 @@ jobs:
msrv:
name: MSRV
runs-on: ${{ github.repository == 'paradigmxyz/reth' && 'depot-ubuntu-latest' || 'ubuntu-latest' }}
runs-on: depot-ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v6
- uses: rui314/setup-mold@v1
- uses: dtolnay/rust-toolchain@master
with:
toolchain: "1.93" # MSRV
toolchain: "1.88" # MSRV
- uses: mozilla-actions/sccache-action@v0.0.9
- uses: Swatinem/rust-cache@v2
with:
@@ -135,7 +135,7 @@ jobs:
docs:
name: docs
runs-on: ${{ github.repository == 'paradigmxyz/reth' && 'depot-ubuntu-latest-4' || 'ubuntu-latest' }}
runs-on: depot-ubuntu-latest-4
timeout-minutes: 30
steps:
- uses: actions/checkout@v6
@@ -153,7 +153,7 @@ jobs:
fmt:
name: fmt
runs-on: ${{ github.repository == 'paradigmxyz/reth' && 'depot-ubuntu-latest' || 'ubuntu-latest' }}
runs-on: depot-ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v6
@@ -167,7 +167,7 @@ jobs:
udeps:
name: udeps
runs-on: ${{ github.repository == 'paradigmxyz/reth' && 'depot-ubuntu-latest' || 'ubuntu-latest' }}
runs-on: depot-ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v6
@@ -182,7 +182,7 @@ jobs:
book:
name: book
runs-on: ${{ github.repository == 'paradigmxyz/reth' && 'depot-ubuntu-latest' || 'ubuntu-latest' }}
runs-on: depot-ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v6
@@ -240,7 +240,7 @@ jobs:
# Checks that selected crates can compile with power set of features
features:
name: features
runs-on: ${{ github.repository == 'paradigmxyz/reth' && 'depot-ubuntu-latest' || 'ubuntu-latest' }}
runs-on: depot-ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v6
@@ -264,7 +264,7 @@ jobs:
# Check crates correctly propagate features
feature-propagation:
runs-on: ${{ github.repository == 'paradigmxyz/reth' && 'depot-ubuntu-latest' || 'ubuntu-latest' }}
runs-on: depot-ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@v6

View File

@@ -1,107 +0,0 @@
name: pgo-profile
on:
workflow_call:
inputs:
pgo_blocks:
description: "Number of blocks to execute for PGO profiling"
required: false
type: string
default: "20"
workflow_dispatch:
inputs:
pgo_blocks:
description: "Number of blocks to execute for PGO profiling"
required: false
type: string
default: "20"
jobs:
collect:
name: collect PGO profiles
runs-on: [self-hosted, Linux, X64]
timeout-minutes: 180
env:
SCHELK_MOUNT: /reth-bench
BENCH_RPC_URL: https://ethereum.reth.rs/rpc
RUSTC_WRAPPER: "sccache"
steps:
- uses: actions/checkout@v6
with:
submodules: true
- uses: rui314/setup-mold@v1
- uses: dtolnay/rust-toolchain@stable
with:
target: x86_64-unknown-linux-gnu
- uses: mozilla-actions/sccache-action@v0.0.9
continue-on-error: true
- uses: Swatinem/rust-cache@v2
with:
cache-on-failure: true
- name: Install dependencies
run: |
sudo apt-get update -qq
sudo apt-get install -y --no-install-recommends \
dmsetup lsb-release wget linux-tools-"$(uname -r)" || \
sudo apt-get install -y --no-install-recommends linux-tools-generic
- name: Download snapshot if needed
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
BENCH_REPO: ${{ github.repository }}
run: |
if ! .github/scripts/bench-reth-snapshot.sh --check; then
echo "Snapshot outdated or missing, downloading..."
.github/scripts/bench-reth-snapshot.sh
fi
- name: Mount snapshot
run: |
sudo pkill -9 reth || true
sleep 1
if mountpoint -q "$SCHELK_MOUNT"; then
sudo umount -l "$SCHELK_MOUNT" || true
sudo schelk recover -y || true
fi
sudo schelk mount -y
sync
sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'
- name: Collect PGO profile
run: |
DATADIR="$SCHELK_MOUNT/datadir" \
RPC_URL="$BENCH_RPC_URL" \
PGO_BLOCKS="${{ inputs.pgo_blocks || '20' }}" \
BOLT_BLOCKS="${{ inputs.pgo_blocks || '20' }}" \
COLLECT_PGO_ONLY=true \
SKIP_BOLT=true \
PROFILE=maxperf-symbols \
FEATURES="jemalloc,asm-keccak,min-debug-logs" \
TARGET=x86_64-unknown-linux-gnu \
EXTRA_RUSTFLAGS="-C target-cpu=x86-64-v3 -C target-feature=+pclmulqdq" \
.github/scripts/build_pgo_bolt.sh
- name: Show PGO profile stats
run: |
LLVM_PROFDATA=$(find "$(rustc --print sysroot)" -name llvm-profdata -type f | head -1)
if [ -z "$LLVM_PROFDATA" ]; then
echo "::error::llvm-profdata not found in rust toolchain"
exit 1
fi
"$LLVM_PROFDATA" show --detailed-summary --topn=20 target/pgo-profiles/merged.profdata
- name: Upload PGO profile
uses: actions/upload-artifact@v7
with:
name: pgo-profdata
path: target/pgo-profiles/merged.profdata
retention-days: 1
- name: Recover snapshot
if: always()
run: |
if mountpoint -q "$SCHELK_MOUNT"; then
sudo umount -l "$SCHELK_MOUNT" || true
sudo schelk recover -y || true
fi

View File

@@ -1,30 +0,0 @@
name: Pull request audit
on:
pull_request:
types: [labeled]
jobs:
publish:
runs-on: ubuntu-latest
if: github.event.label.name == 'cyclops'
steps:
- name: Publish event
run: |
set -euo pipefail
echo "${{ secrets.EVENTS_KEY }}" > ${{ runner.temp }}/key
echo "${{ secrets.EVENTS_CERT }}" > ${{ runner.temp }}/cert
curl -sf -o /dev/null -X POST ${{ secrets.EVENTS_ARGS }} \
-H "Content-Type: application/json" \
--key ${{ runner.temp }}/key \
--cert ${{ runner.temp }}/cert \
-d '{
"repository": "${{ github.repository }}",
"event": "pr_audit",
"data": {
"pr_number": ${{ github.event.pull_request.number }},
"sha": "${{ github.event.pull_request.head.sha }}"
}
}'

View File

@@ -52,7 +52,7 @@ jobs:
uses: docker/setup-buildx-action@v3
- name: Log in to GitHub Container Registry
uses: docker/login-action@v4
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}

View File

@@ -13,10 +13,6 @@ on:
description: "Enable dry run mode (builds artifacts but skips uploads and release creation)"
type: boolean
default: false
pgo_blocks:
description: "Number of blocks to execute for PGO profiling on self-hosted runner"
type: string
default: "20"
env:
REPO_NAME: ${{ github.repository_owner }}/reth
@@ -41,7 +37,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Extract version
run: echo "VERSION=${GITHUB_REF_NAME//\//-}" >> $GITHUB_OUTPUT
run: echo "VERSION=${GITHUB_REF_NAME}" >> $GITHUB_OUTPUT
id: extract_version
outputs:
VERSION: ${{ steps.extract_version.outputs.VERSION }}
@@ -73,22 +69,26 @@ jobs:
fail-fast: true
matrix:
configs:
- target: aarch64-unknown-linux-gnu
os: ubuntu-24.04-arm
- target: x86_64-unknown-linux-gnu
os: ubuntu-24.04
profile: maxperf
allow_fail: false
- target: aarch64-unknown-linux-gnu
os: ubuntu-24.04
profile: maxperf
allow_fail: false
rustflags: ""
native: true
- target: x86_64-apple-darwin
os: macos-14
profile: maxperf
allow_fail: false
rustflags: "-C target-cpu=x86-64-v3 -C target-feature=+pclmulqdq"
- target: aarch64-apple-darwin
os: macos-14
profile: maxperf
allow_fail: false
rustflags: ""
- target: riscv64gc-unknown-linux-gnu
os: ubuntu-24.04
profile: maxperf
allow_fail: true
build:
- command: build
binary: reth
@@ -100,10 +100,9 @@ jobs:
target: ${{ matrix.configs.target }}
- uses: mozilla-actions/sccache-action@v0.0.9
- name: Install cross main
if: ${{ !matrix.configs.native }}
id: cross_main
run: |
cargo install cross --locked --git https://github.com/cross-rs/cross
cargo install cross --git https://github.com/cross-rs/cross
- uses: Swatinem/rust-cache@v2
with:
cache-on-failure: true
@@ -115,12 +114,7 @@ jobs:
echo "MACOSX_DEPLOYMENT_TARGET=$(xcrun -sdk macosx --show-sdk-platform-version)" >> $GITHUB_ENV
- name: Build Reth
run: |
if [ "${{ matrix.configs.native }}" = "true" ]; then
make PROFILE=${{ matrix.configs.profile }} EXTRA_RUSTFLAGS="${{ matrix.configs.rustflags }}" ${{ matrix.build.command }}-native-${{ matrix.configs.target }}
else
make PROFILE=${{ matrix.configs.profile }} EXTRA_RUSTFLAGS="${{ matrix.configs.rustflags }}" ${{ matrix.build.command }}-${{ matrix.configs.target }}
fi
run: make PROFILE=${{ matrix.configs.profile }} ${{ matrix.build.command }}-${{ matrix.configs.target }}
- name: Move binary
run: |
mkdir artifacts
@@ -141,102 +135,22 @@ jobs:
- name: Upload artifact
if: ${{ github.event.inputs.dry_run != 'true' }}
uses: actions/upload-artifact@v7
uses: actions/upload-artifact@v6
with:
name: ${{ matrix.build.binary }}-${{ needs.extract-version.outputs.VERSION }}-${{ matrix.configs.target }}.tar.gz
path: ${{ matrix.build.binary }}-${{ needs.extract-version.outputs.VERSION }}-${{ matrix.configs.target }}.tar.gz
- name: Upload signature
if: ${{ github.event.inputs.dry_run != 'true' }}
uses: actions/upload-artifact@v7
uses: actions/upload-artifact@v6
with:
name: ${{ matrix.build.binary }}-${{ needs.extract-version.outputs.VERSION }}-${{ matrix.configs.target }}.tar.gz.asc
path: ${{ matrix.build.binary }}-${{ needs.extract-version.outputs.VERSION }}-${{ matrix.configs.target }}.tar.gz.asc
collect-pgo-profile:
uses: ./.github/workflows/pgo-profile.yml
with:
pgo_blocks: ${{ github.event_name == 'workflow_dispatch' && inputs.pgo_blocks || '20' }}
secrets: inherit
build-pgo:
name: build release (x86_64-linux PGO+BOLT)
runs-on: [self-hosted, Linux, X64]
needs: [extract-version, collect-pgo-profile]
timeout-minutes: 120
env:
RUSTC_WRAPPER: "sccache"
steps:
- uses: actions/checkout@v6
with:
submodules: true
- uses: rui314/setup-mold@v1
- uses: dtolnay/rust-toolchain@stable
with:
target: x86_64-unknown-linux-gnu
- uses: mozilla-actions/sccache-action@v0.0.9
continue-on-error: true
- uses: Swatinem/rust-cache@v2
with:
cache-on-failure: true
- name: Download pre-collected PGO profile
uses: actions/download-artifact@v7
with:
name: pgo-profdata
path: dist
- name: Verify PGO profile artifact
run: |
test -f dist/merged.profdata
ls -lh dist/merged.profdata
- name: Build Reth with PGO+BOLT
run: |
SKIP_BOLT=true \
PGO_PROFDATA="$PWD/dist/merged.profdata" \
PROFILE=maxperf-symbols \
FEATURES="jemalloc,asm-keccak,min-debug-logs" \
TARGET=x86_64-unknown-linux-gnu \
EXTRA_RUSTFLAGS="-C target-cpu=x86-64-v3 -C target-feature=+pclmulqdq" \
.github/scripts/build_pgo_bolt.sh
- name: Move binary
run: |
mkdir artifacts
mv target/maxperf-symbols/reth ./artifacts
- name: Configure GPG and create artifacts
env:
GPG_SIGNING_KEY: ${{ secrets.GPG_SIGNING_KEY }}
GPG_PASSPHRASE: ${{ secrets.GPG_PASSPHRASE }}
run: |
export GPG_TTY=$(tty)
echo -n "$GPG_SIGNING_KEY" | base64 --decode | gpg --batch --import
cd artifacts
tar -czf reth-${{ needs.extract-version.outputs.VERSION }}-x86_64-unknown-linux-gnu.tar.gz reth*
echo "$GPG_PASSPHRASE" | gpg --passphrase-fd 0 --pinentry-mode loopback --batch -ab reth-${{ needs.extract-version.outputs.VERSION }}-x86_64-unknown-linux-gnu.tar.gz
mv *tar.gz* ..
shell: bash
- name: Upload artifact
if: ${{ github.event.inputs.dry_run != 'true' }}
uses: actions/upload-artifact@v6
with:
name: reth-${{ needs.extract-version.outputs.VERSION }}-x86_64-unknown-linux-gnu.tar.gz
path: reth-${{ needs.extract-version.outputs.VERSION }}-x86_64-unknown-linux-gnu.tar.gz
- name: Upload signature
if: ${{ github.event.inputs.dry_run != 'true' }}
uses: actions/upload-artifact@v6
with:
name: reth-${{ needs.extract-version.outputs.VERSION }}-x86_64-unknown-linux-gnu.tar.gz.asc
path: reth-${{ needs.extract-version.outputs.VERSION }}-x86_64-unknown-linux-gnu.tar.gz.asc
draft-release:
name: draft release
runs-on: ubuntu-latest
needs: [build, build-pgo, extract-version]
needs: [build, extract-version]
if: ${{ github.event.inputs.dry_run != 'true' }}
env:
VERSION: ${{ needs.extract-version.outputs.VERSION }}
@@ -250,7 +164,7 @@ jobs:
with:
fetch-depth: 0
- name: Download artifacts
uses: actions/download-artifact@v8
uses: actions/download-artifact@v7
- name: Generate full changelog
id: changelog
run: |

View File

@@ -43,7 +43,7 @@ jobs:
echo "Binaries SHA256 on ${{ matrix.machine }}: $(cat checksum.sha256)"
- name: Upload the hash
uses: actions/upload-artifact@v7
uses: actions/upload-artifact@v6
with:
name: checksum-${{ matrix.machine }}
path: |
@@ -56,12 +56,12 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Download artifacts from machine-1
uses: actions/download-artifact@v8
uses: actions/download-artifact@v7
with:
name: checksum-machine-1
path: machine-1/
- name: Download artifacts from machine-2
uses: actions/download-artifact@v8
uses: actions/download-artifact@v7
with:
name: checksum-machine-2
path: machine-2/

View File

@@ -23,7 +23,7 @@ jobs:
name: stage-run-test
# Only run stage commands test in merge groups
if: github.event_name == 'merge_group'
runs-on: ${{ github.repository == 'paradigmxyz/reth' && 'depot-ubuntu-latest' || 'ubuntu-latest' }}
runs-on: depot-ubuntu-latest
env:
RUST_LOG: info,sync=error
RUST_BACKTRACE: 1
@@ -38,7 +38,7 @@ jobs:
cache-on-failure: true
- name: Build reth
run: |
cargo install --locked --path bin/reth
cargo install --path bin/reth
- name: Run headers stage
run: |
reth stage run headers --from ${{ env.FROM_BLOCK }} --to ${{ env.TO_BLOCK }} --commit --checkpoints
@@ -51,12 +51,15 @@ jobs:
- name: Run execution stage
run: |
reth stage run execution --from ${{ env.FROM_BLOCK }} --to ${{ env.TO_BLOCK }} --commit --checkpoints
# NOTE: account-hashing, storage-hashing, and hashing stages are omitted.
# With storage v2 (now default), these stages are no-ops because the
# execution stage writes directly to HashedAccounts/HashedStorages.
# Running them here is harmful: `stage run` unwinds before executing,
# and the unwind reverts the hashed state that execution wrote, but
# the no-op execute never restores it — causing merkle to fail.
- name: Run account-hashing stage
run: |
reth stage run account-hashing --from ${{ env.FROM_BLOCK }} --to ${{ env.TO_BLOCK }} --commit --checkpoints
- name: Run storage hashing stage
run: |
reth stage run storage-hashing --from ${{ env.FROM_BLOCK }} --to ${{ env.TO_BLOCK }} --commit --checkpoints
- name: Run hashing stage
run: |
reth stage run hashing --from ${{ env.FROM_BLOCK }} --to ${{ env.TO_BLOCK }} --commit --checkpoints
- name: Run merkle stage
run: |
reth stage run merkle --from ${{ env.FROM_BLOCK }} --to ${{ env.TO_BLOCK }} --commit --checkpoints

View File

@@ -19,7 +19,7 @@ jobs:
sync:
if: github.repository == 'paradigmxyz/reth'
name: sync (${{ matrix.chain.bin }})
runs-on: ${{ github.repository == 'paradigmxyz/reth' && 'depot-ubuntu-latest' || 'ubuntu-latest' }}
runs-on: depot-ubuntu-latest
env:
RUST_LOG: info,sync=error
RUST_BACKTRACE: 1

View File

@@ -19,7 +19,7 @@ jobs:
sync:
if: github.repository == 'paradigmxyz/reth'
name: sync (${{ matrix.chain.bin }})
runs-on: ${{ github.repository == 'paradigmxyz/reth' && 'depot-ubuntu-latest' || 'ubuntu-latest' }}
runs-on: depot-ubuntu-latest
env:
RUST_LOG: info,sync=error
RUST_BACKTRACE: 1

View File

@@ -20,7 +20,7 @@ concurrency:
jobs:
test:
name: test / ${{ matrix.type }} / ${{ matrix.storage }}
runs-on: ${{ github.repository == 'paradigmxyz/reth' && 'depot-ubuntu-latest-4' || 'ubuntu-latest' }}
runs-on: depot-ubuntu-latest-4
env:
RUST_BACKTRACE: 1
EDGE_FEATURES: ${{ matrix.storage == 'edge' && 'edge' || '' }}
@@ -57,7 +57,7 @@ jobs:
state:
name: Ethereum state tests
runs-on: ${{ github.repository == 'paradigmxyz/reth' && 'depot-ubuntu-latest-4' || 'ubuntu-latest' }}
runs-on: depot-ubuntu-latest-4
env:
RUST_LOG: info,sync=error
RUST_BACKTRACE: 1
@@ -92,7 +92,7 @@ jobs:
doc:
name: doc tests
runs-on: ${{ github.repository == 'paradigmxyz/reth' && 'depot-ubuntu-latest' || 'ubuntu-latest' }}
runs-on: depot-ubuntu-latest
env:
RUST_BACKTRACE: 1
timeout-minutes: 30

162
CLAUDE.md
View File

@@ -172,97 +172,10 @@ Before submitting changes, ensure:
2. **Clippy**: No warnings
3. **Tests Pass**: All unit and integration tests
4. **Documentation**: Update relevant docs and add doc comments with `cargo docs --document-private-items`
5. **CLI Docs** (if CLI changed): Run `make update-book-cli` (see below)
6. **Commit Messages**: Follow conventional format (feat:, fix:, chore:, etc.)
### CLI Reference Docs (`book` CI Job)
The CLI reference pages under `docs/vocs/docs/pages/cli/` are **auto-generated** from the `reth` binary's `--help` output. **Do not edit these files manually** — any hand edits will be overwritten and CI will fail regardless.
When you add, remove, or modify CLI commands, subcommands, or flags, regenerate the CLI docs by running:
```bash
make update-book-cli
```
This builds `reth` in debug mode and runs `docs/cli/update.sh` to regenerate all CLI pages. Commit the resulting changes.
The `book` CI job (`.github/workflows/lint.yml`) enforces this by regenerating the docs and running `git diff --exit-code`. If the committed docs don't match the generated output, CI fails. Manually editing these pages is never productive — always use `make update-book-cli`.
5. **Commit Messages**: Follow conventional format (feat:, fix:, chore:, etc.)
### Opening PRs against <https://github.com/paradigmxyz/reth>
#### Titles
Use [Conventional Commits](https://www.conventionalcommits.org/) with an optional scope:
```
<type>(<scope>): <short description>
```
**Types**: `feat`, `fix`, `perf`, `refactor`, `docs`, `test`, `chore`
**Scope** (optional): crate or area, e.g. `evm`, `trie`, `rpc`, `engine`, `net`
Examples:
- `fix(rpc): correct gas estimation for ERC-20 transfers`
- `perf: batch trie updates to reduce cursor overhead`
- `feat(engine): add new_payload_interval metric`
#### Descriptions
Keep it short. Say what changed and why — nothing more.
**Do:**
- Write 13 sentences summarizing the change
- Explain _why_ if the diff doesn't make it obvious
- Link related issues or EIPs
- Include benchmark numbers for perf changes
**Don't:**
- List every file changed — that's what the diff is for
- Repeat the title in the body
- Add "Files changed" or "Changes" sections
- Write walls of text that go stale when the diff is updated
- Use filler like "This PR introduces...", "comprehensive", "robust", "enhance", "leverage"
**Template:**
```
Closes #<issue>
<what changed, 1-3 sentences>
<why, if not obvious from the diff>
```
**Good example:**
```
Closes #16800
Adds fallback for external IP resolution so node startup doesn't fail
when STUN is unreachable. Falls back to the configured default.
```
**Bad example:**
```
## Summary
This PR introduces comprehensive improvements to the IP resolution system.
## Changes
- Modified `crates/net/discv4/src/lib.rs` to add fallback
- Modified `crates/net/discv4/src/config.rs` to add default IP
- Added tests in `crates/net/discv4/src/tests/ip.rs`
## Files Changed
- crates/net/discv4/src/lib.rs
- crates/net/discv4/src/config.rs
- crates/net/discv4/src/tests/ip.rs
```
#### Labels and CI
Label PRs appropriately, first check the available labels and then apply the relevant ones:
* when changes are RPC related, add A-rpc label
* when changes are docs related, add C-docs label
@@ -400,74 +313,6 @@ GLOBAL_COUNTER.fetch_add(1, Ordering::SeqCst);
Before adding a comment, ask: Would someone reading just the current code (no PR, no history) find this helpful?
#### Rust Style Guides
##### Type Ordering in Files
When defining structs, traits, and functions in a file, follow this ordering convention. The file's primary type (matching the file name) comes first, followed by supporting public types, then private types and helpers.
```rust
use ...;
/// The primary type of this file (matches filename).
pub struct PayloadProcessor { ... }
impl PayloadProcessor { ... }
// Followed by public auxiliary types that support the primary type
/// Configuration for the processor.
pub struct PayloadProcessorConfig { ... }
/// Result type returned by processor operations.
pub struct ProcessorResult { ... }
// Followed by public traits related to the primary type
pub trait ProcessorExt { ... }
// Followed by private helper types
struct InternalState { ... }
// Followed by private helper functions
fn validate_input() { ... }
```
❌ **Bad**: Adding new traits and auxiliary types **above** the file's primary type (see [#22133](https://github.com/paradigmxyz/reth/pull/22133)):
```rust
use ...;
// ❌ BAD - new auxiliary struct added before the file's main type
pub struct CacheWaitDurations { ... }
// ❌ BAD - new trait added before the file's main type
pub trait WaitForCaches { ... }
// The file's primary type is buried below unrelated additions
pub struct PayloadProcessor { ... }
```
✅ **Good**: New types go **after** the primary type:
```rust
use ...;
// ✅ The file's primary type stays at the top
pub struct PayloadProcessor { ... }
impl PayloadProcessor { ... }
// ✅ Auxiliary types follow the primary type
pub struct CacheWaitDurations { ... }
pub trait WaitForCaches { ... }
impl WaitForCaches for PayloadProcessor { ... }
```
### Example Contribution Workflow
Let's say you want to fix a bug where external IP resolution fails on startup:
@@ -542,8 +387,5 @@ cargo build --release
cargo check --workspace --all-features
# Check documentation
cargo docs --document-private-items
# Regenerate CLI reference docs (after CLI changes)
make update-book-cli
cargo docs --document-private-items
```

1610
Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,7 +1,7 @@
[workspace.package]
version = "1.11.3"
version = "1.10.2"
edition = "2024"
rust-version = "1.93"
rust-version = "1.88"
license = "MIT OR Apache-2.0"
homepage = "https://paradigmxyz.github.io/reth"
repository = "https://github.com/paradigmxyz/reth"
@@ -27,6 +27,7 @@ members = [
"crates/engine/invalid-block-hooks/",
"crates/engine/local",
"crates/engine/primitives/",
"crates/engine/service",
"crates/engine/tree/",
"crates/engine/util/",
"crates/era",
@@ -138,7 +139,6 @@ members = [
"examples/exex-subscription",
"examples/exex-test",
"examples/full-contract-state",
"examples/migrate-trie-to-packed",
"examples/manual-p2p/",
"examples/network-txpool/",
"examples/network/",
@@ -170,7 +170,6 @@ rust.rust_2018_idioms = { level = "deny", priority = -1 }
rust.unreachable_pub = "warn"
rust.unused_must_use = "deny"
rust.rust_2024_incompatible_pat = "warn"
rust.unexpected_cfgs = { level = "warn", check-cfg = ['cfg(tokio_unstable)'] }
rustdoc.all = "warn"
# rust.unnameable-types = "warn"
@@ -350,6 +349,7 @@ reth-ecies = { path = "crates/net/ecies" }
reth-engine-local = { path = "crates/engine/local" }
reth-engine-primitives = { path = "crates/engine/primitives", default-features = false }
reth-engine-tree = { path = "crates/engine/tree" }
reth-engine-service = { path = "crates/engine/service" }
reth-engine-util = { path = "crates/engine/util" }
reth-era = { path = "crates/era" }
reth-era-downloader = { path = "crates/era-downloader" }
@@ -399,7 +399,7 @@ reth-payload-builder-primitives = { path = "crates/payload/builder-primitives" }
reth-payload-primitives = { path = "crates/payload/primitives" }
reth-payload-validator = { path = "crates/payload/validator" }
reth-payload-util = { path = "crates/payload/util" }
reth-primitives = { path = "crates/primitives", default-features = false, features = ["__internal"] }
reth-primitives = { path = "crates/primitives", default-features = false }
reth-primitives-traits = { path = "crates/primitives-traits", default-features = false }
reth-provider = { path = "crates/storage/provider" }
reth-prune = { path = "crates/prune/prune" }
@@ -437,15 +437,15 @@ reth-trie-sparse = { path = "crates/trie/sparse", default-features = false }
reth-zstd-compressors = { path = "crates/storage/zstd-compressors", default-features = false }
# revm
revm = { version = "36.0.0", default-features = false }
revm-bytecode = { version = "9.0.0", default-features = false }
revm-database = { version = "12.0.0", default-features = false }
revm-state = { version = "10.0.0", default-features = false }
revm-primitives = { version = "22.1.0", default-features = false }
revm-interpreter = { version = "34.0.0", default-features = false }
revm-database-interface = { version = "10.0.0", default-features = false }
op-revm = { version = "17.0.0", default-features = false }
revm-inspectors = "0.36.0"
revm = { version = "34.0.0", default-features = false }
revm-bytecode = { version = "8.0.0", default-features = false }
revm-database = { version = "10.0.0", default-features = false }
revm-state = { version = "9.0.0", default-features = false }
revm-primitives = { version = "22.0.0", default-features = false }
revm-interpreter = { version = "32.0.0", default-features = false }
revm-database-interface = { version = "9.0.0", default-features = false }
op-revm = { version = "15.0.0", default-features = false }
revm-inspectors = "0.34.2"
# eth
alloy-dyn-abi = "1.5.6"
@@ -455,44 +455,49 @@ alloy-sol-types = { version = "1.5.6", default-features = false }
alloy-chains = { version = "0.2.5", default-features = false }
alloy-eip2124 = { version = "0.2.0", default-features = false }
alloy-eip7928 = { version = "0.3.0", default-features = false }
alloy-evm = { version = "0.29.2", default-features = false }
alloy-evm = { version = "0.27.2", default-features = false }
alloy-rlp = { version = "0.3.13", default-features = false, features = ["core-net"] }
alloy-trie = { version = "0.9.4", default-features = false }
alloy-hardforks = "0.4.5"
alloy-consensus = { version = "1.7.3", default-features = false }
alloy-contract = { version = "1.7.3", default-features = false }
alloy-eips = { version = "1.7.3", default-features = false }
alloy-genesis = { version = "1.7.3", default-features = false }
alloy-json-rpc = { version = "1.7.3", default-features = false }
alloy-network = { version = "1.7.3", default-features = false }
alloy-network-primitives = { version = "1.7.3", default-features = false }
alloy-provider = { version = "1.7.3", features = ["reqwest", "debug-api"], default-features = false }
alloy-pubsub = { version = "1.7.3", default-features = false }
alloy-rpc-client = { version = "1.7.3", default-features = false }
alloy-rpc-types = { version = "1.7.3", features = ["eth"], default-features = false }
alloy-rpc-types-admin = { version = "1.7.3", default-features = false }
alloy-rpc-types-anvil = { version = "1.7.3", default-features = false }
alloy-rpc-types-beacon = { version = "1.7.3", default-features = false }
alloy-rpc-types-debug = { version = "1.7.3", default-features = false }
alloy-rpc-types-engine = { version = "1.7.3", default-features = false }
alloy-rpc-types-eth = { version = "1.7.3", default-features = false }
alloy-rpc-types-mev = { version = "1.7.3", default-features = false }
alloy-rpc-types-trace = { version = "1.7.3", default-features = false }
alloy-rpc-types-txpool = { version = "1.7.3", default-features = false }
alloy-serde = { version = "1.7.3", default-features = false }
alloy-signer = { version = "1.7.3", default-features = false }
alloy-signer-local = { version = "1.7.3", default-features = false }
alloy-transport = { version = "1.7.3" }
alloy-transport-http = { version = "1.7.3", features = ["reqwest-rustls-tls"], default-features = false }
alloy-transport-ipc = { version = "1.7.3", default-features = false }
alloy-transport-ws = { version = "1.7.3", default-features = false }
alloy-consensus = { version = "1.6.3", default-features = false }
alloy-contract = { version = "1.6.3", default-features = false }
alloy-eips = { version = "1.6.3", default-features = false }
alloy-genesis = { version = "1.6.3", default-features = false }
alloy-json-rpc = { version = "1.6.3", default-features = false }
alloy-network = { version = "1.6.3", default-features = false }
alloy-network-primitives = { version = "1.6.3", default-features = false }
alloy-provider = { version = "1.6.3", features = ["reqwest", "debug-api"], default-features = false }
alloy-pubsub = { version = "1.6.3", default-features = false }
alloy-rpc-client = { version = "1.6.3", default-features = false }
alloy-rpc-types = { version = "1.6.3", features = ["eth"], default-features = false }
alloy-rpc-types-admin = { version = "1.6.3", default-features = false }
alloy-rpc-types-anvil = { version = "1.6.3", default-features = false }
alloy-rpc-types-beacon = { version = "1.6.3", default-features = false }
alloy-rpc-types-debug = { version = "1.6.3", default-features = false }
alloy-rpc-types-engine = { version = "1.6.3", default-features = false }
alloy-rpc-types-eth = { version = "1.6.3", default-features = false }
alloy-rpc-types-mev = { version = "1.6.3", default-features = false }
alloy-rpc-types-trace = { version = "1.6.3", default-features = false }
alloy-rpc-types-txpool = { version = "1.6.3", default-features = false }
alloy-serde = { version = "1.6.3", default-features = false }
alloy-signer = { version = "1.6.3", default-features = false }
alloy-signer-local = { version = "1.6.3", default-features = false }
alloy-transport = { version = "1.6.3" }
alloy-transport-http = { version = "1.6.3", features = ["reqwest-rustls-tls"], default-features = false }
alloy-transport-ipc = { version = "1.6.3", default-features = false }
alloy-transport-ws = { version = "1.6.3", default-features = false }
# op
op-alloy-rpc-types = { version = "0.24.0", default-features = false }
op-alloy-rpc-types-engine = { version = "0.24.0", default-features = false }
op-alloy-consensus = { version = "0.24.0", default-features = false }
alloy-op-evm = { version = "0.27.2", default-features = false }
alloy-op-hardforks = "0.4.4"
op-alloy-rpc-types = { version = "0.23.1", default-features = false }
op-alloy-rpc-types-engine = { version = "0.23.1", default-features = false }
op-alloy-network = { version = "0.23.1", default-features = false }
op-alloy-consensus = { version = "0.23.1", default-features = false }
op-alloy-rpc-jsonrpsee = { version = "0.23.1", default-features = false }
op-alloy-flz = { version = "0.13.1", default-features = false }
# misc
either = { version = "1.15.0", default-features = false }
@@ -504,7 +509,6 @@ bincode = "1.3"
bitflags = "2.4"
boyer-moore-magiclen = "0.2.16"
bytes = { version = "1.11.1", default-features = false }
blake3 = "1.8"
brotli = "8"
cfg-if = "1.0"
clap = "4"
@@ -520,27 +524,23 @@ humantime = "2.1"
humantime-serde = "1.1"
itertools = { version = "0.14", default-features = false }
linked_hash_set = "0.1"
libc = "0.2"
lz4 = "1.28.1"
modular-bitfield = "0.13.1"
notify = { version = "8.0.0", default-features = false, features = ["macos_fsevent"] }
nybbles = { version = "0.4.8", default-features = false }
once_cell = { version = "1.19", default-features = false, features = ["critical-section"] }
parking_lot = "0.12"
quanta = "0.12"
paste = "1.0"
rand = "0.9"
rayon = "1.7"
thread-priority = "3.0.0"
rustc-hash = { version = "2.0", default-features = false }
schnellru = "0.2"
serde = { version = "1.0", default-features = false }
serde_json = { version = "1.0", default-features = false, features = ["alloc"] }
serde_with = { version = "3", default-features = false, features = ["macros"] }
sha2 = { version = "0.10", default-features = false }
shellexpand = "3.0.0"
shlex = "1.3"
# https://github.com/orlp/slotmap/pull/148
slotmap = { git = "https://github.com/DaniPopes/slotmap.git", branch = "dani/shrink-methods" }
smallvec = "1"
strum = { version = "0.27", default-features = false }
strum_macros = "0.27"
@@ -652,7 +652,7 @@ ethereum_ssz_derive = "0.10.1"
jemalloc_pprof = { version = "0.8", default-features = false }
tikv-jemalloc-ctl = "0.6"
tikv-jemallocator = "0.6"
tracy-client = { version = "0.18.0", features = ["demangle"] }
tracy-client = "0.18.0"
snmalloc-rs = { version = "0.3.7", features = ["build_cc"] }
aes = "0.8.1"
@@ -665,8 +665,6 @@ cipher = "0.4.3"
comfy-table = "7.0"
concat-kdf = "0.1.0"
crossbeam-channel = "0.5.13"
crossbeam-queue = "0.3"
crossbeam-utils = "0.8"
crossterm = "0.29.0"
csv = "1.3.0"
ctrlc = "3.4"
@@ -713,6 +711,28 @@ vergen-git2 = "9.1.0"
ipnet = "2.11"
[patch.crates-io]
# revm staging patches
revm = { git = "https://github.com/bluealloy/revm", rev = "33330a285e621b9170c30a21cfea9ab32e2a2169" }
op-revm = { git = "https://github.com/bluealloy/revm", rev = "33330a285e621b9170c30a21cfea9ab32e2a2169" }
revm-bytecode = { git = "https://github.com/bluealloy/revm", rev = "33330a285e621b9170c30a21cfea9ab32e2a2169" }
revm-context = { git = "https://github.com/bluealloy/revm", rev = "33330a285e621b9170c30a21cfea9ab32e2a2169" }
revm-context-interface = { git = "https://github.com/bluealloy/revm", rev = "33330a285e621b9170c30a21cfea9ab32e2a2169" }
revm-database = { git = "https://github.com/bluealloy/revm", rev = "33330a285e621b9170c30a21cfea9ab32e2a2169" }
revm-database-interface = { git = "https://github.com/bluealloy/revm", rev = "33330a285e621b9170c30a21cfea9ab32e2a2169" }
revm-handler = { git = "https://github.com/bluealloy/revm", rev = "33330a285e621b9170c30a21cfea9ab32e2a2169" }
revm-inspector = { git = "https://github.com/bluealloy/revm", rev = "33330a285e621b9170c30a21cfea9ab32e2a2169" }
revm-interpreter = { git = "https://github.com/bluealloy/revm", rev = "33330a285e621b9170c30a21cfea9ab32e2a2169" }
revm-precompile = { git = "https://github.com/bluealloy/revm", rev = "33330a285e621b9170c30a21cfea9ab32e2a2169" }
revm-primitives = { git = "https://github.com/bluealloy/revm", rev = "33330a285e621b9170c30a21cfea9ab32e2a2169" }
revm-state = { git = "https://github.com/bluealloy/revm", rev = "33330a285e621b9170c30a21cfea9ab32e2a2169" }
# revm-inspectors staging patch
revm-inspectors = { git = "https://github.com/paradigmxyz/revm-inspectors", rev = "e80e2eab72dfa18011e6a99abd37027290a46e83" }
# alloy-evm staging patches
alloy-evm = { git = "https://github.com/alloy-rs/evm", rev = "742dc14749ea0279c03ca27b1c26f26ac19fbefb" }
alloy-op-evm = { git = "https://github.com/alloy-rs/evm", rev = "742dc14749ea0279c03ca27b1c26f26ac19fbefb" }
# alloy-consensus = { git = "https://github.com/alloy-rs/alloy", rev = "3049f232fbb44d1909883e154eb38ec5962f53a3" }
# alloy-contract = { git = "https://github.com/alloy-rs/alloy", rev = "3049f232fbb44d1909883e154eb38ec5962f53a3" }
# alloy-eips = { git = "https://github.com/alloy-rs/alloy", rev = "3049f232fbb44d1909883e154eb38ec5962f53a3" }
@@ -742,8 +762,10 @@ ipnet = "2.11"
# alloy-transport-ws = { git = "https://github.com/alloy-rs/alloy", rev = "3049f232fbb44d1909883e154eb38ec5962f53a3" }
# op-alloy-consensus = { git = "https://github.com/alloy-rs/op-alloy", rev = "a79d6fc" }
# op-alloy-network = { git = "https://github.com/alloy-rs/op-alloy", rev = "a79d6fc" }
# op-alloy-rpc-types = { git = "https://github.com/alloy-rs/op-alloy", rev = "a79d6fc" }
# op-alloy-rpc-types-engine = { git = "https://github.com/alloy-rs/op-alloy", rev = "a79d6fc" }
# op-alloy-rpc-jsonrpsee = { git = "https://github.com/alloy-rs/op-alloy", rev = "a79d6fc" }
#
# revm-inspectors = { git = "https://github.com/paradigmxyz/revm-inspectors", rev = "1207e33" }
#
@@ -753,6 +775,10 @@ ipnet = "2.11"
# jsonrpsee-http-client = { git = "https://github.com/paradigmxyz/jsonrpsee", branch = "matt/make-rpc-service-pub" }
# jsonrpsee-types = { git = "https://github.com/paradigmxyz/jsonrpsee", branch = "matt/make-rpc-service-pub" }
# alloy-evm = { git = "https://github.com/alloy-rs/evm", rev = "9bc2dba" }
# alloy-evm = { git = "https://github.com/alloy-rs/evm", rev = "df124c0" }
# alloy-op-evm = { git = "https://github.com/alloy-rs/evm", rev = "df124c0" }
# revm-inspectors = { git = "https://github.com/paradigmxyz/revm-inspectors", rev = "3020ea8" }
# alloy-evm = { git = "https://github.com/alloy-rs/evm", rev = "072c248" }
# alloy-op-evm = { git = "https://github.com/alloy-rs/evm", rev = "072c248" }

View File

@@ -19,11 +19,10 @@ pre-build = [
image = "ubuntu:24.04"
pre-build = [
"apt update",
"apt install --yes gcc gcc-riscv64-linux-gnu g++-riscv64-linux-gnu libclang-dev make",
"apt install --yes gcc gcc-riscv64-linux-gnu libclang-dev make",
]
env.passthrough = [
"CARGO_TARGET_RISCV64GC_UNKNOWN_LINUX_GNU_LINKER=riscv64-linux-gnu-gcc",
"CXX_riscv64gc_unknown_linux_gnu=riscv64-linux-gnu-g++",
]
[build.env]

View File

@@ -1,6 +1,6 @@
# syntax=docker.io/docker/dockerfile:1.7-labs
FROM lukemathwalker/cargo-chef:latest-rust-1.93 AS chef
FROM lukemathwalker/cargo-chef:latest-rust-1 AS chef
WORKDIR /app
LABEL org.opencontainers.image.source=https://github.com/paradigmxyz/reth

View File

@@ -1,12 +1,10 @@
# syntax=docker/dockerfile:1
# Dockerfile for reth, optimized for Depot builds
# Supports PGO+BOLT optimization for maximum performance
# Usage:
# reth: --build-arg BINARY=reth
# PGO+BOLT: --build-arg USE_PGO_BOLT=true (Linux x86_64/aarch64 only)
FROM rust:1.93 AS builder
FROM rust:1 AS builder
WORKDIR /app
LABEL org.opencontainers.image.source=https://github.com/paradigmxyz/reth
@@ -45,18 +43,6 @@ ENV VERGEN_GIT_SHA=$VERGEN_GIT_SHA
ENV VERGEN_GIT_DESCRIBE=$VERGEN_GIT_DESCRIBE
ENV VERGEN_GIT_DIRTY=$VERGEN_GIT_DIRTY
# Enable PGO+BOLT optimization (Linux only)
ARG USE_PGO_BOLT=false
ENV USE_PGO_BOLT=$USE_PGO_BOLT
# Optional path to a pre-collected merged.profdata file in build context.
ARG PGO_PROFDATA=""
ENV PGO_PROFDATA=$PGO_PROFDATA
# Whether to strip debug symbols from PGO-built binaries.
ARG STRIP_SYMBOLS=true
ENV STRIP_SYMBOLS=$STRIP_SYMBOLS
# Build application
# Platform-specific RUSTFLAGS: amd64 uses x86-64-v3 (Haswell+) with pclmulqdq for rocksdb
ARG TARGETPLATFORM
@@ -67,21 +53,12 @@ RUN --mount=type=secret,id=DEPOT_TOKEN,env=SCCACHE_WEBDAV_TOKEN \
--mount=type=cache,target=$SCCACHE_DIR,sharing=shared \
export RUSTC_WRAPPER=sccache SCCACHE_WEBDAV_ENDPOINT=https://cache.depot.dev SCCACHE_DIR=/sccache && \
sccache --start-server && \
if [ "$USE_PGO_BOLT" = "true" ] && [ "$TARGETPLATFORM" = "linux/amd64" ] && [ -n "$PGO_PROFDATA" ] && [ -f "$PGO_PROFDATA" ]; then \
apt-get update && apt-get install -y -qq lsb-release wget sudo && \
BINARY="$BINARY" PROFILE="$BUILD_PROFILE" FEATURES="$FEATURES" SKIP_BOLT=true STRIP_SYMBOLS="$STRIP_SYMBOLS" PGO_PROFDATA="$PGO_PROFDATA" \
./.github/scripts/build_pgo_bolt.sh; \
else \
if [ "$USE_PGO_BOLT" = "true" ]; then \
echo "PGO requested but pre-collected profile missing at '${PGO_PROFDATA:-<unset>}' - falling back to non-PGO build"; \
fi; \
if [ -n "$RUSTFLAGS" ]; then \
export RUSTFLAGS="$RUSTFLAGS"; \
elif [ "$TARGETPLATFORM" = "linux/amd64" ]; then \
export RUSTFLAGS="-C target-cpu=x86-64-v3 -C target-feature=+pclmulqdq"; \
fi && \
cargo build --profile $BUILD_PROFILE --features "$FEATURES" --locked --bin $BINARY --manifest-path $MANIFEST_PATH/Cargo.toml; \
if [ -n "$RUSTFLAGS" ]; then \
export RUSTFLAGS="$RUSTFLAGS"; \
elif [ "$TARGETPLATFORM" = "linux/amd64" ]; then \
export RUSTFLAGS="-C target-cpu=x86-64-v3 -C target-feature=+pclmulqdq"; \
fi && \
cargo build --profile $BUILD_PROFILE --features "$FEATURES" --locked --bin $BINARY --manifest-path $MANIFEST_PATH/Cargo.toml && \
sccache --show-stats
# Copy binary to a known location (ARG not resolved in COPY)

View File

@@ -17,9 +17,6 @@ FEATURES ?=
# Cargo profile for builds. Default is for local builds, CI uses an override.
PROFILE ?= release
# Extra RUSTFLAGS to append to build targets (e.g., "-C target-cpu=x86-64-v3")
EXTRA_RUSTFLAGS ?=
# Extra flags for Cargo
CARGO_INSTALL_EXTRA_FLAGS ?=
@@ -77,13 +74,13 @@ build-debug: ## Build the reth binary into `target/debug` directory.
cargo build --bin reth --features "$(FEATURES)"
# Builds the reth binary natively.
build-native-%:
$(if $(EXTRA_RUSTFLAGS),RUSTFLAGS="$(EXTRA_RUSTFLAGS)") cargo build --bin reth --target $* --features "$(FEATURES)" --profile "$(PROFILE)"
cargo build --bin reth --target $* --features "$(FEATURES)" --profile "$(PROFILE)"
# The following commands use `cross` to build a cross-compile.
#
# These commands require that:
#
# - `cross` is installed (`cargo install --locked cross`).
# - `cross` is installed (`cargo install cross`).
# - Docker is running.
# - The current user is in the `docker` group.
#
@@ -95,12 +92,11 @@ build-native-%:
# on other systems. JEMALLOC_SYS_WITH_LG_PAGE=16 tells jemalloc to use 64-KiB
# pages. See: https://github.com/paradigmxyz/reth/issues/6742
build-aarch64-unknown-linux-gnu: export JEMALLOC_SYS_WITH_LG_PAGE=16
build-native-aarch64-unknown-linux-gnu: export JEMALLOC_SYS_WITH_LG_PAGE=16
# Note: The additional rustc compiler flags are for intrinsics needed by MDBX.
# See: https://github.com/cross-rs/cross/wiki/FAQ#undefined-reference-with-build-std
build-%:
RUSTFLAGS="-C link-arg=-lgcc -Clink-arg=-static-libgcc $(EXTRA_RUSTFLAGS)" \
RUSTFLAGS="-C link-arg=-lgcc -Clink-arg=-static-libgcc" \
cross build --bin reth --target $* --features "$(FEATURES)" --profile "$(PROFILE)"
# Unfortunately we can't easily use cross to build for Darwin because of licensing issues.
@@ -265,7 +261,7 @@ lint-typos: ensure-typos
ensure-typos:
@if ! command -v typos &> /dev/null; then \
echo "typos not found. Please install it by running the command 'cargo install --locked typos-cli' or refer to the following link for more information: https://github.com/crate-ci/typos"; \
echo "typos not found. Please install it by running the command 'cargo install typos-cli' or refer to the following link for more information: https://github.com/crate-ci/typos"; \
exit 1; \
fi

View File

@@ -93,7 +93,7 @@ When updating this, also update:
- .github/workflows/lint.yml
-->
The Minimum Supported Rust Version (MSRV) of this project is [1.93.0](https://blog.rust-lang.org/2026/01/22/Rust-1.93.0/).
The Minimum Supported Rust Version (MSRV) of this project is [1.88.0](https://blog.rust-lang.org/2025/06/26/Rust-1.88.0/).
See the docs for detailed instructions on how to [build from source](https://reth.rs/installation/source/).

View File

@@ -45,6 +45,9 @@ serde_json.workspace = true
# Time handling
chrono = { workspace = true, features = ["serde"] }
# Path manipulation
shellexpand.workspace = true
# CSV handling
csv.workspace = true

View File

@@ -289,7 +289,11 @@ impl Args {
/// Get the JWT secret path - either provided or derived from datadir
pub(crate) fn jwt_secret_path(&self) -> PathBuf {
match &self.jwt_secret {
Some(path) => path.clone(),
Some(path) => {
let jwt_secret_str = path.to_string_lossy();
let expanded = shellexpand::tilde(&jwt_secret_str);
PathBuf::from(expanded.as_ref())
}
None => {
// Use the same logic as reth: <datadir>/<chain>/jwt.hex
let chain_path = self.datadir.clone().resolve_datadir(self.chain);
@@ -304,9 +308,10 @@ impl Args {
chain_path.data_dir().to_path_buf()
}
/// Get the output directory path
/// Get the expanded output directory path
pub(crate) fn output_dir_path(&self) -> PathBuf {
PathBuf::from(&self.output_dir)
let expanded = shellexpand::tilde(&self.output_dir);
PathBuf::from(expanded.as_ref())
}
/// Get the effective warmup blocks value - either specified or defaults to blocks

View File

@@ -7,7 +7,7 @@ use alloy_primitives::address;
use alloy_provider::{network::AnyNetwork, Provider, RootProvider};
use alloy_rpc_client::ClientBuilder;
use alloy_rpc_types_engine::JwtSecret;
use alloy_transport::layers::{RateLimitRetryPolicy, RetryBackoffLayer};
use alloy_transport::layers::RetryBackoffLayer;
use reqwest::Url;
use reth_node_core::args::BenchmarkArgs;
use tracing::info;
@@ -29,10 +29,6 @@ pub(crate) struct BenchContext {
pub(crate) next_block: u64,
/// Whether the chain is an OP rollup.
pub(crate) is_optimism: bool,
/// Whether to use `reth_newPayload` endpoint instead of `engine_newPayload*`.
pub(crate) use_reth_namespace: bool,
/// Whether to fetch and replay RLP-encoded blocks.
pub(crate) rlp_blocks: bool,
}
impl BenchContext {
@@ -53,15 +49,9 @@ impl BenchContext {
}
}
// set up alloy client for blocks, retrying on 429/503 (default) and 502
let retry_policy =
RateLimitRetryPolicy::default().or(|err: &alloy_transport::TransportError| -> bool {
err.as_transport_err()
.and_then(|t| t.as_http_error())
.is_some_and(|e| e.status == 502)
});
// set up alloy client for blocks
let client = ClientBuilder::default()
.layer(RetryBackoffLayer::new_with_policy(10, 800, u64::MAX, retry_policy))
.layer(RetryBackoffLayer::new(10, 800, u64::MAX))
.http(rpc_url.parse()?);
let block_provider = RootProvider::<AnyNetwork>::new(client);
@@ -94,11 +84,9 @@ impl BenchContext {
// Computes the block range for the benchmark.
//
// - If `--advance` is provided, fetches the latest block from the engine and sets:
// - If `--advance` is provided, fetches the latest block and sets:
// - `from = head + 1`
// - `to = head + advance`
// - If only `--to` is provided, fetches the latest block from the engine and sets:
// - `from = head`
// - Otherwise, uses the values from `--from` and `--to`.
let (from, to) = if let Some(advance) = bench_args.advance {
if advance == 0 {
@@ -111,14 +99,6 @@ impl BenchContext {
.ok_or_else(|| eyre::eyre!("Failed to fetch latest block for --advance"))?;
let head_number = head_block.header.number;
(Some(head_number), Some(head_number + advance))
} else if bench_args.from.is_none() && bench_args.to.is_some() {
let head_block = auth_provider
.get_block_by_number(BlockNumberOrTag::Latest)
.await?
.ok_or_else(|| eyre::eyre!("Failed to fetch latest block from engine"))?;
let head_number = head_block.header.number;
info!(target: "reth-bench", "No --from provided, derived from engine head: {}", head_number);
(Some(head_number), bench_args.to)
} else {
(bench_args.from, bench_args.to)
};
@@ -130,7 +110,7 @@ impl BenchContext {
.full()
.await?
.ok_or_else(|| eyre::eyre!("Failed to fetch latest block from RPC"))?;
let mut benchmark_mode = BenchMode::new(from, to, latest_block.into_inner().number());
let mut benchmark_mode = BenchMode::new(from, to, latest_block.into_inner().number())?;
let first_block = match benchmark_mode {
BenchMode::Continuous(start) => {
@@ -160,16 +140,6 @@ impl BenchContext {
};
let next_block = first_block.header.number + 1;
let rlp_blocks = bench_args.rlp_blocks;
let use_reth_namespace = bench_args.reth_new_payload || rlp_blocks;
Ok(Self {
auth_provider,
block_provider,
benchmark_mode,
next_block,
is_optimism,
use_reth_namespace,
rlp_blocks,
})
Ok(Self { auth_provider, block_provider, benchmark_mode, next_block, is_optimism })
}
}

View File

@@ -6,9 +6,7 @@ use crate::{
helpers::{build_payload, parse_gas_limit, prepare_payload_request, rpc_block_to_header},
output::GasRampPayloadFile,
},
valid_payload::{
call_forkchoice_updated_with_reth, call_new_payload_with_reth, payload_to_new_payload,
},
valid_payload::{call_forkchoice_updated, call_new_payload, payload_to_new_payload},
};
use alloy_eips::BlockNumberOrTag;
use alloy_provider::{network::AnyNetwork, Provider, RootProvider};
@@ -21,7 +19,6 @@ use reth_chainspec::ChainSpec;
use reth_cli_runner::CliContext;
use reth_ethereum_primitives::TransactionSigned;
use reth_primitives_traits::constants::{GAS_LIMIT_BOUND_DIVISOR, MAXIMUM_GAS_LIMIT_BLOCK};
use reth_rpc_api::RethNewPayloadInput;
use std::{path::PathBuf, time::Instant};
use tracing::info;
@@ -50,14 +47,6 @@ pub struct Command {
/// Output directory for benchmark results and generated payloads.
#[arg(long, value_name = "OUTPUT")]
output: PathBuf,
/// Use `reth_newPayload` endpoint instead of `engine_newPayload*`.
///
/// The `reth_newPayload` endpoint is a reth-specific extension that takes `ExecutionData`
/// directly, waits for persistence and cache updates to complete before processing,
/// and returns server-side timing breakdowns (latency, persistence wait, cache wait).
#[arg(long, default_value = "false", verbatim_doc_comment)]
reth_new_payload: bool,
}
/// Mode for determining when to stop ramping.
@@ -149,9 +138,6 @@ impl Command {
);
}
}
if self.reth_new_payload {
info!("Using reth_newPayload and reth_forkchoiceUpdated endpoints");
}
let mut blocks_processed = 0u64;
let total_benchmark_duration = Instant::now();
@@ -177,7 +163,7 @@ impl Command {
// Regenerate the payload from the modified block, but keep the original sidecar
// which contains the actual execution requests data (not just the hash)
let (payload, _) = ExecutionPayload::from_block_unchecked(block_hash, &block);
let (version, params, execution_data) = payload_to_new_payload(
let (version, params) = payload_to_new_payload(
payload,
sidecar,
false,
@@ -185,45 +171,27 @@ impl Command {
Some(new_payload_version),
)?;
let (version, params) = if self.reth_new_payload {
(None, serde_json::to_value((RethNewPayloadInput::ExecutionData(execution_data),))?)
} else {
(Some(version), params)
};
// Save payload to file with version info for replay
let payload_path =
self.output.join(format!("payload_block_{}.json", block.header.number));
let file = GasRampPayloadFile {
version: version.map(|v| v as u8),
block_hash,
params: params.clone(),
};
let file =
GasRampPayloadFile { version: version as u8, block_hash, params: params.clone() };
let payload_json = serde_json::to_string_pretty(&file)?;
std::fs::write(&payload_path, &payload_json)?;
info!(target: "reth-bench", block_number = block.header.number, path = %payload_path.display(), "Saved payload");
let _ = call_new_payload_with_reth(&provider, version, params).await?;
call_new_payload(&provider, version, params).await?;
let forkchoice_state = ForkchoiceState {
head_block_hash: block_hash,
safe_block_hash: block_hash,
finalized_block_hash: block_hash,
};
call_forkchoice_updated_with_reth(&provider, version, forkchoice_state).await?;
call_forkchoice_updated(&provider, version, forkchoice_state, None).await?;
parent_header = block.header;
parent_hash = block_hash;
blocks_processed += 1;
let progress = match mode {
RampMode::Blocks(total) => format!("{blocks_processed}/{total}"),
RampMode::TargetGasLimit(target) => {
let pct = (parent_header.gas_limit as f64 / target as f64 * 100.0).min(100.0);
format!("{pct:.1}%")
}
};
info!(target: "reth-bench", progress, block_number = parent_header.number, gas_limit = parent_header.gas_limit, "Block processed");
}
let final_gas_limit = parent_header.gas_limit;

View File

@@ -1,156 +0,0 @@
//! Prometheus metrics scraper for reth-bench.
//!
//! Scrapes a node's Prometheus metrics endpoint after each block to record
//! execution and state root durations with block-level granularity.
use csv::Writer;
use eyre::Context;
use reqwest::Client;
use serde::Serialize;
use std::{path::Path, time::Duration};
use tracing::info;
/// Suffix for the metrics CSV output file.
pub(crate) const METRICS_OUTPUT_SUFFIX: &str = "metrics.csv";
/// A single row of scraped prometheus metrics for one block.
#[derive(Debug, Clone, Serialize)]
pub(crate) struct MetricsRow {
/// The block number.
pub(crate) block_number: u64,
/// EVM execution duration in seconds (from `sync_execution_execution_duration` gauge).
pub(crate) execution_duration_secs: Option<f64>,
/// State root computation duration in seconds (from
/// `sync_block_validation_state_root_duration` gauge).
pub(crate) state_root_duration_secs: Option<f64>,
}
/// Scrapes a Prometheus metrics endpoint after each block to collect
/// execution and state root durations.
pub(crate) struct MetricsScraper {
/// The full URL of the Prometheus metrics endpoint.
url: String,
/// Reusable HTTP client.
client: Client,
/// Collected metrics rows, one per block.
rows: Vec<MetricsRow>,
}
impl MetricsScraper {
/// Creates a new scraper if a URL is provided.
pub(crate) fn maybe_new(url: Option<String>) -> Option<Self> {
url.map(|url| {
info!(target: "reth-bench", %url, "Prometheus metrics scraping enabled");
let client = Client::builder()
.timeout(Duration::from_secs(5))
.build()
.expect("failed to build reqwest client");
Self { url, client, rows: Vec::new() }
})
}
/// Scrapes the metrics endpoint and records values for the given block.
pub(crate) async fn scrape_after_block(&mut self, block_number: u64) -> eyre::Result<()> {
let text = self
.client
.get(&self.url)
.send()
.await
.wrap_err("failed to fetch metrics endpoint")?
.error_for_status()
.wrap_err("metrics endpoint returned error status")?
.text()
.await
.wrap_err("failed to read metrics response body")?;
let execution = parse_gauge(&text, "sync_execution_execution_duration");
let state_root = parse_gauge(&text, "sync_block_validation_state_root_duration");
self.rows.push(MetricsRow {
block_number,
execution_duration_secs: execution,
state_root_duration_secs: state_root,
});
Ok(())
}
/// Writes collected metrics to a CSV file in the output directory.
pub(crate) fn write_csv(&self, output_dir: &Path) -> eyre::Result<()> {
let path = output_dir.join(METRICS_OUTPUT_SUFFIX);
info!(target: "reth-bench", "Writing scraped metrics to file: {:?}", path);
let mut writer = Writer::from_path(&path)?;
for row in &self.rows {
writer.serialize(row)?;
}
writer.flush()?;
Ok(())
}
}
/// Parses a Prometheus gauge value from exposition-format text.
///
/// Searches for lines starting with `name` followed by either a space or `{`
/// (for labeled metrics), then parses the numeric value. Returns the last
/// matching sample to handle metrics emitted with multiple label sets.
fn parse_gauge(text: &str, name: &str) -> Option<f64> {
let mut result = None;
for line in text.lines() {
let line = line.trim();
if line.is_empty() || line.starts_with('#') {
continue;
}
if !line.starts_with(name) {
continue;
}
// Ensure we match the full metric name, not a prefix of another metric.
let rest = &line[name.len()..];
if !rest.starts_with(' ') && !rest.starts_with('{') {
continue;
}
// Format: `metric_name{labels} value [timestamp]` or `metric_name value [timestamp]`
// Value is always the second whitespace-separated token.
let mut parts = line.split_whitespace();
if let Some(value_str) = parts.nth(1) &&
let Ok(v) = value_str.parse::<f64>()
{
result = Some(v);
}
}
result
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_parse_gauge_simple() {
let text = r#"# HELP sync_execution_execution_duration Duration of execution
# TYPE sync_execution_execution_duration gauge
sync_execution_execution_duration 0.123456
"#;
assert_eq!(parse_gauge(text, "sync_execution_execution_duration"), Some(0.123456));
}
#[test]
fn test_parse_gauge_missing() {
let text = "some_other_metric 1.0\n";
assert_eq!(parse_gauge(text, "sync_execution_execution_duration"), None);
}
#[test]
fn test_parse_gauge_with_labels() {
let text = "sync_block_validation_state_root_duration{instance=\"node1\"} 0.5\n";
assert_eq!(parse_gauge(text, "sync_block_validation_state_root_duration"), Some(0.5));
}
#[test]
fn test_parse_gauge_prefix_no_false_match() {
let text =
"sync_execution_execution_duration_total 99.0\nsync_execution_execution_duration 0.5\n";
assert_eq!(parse_gauge(text, "sync_execution_execution_duration"), Some(0.5));
}
}

View File

@@ -12,7 +12,6 @@ pub(crate) mod helpers;
pub use generate_big_block::{
RawTransaction, RpcTransactionSource, TransactionCollector, TransactionSource,
};
pub(crate) mod metrics_scraper;
mod new_payload_fcu;
mod new_payload_only;
mod output;

View File

@@ -13,7 +13,6 @@ use crate::{
bench::{
context::BenchContext,
helpers::parse_duration,
metrics_scraper::MetricsScraper,
output::{
write_benchmark_results, CombinedResult, NewPayloadResult, TotalGasOutput, TotalGasRow,
},
@@ -21,11 +20,9 @@ use crate::{
derive_ws_rpc_url, setup_persistence_subscription, PersistenceWaiter,
},
},
valid_payload::{
block_to_new_payload, call_forkchoice_updated_with_reth, call_new_payload_with_reth,
},
valid_payload::{block_to_new_payload, call_forkchoice_updated, call_new_payload},
};
use alloy_provider::{ext::DebugApi, Provider};
use alloy_provider::Provider;
use alloy_rpc_types_engine::ForkchoiceState;
use clap::Parser;
use eyre::{Context, OptionExt};
@@ -33,7 +30,7 @@ use reth_cli_runner::CliContext;
use reth_engine_primitives::config::DEFAULT_PERSISTENCE_THRESHOLD;
use reth_node_core::args::BenchmarkArgs;
use std::time::{Duration, Instant};
use tracing::{debug, info, warn};
use tracing::{debug, info};
/// `reth benchmark new-payload-fcu` command
#[derive(Debug, Parser)]
@@ -153,18 +150,9 @@ impl Command {
auth_provider,
mut next_block,
is_optimism,
use_reth_namespace,
rlp_blocks,
..
} = BenchContext::new(&self.benchmark, self.rpc_url).await?;
let total_blocks = benchmark_mode.total_blocks();
let mut metrics_scraper = MetricsScraper::maybe_new(self.benchmark.metrics_url.clone());
if use_reth_namespace {
info!("Using reth_newPayload and reth_forkchoiceUpdated endpoints");
}
let buffer_size = self.rpc_block_buffer_size;
// Use a oneshot channel to propagate errors from the spawned task
@@ -187,21 +175,6 @@ impl Command {
}
};
let rlp = if rlp_blocks {
let rlp = match block_provider.debug_get_raw_block(next_block.into()).await {
Ok(rlp) => rlp,
Err(e) => {
tracing::error!(target: "reth-bench", "Failed to fetch raw block {next_block}: {e}");
let _ = error_sender
.send(eyre::eyre!("Failed to fetch raw block {next_block}: {e}"));
break;
}
};
Some(rlp)
} else {
None
};
let head_block_hash = block.header.hash;
let safe_block_hash = block_provider
.get_block_by_number(block.header.number.saturating_sub(32).into());
@@ -223,7 +196,7 @@ impl Command {
next_block += 1;
if let Err(e) = sender
.send((block, head_block_hash, safe_block_hash, finalized_block_hash, rlp))
.send((block, head_block_hash, safe_block_hash, finalized_block_hash))
.await
{
tracing::error!(target: "reth-bench", "Failed to send block data: {e}");
@@ -233,11 +206,10 @@ impl Command {
});
let mut results = Vec::new();
let mut blocks_processed = 0u64;
let total_benchmark_duration = Instant::now();
let mut total_wait_time = Duration::ZERO;
while let Some((block, head, safe, finalized, rlp)) = {
while let Some((block, head, safe, finalized)) = {
let wait_start = Instant::now();
let result = receiver.recv().await;
total_wait_time += wait_start.elapsed();
@@ -256,40 +228,16 @@ impl Command {
finalized_block_hash: finalized,
};
let (version, params) =
block_to_new_payload(block, is_optimism, rlp, use_reth_namespace)?;
let (version, params) = block_to_new_payload(block, is_optimism)?;
let start = Instant::now();
let server_timings =
call_new_payload_with_reth(&auth_provider, version, params).await?;
call_new_payload(&auth_provider, version, params).await?;
let np_latency =
server_timings.as_ref().map(|t| t.latency).unwrap_or_else(|| start.elapsed());
let new_payload_result = NewPayloadResult {
gas_used,
latency: np_latency,
persistence_wait: server_timings.as_ref().and_then(|t| t.persistence_wait),
execution_cache_wait: server_timings
.as_ref()
.map(|t| t.execution_cache_wait)
.unwrap_or_default(),
sparse_trie_wait: server_timings
.as_ref()
.map(|t| t.sparse_trie_wait)
.unwrap_or_default(),
};
let new_payload_result = NewPayloadResult { gas_used, latency: start.elapsed() };
let fcu_start = Instant::now();
call_forkchoice_updated_with_reth(&auth_provider, version, forkchoice_state).await?;
let fcu_latency = fcu_start.elapsed();
call_forkchoice_updated(&auth_provider, version, forkchoice_state, None).await?;
let total_latency = if server_timings.is_some() {
// When using server-side latency for newPayload, derive total from the
// independently measured components to avoid mixing server-side and
// client-side (network-inclusive) timings.
np_latency + fcu_latency
} else {
start.elapsed()
};
let total_latency = start.elapsed();
let fcu_latency = total_latency - new_payload_result.latency;
let combined_result = CombinedResult {
block_number,
gas_limit,
@@ -301,19 +249,8 @@ impl Command {
// Exclude time spent waiting on the block prefetch channel from the benchmark duration.
// We want to measure engine throughput, not RPC fetch latency.
blocks_processed += 1;
let current_duration = total_benchmark_duration.elapsed() - total_wait_time;
let progress = match total_blocks {
Some(total) => format!("{blocks_processed}/{total}"),
None => format!("{blocks_processed}"),
};
info!(target: "reth-bench", progress, %combined_result);
if let Some(scraper) = metrics_scraper.as_mut() &&
let Err(err) = scraper.scrape_after_block(block_number).await
{
warn!(target: "reth-bench", %err, block_number, "Failed to scrape metrics");
}
info!(target: "reth-bench", %combined_result);
if let Some(w) = &mut waiter {
w.on_block(block_number).await?;
@@ -340,10 +277,6 @@ impl Command {
write_benchmark_results(path, &gas_output_results, &combined_results)?;
}
if let (Some(path), Some(scraper)) = (&self.benchmark.output, &metrics_scraper) {
scraper.write_csv(path)?;
}
let gas_output =
TotalGasOutput::with_combined_results(gas_output_results, &combined_results)?;

View File

@@ -3,15 +3,14 @@
use crate::{
bench::{
context::BenchContext,
metrics_scraper::MetricsScraper,
output::{
NewPayloadResult, TotalGasOutput, TotalGasRow, GAS_OUTPUT_SUFFIX,
NEW_PAYLOAD_OUTPUT_SUFFIX,
},
},
valid_payload::{block_to_new_payload, call_new_payload_with_reth},
valid_payload::{block_to_new_payload, call_new_payload},
};
use alloy_provider::{ext::DebugApi, Provider};
use alloy_provider::Provider;
use clap::Parser;
use csv::Writer;
use eyre::{Context, OptionExt};
@@ -50,18 +49,9 @@ impl Command {
auth_provider,
mut next_block,
is_optimism,
use_reth_namespace,
rlp_blocks,
..
} = BenchContext::new(&self.benchmark, self.rpc_url).await?;
let total_blocks = benchmark_mode.total_blocks();
let mut metrics_scraper = MetricsScraper::maybe_new(self.benchmark.metrics_url.clone());
if use_reth_namespace {
info!("Using reth_newPayload endpoint");
}
let buffer_size = self.rpc_block_buffer_size;
// Use a oneshot channel to propagate errors from the spawned task
@@ -84,33 +74,20 @@ impl Command {
}
};
let rlp = if rlp_blocks {
let Ok(rlp) = block_provider.debug_get_raw_block(next_block.into()).await
else {
tracing::error!(target: "reth-bench", "Failed to fetch raw block {next_block}");
let _ = error_sender
.send(eyre::eyre!("Failed to fetch raw block {next_block}"));
break;
};
Some(rlp)
} else {
None
};
next_block += 1;
if let Err(e) = sender.send((block, rlp)).await {
if let Err(e) = sender.send(block).await {
tracing::error!(target: "reth-bench", "Failed to send block data: {e}");
break;
}
}
});
// put results in a summary vec so they can be printed at the end
let mut results = Vec::new();
let mut blocks_processed = 0u64;
let total_benchmark_duration = Instant::now();
let mut total_wait_time = Duration::ZERO;
while let Some((block, rlp)) = {
while let Some(block) = {
let wait_start = Instant::now();
let result = receiver.recv().await;
total_wait_time += wait_start.elapsed();
@@ -122,34 +99,13 @@ impl Command {
debug!(target: "reth-bench", number=?block.header.number, "Sending payload to engine");
let (version, params) =
block_to_new_payload(block, is_optimism, rlp, use_reth_namespace)?;
let (version, params) = block_to_new_payload(block, is_optimism)?;
let start = Instant::now();
let server_timings =
call_new_payload_with_reth(&auth_provider, version, params).await?;
call_new_payload(&auth_provider, version, params).await?;
let latency =
server_timings.as_ref().map(|t| t.latency).unwrap_or_else(|| start.elapsed());
let new_payload_result = NewPayloadResult {
gas_used,
latency,
persistence_wait: server_timings.as_ref().and_then(|t| t.persistence_wait),
execution_cache_wait: server_timings
.as_ref()
.map(|t| t.execution_cache_wait)
.unwrap_or_default(),
sparse_trie_wait: server_timings
.as_ref()
.map(|t| t.sparse_trie_wait)
.unwrap_or_default(),
};
blocks_processed += 1;
let progress = match total_blocks {
Some(total) => format!("{blocks_processed}/{total}"),
None => format!("{blocks_processed}"),
};
info!(target: "reth-bench", progress, %new_payload_result);
let new_payload_result = NewPayloadResult { gas_used, latency: start.elapsed() };
info!(target: "reth-bench", %new_payload_result);
// current duration since the start of the benchmark minus the time
// waiting for blocks
@@ -159,12 +115,6 @@ impl Command {
let row =
TotalGasRow { block_number, transaction_count, gas_used, time: current_duration };
results.push((row, new_payload_result));
if let Some(scraper) = metrics_scraper.as_mut() &&
let Err(err) = scraper.scrape_after_block(block_number).await
{
tracing::warn!(target: "reth-bench", %err, block_number, "Failed to scrape metrics");
}
}
// Check if the spawned task encountered an error
@@ -195,10 +145,6 @@ impl Command {
}
writer.flush()?;
if let Some(scraper) = &metrics_scraper {
scraper.write_csv(&path)?;
}
info!(target: "reth-bench", "Finished writing benchmark output files to {:?}.", path);
}

View File

@@ -22,10 +22,7 @@ pub(crate) const NEW_PAYLOAD_OUTPUT_SUFFIX: &str = "new_payload_latency.csv";
#[derive(Debug, Serialize, Deserialize)]
pub(crate) struct GasRampPayloadFile {
/// Engine API version (1-5).
///
/// `None` indicates that `reth_newPayload` should be used.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub(crate) version: Option<u8>,
pub(crate) version: u8,
/// The block hash for FCU.
pub(crate) block_hash: B256,
/// The params to pass to newPayload.
@@ -40,12 +37,6 @@ pub(crate) struct NewPayloadResult {
pub(crate) gas_used: u64,
/// The latency of the `newPayload` call.
pub(crate) latency: Duration,
/// Time spent waiting for persistence. `None` when no persistence was in-flight.
pub(crate) persistence_wait: Option<Duration>,
/// Time spent waiting for execution cache lock.
pub(crate) execution_cache_wait: Duration,
/// Time spent waiting for sparse trie lock.
pub(crate) sparse_trie_wait: Duration,
}
impl NewPayloadResult {
@@ -76,12 +67,9 @@ impl Serialize for NewPayloadResult {
{
// convert the time to microseconds
let time = self.latency.as_micros();
let mut state = serializer.serialize_struct("NewPayloadResult", 5)?;
let mut state = serializer.serialize_struct("NewPayloadResult", 2)?;
state.serialize_field("gas_used", &self.gas_used)?;
state.serialize_field("latency", &time)?;
state.serialize_field("persistence_wait", &self.persistence_wait.map(|d| d.as_micros()))?;
state.serialize_field("execution_cache_wait", &self.execution_cache_wait.as_micros())?;
state.serialize_field("sparse_trie_wait", &self.sparse_trie_wait.as_micros())?;
state.end()
}
}
@@ -114,27 +102,16 @@ impl CombinedResult {
impl std::fmt::Display for CombinedResult {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
let np = &self.new_payload_result;
write!(
f,
"Block {} processed at {:.4} Ggas/s, used {} total gas. Combined: {:.4} Ggas/s. fcu: {:?}, newPayload: {:?}",
self.block_number,
np.gas_per_second() / GIGAGAS as f64,
np.gas_used,
self.new_payload_result.gas_per_second() / GIGAGAS as f64,
self.new_payload_result.gas_used,
self.combined_gas_per_second() / GIGAGAS as f64,
self.fcu_latency,
np.latency,
)?;
if !np.execution_cache_wait.is_zero() {
write!(f, ", execution cache wait: {:?}", np.execution_cache_wait)?;
}
if !np.sparse_trie_wait.is_zero() {
write!(f, ", trie cache wait: {:?}", np.sparse_trie_wait)?;
}
if let Some(d) = np.persistence_wait {
write!(f, ", persistence wait: {d:?}")?;
}
Ok(())
self.new_payload_result.latency
)
}
}
@@ -149,7 +126,7 @@ impl Serialize for CombinedResult {
let fcu_latency = self.fcu_latency.as_micros();
let new_payload_latency = self.new_payload_result.latency.as_micros();
let total_latency = self.total_latency.as_micros();
let mut state = serializer.serialize_struct("CombinedResult", 10)?;
let mut state = serializer.serialize_struct("CombinedResult", 7)?;
// flatten the new payload result because this is meant for CSV writing
state.serialize_field("block_number", &self.block_number)?;
@@ -159,18 +136,6 @@ impl Serialize for CombinedResult {
state.serialize_field("new_payload_latency", &new_payload_latency)?;
state.serialize_field("fcu_latency", &fcu_latency)?;
state.serialize_field("total_latency", &total_latency)?;
state.serialize_field(
"persistence_wait",
&self.new_payload_result.persistence_wait.map(|d| d.as_micros()),
)?;
state.serialize_field(
"execution_cache_wait",
&self.new_payload_result.execution_cache_wait.as_micros(),
)?;
state.serialize_field(
"sparse_trie_wait",
&self.new_payload_result.sparse_trie_wait.as_micros(),
)?;
state.end()
}
}

View File

@@ -15,7 +15,6 @@ use crate::{
authenticated_transport::AuthenticatedTransportConnect,
bench::{
helpers::parse_duration,
metrics_scraper::MetricsScraper,
output::{
write_benchmark_results, CombinedResult, GasRampPayloadFile, NewPayloadResult,
TotalGasOutput, TotalGasRow,
@@ -24,21 +23,17 @@ use crate::{
derive_ws_rpc_url, setup_persistence_subscription, PersistenceWaiter,
},
},
valid_payload::{call_forkchoice_updated_with_reth, call_new_payload_with_reth},
valid_payload::{call_forkchoice_updated, call_new_payload},
};
use alloy_primitives::B256;
use alloy_provider::{network::AnyNetwork, Provider, RootProvider};
use alloy_provider::{ext::EngineApi, network::AnyNetwork, Provider, RootProvider};
use alloy_rpc_client::ClientBuilder;
use alloy_rpc_types_engine::{
CancunPayloadFields, ExecutionData, ExecutionPayloadEnvelopeV4, ExecutionPayloadSidecar,
ForkchoiceState, JwtSecret, PraguePayloadFields,
};
use alloy_rpc_types_engine::{ExecutionPayloadEnvelopeV4, ForkchoiceState, JwtSecret};
use clap::Parser;
use eyre::Context;
use reth_cli_runner::CliContext;
use reth_engine_primitives::config::DEFAULT_PERSISTENCE_THRESHOLD;
use reth_node_api::EngineApiMessageVersion;
use reth_rpc_api::RethNewPayloadInput;
use std::{
path::PathBuf,
time::{Duration, Instant},
@@ -129,22 +124,6 @@ pub struct Command {
/// If not provided, derives from engine RPC URL by changing scheme to ws and port to 8546.
#[arg(long, value_name = "WS_RPC_URL", verbatim_doc_comment)]
ws_rpc_url: Option<String>,
/// Use `reth_newPayload` endpoint instead of `engine_newPayload*`.
///
/// The `reth_newPayload` endpoint is a reth-specific extension that takes `ExecutionData`
/// directly, waits for persistence and cache updates to complete before processing,
/// and returns server-side timing breakdowns (latency, persistence wait, cache wait).
#[arg(long, default_value = "false", verbatim_doc_comment)]
reth_new_payload: bool,
/// Optional Prometheus metrics endpoint to scrape after each block.
///
/// When provided, reth-bench will fetch metrics from this URL after each
/// payload, recording per-block execution and state root durations.
/// Results are written to `metrics.csv` in the output directory.
#[arg(long = "metrics-url", value_name = "URL", verbatim_doc_comment)]
metrics_url: Option<String>,
}
/// A loaded payload ready for execution.
@@ -162,9 +141,7 @@ struct GasRampPayload {
/// Block number from filename.
block_number: u64,
/// Engine API version for newPayload.
///
/// `None` indicates that `reth_newPayload` should be used.
version: Option<EngineApiMessageVersion>,
version: EngineApiMessageVersion,
/// The file contents.
file: GasRampPayloadFile,
}
@@ -186,9 +163,6 @@ impl Command {
self.persistence_threshold
);
}
if self.reth_new_payload {
info!("Using reth_newPayload and reth_forkchoiceUpdated endpoints");
}
// Set up waiter based on configured options
// When both are set: wait at least wait_time, and also wait for persistence if needed
@@ -216,8 +190,6 @@ impl Command {
(None, false) => None,
};
let mut metrics_scraper = MetricsScraper::maybe_new(self.metrics_url.clone());
// Set up authenticated engine provider
let jwt =
std::fs::read_to_string(&self.jwt_secret).wrap_err("Failed to read JWT secret file")?;
@@ -276,22 +248,21 @@ impl Command {
"Executing gas ramp payload (newPayload + FCU)"
);
let _ = call_new_payload_with_reth(
&auth_provider,
payload.version,
payload.file.params.clone(),
)
.await?;
call_new_payload(&auth_provider, payload.version, payload.file.params.clone()).await?;
let fcu_state = ForkchoiceState {
head_block_hash: payload.file.block_hash,
safe_block_hash: parent_hash,
finalized_block_hash: parent_hash,
};
call_forkchoice_updated_with_reth(&auth_provider, payload.version, fcu_state).await?;
call_forkchoice_updated(&auth_provider, payload.version, fcu_state, None).await?;
info!(target: "reth-bench", gas_ramp_payload = i + 1, "Gas ramp payload executed successfully");
if let Some(w) = &mut waiter {
w.on_block(payload.block_number).await?;
}
parent_hash = payload.file.block_hash;
}
@@ -332,50 +303,20 @@ impl Command {
"Sending newPayload"
);
let (version, params) = if self.reth_new_payload {
let reth_data = ExecutionData {
payload: execution_payload.clone().into(),
sidecar: ExecutionPayloadSidecar::v4(
CancunPayloadFields {
versioned_hashes: Vec::new(),
parent_beacon_block_root: B256::ZERO,
},
PraguePayloadFields {
requests: envelope.execution_requests.clone().into(),
},
),
};
(None, serde_json::to_value((RethNewPayloadInput::ExecutionData(reth_data),))?)
} else {
(
Some(EngineApiMessageVersion::V4),
serde_json::to_value((
execution_payload.clone(),
Vec::<B256>::new(),
B256::ZERO,
envelope.execution_requests.to_vec(),
))?,
let status = auth_provider
.new_payload_v4(
execution_payload.clone(),
vec![],
B256::ZERO,
envelope.execution_requests.to_vec(),
)
};
.await?;
let server_timings =
call_new_payload_with_reth(&auth_provider, version, params).await?;
let new_payload_result = NewPayloadResult { gas_used, latency: start.elapsed() };
let np_latency =
server_timings.as_ref().map(|t| t.latency).unwrap_or_else(|| start.elapsed());
let new_payload_result = NewPayloadResult {
gas_used,
latency: np_latency,
persistence_wait: server_timings.as_ref().and_then(|t| t.persistence_wait),
execution_cache_wait: server_timings
.as_ref()
.map(|t| t.execution_cache_wait)
.unwrap_or_default(),
sparse_trie_wait: server_timings
.as_ref()
.map(|t| t.sparse_trie_wait)
.unwrap_or_default(),
};
if !status.is_valid() {
return Err(eyre::eyre!("Payload rejected: {:?}", status));
}
let fcu_state = ForkchoiceState {
head_block_hash: block_hash,
@@ -383,12 +324,12 @@ impl Command {
finalized_block_hash: parent_hash,
};
let fcu_start = Instant::now();
call_forkchoice_updated_with_reth(&auth_provider, version, fcu_state).await?;
let fcu_latency = fcu_start.elapsed();
debug!(target: "reth-bench", method = "engine_forkchoiceUpdatedV3", ?fcu_state, "Sending forkchoiceUpdated");
let total_latency =
if server_timings.is_some() { np_latency + fcu_latency } else { start.elapsed() };
let fcu_result = auth_provider.fork_choice_updated_v3(fcu_state, None).await?;
let total_latency = start.elapsed();
let fcu_latency = total_latency - new_payload_result.latency;
let combined_result = CombinedResult {
block_number,
@@ -400,14 +341,7 @@ impl Command {
};
let current_duration = total_benchmark_duration.elapsed();
let progress = format!("{}/{}", i + 1, payloads.len());
info!(target: "reth-bench", progress, %combined_result);
if let Some(scraper) = metrics_scraper.as_mut() &&
let Err(err) = scraper.scrape_after_block(block_number).await
{
tracing::warn!(target: "reth-bench", %err, block_number, "Failed to scrape metrics");
}
info!(target: "reth-bench", %combined_result);
if let Some(w) = &mut waiter {
w.on_block(block_number).await?;
@@ -417,6 +351,7 @@ impl Command {
TotalGasRow { block_number, transaction_count, gas_used, time: current_duration };
results.push((gas_row, combined_result));
debug!(target: "reth-bench", ?status, ?fcu_result, "Payload executed successfully");
parent_hash = block_hash;
}
@@ -431,10 +366,6 @@ impl Command {
write_benchmark_results(path, &gas_output_results, &combined_results)?;
}
if let (Some(path), Some(scraper)) = (&self.output, &metrics_scraper) {
scraper.write_csv(path)?;
}
let gas_output =
TotalGasOutput::with_combined_results(gas_output_results, &combined_results)?;
info!(
@@ -545,18 +476,13 @@ impl Command {
let file: GasRampPayloadFile = serde_json::from_str(&content)
.wrap_err_with(|| format!("Failed to parse {:?}", path))?;
let version = if let Some(version) = file.version {
match version {
1 => EngineApiMessageVersion::V1,
2 => EngineApiMessageVersion::V2,
3 => EngineApiMessageVersion::V3,
4 => EngineApiMessageVersion::V4,
5 => EngineApiMessageVersion::V5,
v => return Err(eyre::eyre!("Invalid version {} in {:?}", v, path)),
}
.into()
} else {
None
let version = match file.version {
1 => EngineApiMessageVersion::V1,
2 => EngineApiMessageVersion::V2,
3 => EngineApiMessageVersion::V3,
4 => EngineApiMessageVersion::V4,
5 => EngineApiMessageVersion::V5,
v => return Err(eyre::eyre!("Invalid version {} in {:?}", v, path)),
};
info!(

View File

@@ -20,30 +20,18 @@ impl BenchMode {
}
}
/// Returns the total number of blocks in the benchmark, if known.
///
/// For [`BenchMode::Range`] this is the length of the range.
/// For [`BenchMode::Continuous`] the total is unbounded, so `None` is returned.
pub const fn total_blocks(&self) -> Option<u64> {
match self {
Self::Continuous(_) => None,
Self::Range(range) => {
Some(range.end().saturating_sub(*range.start()).saturating_add(1))
}
}
}
/// Create a [`BenchMode`] from optional `from` and `to` fields.
///
/// If only `--to` is provided, `from` is derived as `latest_block + 1`.
pub const fn new(from: Option<u64>, to: Option<u64>, latest_block: u64) -> Self {
pub fn new(from: Option<u64>, to: Option<u64>, latest_block: u64) -> Result<Self, eyre::Error> {
// If neither `--from` nor `--to` are provided, we will run the benchmark continuously,
// starting at the latest block.
match (from, to) {
(Some(from), Some(to)) => Self::Range(from..=to),
(None, None) => Self::Continuous(latest_block),
(Some(start), None) => Self::Continuous(start),
(None, Some(to)) => Self::Range(latest_block + 1..=to),
(Some(from), Some(to)) => Ok(Self::Range(from..=to)),
(None, None) => Ok(Self::Continuous(latest_block)),
(Some(start), None) => Ok(Self::Continuous(start)),
_ => {
// both or neither are allowed, everything else is ambiguous
Err(eyre::eyre!("`from` and `to` must be provided together, or not at all."))
}
}
}
}

Some files were not shown because too many files have changed in this diff Show More