Merge branch 'main' into dan/static-file-split

Co-authored-by: Amp <amp@ampcode.com>

.changelog/cool-owls-rest.md

@@ -0,0 +1,5 @@
+---
+reth-trie-sparse: patch
+---
+
+Fixed a bug in `merge_subtrie_updates` where source insertions did not cancel destination removals (and vice versa), causing inconsistent trie updates accumulated across multiple `root()` calls without intermediate `take_updates()`. Added a test covering the cross-cancellation behavior.

.changelog/dark-ants-write.md

@@ -0,0 +1,5 @@
+---
+reth-tasks: patch
+---
+
+Added panic handler to all rayon thread pools that logs panics via `tracing::error` instead of aborting the process.

.changelog/dry-koalas-wink.md

@@ -0,0 +1,5 @@
+---
+reth-network-types: patch
+---
+
+Increased default maximum concurrent outbound dials from 15 to 30.

.changelog/dry-lakes-sleep.md

@@ -0,0 +1,6 @@
+---
+reth-trie-common: minor
+reth-trie: minor
+---
+
+Added `contains_range` method to `PrefixSet` for checking if any key falls within a half-open range. Added prefix set support to `ProofCalculator` via `with_prefix_set`, enabling stale cached hash invalidation and branch collapse detection when keys are inserted or removed; propagated storage prefix sets through `SyncAccountValueEncoder`.

.changelog/dull-clams-pack.md

@@ -0,0 +1,5 @@
+---
+reth-trie-sparse: patch
+---
+
+Refactored arena trie internals by adding a `BranchChildIdx::sibling()` helper, deduplicating `Index`/`NodeArena` type aliases, and replacing `is_empty()` with a `drop_root()` method. Fixed a bug where `cursor.pop()` was called before checking if the leaf was the root node, which could cause incorrect dirty-state propagation.

.changelog/dull-seals-laugh.md

@@ -0,0 +1,5 @@
+---
+reth-trie-sparse: minor
+---
+
+Fixed a bug in `ArenaParallelSparseTrie` where subtrie updates that would completely empty a subtrie were incorrectly dispatched to parallel workers instead of being processed inline, preventing correct branch collapse detection when blinded siblings are present. Refactored the `SparseTrie` test suite to accept a `fn() -> T` factory instead of requiring `T: Default`, enabling a new `arena_parallel_sparse_trie_always_parallel` test variant that exercises all tests with parallelism thresholds set to 1. Added `test_branch_collapse_multi_empty_subtries_blinded_remaining` to cover the case where removing multiple revealed leaves empties their subtries and leaves a single blinded sibling requiring a proof.

.changelog/fair-wolves-smile.md

@@ -0,0 +1,5 @@
+---
+reth-payload-builder: minor
+---
+
+Added observability metrics for payload resolve latency and new payload job creation latency to the payload builder service.

.changelog/fine-horses-whisper.md

@@ -0,0 +1,5 @@
+---
+reth-trie-sparse: patch
+---
+
+Fixed another branch collapse edge case where `check_subtrie_collapse_needs_proof` incorrectly compared removal count against total update count (including `Touched` entries), causing it to skip proof requests for blinded siblings and panic when the subtrie emptied. Added a regression test covering the removals + `Touched` + blinded sibling scenario.

.changelog/fine-koalas-shout.md

@@ -0,0 +1,10 @@
+---
+reth-chain-state: minor
+reth-engine-primitives: minor
+reth-engine-tree: minor
+reth-node-core: minor
+reth-node-events: minor
+reth: patch
+---
+
+Added configurable slow block logging (`--engine.slow-block-threshold`) that emits a structured `warn!` log with detailed timing, state-operation counts, and cache hit-rate metrics for blocks whose total processing time exceeds the threshold. Introduced `ExecutionTimingStats`, `CacheStats`, `StateProviderStats`, and `SlowBlockInfo` types to carry execution statistics from block validation through persistence, and refactored `PersistenceResult` to carry commit duration alongside the last persisted block.

.changelog/fix-download-api-url.md

@@ -0,0 +1,5 @@
+---
+reth-cli-commands: patch
+---
+
+Added `snapshot_api_url` field to `DownloadDefaults` so downstream projects can override the snapshot discovery API endpoint. Previously, `discover_manifest_url`, `fetch_snapshot_api_entries`, and `print_snapshot_listing` used a hardcoded `snapshots.reth.rs` URL that bypassed the `DownloadDefaults` override mechanism.

.changelog/icy-tigers-buzz.md

@@ -0,0 +1,6 @@
+---
+reth-trie-db: minor
+reth-engine-tree: minor
+---
+
+Added `PendingChangeset` and `PendingChangesetGuard` to `ChangesetCache` so concurrent readers wait for an in-progress computation instead of falling back to the expensive DB-based path. The guard automatically cancels the pending entry on drop (e.g. task panic), ensuring waiters always make progress.

.changelog/kind-eagles-hum.md

@@ -0,0 +1,6 @@
+---
+reth-primitives-traits: major
+reth-downloaders: patch
+---
+
+Removed the local `size` module from `reth-primitives-traits` and replaced it with `alloy_consensus::InMemorySize`. Simplified `SignedTransaction` to a blanket impl covering all types satisfying the required bounds, removing `is_system_tx`, `auto_impl` attributes, and explicit impls for `EthereumTxEnvelope` and OP types. Updated import paths in `reth-downloaders` accordingly.

.changelog/kind-frogs-meow.md

@@ -0,0 +1,13 @@
+---
+reth-primitives-traits: minor
+reth-engine-local: patch
+reth-evm: patch
+reth-node-builder: patch
+reth-payload-primitives: patch
+reth-rpc-convert: patch
+reth-rpc-eth-api: patch
+reth-db-api: patch
+reth-db: patch
+---
+
+Removed the unused `Extended` type and `op` feature (including `op-alloy-consensus` dependency) from `reth-primitives-traits`. Updated all dependent crates to remove the now-unnecessary `reth-primitives-traits/op` feature flag propagation.

.changelog/merry-bees-break.md

@@ -0,0 +1,8 @@
+---
+reth-engine-primitives: minor
+reth-engine-tree: minor
+reth-node-core: minor
+reth-trie-parallel: minor
+---
+
+Added `--engine.proof-jitter` CLI option behind the `trie-debug` feature flag. When set, each proof worker sleeps for a random duration up to the specified value before starting proof computation, useful for stress-testing timing-sensitive proof logic.

.changelog/mild-foxes-cook.md

@@ -0,0 +1,5 @@
+---
+reth-rpc-convert: patch
+---
+
+Updated `alloy-evm` dependency to git revision `9bc2dba` and adapted `TxEnvConverter` impl to the updated `TryIntoTxEnv` trait signature that now includes a `Spec` generic parameter.

.changelog/neat-ducks-whisper.md

@@ -0,0 +1,6 @@
+---
+reth-trie: patch
+reth-trie-sparse: patch
+---
+
+Refactored test harness for sparse trie tests by extracting `TrieTestHarness` into a shared `reth-trie` test utility, replacing duplicated inline harness code across multiple test modules. Updated `proof_v2` return type to include an optional root hash, and converted `original_root` and `storage` from public fields to accessor methods.

.changelog/nice-bears-cook.md

@@ -0,0 +1,5 @@
+---
+reth-trie: patch
+---
+
+Removed the local `increment_and_strip_trailing_zeros` function and `PATH_ALL_ZEROS` static in `proof_v2`, replacing them with the equivalent `Nibbles::next_without_prefix` and `Nibbles::is_zeroes` builtins. Also replaced manual `.get()` calls on `state_mask`/`hash_mask` with direct field access and switched to `Nibbles::unpack_array` over the unsafe `unpack_unchecked`.

.changelog/nice-whales-buzz.md

@@ -0,0 +1,5 @@
+---
+reth-engine-tree: patch
+---
+
+Added idle-time pre-computation of account trie upper hashes in the sparse trie payload processor when no pending proof results are available.

.changelog/odd-birds-meow.md

@@ -0,0 +1,6 @@
+---
+reth-engine-local: patch
+reth-node-builder: patch
+---
+
+Removed the `op` feature flag and `OpPayloadAttributes` `PayloadAttributesBuilder` implementation from `reth-engine-local`, along with the `op-alloy-rpc-types-engine` dependency. Updated `reth-node-builder` to no longer enable the removed `op` feature on `reth-engine-local`.

.changelog/odd-frogs-break.md

@@ -0,0 +1,7 @@
+---
+reth-cli-commands: minor
+reth-node-core: minor
+reth: patch
+---
+
+Made v2 storage the default for all new databases, deprecating the `--storage.v2` flag to a hidden no-op kept for backwards compatibility. Updated CLI reference docs to remove the now-hidden flag from all command help pages.

.changelog/proud-slugs-fly.md

@@ -0,0 +1,7 @@
+---
+reth-engine-tree: patch
+reth-trie-sparse: patch
+reth-tasks: patch
+---
+
+Offloaded deallocation of expensive proof node buffers to a persistent background thread (`Runtime::spawn_drop`) to avoid blocking state root computation or lock-holding code.

.changelog/rare-tigers-nod.md

@@ -0,0 +1,6 @@
+---
+reth-trie-sparse: patch
+reth-engine-tree: patch
+---
+
+Removed the `skip_proof_node_filtering` flag, `revealed_account_paths`/`revealed_paths` tracking, and the `filter_revealed_v2_proof_nodes` function from the sparse trie implementation. Also removed the corresponding skipped-nodes metrics, simplifying the proof node reveal path to always pass nodes directly to the sparse trie without pre-filtering.

.changelog/rare-whales-pack.md

@@ -0,0 +1,5 @@
+---
+reth-trie-sparse: minor
+---
+
+Added a comprehensive generic `SparseTrie` test suite covering `set_root`, `reveal_nodes`, `update_leaves`, `root`, `take_updates`, `commit_updates`, `prune`, `wipe`/`clear`, `get_leaf_value`, `find_leaf`, `size_hint`, and integration lifecycle scenarios. Tests are stamped out for all concrete `SparseTrie` implementations via a macro.

.changelog/rich-pigs-shake.md

@@ -0,0 +1,6 @@
+---
+reth-payload-primitives: patch
+reth-engine-local: patch
+---
+
+Removed the `op` feature and `op-alloy-rpc-types-engine` dependency from `reth-payload-primitives`, along with the `ExecutionPayload` impl for `OpExecutionData`. Updated `reth-engine-local` to drop the corresponding feature flag dependency.

.changelog/safe-waves-cook.md

@@ -0,0 +1,5 @@
+---
+reth-engine-tree: patch
+---
+
+Downgraded per-transaction prewarm span from `debug_span!` to `trace_span!` to reduce noise in debug-level logging.

.changelog/tall-fish-climb.md

@@ -0,0 +1,5 @@
+---
+reth-cli-commands: minor
+---
+
+Added `reth_version` field to `SnapshotManifest` to record the Reth version that produced a snapshot. The field is optional and populated automatically during manifest generation.

.changelog/tidy-ducks-bake.md

@@ -0,0 +1,5 @@
+---
+reth-trie-sparse: patch
+---
+
+Fixed a panic in `ParallelSparseTrie::reveal_nodes` when a boundary node's upper parent is absent or non-branch (e.g. when an upper extension crosses the boundary). The code now skips gracefully instead of unwrapping. Added a regression test covering this case.

.changelog/vain-bats-bark.md

@@ -0,0 +1,8 @@
+---
+reth-trie-common: minor
+reth-trie: minor
+reth-trie-parallel: minor
+reth-engine-tree: patch
+---
+
+Moved `ProofV2Target`, `MultiProofTargetsV2`, and `ChunkedMultiProofTargetsV2` from `reth-trie-parallel::targets_v2` into a new `reth-trie-common::target_v2` module, making these types available at a lower level without pulling in the full parallel trie crate. Added a `multiproof_v2` method to `Proof` in `reth-trie` that generates a state multiproof using the V2 proof calculator with synchronous account value encoding.

.changelog/vain-birds-laugh.md

@@ -0,0 +1,14 @@
+---
+reth-engine-tree: patch
+reth-evm-ethereum: patch
+reth-evm: patch
+reth-primitives-traits: patch
+reth-revm: patch
+reth-rpc-eth-api: patch
+reth-rpc-eth-types: patch
+reth-provider: patch
+example-custom-evm: patch
+example-precompile-cache: patch
+---
+
+Bumped revm to v35.0.0, revm-inspectors to 0.35.0, and alloy-evm to 0.29.0. Updated call sites throughout the codebase to align with the new APIs, including `ExecutionResult` field changes (`gas_used` → `gas.used()`/`gas.final_refunded()`), removal of `.without_state_clear()`, updated `EthPrecompiles::new(spec)` constructor, and updated `block_hashes.lowest()` access.

.changelog/vain-goats-cook.md

@@ -0,0 +1,9 @@
+---
+reth-trie-sparse: minor
+reth-engine-primitives: minor
+reth-engine-tree: minor
+reth-node-core: minor
+reth-trie-common: patch
+---
+
+Added an arena-based sparse trie implementation (`ArenaParallelSparseTrie`) using `slotmap` arena allocation for node storage, enabling parallel subtrie mutation without per-node hashing overhead. Added `ConfigurableSparseTrie` enum to switch between the arena and hash-map implementations, and a `--engine.enable-arena-sparse-trie` CLI flag to opt in at runtime.

.changelog/warm-slugs-crawl.md

@@ -0,0 +1,5 @@
+---
+reth-transaction-pool: minor
+---
+
+Added `TransactionValidationTaskExecutor::spawn` as a dedicated constructor that encapsulates spawning validation tasks on a runtime, and refactored `EthTransactionValidatorBuilder::build_with_tasks` to use it.

.changelog/warm-trees-grow.md

@@ -0,0 +1,7 @@
+---
+reth-engine-primitives: patch
+reth-engine-tree: patch
+reth-node-core: patch
+---
+
+Removed `--engine.enable-arena-sparse-trie` CLI flag and made the arena-based sparse trie the default implementation. The hash-map-based `ParallelSparseTrie` variant is no longer selectable.

.changelog/young-horses-play.md

@@ -0,0 +1,7 @@
+---
+reth-trie: major
+reth-trie-db: major
+reth-provider: minor
+---
+
+Added `MaskedTrieCursorFactory` and `MaskedTrieCursor` to handle prefix-set-based hash invalidation at the cursor layer, replacing the `DatabaseTrieWitness` trait abstraction. Removed `with_prefix_sets_mut` from `TrieWitness` and deleted `DatabaseTrieWitness` — callers should now wrap their cursor factory with `MaskedTrieCursorFactory` to apply prefix sets during witness/proof computation.

.changelog/zesty-bees-drink.md

@@ -0,0 +1,8 @@
+---
+reth-engine-primitives: minor
+reth-engine-tree: major
+reth-node-core: minor
+reth-cli-commands: minor
+---
+
+Added persistence backpressure to the engine tree: when the canonical-minus-persisted block gap exceeds a configurable threshold (`--engine.persistence-backpressure-threshold`, default 16), the engine loop stalls on the persistence receiver instead of processing new incoming messages. Added CLI argument, cross-field validation, metrics (`backpressure_active`, `backpressure_stall_duration`), and tests.

.changelog/zesty-clouds-wave.md

@@ -0,0 +1,5 @@
+---
+reth-trie: patch
+---
+
+Fixed a potential panic in `ProofCalculator` by clearing internal computation state (`branch_stack`, `child_stack`, `branch_path`, etc.) after errors, preventing stale state from causing `usize` underflow panics when the calculator is reused. Added a test verifying correct behavior after simulated mid-computation errors.

.config/zepter.yaml

@@ -12,7 +12,7 @@ workflows:
         # Check that `A` activates the features of `B`.
         "propagate-feature",
         # These are the features to check:
-        "--features=std,op,dev,asm-keccak,jemalloc,jemalloc-prof,tracy-allocator,tracy,serde-bincode-compat,serde,test-utils,arbitrary,bench,alloy-compat,min-error-logs,min-warn-logs,min-info-logs,min-debug-logs,min-trace-logs,otlp,otlp-logs,js-tracer,portable,keccak-cache-global",
+        "--features=std,op,dev,asm-keccak,jemalloc,jemalloc-prof,tracy-allocator,tracy,serde-bincode-compat,serde,test-utils,arbitrary,bench,alloy-compat,min-error-logs,min-warn-logs,min-info-logs,min-debug-logs,min-trace-logs,otlp,otlp-logs,js-tracer,portable,keccak-cache-global,trie-debug",
         # Do not try to add a new section to `[features]` of `A` only because `B` exposes that feature. There are edge-cases where this is still needed, but we can add them manually.
         "--left-side-feature-missing=ignore",
         # Ignore the case that `A` it outside of the workspace. Otherwise it will report errors in external dependencies that we have no influence on.

.dockerignore

@@ -20,6 +20,11 @@
 # include dist directory, where the reth binary is located after compilation
 !/dist
 
+# include PGO build helper used by Dockerfile.depot
+!/.github
+!/.github/scripts
+!/.github/scripts/build_pgo_bolt.sh
+
 # include licenses
 !LICENSE-*
 

.github/actionlint.yaml

@@ -5,3 +5,4 @@ self-hosted-runner:
     - depot-ubuntu-latest-4
     - depot-ubuntu-latest-8
     - depot-ubuntu-latest-16
+    - available

.github/scripts/bench-job-summary.js

@@ -0,0 +1,106 @@
+// Generates a rich GitHub Actions job summary for reth-bench results.
+//
+// Reads from environment:
+//   BENCH_WORK_DIR       – Directory containing summary.json
+//   BENCH_PR             – PR number (may be empty)
+//   BENCH_ACTOR          – GitHub user who triggered the bench
+//   BENCH_CORES          – CPU core limit (0 = all)
+//   BENCH_WARMUP_BLOCKS  – Number of warmup blocks
+//   BENCH_SAMPLY         – 'true' if samply profiling was enabled
+//   BENCH_ABBA           – 'true' if ABBA interleaved order was used
+//
+// Usage from actions/github-script:
+//   const jobSummary = require('./.github/scripts/bench-job-summary.js');
+//   await jobSummary({ core, context, chartSha, grafanaUrl, runId });
+
+const fs = require('fs');
+const { verdict, loadSamplyUrls, blocksLabel, metricRows, waitTimeRows } = require('./bench-utils');
+
+module.exports = async function ({ core, context, chartSha, grafanaUrl, runId }) {
+  let summary;
+  try {
+    summary = JSON.parse(fs.readFileSync(process.env.BENCH_WORK_DIR + '/summary.json', 'utf8'));
+  } catch (e) {
+    await core.summary.addRaw('⚠️ Benchmark completed but failed to load summary.').write();
+    return;
+  }
+
+  const repo = `${context.repo.owner}/${context.repo.repo}`;
+  const prNumber = process.env.BENCH_PR;
+  const actor = process.env.BENCH_ACTOR;
+  const commitUrl = `https://github.com/${repo}/commit`;
+
+  const { emoji, label } = verdict(summary.changes);
+  const baselineLink = `[\`${summary.baseline.name}\`](${commitUrl}/${summary.baseline.ref})`;
+  const featureLink = `[\`${summary.feature.name}\`](${commitUrl}/${summary.feature.ref})`;
+  const diffUrl = `https://github.com/${repo}/compare/${summary.baseline.ref}...${summary.feature.ref}`;
+
+  // Header & metadata
+  const metaParts = [];
+  if (prNumber) metaParts.push(`**[PR #${prNumber}](https://github.com/${repo}/pull/${prNumber})**`);
+  metaParts.push(`triggered by @${actor}`);
+
+  let md = `# ${emoji} ${label}\n\n`;
+  md += metaParts.join(' · ') + '\n\n';
+  md += `**Baseline:** ${baselineLink}\n`;
+  md += `**Feature:** ${featureLink} ([diff](${diffUrl}))\n`;
+  md += blocksLabel(summary).map(p => `**${p.key}:** ${p.value}`).join(' · ') + '\n\n';
+
+  // Main comparison table
+  const rows = metricRows(summary);
+  md += `| Metric | Baseline | Feature | Change |\n`;
+  md += `|--------|----------|---------|--------|\n`;
+  for (const r of rows) {
+    md += `| ${r.label} | ${r.baseline} | ${r.feature} | ${r.change} |\n`;
+  }
+  md += '\n';
+
+  // Wait time breakdown
+  const wtRows = waitTimeRows(summary);
+  if (wtRows.length > 0) {
+    md += `### Wait Time Breakdown\n\n`;
+    md += `| Metric | Baseline | Feature |\n`;
+    md += `|--------|----------|--------|\n`;
+    for (const r of wtRows) {
+      md += `| ${r.title} | ${r.baseline} | ${r.feature} |\n`;
+    }
+    md += '\n';
+  }
+
+  // Charts
+  if (chartSha) {
+    const prNum = prNumber || '0';
+    const baseUrl = `https://raw.githubusercontent.com/decofe/reth-bench-charts/${chartSha}/pr/${prNum}/${runId}`;
+    const charts = [
+      { file: 'latency_throughput.png', label: 'Latency, Throughput & Diff' },
+      { file: 'wait_breakdown.png', label: 'Wait Time Breakdown' },
+      { file: 'gas_vs_latency.png', label: 'Gas vs Latency' },
+    ];
+    md += `### Charts\n\n`;
+    for (const chart of charts) {
+      md += `<details><summary>${chart.label}</summary>\n\n`;
+      md += `![${chart.label}](${baseUrl}/${chart.file})\n\n`;
+      md += `</details>\n\n`;
+    }
+  }
+
+  // Samply profiles
+  const samplyUrls = loadSamplyUrls(process.env.BENCH_WORK_DIR);
+  const samplyLinks = Object.entries(samplyUrls).map(([run, url]) => `- **${run}**: [Firefox Profiler](${url})`);
+  if (samplyLinks.length > 0) {
+    md += `### Samply Profiles\n\n${samplyLinks.join('\n')}\n\n`;
+  }
+
+  // Grafana
+  if (grafanaUrl) {
+    md += `### Grafana Dashboard\n\n[View real-time metrics](${grafanaUrl})\n\n`;
+  }
+
+  // Node errors
+  try {
+    const errors = fs.readFileSync(process.env.BENCH_WORK_DIR + '/errors.md', 'utf8');
+    if (errors.trim()) md += '\n' + errors + '\n';
+  } catch {}
+
+  await core.summary.addRaw(md).write();
+};

.github/scripts/bench-metrics-proxy.py

@@ -0,0 +1,276 @@
+#!/usr/bin/env python3
+"""
+Prometheus metrics proxy that fetches from a local reth node and
+re-exposes with additional benchmark labels.
+
+Reads labels from a JSON file (updated by local-reth-bench.sh between runs)
+and injects them into every Prometheus metric line.
+
+Returns empty 200 when reth is not running (clean Grafana gaps).
+"""
+import argparse
+import ipaddress
+import json
+import subprocess
+import sys
+import time
+from http.server import HTTPServer, BaseHTTPRequestHandler
+from urllib.request import urlopen
+from urllib.error import URLError
+
+
+def read_labels(path):
+    try:
+        with open(path) as f:
+            return json.load(f)
+    except (FileNotFoundError, json.JSONDecodeError):
+        return {}
+
+
+def inject_labels(metrics_bytes, label_str, label_names):
+    """Inject labels into Prometheus text format.
+
+    Operates on bytes and uses simple string ops instead of regex
+    for speed on large payloads (reth exposes thousands of metrics).
+
+    Skips injecting into lines that already contain any of the label names
+    to avoid duplicate labels (which Prometheus rejects).
+    """
+    if not label_str:
+        return metrics_bytes
+
+    label_bytes = label_str.encode("utf-8")
+    # Pre-encode label names for fast duplicate detection
+    label_name_bytes = [n.encode("utf-8") for n in label_names]
+    out = []
+    for line in metrics_bytes.split(b"\n"):
+        # Skip comments and blank lines
+        if line.startswith(b"#") or not line:
+            out.append(line)
+            continue
+
+        brace = line.find(b"{")
+        space = line.find(b" ")
+
+        if space == -1:
+            # Malformed, pass through
+            out.append(line)
+        elif brace != -1 and brace < space:
+            # Has labels: metric{existing="val"} 123
+            close = line.find(b"}", brace)
+            if close == -1:
+                out.append(line)
+                continue
+
+            # Filter out labels that already exist in this line
+            existing = line[brace + 1:close]
+            inject = label_bytes
+            if existing:
+                for name in label_name_bytes:
+                    if name + b"=" in existing:
+                        # Rebuild inject string excluding this label
+                        inject = _remove_label(inject, name)
+                if not inject:
+                    out.append(line)
+                    continue
+
+            if close == brace + 1:
+                # Empty braces: metric{} 123
+                out.append(line[:close] + inject + line[close:])
+            else:
+                out.append(line[:close] + b"," + inject + line[close:])
+        else:
+            # No labels: metric 123
+            out.append(line[:space] + b"{" + label_bytes + b"}" + line[space:])
+
+    return b"\n".join(out)
+
+
+def _remove_label(label_bytes, name):
+    """Remove a single label (name=\"...\") from a comma-separated label string."""
+    parts = []
+    for part in label_bytes.split(b","):
+        if not part.startswith(name + b"="):
+            parts.append(part)
+    return b",".join(parts)
+
+
+def build_label_str(labels):
+    """Pre-format the label injection string: key1="val1",key2="val2" """
+    if not labels:
+        return ""
+    return ",".join(f'{k}="{v}"' for k, v in sorted(labels.items()))
+
+
+def build_elapsed_gauge(labels):
+    """Build a bench_elapsed_seconds gauge from run_start_epoch in labels."""
+    start = labels.get("run_start_epoch")
+    if not start:
+        return b""
+    try:
+        elapsed = time.time() - float(start)
+    except (ValueError, TypeError):
+        return b""
+    # Build labels excluding internal keys
+    display = {k: v for k, v in labels.items()
+               if k not in ("run_start_epoch", "reference_epoch")}
+    lstr = build_label_str(display)
+    return (
+        f"# HELP bench_elapsed_seconds Seconds since benchmark run started\n"
+        f"# TYPE bench_elapsed_seconds gauge\n"
+        f"bench_elapsed_seconds{{{lstr}}} {elapsed:.1f}\n"
+    ).encode("utf-8")
+
+
+def compute_timestamp_ms(labels):
+    """Compute a synthetic timestamp so all runs share a common time origin.
+
+    Returns the timestamp in milliseconds, or None if not enough info.
+    Uses: reference_epoch + (now - run_start_epoch) → all runs overlay at
+    the same Grafana time range.
+    """
+    ref = labels.get("reference_epoch")
+    start = labels.get("run_start_epoch")
+    if not ref or not start:
+        return None
+    try:
+        elapsed = time.time() - float(start)
+        return int((float(ref) + elapsed) * 1000)
+    except (ValueError, TypeError):
+        return None
+
+
+def inject_timestamps(metrics_bytes, timestamp_ms):
+    """Append a Prometheus timestamp (ms) to every data line.
+
+    Prometheus text format: metric{labels} value [timestamp_ms]
+    Adding timestamps causes Prometheus to store all runs' samples
+    at the same relative time, enabling natural overlay in Grafana.
+    """
+    if timestamp_ms is None:
+        return metrics_bytes
+
+    ts = str(timestamp_ms).encode("utf-8")
+    out = []
+    for line in metrics_bytes.split(b"\n"):
+        if line.startswith(b"#") or not line:
+            out.append(line)
+        else:
+            out.append(line + b" " + ts)
+    return b"\n".join(out)
+
+
+class MetricsHandler(BaseHTTPRequestHandler):
+    # Use HTTP/1.1 so Content-Length is respected and Prometheus
+    # doesn't have to rely on connection close to detect end of body.
+    protocol_version = "HTTP/1.1"
+
+    def do_GET(self):
+        src = self.client_address[0]
+        try:
+            resp = urlopen(self.server.upstream, timeout=2)
+            metrics = resp.read()
+        except (URLError, ConnectionError, OSError):
+            # reth not running — return empty 200
+            self._send(b"")
+            #print(f"  scrape from {src}: empty (reth not running)", flush=True)
+            return
+
+        all_labels = read_labels(self.server.labels_file)
+        # Internal keys — not injected as Prometheus labels
+        internal = ("run_start_epoch", "reference_epoch")
+        labels = {k: v for k, v in all_labels.items() if k not in internal}
+        label_str = build_label_str(labels)
+        label_names = sorted(labels.keys())
+
+        t0 = time.monotonic()
+        result = inject_labels(metrics, label_str, label_names)
+        result += build_elapsed_gauge(all_labels)
+        ts_ms = compute_timestamp_ms(all_labels)
+        result = inject_timestamps(result, ts_ms)
+        dt = time.monotonic() - t0
+
+        self._send(result)
+        print(f"  scrape from {src}: {len(metrics)} -> {len(result)} bytes, "
+              f"inject {dt*1000:.1f}ms", flush=True)
+
+    def _send(self, body):
+        self.send_response(200)
+        self.send_header("Content-Type", "text/plain; version=0.0.4")
+        self.send_header("Content-Length", str(len(body)))
+        self.send_header("Connection", "close")
+        self.end_headers()
+        if body:
+            self.wfile.write(body)
+
+    def log_message(self, format, *args):
+        pass  # suppress per-request logging
+
+
+def resolve_bind_address(subnet_cidr):
+    """Find the local IP address that belongs to the given subnet.
+
+    Uses ``ip -j addr show`` to enumerate interfaces and returns the first
+    address that falls within *subnet_cidr* (e.g. ``10.10.0.0/24``).
+    """
+    network = ipaddress.ip_network(subnet_cidr, strict=False)
+    try:
+        result = subprocess.run(
+            ["ip", "-j", "addr", "show"],
+            capture_output=True, text=True, check=True,
+        )
+        interfaces = json.loads(result.stdout)
+    except (subprocess.CalledProcessError, FileNotFoundError, json.JSONDecodeError) as exc:
+        print(f"Error: cannot enumerate interfaces: {exc}", file=sys.stderr)
+        sys.exit(1)
+
+    for iface in interfaces:
+        for addr_info in iface.get("addr_info", []):
+            try:
+                addr = ipaddress.ip_address(addr_info["local"])
+            except (KeyError, ValueError):
+                continue
+            if addr in network:
+                return str(addr)
+
+    print(f"Error: no interface address found in subnet {subnet_cidr}", file=sys.stderr)
+    sys.exit(1)
+
+
+def main():
+    parser = argparse.ArgumentParser(description="Prometheus metrics proxy with label injection")
+    parser.add_argument("--labels", default="/tmp/bench-metrics-labels.json",
+                        help="Path to JSON file with labels to inject (default: /tmp/bench-metrics-labels.json)")
+    parser.add_argument("--upstream", default="http://127.0.0.1:9100/",
+                        help="Upstream reth metrics URL (default: http://127.0.0.1:9100/)")
+
+    bind_group = parser.add_mutually_exclusive_group()
+    bind_group.add_argument("--bind", default=None,
+                            help="Address to bind the proxy (default: 0.0.0.0)")
+    bind_group.add_argument("--subnet", default=None,
+                            help="Auto-detect bind address from a local interface in this subnet (e.g. 10.10.0.0/24)")
+
+    parser.add_argument("--port", type=int, default=9090,
+                        help="Port to bind the proxy (default: 9090)")
+    args = parser.parse_args()
+
+    if args.subnet:
+        bind_addr = resolve_bind_address(args.subnet)
+    elif args.bind:
+        bind_addr = args.bind
+    else:
+        bind_addr = "0.0.0.0"
+
+    server = HTTPServer((bind_addr, args.port), MetricsHandler)
+    server.upstream = args.upstream
+    server.labels_file = args.labels
+
+    print(f"bench-metrics-proxy listening on {bind_addr}:{args.port}")
+    print(f"  upstream: {args.upstream}")
+    print(f"  labels:   {args.labels}")
+    sys.stdout.flush()
+    server.serve_forever()
+
+
+if __name__ == "__main__":
+    main()

.github/scripts/bench-reth-build.sh

@@ -11,10 +11,11 @@
 #              optional branch-sha is the PR head commit for cache key
 #
 # Outputs:
-#   baseline: <source-dir>/target/profiling/reth
-#   feature:  <source-dir>/target/profiling/reth, reth-bench installed to cargo bin
+#   baseline: <source-dir>/target/profiling/reth (or reth-bb if BENCH_BIG_BLOCKS=true)
+#   feature:  <source-dir>/target/profiling/reth (or reth-bb), reth-bench installed to cargo bin
 #
 # Required: mc (MinIO client) with a configured alias
+# Optional env: BENCH_BIG_BLOCKS (true/false) — build reth-bb instead of reth
 set -euo pipefail
 
 MC="mc"
@@ -22,6 +23,32 @@ MODE="$1"
 SOURCE_DIR="$2"
 COMMIT="$3"
 
+BIG_BLOCKS="${BENCH_BIG_BLOCKS:-false}"
+# The node binary to build: reth-bb for big blocks, reth otherwise
+if [ "$BIG_BLOCKS" = "true" ]; then
+  NODE_BIN="reth-bb"
+  NODE_PKG="-p reth-bb"
+else
+  NODE_BIN="reth"
+  NODE_PKG="--bin reth"
+fi
+
+# Tracy support: when BENCH_TRACY is "on" or "full", add Tracy cargo features
+# and frame pointers for accurate stack traces.
+EXTRA_FEATURES=""
+EXTRA_RUSTFLAGS=""
+if [ "${BENCH_TRACY:-off}" != "off" ]; then
+  EXTRA_FEATURES="tracy,tracy-client/ondemand"
+  EXTRA_RUSTFLAGS=" -C force-frame-pointers=yes"
+fi
+
+# Cache suffix: hash of features+rustflags so different build configs get separate cache entries
+if [ -n "$EXTRA_FEATURES" ] || [ -n "$EXTRA_RUSTFLAGS" ]; then
+  BUILD_SUFFIX="-$(echo "${EXTRA_FEATURES}${EXTRA_RUSTFLAGS}" | sha256sum | cut -c1-12)"
+else
+  BUILD_SUFFIX=""
+fi
+
 # Verify a cached reth binary was built from the expected commit.
 # `reth --version` outputs "Commit SHA: <full-sha>" on its own line.
 verify_binary() {
@@ -42,52 +69,70 @@ verify_binary() {
 
 case "$MODE" in
   baseline|main)
-    BUCKET="minio/reth-binaries/${COMMIT}"
+    BUCKET="minio/reth-binaries/${COMMIT}${BUILD_SUFFIX}"
     mkdir -p "${SOURCE_DIR}/target/profiling"
 
     CACHE_VALID=false
-    if $MC stat "${BUCKET}/reth" &>/dev/null; then
-      echo "Cache hit for baseline (${COMMIT}), downloading binary..."
-      $MC cp "${BUCKET}/reth" "${SOURCE_DIR}/target/profiling/reth"
-      chmod +x "${SOURCE_DIR}/target/profiling/reth"
-      if verify_binary "${SOURCE_DIR}/target/profiling/reth" "${COMMIT}"; then
+    if $MC stat "${BUCKET}/${NODE_BIN}" &>/dev/null; then
+      echo "Cache hit for baseline (${COMMIT}), downloading ${NODE_BIN}..."
+      $MC cp "${BUCKET}/${NODE_BIN}" "${SOURCE_DIR}/target/profiling/${NODE_BIN}"
+      chmod +x "${SOURCE_DIR}/target/profiling/${NODE_BIN}"
+      if verify_binary "${SOURCE_DIR}/target/profiling/${NODE_BIN}" "${COMMIT}"; then
         CACHE_VALID=true
       else
         echo "Cached baseline binary is stale, rebuilding..."
       fi
     fi
     if [ "$CACHE_VALID" = false ]; then
-      echo "Building baseline (${COMMIT}) from source..."
+      echo "Building baseline ${NODE_BIN} (${COMMIT}) from source..."
       cd "${SOURCE_DIR}"
-      cargo build --profile profiling --bin reth
-      $MC cp target/profiling/reth "${BUCKET}/reth"
+      FEATURES_ARG=""
+      WORKSPACE_ARG=""
+      if [ -n "$EXTRA_FEATURES" ]; then
+        # --workspace is needed for cross-package feature syntax (tracy-client/ondemand)
+        FEATURES_ARG="--features ${EXTRA_FEATURES}"
+        WORKSPACE_ARG="--workspace"
+      fi
+      # shellcheck disable=SC2086
+      RUSTFLAGS="-C target-cpu=native${EXTRA_RUSTFLAGS}" \
+        cargo build --profile profiling $NODE_PKG $WORKSPACE_ARG $FEATURES_ARG
+      $MC cp "target/profiling/${NODE_BIN}" "${BUCKET}/${NODE_BIN}"
     fi
     ;;
 
   feature|branch)
     BRANCH_SHA="${4:-$COMMIT}"
-    BUCKET="minio/reth-binaries/${BRANCH_SHA}"
+    BUCKET="minio/reth-binaries/${BRANCH_SHA}${BUILD_SUFFIX}"
 
     CACHE_VALID=false
-    if $MC stat "${BUCKET}/reth" &>/dev/null && $MC stat "${BUCKET}/reth-bench" &>/dev/null; then
+    if $MC stat "${BUCKET}/${NODE_BIN}" &>/dev/null && $MC stat "${BUCKET}/reth-bench" &>/dev/null; then
       echo "Cache hit for ${BRANCH_SHA}, downloading binaries..."
       mkdir -p "${SOURCE_DIR}/target/profiling"
-      $MC cp "${BUCKET}/reth" "${SOURCE_DIR}/target/profiling/reth"
+      $MC cp "${BUCKET}/${NODE_BIN}" "${SOURCE_DIR}/target/profiling/${NODE_BIN}"
       $MC cp "${BUCKET}/reth-bench" /home/ubuntu/.cargo/bin/reth-bench
-      chmod +x "${SOURCE_DIR}/target/profiling/reth" /home/ubuntu/.cargo/bin/reth-bench
-      if verify_binary "${SOURCE_DIR}/target/profiling/reth" "${COMMIT}"; then
+      chmod +x "${SOURCE_DIR}/target/profiling/${NODE_BIN}" /home/ubuntu/.cargo/bin/reth-bench
+      if verify_binary "${SOURCE_DIR}/target/profiling/${NODE_BIN}" "${COMMIT}"; then
         CACHE_VALID=true
       else
         echo "Cached feature binary is stale, rebuilding..."
       fi
     fi
     if [ "$CACHE_VALID" = false ]; then
-      echo "Building feature (${COMMIT}) from source..."
+      echo "Building feature ${NODE_BIN} (${COMMIT}) from source..."
       cd "${SOURCE_DIR}"
       rustup show active-toolchain || rustup default stable
-      make profiling
+      if [ -n "$EXTRA_FEATURES" ]; then
+        # Can't use `make profiling` when adding features; build explicitly
+        # --workspace is needed for cross-package feature syntax (tracy-client/ondemand)
+        RUSTFLAGS="-C target-cpu=native${EXTRA_RUSTFLAGS}" \
+          cargo build --profile profiling --workspace $NODE_PKG --features "${EXTRA_FEATURES}"
+      else
+        # shellcheck disable=SC2086
+        RUSTFLAGS="-C target-cpu=native${EXTRA_RUSTFLAGS}" \
+          cargo build --profile profiling $NODE_PKG
+      fi
       make install-reth-bench
-      $MC cp target/profiling/reth "${BUCKET}/reth"
+      $MC cp "target/profiling/${NODE_BIN}" "${BUCKET}/${NODE_BIN}"
       $MC cp "$(which reth-bench)" "${BUCKET}/reth-bench"
     fi
     ;;

.github/scripts/bench-reth-local.sh

@@ -0,0 +1,581 @@
+#!/usr/bin/env bash
+#
+# local-reth-bench.sh — Run the reth Engine API benchmark locally.
+#
+# Replicates the CI bench.yml workflow (build, snapshot, system tuning,
+# interleaved B-F-F-B execution, summary, charts) without any GitHub
+# Actions glue (no PR comments, no artifact upload, no Slack).
+#
+# Usage:
+#   local-reth-bench.sh <baseline-ref> <feature-ref> [options]
+#
+# Options:
+#   --blocks N      Number of blocks to benchmark (default: 500)
+#   --warmup N      Number of warmup blocks (default: 100)
+#   --cores N       Limit reth to N CPU cores, 0 = all available (default: 0)
+#   --samply        Enable samply profiling
+#   --tracy MODE    Tracy profiling: off, on, full (default: off)
+#   --tracy-filter F Tracy tracing filter (default: debug)
+#   --no-tune       Skip system tuning (useful on dev machines / macOS)
+#
+# Requires: the reth repo at RETH_REPO (default: ~/reth)
+#
+# Dependencies (install before first run):
+#   mc (MinIO client), schelk, cpupower, taskset, stdbuf, python3, curl,
+#   make, uv, pzstd, jq, Rust toolchain (cargo/rustup)
+#
+# The script delegates to the existing bench-reth-*.sh scripts in the reth
+# repo for the actual build, snapshot, and run steps.
+set -euo pipefail
+
+# ── PATH ──────────────────────────────────────────────────────────────
+# Ensure cargo and user-local bins (mc, uv) are visible
+export PATH="$HOME/.local/bin:$HOME/.cargo/bin:$PATH"
+
+# ── Defaults ──────────────────────────────────────────────────────────
+RETH_REPO="${RETH_REPO:-$HOME/reth}"
+BLOCKS=500
+WARMUP=100
+CORES=0
+SAMPLY=false
+TRACY="off"
+TRACY_FILTER="debug"
+TUNE=true
+BASELINE_REF=""
+FEATURE_REF=""
+
+# ── Parse arguments ──────────────────────────────────────────────────
+usage() {
+  cat <<EOF
+Usage: $(basename "$0") <baseline-ref> <feature-ref> [options]
+
+Options:
+  --blocks N         Number of blocks to benchmark (default: 500)
+  --warmup N         Number of warmup blocks (default: 100)
+  --cores N          Limit reth to N CPU cores (default: 0 = all)
+  --samply           Enable samply profiling
+  --tracy MODE       Tracy profiling: off, on, full (default: off)
+                       on   = tracing only (lower overhead)
+                       full = tracing + CPU sampling (higher overhead)
+  --tracy-filter F   Tracy tracing filter (default: debug)
+  --no-tune          Skip system tuning
+EOF
+  exit 1
+}
+
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+    --blocks)       BLOCKS="$2"; shift 2 ;;
+    --warmup)       WARMUP="$2"; shift 2 ;;
+    --cores)        CORES="$2"; shift 2 ;;
+    --samply)       SAMPLY=true; shift ;;
+    --tracy)        TRACY="$2"; shift 2 ;;
+    --tracy-filter) TRACY_FILTER="$2"; shift 2 ;;
+    --no-tune)      TUNE=false; shift ;;
+    --help|-h)  usage ;;
+    -*)         echo "Unknown option: $1"; usage ;;
+    *)
+      if [ -z "$BASELINE_REF" ]; then
+        BASELINE_REF="$1"
+      elif [ -z "$FEATURE_REF" ]; then
+        FEATURE_REF="$1"
+      else
+        echo "Unexpected argument: $1"; usage
+      fi
+      shift
+      ;;
+  esac
+done
+
+if [ -z "$BASELINE_REF" ] || [ -z "$FEATURE_REF" ]; then
+  echo "Error: both <baseline-ref> and <feature-ref> are required."
+  usage
+fi
+
+# Validate --tracy value
+case "$TRACY" in
+  off|on|full) ;;
+  *) echo "Error: --tracy must be off, on, or full (got: $TRACY)"; usage ;;
+esac
+
+# Samply + tracy=full are mutually exclusive (both use perf sampling)
+if [ "$SAMPLY" = "true" ] && [ "$TRACY" = "full" ]; then
+  echo "Warning: samply and tracy=full both use perf sampling; downgrading tracy to 'on'."
+  TRACY="on"
+fi
+
+# ── Check dependencies ───────────────────────────────────────────────
+missing=()
+for cmd in mc schelk cpupower taskset stdbuf python3 curl make uv pzstd jq cargo; do
+  command -v "$cmd" &>/dev/null || missing+=("$cmd")
+done
+if [ ${#missing[@]} -gt 0 ]; then
+  echo "Error: missing required tools: ${missing[*]}"
+  echo "See the CI 'Install dependencies' step in .github/workflows/bench.yml for install instructions."
+  exit 1
+fi
+
+if [ "$TRACY" != "off" ]; then
+  if ! command -v tracy-capture &>/dev/null; then
+    echo "Error: tracy-capture is required for --tracy $TRACY"
+    exit 1
+  fi
+fi
+
+# Ensure tools that run via sudo are in a sudo-visible path.
+# The bench scripts use `sudo schelk` / `sudo samply` but cargo installs
+# them to ~/.cargo/bin which sudo's secure_path doesn't include.
+for cmd in schelk samply; do
+  if command -v "$cmd" &>/dev/null && ! sudo sh -c "command -v $cmd" &>/dev/null; then
+    echo "Installing $cmd to /usr/local/bin (needed for sudo)..."
+    sudo install "$(command -v "$cmd")" /usr/local/bin/
+  fi
+done
+
+if [ ! -d "$RETH_REPO/.git" ]; then
+  echo "Error: RETH_REPO=$RETH_REPO is not a git repository."
+  echo "Set RETH_REPO or clone reth to ~/reth"
+  exit 1
+fi
+
+# ── Resolve paths ────────────────────────────────────────────────────
+SELF_DIR="$(cd "$(dirname "$0")" && pwd)"
+SCRIPTS_DIR="${RETH_REPO}/.github/scripts"
+BENCH_WORK_DIR="${RETH_REPO}/../bench-work-$(date +%Y%m%d-%H%M%S)"
+BASELINE_SRC="${RETH_REPO}/../reth-baseline"
+FEATURE_SRC="${RETH_REPO}/../reth-feature"
+
+mkdir -p "$BENCH_WORK_DIR"
+BENCH_WORK_DIR="$(cd "$BENCH_WORK_DIR" && pwd)"
+
+# ── Global cleanup trap (restores system tuning on any exit) ─────────
+TUNING_APPLIED=false
+CSTATE_PID=
+METRICS_PROXY_PID=
+cleanup_global() {
+  [ -n "$METRICS_PROXY_PID" ] && kill "$METRICS_PROXY_PID" 2>/dev/null || true
+  if [ "$TUNING_APPLIED" = true ]; then
+    echo
+    echo "▸ Restoring system settings..."
+    [ -n "$CSTATE_PID" ] && kill "$CSTATE_PID" 2>/dev/null || true
+    sudo systemctl start irqbalance cron atd 2>/dev/null || true
+    echo "  System settings restored."
+  fi
+}
+trap cleanup_global EXIT
+
+echo "═══════════════════════════════════════════════════════════"
+echo "  reth local benchmark"
+echo "═══════════════════════════════════════════════════════════"
+echo "  Baseline ref : $BASELINE_REF"
+echo "  Feature ref  : $FEATURE_REF"
+echo "  Blocks       : $BLOCKS"
+echo "  Warmup       : $WARMUP"
+echo "  Cores        : $CORES"
+echo "  Samply       : $SAMPLY"
+echo "  Tracy        : $TRACY"
+echo "  Tracy filter : $TRACY_FILTER"
+echo "  System tune  : $TUNE"
+echo "  Work dir     : $BENCH_WORK_DIR"
+echo "  Reth repo    : $RETH_REPO"
+echo "═══════════════════════════════════════════════════════════"
+echo
+
+# Enable sccache if available (matches CI's RUSTC_WRAPPER=sccache)
+if command -v sccache &>/dev/null; then
+  export RUSTC_WRAPPER="sccache"
+fi
+
+# Export env vars expected by the bench-reth-*.sh scripts
+export BENCH_BLOCKS="$BLOCKS"
+export BENCH_WARMUP_BLOCKS="$WARMUP"
+export BENCH_CORES="$CORES"
+export BENCH_SAMPLY="$SAMPLY"
+export BENCH_TRACY="$TRACY"
+export BENCH_TRACY_FILTER="$TRACY_FILTER"
+export BENCH_WORK_DIR
+export SCHELK_MOUNT="${SCHELK_MOUNT:-/reth-bench}"
+export BENCH_RPC_URL="${BENCH_RPC_URL:-https://ethereum.reth.rs/rpc}"
+export BENCH_METRICS_ADDR="127.0.0.1:9100"
+
+# ── Step 1: Resolve refs to full SHAs ────────────────────────────────
+echo "▸ Resolving git refs..."
+cd "$RETH_REPO"
+
+resolve_ref() {
+  local ref="$1"
+  git fetch origin "$ref" --quiet 2>/dev/null || true
+  git rev-parse "$ref" 2>/dev/null \
+    || git rev-parse "origin/$ref" 2>/dev/null \
+    || { echo "Error: cannot resolve ref '$ref'"; exit 1; }
+}
+
+BASELINE_SHA="$(resolve_ref "$BASELINE_REF")"
+FEATURE_SHA="$(resolve_ref "$FEATURE_REF")"
+echo "  Baseline SHA : $BASELINE_SHA"
+echo "  Feature SHA  : $FEATURE_SHA"
+echo
+
+# ── Step 2: Prepare source directories ───────────────────────────────
+echo "▸ Preparing source directories..."
+
+prepare_source() {
+  local src_dir="$1" ref="$2"
+  if [ -d "$src_dir" ]; then
+    git -C "$src_dir" fetch origin "$ref" 2>/dev/null || true
+  else
+    git clone --recurse-submodules "$RETH_REPO" "$src_dir"
+  fi
+  git -C "$src_dir" checkout "$ref" --force
+  git -C "$src_dir" submodule update --init --recursive
+}
+
+prepare_source "$BASELINE_SRC" "$BASELINE_SHA"
+prepare_source "$FEATURE_SRC" "$FEATURE_SHA"
+BASELINE_SRC="$(cd "$BASELINE_SRC" && pwd)"
+FEATURE_SRC="$(cd "$FEATURE_SRC" && pwd)"
+echo "  Baseline src : $BASELINE_SRC"
+echo "  Feature src  : $FEATURE_SRC"
+echo
+
+# ── Step 3: Check / download snapshot ────────────────────────────────
+echo "▸ Checking snapshot..."
+cd "$RETH_REPO"
+SNAPSHOT_NEEDED=false
+if ! "${SCRIPTS_DIR}/bench-reth-snapshot.sh" --check; then
+  SNAPSHOT_NEEDED=true
+  echo "  Snapshot needs update."
+else
+  echo "  Snapshot is up-to-date."
+fi
+echo
+
+# ── Step 4: Build binaries (+ snapshot download) in parallel ─────────
+echo "▸ Building binaries (parallel)..."
+cd "$RETH_REPO"
+
+FAIL=0
+
+"${SCRIPTS_DIR}/bench-reth-build.sh" baseline "$BASELINE_SRC" "$BASELINE_SHA" &
+PID_BASELINE=$!
+
+"${SCRIPTS_DIR}/bench-reth-build.sh" feature "$FEATURE_SRC" "$FEATURE_SHA" &
+PID_FEATURE=$!
+
+PID_SNAPSHOT=
+if [ "$SNAPSHOT_NEEDED" = "true" ]; then
+  echo "  Also downloading snapshot in parallel..."
+  "${SCRIPTS_DIR}/bench-reth-snapshot.sh" &
+  PID_SNAPSHOT=$!
+fi
+
+wait $PID_BASELINE || FAIL=1
+wait $PID_FEATURE  || FAIL=1
+[ -n "$PID_SNAPSHOT" ] && { wait $PID_SNAPSHOT || FAIL=1; }
+
+if [ $FAIL -ne 0 ]; then
+  echo "Error: one or more parallel tasks failed (builds / snapshot)"
+  exit 1
+fi
+echo "  Binaries built successfully."
+echo
+
+# ── Step 5: System tuning (optional) ────────────────────────────────
+if [ "$TUNE" = "true" ]; then
+  echo "▸ Applying system tuning..."
+
+  sudo cpupower frequency-set -g performance 2>/dev/null || true
+
+  # Disable turbo boost (Intel + AMD)
+  echo 1 | sudo tee /sys/devices/system/cpu/intel_pstate/no_turbo 2>/dev/null || true
+  echo 0 | sudo tee /sys/devices/system/cpu/cpufreq/boost 2>/dev/null || true
+
+  sudo swapoff -a 2>/dev/null || true
+  echo 0 | sudo tee /proc/sys/kernel/randomize_va_space 2>/dev/null || true
+
+  # Disable SMT (hyperthreading)
+  for cpu in /sys/devices/system/cpu/cpu*/topology/thread_siblings_list; do
+    [ -f "$cpu" ] || continue
+    first=$(cut -d, -f1 < "$cpu" | cut -d- -f1)
+    current=$(echo "$cpu" | grep -o 'cpu[0-9]*' | grep -o '[0-9]*')
+    if [ "$current" != "$first" ]; then
+      echo 0 | sudo tee "/sys/devices/system/cpu/cpu${current}/online" 2>/dev/null || true
+    fi
+  done
+  echo "  Online CPUs: $(nproc)"
+
+  # Disable transparent huge pages
+  for p in /sys/kernel/mm/transparent_hugepage /sys/kernel/mm/transparent_hugepages; do
+    if [ -d "$p" ]; then
+      echo never | sudo tee "$p/enabled" 2>/dev/null || true
+      echo never | sudo tee "$p/defrag" 2>/dev/null || true
+      break
+    fi
+  done
+
+  # Prevent deep C-states
+  sudo sh -c 'exec 3<>/dev/cpu_dma_latency; echo -ne "\x00\x00\x00\x00" >&3; sleep infinity' &
+  CSTATE_PID=$!
+
+  # Pin IRQs to core 0
+  for irq in /proc/irq/*/smp_affinity_list; do
+    echo 0 | sudo tee "$irq" 2>/dev/null || true
+  done
+
+  # Stop noisy background services
+  sudo systemctl stop irqbalance cron atd unattended-upgrades snapd 2>/dev/null || true
+
+  TUNING_APPLIED=true
+
+  # Log environment for reproducibility (matches CI)
+  echo "  === Benchmark environment ==="
+  echo "  Kernel : $(uname -r)"
+  lscpu | grep -E 'Model name|CPU\(s\)|MHz|NUMA' | sed 's/^/  /'
+  echo "  Governor : $(cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor 2>/dev/null || echo unknown)"
+  echo "  Freq     : $(cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq 2>/dev/null || echo unknown)"
+  echo "  THP      : $(cat /sys/kernel/mm/transparent_hugepage/enabled 2>/dev/null || cat /sys/kernel/mm/transparent_hugepages/enabled 2>/dev/null || echo unknown)"
+  free -h | sed 's/^/  /'
+  echo "  System tuning applied."
+  echo
+fi
+
+# ── Step 5b: Tracefs mount (tracy=full only) ─────────────────────────
+if [ "$TRACY" = "full" ] && [ "$(uname)" = "Linux" ]; then
+  echo "▸ Mounting tracefs for Tracy full mode..."
+  sudo mount -t tracefs tracefs /sys/kernel/tracing -o mode=755 2>/dev/null || true
+fi
+
+# ── Tracy upload & viewer helpers ────────────────────────────────────
+TRACY_VIEWER_BASE="${TRACY_VIEWER_BASE:-}"
+
+tracy_viewer_url() {
+  local profile_url="$1"
+  if [ -z "$TRACY_VIEWER_BASE" ]; then
+    echo ""
+    return
+  fi
+  local encoded
+  encoded=$(python3 -c "import urllib.parse, sys; print(urllib.parse.quote(sys.argv[1], safe=''))" "$profile_url")
+  echo "${TRACY_VIEWER_BASE}?profile_url=${encoded}"
+}
+
+upload_tracy() {
+  local label="$1" output_dir="$2" sha="$3"
+  local tracy_file="$output_dir/tracy-profile.tracy"
+
+  if [ ! -f "$tracy_file" ]; then
+    echo "  Tracy: no profile found, skipping upload."
+    return
+  fi
+
+  local timestamp short_sha remote_name bucket mc_alias
+  timestamp=$(date +%Y%m%d-%H%M%S)
+  short_sha="${sha:0:7}"
+  remote_name="${label}-${short_sha}-${timestamp}.tracy"
+  bucket="${TRACY_BUCKET:-tracy-profiles}"
+  mc_alias="${MC_ALIAS:-minio}"
+  local minio_base="${TRACY_MINIO_URL:-http://minio.minio.svc.cluster.local:9000}"
+
+  echo "  Tracy: uploading profile..."
+  if mc cp "$tracy_file" "${mc_alias}/${bucket}/${remote_name}"; then
+    local url="${minio_base}/${bucket}/${remote_name}"
+    echo "$url" > "$output_dir/tracy_url.txt"
+    local viewer
+    viewer=$(tracy_viewer_url "$url")
+    if [ -n "$viewer" ]; then
+      echo "$viewer" > "$output_dir/tracy_viewer_url.txt"
+      echo "  Tracy: uploaded → $viewer"
+    else
+      echo "  Tracy: uploaded → $url"
+    fi
+  else
+    echo "  Tracy: upload failed (non-fatal)."
+  fi
+
+  # Delete large profile to free disk
+  rm -f "$tracy_file"
+}
+
+# ── Step 6: Pre-flight cleanup ───────────────────────────────────────
+echo "▸ Pre-flight cleanup..."
+pkill -f bench-metrics-proxy 2>/dev/null || true
+sudo pkill -9 reth 2>/dev/null || true
+sleep 1
+if mountpoint -q "$SCHELK_MOUNT" 2>/dev/null; then
+  sudo umount -l "$SCHELK_MOUNT" 2>/dev/null || true
+  sudo schelk recover -y 2>/dev/null || true
+fi
+echo
+
+# ── Step 7: Interleaved benchmark runs (B-F-F-B) ────────────────────
+# This ordering reduces systematic bias from thermal drift and cache warming.
+BASELINE_BIN="${BASELINE_SRC}/target/profiling/reth"
+FEATURE_BIN="${FEATURE_SRC}/target/profiling/reth"
+
+# Start metrics proxy (reth → label injection → Prometheus)
+LABELS_FILE="/tmp/bench-metrics-labels.json"
+echo '{}' > "$LABELS_FILE"
+METRICS_SUBNET="${METRICS_SUBNET:-10.10.0.0/24}"
+METRICS_PORT="${METRICS_PORT:-9090}"
+python3 "${SELF_DIR}/bench-metrics-proxy.py" \
+  --labels "$LABELS_FILE" \
+  --upstream "http://${BENCH_METRICS_ADDR}/" \
+  --subnet "$METRICS_SUBNET" \
+  --port "$METRICS_PORT" &
+METRICS_PROXY_PID=$!
+echo "▸ Metrics proxy started (PID $METRICS_PROXY_PID) on subnet ${METRICS_SUBNET}, port ${METRICS_PORT}"
+
+# Unique benchmark ID: local-<timestamp> for local runs, ci-<run_id> for CI
+BENCH_ID="local-$(basename "$BENCH_WORK_DIR" | sed 's/bench-work-//')"
+# Reference epoch: shared time origin so all runs overlay in Grafana.
+# The proxy maps each run's elapsed time onto this common origin.
+BENCH_REFERENCE_EPOCH=$(date +%s)
+
+write_labels() {
+  local run_label="$1" run_type="$2" ref="$3" sha="$4"
+  LAST_RUN_START=$(date +%s)
+  cat > "$LABELS_FILE" <<-EOF
+	{"benchmark_run":"${run_label}","run_type":"${run_type}","git_ref":"${ref}","bench_sha":"${sha}","benchmark_id":"${BENCH_ID}","run_start_epoch":"${LAST_RUN_START}","reference_epoch":"${BENCH_REFERENCE_EPOCH}"}
+	EOF
+}
+
+run_bench() {
+  local label="$1" binary="$2" output_dir="$3"
+  echo "▸ Running benchmark: ${label}..."
+  cd "$RETH_REPO"
+  if command -v taskset &>/dev/null; then
+    taskset -c 0 "${SCRIPTS_DIR}/bench-reth-run.sh" "$label" "$binary" "$output_dir"
+  else
+    "${SCRIPTS_DIR}/bench-reth-run.sh" "$label" "$binary" "$output_dir"
+  fi
+  echo "  ✓ ${label} complete."
+  echo
+}
+
+write_labels "baseline-1" "baseline" "$BASELINE_REF" "$BASELINE_SHA"
+run_bench "baseline-1" "$BASELINE_BIN" "$BENCH_WORK_DIR/baseline-1"
+
+write_labels "feature-1" "feature" "$FEATURE_REF" "$FEATURE_SHA"
+run_bench "feature-1"  "$FEATURE_BIN"  "$BENCH_WORK_DIR/feature-1"
+
+write_labels "feature-2" "feature" "$FEATURE_REF" "$FEATURE_SHA"
+run_bench "feature-2"  "$FEATURE_BIN"  "$BENCH_WORK_DIR/feature-2"
+
+write_labels "baseline-2" "baseline" "$BASELINE_REF" "$BASELINE_SHA"
+run_bench "baseline-2" "$BASELINE_BIN" "$BENCH_WORK_DIR/baseline-2"
+
+# ── Compute Grafana URL ──────────────────────────────────────────────
+GRAFANA_BASE_URL="https://tempoxyz.grafana.net/d/reth-bench-ghr/reth-bench-ghr"
+GRAFANA_DATASOURCE="ef57fux92e9z4e"
+LAST_RUN_DURATION=$(( $(date +%s) - LAST_RUN_START ))
+FROM_MS=$(( BENCH_REFERENCE_EPOCH * 1000 ))
+TO_MS=$(( (BENCH_REFERENCE_EPOCH + LAST_RUN_DURATION) * 1000 ))
+GRAFANA_URL="${GRAFANA_BASE_URL}?orgId=1&from=${FROM_MS}&to=${TO_MS}&timezone=browser&var-datasource=${GRAFANA_DATASOURCE}&var-job=reth-bench&var-benchmark_id=${BENCH_ID}&var-benchmark_run=\$__all"
+
+# ── Step 8: Scan logs for errors ─────────────────────────────────────
+echo "▸ Scanning logs for errors..."
+ERRORS_FILE="$BENCH_WORK_DIR/errors.md"
+found_errors=false
+for run_dir in baseline-1 feature-1 feature-2 baseline-2; do
+  LOG="$BENCH_WORK_DIR/$run_dir/node.log"
+  [ -f "$LOG" ] || continue
+  panics=$(grep -c -E 'panicked at' "$LOG" 2>/dev/null || true)
+  errors=$(grep -c ' ERROR ' "$LOG" 2>/dev/null || true)
+  if [ "$panics" -gt 0 ] || [ "$errors" -gt 0 ]; then
+    if [ "$found_errors" = false ]; then
+      printf '### ⚠️ Node Errors\n\n' >> "$ERRORS_FILE"
+      found_errors=true
+    fi
+    printf '<details><summary><b>%s</b>: %d panic(s), %d error(s)</summary>\n\n' \
+      "$run_dir" "$panics" "$errors" >> "$ERRORS_FILE"
+    if [ "$panics" -gt 0 ]; then
+      printf '**Panics:**\n```\n' >> "$ERRORS_FILE"
+      grep -E 'panicked at' "$LOG" | head -10 >> "$ERRORS_FILE"
+      printf '```\n' >> "$ERRORS_FILE"
+    fi
+    if [ "$errors" -gt 0 ]; then
+      printf '**Errors (first 20):**\n```\n' >> "$ERRORS_FILE"
+      grep ' ERROR ' "$LOG" | head -20 >> "$ERRORS_FILE"
+      printf '```\n' >> "$ERRORS_FILE"
+    fi
+    printf '\n</details>\n\n' >> "$ERRORS_FILE"
+  fi
+done
+if [ "$found_errors" = true ]; then
+  echo "  ⚠ Errors found — see $ERRORS_FILE"
+else
+  echo "  No errors found."
+fi
+echo
+
+# ── Step 9: Parse results ───────────────────────────────────────────
+echo "▸ Parsing results..."
+cd "$RETH_REPO"
+
+SUMMARY_ARGS=(
+  --output-summary "$BENCH_WORK_DIR/summary.json"
+  --output-markdown "$BENCH_WORK_DIR/comment.md"
+  --repo "paradigmxyz/reth"
+  --baseline-ref "$BASELINE_SHA"
+  --baseline-name "$BASELINE_REF"
+  --feature-name "$FEATURE_REF"
+  --feature-ref "$FEATURE_SHA"
+  --baseline-csv "$BENCH_WORK_DIR/baseline-1/combined_latency.csv" "$BENCH_WORK_DIR/baseline-2/combined_latency.csv"
+  --feature-csv "$BENCH_WORK_DIR/feature-1/combined_latency.csv" "$BENCH_WORK_DIR/feature-2/combined_latency.csv"
+  --gas-csv "$BENCH_WORK_DIR/feature-1/total_gas.csv"
+  --grafana-url "$GRAFANA_URL"
+)
+
+python3 "${SCRIPTS_DIR}/bench-reth-summary.py" "${SUMMARY_ARGS[@]}"
+echo
+
+# ── Step 10: Generate charts ─────────────────────────────────────────
+echo "▸ Generating charts..."
+CHART_ARGS=(
+  --output-dir "$BENCH_WORK_DIR/charts"
+  --feature "$BENCH_WORK_DIR/feature-1/combined_latency.csv" "$BENCH_WORK_DIR/feature-2/combined_latency.csv"
+  --baseline "$BENCH_WORK_DIR/baseline-1/combined_latency.csv" "$BENCH_WORK_DIR/baseline-2/combined_latency.csv"
+  --baseline-name "$BASELINE_REF"
+  --feature-name "$FEATURE_REF"
+)
+
+if python3 -c "import matplotlib" 2>/dev/null; then
+  python3 "${SCRIPTS_DIR}/bench-reth-charts.py" "${CHART_ARGS[@]}"
+elif command -v uv &>/dev/null; then
+  uv run --with matplotlib python3 "${SCRIPTS_DIR}/bench-reth-charts.py" "${CHART_ARGS[@]}"
+else
+  echo "  Warning: matplotlib not available, skipping chart generation."
+fi
+echo
+
+# ── Step 11: Upload Tracy profiles ────────────────────────────────────
+if [ "$TRACY" != "off" ]; then
+  echo "▸ Uploading Tracy profiles..."
+  upload_tracy "baseline-1" "$BENCH_WORK_DIR/baseline-1" "$BASELINE_SHA"
+  upload_tracy "feature-1"  "$BENCH_WORK_DIR/feature-1"  "$FEATURE_SHA"
+  upload_tracy "feature-2"  "$BENCH_WORK_DIR/feature-2"  "$FEATURE_SHA"
+  upload_tracy "baseline-2" "$BENCH_WORK_DIR/baseline-2" "$BASELINE_SHA"
+  echo
+fi
+
+# ── Done (system restore happens via EXIT trap) ─────────────────────
+echo "═══════════════════════════════════════════════════════════"
+echo "  Benchmark complete!"
+echo "═══════════════════════════════════════════════════════════"
+echo "  Results  : $BENCH_WORK_DIR/summary.json"
+echo "  Markdown : $BENCH_WORK_DIR/comment.md"
+echo "  Charts   : $BENCH_WORK_DIR/charts/"
+if [ -f "$ERRORS_FILE" ]; then
+  echo "  Errors   : $ERRORS_FILE"
+fi
+echo "  Grafana  : $GRAFANA_URL"
+if [ "$TRACY" != "off" ]; then
+  echo "  ─── Tracy Profiles ───"
+  for run_dir in baseline-1 feature-1 feature-2 baseline-2; do
+    url_file="$BENCH_WORK_DIR/$run_dir/tracy_viewer_url.txt"
+    if [ -f "$url_file" ]; then
+      echo "  $run_dir : $(cat "$url_file")"
+    fi
+  done
+fi
+echo "═══════════════════════════════════════════════════════════"

.github/scripts/bench-reth-run.sh

@@ -6,17 +6,46 @@
 # Usage: bench-reth-run.sh <label> <binary> <output-dir>
 #
 # Required env: SCHELK_MOUNT, BENCH_RPC_URL, BENCH_BLOCKS, BENCH_WARMUP_BLOCKS
+# Optional env: BENCH_BIG_BLOCKS (true/false), BENCH_WORK_DIR (for big blocks path)
+#               BENCH_RETH_NEW_PAYLOAD (true/false, default true)
+#               BENCH_WAIT_TIME (duration like 500ms, default empty)
+#               BENCH_BASELINE_ARGS (extra reth node args for baseline runs)
+#               BENCH_FEATURE_ARGS (extra reth node args for feature runs)
+#               BENCH_OTLP_TRACES_ENDPOINT (OTLP HTTP endpoint for traces, e.g. https://host/insert/opentelemetry/v1/traces)
+#               BENCH_OTLP_LOGS_ENDPOINT (OTLP HTTP endpoint for logs, e.g. https://host/insert/opentelemetry/v1/logs)
 set -euo pipefail
 
 LABEL="$1"
 BINARY="$2"
 OUTPUT_DIR="$3"
-DATADIR="$SCHELK_MOUNT/datadir"
+DATADIR_NAME="datadir"
+if [ "${BENCH_BIG_BLOCKS:-false}" = "true" ]; then
+  DATADIR_NAME="datadir-big-blocks"
+fi
+DATADIR="$SCHELK_MOUNT/$DATADIR_NAME"
 mkdir -p "$OUTPUT_DIR"
 LOG="${OUTPUT_DIR}/node.log"
 
 cleanup() {
   kill "$TAIL_PID" 2>/dev/null || true
+  # Stop tracy-capture first (SIGINT makes it disconnect and flush to disk)
+  # Must happen before killing reth, otherwise reth keeps streaming data.
+  if [ -n "${TRACY_PID:-}" ] && kill -0 "$TRACY_PID" 2>/dev/null; then
+    echo "Stopping tracy-capture..."
+    kill -INT "$TRACY_PID" 2>/dev/null || true
+    for i in $(seq 1 30); do
+      kill -0 "$TRACY_PID" 2>/dev/null || break
+      if [ $((i % 10)) -eq 0 ]; then
+        echo "Waiting for tracy-capture to finish writing... (${i}s)"
+      fi
+      sleep 1
+    done
+    if kill -0 "$TRACY_PID" 2>/dev/null; then
+      echo "tracy-capture still running after 30s, killing..."
+      kill -9 "$TRACY_PID" 2>/dev/null || true
+    fi
+    wait "$TRACY_PID" 2>/dev/null || true
+  fi
   if [ -n "${RETH_PID:-}" ] && sudo kill -0 "$RETH_PID" 2>/dev/null; then
     if [ "${BENCH_SAMPLY:-false}" = "true" ]; then
       # Send SIGINT to the inner reth process by exact name (not -f which
@@ -53,8 +82,14 @@ cleanup() {
   fi
 }
 TAIL_PID=
+TRACY_PID=
 trap cleanup EXIT
 
+# Clean up stale schelk state from a previous cancelled run.
+# If schelk thinks it's still mounted (e.g. a cancelled run skipped cleanup),
+# recover first to reset state.
+sudo schelk recover -y -k || true
+
 # Mount
 sudo schelk mount -y
 sync
@@ -73,6 +108,8 @@ if [ "${BENCH_CORES:-0}" -gt 0 ] && [ "$BENCH_CORES" -lt "$MAX_RETH" ]; then
 fi
 RETH_CPUS="1-${MAX_RETH}"
 
+BIG_BLOCKS="${BENCH_BIG_BLOCKS:-false}"
+
 RETH_ARGS=(
   node
   --datadir "$DATADIR"
@@ -87,16 +124,71 @@ RETH_ARGS=(
   --no-persist-peers
 )
 
+# Gate flag on binary support (older baselines may not have it).
+# Uses --help which exits immediately via clap without node init.
+SYNC_STATE_IDLE=false
+if "$BINARY" node --help 2>/dev/null | grep -qF -- '--debug.startup-sync-state-idle'; then
+  RETH_ARGS+=(--debug.startup-sync-state-idle)
+  SYNC_STATE_IDLE=true
+fi
+
+# Append per-label extra node args (baseline or feature)
+EXTRA_NODE_ARGS=""
+case "$LABEL" in
+  baseline*) EXTRA_NODE_ARGS="${BENCH_BASELINE_ARGS:-}" ;;
+  feature*)  EXTRA_NODE_ARGS="${BENCH_FEATURE_ARGS:-}" ;;
+esac
+if [ -n "$EXTRA_NODE_ARGS" ]; then
+  # Word-split the string into individual args
+  # shellcheck disable=SC2206
+  RETH_ARGS+=($EXTRA_NODE_ARGS)
+fi
+
+if [ -n "${BENCH_METRICS_ADDR:-}" ]; then
+  RETH_ARGS+=(--metrics "$BENCH_METRICS_ADDR")
+fi
+
+# OTLP traces and logs export
+if [ -n "${BENCH_OTLP_TRACES_ENDPOINT:-}" ]; then
+  RETH_ARGS+=(--tracing-otlp="${BENCH_OTLP_TRACES_ENDPOINT}" --tracing-otlp.service-name=reth-bench)
+fi
+if [ -n "${BENCH_OTLP_LOGS_ENDPOINT:-}" ]; then
+  RETH_ARGS+=(--logs-otlp="${BENCH_OTLP_LOGS_ENDPOINT}" --logs-otlp.filter=debug)
+fi
+
+# Tracy profiling: add --log.tracy flags and set environment
+if [ "${BENCH_TRACY:-off}" != "off" ]; then
+  RETH_ARGS+=(--log.tracy --log.tracy.filter "${BENCH_TRACY_FILTER:-debug}")
+  if [ "${BENCH_TRACY}" = "on" ]; then
+    export TRACY_NO_SYS_TRACE=1
+  elif [ "${BENCH_TRACY}" = "full" ]; then
+    export TRACY_SAMPLING_HZ="${BENCH_TRACY_SAMPLING_HZ:-1}"
+  fi
+fi
+
+SUDO_ENV=()
+if [ -n "${OTEL_RESOURCE_ATTRIBUTES:-}" ]; then
+  SUDO_ENV+=("OTEL_RESOURCE_ATTRIBUTES=${OTEL_RESOURCE_ATTRIBUTES}")
+  SUDO_ENV+=("OTEL_BSP_MAX_QUEUE_SIZE=65536" "OTEL_BLRP_MAX_QUEUE_SIZE=65536")
+fi
+
+# Limit reth memory to 95% of available RAM to prevent OOM kills
+TOTAL_MEM_KB=$(awk '/^MemTotal:/ {print $2}' /proc/meminfo)
+MEM_LIMIT=$(( TOTAL_MEM_KB * 95 / 100 * 1024 ))
+echo "Memory limit: $(( MEM_LIMIT / 1024 / 1024 ))MB (95% of $(( TOTAL_MEM_KB / 1024 ))MB)"
+
 if [ "${BENCH_SAMPLY:-false}" = "true" ]; then
   RETH_ARGS+=(--log.samply)
   SAMPLY="$(which samply)"
-  sudo taskset -c "$RETH_CPUS" nice -n -20 \
+  sudo systemd-run --scope -p MemoryMax="$MEM_LIMIT" -p AllowedCPUs="$RETH_CPUS" \
+    env "${SUDO_ENV[@]}" nice -n -20 \
     "$SAMPLY" record --save-only --presymbolicate --rate 10000 \
     --output "$OUTPUT_DIR/samply-profile.json.gz" \
     -- "$BINARY" "${RETH_ARGS[@]}" \
     > "$LOG" 2>&1 &
 else
-  sudo taskset -c "$RETH_CPUS" nice -n -20 "$BINARY" "${RETH_ARGS[@]}" \
+  sudo systemd-run --scope -p MemoryMax="$MEM_LIMIT" -p AllowedCPUs="$RETH_CPUS" \
+    env "${SUDO_ENV[@]}" nice -n -20 "$BINARY" "${RETH_ARGS[@]}" \
     > "$LOG" 2>&1 &
 fi
 
@@ -109,7 +201,7 @@ for i in $(seq 1 60); do
     -H 'Content-Type: application/json' \
     -d '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}' \
     > /dev/null 2>&1; then
-    echo "reth (${LABEL}) is ready after ${i}s"
+    echo "reth (${LABEL}) RPC is up after ${i}s"
     break
   fi
   if [ "$i" -eq 60 ]; then
@@ -120,25 +212,96 @@ for i in $(seq 1 60); do
   sleep 1
 done
 
+# Wait for the pipeline to finish (eth_syncing returns false) so the
+# engine is in live mode and can accept newPayload calls.
+# Only possible when --debug.startup-sync-state-idle is supported.
+if [ "$SYNC_STATE_IDLE" = "true" ]; then
+  for i in $(seq 1 300); do
+    SYNC_RESULT=$(curl -sf http://127.0.0.1:8545 -X POST \
+      -H 'Content-Type: application/json' \
+      -d '{"jsonrpc":"2.0","method":"eth_syncing","params":[],"id":1}' 2>/dev/null || true)
+    if [ -n "$SYNC_RESULT" ] && jq -e '.result == false' <<< "$SYNC_RESULT" > /dev/null 2>&1; then
+      echo "reth (${LABEL}) pipeline finished after ${i}s, engine is live"
+      break
+    fi
+    if [ "$i" -eq 300 ]; then
+      echo "::error::reth (${LABEL}) pipeline did not finish within 300s"
+      cat "$LOG"
+      exit 1
+    fi
+    sleep 1
+  done
+else
+  echo "reth (${LABEL}) binary does not support --debug.startup-sync-state-idle, skipping sync wait"
+fi
+
 # Run reth-bench with high priority but as the current user so output
 # files are not root-owned (avoids EACCES on next checkout).
 BENCH_NICE="sudo nice -n -20 sudo -u $(id -un)"
 
-# Warmup
-$BENCH_NICE "$RETH_BENCH" new-payload-fcu \
-  --rpc-url "$BENCH_RPC_URL" \
-  --engine-rpc-url http://127.0.0.1:8551 \
-  --jwt-secret "$DATADIR/jwt.hex" \
-  --advance "${BENCH_WARMUP_BLOCKS:-50}" \
-  --reth-new-payload 2>&1 | sed -u "s/^/[bench] /"
+# Build optional flags
+EXTRA_BENCH_ARGS=()
+if [ "${BENCH_RETH_NEW_PAYLOAD:-true}" != "false" ]; then
+  EXTRA_BENCH_ARGS+=(--reth-new-payload --wait-for-persistence)
+fi
+if [ -n "${BENCH_WAIT_TIME:-}" ]; then
+  EXTRA_BENCH_ARGS+=(--wait-time "$BENCH_WAIT_TIME")
+fi
 
-# Benchmark
-$BENCH_NICE "$RETH_BENCH" new-payload-fcu \
-  --rpc-url "$BENCH_RPC_URL" \
-  --engine-rpc-url http://127.0.0.1:8551 \
-  --jwt-secret "$DATADIR/jwt.hex" \
-  --advance "$BENCH_BLOCKS" \
-  --reth-new-payload \
-  --output "$OUTPUT_DIR" 2>&1 | sed -u "s/^/[bench] /"
+if [ "$BIG_BLOCKS" = "true" ]; then
+  # Big blocks mode: replay pre-generated payloads
+  BIG_BLOCKS_DIR="${BENCH_BIG_BLOCKS_DIR:-${BENCH_WORK_DIR}/big-blocks}"
+
+  # Start tracy-capture so profile only covers the benchmark
+  if [ "${BENCH_TRACY:-off}" != "off" ]; then
+    echo "Starting tracy-capture..."
+    tracy-capture -f -o "$OUTPUT_DIR/tracy-profile.tracy" &
+    TRACY_PID=$!
+    sleep 0.5  # give tracy-capture time to connect
+  fi
+
+  BB_BENCH_ARGS=(--reth-new-payload --wait-for-persistence)
+  if [ -n "${BENCH_WAIT_TIME:-}" ]; then
+    BB_BENCH_ARGS+=(--wait-time "$BENCH_WAIT_TIME")
+  fi
+  # Limit number of payloads if blocks count is specified
+  if [ "${BENCH_BLOCKS:-0}" -gt 0 ] 2>/dev/null; then
+    BB_BENCH_ARGS+=(--count "$BENCH_BLOCKS")
+  fi
+
+  echo "Running big blocks benchmark (replay-payloads)..."
+  $BENCH_NICE "$RETH_BENCH" replay-payloads \
+    "${BB_BENCH_ARGS[@]}" \
+    --payload-dir "$BIG_BLOCKS_DIR/payloads" \
+    --engine-rpc-url http://127.0.0.1:8551 \
+    --jwt-secret "$DATADIR/jwt.hex" \
+    --output "$OUTPUT_DIR" 2>&1 | sed -u "s/^/[bench] /"
+else
+  # Standard mode: warmup + new-payload-fcu
+  # Warmup
+  $BENCH_NICE "$RETH_BENCH" new-payload-fcu \
+    --rpc-url "$BENCH_RPC_URL" \
+    --engine-rpc-url http://127.0.0.1:8551 \
+    --jwt-secret "$DATADIR/jwt.hex" \
+    --advance "${BENCH_WARMUP_BLOCKS:-50}" \
+    "${EXTRA_BENCH_ARGS[@]}" 2>&1 | sed -u "s/^/[bench] /"
+
+  # Start tracy-capture after warmup so profile only covers the benchmark
+  if [ "${BENCH_TRACY:-off}" != "off" ]; then
+    echo "Starting tracy-capture..."
+    tracy-capture -f -o "$OUTPUT_DIR/tracy-profile.tracy" &
+    TRACY_PID=$!
+    sleep 0.5  # give tracy-capture time to connect
+  fi
+
+  # Benchmark
+  $BENCH_NICE "$RETH_BENCH" new-payload-fcu \
+    --rpc-url "$BENCH_RPC_URL" \
+    --engine-rpc-url http://127.0.0.1:8551 \
+    --jwt-secret "$DATADIR/jwt.hex" \
+    --advance "$BENCH_BLOCKS" \
+    "${EXTRA_BENCH_ARGS[@]}" \
+    --output "$OUTPUT_DIR" 2>&1 | sed -u "s/^/[bench] /"
+fi
 
 # cleanup runs via trap

.github/scripts/bench-reth-snapshot.sh

@@ -1,15 +1,17 @@
 #!/usr/bin/env bash
 #
-# Downloads the latest nightly snapshot into the schelk volume with
-# progress reporting to the GitHub PR comment.
+# Downloads the latest snapshot into the schelk volume using
+# `reth download` with progress reporting to the GitHub PR comment.
 #
-# Skips the download if the local ETag marker matches the remote one.
+# Skips the download if the manifest content hasn't changed since
+# the last successful download (checked via SHA-256 of the manifest).
 #
 # Usage: bench-reth-snapshot.sh [--check]
-#   --check   Only check if a download is needed; exits 0 if up-to-date, 1 if not.
+#   --check   Only check if a download is needed; exits 0 if up-to-date, 10 if not.
 #
 # Required env:
 #   SCHELK_MOUNT       – schelk mount point (e.g. /reth-bench)
+#   BENCH_RETH_BINARY  – path to the reth binary
 #   GITHUB_TOKEN       – token for GitHub API calls (only for download)
 #   BENCH_COMMENT_ID   – PR comment ID to update (optional)
 #   BENCH_REPO         – owner/repo (e.g. paradigmxyz/reth)
@@ -18,52 +20,74 @@
 #   BENCH_CONFIG       – config summary line
 set -euo pipefail
 
-BUCKET="minio/reth-snapshots/reth-1-minimal-nightly-previous.tar.zst"
-DATADIR="$SCHELK_MOUNT/datadir"
-ETAG_FILE="$HOME/.reth-bench-snapshot-etag"
-
-# Get remote metadata via JSON for reliable parsing
-MC_STAT=$(mc stat --json "$BUCKET" 2>/dev/null || true)
-REMOTE_ETAG=$(echo "$MC_STAT" | jq -r '.etag // empty')
-if [ -z "$REMOTE_ETAG" ]; then
-  echo "::warning::Failed to get ETag from mc stat, will re-download"
-  REMOTE_ETAG="unknown-$(date +%s)"
+MC="mc"
+BUCKET="minio/reth-snapshots"
+# Allow overriding the snapshot name (e.g. for big-blocks mode where the
+# big-blocks manifest specifies which base snapshot to use).
+SNAPSHOT_NAME="${BENCH_SNAPSHOT_NAME:-reth-1-minimal-stable}"
+MANIFEST_PATH="${SNAPSHOT_NAME}/manifest.json"
+DATADIR_NAME="datadir"
+HASH_MODE_SUFFIX=""
+if [ "${BENCH_BIG_BLOCKS:-false}" = "true" ]; then
+  DATADIR_NAME="datadir-big-blocks"
+  HASH_MODE_SUFFIX="-big-blocks"
 fi
+DATADIR="$SCHELK_MOUNT/$DATADIR_NAME"
+HASH_FILE="$HOME/.reth-bench-snapshot-hash${HASH_MODE_SUFFIX}"
 
-LOCAL_ETAG=""
-[ -f "$ETAG_FILE" ] && LOCAL_ETAG=$(cat "$ETAG_FILE")
+# Fetch manifest and compute content hash for reliable freshness check
+MANIFEST_CONTENT=$($MC cat "${BUCKET}/${MANIFEST_PATH}" 2>/dev/null) || {
+  echo "::error::Failed to fetch snapshot manifest from ${BUCKET}/${MANIFEST_PATH}"
+  exit 2
+}
+REMOTE_HASH=$(echo "$MANIFEST_CONTENT" | sha256sum | awk '{print $1}')
 
-if [ "$REMOTE_ETAG" = "$LOCAL_ETAG" ]; then
-  echo "Snapshot is up-to-date (ETag: ${REMOTE_ETAG})"
-  if [ "${1:-}" = "--check" ]; then
-    exit 0
-  fi
+LOCAL_HASH=""
+[ -f "$HASH_FILE" ] && LOCAL_HASH=$(cat "$HASH_FILE")
+
+if [ "$REMOTE_HASH" = "$LOCAL_HASH" ]; then
+  echo "Snapshot is up-to-date (manifest hash: ${REMOTE_HASH:0:16}…)"
   exit 0
 fi
 
-echo "Snapshot needs update (local: ${LOCAL_ETAG:-<none>}, remote: ${REMOTE_ETAG})"
+echo "Snapshot needs update (local: ${LOCAL_HASH:+${LOCAL_HASH:0:16}…}${LOCAL_HASH:-<none>}, remote: ${REMOTE_HASH:0:16}…)"
 if [ "${1:-}" = "--check" ]; then
+  exit 10
+fi
+
+RETH="${BENCH_RETH_BINARY:?BENCH_RETH_BINARY must be set}"
+if [ ! -x "$RETH" ]; then
+  echo "::error::reth binary not found or not executable at $RETH"
   exit 1
 fi
 
-# Get compressed size for progress tracking
-TOTAL_BYTES=$(echo "$MC_STAT" | jq -r '.size // empty')
-if [ -z "$TOTAL_BYTES" ] || [ "$TOTAL_BYTES" = "0" ]; then
-  echo "::error::Failed to get snapshot size from mc stat"
+# Resolve the MinIO HTTP endpoint from the mc alias so reth can
+# fetch archives over HTTP (the manifest's embedded base_url points
+# to the cluster-internal address which is unreachable from runners).
+MINIO_ENDPOINT=$($MC alias list minio --json 2>/dev/null | jq -r '.URL // empty') || true
+if [ -z "$MINIO_ENDPOINT" ]; then
+  echo "::error::Failed to resolve MinIO endpoint from mc alias 'minio'"
   exit 1
 fi
-echo "Snapshot size: $TOTAL_BYTES bytes ($(numfmt --to=iec "$TOTAL_BYTES"))"
+BASE_URL="${MINIO_ENDPOINT}/reth-snapshots/${SNAPSHOT_NAME}"
+
+# Rewrite manifest's base_url with the runner-reachable endpoint
+MANIFEST_TMP=$(mktemp --suffix=.json)
+trap 'rm -f -- "$MANIFEST_TMP"' EXIT
+echo "$MANIFEST_CONTENT" \
+  | jq --arg base "$BASE_URL" '.base_url = $base' > "$MANIFEST_TMP"
 
 # Prepare mount
 mountpoint -q "$SCHELK_MOUNT" && sudo schelk recover -y || true
 sudo schelk mount -y
 sudo rm -rf "$DATADIR"
 sudo mkdir -p "$DATADIR"
+# reth download runs as current user (not root), needs write access
+sudo chown -R "$(id -u):$(id -g)" "$DATADIR"
 
 update_comment() {
-  local pct="$1"
+  local status="$1"
   [ -z "${BENCH_COMMENT_ID:-}" ] && return 0
-  local status="Building binaries & downloading snapshot… ${pct}%"
   local body
   body="$(printf 'cc @%s\n\n🚀 Benchmark started! [View job](%s)\n\n⏳ **Status:** %s\n\n%s' \
     "$BENCH_ACTOR" "$BENCH_JOB_URL" "$status" "$BENCH_CONFIG")"
@@ -75,53 +99,31 @@ update_comment() {
     > /dev/null 2>&1 || true
 }
 
-# Track compressed bytes flowing through the pipe
-DL_BYTES_FILE=$(mktemp)
-echo 0 > "$DL_BYTES_FILE"
+update_comment "Downloading snapshot…"
 
-# Start progress reporter in background
-(
-  while true; do
-    sleep 10
-    CURRENT=$(cat "$DL_BYTES_FILE" 2>/dev/null || echo 0)
-    if [ "$TOTAL_BYTES" -gt 0 ]; then
-      PCT=$(( CURRENT * 100 / TOTAL_BYTES ))
-      [ "$PCT" -gt 100 ] && PCT=100
-      echo "Snapshot download: $(numfmt --to=iec "$CURRENT") / $(numfmt --to=iec "$TOTAL_BYTES") (${PCT}%)"
-      update_comment "$PCT"
-    fi
-  done
-) &
-PROGRESS_PID=$!
-trap 'kill $PROGRESS_PID 2>/dev/null || true; rm -f "$DL_BYTES_FILE"' EXIT
+# Download using reth download (manifest-path with rewritten base_url)
+"$RETH" download \
+  --manifest-path "$MANIFEST_TMP" \
+  -y \
+  --minimal \
+  --datadir "$DATADIR"
 
-# Download and extract; python byte counter tracks compressed bytes received
-mc cat "$BUCKET" | python3 -c "
-import sys
-count = 0
-while True:
-    data = sys.stdin.buffer.read(1048576)
-    if not data:
-        break
-    count += len(data)
-    sys.stdout.buffer.write(data)
-    with open('$DL_BYTES_FILE', 'w') as f:
-        f.write(str(count))
-" | pzstd -d -p 6 | sudo tar -xf - -C "$DATADIR"
-
-# Stop progress reporter
-kill $PROGRESS_PID 2>/dev/null || true
-wait $PROGRESS_PID 2>/dev/null || true
-
-update_comment "100"
+update_comment "Downloading snapshot… done"
 echo "Snapshot download complete"
 
+# Sanity check: verify expected directories exist
+if [ ! -d "$DATADIR/db" ] || [ ! -d "$DATADIR/static_files" ]; then
+  echo "::error::Snapshot download did not produce expected directory layout (missing db/ or static_files/)"
+  ls -la "$DATADIR" || true
+  exit 1
+fi
+
 # Promote the new snapshot to become the schelk baseline (virgin volume).
 # This copies changed blocks from scratch → virgin so that future
 # `schelk recover` calls restore to this new state.
 sync
 sudo schelk promote -y
 
-# Save ETag marker
-echo "$REMOTE_ETAG" > "$ETAG_FILE"
-echo "Snapshot promoted to schelk baseline (ETag: ${REMOTE_ETAG})"
+# Save manifest hash
+echo "$REMOTE_HASH" > "$HASH_FILE"
+echo "Snapshot promoted to schelk baseline (manifest hash: ${REMOTE_HASH:0:16}…)"

.github/scripts/bench-reth-summary.py

@@ -107,6 +107,10 @@ def compute_stats(combined: list[dict]) -> dict:
             mgas_s_values.append(r["gas_used"] / lat_s / 1_000_000)
     mean_mgas_s = sum(mgas_s_values) / len(mgas_s_values) if mgas_s_values else 0
 
+    total_latencies_ms = [r["total_latency_us"] / 1_000 for r in combined]
+    wall_clock_s = sum(total_latencies_ms) / 1_000
+    mean_total_lat_ms = sum(total_latencies_ms) / n
+
     return {
         "n": n,
         "mean_ms": mean_lat,
@@ -115,6 +119,8 @@ def compute_stats(combined: list[dict]) -> dict:
         "p90_ms": percentile(sorted_lat, 90),
         "p99_ms": percentile(sorted_lat, 99),
         "mean_mgas_s": mean_mgas_s,
+        "wall_clock_s": wall_clock_s,
+        "mean_total_lat_ms": mean_total_lat_ms,
     }
 
 
@@ -139,13 +145,14 @@ def compute_wait_stats(combined: list[dict], field: str) -> dict:
 
 def _paired_data(
     baseline: list[dict], feature: list[dict]
-) -> tuple[list[tuple[float, float]], list[float], list[float]]:
+) -> tuple[list[tuple[float, float]], list[float], list[float], list[float]]:
     """Match blocks and return paired latencies and per-block diffs.
 
     Returns:
         pairs: list of (baseline_ms, feature_ms) tuples
         lat_diffs_ms: list of feature − baseline latency diffs in ms
         mgas_diffs: list of feature − baseline Mgas/s diffs
+        total_lat_diffs_ms: list of feature − baseline total latency diffs in ms
     """
     baseline_by_block = {r["block_number"]: r for r in baseline}
     feature_by_block = {r["block_number"]: r for r in feature}
@@ -154,6 +161,7 @@ def _paired_data(
     pairs = []
     lat_diffs_ms = []
     mgas_diffs = []
+    total_lat_diffs_ms = []
     for bn in common_blocks:
         b = baseline_by_block[bn]
         f = feature_by_block[bn]
@@ -168,7 +176,10 @@ def _paired_data(
                 f["gas_used"] / f_lat_s / 1_000_000
                 - b["gas_used"] / b_lat_s / 1_000_000
             )
-    return pairs, lat_diffs_ms, mgas_diffs
+        total_lat_diffs_ms.append(
+            f["total_latency_us"] / 1_000 - b["total_latency_us"] / 1_000
+        )
+    return pairs, lat_diffs_ms, mgas_diffs, total_lat_diffs_ms
 
 
 def compute_paired_stats(
@@ -183,12 +194,14 @@ def compute_paired_stats(
     all_pairs = []
     all_lat_diffs = []
     all_mgas_diffs = []
+    all_total_lat_diffs = []
     blocks_per_pair = []
     for baseline, feature in zip(baseline_runs, feature_runs):
-        pairs, lat_diffs, mgas_diffs = _paired_data(baseline, feature)
+        pairs, lat_diffs, mgas_diffs, total_lat_diffs = _paired_data(baseline, feature)
         all_pairs.extend(pairs)
         all_lat_diffs.extend(lat_diffs)
         all_mgas_diffs.extend(mgas_diffs)
+        all_total_lat_diffs.extend(total_lat_diffs)
         blocks_per_pair.append(len(pairs))
 
     if not all_lat_diffs:
@@ -227,6 +240,11 @@ def compute_paired_stats(
     mgas_se = std_mgas_diff / math.sqrt(len(all_mgas_diffs)) if all_mgas_diffs else 0.0
     mgas_ci = T_CRITICAL * mgas_se
 
+    mean_total_diff = sum(all_total_lat_diffs) / len(all_total_lat_diffs) if all_total_lat_diffs else 0.0
+    std_total_diff = stddev(all_total_lat_diffs, mean_total_diff) if len(all_total_lat_diffs) > 1 else 0.0
+    total_se = std_total_diff / math.sqrt(len(all_total_lat_diffs)) if all_total_lat_diffs else 0.0
+    wall_clock_ci_ms = T_CRITICAL * total_se
+
     return {
         "n": n,
         "mean_diff_ms": mean_diff,
@@ -239,6 +257,7 @@ def compute_paired_stats(
         "p99_ci_ms": (p99_boot[hi] - p99_boot[lo]) / 2,
         "mean_mgas_diff": mean_mgas_diff,
         "mgas_ci": mgas_ci,
+        "wall_clock_ci_ms": wall_clock_ci_ms,
         "blocks": max(blocks_per_pair),
     }
 
@@ -267,6 +286,10 @@ def fmt_mgas(v: float) -> str:
     return f"{v:.2f}"
 
 
+def fmt_s(v: float) -> str:
+    return f"{v:.2f}s"
+
+
 def significance(pct: float, ci_pct: float, lower_is_better: bool) -> str:
     """Return significance label: 'good', 'bad', or 'neutral'."""
     significant = abs(pct) > ci_pct
@@ -304,6 +327,7 @@ def compute_changes(
         ("p90", "p90_ms", "p90_ci_ms", "p90_ms", True),
         ("p99", "p99_ms", "p99_ci_ms", "p99_ms", True),
         ("mgas_s", "mean_mgas_s", "mgas_ci", "mean_mgas_s", False),
+        ("wall_clock", "wall_clock_s", "wall_clock_ci_ms", "mean_total_lat_ms", True),
     ]
     changes = {}
     for name, stat_key, ci_key, base_key, lower_is_better in metrics:
@@ -326,6 +350,7 @@ def generate_comparison_table(
     baseline_name: str,
     feature_name: str,
     feature_sha: str,
+    big_blocks: bool = False,
 ) -> str:
     """Generate a markdown comparison table between baseline and feature."""
     n = paired["blocks"]
@@ -335,6 +360,7 @@ def generate_comparison_table(
 
     mean_pct = pct(run1["mean_ms"], run2["mean_ms"])
     gas_pct = pct(run1["mean_mgas_s"], run2["mean_mgas_s"])
+    wall_pct = pct(run1["wall_clock_s"], run2["wall_clock_s"])
 
     p50_pct = pct(run1["p50_ms"], run2["p50_ms"])
     p90_pct = pct(run1["p90_ms"], run2["p90_ms"])
@@ -348,6 +374,7 @@ def generate_comparison_table(
     # CI as a percentage of baseline mean
     lat_ci_pct = paired["ci_ms"] / run1["mean_ms"] * 100.0 if run1["mean_ms"] > 0 else 0.0
     mgas_ci_pct = paired["mgas_ci"] / run1["mean_mgas_s"] * 100.0 if run1["mean_mgas_s"] > 0 else 0.0
+    wall_ci_pct = paired["wall_clock_ci_ms"] / run1["mean_total_lat_ms"] * 100.0 if run1["mean_total_lat_ms"] > 0 else 0.0
 
     base_url = f"https://github.com/{repo}/commit"
     baseline_label = f"[`{baseline_name}`]({base_url}/{baseline_ref})"
@@ -362,8 +389,9 @@ def generate_comparison_table(
         f"| P90 | {fmt_ms(run1['p90_ms'])} | {fmt_ms(run2['p90_ms'])} | {change_str(p90_pct, p90_ci_pct, lower_is_better=True)} |",
         f"| P99 | {fmt_ms(run1['p99_ms'])} | {fmt_ms(run2['p99_ms'])} | {change_str(p99_pct, p99_ci_pct, lower_is_better=True)} |",
         f"| Mgas/s | {fmt_mgas(run1['mean_mgas_s'])} | {fmt_mgas(run2['mean_mgas_s'])} | {change_str(gas_pct, mgas_ci_pct, lower_is_better=False)} |",
+        f"| Wall Clock | {fmt_s(run1['wall_clock_s'])} | {fmt_s(run2['wall_clock_s'])} | {change_str(wall_pct, wall_ci_pct, lower_is_better=True)} |",
         "",
-        f"*{n} blocks*",
+        f"*{n} {'big blocks' if big_blocks else 'blocks'}*",
     ]
     return "\n".join(lines)
 
@@ -394,6 +422,7 @@ def generate_markdown(
     summary: dict, comparison_table: str,
     wait_time_tables: list[str] | None = None,
     behind_baseline: int = 0, repo: str = "", baseline_ref: str = "", baseline_name: str = "",
+    grafana_url: str | None = None,
 ) -> str:
     """Generate a markdown comment body."""
     lines = ["## Benchmark Results", ""]
@@ -413,6 +442,9 @@ def generate_markdown(
                 lines.append(table)
                 lines.append("")
         lines.append("</details>")
+    if grafana_url:
+        lines.append("")
+        lines.append(f"**[Grafana Dashboard]({grafana_url})**")
     return "\n".join(lines)
 
 
@@ -439,6 +471,8 @@ def main():
     parser.add_argument("--feature-name", "--branch-name", default=None, help="Feature branch name")
     parser.add_argument("--feature-ref", "--branch-sha", "--feature-sha", default=None, help="Feature commit SHA")
     parser.add_argument("--behind-baseline", "--behind-main", type=int, default=0, help="Commits behind baseline")
+    parser.add_argument("--big-blocks", action="store_true", default=False, help="Big blocks mode")
+    parser.add_argument("--grafana-url", default=None, help="Grafana dashboard URL for this benchmark run")
     args = parser.parse_args()
 
     if len(args.baseline_csv) != len(args.feature_csv):
@@ -487,6 +521,7 @@ def main():
         baseline_name=baseline_name,
         feature_name=feature_name,
         feature_sha=feature_sha,
+        big_blocks=args.big_blocks,
     )
     print(f"Generated comparison ({paired_stats['n']} paired blocks, "
           f"mean diff {paired_stats['mean_diff_ms']:+.3f}ms ± {paired_stats['ci_ms']:.3f}ms)")
@@ -517,6 +552,7 @@ def main():
 
     summary = {
         "blocks": paired_stats["blocks"],
+        "big_blocks": args.big_blocks,
         "baseline": {
             "name": baseline_name,
             "ref": baseline_ref,
@@ -542,6 +578,7 @@ def main():
         repo=args.repo,
         baseline_ref=baseline_ref,
         baseline_name=baseline_name,
+        grafana_url=args.grafana_url,
     )
 
     with open(args.output_markdown, "w") as f:

.github/scripts/bench-scheduled-refs.sh

@@ -0,0 +1,231 @@
+#!/usr/bin/env bash
+#
+# Resolves baseline and feature refs for scheduled benchmark runs.
+#
+# Supports two modes:
+#   nightly — Queries the latest successful scheduled docker.yml run via
+#             GitHub API to find the nightly Docker image commit. Compares
+#             with the last successful feature ref to detect staleness.
+#   hourly  — Compares origin/main HEAD against the last successfully
+#             benchmarked commit (falls back to HEAD~1 on first run).
+#             Checks for in-progress sibling runs to avoid overlap.
+#
+# Usage: bench-scheduled-refs.sh <force> <mode>
+#   force — "true" to run even if no new commit (bypass skip logic)
+#   mode  — "nightly" or "hourly"
+#
+# Outputs (via GITHUB_OUTPUT):
+#   baseline-ref    — commit SHA for baseline
+#   feature-ref     — commit SHA for feature
+#   should-skip     — "true" if no new commit since last run or sibling in progress
+#   is-stale        — "true" if latest nightly build is >24h old (nightly only)
+#   stale-age-hours — age of the nightly build in hours (nightly only)
+#   nightly-created — ISO timestamp of the nightly build (nightly only)
+#
+# Reads:
+#   .nightly-state/last-feature-ref  (nightly, from GH Actions cache)
+#   .hourly-state/last-feature-ref   (hourly, from GH Actions cache)
+#
+# Requires: gh (GitHub CLI), jq, date, git (hourly mode)
+set -euo pipefail
+
+FORCE="${1:-false}"
+MODE="${2:-nightly}"
+REPO="${GITHUB_REPOSITORY:-paradigmxyz/reth}"
+
+echo "Mode: $MODE, Force: $FORCE"
+
+# ==========================================================================
+# Hourly mode: compare origin/main HEAD vs HEAD~1
+# ==========================================================================
+if [ "$MODE" = "hourly" ]; then
+
+  # --- Step 1: Resolve feature ref from git ---
+  echo "::group::Resolving hourly refs from git"
+  git fetch origin main --quiet
+  FEATURE_REF=$(git rev-parse origin/main)
+  echo "Feature (HEAD): $FEATURE_REF"
+  echo "::endgroup::"
+
+  # --- Step 2: Check for in-progress sibling runs ---
+  echo "::group::Checking for in-progress sibling runs"
+  CURRENT_RUN_ID="${GITHUB_RUN_ID:-0}"
+  IN_PROGRESS=$(gh run list \
+    -R "$REPO" \
+    --workflow=bench-scheduled.yml \
+    --status=in_progress \
+    --json databaseId \
+    --jq "[.[] | select(.databaseId != $CURRENT_RUN_ID)] | length")
+
+  SHOULD_SKIP="false"
+  if [ "$IN_PROGRESS" -gt 0 ]; then
+    echo "::warning::Previous bench run still in progress ($IN_PROGRESS sibling run(s) found). Skipping."
+    SHOULD_SKIP="true"
+    # Output a flag so the workflow can send a Slack alert
+    echo "long-running=true" >> "$GITHUB_OUTPUT"
+  else
+    echo "No in-progress sibling runs"
+    echo "long-running=false" >> "$GITHUB_OUTPUT"
+  fi
+  echo "::endgroup::"
+
+  # --- Step 3: Read last successful feature ref from cache ---
+  echo "::group::Reading cached state"
+  LAST_FEATURE_REF=""
+  STATE_FILE=".hourly-state/last-feature-ref"
+  if [ -f "$STATE_FILE" ]; then
+    LAST_FEATURE_REF=$(tr -d '[:space:]' < "$STATE_FILE")
+    echo "Previous feature ref: $LAST_FEATURE_REF"
+  else
+    echo "No cached state found (first run)"
+  fi
+  echo "::endgroup::"
+
+  # --- Step 4: Determine baseline and skip logic ---
+  echo "::group::Resolving baseline and skip logic"
+  if [ "$SHOULD_SKIP" = "true" ]; then
+    BASELINE_REF=$(git rev-parse origin/main~1)
+    echo "Already marked skip (sibling in progress)"
+  elif [ -z "$LAST_FEATURE_REF" ]; then
+    # First run: no previous state, fall back to HEAD~1
+    BASELINE_REF=$(git rev-parse origin/main~1)
+    echo "First run — using HEAD~1 as baseline"
+  elif [ "$LAST_FEATURE_REF" = "$FEATURE_REF" ]; then
+    BASELINE_REF="$LAST_FEATURE_REF"
+    if [ "$FORCE" = "true" ] || [ "$FORCE" = "--force" ]; then
+      echo "No new commits on main, but force=true — running anyway"
+    else
+      SHOULD_SKIP="true"
+      echo "No new commits on main since last run — will skip"
+    fi
+  else
+    # Normal case: use last benchmarked commit as baseline
+    BASELINE_REF="$LAST_FEATURE_REF"
+    echo "New commit(s) on main detected — comparing against last benchmarked commit"
+  fi
+
+  echo "Baseline: $BASELINE_REF"
+  echo "Feature:  $FEATURE_REF"
+  echo "Skip:     $SHOULD_SKIP"
+  echo "::endgroup::"
+
+  # --- Step 5: Write outputs ---
+  {
+    echo "baseline-ref=$BASELINE_REF"
+    echo "feature-ref=$FEATURE_REF"
+    echo "should-skip=$SHOULD_SKIP"
+    echo "is-stale=false"
+    echo "stale-age-hours=0"
+    echo "nightly-created="
+  } >> "$GITHUB_OUTPUT"
+  exit 0
+fi
+
+# ==========================================================================
+# Nightly mode: query latest Docker nightly build (original logic)
+# ==========================================================================
+
+# --- Step 1: Query latest successful scheduled docker.yml run ---
+echo "::group::Querying latest nightly docker build"
+
+RUNS_JSON=$(gh run list \
+  -R "$REPO" \
+  --workflow=docker.yml \
+  --event=schedule \
+  --status=completed \
+  --limit 5 \
+  --json headSha,createdAt,conclusion)
+
+# Find the most recent successful run
+LATEST=$(echo "$RUNS_JSON" | jq -r '[.[] | select(.conclusion == "success")] | first // empty')
+
+if [ -z "$LATEST" ]; then
+  echo "::error::No successful scheduled docker.yml run found in the last 5 runs"
+  echo "Runs found: $RUNS_JSON"
+  exit 1
+fi
+
+FEATURE_REF=$(echo "$LATEST" | jq -r '.headSha')
+CREATED_AT=$(echo "$LATEST" | jq -r '.createdAt')
+echo "Latest nightly commit: $FEATURE_REF"
+echo "Built at: $CREATED_AT"
+echo "::endgroup::"
+
+# --- Step 2: Staleness check ---
+echo "::group::Checking staleness"
+NOW_EPOCH=$(date +%s)
+# Handle both GNU date (-d) and BSD date (-j -f) for cross-platform compat
+CREATED_EPOCH=$(date -d "$CREATED_AT" +%s 2>/dev/null || \
+  date -j -f "%Y-%m-%dT%H:%M:%SZ" "$CREATED_AT" +%s 2>/dev/null || \
+  date -j -f "%Y-%m-%dT%T%z" "$CREATED_AT" +%s 2>/dev/null || \
+  { echo "::error::Cannot parse date: $CREATED_AT"; exit 1; })
+
+AGE_SECONDS=$(( NOW_EPOCH - CREATED_EPOCH ))
+AGE_HOURS=$(( AGE_SECONDS / 3600 ))
+IS_STALE="false"
+
+if [ "$AGE_HOURS" -gt 24 ]; then
+  IS_STALE="true"
+  echo "::warning::STALE NIGHTLY: Build is ${AGE_HOURS}h old (>24h threshold)"
+  echo "This indicates the nightly docker build failed — no new image was produced"
+else
+  echo "Nightly build age: ${AGE_HOURS}h (within 24h threshold)"
+fi
+echo "::endgroup::"
+
+# --- Step 3: Read last successful feature ref from cache ---
+echo "::group::Reading cached state"
+LAST_FEATURE_REF=""
+STATE_FILE=".nightly-state/last-feature-ref"
+if [ -f "$STATE_FILE" ]; then
+  LAST_FEATURE_REF=$(tr -d '[:space:]' < "$STATE_FILE")
+  echo "Previous feature ref: $LAST_FEATURE_REF"
+else
+  echo "No cached state found (first run)"
+fi
+echo "::endgroup::"
+
+# --- Step 4: Determine baseline and skip logic ---
+echo "::group::Resolving refs"
+SHOULD_SKIP="false"
+BASELINE_REF="$FEATURE_REF"  # default for first run
+
+if [ "$IS_STALE" = "true" ]; then
+  # Stale = error path, don't skip (will alert and fail downstream)
+  SHOULD_SKIP="false"
+  BASELINE_REF="${LAST_FEATURE_REF:-$FEATURE_REF}"
+  echo "Stale nightly detected — will alert and fail"
+elif [ -z "$LAST_FEATURE_REF" ]; then
+  # First run: baseline = feature (self-comparison to establish baseline)
+  BASELINE_REF="$FEATURE_REF"
+  echo "First run — will benchmark nightly against itself to establish baseline"
+elif [ "$LAST_FEATURE_REF" = "$FEATURE_REF" ]; then
+  # No new nightly since last successful run
+  if [ "$FORCE" = "true" ] || [ "$FORCE" = "--force" ]; then
+    echo "No new nightly, but force=true — running anyway"
+    BASELINE_REF="$LAST_FEATURE_REF"
+  else
+    SHOULD_SKIP="true"
+    echo "No new nightly since last run — will skip"
+  fi
+else
+  # Normal case: new nightly available
+  BASELINE_REF="$LAST_FEATURE_REF"
+  echo "New nightly detected"
+fi
+
+echo "Baseline: $BASELINE_REF"
+echo "Feature:  $FEATURE_REF"
+echo "Skip:     $SHOULD_SKIP"
+echo "Stale:    $IS_STALE"
+echo "::endgroup::"
+
+# --- Step 5: Write outputs ---
+{
+  echo "baseline-ref=$BASELINE_REF"
+  echo "feature-ref=$FEATURE_REF"
+  echo "should-skip=$SHOULD_SKIP"
+  echo "is-stale=$IS_STALE"
+  echo "stale-age-hours=$AGE_HOURS"
+  echo "nightly-created=$CREATED_AT"
+} >> "$GITHUB_OUTPUT"

.github/scripts/bench-slack-notify.js

@@ -7,6 +7,8 @@
 //   BENCH_PR               – PR number (may be empty)
 //   BENCH_ACTOR            – GitHub user who triggered the bench
 //   BENCH_JOB_URL          – URL to the Actions job page
+//   BENCH_BASELINE_ARGS    – Extra CLI args for the baseline reth node
+//   BENCH_FEATURE_ARGS     – Extra CLI args for the feature reth node
 //   BENCH_SAMPLY           – 'true' if samply profiling was enabled
 //
 // Usage from actions/github-script:
@@ -16,6 +18,7 @@
 
 const fs = require('fs');
 const path = require('path');
+const { fmtChange, fmtMs, verdict, loadSamplyUrls, blocksLabel, metricRows, waitTimeRows } = require('./bench-utils');
 
 const SLACK_API = 'https://slack.com/api/chat.postMessage';
 
@@ -59,40 +62,17 @@ function cell(text) {
   return { type: 'raw_text', text: s || ' ' };
 }
 
+// Slack shortcodes for verdict (Block Kit header doesn't support unicode emoji)
+const SLACK_VERDICT = {
+  '⚠️': ':warning:',
+  '❌': ':x:',
+  '✅': ':white_check_mark:',
+  '⚪': ':white_circle:',
+};
+
 function buildSuccessBlocks({ summary, prNumber, actor, actorSlackId, jobUrl, repo, samplyUrls }) {
-  const b = summary.baseline.stats;
-  const f = summary.feature.stats;
-  const c = summary.changes;
-
-  const sigEmoji = { good: '\u2705', bad: '\u274c', neutral: '\u26aa' };
-
-  function fmtMs(v) { return v.toFixed(2) + 'ms'; }
-  function fmtMgas(v) { return v.toFixed(2); }
-  function fmtChange(ch) {
-    if (!ch.pct && !ch.ci_pct) return ' ';
-    const pctStr = `${ch.pct >= 0 ? '+' : ''}${ch.pct.toFixed(2)}%`;
-    const ciStr = ch.ci_pct ? ` (\u00b1${ch.ci_pct.toFixed(2)}%)` : '';
-    return `${pctStr}${ciStr} ${sigEmoji[ch.sig]}`;
-  }
-
-  // Overall result for header
-  const vals = Object.values(c);
-  const hasBad = vals.some(v => v.sig === 'bad');
-  const hasGood = vals.some(v => v.sig === 'good');
-  let headerEmoji, headerResult;
-  if (hasBad && hasGood) {
-    headerEmoji = ':warning:';
-    headerResult = 'Mixed Results';
-  } else if (hasBad) {
-    headerEmoji = ':x:';
-    headerResult = 'Regression';
-  } else if (hasGood) {
-    headerEmoji = ':white_check_mark:';
-    headerResult = 'Improvement';
-  } else {
-    headerEmoji = ':white_circle:';
-    headerResult = 'No Difference';
-  }
+  const { emoji, label } = verdict(summary.changes);
+  const headerEmoji = SLACK_VERDICT[emoji] || emoji;
 
   const prUrl = prNumber ? `https://github.com/${repo}/pull/${prNumber}` : '';
   const commitUrl = `https://github.com/${repo}/commit`;
@@ -117,12 +97,15 @@ function buildSuccessBlocks({ summary, prNumber, actor, actorSlackId, jobUrl, re
   if (fl1) featureLine += ` | <${fl1}|Samply 1>`;
   if (fl2) featureLine += ` | <${fl2}|Samply 2>`;
 
-  const warmup = summary.warmup_blocks || process.env.BENCH_WARMUP_BLOCKS || '';
-  const countsLine = warmup
-    ? `*Warmup:* ${warmup} | *Blocks:* ${summary.blocks}`
-    : `*Blocks:* ${summary.blocks}`;
+  const countsLine = blocksLabel(summary).map(p => `*${p.key}:* ${p.value}`).join(' | ');
 
-  const sectionText = [metaParts.join(' | '), '', baselineLine, featureLine, countsLine].join('\n');
+  const baselineArgs = process.env.BENCH_BASELINE_ARGS || '';
+  const featureArgs = process.env.BENCH_FEATURE_ARGS || '';
+  const argsLines = [];
+  if (baselineArgs) argsLines.push(`*Baseline Args:* \`${baselineArgs}\``);
+  if (featureArgs) argsLines.push(`*Feature Args:* \`${featureArgs}\``);
+
+  const sectionText = [metaParts.join(' | '), '', baselineLine, featureLine, ...argsLines, countsLine].join('\n');
 
   // Action buttons
   const diffUrl = `https://github.com/${repo}/compare/${summary.baseline.ref}...${summary.feature.ref}`;
@@ -141,10 +124,17 @@ function buildSuccessBlocks({ summary, prNumber, actor, actorSlackId, jobUrl, re
     },
   ];
 
+  // Build table rows from shared metricRows
+  const rows = metricRows(summary);
+  const tableRows = [
+    [cell('Metric'), cell('Baseline'), cell('Feature'), cell('Change')],
+    ...rows.map(r => [cell(r.label), cell(r.baseline), cell(r.feature), cell(r.change || ' ')]),
+  ];
+
   const blocks = [
     {
       type: 'header',
-      text: { type: 'plain_text', text: `${headerEmoji} ${headerResult}`, emoji: true },
+      text: { type: 'plain_text', text: `${headerEmoji} ${label}`, emoji: true },
     },
     {
       type: 'section',
@@ -158,15 +148,7 @@ function buildSuccessBlocks({ summary, prNumber, actor, actorSlackId, jobUrl, re
         { align: 'right' },
         { align: 'right' },
       ],
-      rows: [
-        [cell('Metric'),  cell('Baseline'), cell('Feature'), cell('Change')],
-        [cell('Mean'),     cell(fmtMs(b.mean_ms)),      cell(fmtMs(f.mean_ms)),      cell(fmtChange(c.mean))],
-        [cell('StdDev'),   cell(fmtMs(b.stddev_ms)),    cell(fmtMs(f.stddev_ms)),    cell(' ')],
-        [cell('P50'),      cell(fmtMs(b.p50_ms)),       cell(fmtMs(f.p50_ms)),       cell(fmtChange(c.p50))],
-        [cell('P90'),      cell(fmtMs(b.p90_ms)),       cell(fmtMs(f.p90_ms)),       cell(fmtChange(c.p90))],
-        [cell('P99'),      cell(fmtMs(b.p99_ms)),       cell(fmtMs(f.p99_ms)),       cell(fmtChange(c.p99))],
-        [cell('Mgas/s'),   cell(fmtMgas(b.mean_mgas_s)), cell(fmtMgas(f.mean_mgas_s)), cell(fmtChange(c.mgas_s))],
-      ],
+      rows: tableRows,
     },
     {
       type: 'actions',
@@ -176,16 +158,12 @@ function buildSuccessBlocks({ summary, prNumber, actor, actorSlackId, jobUrl, re
 
   // Wait times as a separate table block (sent as threaded reply due to Slack one-table limit)
   const threadBlocks = [];
-  const waitTimes = summary.wait_times || {};
-  const waitKeys = Object.keys(waitTimes);
-  if (waitKeys.length > 0) {
-    const waitRows = [
+  const wtRows = waitTimeRows(summary);
+  if (wtRows.length > 0) {
+    const waitTableRows = [
       [cell('Wait Time'), cell('Baseline'), cell('Feature')],
+      ...wtRows.map(r => [cell(r.title), cell(r.baseline), cell(r.feature)]),
     ];
-    for (const key of waitKeys) {
-      const wt = waitTimes[key];
-      waitRows.push([cell(wt.title), cell(fmtMs(wt.baseline.mean_ms)), cell(fmtMs(wt.feature.mean_ms))]);
-    }
     threadBlocks.push({
       type: 'table',
       column_settings: [
@@ -193,7 +171,7 @@ function buildSuccessBlocks({ summary, prNumber, actor, actorSlackId, jobUrl, re
         { align: 'right' },
         { align: 'right' },
       ],
-      rows: waitRows,
+      rows: waitTableRows,
     });
   }
 
@@ -255,16 +233,7 @@ async function success({ core, context }) {
   const jobUrl = process.env.BENCH_JOB_URL ||
     `${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;
 
-  // Load samply profile URLs (files exist when samply profiling was enabled)
-  const samplyUrls = {};
-  for (const run of ['baseline-1', 'baseline-2', 'feature-1', 'feature-2']) {
-    try {
-      const url = fs.readFileSync(
-        path.join(process.env.BENCH_WORK_DIR, run, 'samply-profile-url.txt'), 'utf8'
-      ).trim();
-      if (url) samplyUrls[run] = url;
-    } catch {}
-  }
+  const samplyUrls = loadSamplyUrls(process.env.BENCH_WORK_DIR);
 
   const slackUsers = loadSlackUsers(process.env.GITHUB_WORKSPACE || '.');
   const actorSlackId = slackUsers[actor];

.github/scripts/bench-slack-users.json

@@ -9,5 +9,17 @@
   "gakonst": "U092SEPDM40",
   "Rjected": "U09F6SCKRGT",
   "DaniPopes": "U09FAT8EK2A",
-  "emmajam": "U0A34UN92HW"
+  "emmajam": "U0A34UN92HW",
+  "onbjerg": "U09FB0UK5AA",
+  "fgimenez": "U09G3GP7CSU",
+  "rakita": "U09FB3Z2M7Y",
+  "jxom": "U09F72MG083",
+  "tmm": "U0AD0U8E88N",
+  "pepyakin": "U0A7HKMGEHJ",
+  "grandizzy": "U09F8DBDDRT",
+  "SuperFluffy": "U095BKHB2Q4",
+  "kamsz": "U0A2563UBRD",
+  "zerosnacks": "U09FARPMN74",
+  "samczsun": "U096R14E4H3",
+  "laibe": "U09FARE0B9Q"
 }

.github/scripts/bench-utils.js

@@ -0,0 +1,95 @@
+// Shared utilities for reth-bench result rendering.
+//
+// Used by bench-job-summary.js and bench-slack-notify.js.
+
+const fs = require('fs');
+const path = require('path');
+
+const SIG_EMOJI = { good: '✅', bad: '❌', neutral: '⚪' };
+
+function fmtMs(v) { return v.toFixed(2) + 'ms'; }
+function fmtMgas(v) { return v.toFixed(2); }
+function fmtS(v) { return v.toFixed(2) + 's'; }
+
+function fmtChange(ch) {
+  if (!ch || (!ch.pct && !ch.ci_pct)) return '';
+  const pctStr = `${ch.pct >= 0 ? '+' : ''}${ch.pct.toFixed(2)}%`;
+  const ciStr = ch.ci_pct ? ` (±${ch.ci_pct.toFixed(2)}%)` : '';
+  return `${pctStr}${ciStr} ${SIG_EMOJI[ch.sig]}`;
+}
+
+function verdict(changes) {
+  const vals = Object.values(changes);
+  const hasBad = vals.some(v => v.sig === 'bad');
+  const hasGood = vals.some(v => v.sig === 'good');
+  if (hasBad && hasGood) return { emoji: '⚠️', label: 'Mixed Results' };
+  if (hasBad) return { emoji: '❌', label: 'Regression' };
+  if (hasGood) return { emoji: '✅', label: 'Improvement' };
+  return { emoji: '⚪', label: 'No Difference' };
+}
+
+function loadSamplyUrls(workDir) {
+  const urls = {};
+  for (const run of ['baseline-1', 'baseline-2', 'feature-1', 'feature-2']) {
+    try {
+      const url = fs.readFileSync(path.join(workDir, run, 'samply-profile-url.txt'), 'utf8').trim();
+      if (url) urls[run] = url;
+    } catch {}
+  }
+  return urls;
+}
+
+function blocksLabel(summary) {
+  const parts = [];
+  if (summary.big_blocks) {
+    parts.push({ key: 'Big Blocks', value: summary.blocks });
+  } else {
+    const warmup = summary.warmup_blocks || process.env.BENCH_WARMUP_BLOCKS || '';
+    if (warmup) parts.push({ key: 'Warmup', value: warmup });
+    parts.push({ key: 'Blocks', value: summary.blocks });
+  }
+  const cores = process.env.BENCH_CORES || '0';
+  if (cores !== '0') parts.push({ key: 'Cores', value: cores });
+  return parts;
+}
+
+// The 7 metric rows shared by all renderers.
+// Returns an array of { label, baseline, feature, change } objects.
+function metricRows(summary) {
+  const b = summary.baseline.stats;
+  const f = summary.feature.stats;
+  const c = summary.changes;
+  return [
+    { label: 'Mean',       baseline: fmtMs(b.mean_ms),       feature: fmtMs(f.mean_ms),       change: fmtChange(c.mean) },
+    { label: 'StdDev',     baseline: fmtMs(b.stddev_ms),     feature: fmtMs(f.stddev_ms),     change: '' },
+    { label: 'P50',        baseline: fmtMs(b.p50_ms),        feature: fmtMs(f.p50_ms),        change: fmtChange(c.p50) },
+    { label: 'P90',        baseline: fmtMs(b.p90_ms),        feature: fmtMs(f.p90_ms),        change: fmtChange(c.p90) },
+    { label: 'P99',        baseline: fmtMs(b.p99_ms),        feature: fmtMs(f.p99_ms),        change: fmtChange(c.p99) },
+    { label: 'Mgas/s',     baseline: fmtMgas(b.mean_mgas_s), feature: fmtMgas(f.mean_mgas_s), change: fmtChange(c.mgas_s) },
+    { label: 'Wall Clock', baseline: fmtS(b.wall_clock_s),   feature: fmtS(f.wall_clock_s),   change: fmtChange(c.wall_clock) },
+  ];
+}
+
+// Wait time rows: one row per metric showing mean values.
+function waitTimeRows(summary) {
+  const waitTimes = summary.wait_times || {};
+  const rows = [];
+  for (const key of Object.keys(waitTimes)) {
+    const wt = waitTimes[key];
+    rows.push({ title: wt.title, baseline: fmtMs(wt.baseline.mean_ms), feature: fmtMs(wt.feature.mean_ms) });
+  }
+  return rows;
+}
+
+module.exports = {
+  SIG_EMOJI,
+  fmtMs,
+  fmtMgas,
+  fmtS,
+  fmtChange,
+  verdict,
+  loadSamplyUrls,
+  blocksLabel,
+  metricRows,
+  waitTimeRows,
+};

.github/scripts/build_pgo_bolt.sh

@@ -0,0 +1,414 @@
+#!/usr/bin/env bash
+#
+# Full PGO+BOLT optimized build for reth using real reth-bench workloads.
+#
+# Phases:
+#   1. Build PGO-instrumented reth, run reth-bench → collect PGO profiles
+#   2. Build BOLT-instrumented reth (with PGO), run reth-bench → collect BOLT profiles
+#   3. Build final PGO+BOLT optimized binary
+#
+# Required environment variables:
+#   DATADIR    - Path to reth datadir (must already contain chain data)
+#   RPC_URL    - Source RPC URL for reth-bench to fetch payloads from
+#
+# Optional environment variables:
+#   PGO_BLOCKS      - Number of blocks for PGO profiling (default: 20)
+#   BOLT_BLOCKS     - Number of blocks for BOLT profiling (default: 20)
+#   SKIP_BOLT       - Temporarily skip BOLT phases (default: false)
+#   STRIP_SYMBOLS   - Strip debug symbols from output binary (default: true)
+#   COLLECT_PGO_ONLY - Stop after producing merged.profdata (default: false)
+#   PGO_PROFDATA    - Path to pre-collected merged.profdata (optional)
+#   PROFILE         - Cargo profile (default: maxperf-symbols)
+#   FEATURES        - Cargo features (default: jemalloc,asm-keccak,min-debug-logs)
+#   TARGET          - Target triple (default: auto-detected)
+#   EXTRA_RUSTFLAGS - Additional RUSTFLAGS (e.g. -C target-cpu=x86-64-v3)
+#
+# Output:
+#   target/$PROFILE_DIR/reth  — final optimized binary
+set -euo pipefail
+
+gha_section_start() {
+    local title="$1"
+    if [ -n "${GITHUB_ACTIONS:-}" ]; then
+        echo "::group::$title"
+    else
+        echo ""
+        echo "=== $title ==="
+    fi
+}
+
+gha_section_end() {
+    if [ -n "${GITHUB_ACTIONS:-}" ]; then
+        echo "::endgroup::"
+    fi
+}
+
+cd "$(dirname "$0")/../.."
+
+# ── Configuration ──────────────────────────────────────────────────────────────
+PGO_BLOCKS="${PGO_BLOCKS:-20}"
+BOLT_BLOCKS="${BOLT_BLOCKS:-20}"
+SKIP_BOLT="${SKIP_BOLT:-false}"
+STRIP_SYMBOLS="${STRIP_SYMBOLS:-true}"
+COLLECT_PGO_ONLY="${COLLECT_PGO_ONLY:-false}"
+PROFILE="${PROFILE:-maxperf-symbols}"
+FEATURES="${FEATURES:-jemalloc,asm-keccak,min-debug-logs}"
+TARGET="${TARGET:-$(rustc -Vv | grep host | cut -d' ' -f2)}"
+BASE_RUSTFLAGS="${RUSTFLAGS:-}"
+EXTRA_RUSTFLAGS="${EXTRA_RUSTFLAGS:-}"
+COMBINED_RUSTFLAGS="$BASE_RUSTFLAGS $EXTRA_RUSTFLAGS"
+PGO_PROFDATA="${PGO_PROFDATA:-}"
+DATADIR="${DATADIR:-}"
+RPC_URL="${RPC_URL:-}"
+
+SKIP_BOLT_BOOL=false
+if [[ "${SKIP_BOLT,,}" == "true" || "$SKIP_BOLT" == "1" ]]; then
+    SKIP_BOLT_BOOL=true
+fi
+
+STRIP_SYMBOLS_BOOL=false
+if [[ "${STRIP_SYMBOLS,,}" == "true" || "$STRIP_SYMBOLS" == "1" ]]; then
+    STRIP_SYMBOLS_BOOL=true
+fi
+
+COLLECT_PGO_ONLY_BOOL=false
+if [[ "${COLLECT_PGO_ONLY,,}" == "true" || "$COLLECT_PGO_ONLY" == "1" ]]; then
+    COLLECT_PGO_ONLY_BOOL=true
+fi
+
+USE_PRECOLLECTED_PGO=false
+if [ -n "$PGO_PROFDATA" ]; then
+    if [ ! -f "$PGO_PROFDATA" ]; then
+        echo "error: PGO_PROFDATA points to a missing file: $PGO_PROFDATA"
+        exit 1
+    fi
+    USE_PRECOLLECTED_PGO=true
+fi
+
+NEEDS_BENCH_WORKLOAD=true
+if [ "$USE_PRECOLLECTED_PGO" = true ] && [ "$SKIP_BOLT_BOOL" = true ]; then
+    NEEDS_BENCH_WORKLOAD=false
+fi
+
+if [ "$NEEDS_BENCH_WORKLOAD" = true ]; then
+    : "${DATADIR:?DATADIR must be set to the reth data directory}"
+    : "${RPC_URL:?RPC_URL must be set}"
+fi
+
+if [[ "$PROFILE" == dev ]]; then
+    PROFILE_DIR=debug
+else
+    PROFILE_DIR=$PROFILE
+fi
+
+MANIFEST_PATH="bin/reth"
+
+LLVM_VERSION=$(rustc -Vv | grep -oP 'LLVM version: \K\d+')
+PGO_DIR="$PWD/target/pgo-profiles"
+BOLT_DIR="$PWD/target/bolt-profiles"
+CARGO_ARGS=(--profile "$PROFILE" --features "$FEATURES" --manifest-path "$MANIFEST_PATH/Cargo.toml" --bin "reth" --locked)
+
+# Enable debug symbols for BOLT (requires symbols to reorder code).
+# Strip them at the end.
+PROFILE_UPPER=$(echo "$PROFILE" | tr '[:lower:]-' '[:upper:]_')
+export "CARGO_PROFILE_${PROFILE_UPPER}_STRIP=debuginfo"
+
+gha_section_start "Full PGO+BOLT Build"
+echo "Binary:      reth"
+echo "Manifest:    $MANIFEST_PATH"
+echo "Target:      $TARGET"
+echo "Profile:     $PROFILE"
+echo "Features:    $FEATURES"
+echo "LLVM:        $LLVM_VERSION"
+echo "PGO blocks:  $PGO_BLOCKS"
+echo "BOLT blocks: $BOLT_BLOCKS"
+echo "Skip BOLT:   $SKIP_BOLT"
+echo "Strip symbols: $STRIP_SYMBOLS"
+echo "Collect only: $COLLECT_PGO_ONLY"
+echo "PGO profdata: ${PGO_PROFDATA:-<collect with reth-bench>}"
+echo "RUSTFLAGS:   ${BASE_RUSTFLAGS:-<unset>}"
+echo "EXTRA_RUSTFLAGS: ${EXTRA_RUSTFLAGS:-<unset>}"
+if [ "$NEEDS_BENCH_WORKLOAD" = true ]; then
+    echo "Datadir:     $DATADIR"
+    echo "RPC URL:     $RPC_URL"
+else
+    echo "Datadir:     <not required>"
+    echo "RPC URL:     <not required>"
+fi
+gha_section_end
+
+# ── Prerequisites ──────────────────────────────────────────────────────────────
+gha_section_start "Installing prerequisites"
+rustup component add llvm-tools-preview
+
+LLVM_PROFDATA=$(find "$(rustc --print sysroot)" -name llvm-profdata -type f | head -1)
+if [ -z "$LLVM_PROFDATA" ]; then
+    echo "error: llvm-profdata not found"
+    exit 1
+fi
+
+install_bolt() {
+    if command -v llvm-bolt &>/dev/null; then
+        echo "BOLT already installed"
+        return
+    fi
+    echo "Installing BOLT from apt.llvm.org..."
+    wget -qO- https://apt.llvm.org/llvm-snapshot.gpg.key | sudo tee /etc/apt/trusted.gpg.d/apt.llvm.org.asc >/dev/null
+    CODENAME=$(lsb_release -cs)
+    echo "deb http://apt.llvm.org/$CODENAME/ llvm-toolchain-$CODENAME-$LLVM_VERSION main" | sudo tee /etc/apt/sources.list.d/llvm.list >/dev/null
+    sudo apt-get update -qq
+    sudo apt-get install -y -qq "bolt-$LLVM_VERSION"
+    sudo ln -sf "/usr/bin/llvm-bolt-$LLVM_VERSION" /usr/local/bin/llvm-bolt
+    sudo ln -sf "/usr/bin/merge-fdata-$LLVM_VERSION" /usr/local/bin/merge-fdata
+}
+if [ "$SKIP_BOLT_BOOL" = true ]; then
+    echo "Skipping BOLT installation (SKIP_BOLT=$SKIP_BOLT)"
+else
+    install_bolt
+fi
+gha_section_end
+
+if [ "$NEEDS_BENCH_WORKLOAD" = true ]; then
+    # Build reth-bench once (non-instrumented) — reused for both phases.
+    gha_section_start "Building reth-bench"
+    RUSTFLAGS="$COMBINED_RUSTFLAGS" \
+        cargo build --profile "$PROFILE" --features "$FEATURES" \
+        --manifest-path bin/reth-bench/Cargo.toml --bin reth-bench --locked
+    RETH_BENCH_BIN="$(find target -name reth-bench -type f -executable | head -1)"
+    echo "reth-bench: $RETH_BENCH_BIN"
+    gha_section_end
+else
+    gha_section_start "Building reth-bench"
+    echo "Skipping reth-bench build (pre-collected PGO with SKIP_BOLT=true)"
+    gha_section_end
+fi
+
+# ── Helpers ────────────────────────────────────────────────────────────────────
+RETH_PID=
+cleanup() {
+    if [ -n "${RETH_PID:-}" ] && kill -0 "$RETH_PID" 2>/dev/null; then
+        echo "Stopping reth (pid $RETH_PID)..."
+        sudo kill "$RETH_PID" 2>/dev/null || true
+        for i in $(seq 1 60); do
+            sudo kill -0 "$RETH_PID" 2>/dev/null || break
+            if [ $((i % 10)) -eq 0 ]; then
+                echo "  waiting... (${i}s)"
+            fi
+            sleep 1
+        done
+        sudo kill -9 "$RETH_PID" 2>/dev/null || true
+    fi
+}
+trap cleanup EXIT
+
+# Start reth, wait for RPC, run reth-bench, then stop reth.
+# Arguments: $1 = reth binary path, $2 = number of blocks, $3 = log label
+run_bench_workload() {
+    local reth_bin="$1" blocks="$2" label="$3"
+    local http_port=8545 authrpc_port=8551
+
+    echo "--- Starting reth ($label) ---"
+    sudo "$reth_bin" node \
+        --datadir "$DATADIR" \
+        --log.file.directory "/tmp/reth-${label}-logs" \
+        --engine.accept-execution-requests-hash \
+        --http --http.port "$http_port" \
+        --authrpc.port "$authrpc_port" \
+        --disable-discovery --no-persist-peers \
+        > "/tmp/reth-${label}.log" 2>&1 &
+    RETH_PID=$!
+
+    echo "Waiting for reth RPC..."
+    for i in $(seq 1 120); do
+        if curl -sf "http://127.0.0.1:$http_port" -X POST \
+            -H 'Content-Type: application/json' \
+            -d '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}' \
+            > /dev/null 2>&1; then
+            echo "reth is ready after ${i}s"
+            break
+        fi
+        if [ "$i" -eq 120 ]; then
+            echo "error: reth failed to start within 120s"
+            cat "/tmp/reth-${label}.log"
+            exit 1
+        fi
+        sleep 1
+    done
+
+    echo "Running reth-bench ($blocks blocks)..."
+    "$RETH_BENCH_BIN" new-payload-fcu \
+        --rpc-url "$RPC_URL" \
+        --engine-rpc-url "http://127.0.0.1:$authrpc_port" \
+        --jwt-secret "$DATADIR/jwt.hex" \
+        --advance "$blocks" \
+        --reth-new-payload 2>&1 | sed -u "s/^/[$label] /"
+
+    echo "Stopping reth ($label)..."
+    sudo kill "$RETH_PID" 2>/dev/null || true
+    for i in $(seq 1 60); do
+        sudo kill -0 "$RETH_PID" 2>/dev/null || break
+        sleep 1
+    done
+    sudo kill -9 "$RETH_PID" 2>/dev/null || true
+    RETH_PID=
+}
+
+publish_binary() {
+    local source_bin="$1"
+    for out in "target/$TARGET/$PROFILE_DIR" "target/$PROFILE_DIR"; do
+        local destination="$out/reth"
+        mkdir -p "$out"
+        # Skip copying when source and destination resolve to the same inode.
+        if [ -e "$destination" ] && [ "$source_bin" -ef "$destination" ]; then
+            continue
+        fi
+        cp "$source_bin" "$destination"
+    done
+}
+
+if [ "$USE_PRECOLLECTED_PGO" = true ]; then
+    gha_section_start "Phase 1: Using Pre-Collected PGO Profile"
+    rm -rf "$PGO_DIR"
+    mkdir -p "$PGO_DIR"
+    cp "$PGO_PROFDATA" "$PGO_DIR/merged.profdata"
+    echo "Using pre-collected profile: $PGO_PROFDATA"
+    echo "PGO profile: $PGO_DIR/merged.profdata ($(ls -lh "$PGO_DIR/merged.profdata" | awk '{print $5}'))"
+    gha_section_end
+else
+    # ── Phase 1: PGO profile collection ───────────────────────────────────────
+    gha_section_start "Phase 1: PGO Profile Collection"
+
+    rm -rf "$PGO_DIR"
+    mkdir -p "$PGO_DIR"
+
+    echo "Building PGO-instrumented binary..."
+    RUSTFLAGS="-Cprofile-generate=$PGO_DIR -Crelocation-model=pic $COMBINED_RUSTFLAGS" \
+        cargo build "${CARGO_ARGS[@]}" --target "$TARGET"
+
+    PGO_RETH_BIN="$PWD/target/$TARGET/$PROFILE_DIR/reth"
+    echo "Instrumented binary: $PGO_RETH_BIN ($(ls -lh "$PGO_RETH_BIN" | awk '{print $5}'))"
+
+    run_bench_workload "$PGO_RETH_BIN" "$PGO_BLOCKS" "pgo"
+
+    # Fix ownership if reth ran as root.
+    sudo chown -R "$(id -un):$(id -gn)" "$PGO_DIR" 2>/dev/null || true
+
+    # Merge PGO profiles.
+    echo "Merging PGO profiles..."
+    PROFRAW_COUNT=$(find "$PGO_DIR" -name '*.profraw' | wc -l)
+    echo "Found $PROFRAW_COUNT .profraw files"
+    if [ "$PROFRAW_COUNT" -eq 0 ]; then
+        echo "error: no .profraw files — instrumented binary did not produce profiles"
+        exit 1
+    fi
+    "$LLVM_PROFDATA" merge -o "$PGO_DIR/merged.profdata" "$PGO_DIR"/*.profraw
+    echo "PGO profile: $PGO_DIR/merged.profdata ($(ls -lh "$PGO_DIR/merged.profdata" | awk '{print $5}'))"
+    gha_section_end
+fi
+
+if [ "$COLLECT_PGO_ONLY_BOOL" = true ]; then
+    gha_section_start "PGO Collection Complete"
+    echo "COLLECT_PGO_ONLY=true, skipping PGO/BOLT optimized binary build"
+    echo "Profile: $PGO_DIR/merged.profdata"
+    gha_section_end
+    exit 0
+fi
+
+if [ "$SKIP_BOLT_BOOL" = true ]; then
+    gha_section_start "BOLT Phase Skipped"
+    echo "SKIP_BOLT=$SKIP_BOLT, building PGO-only binary"
+    echo "Building PGO-optimized binary..."
+    RUSTFLAGS="-Cprofile-use=$PGO_DIR/merged.profdata $COMBINED_RUSTFLAGS" \
+        cargo build "${CARGO_ARGS[@]}" --target "$TARGET"
+
+    BUILT_BIN="$PWD/target/$TARGET/$PROFILE_DIR/reth"
+    if [ "$STRIP_SYMBOLS_BOOL" = true ]; then
+        echo "Stripping debug symbols..."
+        strip "$BUILT_BIN"
+    else
+        echo "Skipping strip (STRIP_SYMBOLS=$STRIP_SYMBOLS)"
+    fi
+    publish_binary "$BUILT_BIN"
+    gha_section_end
+else
+    # ── Phase 2: BOLT profile collection (with PGO) ──────────────────────────
+    gha_section_start "Phase 2: BOLT Profile Collection (with PGO)"
+
+    rm -rf "$BOLT_DIR"
+    mkdir -p "$BOLT_DIR"
+
+    echo "Building BOLT-instrumented binary with PGO..."
+    # --emit-relocs preserves relocation entries in the binary, required by llvm-bolt -instrument
+    RUSTFLAGS="-Cprofile-use=$PGO_DIR/merged.profdata -Clink-arg=-Wl,--emit-relocs $COMBINED_RUSTFLAGS" \
+        cargo build "${CARGO_ARGS[@]}" --target "$TARGET"
+
+    # Instrument with BOLT
+    BUILT_BIN="$PWD/target/$TARGET/$PROFILE_DIR/reth"
+    BOLT_INSTRUMENTED_BIN="$BUILT_BIN-bolt-instrumented"
+
+    echo "Instrumenting binary with BOLT..."
+    # --skip-funcs: skip compiler-generated drop_in_place functions that BOLT can't handle
+    # as split functions in relocation mode (triggered by --emit-relocs)
+    llvm-bolt "$BUILT_BIN" \
+        -instrument \
+        --instrumentation-file-append-pid \
+        --instrumentation-file="$BOLT_DIR/prof" \
+        --skip-funcs='.*drop_in_place.*' \
+        -o "$BOLT_INSTRUMENTED_BIN"
+    echo "BOLT-instrumented binary: $BOLT_INSTRUMENTED_BIN ($(ls -lh "$BOLT_INSTRUMENTED_BIN" | awk '{print $5}'))"
+
+    run_bench_workload "$BOLT_INSTRUMENTED_BIN" "$BOLT_BLOCKS" "bolt"
+
+    # Fix ownership for BOLT profiles
+    sudo chown -R "$(id -un):$(id -gn)" "$BOLT_DIR" 2>/dev/null || true
+
+    # Merge BOLT profiles
+    echo "Merging BOLT profiles..."
+    FDATA_COUNT=$(find "$BOLT_DIR" -name '*.fdata' | wc -l)
+    echo "Found $FDATA_COUNT .fdata files"
+    if [ "$FDATA_COUNT" -eq 0 ]; then
+        echo "error: no .fdata files — BOLT-instrumented binary did not produce profiles"
+        exit 1
+    fi
+    merge-fdata "$BOLT_DIR"/*.fdata > "$BOLT_DIR/merged.fdata"
+    echo "BOLT profile: $BOLT_DIR/merged.fdata ($(ls -lh "$BOLT_DIR/merged.fdata" | awk '{print $5}'))"
+    gha_section_end
+
+    # ── Phase 3: Final optimized build ───────────────────────────────────────
+    gha_section_start "Phase 3: Final PGO+BOLT Optimized Build"
+
+    echo "Building PGO-optimized binary..."
+    # --emit-relocs preserves relocation entries in the binary, required by llvm-bolt for code reordering
+    RUSTFLAGS="-Cprofile-use=$PGO_DIR/merged.profdata -Clink-arg=-Wl,--emit-relocs $COMBINED_RUSTFLAGS" \
+        cargo build "${CARGO_ARGS[@]}" --target "$TARGET"
+
+    BUILT_BIN="$PWD/target/$TARGET/$PROFILE_DIR/reth"
+    OPTIMIZED_BIN="$BUILT_BIN-bolt-optimized"
+
+    echo "Optimizing with BOLT..."
+    llvm-bolt "$BUILT_BIN" \
+        -o "$OPTIMIZED_BIN" \
+        --data "$BOLT_DIR/merged.fdata" \
+        -reorder-blocks=ext-tsp \
+        -reorder-functions=cdsort \
+        -split-functions \
+        -split-all-cold \
+        -dyno-stats \
+        -icf=1 \
+        -use-gnu-stack \
+        --skip-funcs='.*drop_in_place.*'
+
+    if [ "$STRIP_SYMBOLS_BOOL" = true ]; then
+        echo "Stripping debug symbols..."
+        strip "$OPTIMIZED_BIN"
+    else
+        echo "Skipping strip (STRIP_SYMBOLS=$STRIP_SYMBOLS)"
+    fi
+    publish_binary "$OPTIMIZED_BIN"
+    gha_section_end
+fi
+
+gha_section_start "Build Complete"
+ls -lh "target/$PROFILE_DIR/reth"
+echo "Output: target/$PROFILE_DIR/reth"
+gha_section_end

.github/scripts/check_rv32imac.sh

@@ -2,9 +2,6 @@
 set -uo pipefail
 
 crates_to_check=(
-    reth-codecs-derive
-    reth-primitives
-    reth-primitives-traits
     reth-network-peers
     reth-trie-common
     reth-trie-sparse

.github/scripts/check_wasm.sh

@@ -22,6 +22,7 @@ exclude_crates=(
   reth-downloaders
   reth-e2e-test-utils
   reth-engine-service
+  reth-execution-cache
   reth-engine-tree
   reth-engine-util
   reth-eth-wire
@@ -55,6 +56,7 @@ exclude_crates=(
   reth-ress-provider
   # The following are not supposed to be working
   reth # all of the crates below
+  reth-bb # binary-only, uses tokio features unsupported on wasm
   reth-storage-rpc-provider
   reth-invalid-block-hooks # reth-provider
   reth-libmdbx # mdbx

.github/scripts/hive/Dockerfile

@@ -7,7 +7,7 @@ FROM lukemathwalker/cargo-chef:latest-rust-1 AS chef
 WORKDIR /app
 
 # Install system dependencies
-RUN apt-get update && apt-get -y upgrade && apt-get install -y libclang-dev pkg-config
+RUN apt-get update && apt-get install -y libclang-dev pkg-config
 
 # 
 # We prepare the build plan

.github/scripts/hive/build_simulators.sh

@@ -11,8 +11,14 @@ go build .
 
 # Run each hive command in the background for each simulator and wait
 echo "Building images"
-# TODO: test code has been moved from https://github.com/ethereum/execution-spec-tests to https://github.com/ethereum/execution-specs  we need to pin eels branch with `--sim.buildarg branch=<release-branch-name>` once we have the fusaka release tagged on the new repo
-./hive -client reth --sim "ethereum/eels" --sim.buildarg fixtures=https://github.com/ethereum/execution-spec-tests/releases/download/v5.3.0/fixtures_develop.tar.gz -sim.timelimit 1s || true &
+./hive -client reth --sim "ethereum/eels/consume-engine" \
+    --sim.buildarg fixtures=https://github.com/ethereum/execution-spec-tests/releases/download/v5.3.0/fixtures_develop.tar.gz \
+    --sim.buildarg branch=forks/osaka \
+    --sim.timelimit 1s || true &
+./hive -client reth --sim "ethereum/eels/consume-rlp" \
+    --sim.buildarg fixtures=https://github.com/ethereum/execution-spec-tests/releases/download/v5.3.0/fixtures_develop.tar.gz \
+    --sim.buildarg branch=forks/osaka \
+    --sim.timelimit 1s || true &
 ./hive -client reth --sim "ethereum/engine" -sim.timelimit 1s || true &
 ./hive -client reth --sim "devp2p" -sim.timelimit 1s || true &
 ./hive -client reth --sim "ethereum/rpc-compat" -sim.timelimit 1s || true &

.github/scripts/hive/expected_failures.yaml

@@ -1,9 +1,6 @@
 # tracked by https://github.com/paradigmxyz/reth/issues/13879
 rpc-compat:
   - debug_getRawBlock/get-invalid-number (reth)
-  - debug_getRawHeader/get-invalid-number (reth)
-  - debug_getRawReceipts/get-invalid-number (reth)
-  - debug_getRawReceipts/get-block-n (reth)
   - debug_getRawTransaction/get-invalid-hash (reth)
 
   - eth_getStorageAt/get-storage-invalid-key-too-large (reth)
@@ -16,28 +13,11 @@ rpc-compat:
   # syncing mode, the test expects syncing to be false on start
   - eth_syncing/check-syncing (reth)
 
-# no fix due to https://github.com/paradigmxyz/reth/issues/8732
-engine-withdrawals:
-  - Withdrawals Fork On Genesis (Paris) (reth)
-  - Withdrawals Fork on Block 1 (Paris) (reth)
-  - Withdrawals Fork on Block 2 (Paris) (reth)
-  - Withdrawals Fork on Block 3 (Paris) (reth)
-  - Withdraw to a single account (Paris) (reth)
-  - Withdraw to two accounts (Paris) (reth)
-  - Withdraw many accounts (Paris) (reth)
-  - Withdraw zero amount (Paris) (reth)
-  - Empty Withdrawals (Paris) (reth)
-  - Corrupted Block Hash Payload (INVALID) (Paris) (reth)
-  - Withdrawals Fork on Canonical Block 8 / Side Block 7 - 10 Block Re-Org (Paris) (reth)
+engine-withdrawals: [ ]
 
 engine-api: [ ]
 
-# no fix due to https://github.com/paradigmxyz/reth/issues/8732
-engine-cancun:
-  - Invalid PayloadAttributes, Missing BeaconRoot, Syncing=True (Cancun) (reth)
-  # the test fails with older versions of the code for which it passed before, probably related to changes
-  # in hive or its dependencies
-  - Blob Transaction Ordering, Multiple Clients (Cancun) (reth)
+engine-cancun: [ ]
 
 sync: [ ]
 
@@ -49,7 +29,7 @@ engine-auth: [ ]
 #   The test artificially creates an empty account with storage, then tests EIP-7610's behavior.
 #   On mainnet, ~25 such accounts exist as contract addresses (derived from keccak(prefix, caller,
 #   nonce/salt), not from public keys). No private key exists for contract addresses. To trigger
-#   this with EIP-7702, you'd need to recover a private key from one of the already deployed contract addresses - mathematically impossible. 
+#   this with EIP-7702, you'd need to recover a private key from one of the already deployed contract addresses - mathematically impossible.
 #
 # tests/paris/eip7610_create_collision/test_initcollision.py::test_init_collision_*
 #   Requires hash collision on create2 address to target already deployed accounts with storage.

.github/scripts/hive/ignored_tests.yaml

@@ -11,17 +11,6 @@
 #
 # When a test should no longer be ignored, remove it from this list.
 
-# flaky
-engine-withdrawals:
-  - Withdrawals Fork on Block 1 - 8 Block Re-Org NewPayload (Paris) (reth)
-  - Withdrawals Fork on Block 8 - 10 Block Re-Org NewPayload (Paris) (reth)
-  - Withdrawals Fork on Canonical Block 8 / Side Block 7 - 10 Block Re-Org (Paris) (reth)
-  # P2P sync timing issue in hive Docker environment: secondary client returns SYNCING but
-  # peer discovery/connection doesn't complete within the timeout when running with
-  # --sim.parallelism 16. Not a correctness bug, purely a CI timing issue.
-  - Sync after 2 blocks - Withdrawals on Block 2 - Multiple Withdrawal Accounts (Paris) (reth)
-  - Sync after 2 blocks - Withdrawals on Block 2 - Multiple Withdrawal Accounts - No Transactions (Paris) (reth)
-  - Sync after 128 blocks - Withdrawals on Block 2 - Multiple Withdrawal Accounts (Paris) (reth)
 engine-cancun:
   - Transaction Re-Org, New Payload on Revert Back (Cancun) (reth)
   - Transaction Re-Org, Re-Org to Different Block (Cancun) (reth)

.github/scripts/hive/run_simulator.sh

@@ -13,7 +13,13 @@ if [[ "${sim}" == *"eels"* ]]; then
 fi
 
 run_hive() {
-    hive --sim "${sim}" --sim.limit "${limit}" --sim.parallelism "${parallelism}" --client reth 2>&1 | tee /tmp/log || true
+    hive \
+  --sim "${sim}" \
+  --sim.limit "${limit}" \
+  --sim.limit.exact=false \
+  --sim.parallelism "${parallelism}" \
+  --client reth \
+  2>&1 | tee /tmp/log || true
 }
 
 check_log() {

.github/workflows/bench-scheduled.yml

@@ -0,0 +1,966 @@
+# Scheduled regression benchmarks (nightly + hourly).
+#
+# Two modes:
+#   nightly — Compares the previous nightly Docker build against the current one.
+#             Runs daily after docker.yml produces a new nightly image.
+#   hourly  — Compares main HEAD against the last benchmarked commit to catch
+#             regressions quickly. Falls back to HEAD~1 on first run.
+#             Skips if no new commits or if a previous run is still in progress.
+#
+# State is persisted between runs via GitHub Actions cache: each successful
+# run saves the feature commit SHA so the next run knows what to compare against.
+
+on:
+  schedule:
+    # Nightly: compares previous vs current nightly Docker build
+    - cron: "30 5 * * *"
+    # Hourly: compares main HEAD vs last benchmarked commit, skips if no new commits
+    - cron: "0 * * * *"
+  workflow_dispatch:
+    inputs:
+      force:
+        description: "Force run even if no new commit (bypass skip logic)"
+        required: false
+        default: false
+        type: boolean
+      no_slack:
+        description: "Suppress Slack notifications"
+        required: false
+        default: true
+        type: boolean
+      mode:
+        description: "Benchmark mode"
+        required: false
+        default: "nightly"
+        type: choice
+        options:
+          - nightly
+          - hourly
+
+env:
+  CARGO_TERM_COLOR: always
+  RUSTC_WRAPPER: "sccache"
+
+name: bench-scheduled
+
+permissions:
+  contents: read
+  actions: read
+
+jobs:
+  # ---------------------------------------------------------------------------
+  # Job 1: Resolve refs, check staleness, manage state
+  # ---------------------------------------------------------------------------
+  resolve-refs:
+    name: resolve-refs
+    runs-on: ubuntu-latest
+    outputs:
+      mode: ${{ steps.mode.outputs.mode }}
+      baseline-ref: ${{ steps.refs.outputs.baseline-ref }}
+      feature-ref: ${{ steps.refs.outputs.feature-ref }}
+      should-skip: ${{ steps.refs.outputs.should-skip }}
+      is-stale: ${{ steps.refs.outputs.is-stale }}
+      stale-age-hours: ${{ steps.refs.outputs.stale-age-hours }}
+      nightly-created: ${{ steps.refs.outputs.nightly-created }}
+      long-running: ${{ steps.refs.outputs.long-running }}
+    steps:
+      - uses: actions/checkout@v6
+        with:
+          sparse-checkout: .github/scripts
+          sparse-checkout-cone-mode: true
+          fetch-depth: 2
+
+      - name: Detect mode
+        id: mode
+        run: |
+          # Maps cron schedules to modes (must match the schedule entries above)
+          if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
+            MODE="${{ inputs.mode || 'nightly' }}"
+          elif [ "${{ github.event.schedule }}" = "30 5 * * *" ]; then
+            MODE="nightly"
+          else
+            MODE="hourly"
+          fi
+          echo "mode=$MODE" >> "$GITHUB_OUTPUT"
+          echo "Detected mode: $MODE"
+
+      - name: Restore state cache
+        id: state-cache
+        uses: actions/cache/restore@v4
+        with:
+          path: .${{ steps.mode.outputs.mode == 'hourly' && 'hourly' || 'nightly' }}-state
+          key: bench-${{ steps.mode.outputs.mode }}-state-dummy
+          restore-keys: |
+            bench-${{ steps.mode.outputs.mode }}-state-
+
+      - name: Resolve refs
+        id: refs
+        env:
+          GH_TOKEN: ${{ github.token }}
+          GITHUB_REPOSITORY: ${{ github.repository }}
+          GITHUB_RUN_ID: ${{ github.run_id }}
+        run: |
+          FORCE="${{ inputs.force || 'false' }}"
+          MODE="${{ steps.mode.outputs.mode }}"
+          .github/scripts/bench-scheduled-refs.sh "$FORCE" "$MODE"
+
+      - name: Alert on long-running hourly
+        if: steps.mode.outputs.mode == 'hourly' && steps.refs.outputs.long-running == 'true'
+        uses: actions/github-script@v8
+        env:
+          SLACK_BENCH_BOT_TOKEN: ${{ secrets.SLACK_BENCH_BOT_TOKEN }}
+          SLACK_BENCH_CHANNEL: ${{ secrets.SLACK_BENCH_CHANNEL }}
+        with:
+          script: |
+            const token = process.env.SLACK_BENCH_BOT_TOKEN;
+            const channel = process.env.SLACK_BENCH_CHANNEL;
+            if (!token || !channel) return;
+
+            const repo = '${{ github.repository }}';
+            const runUrl = `${context.serverUrl}/${repo}/actions/runs/${context.runId}`;
+            const blocks = [
+              {
+                type: 'header',
+                text: { type: 'plain_text', text: ':warning: Hourly Bench: previous run still in progress', emoji: true },
+              },
+              {
+                type: 'section',
+                text: {
+                  type: 'mrkdwn',
+                  text: 'A previous hourly benchmark run is still in progress. This invocation will be skipped.\nThis may indicate a long-running or stuck job.',
+                },
+              },
+              {
+                type: 'actions',
+                elements: [{
+                  type: 'button',
+                  text: { type: 'plain_text', text: 'View Run :github:', emoji: true },
+                  url: runUrl,
+                  action_id: 'ci_button',
+                }],
+              },
+            ];
+            await fetch('https://slack.com/api/chat.postMessage', {
+              method: 'POST',
+              headers: { 'Authorization': `Bearer ${token}`, 'Content-Type': 'application/json' },
+              body: JSON.stringify({ channel, blocks, text: 'Hourly bench: previous run still in progress', unfurl_links: false }),
+            });
+
+      - name: Alert on stale nightly
+        if: steps.mode.outputs.mode == 'nightly' && steps.refs.outputs.is-stale == 'true'
+        uses: actions/github-script@v8
+        env:
+          SLACK_BENCH_BOT_TOKEN: ${{ secrets.SLACK_BENCH_BOT_TOKEN }}
+          SLACK_BENCH_CHANNEL: ${{ secrets.SLACK_BENCH_CHANNEL }}
+        with:
+          script: |
+            const token = process.env.SLACK_BENCH_BOT_TOKEN;
+            const channel = process.env.SLACK_BENCH_CHANNEL;
+            if (!token || !channel) {
+              core.warning('Slack credentials not set, skipping stale nightly alert');
+              return;
+            }
+
+            const ageHours = '${{ steps.refs.outputs.stale-age-hours }}';
+            const created = '${{ steps.refs.outputs.nightly-created }}';
+            const featureRef = '${{ steps.refs.outputs.feature-ref }}';
+            const shortSha = featureRef.slice(0, 8);
+            const repo = '${{ github.repository }}';
+            const runUrl = `${context.serverUrl}/${repo}/actions/runs/${context.runId}`;
+
+            const blocks = [
+              {
+                type: 'header',
+                text: { type: 'plain_text', text: ':rotating_light: Nightly Regression: nightly build is stale', emoji: true },
+              },
+              {
+                type: 'section',
+                text: {
+                  type: 'mrkdwn',
+                  text: [
+                    '*Nightly regression did not run* — nightly build is stale',
+                    '',
+                    `The latest nightly image was built from a commit that is *${ageHours}h old* (threshold: 24h).`,
+                    `This means today's nightly docker build likely failed and no new image was produced.`,
+                    '',
+                    `Stale commit: \`${shortSha}\` (built at ${created})`,
+                    '',
+                    '*Action required:* Check the <https://github.com/' + repo + '/actions/workflows/docker.yml|docker.yml> workflow for failures.',
+                  ].join('\n'),
+                },
+              },
+              {
+                type: 'actions',
+                elements: [
+                  {
+                    type: 'button',
+                    text: { type: 'plain_text', text: 'View Run :github:', emoji: true },
+                    url: runUrl,
+                    action_id: 'ci_button',
+                  },
+                ],
+              },
+            ];
+
+            const resp = await fetch('https://slack.com/api/chat.postMessage', {
+              method: 'POST',
+              headers: {
+                'Authorization': `Bearer ${token}`,
+                'Content-Type': 'application/json',
+              },
+              body: JSON.stringify({
+                channel,
+                blocks,
+                text: 'Nightly regression: nightly build is stale',
+                unfurl_links: false,
+              }),
+            });
+            const data = await resp.json();
+            if (!data.ok) {
+              core.warning(`Slack API error: ${JSON.stringify(data)}`);
+            }
+
+      - name: Fail on stale nightly
+        if: steps.mode.outputs.mode == 'nightly' && steps.refs.outputs.is-stale == 'true'
+        run: |
+          echo "::error::Nightly build is stale (>24h old). Aborting."
+          exit 1
+
+  # ---------------------------------------------------------------------------
+  # Job 2: Run the benchmark
+  # ---------------------------------------------------------------------------
+  bench-scheduled:
+    needs: resolve-refs
+    if: |
+      needs.resolve-refs.outputs.should-skip != 'true' &&
+      needs.resolve-refs.outputs.is-stale != 'true'
+    name: bench-scheduled
+    runs-on: [self-hosted, Linux, X64, available]
+    timeout-minutes: 120
+    env:
+      BENCH_RPC_URL: https://ethereum.reth.rs/rpc
+      SCHELK_MOUNT: /reth-bench
+      BENCH_WORK_DIR: ${{ github.workspace }}/bench-work
+      BENCH_PR: ""
+      BENCH_MODE: ${{ needs.resolve-refs.outputs.mode }}
+      BENCH_ACTOR: "${{ needs.resolve-refs.outputs.mode }}-regression"
+      BENCH_BLOCKS: "2000"
+      BENCH_WARMUP_BLOCKS: "500"
+      BENCH_SAMPLY: "false"
+      BENCH_CORES: "0"
+      BENCH_BIG_BLOCKS: "false"
+      BENCH_RETH_NEW_PAYLOAD: "true"
+      BENCH_WAIT_TIME: ""
+      BENCH_BASELINE_ARGS: ""
+      BENCH_FEATURE_ARGS: ""
+      BENCH_ABBA: "true"
+      BENCH_COMMENT_ID: ""
+      BENCH_NO_SLACK: ${{ github.event_name == 'workflow_dispatch' && inputs.no_slack == true && 'true' || 'false' }}
+      BENCH_METRICS_ADDR: "127.0.0.1:9100"
+      BENCH_OTLP_TRACES_ENDPOINT: ${{ secrets.BENCH_OTLP_TRACES_ENDPOINT }}
+      BENCH_OTLP_LOGS_ENDPOINT: ${{ secrets.BENCH_OTLP_LOGS_ENDPOINT }}
+      BASELINE_REF: ${{ needs.resolve-refs.outputs.baseline-ref }}
+      FEATURE_REF: ${{ needs.resolve-refs.outputs.feature-ref }}
+    steps:
+      - name: Clean up previous bench-work
+        run: sudo rm -rf "$BENCH_WORK_DIR" 2>/dev/null || true
+
+      - uses: actions/checkout@v6
+        with:
+          submodules: true
+          fetch-depth: 0
+          ref: ${{ needs.resolve-refs.outputs.feature-ref }}
+
+      - name: Resolve job URL
+        id: job-url
+        uses: actions/github-script@v8
+        with:
+          script: |
+            const { data: jobs } = await github.rest.actions.listJobsForWorkflowRun({
+              owner: context.repo.owner,
+              repo: context.repo.repo,
+              run_id: context.runId,
+            });
+            const job = jobs.jobs.find(j => j.name === 'bench-scheduled');
+            const jobUrl = job ? job.html_url : `${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;
+            core.exportVariable('BENCH_JOB_URL', jobUrl);
+
+      - uses: dtolnay/rust-toolchain@stable
+      - uses: mozilla-actions/sccache-action@v0.0.9
+        continue-on-error: true
+
+      - name: Install dependencies
+        env:
+          DEREK_TOKEN: ${{ secrets.DEREK_TOKEN }}
+        run: |
+          mkdir -p "$HOME/.local/bin"
+
+          # apt packages
+          sudo apt-get update -qq
+          sudo apt-get install -y --no-install-recommends \
+            python3 make jq zstd curl dmsetup \
+            linux-tools-"$(uname -r)" || \
+            sudo apt-get install -y --no-install-recommends linux-tools-generic
+
+          # mc (MinIO client)
+          if ! command -v mc &>/dev/null; then
+            curl -sSfL https://dl.min.io/client/mc/release/linux-amd64/mc -o "$HOME/.local/bin/mc"
+            chmod +x "$HOME/.local/bin/mc"
+          fi
+
+          # uv (Python package manager)
+          if ! command -v uv &>/dev/null; then
+            curl -LsSf https://astral.sh/uv/install.sh | env UV_INSTALL_DIR="$HOME/.local/bin" sh
+          fi
+
+          # Configure git auth for private repos
+          git config --global url."https://x-access-token:${DEREK_TOKEN}@github.com/".insteadOf "https://github.com/"
+
+          # thin-provisioning-tools (era_invalidate, required by schelk)
+          if ! command -v era_invalidate &>/dev/null; then
+            git clone --depth 1 https://github.com/jthornber/thin-provisioning-tools /tmp/tpt
+            sudo make -C /tmp/tpt install
+            rm -rf /tmp/tpt
+          fi
+
+          # schelk (snapshot rollback tool, invoked via sudo)
+          if ! sudo sh -c 'command -v schelk' &>/dev/null; then
+            cargo install --git https://github.com/tempoxyz/schelk --locked
+            sudo install "$HOME/.cargo/bin/schelk" /usr/local/bin/
+          fi
+
+      - name: Check dependencies
+        run: |
+          export PATH="$HOME/.local/bin:$HOME/.cargo/bin:$PATH"
+          echo "$HOME/.local/bin" >> "$GITHUB_PATH"
+          echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
+          missing=()
+          for cmd in mc schelk cpupower taskset stdbuf python3 curl make uv pzstd jq; do
+            command -v "$cmd" &>/dev/null || missing+=("$cmd")
+          done
+          if [ ${#missing[@]} -gt 0 ]; then
+            echo "::error::Missing required tools: ${missing[*]}"
+            exit 1
+          fi
+          echo "All dependencies found"
+
+      - name: Resolve display names
+        id: refs
+        run: |
+          BASELINE_SHORT=$(echo "$BASELINE_REF" | cut -c1-8)
+          FEATURE_SHORT=$(echo "$FEATURE_REF" | cut -c1-8)
+          echo "baseline-name=${BENCH_MODE}-${BASELINE_SHORT}" >> "$GITHUB_OUTPUT"
+          echo "feature-name=${BENCH_MODE}-${FEATURE_SHORT}" >> "$GITHUB_OUTPUT"
+          echo "baseline-ref=$BASELINE_REF" >> "$GITHUB_OUTPUT"
+          echo "feature-ref=$FEATURE_REF" >> "$GITHUB_OUTPUT"
+
+      - name: Check if snapshot needs update
+        id: snapshot-check
+        run: |
+          set +e
+          .github/scripts/bench-reth-snapshot.sh --check
+          rc=$?
+          set -e
+          case "$rc" in
+            0)  echo "needed=false" >> "$GITHUB_OUTPUT" ;;
+            10) echo "needed=true"  >> "$GITHUB_OUTPUT" ;;
+            *)  echo "::error::Snapshot check failed (exit $rc)"
+                exit "$rc" ;;
+          esac
+
+      - name: Prepare source dirs
+        run: |
+          if [ -d ../reth-baseline ]; then
+            git -C ../reth-baseline fetch origin "$BASELINE_REF"
+          else
+            git clone . ../reth-baseline
+          fi
+          git -C ../reth-baseline checkout "$BASELINE_REF"
+
+          if [ -d ../reth-feature ]; then
+            git -C ../reth-feature fetch origin "$FEATURE_REF"
+          else
+            git clone . ../reth-feature
+          fi
+          git -C ../reth-feature checkout "$FEATURE_REF"
+
+      - name: Build binaries
+        id: build
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          BENCH_REPO: ${{ github.repository }}
+        run: |
+          BASELINE_DIR="$(cd ../reth-baseline && pwd)"
+          FEATURE_DIR="$(cd ../reth-feature && pwd)"
+
+          .github/scripts/bench-reth-build.sh baseline "${BASELINE_DIR}" "$BASELINE_REF" &
+          PID_BASELINE=$!
+          .github/scripts/bench-reth-build.sh feature "${FEATURE_DIR}" "$FEATURE_REF" &
+          PID_FEATURE=$!
+
+          FAIL=0
+          wait $PID_BASELINE || FAIL=1
+          wait $PID_FEATURE || FAIL=1
+          if [ $FAIL -ne 0 ]; then
+            echo "::error::One or more build tasks failed"
+            exit 1
+          fi
+
+      - name: Download snapshot
+        id: snapshot-download
+        if: steps.snapshot-check.outputs.needed == 'true'
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          BENCH_REPO: ${{ github.repository }}
+          BENCH_RETH_BINARY: ${{ github.workspace }}/../reth-feature/target/profiling/reth
+        run: .github/scripts/bench-reth-snapshot.sh
+
+      # System tuning for reproducible benchmarks
+      - name: System setup
+        run: |
+          sudo cpupower frequency-set -g performance || true
+          # Disable turbo boost (Intel and AMD paths)
+          echo 1 | sudo tee /sys/devices/system/cpu/intel_pstate/no_turbo 2>/dev/null || true
+          echo 0 | sudo tee /sys/devices/system/cpu/cpufreq/boost 2>/dev/null || true
+          sudo swapoff -a || true
+          echo 0 | sudo tee /proc/sys/kernel/randomize_va_space || true
+          # Disable SMT (hyperthreading)
+          for cpu in /sys/devices/system/cpu/cpu*/topology/thread_siblings_list; do
+            first=$(cut -d, -f1 < "$cpu" | cut -d- -f1)
+            current=$(echo "$cpu" | grep -o 'cpu[0-9]*' | grep -o '[0-9]*')
+            if [ "$current" != "$first" ]; then
+              echo 0 | sudo tee "/sys/devices/system/cpu/cpu${current}/online" || true
+            fi
+          done
+          echo "Online CPUs: $(nproc)"
+          # Disable transparent huge pages
+          for p in /sys/kernel/mm/transparent_hugepage /sys/kernel/mm/transparent_hugepages; do
+            [ -d "$p" ] && echo never | sudo tee "$p/enabled" && echo never | sudo tee "$p/defrag" && break
+          done || true
+          # Prevent deep C-states
+          sudo sh -c 'exec 3<>/dev/cpu_dma_latency; echo -ne "\x00\x00\x00\x00" >&3; sleep infinity' &
+          # Move all IRQs to core 0
+          for irq in /proc/irq/*/smp_affinity_list; do
+            echo 0 | sudo tee "$irq" 2>/dev/null || true
+          done
+          # Stop noisy background services
+          sudo systemctl stop irqbalance cron atd unattended-upgrades snapd 2>/dev/null || true
+          echo "=== Benchmark environment ==="
+          uname -r
+          lscpu | grep -E 'Model name|CPU\(s\)|MHz|NUMA'
+          cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
+          cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq
+          cat /sys/kernel/mm/transparent_hugepage/enabled 2>/dev/null || cat /sys/kernel/mm/transparent_hugepages/enabled 2>/dev/null || echo "THP: unknown"
+          free -h
+
+      - name: Pre-flight cleanup
+        run: |
+          sudo pkill -9 reth || true
+          sleep 1
+          if mountpoint -q "$SCHELK_MOUNT"; then
+            sudo umount -l "$SCHELK_MOUNT" || true
+            sudo schelk recover -y || true
+          fi
+          rm -rf "$BENCH_WORK_DIR"
+          mkdir -p "$BENCH_WORK_DIR"
+
+      - name: Start metrics proxy
+        run: |
+          BENCH_ID="${BENCH_MODE}-${{ github.run_id }}"
+          BENCH_REFERENCE_EPOCH=$(date +%s)
+          echo "BENCH_ID=${BENCH_ID}" >> "$GITHUB_ENV"
+          echo "BENCH_REFERENCE_EPOCH=${BENCH_REFERENCE_EPOCH}" >> "$GITHUB_ENV"
+
+          LABELS_FILE="/tmp/bench-metrics-labels.json"
+          echo '{}' > "$LABELS_FILE"
+          echo "BENCH_LABELS_FILE=${LABELS_FILE}" >> "$GITHUB_ENV"
+
+          python3 .github/scripts/bench-metrics-proxy.py \
+            --labels "$LABELS_FILE" \
+            --upstream "http://${BENCH_METRICS_ADDR}/" \
+            --subnet 10.10.0.0/24 \
+            --port 9090 &
+          PROXY_PID=$!
+          echo "BENCH_METRICS_PROXY_PID=${PROXY_PID}" >> "$GITHUB_ENV"
+          echo "Metrics proxy started (PID $PROXY_PID)"
+
+      # Interleaved run order (B-F-F-B) to reduce systematic bias
+      - name: "Run benchmark: baseline (1/2)"
+        id: run-baseline-1
+        run: |
+          cat > "$BENCH_LABELS_FILE" <<LABELS
+          {"benchmark_run":"baseline-1","run_type":"baseline","git_ref":"${BASELINE_REF}","bench_sha":"${BASELINE_REF}","benchmark_id":"${BENCH_ID}","run_start_epoch":"$(date +%s)","reference_epoch":"${BENCH_REFERENCE_EPOCH}"}
+          LABELS
+          taskset -c 0 .github/scripts/bench-reth-run.sh baseline ../reth-baseline/target/profiling/reth "$BENCH_WORK_DIR/baseline-1"
+
+      - name: "Run benchmark: feature (1/2)"
+        id: run-feature-1
+        run: |
+          cat > "$BENCH_LABELS_FILE" <<LABELS
+          {"benchmark_run":"feature-1","run_type":"feature","git_ref":"${FEATURE_REF}","bench_sha":"${FEATURE_REF}","benchmark_id":"${BENCH_ID}","run_start_epoch":"$(date +%s)","reference_epoch":"${BENCH_REFERENCE_EPOCH}"}
+          LABELS
+          taskset -c 0 .github/scripts/bench-reth-run.sh feature ../reth-feature/target/profiling/reth "$BENCH_WORK_DIR/feature-1"
+
+      - name: "Run benchmark: feature (2/2)"
+        id: run-feature-2
+        run: |
+          cat > "$BENCH_LABELS_FILE" <<LABELS
+          {"benchmark_run":"feature-2","run_type":"feature","git_ref":"${FEATURE_REF}","bench_sha":"${FEATURE_REF}","benchmark_id":"${BENCH_ID}","run_start_epoch":"$(date +%s)","reference_epoch":"${BENCH_REFERENCE_EPOCH}"}
+          LABELS
+          taskset -c 0 .github/scripts/bench-reth-run.sh feature ../reth-feature/target/profiling/reth "$BENCH_WORK_DIR/feature-2"
+
+      - name: "Run benchmark: baseline (2/2)"
+        id: run-baseline-2
+        run: |
+          LAST_RUN_START=$(date +%s)
+          echo "BENCH_LAST_RUN_START=${LAST_RUN_START}" >> "$GITHUB_ENV"
+          cat > "$BENCH_LABELS_FILE" <<LABELS
+          {"benchmark_run":"baseline-2","run_type":"baseline","git_ref":"${BASELINE_REF}","bench_sha":"${BASELINE_REF}","benchmark_id":"${BENCH_ID}","run_start_epoch":"${LAST_RUN_START}","reference_epoch":"${BENCH_REFERENCE_EPOCH}"}
+          LABELS
+          taskset -c 0 .github/scripts/bench-reth-run.sh baseline ../reth-baseline/target/profiling/reth "$BENCH_WORK_DIR/baseline-2"
+
+      - name: Stop metrics proxy & generate Grafana URL
+        id: metrics
+        if: "!cancelled()"
+        run: |
+          kill "$BENCH_METRICS_PROXY_PID" 2>/dev/null || true
+
+          LAST_RUN_DURATION=$(( $(date +%s) - BENCH_LAST_RUN_START ))
+          FROM_MS=$(( BENCH_REFERENCE_EPOCH * 1000 ))
+          TO_MS=$(( (BENCH_REFERENCE_EPOCH + LAST_RUN_DURATION) * 1000 ))
+          GRAFANA_URL="https://tempoxyz.grafana.net/d/reth-bench-ghr/reth-bench-ghr?orgId=1&from=${FROM_MS}&to=${TO_MS}&timezone=browser&var-datasource=ef57fux92e9z4e&var-job=reth-bench&var-benchmark_id=${BENCH_ID}&var-benchmark_run=\$__all"
+          echo "grafana-url=${GRAFANA_URL}" >> "$GITHUB_OUTPUT"
+          echo "Grafana URL: ${GRAFANA_URL}"
+
+      - name: Scan logs for errors
+        if: "!cancelled()"
+        run: |
+          ERRORS_FILE="$BENCH_WORK_DIR/errors.md"
+          found=false
+          for run_dir in baseline-1 feature-1 feature-2 baseline-2; do
+            LOG="$BENCH_WORK_DIR/$run_dir/node.log"
+            if [ ! -f "$LOG" ]; then continue; fi
+
+            panics=$(grep -c -E 'panicked at' "$LOG" || true)
+            errors=$(grep -c ' ERROR ' "$LOG" || true)
+
+            if [ "$panics" -gt 0 ] || [ "$errors" -gt 0 ]; then
+              if [ "$found" = false ]; then
+                printf '### ⚠️ Node Errors\n\n' >> "$ERRORS_FILE"
+                found=true
+              fi
+              printf '<details><summary><b>%s</b>: %d panic(s), %d error(s)</summary>\n\n' "$run_dir" "$panics" "$errors" >> "$ERRORS_FILE"
+              if [ "$panics" -gt 0 ]; then
+                printf '**Panics:**\n```\n' >> "$ERRORS_FILE"
+                grep -E 'panicked at' "$LOG" | head -10 >> "$ERRORS_FILE"
+                printf '```\n' >> "$ERRORS_FILE"
+              fi
+              if [ "$errors" -gt 0 ]; then
+                printf '**Errors (first 20):**\n```\n' >> "$ERRORS_FILE"
+                grep ' ERROR ' "$LOG" | head -20 >> "$ERRORS_FILE"
+                printf '```\n' >> "$ERRORS_FILE"
+              fi
+              printf '\n</details>\n\n' >> "$ERRORS_FILE"
+            fi
+          done
+
+      - name: Parse results
+        id: results
+        if: success()
+        env:
+          BASELINE_NAME: ${{ steps.refs.outputs.baseline-name }}
+          FEATURE_NAME: ${{ steps.refs.outputs.feature-name }}
+        run: |
+          SUMMARY_ARGS="--output-summary $BENCH_WORK_DIR/summary.json"
+          SUMMARY_ARGS="$SUMMARY_ARGS --output-markdown $BENCH_WORK_DIR/comment.md"
+          SUMMARY_ARGS="$SUMMARY_ARGS --repo ${{ github.repository }}"
+          SUMMARY_ARGS="$SUMMARY_ARGS --baseline-ref ${BASELINE_REF}"
+          SUMMARY_ARGS="$SUMMARY_ARGS --baseline-name ${BASELINE_NAME}"
+          SUMMARY_ARGS="$SUMMARY_ARGS --feature-name ${FEATURE_NAME}"
+          SUMMARY_ARGS="$SUMMARY_ARGS --feature-ref ${FEATURE_REF}"
+
+          BASELINE_CSVS="$BENCH_WORK_DIR/baseline-1/combined_latency.csv"
+          FEATURE_CSVS="$BENCH_WORK_DIR/feature-1/combined_latency.csv"
+          BASELINE_CSVS="$BASELINE_CSVS $BENCH_WORK_DIR/baseline-2/combined_latency.csv"
+          FEATURE_CSVS="$FEATURE_CSVS $BENCH_WORK_DIR/feature-2/combined_latency.csv"
+          SUMMARY_ARGS="$SUMMARY_ARGS --baseline-csv $BASELINE_CSVS"
+          SUMMARY_ARGS="$SUMMARY_ARGS --feature-csv $FEATURE_CSVS"
+          SUMMARY_ARGS="$SUMMARY_ARGS --gas-csv $BENCH_WORK_DIR/feature-1/total_gas.csv"
+
+          GRAFANA_URL='${{ steps.metrics.outputs.grafana-url }}'
+          if [ -n "$GRAFANA_URL" ]; then
+            SUMMARY_ARGS="$SUMMARY_ARGS --grafana-url $GRAFANA_URL"
+          fi
+          # shellcheck disable=SC2086
+          python3 .github/scripts/bench-reth-summary.py $SUMMARY_ARGS
+
+      - name: Generate charts
+        if: success() && env.BENCH_MODE != 'hourly'
+        env:
+          BASELINE_NAME: ${{ steps.refs.outputs.baseline-name }}
+          FEATURE_NAME: ${{ steps.refs.outputs.feature-name }}
+        run: |
+          CHART_ARGS="--output-dir $BENCH_WORK_DIR/charts"
+          FEATURE_CSVS="$BENCH_WORK_DIR/feature-1/combined_latency.csv"
+          BASELINE_CSVS="$BENCH_WORK_DIR/baseline-1/combined_latency.csv"
+          FEATURE_CSVS="$FEATURE_CSVS $BENCH_WORK_DIR/feature-2/combined_latency.csv"
+          BASELINE_CSVS="$BASELINE_CSVS $BENCH_WORK_DIR/baseline-2/combined_latency.csv"
+          CHART_ARGS="$CHART_ARGS --feature $FEATURE_CSVS"
+          CHART_ARGS="$CHART_ARGS --baseline $BASELINE_CSVS"
+          CHART_ARGS="$CHART_ARGS --baseline-name ${BASELINE_NAME}"
+          CHART_ARGS="$CHART_ARGS --feature-name ${FEATURE_NAME}"
+          # shellcheck disable=SC2086
+          uv run --with matplotlib python3 .github/scripts/bench-reth-charts.py $CHART_ARGS
+
+      - name: Upload results
+        if: "!cancelled()"
+        uses: actions/upload-artifact@v7
+        with:
+          name: bench-scheduled-results
+          path: ${{ env.BENCH_WORK_DIR }}
+
+      - name: Push charts
+        id: push-charts
+        if: success() && env.BENCH_MODE != 'hourly'
+        run: |
+          RUN_ID=${{ github.run_id }}
+          CHART_DIR="nightly/${RUN_ID}"
+          CHARTS_REPO="https://x-access-token:${{ secrets.DEREK_TOKEN }}@github.com/decofe/reth-bench-charts.git"
+
+          TMP_DIR=$(mktemp -d)
+          if git clone --depth 1 "${CHARTS_REPO}" "${TMP_DIR}" 2>/dev/null; then
+            true
+          else
+            git init "${TMP_DIR}"
+            git -C "${TMP_DIR}" remote add origin "${CHARTS_REPO}"
+          fi
+
+          mkdir -p "${TMP_DIR}/${CHART_DIR}"
+          cp "$BENCH_WORK_DIR"/charts/*.png "${TMP_DIR}/${CHART_DIR}/"
+          git -C "${TMP_DIR}" add "${CHART_DIR}"
+          git -C "${TMP_DIR}" -c user.name="github-actions" -c user.email="github-actions@github.com" \
+            commit -m "nightly bench charts for run ${RUN_ID}"
+          git -C "${TMP_DIR}" push origin HEAD:main
+          echo "sha=$(git -C "${TMP_DIR}" rev-parse HEAD)" >> "$GITHUB_OUTPUT"
+          rm -rf "${TMP_DIR}"
+
+      - name: Write job summary
+        if: success()
+        uses: actions/github-script@v8
+        with:
+          script: |
+            const fs = require('fs');
+            const { verdict, metricRows, waitTimeRows, blocksLabel } = require('./.github/scripts/bench-utils');
+
+            let summary;
+            try {
+              summary = JSON.parse(fs.readFileSync(process.env.BENCH_WORK_DIR + '/summary.json', 'utf8'));
+            } catch (e) {
+              await core.summary.addRaw('⚠️ Benchmark completed but failed to load summary.').write();
+              return;
+            }
+
+            const repo = `${context.repo.owner}/${context.repo.repo}`;
+            const commitUrl = `https://github.com/${repo}/commit`;
+            const { emoji, label } = verdict(summary.changes);
+            const baselineLink = `[\`${summary.baseline.name}\`](${commitUrl}/${summary.baseline.ref})`;
+            const featureLink = `[\`${summary.feature.name}\`](${commitUrl}/${summary.feature.ref})`;
+            const diffUrl = `https://github.com/${repo}/compare/${summary.baseline.ref}...${summary.feature.ref}`;
+
+            const mode = process.env.BENCH_MODE || 'nightly';
+            const modeLabel = mode === 'hourly' ? 'Hourly Regression' : 'Nightly Regression';
+            let md = `# ${emoji} ${modeLabel}: ${label}\n\n`;
+            md += `**Baseline:** ${baselineLink}\n`;
+            md += `**Feature:** ${featureLink} ([diff](${diffUrl}))\n`;
+            md += blocksLabel(summary).map(p => `**${p.key}:** ${p.value}`).join(' · ') + '\n\n';
+
+            const rows = metricRows(summary);
+            md += `| Metric | Baseline | Feature | Change |\n`;
+            md += `|--------|----------|---------|--------|\n`;
+            for (const r of rows) {
+              md += `| ${r.label} | ${r.baseline} | ${r.feature} | ${r.change} |\n`;
+            }
+            md += '\n';
+
+            const wtRows = waitTimeRows(summary);
+            if (wtRows.length > 0) {
+              md += `### Wait Time Breakdown\n\n`;
+              md += `| Metric | Baseline | Feature |\n`;
+              md += `|--------|----------|--------|\n`;
+              for (const r of wtRows) {
+                md += `| ${r.title} | ${r.baseline} | ${r.feature} |\n`;
+              }
+              md += '\n';
+            }
+
+            // Charts
+            const chartSha = '${{ steps.push-charts.outputs.sha }}';
+            if (chartSha) {
+              const runId = '${{ github.run_id }}';
+              const baseUrl = `https://raw.githubusercontent.com/decofe/reth-bench-charts/${chartSha}/nightly/${runId}`;
+              const charts = [
+                { file: 'latency_throughput.png', label: 'Latency, Throughput & Diff' },
+                { file: 'wait_breakdown.png', label: 'Wait Time Breakdown' },
+                { file: 'gas_vs_latency.png', label: 'Gas vs Latency' },
+              ];
+              md += `### Charts\n\n`;
+              for (const chart of charts) {
+                md += `<details><summary>${chart.label}</summary>\n\n`;
+                md += `![${chart.label}](${baseUrl}/${chart.file})\n\n`;
+                md += `</details>\n\n`;
+              }
+            }
+
+            const grafanaUrl = '${{ steps.metrics.outputs.grafana-url }}';
+            if (grafanaUrl) {
+              md += `### Grafana Dashboard\n\n[View real-time metrics](${grafanaUrl})\n\n`;
+            }
+
+            try {
+              const errors = fs.readFileSync(process.env.BENCH_WORK_DIR + '/errors.md', 'utf8');
+              if (errors.trim()) md += '\n' + errors + '\n';
+            } catch {}
+
+            await core.summary.addRaw(md).write();
+
+      - name: Send Slack notification (success)
+        if: success() && env.BENCH_NO_SLACK != 'true'
+        uses: actions/github-script@v8
+        env:
+          SLACK_BENCH_BOT_TOKEN: ${{ secrets.SLACK_BENCH_BOT_TOKEN }}
+          SLACK_BENCH_CHANNEL: ${{ secrets.SLACK_BENCH_CHANNEL }}
+        with:
+          script: |
+            const fs = require('fs');
+            const { verdict, fmtChange, fmtMs, metricRows, waitTimeRows, blocksLabel } = require('./.github/scripts/bench-utils');
+
+            const token = process.env.SLACK_BENCH_BOT_TOKEN;
+            const channel = process.env.SLACK_BENCH_CHANNEL;
+            if (!token || !channel) {
+              core.info('Slack credentials not set, skipping notification');
+              return;
+            }
+
+            let summary;
+            try {
+              summary = JSON.parse(fs.readFileSync(process.env.BENCH_WORK_DIR + '/summary.json', 'utf8'));
+            } catch (e) {
+              core.warning('Could not read summary.json for Slack notification');
+              return;
+            }
+
+            // Filter notifications based on mode
+            const changes = summary.changes || {};
+            const mode = process.env.BENCH_MODE || 'nightly';
+            const hasRegression = Object.values(changes).some(c => c.sig === 'bad');
+            const hasSignificant = Object.values(changes).some(c => c.sig === 'good' || c.sig === 'bad');
+
+            // Hourly mode: only notify on regressions
+            if (mode === 'hourly' && !hasRegression) {
+              core.info('Hourly mode: no regression detected, skipping Slack notification');
+              return;
+            }
+
+            // Nightly mode: notify on any significant change (regression or improvement)
+            if (!hasSignificant) {
+              core.info('No significant changes detected, skipping nightly Slack notification');
+              return;
+            }
+
+            const SLACK_VERDICT = {
+              '⚠️': ':warning:',
+              '❌': ':x:',
+              '✅': ':white_check_mark:',
+              '⚪': ':white_circle:',
+            };
+
+            const repo = `${context.repo.owner}/${context.repo.repo}`;
+            const { emoji, label } = verdict(changes);
+            const headerEmoji = SLACK_VERDICT[emoji] || emoji;
+            const commitUrl = `https://github.com/${repo}/commit`;
+            const baselineLink = `<${commitUrl}/${summary.baseline.ref}|${summary.baseline.name}>`;
+            const featureLink = `<${commitUrl}/${summary.feature.ref}|${summary.feature.name}>`;
+            const diffUrl = `https://github.com/${repo}/compare/${summary.baseline.ref}...${summary.feature.ref}`;
+            const jobUrl = process.env.BENCH_JOB_URL || `${context.serverUrl}/${repo}/actions/runs/${context.runId}`;
+
+            function cell(text) { return { type: 'raw_text', text: String(text) || ' ' }; }
+
+            const modeLabel = mode === 'hourly' ? 'Hourly Regression' : 'Nightly Regression';
+            const sectionText = [
+              `*${modeLabel}*`,
+              '',
+              `*Baseline:* ${baselineLink}`,
+              `*Feature:* ${featureLink}`,
+              blocksLabel(summary).map(p => `*${p.key}:* ${p.value}`).join(' | '),
+            ].join('\n');
+
+            const rows = metricRows(summary);
+            const tableRows = [
+              [cell('Metric'), cell('Baseline'), cell('Feature'), cell('Change')],
+              ...rows.map(r => [cell(r.label), cell(r.baseline), cell(r.feature), cell(r.change || ' ')]),
+            ];
+
+            const blocks = [
+              {
+                type: 'header',
+                text: { type: 'plain_text', text: `${headerEmoji} ${modeLabel}: ${label}`, emoji: true },
+              },
+              {
+                type: 'section',
+                text: { type: 'mrkdwn', text: sectionText },
+              },
+              {
+                type: 'table',
+                column_settings: [{ align: 'left' }, { align: 'right' }, { align: 'right' }, { align: 'right' }],
+                rows: tableRows,
+              },
+              {
+                type: 'actions',
+                elements: [
+                  {
+                    type: 'button',
+                    text: { type: 'plain_text', text: 'CI :github:', emoji: true },
+                    url: jobUrl,
+                    action_id: 'ci_button',
+                  },
+                  {
+                    type: 'button',
+                    text: { type: 'plain_text', text: 'Diff :github:', emoji: true },
+                    url: diffUrl,
+                    action_id: 'diff_button',
+                  },
+                ],
+              },
+            ];
+
+            const text = `${modeLabel}: ${summary.baseline.name} vs ${summary.feature.name}`;
+            const resp = await fetch('https://slack.com/api/chat.postMessage', {
+              method: 'POST',
+              headers: {
+                'Authorization': `Bearer ${token}`,
+                'Content-Type': 'application/json',
+              },
+              body: JSON.stringify({ channel, blocks, text, unfurl_links: false }),
+            });
+            const data = await resp.json();
+            if (!data.ok) {
+              core.warning(`Slack API error: ${JSON.stringify(data)}`);
+              return;
+            }
+
+            // Post wait time breakdown as threaded reply
+            const wtRows = waitTimeRows(summary);
+            if (data.ts && wtRows.length > 0) {
+              const waitTableRows = [
+                [cell('Wait Time'), cell('Baseline'), cell('Feature')],
+                ...wtRows.map(r => [cell(r.title), cell(r.baseline), cell(r.feature)]),
+              ];
+              await fetch('https://slack.com/api/chat.postMessage', {
+                method: 'POST',
+                headers: {
+                  'Authorization': `Bearer ${token}`,
+                  'Content-Type': 'application/json',
+                },
+                body: JSON.stringify({
+                  channel,
+                  thread_ts: data.ts,
+                  blocks: [{
+                    type: 'table',
+                    column_settings: [{ align: 'left' }, { align: 'right' }, { align: 'right' }],
+                    rows: waitTableRows,
+                  }],
+                  text: 'Wait time breakdown',
+                  unfurl_links: false,
+                }),
+              });
+            }
+
+      - name: Send Slack notification (failure)
+        if: failure()
+        uses: actions/github-script@v8
+        env:
+          SLACK_BENCH_BOT_TOKEN: ${{ secrets.SLACK_BENCH_BOT_TOKEN }}
+          SLACK_BENCH_CHANNEL: ${{ secrets.SLACK_BENCH_CHANNEL }}
+        with:
+          script: |
+            const token = process.env.SLACK_BENCH_BOT_TOKEN;
+            const channel = process.env.SLACK_BENCH_CHANNEL;
+            if (!token || !channel) return;
+
+            const steps_status = [
+              ['building binaries', '${{ steps.build.outcome }}'],
+              ['downloading snapshot', '${{ steps.snapshot-download.outcome }}'],
+              ['running baseline benchmark (1/2)', '${{ steps.run-baseline-1.outcome }}'],
+              ['running feature benchmark (1/2)', '${{ steps.run-feature-1.outcome }}'],
+              ['running feature benchmark (2/2)', '${{ steps.run-feature-2.outcome }}'],
+              ['running baseline benchmark (2/2)', '${{ steps.run-baseline-2.outcome }}'],
+            ];
+            const failed = steps_status.find(([, o]) => o === 'failure');
+            const failedStep = failed ? failed[0] : 'unknown step';
+
+            const repo = `${context.repo.owner}/${context.repo.repo}`;
+            const jobUrl = process.env.BENCH_JOB_URL || `${context.serverUrl}/${repo}/actions/runs/${context.runId}`;
+
+            const mode = process.env.BENCH_MODE || 'nightly';
+            const modeLabel = mode === 'hourly' ? 'Hourly' : 'Nightly';
+
+            const blocks = [
+              {
+                type: 'header',
+                text: { type: 'plain_text', text: `:rotating_light: ${modeLabel} Bench Failed`, emoji: true },
+              },
+              {
+                type: 'section',
+                text: { type: 'mrkdwn', text: `*${modeLabel} regression* failed while *${failedStep}*\ncc <@U09FARE0B9Q> <@U09FAL2UMLJ>` },
+              },
+              {
+                type: 'actions',
+                elements: [{
+                  type: 'button',
+                  text: { type: 'plain_text', text: 'View Logs :github:', emoji: true },
+                  url: jobUrl,
+                  action_id: 'ci_button',
+                }],
+              },
+            ];
+
+            await fetch('https://slack.com/api/chat.postMessage', {
+              method: 'POST',
+              headers: {
+                'Authorization': `Bearer ${token}`,
+                'Content-Type': 'application/json',
+              },
+              body: JSON.stringify({
+                channel,
+                blocks,
+                text: `${modeLabel} bench failed while ${failedStep}`,
+                unfurl_links: false,
+              }),
+            });
+
+      - name: Restore system settings
+        if: always()
+        run: |
+          sudo systemctl start irqbalance cron atd 2>/dev/null || true
+
+  # ---------------------------------------------------------------------------
+  # Job 3: Save state on success
+  # ---------------------------------------------------------------------------
+  save-state:
+    needs: [resolve-refs, bench-scheduled]
+    if: success()
+    name: save-state
+    runs-on: ubuntu-latest
+    steps:
+      - name: Write state file
+        run: |
+          MODE="${{ needs.resolve-refs.outputs.mode }}"
+          STATE_DIR=".${MODE}-state"
+          mkdir -p "$STATE_DIR"
+          echo "${{ needs.resolve-refs.outputs.feature-ref }}" > "$STATE_DIR/last-feature-ref"
+
+      - name: Save state
+        uses: actions/cache/save@v4
+        with:
+          path: .${{ needs.resolve-refs.outputs.mode }}-state
+          key: bench-${{ needs.resolve-refs.outputs.mode }}-state-${{ needs.resolve-refs.outputs.feature-ref }}

.github/workflows/bench.yml

@@ -7,12 +7,8 @@
 # same block range (snapshot recovered between runs) to compare performance.
 
 on:
-  # TODO: Disabled temporarily for https://github.com/CodSpeedHQ/runner/issues/55
-  # merge_group:
-  push:
-    branches: [main]
   issue_comment:
-    types: [created, edited]
+    types: [created]
   workflow_dispatch:
     inputs:
       blocks:
@@ -20,6 +16,11 @@ on:
         required: false
         default: "500"
         type: string
+      big_blocks:
+        description: "Use big blocks mode (pre-generated merged payloads with reth-bb)"
+        required: false
+        default: "false"
+        type: boolean
       warmup:
         description: "Number of warmup blocks"
         required: false
@@ -35,21 +36,54 @@ on:
         required: false
         default: ""
         type: string
+      wait_time:
+        description: "Fixed wait time between blocks (e.g. 500ms, 1s)"
+        required: false
+        default: ""
+        type: string
+      baseline_args:
+        description: "Extra CLI args for the baseline reth node"
+        required: false
+        default: ""
+        type: string
+      feature_args:
+        description: "Extra CLI args for the feature reth node"
+        required: false
+        default: ""
+        type: string
       samply:
         description: "Enable samply profiling"
         required: false
         default: "false"
         type: boolean
+      reth_newPayload:
+        description: "Use reth_newPayload RPC (server-side timing)"
+        required: false
+        default: "true"
+        type: boolean
       cores:
         description: "Limit reth to N CPU cores (0 = all available)"
         required: false
         default: "0"
         type: string
+      no_slack:
+        description: "Suppress Slack notifications for benchmark results"
+        required: false
+        default: "true"
+        type: boolean
+      abba:
+        description: "Run ABBA (BFFB) interleaved order; false = single AB pass"
+        required: false
+        default: "true"
+        type: boolean
+      otlp:
+        description: "Export OTLP traces and logs"
+        required: false
+        default: "true"
+        type: boolean
 
 env:
   CARGO_TERM_COLOR: always
-  BASELINE: base
-  SEED: reth
   RUSTC_WRAPPER: "sccache"
   BENCH_RUNNERS: 2
 
@@ -60,46 +94,6 @@ permissions:
   pull-requests: write
 
 jobs:
-  codspeed:
-    if: github.event_name == 'push'
-    runs-on: depot-ubuntu-latest
-    concurrency:
-      group: bench-codspeed-${{ github.head_ref || github.run_id }}
-      cancel-in-progress: true
-    strategy:
-      matrix:
-        partition: [1, 2]
-        total_partitions: [2]
-        include:
-          - partition: 1
-            crates: "-p reth-primitives -p reth-trie-common -p reth-trie-sparse"
-          - partition: 2
-            crates: "-p reth-trie"
-    name: codspeed (${{ matrix.partition }}/${{ matrix.total_partitions }})
-    steps:
-      - uses: actions/checkout@v6
-        with:
-          submodules: true
-          ref: ${{ github.event_name == 'issue_comment' && format('refs/pull/{0}/merge', github.event.issue.number) || '' }}
-      - uses: rui314/setup-mold@v1
-      - uses: dtolnay/rust-toolchain@stable
-      - uses: mozilla-actions/sccache-action@v0.0.9
-      - uses: Swatinem/rust-cache@v2
-        with:
-          cache-on-failure: true
-      - name: Install cargo-codspeed
-        uses: taiki-e/install-action@v2
-        with:
-          tool: cargo-codspeed
-      - name: Build the benchmark target(s)
-        run: cargo codspeed build --profile profiling --features test-utils ${{ matrix.crates }}
-      - name: Run the benchmarks
-        uses: CodSpeedHQ/action@v4
-        with:
-          run: cargo codspeed run ${{ matrix.crates }}
-          mode: instrumentation
-          token: ${{ secrets.CODSPEED_TOKEN }}
-
   reth-bench-ack:
     if: |
       (github.event_name == 'issue_comment' && github.event.issue.pull_request && (startsWith(github.event.comment.body, '@decofe bench') || startsWith(github.event.comment.body, 'derek bench'))) ||
@@ -116,7 +110,15 @@ jobs:
       baseline-name: ${{ steps.args.outputs.baseline-name }}
       feature-name: ${{ steps.args.outputs.feature-name }}
       samply: ${{ steps.args.outputs.samply }}
+      no-slack: ${{ steps.args.outputs.no-slack }}
       cores: ${{ steps.args.outputs.cores }}
+      big-blocks: ${{ steps.args.outputs.big-blocks }}
+      reth-new-payload: ${{ steps.args.outputs.reth-new-payload }}
+      wait-time: ${{ steps.args.outputs.wait-time }}
+      baseline-args: ${{ steps.args.outputs.baseline-args }}
+      feature-args: ${{ steps.args.outputs.feature-args }}
+      abba: ${{ steps.args.outputs.abba }}
+      otlp: ${{ steps.args.outputs.otlp }}
       comment-id: ${{ steps.ack.outputs.comment-id }}
     steps:
       - name: Check org membership
@@ -144,7 +146,7 @@ jobs:
         with:
           github-token: ${{ secrets.DEREK_PAT }}
           script: |
-            let pr, actor, blocks, warmup, baseline, feature, samply, cores;
+            let pr, actor, blocks, warmup, baseline, feature, samply, cores, bigBlocks;
 
             if (context.eventName === 'workflow_dispatch') {
               actor = '${{ github.actor }}';
@@ -153,7 +155,15 @@ jobs:
               baseline = '${{ github.event.inputs.baseline }}';
               feature = '${{ github.event.inputs.feature }}';
               samply = '${{ github.event.inputs.samply }}' === 'true' ? 'true' : 'false';
+              var noSlack = '${{ github.event.inputs.no_slack }}' !== 'false' ? 'true' : 'false';
               cores = '${{ github.event.inputs.cores }}' || '0';
+              bigBlocks = '${{ github.event.inputs.big_blocks }}' === 'true' ? 'true' : 'false';
+              var rethNewPayload = '${{ github.event.inputs.reth_newPayload }}' !== 'false' ? 'true' : 'false';
+              var abba = '${{ github.event.inputs.abba }}' !== 'false' ? 'true' : 'false';
+              var otlp = '${{ github.event.inputs.otlp }}' !== 'false' ? 'true' : 'false';
+              var waitTime = '${{ github.event.inputs.wait_time }}' || '';
+              var baselineNodeArgs = '${{ github.event.inputs.baseline_args }}' || '';
+              var featureNodeArgs = '${{ github.event.inputs.feature_args }}' || '';
 
               // Find PR for the selected branch
               const branch = '${{ github.ref_name }}';
@@ -173,26 +183,52 @@ jobs:
               actor = context.payload.comment.user.login;
 
               const body = context.payload.comment.body.trim();
-              const intArgs = new Set(['blocks', 'warmup', 'cores']);
+              const intArgs = new Set(['warmup', 'cores', 'blocks']);
               const refArgs = new Set(['baseline', 'feature']);
-              const boolArgs = new Set(['samply']);
-              const defaults = { blocks: '500', warmup: '100', baseline: '', feature: '', samply: 'false', cores: '0' };
+              const boolArgs = new Set(['samply', 'no-slack', 'big-blocks']);
+              const boolDefaultTrue = new Set(['reth_newPayload', 'abba', 'otlp']);
+              const durationArgs = new Set(['wait-time']);
+              const stringArgs = new Set(['baseline-args', 'feature-args']);
+              const defaults = { blocks: '500', warmup: '100', baseline: '', feature: '', samply: 'false', 'no-slack': 'false', 'big-blocks': 'false', cores: '0', reth_newPayload: 'true', abba: 'true', otlp: 'true', 'wait-time': '', 'baseline-args': '', 'feature-args': '' };
               const unknown = [];
               const invalid = [];
               const args = body.replace(/^(?:@decofe|derek) bench\s*/, '');
-              for (const part of args.split(/\s+/).filter(Boolean)) {
+              // Parse args, handling quoted values like key="value with spaces"
+              const parts = [];
+              const argRegex = /(\S+?="[^"]*"|\S+?='[^']*'|\S+)/g;
+              let m;
+              while ((m = argRegex.exec(args)) !== null) parts.push(m[1]);
+              for (const part of parts) {
                 const eq = part.indexOf('=');
                 if (eq === -1) {
                   if (boolArgs.has(part)) {
                     defaults[part] = 'true';
+                  } else if (boolDefaultTrue.has(part)) {
+                    defaults[part] = 'true';
                   } else {
                     unknown.push(part);
                   }
                   continue;
                 }
                 const key = part.slice(0, eq);
-                const value = part.slice(eq + 1);
-                if (intArgs.has(key)) {
+                let value = part.slice(eq + 1);
+                // Strip surrounding quotes
+                if ((value.startsWith('"') && value.endsWith('"')) || (value.startsWith("'") && value.endsWith("'"))) {
+                  value = value.slice(1, -1);
+                }
+                if (boolDefaultTrue.has(key)) {
+                  if (value === 'true' || value === 'false') {
+                    defaults[key] = value;
+                  } else {
+                    invalid.push(`\`${key}=${value}\` (must be true or false)`);
+                  }
+                } else if (durationArgs.has(key)) {
+                  if (/^\d+(ms|s|m)$/.test(value)) {
+                    defaults[key] = value;
+                  } else {
+                    invalid.push(`\`${key}=${value}\` (must be a duration like 500ms, 1s, 2m)`);
+                  }
+                } else if (intArgs.has(key)) {
                   if (!/^\d+$/.test(value)) {
                     invalid.push(`\`${key}=${value}\` (must be a positive integer)`);
                   } else {
@@ -204,6 +240,8 @@ jobs:
                   } else {
                     defaults[key] = value;
                   }
+                } else if (stringArgs.has(key)) {
+                  defaults[key] = value;
                 } else {
                   unknown.push(key);
                 }
@@ -212,7 +250,7 @@ jobs:
               if (unknown.length) errors.push(`Unknown argument(s): \`${unknown.join('`, `')}\``);
               if (invalid.length) errors.push(`Invalid value(s): ${invalid.join(', ')}`);
               if (errors.length) {
-                const msg = `❌ **Invalid bench command**\n\n${errors.join('\n')}\n\n**Usage:** \`@decofe bench [blocks=N] [warmup=N] [baseline=REF] [feature=REF] [samply] [cores=N]\``;
+                const msg = `❌ **Invalid bench command**\n\n${errors.join('\n')}\n\n**Usage:** \`@decofe bench [blocks=N] [big-blocks] [warmup=N] [baseline=REF] [feature=REF] [samply] [no-slack] [cores=N] [reth_newPayload=true|false] [abba=true|false] [otlp=true|false] [wait-time=DURATION] [baseline-args="..."] [feature-args="..."]\``;
                 await github.rest.issues.createComment({
                   owner: context.repo.owner,
                   repo: context.repo.repo,
@@ -227,7 +265,15 @@ jobs:
               baseline = defaults.baseline;
               feature = defaults.feature;
               samply = defaults.samply;
+              var noSlack = defaults['no-slack'];
               cores = defaults.cores;
+              bigBlocks = defaults['big-blocks'];
+              var rethNewPayload = defaults.reth_newPayload;
+              var abba = defaults.abba;
+              var otlp = defaults.otlp;
+              var waitTime = defaults['wait-time'];
+              var baselineNodeArgs = defaults['baseline-args'];
+              var featureNodeArgs = defaults['feature-args'];
             }
 
             // Resolve display names for baseline/feature
@@ -255,7 +301,15 @@ jobs:
             core.setOutput('baseline-name', baselineName);
             core.setOutput('feature-name', featureName);
             core.setOutput('samply', samply);
+            core.setOutput('no-slack', noSlack);
             core.setOutput('cores', cores);
+            core.setOutput('big-blocks', bigBlocks);
+            core.setOutput('reth-new-payload', rethNewPayload);
+            core.setOutput('wait-time', waitTime);
+            core.setOutput('baseline-args', baselineNodeArgs);
+            core.setOutput('feature-args', featureNodeArgs);
+            core.setOutput('abba', abba);
+            core.setOutput('otlp', otlp);
 
       - name: Acknowledge request
         id: ack
@@ -295,7 +349,6 @@ jobs:
                 });
                 allRuns.push(...r);
               }
-              // Only count runs that trigger reth-bench (not push-triggered codspeed runs)
               const benchRuns = allRuns.filter(r => r.event === 'issue_comment' || r.event === 'workflow_dispatch');
               const thisRun = benchRuns.find(r => r.id === context.runId);
               const thisCreatedAt = thisRun ? new Date(thisRun.created_at) : new Date();
@@ -314,10 +367,26 @@ jobs:
             const baseline = '${{ steps.args.outputs.baseline-name }}';
             const feature = '${{ steps.args.outputs.feature-name }}';
             const samply = '${{ steps.args.outputs.samply }}' === 'true';
+            const noSlack = '${{ steps.args.outputs.no-slack }}' === 'true';
+            const bigBlocks = '${{ steps.args.outputs.big-blocks }}' === 'true';
             const samplyNote = samply ? ', samply: `enabled`' : '';
+            const noSlackNote = noSlack ? ', no-slack' : '';
             const cores = '${{ steps.args.outputs.cores }}';
             const coresNote = cores && cores !== '0' ? `, cores: \`${cores}\`` : '';
-            const config = `**Config:** ${blocks} blocks, ${warmup} warmup blocks, baseline: \`${baseline}\`, feature: \`${feature}\`${samplyNote}${coresNote}`;
+            const rethNP = '${{ steps.args.outputs.reth-new-payload }}' !== 'false';
+            const rethNPNote = !rethNP ? ', reth_newPayload: `disabled`' : '';
+            const abbaEnabled = '${{ steps.args.outputs.abba }}' !== 'false';
+            const abbaNote = !abbaEnabled ? ', abba: `disabled`' : '';
+            const otlpEnabled = '${{ steps.args.outputs.otlp }}' !== 'false';
+            const otlpNote = !otlpEnabled ? ', otlp: `disabled`' : '';
+            const waitTimeVal = '${{ steps.args.outputs.wait-time }}';
+            const waitTimeNote = waitTimeVal ? `, wait-time: \`${waitTimeVal}\`` : '';
+            const baselineArgsVal = '${{ steps.args.outputs.baseline-args }}';
+            const baselineArgsNote = baselineArgsVal ? `, baseline-args: \`${baselineArgsVal}\`` : '';
+            const featureArgsVal = '${{ steps.args.outputs.feature-args }}';
+            const featureArgsNote = featureArgsVal ? `, feature-args: \`${featureArgsVal}\`` : '';
+            const blocksDesc = bigBlocks ? 'blocks: `big`' : `${blocks} blocks, ${warmup} warmup blocks`;
+            const config = `**Config:** ${blocksDesc}, baseline: \`${baseline}\`, feature: \`${feature}\`${samplyNote}${noSlackNote}${coresNote}${rethNPNote}${abbaNote}${otlpNote}${waitTimeNote}${baselineArgsNote}${featureArgsNote}`;
 
             const { data: comment } = await github.rest.issues.createComment({
               owner: context.repo.owner,
@@ -342,10 +411,26 @@ jobs:
             const baseline = '${{ steps.args.outputs.baseline-name }}';
             const feature = '${{ steps.args.outputs.feature-name }}';
             const samply = '${{ steps.args.outputs.samply }}' === 'true';
+            const noSlack = '${{ steps.args.outputs.no-slack }}' === 'true';
+            const bigBlocks = '${{ steps.args.outputs.big-blocks }}' === 'true';
             const samplyNote = samply ? ', samply: `enabled`' : '';
+            const noSlackNote = noSlack ? ', no-slack' : '';
             const cores = '${{ steps.args.outputs.cores }}';
             const coresNote = cores && cores !== '0' ? `, cores: \`${cores}\`` : '';
-            const config = `**Config:** ${blocks} blocks, ${warmup} warmup blocks, baseline: \`${baseline}\`, feature: \`${feature}\`${samplyNote}${coresNote}`;
+            const rethNP = '${{ steps.args.outputs.reth-new-payload }}' !== 'false';
+            const rethNPNote = !rethNP ? ', reth_newPayload: `disabled`' : '';
+            const abbaEnabled = '${{ steps.args.outputs.abba }}' !== 'false';
+            const abbaNote = !abbaEnabled ? ', abba: `disabled`' : '';
+            const otlpEnabled = '${{ steps.args.outputs.otlp }}' !== 'false';
+            const otlpNote = !otlpEnabled ? ', otlp: `disabled`' : '';
+            const waitTimeVal = '${{ steps.args.outputs.wait-time }}';
+            const waitTimeNote = waitTimeVal ? `, wait-time: \`${waitTimeVal}\`` : '';
+            const baselineArgsVal = '${{ steps.args.outputs.baseline-args }}';
+            const baselineArgsNote = baselineArgsVal ? `, baseline-args: \`${baselineArgsVal}\`` : '';
+            const featureArgsVal = '${{ steps.args.outputs.feature-args }}';
+            const featureArgsNote = featureArgsVal ? `, feature-args: \`${featureArgsVal}\`` : '';
+            const blocksDesc = bigBlocks ? 'blocks: `big`' : `${blocks} blocks, ${warmup} warmup blocks`;
+            const config = `**Config:** ${blocksDesc}, baseline: \`${baseline}\`, feature: \`${feature}\`${samplyNote}${noSlackNote}${coresNote}${rethNPNote}${abbaNote}${otlpNote}${waitTimeNote}${baselineArgsNote}${featureArgsNote}`;
             const runUrl = `${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;
 
             const numRunners = parseInt(process.env.BENCH_RUNNERS) || 1;
@@ -397,7 +482,7 @@ jobs:
   reth-bench:
     needs: reth-bench-ack
     name: reth-bench
-    runs-on: [self-hosted, Linux, X64]
+    runs-on: [self-hosted, Linux, X64, available]
     timeout-minutes: 120
     env:
       BENCH_RPC_URL: https://ethereum.reth.rs/rpc
@@ -409,8 +494,23 @@ jobs:
       BENCH_WARMUP_BLOCKS: ${{ needs.reth-bench-ack.outputs.warmup }}
       BENCH_SAMPLY: ${{ needs.reth-bench-ack.outputs.samply }}
       BENCH_CORES: ${{ needs.reth-bench-ack.outputs.cores }}
+      BENCH_BIG_BLOCKS: ${{ needs.reth-bench-ack.outputs.big-blocks }}
+      BENCH_RETH_NEW_PAYLOAD: ${{ needs.reth-bench-ack.outputs.reth-new-payload }}
+      BENCH_WAIT_TIME: ${{ needs.reth-bench-ack.outputs.wait-time }}
+      BENCH_BASELINE_ARGS: ${{ needs.reth-bench-ack.outputs.baseline-args }}
+      BENCH_FEATURE_ARGS: ${{ needs.reth-bench-ack.outputs.feature-args }}
+      BENCH_ABBA: ${{ needs.reth-bench-ack.outputs.abba }}
+      BENCH_OTLP: ${{ needs.reth-bench-ack.outputs.otlp }}
       BENCH_COMMENT_ID: ${{ needs.reth-bench-ack.outputs.comment-id }}
+      BENCH_NO_SLACK: ${{ needs.reth-bench-ack.outputs.no-slack }}
+      BENCH_NODE_BIN: ${{ needs.reth-bench-ack.outputs.big-blocks == 'true' && 'reth-bb' || 'reth' }}
+      BENCH_METRICS_ADDR: "127.0.0.1:9100"
+      BENCH_OTLP_TRACES_ENDPOINT: ${{ needs.reth-bench-ack.outputs.otlp != 'false' && secrets.BENCH_OTLP_TRACES_ENDPOINT || '' }}
+      BENCH_OTLP_LOGS_ENDPOINT: ${{ needs.reth-bench-ack.outputs.otlp != 'false' && secrets.BENCH_OTLP_LOGS_ENDPOINT || '' }}
     steps:
+      - name: Clean up previous bench-work
+        run: sudo rm -rf "$BENCH_WORK_DIR" 2>/dev/null || true
+
       - name: Resolve checkout ref
         id: checkout-ref
         uses: actions/github-script@v8
@@ -425,13 +525,11 @@ jobs:
               repo: context.repo.repo,
               pull_number: parseInt(process.env.BENCH_PR),
             });
-            // For closed/merged PRs, the merge ref doesn't exist — use head SHA
-            if (pr.state !== 'open') {
-              core.info(`PR #${process.env.BENCH_PR} is ${pr.state}, using head SHA ${pr.head.sha}`);
-              core.setOutput('ref', pr.head.sha);
-            } else {
-              core.setOutput('ref', `refs/pull/${process.env.BENCH_PR}/merge`);
-            }
+            // Always use head SHA — the merge ref (refs/pull/N/merge) may not
+            // exist if the PR has conflicts, was force-pushed, or was
+            // merged/closed between this step and checkout.
+            core.info(`PR #${process.env.BENCH_PR} (${pr.state}), using head SHA ${pr.head.sha}`);
+            core.setOutput('ref', pr.head.sha);
 
       - uses: actions/checkout@v6
         with:
@@ -459,10 +557,26 @@ jobs:
             const baseline = '${{ needs.reth-bench-ack.outputs.baseline-name }}';
             const feature = '${{ needs.reth-bench-ack.outputs.feature-name }}';
             const samply = process.env.BENCH_SAMPLY === 'true';
+            const noSlack = process.env.BENCH_NO_SLACK === 'true';
+            const bigBlocks = process.env.BENCH_BIG_BLOCKS === 'true';
             const samplyNote = samply ? ', samply: `enabled`' : '';
+            const noSlackNote = noSlack ? ', no-slack' : '';
             const cores = process.env.BENCH_CORES || '0';
             const coresNote = cores && cores !== '0' ? `, cores: \`${cores}\`` : '';
-            core.exportVariable('BENCH_CONFIG', `**Config:** ${blocks} blocks, ${warmup} warmup blocks, baseline: \`${baseline}\`, feature: \`${feature}\`${samplyNote}${coresNote}`);
+            const rethNP = (process.env.BENCH_RETH_NEW_PAYLOAD || 'true') !== 'false';
+            const rethNPNote = !rethNP ? ', reth_newPayload: `disabled`' : '';
+            const abbaEnabled = (process.env.BENCH_ABBA || 'true') !== 'false';
+            const abbaNote = !abbaEnabled ? ', abba: `disabled`' : '';
+            const otlpEnabled = (process.env.BENCH_OTLP || 'true') !== 'false';
+            const otlpNote = !otlpEnabled ? ', otlp: `disabled`' : '';
+            const waitTimeVal = process.env.BENCH_WAIT_TIME || '';
+            const waitTimeNote = waitTimeVal ? `, wait-time: \`${waitTimeVal}\`` : '';
+            const baselineArgsVal = process.env.BENCH_BASELINE_ARGS || '';
+            const baselineArgsNote = baselineArgsVal ? `, baseline-args: \`${baselineArgsVal}\`` : '';
+            const featureArgsVal = process.env.BENCH_FEATURE_ARGS || '';
+            const featureArgsNote = featureArgsVal ? `, feature-args: \`${featureArgsVal}\`` : '';
+            const blocksDesc = bigBlocks ? 'blocks: `big`' : `${blocks} blocks, ${warmup} warmup blocks`;
+            core.exportVariable('BENCH_CONFIG', `**Config:** ${blocksDesc}, baseline: \`${baseline}\`, feature: \`${feature}\`${samplyNote}${noSlackNote}${coresNote}${rethNPNote}${abbaNote}${otlpNote}${waitTimeNote}${baselineArgsNote}${featureArgsNote}`);
 
             const { buildBody } = require('./.github/scripts/bench-update-status.js');
             await github.rest.issues.updateComment({
@@ -605,14 +719,52 @@ jobs:
             core.setOutput('feature-ref', featureRef);
             core.setOutput('feature-name', featureName);
 
+      - name: Check big-blocks freshness
+        if: env.BENCH_BIG_BLOCKS == 'true'
+        id: big-blocks-check
+        run: |
+          set -euo pipefail
+          MC="mc --config-dir /home/ubuntu/.mc"
+          MANIFEST="minio/reth-snapshots/reth-1-minimal-stable-big-blocks.json"
+          HASH_FILE="$HOME/.reth-bench-big-blocks-hash"
+          echo "Fetching big-blocks manifest from $MANIFEST..."
+          BB_MANIFEST=$($MC cat "$MANIFEST" 2>/dev/null) || {
+            echo "::error::Failed to fetch big-blocks manifest from $MANIFEST"
+            exit 1
+          }
+          BASE_SNAPSHOT=$(echo "$BB_MANIFEST" | jq -r '.base_snapshot // empty')
+          if [ -z "$BASE_SNAPSHOT" ]; then
+            echo "::error::Big-blocks manifest missing base_snapshot field"
+            exit 1
+          fi
+          echo "Big-blocks base snapshot: $BASE_SNAPSHOT"
+          echo "BENCH_SNAPSHOT_NAME=${BASE_SNAPSHOT}" >> "$GITHUB_ENV"
+
+          REMOTE_HASH=$(echo "$BB_MANIFEST" | sha256sum | awk '{print $1}')
+          LOCAL_HASH=""
+          [ -f "$HASH_FILE" ] && LOCAL_HASH=$(cat "$HASH_FILE")
+          if [ "$REMOTE_HASH" = "$LOCAL_HASH" ]; then
+            echo "Big blocks up-to-date (hash: ${REMOTE_HASH:0:16}…)"
+            echo "needed=false" >> "$GITHUB_OUTPUT"
+          else
+            echo "Big blocks need update (local: ${LOCAL_HASH:+${LOCAL_HASH:0:16}…}${LOCAL_HASH:-<none>}, remote: ${REMOTE_HASH:0:16}…)"
+            echo "needed=true" >> "$GITHUB_OUTPUT"
+            echo "remote-hash=${REMOTE_HASH}" >> "$GITHUB_OUTPUT"
+          fi
+
       - name: Check if snapshot needs update
         id: snapshot-check
         run: |
-          if .github/scripts/bench-reth-snapshot.sh --check; then
-            echo "needed=false" >> "$GITHUB_OUTPUT"
-          else
-            echo "needed=true" >> "$GITHUB_OUTPUT"
-          fi
+          set +e
+          .github/scripts/bench-reth-snapshot.sh --check
+          rc=$?
+          set -e
+          case "$rc" in
+            0)  echo "needed=false" >> "$GITHUB_OUTPUT" ;;
+            10) echo "needed=true"  >> "$GITHUB_OUTPUT" ;;
+            *)  echo "::error::Snapshot check failed (exit $rc)"
+                exit "$rc" ;;
+          esac
 
       - name: Update status (snapshot needed)
         if: env.BENCH_COMMENT_ID && steps.snapshot-check.outputs.needed == 'true'
@@ -621,7 +773,7 @@ jobs:
           github-token: ${{ secrets.DEREK_PAT }}
           script: |
             const s = require('./.github/scripts/bench-update-status.js');
-            await s({github, context, status: 'Building binaries & downloading snapshot...'});
+            await s({github, context, status: 'Building binaries (snapshot update pending)...'});
 
       - name: Prepare source dirs
         run: |
@@ -641,12 +793,11 @@ jobs:
           fi
           git -C ../reth-feature checkout "$FEATURE_REF"
 
-      - name: Build binaries and download snapshot in parallel
+      - name: Build binaries
         id: build
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
           BENCH_REPO: ${{ github.repository }}
-          SNAPSHOT_NEEDED: ${{ steps.snapshot-check.outputs.needed }}
         run: |
           BASELINE_DIR="$(cd ../reth-baseline && pwd)"
           FEATURE_DIR="$(cd ../reth-feature && pwd)"
@@ -656,21 +807,23 @@ jobs:
           .github/scripts/bench-reth-build.sh feature "${FEATURE_DIR}" "${{ steps.refs.outputs.feature-ref }}" &
           PID_FEATURE=$!
 
-          PID_SNAPSHOT=
-          if [ "$SNAPSHOT_NEEDED" = "true" ]; then
-            .github/scripts/bench-reth-snapshot.sh &
-            PID_SNAPSHOT=$!
-          fi
-
           FAIL=0
           wait $PID_BASELINE || FAIL=1
           wait $PID_FEATURE || FAIL=1
-          [ -n "$PID_SNAPSHOT" ] && { wait $PID_SNAPSHOT || FAIL=1; }
           if [ $FAIL -ne 0 ]; then
-            echo "::error::One or more parallel tasks failed (builds / snapshot download)"
+            echo "::error::One or more build tasks failed"
             exit 1
           fi
 
+      - name: Download snapshot
+        id: snapshot-download
+        if: steps.snapshot-check.outputs.needed == 'true'
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          BENCH_REPO: ${{ github.repository }}
+          BENCH_RETH_BINARY: ${{ github.workspace }}/../reth-feature/target/profiling/${{ needs.reth-bench-ack.outputs.big-blocks == 'true' && 'reth-bb' || 'reth' }}
+        run: .github/scripts/bench-reth-snapshot.sh
+
       # System tuning for reproducible benchmarks
       - name: System setup
         run: |
@@ -722,6 +875,65 @@ jobs:
           rm -rf "$BENCH_WORK_DIR"
           mkdir -p "$BENCH_WORK_DIR"
 
+      - name: Download big blocks
+        if: env.BENCH_BIG_BLOCKS == 'true'
+        run: |
+          set -euo pipefail
+          BIG_BLOCKS_DIR="$HOME/.reth-bench-big-blocks"
+          echo "BENCH_BIG_BLOCKS_DIR=${BIG_BLOCKS_DIR}" >> "$GITHUB_ENV"
+          if [ "${{ steps.big-blocks-check.outputs.needed }}" = "false" ]; then
+            echo "Big blocks cached at $BIG_BLOCKS_DIR, skipping download"
+            echo "Payload files: $(find "$BIG_BLOCKS_DIR/payloads" -name '*.json' | wc -l)"
+            exit 0
+          fi
+          MC="mc --config-dir /home/ubuntu/.mc"
+          MANIFEST="minio/reth-snapshots/reth-1-minimal-stable-big-blocks.json"
+          rm -rf "$BIG_BLOCKS_DIR"; mkdir -p "$BIG_BLOCKS_DIR"
+
+          # Download and parse manifest
+          echo "Downloading manifest from $MANIFEST..."
+          $MC cat "$MANIFEST" > "$BIG_BLOCKS_DIR/manifest.json"
+          UPLOAD_PATH=$(jq -r '.upload_path' "$BIG_BLOCKS_DIR/manifest.json")
+          COUNT=$(jq -r '.count' "$BIG_BLOCKS_DIR/manifest.json")
+          TARGET_GAS=$(jq -r '.target_gas' "$BIG_BLOCKS_DIR/manifest.json")
+          echo "Manifest: count=$COUNT, target_gas=$TARGET_GAS, archive=$UPLOAD_PATH"
+
+          # Download and extract archive
+          ARCHIVE="minio/$UPLOAD_PATH"
+          echo "Downloading big blocks from $ARCHIVE..."
+          $MC cat "$ARCHIVE" | pzstd -d -p 6 | tar -xf - -C "$BIG_BLOCKS_DIR"
+          echo "Big blocks downloaded to $BIG_BLOCKS_DIR"
+
+          # Verify expected directory structure
+          if [ ! -d "$BIG_BLOCKS_DIR/payloads" ]; then
+            echo "::error::Big blocks archive missing expected payloads/ directory"
+            ls -laR "$BIG_BLOCKS_DIR"
+            exit 1
+          fi
+          echo "Payload files: $(find "$BIG_BLOCKS_DIR/payloads" -name '*.json' | wc -l)"
+          # Save manifest hash for freshness check on next run
+          echo "${{ steps.big-blocks-check.outputs.remote-hash }}" > "$HOME/.reth-bench-big-blocks-hash"
+
+      - name: Start metrics proxy
+        run: |
+          BENCH_ID="ci-${{ github.run_id }}"
+          BENCH_REFERENCE_EPOCH=$(date +%s)
+          echo "BENCH_ID=${BENCH_ID}" >> "$GITHUB_ENV"
+          echo "BENCH_REFERENCE_EPOCH=${BENCH_REFERENCE_EPOCH}" >> "$GITHUB_ENV"
+
+          LABELS_FILE="/tmp/bench-metrics-labels.json"
+          echo '{}' > "$LABELS_FILE"
+          echo "BENCH_LABELS_FILE=${LABELS_FILE}" >> "$GITHUB_ENV"
+
+          python3 .github/scripts/bench-metrics-proxy.py \
+            --labels "$LABELS_FILE" \
+            --upstream "http://${BENCH_METRICS_ADDR}/" \
+            --subnet 10.10.0.0/24 \
+            --port 9090 &
+          PROXY_PID=$!
+          echo "BENCH_METRICS_PROXY_PID=${PROXY_PID}" >> "$GITHUB_ENV"
+          echo "Metrics proxy started (PID $PROXY_PID)"
+
       - name: Update status (running benchmarks)
         if: success() && env.BENCH_COMMENT_ID
         uses: actions/github-script@v8
@@ -735,19 +947,64 @@ jobs:
       # thermal drift and cache warming.
       - name: "Run benchmark: baseline (1/2)"
         id: run-baseline-1
-        run: taskset -c 0 .github/scripts/bench-reth-run.sh baseline ../reth-baseline/target/profiling/reth "$BENCH_WORK_DIR/baseline-1"
+        env:
+          BASELINE_REF: ${{ steps.refs.outputs.baseline-ref }}
+          OTEL_RESOURCE_ATTRIBUTES: "benchmark_id=${{ env.BENCH_ID }},benchmark_run=baseline-1,run_type=baseline,git_ref=${{ steps.refs.outputs.baseline-ref }}"
+        run: |
+          cat > "$BENCH_LABELS_FILE" <<LABELS
+          {"benchmark_run":"baseline-1","run_type":"baseline","git_ref":"${BASELINE_REF}","bench_sha":"${BASELINE_REF}","benchmark_id":"${BENCH_ID}","run_start_epoch":"$(date +%s)","reference_epoch":"${BENCH_REFERENCE_EPOCH}"}
+          LABELS
+          taskset -c 0 .github/scripts/bench-reth-run.sh baseline "../reth-baseline/target/profiling/${BENCH_NODE_BIN}" "$BENCH_WORK_DIR/baseline-1"
 
       - name: "Run benchmark: feature (1/2)"
         id: run-feature-1
-        run: taskset -c 0 .github/scripts/bench-reth-run.sh feature ../reth-feature/target/profiling/reth "$BENCH_WORK_DIR/feature-1"
+        env:
+          FEATURE_REF: ${{ steps.refs.outputs.feature-ref }}
+          OTEL_RESOURCE_ATTRIBUTES: "benchmark_id=${{ env.BENCH_ID }},benchmark_run=feature-1,run_type=feature,git_ref=${{ steps.refs.outputs.feature-ref }}"
+        run: |
+          cat > "$BENCH_LABELS_FILE" <<LABELS
+          {"benchmark_run":"feature-1","run_type":"feature","git_ref":"${FEATURE_REF}","bench_sha":"${FEATURE_REF}","benchmark_id":"${BENCH_ID}","run_start_epoch":"$(date +%s)","reference_epoch":"${BENCH_REFERENCE_EPOCH}"}
+          LABELS
+          taskset -c 0 .github/scripts/bench-reth-run.sh feature "../reth-feature/target/profiling/${BENCH_NODE_BIN}" "$BENCH_WORK_DIR/feature-1"
 
       - name: "Run benchmark: feature (2/2)"
+        if: env.BENCH_ABBA != 'false'
         id: run-feature-2
-        run: taskset -c 0 .github/scripts/bench-reth-run.sh feature ../reth-feature/target/profiling/reth "$BENCH_WORK_DIR/feature-2"
+        env:
+          FEATURE_REF: ${{ steps.refs.outputs.feature-ref }}
+          OTEL_RESOURCE_ATTRIBUTES: "benchmark_id=${{ env.BENCH_ID }},benchmark_run=feature-2,run_type=feature,git_ref=${{ steps.refs.outputs.feature-ref }}"
+        run: |
+          cat > "$BENCH_LABELS_FILE" <<LABELS
+          {"benchmark_run":"feature-2","run_type":"feature","git_ref":"${FEATURE_REF}","bench_sha":"${FEATURE_REF}","benchmark_id":"${BENCH_ID}","run_start_epoch":"$(date +%s)","reference_epoch":"${BENCH_REFERENCE_EPOCH}"}
+          LABELS
+          taskset -c 0 .github/scripts/bench-reth-run.sh feature "../reth-feature/target/profiling/${BENCH_NODE_BIN}" "$BENCH_WORK_DIR/feature-2"
 
       - name: "Run benchmark: baseline (2/2)"
+        if: env.BENCH_ABBA != 'false'
         id: run-baseline-2
-        run: taskset -c 0 .github/scripts/bench-reth-run.sh baseline ../reth-baseline/target/profiling/reth "$BENCH_WORK_DIR/baseline-2"
+        env:
+          BASELINE_REF: ${{ steps.refs.outputs.baseline-ref }}
+          OTEL_RESOURCE_ATTRIBUTES: "benchmark_id=${{ env.BENCH_ID }},benchmark_run=baseline-2,run_type=baseline,git_ref=${{ steps.refs.outputs.baseline-ref }}"
+        run: |
+          LAST_RUN_START=$(date +%s)
+          echo "BENCH_LAST_RUN_START=${LAST_RUN_START}" >> "$GITHUB_ENV"
+          cat > "$BENCH_LABELS_FILE" <<LABELS
+          {"benchmark_run":"baseline-2","run_type":"baseline","git_ref":"${BASELINE_REF}","bench_sha":"${BASELINE_REF}","benchmark_id":"${BENCH_ID}","run_start_epoch":"${LAST_RUN_START}","reference_epoch":"${BENCH_REFERENCE_EPOCH}"}
+          LABELS
+          taskset -c 0 .github/scripts/bench-reth-run.sh baseline "../reth-baseline/target/profiling/${BENCH_NODE_BIN}" "$BENCH_WORK_DIR/baseline-2"
+
+      - name: Stop metrics proxy & generate Grafana URL
+        id: metrics
+        if: "!cancelled()"
+        run: |
+          kill "$BENCH_METRICS_PROXY_PID" 2>/dev/null || true
+
+          LAST_RUN_DURATION=$(( $(date +%s) - BENCH_LAST_RUN_START ))
+          FROM_MS=$(( BENCH_REFERENCE_EPOCH * 1000 ))
+          TO_MS=$(( (BENCH_REFERENCE_EPOCH + LAST_RUN_DURATION) * 1000 ))
+          GRAFANA_URL="https://tempoxyz.grafana.net/d/reth-bench-ghr/reth-bench-ghr?orgId=1&from=${FROM_MS}&to=${TO_MS}&timezone=browser&var-datasource=ef57fux92e9z4e&var-job=reth-bench&var-benchmark_id=${BENCH_ID}&var-benchmark_run=\$__all"
+          echo "grafana-url=${GRAFANA_URL}" >> "$GITHUB_OUTPUT"
+          echo "Grafana URL: ${GRAFANA_URL}"
 
       - name: Scan logs for errors
         if: "!cancelled()"
@@ -849,12 +1106,25 @@ jobs:
           SUMMARY_ARGS="$SUMMARY_ARGS --baseline-name ${BASELINE_NAME}"
           SUMMARY_ARGS="$SUMMARY_ARGS --feature-name ${FEATURE_NAME}"
           SUMMARY_ARGS="$SUMMARY_ARGS --feature-ref ${FEATURE_REF}"
-          SUMMARY_ARGS="$SUMMARY_ARGS --baseline-csv $BENCH_WORK_DIR/baseline-1/combined_latency.csv $BENCH_WORK_DIR/baseline-2/combined_latency.csv"
-          SUMMARY_ARGS="$SUMMARY_ARGS --feature-csv $BENCH_WORK_DIR/feature-1/combined_latency.csv $BENCH_WORK_DIR/feature-2/combined_latency.csv"
+          BASELINE_CSVS="$BENCH_WORK_DIR/baseline-1/combined_latency.csv"
+          FEATURE_CSVS="$BENCH_WORK_DIR/feature-1/combined_latency.csv"
+          if [ "${BENCH_ABBA:-true}" = "true" ]; then
+            BASELINE_CSVS="$BASELINE_CSVS $BENCH_WORK_DIR/baseline-2/combined_latency.csv"
+            FEATURE_CSVS="$FEATURE_CSVS $BENCH_WORK_DIR/feature-2/combined_latency.csv"
+          fi
+          SUMMARY_ARGS="$SUMMARY_ARGS --baseline-csv $BASELINE_CSVS"
+          SUMMARY_ARGS="$SUMMARY_ARGS --feature-csv $FEATURE_CSVS"
           SUMMARY_ARGS="$SUMMARY_ARGS --gas-csv $BENCH_WORK_DIR/feature-1/total_gas.csv"
           if [ "$BEHIND_BASELINE" -gt 0 ]; then
             SUMMARY_ARGS="$SUMMARY_ARGS --behind-baseline $BEHIND_BASELINE"
           fi
+          if [ "${BENCH_BIG_BLOCKS:-false}" = "true" ]; then
+            SUMMARY_ARGS="$SUMMARY_ARGS --big-blocks"
+          fi
+          GRAFANA_URL='${{ steps.metrics.outputs.grafana-url }}'
+          if [ -n "$GRAFANA_URL" ]; then
+            SUMMARY_ARGS="$SUMMARY_ARGS --grafana-url $GRAFANA_URL"
+          fi
           # shellcheck disable=SC2086
           python3 .github/scripts/bench-reth-summary.py $SUMMARY_ARGS
 
@@ -865,8 +1135,14 @@ jobs:
           FEATURE_NAME: ${{ steps.refs.outputs.feature-name }}
         run: |
           CHART_ARGS="--output-dir $BENCH_WORK_DIR/charts"
-          CHART_ARGS="$CHART_ARGS --feature $BENCH_WORK_DIR/feature-1/combined_latency.csv $BENCH_WORK_DIR/feature-2/combined_latency.csv"
-          CHART_ARGS="$CHART_ARGS --baseline $BENCH_WORK_DIR/baseline-1/combined_latency.csv $BENCH_WORK_DIR/baseline-2/combined_latency.csv"
+          FEATURE_CSVS="$BENCH_WORK_DIR/feature-1/combined_latency.csv"
+          BASELINE_CSVS="$BENCH_WORK_DIR/baseline-1/combined_latency.csv"
+          if [ "${BENCH_ABBA:-true}" = "true" ]; then
+            FEATURE_CSVS="$FEATURE_CSVS $BENCH_WORK_DIR/feature-2/combined_latency.csv"
+            BASELINE_CSVS="$BASELINE_CSVS $BENCH_WORK_DIR/baseline-2/combined_latency.csv"
+          fi
+          CHART_ARGS="$CHART_ARGS --feature $FEATURE_CSVS"
+          CHART_ARGS="$CHART_ARGS --baseline $BASELINE_CSVS"
           CHART_ARGS="$CHART_ARGS --baseline-name ${BASELINE_NAME}"
           CHART_ARGS="$CHART_ARGS --feature-name ${FEATURE_NAME}"
           # shellcheck disable=SC2086
@@ -874,7 +1150,7 @@ jobs:
 
       - name: Upload results
         if: "!cancelled()"
-        uses: actions/upload-artifact@v6
+        uses: actions/upload-artifact@v7
         with:
           name: bench-reth-results
           path: ${{ env.BENCH_WORK_DIR }}
@@ -906,7 +1182,7 @@ jobs:
           rm -rf "${TMP_DIR}"
 
       - name: Compare & comment
-        if: success()
+        if: success() && env.BENCH_COMMENT_ID
         uses: actions/github-script@v8
         with:
           github-token: ${{ secrets.DEREK_PAT }}
@@ -942,7 +1218,8 @@ jobs:
 
             // Samply profile links (URLs point directly to Firefox Profiler)
             if (process.env.BENCH_SAMPLY === 'true') {
-              const runs = ['baseline-1', 'feature-1', 'feature-2', 'baseline-2'];
+              const abba = (process.env.BENCH_ABBA || 'true') !== 'false';
+              const runs = abba ? ['baseline-1', 'feature-1', 'feature-2', 'baseline-2'] : ['baseline-1', 'feature-1'];
               const links = [];
               for (const run of runs) {
                 try {
@@ -957,6 +1234,12 @@ jobs:
               }
             }
 
+            // Grafana dashboard link
+            const grafanaUrl = '${{ steps.metrics.outputs.grafana-url }}';
+            if (grafanaUrl) {
+              comment += `\n\n### Grafana Dashboard\n\n[View real-time metrics](${grafanaUrl})\n`;
+            }
+
             // Node errors (panics / ERROR logs)
             try {
               const errors = fs.readFileSync(process.env.BENCH_WORK_DIR + '/errors.md', 'utf8');
@@ -976,14 +1259,25 @@ jobs:
                 comment_id: parseInt(ackCommentId),
                 body,
               });
-            } else {
-              // No PR — write results to job summary
-              await core.summary.addRaw(body).write();
             }
 
-      - name: Send Slack notification (success)
+      - name: Write job summary
         if: success()
         uses: actions/github-script@v8
+        with:
+          script: |
+            const jobSummary = require('./.github/scripts/bench-job-summary.js');
+            await jobSummary({
+              core,
+              context,
+              chartSha: '${{ steps.push-charts.outputs.sha }}',
+              grafanaUrl: '${{ steps.metrics.outputs.grafana-url }}',
+              runId: '${{ github.run_id }}',
+            });
+
+      - name: Send Slack notification (success)
+        if: success() && env.BENCH_NO_SLACK != 'true'
+        uses: actions/github-script@v8
         env:
           SLACK_BENCH_BOT_TOKEN: ${{ secrets.SLACK_BENCH_BOT_TOKEN }}
           SLACK_BENCH_CHANNEL: ${{ secrets.SLACK_BENCH_CHANNEL }}
@@ -998,12 +1292,14 @@ jobs:
         with:
           github-token: ${{ secrets.DEREK_PAT }}
           script: |
+            const abba = (process.env.BENCH_ABBA || 'true') !== 'false';
             const steps_status = [
-              ['building binaries${{ steps.snapshot-check.outputs.needed == 'true' && ' & downloading snapshot' || '' }}', '${{ steps.build.outcome }}'],
+              ['building binaries', '${{ steps.build.outcome }}'],
+              ['downloading snapshot', '${{ steps.snapshot-download.outcome }}'],
               ['running baseline benchmark (1/2)', '${{ steps.run-baseline-1.outcome }}'],
               ['running feature benchmark (1/2)', '${{ steps.run-feature-1.outcome }}'],
-              ['running feature benchmark (2/2)', '${{ steps.run-feature-2.outcome }}'],
-              ['running baseline benchmark (2/2)', '${{ steps.run-baseline-2.outcome }}'],
+              ...(abba ? [['running feature benchmark (2/2)', '${{ steps.run-feature-2.outcome }}']] : []),
+              ...(abba ? [['running baseline benchmark (2/2)', '${{ steps.run-baseline-2.outcome }}']] : []),
             ];
             const failed = steps_status.find(([, o]) => o === 'failure');
             const failedStep = failed ? failed[0] : 'unknown step';
@@ -1032,12 +1328,14 @@ jobs:
           SLACK_BENCH_CHANNEL: ${{ secrets.SLACK_BENCH_CHANNEL }}
         with:
           script: |
+            const abba = (process.env.BENCH_ABBA || 'true') !== 'false';
             const steps_status = [
-              ['building binaries${{ steps.snapshot-check.outputs.needed == 'true' && ' & downloading snapshot' || '' }}', '${{ steps.build.outcome }}'],
+              ['building binaries', '${{ steps.build.outcome }}'],
+              ['downloading snapshot', '${{ steps.snapshot-download.outcome }}'],
               ['running baseline benchmark (1/2)', '${{ steps.run-baseline-1.outcome }}'],
               ['running feature benchmark (1/2)', '${{ steps.run-feature-1.outcome }}'],
-              ['running feature benchmark (2/2)', '${{ steps.run-feature-2.outcome }}'],
-              ['running baseline benchmark (2/2)', '${{ steps.run-baseline-2.outcome }}'],
+              ...(abba ? [['running feature benchmark (2/2)', '${{ steps.run-feature-2.outcome }}']] : []),
+              ...(abba ? [['running baseline benchmark (2/2)', '${{ steps.run-baseline-2.outcome }}']] : []),
             ];
             const failed = steps_status.find(([, o]) => o === 'failure');
             const failedStep = failed ? failed[0] : 'unknown step';

.github/workflows/docker-test.yml

@@ -6,7 +6,7 @@ on:
       hive_target:
         required: true
         type: string
-        description: "Docker bake target to build (e.g. hive-stable, hive-edge)"
+        description: "Docker bake target to build (e.g. hive)"
       artifact_name:
         required: false
         type: string
@@ -76,7 +76,7 @@ jobs:
             *.dockerfile=Dockerfile
 
       - name: Upload reth image
-        uses: actions/upload-artifact@v6
+        uses: actions/upload-artifact@v7
         with:
           name: ${{ inputs.artifact_name }}
           path: ./artifacts

.github/workflows/docker.yml

@@ -28,12 +28,30 @@ on:
         required: false
         type: boolean
         default: false
+      pgo:
+        description: "Enable PGO profiling"
+        required: false
+        type: boolean
+        default: false
+      pgo_blocks:
+        description: "Number of blocks to execute for PGO profiling"
+        required: false
+        type: string
+        default: "20"
 
 jobs:
+  collect-pgo-profile:
+    if: github.repository == 'paradigmxyz/reth' && github.event_name == 'workflow_dispatch' && inputs.pgo
+    uses: ./.github/workflows/pgo-profile.yml
+    with:
+      pgo_blocks: ${{ inputs.pgo_blocks || '20' }}
+    secrets: inherit
+
   build:
-    if: github.repository == 'paradigmxyz/reth'
+    if: github.repository == 'paradigmxyz/reth' && !failure() && !cancelled()
     name: Build Docker images
     runs-on: ubuntu-24.04
+    needs: collect-pgo-profile
     permissions:
       packages: write
       contents: read
@@ -45,7 +63,7 @@ jobs:
         uses: depot/setup-action@v1
 
       - name: Log in to GHCR
-        uses: docker/login-action@v3
+        uses: docker/login-action@v4
         with:
           registry: ghcr.io
           username: ${{ github.actor }}
@@ -58,6 +76,30 @@ jobs:
           echo "describe=$(git describe --always --tags)" >> "$GITHUB_OUTPUT"
           echo "dirty=false" >> "$GITHUB_OUTPUT"
 
+      - name: Download pre-collected PGO profile
+        if: ${{ github.event_name == 'workflow_dispatch' && inputs.pgo }}
+        uses: actions/download-artifact@v7
+        with:
+          name: pgo-profdata
+          path: dist
+
+      - name: Configure PGO build args
+        id: pgo
+        run: |
+          if [[ "${{ github.event_name }}" == "workflow_dispatch" ]] && [[ "${{ inputs.pgo }}" == "true" ]]; then
+            if [ ! -f dist/merged.profdata ]; then
+              echo "::error::Expected dist/merged.profdata from collect-pgo-profile job"
+              exit 1
+            fi
+            echo "use_pgo_bolt=true" >> "$GITHUB_OUTPUT"
+            echo "pgo_profdata=dist/merged.profdata" >> "$GITHUB_OUTPUT"
+            echo "Using pre-collected PGO profile from collect-pgo-profile job"
+          else
+            echo "use_pgo_bolt=false" >> "$GITHUB_OUTPUT"
+            echo "pgo_profdata=" >> "$GITHUB_OUTPUT"
+            echo "PGO disabled"
+          fi
+
       - name: Determine build parameters
         id: params
         run: |
@@ -107,6 +149,9 @@ jobs:
           push: ${{ !(github.event_name == 'workflow_dispatch' && inputs.dry_run) }}
           set: |
             ${{ steps.params.outputs.ethereum_set }}
+            *.args.USE_PGO_BOLT=${{ steps.pgo.outputs.use_pgo_bolt }}
+            *.args.PGO_PROFDATA=${{ steps.pgo.outputs.pgo_profdata }}
+            *.args.STRIP_SYMBOLS=false
 
       - name: Verify image architectures
         env:
@@ -138,6 +183,8 @@ jobs:
             *Run:* <https://github.com/paradigmxyz/reth/actions/runs/${{ github.run_id }}|View logs>
 
             *Action required:* Re-run the workflow or investigate the build failure.
+
+            <@U0AAA8F0JEM> investigate and re-run if flaky
           SLACK_FOOTER: "paradigmxyz/reth · docker.yml"
           MSG_MINIMAL: true
           SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK_URL }}

.github/workflows/e2e.yml

@@ -63,6 +63,6 @@ jobs:
         run: |
           cargo nextest run \
             --no-fail-fast \
-            --locked --features "edge" \
+            --locked \
             -p reth-e2e-test-utils \
             -E 'binary(rocksdb)'

.github/workflows/hive.yml

@@ -15,18 +15,11 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  build-reth-stable:
+  build-reth:
     uses: ./.github/workflows/docker-test.yml
     with:
-      hive_target: hive-stable
-      artifact_name: "reth-stable"
-    secrets: inherit
-
-  build-reth-edge:
-    uses: ./.github/workflows/docker-test.yml
-    with:
-      hive_target: hive-edge
-      artifact_name: "reth-edge"
+      hive_target: hive
+      artifact_name: "reth"
     secrets: inherit
 
   prepare-hive:
@@ -75,7 +68,7 @@ jobs:
           chmod +x hive
 
       - name: Upload hive assets
-        uses: actions/upload-artifact@v6
+        uses: actions/upload-artifact@v7
         with:
           name: hive_assets
           path: ./hive_assets
@@ -84,7 +77,6 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        storage: [stable, edge]
         # ethereum/rpc to be deprecated:
         # https://github.com/ethereum/hive/pull/1117
         scenario:
@@ -184,10 +176,9 @@ jobs:
           - sim: ethereum/eels/consume-rlp
             limit: .*tests/paris.*
     needs:
-      - build-reth-stable
-      - build-reth-edge
+      - build-reth
       - prepare-hive
-    name: ${{ matrix.storage }} / ${{ matrix.scenario.sim }}${{ matrix.scenario.limit && format(' - {0}', matrix.scenario.limit) }}
+    name: ${{ matrix.scenario.sim }}${{ matrix.scenario.limit && format(' - {0}', matrix.scenario.limit) }}
     # Use larger runners for eels tests to avoid OOM runner crashes
     runs-on: ${{ github.repository == 'paradigmxyz/reth' && (contains(matrix.scenario.sim, 'eels') && 'depot-ubuntu-latest-8' || 'depot-ubuntu-latest-4') || 'ubuntu-latest' }}
     permissions:
@@ -198,15 +189,15 @@ jobs:
           fetch-depth: 0
 
       - name: Download hive assets
-        uses: actions/download-artifact@v7
+        uses: actions/download-artifact@v8
         with:
           name: hive_assets
           path: /tmp
 
       - name: Download reth image
-        uses: actions/download-artifact@v7
+        uses: actions/download-artifact@v8
         with:
-          name: reth-${{ matrix.storage }}
+          name: reth
           path: /tmp
 
       - name: Load Docker images

.github/workflows/integration.yml

@@ -22,7 +22,7 @@ concurrency:
 
 jobs:
   test:
-    name: test / ${{ matrix.network }} / ${{ matrix.storage }}
+    name: test / ${{ matrix.network }}
     if: github.event_name != 'schedule'
     runs-on: ${{ github.repository == 'paradigmxyz/reth' && 'depot-ubuntu-latest-4' || 'ubuntu-latest' }}
     env:
@@ -30,7 +30,6 @@ jobs:
     strategy:
       matrix:
         network: ["ethereum"]
-        storage: ["stable", "edge"]
     timeout-minutes: 60
     steps:
       - uses: actions/checkout@v6
@@ -47,7 +46,7 @@ jobs:
         run: |
           cargo nextest run \
             --no-fail-fast \
-            --locked --features "asm-keccak ${{ matrix.network }} ${{ matrix.storage == 'edge' && 'edge' || '' }}" \
+            --locked --features "asm-keccak ${{ matrix.network }}" \
             --workspace --exclude ef-tests \
             -E "kind(test) and not binary(e2e_testsuite)"
 

.github/workflows/kurtosis.yml

@@ -40,7 +40,7 @@ jobs:
           fetch-depth: 0
 
       - name: Download reth image
-        uses: actions/download-artifact@v7
+        uses: actions/download-artifact@v8
         with:
           name: artifacts
           path: /tmp

.github/workflows/lint.yml

@@ -237,31 +237,6 @@ jobs:
       - name: Ensure no arbitrary or proptest dependency on default build
         run: cargo tree --package reth -e=features,no-dev | grep -Eq "arbitrary|proptest" && exit 1 || exit 0
 
-  # Checks that selected crates can compile with power set of features
-  features:
-    name: features
-    runs-on: ${{ github.repository == 'paradigmxyz/reth' && 'depot-ubuntu-latest' || 'ubuntu-latest' }}
-    timeout-minutes: 30
-    steps:
-      - uses: actions/checkout@v6
-      - uses: rui314/setup-mold@v1
-      - uses: dtolnay/rust-toolchain@clippy
-      - uses: mozilla-actions/sccache-action@v0.0.9
-      - uses: Swatinem/rust-cache@v2
-        with:
-          cache-on-failure: true
-      - name: cargo install cargo-hack
-        uses: taiki-e/install-action@cargo-hack
-      - run: |
-          cargo hack check \
-            --package reth-codecs \
-            --package reth-primitives-traits \
-            --package reth-primitives \
-            --feature-powerset \
-            --depth 2
-        env:
-          RUSTFLAGS: -D warnings
-
   # Check crates correctly propagate features
   feature-propagation:
     runs-on: ${{ github.repository == 'paradigmxyz/reth' && 'depot-ubuntu-latest' || 'ubuntu-latest' }}
@@ -297,7 +272,6 @@ jobs:
       - typos
       - grafana
       - no-test-deps
-      - features
       - feature-propagation
       - deny
     timeout-minutes: 30

.github/workflows/pgo-profile.yml

@@ -0,0 +1,107 @@
+name: pgo-profile
+
+on:
+  workflow_call:
+    inputs:
+      pgo_blocks:
+        description: "Number of blocks to execute for PGO profiling"
+        required: false
+        type: string
+        default: "20"
+  workflow_dispatch:
+    inputs:
+      pgo_blocks:
+        description: "Number of blocks to execute for PGO profiling"
+        required: false
+        type: string
+        default: "20"
+
+jobs:
+  collect:
+    name: collect PGO profiles
+    runs-on: [self-hosted, Linux, X64]
+    timeout-minutes: 180
+    env:
+      SCHELK_MOUNT: /reth-bench
+      BENCH_RPC_URL: https://ethereum.reth.rs/rpc
+      RUSTC_WRAPPER: "sccache"
+    steps:
+      - uses: actions/checkout@v6
+        with:
+          submodules: true
+      - uses: rui314/setup-mold@v1
+      - uses: dtolnay/rust-toolchain@stable
+        with:
+          target: x86_64-unknown-linux-gnu
+      - uses: mozilla-actions/sccache-action@v0.0.9
+        continue-on-error: true
+      - uses: Swatinem/rust-cache@v2
+        with:
+          cache-on-failure: true
+
+      - name: Install dependencies
+        run: |
+          sudo apt-get update -qq
+          sudo apt-get install -y --no-install-recommends \
+            dmsetup lsb-release wget linux-tools-"$(uname -r)" || \
+            sudo apt-get install -y --no-install-recommends linux-tools-generic
+
+      - name: Download snapshot if needed
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          BENCH_REPO: ${{ github.repository }}
+        run: |
+          if ! .github/scripts/bench-reth-snapshot.sh --check; then
+            echo "Snapshot outdated or missing, downloading..."
+            .github/scripts/bench-reth-snapshot.sh
+          fi
+
+      - name: Mount snapshot
+        run: |
+          sudo pkill -9 reth || true
+          sleep 1
+          if mountpoint -q "$SCHELK_MOUNT"; then
+            sudo umount -l "$SCHELK_MOUNT" || true
+            sudo schelk recover -y || true
+          fi
+          sudo schelk mount -y
+          sync
+          sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'
+
+      - name: Collect PGO profile
+        run: |
+          DATADIR="$SCHELK_MOUNT/datadir" \
+          RPC_URL="$BENCH_RPC_URL" \
+          PGO_BLOCKS="${{ inputs.pgo_blocks || '20' }}" \
+          BOLT_BLOCKS="${{ inputs.pgo_blocks || '20' }}" \
+          COLLECT_PGO_ONLY=true \
+          SKIP_BOLT=true \
+          PROFILE=maxperf-symbols \
+          FEATURES="jemalloc,asm-keccak,min-debug-logs" \
+          TARGET=x86_64-unknown-linux-gnu \
+          EXTRA_RUSTFLAGS="-C target-cpu=x86-64-v3 -C target-feature=+pclmulqdq" \
+            .github/scripts/build_pgo_bolt.sh
+
+      - name: Show PGO profile stats
+        run: |
+          LLVM_PROFDATA=$(find "$(rustc --print sysroot)" -name llvm-profdata -type f | head -1)
+          if [ -z "$LLVM_PROFDATA" ]; then
+            echo "::error::llvm-profdata not found in rust toolchain"
+            exit 1
+          fi
+          "$LLVM_PROFDATA" show --detailed-summary --topn=20 target/pgo-profiles/merged.profdata
+
+      - name: Upload PGO profile
+        uses: actions/upload-artifact@v7
+        with:
+          name: pgo-profdata
+          path: target/pgo-profiles/merged.profdata
+          retention-days: 1
+
+      - name: Recover snapshot
+        if: always()
+        run: |
+          if mountpoint -q "$SCHELK_MOUNT"; then
+            sudo umount -l "$SCHELK_MOUNT" || true
+            sudo schelk recover -y || true
+          fi

.github/workflows/pr-audit.yml

@@ -0,0 +1,30 @@
+name: Pull request audit
+
+on:
+  pull_request:
+    types: [labeled]
+
+jobs:
+  publish:
+    runs-on: ubuntu-latest
+    if: github.event.label.name == 'cyclops'
+    steps:
+      - name: Publish event
+        run: |
+          set -euo pipefail
+
+          echo "${{ secrets.EVENTS_KEY }}" > ${{ runner.temp }}/key
+          echo "${{ secrets.EVENTS_CERT }}" > ${{ runner.temp }}/cert
+
+          curl -sf -o /dev/null -X POST ${{ secrets.EVENTS_ARGS }} \
+            -H "Content-Type: application/json" \
+            --key ${{ runner.temp }}/key \
+            --cert ${{ runner.temp }}/cert \
+            -d '{
+              "repository": "${{ github.repository }}",
+              "event": "pr_audit",
+              "data": {
+                "pr_number": ${{ github.event.pull_request.number }},
+                "sha": "${{ github.event.pull_request.head.sha }}"
+              }
+            }'

.github/workflows/release-reproducible.yml

@@ -52,7 +52,7 @@ jobs:
         uses: docker/setup-buildx-action@v3
 
       - name: Log in to GitHub Container Registry
-        uses: docker/login-action@v3
+        uses: docker/login-action@v4
         with:
           registry: ghcr.io
           username: ${{ github.actor }}

.github/workflows/release.yml

@@ -13,6 +13,14 @@ on:
         description: "Enable dry run mode (builds artifacts but skips uploads and release creation)"
         type: boolean
         default: false
+      pgo:
+        description: "Enable PGO profiling"
+        type: boolean
+        default: false
+      pgo_blocks:
+        description: "Number of blocks to execute for PGO profiling on self-hosted runner"
+        type: string
+        default: "20"
 
 env:
   REPO_NAME: ${{ github.repository_owner }}/reth
@@ -37,7 +45,7 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Extract version
-        run: echo "VERSION=${GITHUB_REF_NAME}" >> $GITHUB_OUTPUT
+        run: echo "VERSION=${GITHUB_REF_NAME//\//-}" >> $GITHUB_OUTPUT
         id: extract_version
     outputs:
       VERSION: ${{ steps.extract_version.outputs.VERSION }}
@@ -69,22 +77,22 @@ jobs:
       fail-fast: true
       matrix:
         configs:
-          - target: x86_64-unknown-linux-gnu
-            os: ubuntu-24.04
-            profile: maxperf
-            allow_fail: false
           - target: aarch64-unknown-linux-gnu
             os: ubuntu-24.04-arm
             profile: maxperf
             allow_fail: false
+            rustflags: ""
+            native: true
           - target: x86_64-apple-darwin
             os: macos-14
             profile: maxperf
             allow_fail: false
+            rustflags: "-C target-cpu=x86-64-v3 -C target-feature=+pclmulqdq"
           - target: aarch64-apple-darwin
             os: macos-14
             profile: maxperf
             allow_fail: false
+            rustflags: ""
         build:
           - command: build
             binary: reth
@@ -96,6 +104,7 @@ jobs:
           target: ${{ matrix.configs.target }}
       - uses: mozilla-actions/sccache-action@v0.0.9
       - name: Install cross main
+        if: ${{ !matrix.configs.native }}
         id: cross_main
         run: |
           cargo install cross --locked --git https://github.com/cross-rs/cross
@@ -110,7 +119,12 @@ jobs:
           echo "MACOSX_DEPLOYMENT_TARGET=$(xcrun -sdk macosx --show-sdk-platform-version)" >> $GITHUB_ENV
 
       - name: Build Reth
-        run: make PROFILE=${{ matrix.configs.profile }} ${{ matrix.build.command }}-${{ matrix.configs.target }}
+        run: |
+          if [ "${{ matrix.configs.native }}" = "true" ]; then
+            make PROFILE=${{ matrix.configs.profile }} EXTRA_RUSTFLAGS="${{ matrix.configs.rustflags }}" ${{ matrix.build.command }}-native-${{ matrix.configs.target }}
+          else
+            make PROFILE=${{ matrix.configs.profile }} EXTRA_RUSTFLAGS="${{ matrix.configs.rustflags }}" ${{ matrix.build.command }}-${{ matrix.configs.target }}
+          fi
       - name: Move binary
         run: |
           mkdir artifacts
@@ -131,23 +145,105 @@ jobs:
 
       - name: Upload artifact
         if: ${{ github.event.inputs.dry_run != 'true' }}
-        uses: actions/upload-artifact@v6
+        uses: actions/upload-artifact@v7
         with:
           name: ${{ matrix.build.binary }}-${{ needs.extract-version.outputs.VERSION }}-${{ matrix.configs.target }}.tar.gz
           path: ${{ matrix.build.binary }}-${{ needs.extract-version.outputs.VERSION }}-${{ matrix.configs.target }}.tar.gz
 
       - name: Upload signature
         if: ${{ github.event.inputs.dry_run != 'true' }}
-        uses: actions/upload-artifact@v6
+        uses: actions/upload-artifact@v7
         with:
           name: ${{ matrix.build.binary }}-${{ needs.extract-version.outputs.VERSION }}-${{ matrix.configs.target }}.tar.gz.asc
           path: ${{ matrix.build.binary }}-${{ needs.extract-version.outputs.VERSION }}-${{ matrix.configs.target }}.tar.gz.asc
 
+  collect-pgo-profile:
+    if: github.event_name == 'workflow_dispatch' && inputs.pgo
+    uses: ./.github/workflows/pgo-profile.yml
+    with:
+      pgo_blocks: ${{ inputs.pgo_blocks || '20' }}
+    secrets: inherit
+
+  build-pgo:
+    if: github.event_name == 'workflow_dispatch' && inputs.pgo
+    name: build release (x86_64-linux PGO+BOLT)
+    runs-on: [self-hosted, Linux, X64]
+    needs: [extract-version, collect-pgo-profile]
+    timeout-minutes: 120
+    env:
+      RUSTC_WRAPPER: "sccache"
+    steps:
+      - uses: actions/checkout@v6
+        with:
+          submodules: true
+      - uses: rui314/setup-mold@v1
+      - uses: dtolnay/rust-toolchain@stable
+        with:
+          target: x86_64-unknown-linux-gnu
+      - uses: mozilla-actions/sccache-action@v0.0.9
+        continue-on-error: true
+      - uses: Swatinem/rust-cache@v2
+        with:
+          cache-on-failure: true
+
+      - name: Download pre-collected PGO profile
+        uses: actions/download-artifact@v7
+        with:
+          name: pgo-profdata
+          path: dist
+
+      - name: Verify PGO profile artifact
+        run: |
+          test -f dist/merged.profdata
+          ls -lh dist/merged.profdata
+
+      - name: Build Reth with PGO+BOLT
+        run: |
+          SKIP_BOLT=true \
+          PGO_PROFDATA="$PWD/dist/merged.profdata" \
+          PROFILE=maxperf-symbols \
+          FEATURES="jemalloc,asm-keccak,min-debug-logs" \
+          TARGET=x86_64-unknown-linux-gnu \
+          EXTRA_RUSTFLAGS="-C target-cpu=x86-64-v3 -C target-feature=+pclmulqdq" \
+            .github/scripts/build_pgo_bolt.sh
+
+      - name: Move binary
+        run: |
+          mkdir artifacts
+          mv target/maxperf-symbols/reth ./artifacts
+
+      - name: Configure GPG and create artifacts
+        env:
+          GPG_SIGNING_KEY: ${{ secrets.GPG_SIGNING_KEY }}
+          GPG_PASSPHRASE: ${{ secrets.GPG_PASSPHRASE }}
+        run: |
+          export GPG_TTY=$(tty)
+          echo -n "$GPG_SIGNING_KEY" | base64 --decode | gpg --batch --import
+          cd artifacts
+          tar -czf reth-${{ needs.extract-version.outputs.VERSION }}-x86_64-unknown-linux-gnu.tar.gz reth*
+          echo "$GPG_PASSPHRASE" | gpg --passphrase-fd 0 --pinentry-mode loopback --batch -ab reth-${{ needs.extract-version.outputs.VERSION }}-x86_64-unknown-linux-gnu.tar.gz
+          mv *tar.gz* ..
+        shell: bash
+
+      - name: Upload artifact
+        if: ${{ github.event.inputs.dry_run != 'true' }}
+        uses: actions/upload-artifact@v7
+        with:
+          name: reth-${{ needs.extract-version.outputs.VERSION }}-x86_64-unknown-linux-gnu.tar.gz
+          path: reth-${{ needs.extract-version.outputs.VERSION }}-x86_64-unknown-linux-gnu.tar.gz
+
+      - name: Upload signature
+        if: ${{ github.event.inputs.dry_run != 'true' }}
+        uses: actions/upload-artifact@v7
+        with:
+          name: reth-${{ needs.extract-version.outputs.VERSION }}-x86_64-unknown-linux-gnu.tar.gz.asc
+          path: reth-${{ needs.extract-version.outputs.VERSION }}-x86_64-unknown-linux-gnu.tar.gz.asc
+
   draft-release:
     name: draft release
     runs-on: ubuntu-latest
-    needs: [build, extract-version]
-    if: ${{ github.event.inputs.dry_run != 'true' }}
+    needs: [build, build-pgo, extract-version]
+    if: ${{ !failure() && !cancelled() && github.event.inputs.dry_run != 'true' }}
     env:
       VERSION: ${{ needs.extract-version.outputs.VERSION }}
     permissions:
@@ -160,7 +256,7 @@ jobs:
         with:
           fetch-depth: 0
       - name: Download artifacts
-        uses: actions/download-artifact@v7
+        uses: actions/download-artifact@v8
       - name: Generate full changelog
         id: changelog
         run: |

.github/workflows/reproducible-build.yml

@@ -43,7 +43,7 @@ jobs:
           echo "Binaries SHA256 on ${{ matrix.machine }}: $(cat checksum.sha256)"
 
       - name: Upload the hash
-        uses: actions/upload-artifact@v6
+        uses: actions/upload-artifact@v7
         with:
           name: checksum-${{ matrix.machine }}
           path: |
@@ -56,12 +56,12 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Download artifacts from machine-1
-        uses: actions/download-artifact@v7
+        uses: actions/download-artifact@v8
         with:
           name: checksum-machine-1
           path: machine-1/
       - name: Download artifacts from machine-2
-        uses: actions/download-artifact@v7
+        uses: actions/download-artifact@v8
         with:
           name: checksum-machine-2
           path: machine-2/

.github/workflows/stage.yml

@@ -51,15 +51,12 @@ jobs:
       - name: Run execution stage
         run: |
           reth stage run execution --from ${{ env.FROM_BLOCK }} --to ${{ env.TO_BLOCK }} --commit --checkpoints
-      - name: Run account-hashing stage
-        run: |
-          reth stage run account-hashing --from ${{ env.FROM_BLOCK }} --to ${{ env.TO_BLOCK }} --commit --checkpoints
-      - name: Run storage hashing stage
-        run: |
-          reth stage run storage-hashing --from ${{ env.FROM_BLOCK }} --to ${{ env.TO_BLOCK }} --commit --checkpoints
-      - name: Run hashing stage
-        run: |
-          reth stage run hashing --from ${{ env.FROM_BLOCK }} --to ${{ env.TO_BLOCK }} --commit --checkpoints
+      # NOTE: account-hashing, storage-hashing, and hashing stages are omitted.
+      # With storage v2 (now default), these stages are no-ops because the
+      # execution stage writes directly to HashedAccounts/HashedStorages.
+      # Running them here is harmful: `stage run` unwinds before executing,
+      # and the unwind reverts the hashed state that execution wrote, but
+      # the no-op execute never restores it — causing merkle to fail.
       - name: Run merkle stage
         run: |
           reth stage run merkle --from ${{ env.FROM_BLOCK }} --to ${{ env.TO_BLOCK }} --commit --checkpoints

.github/workflows/unit.yml

@@ -19,15 +19,13 @@ concurrency:
 
 jobs:
   test:
-    name: test / ${{ matrix.type }} / ${{ matrix.storage }}
+    name: test / ${{ matrix.type }}
     runs-on: ${{ github.repository == 'paradigmxyz/reth' && 'depot-ubuntu-latest-4' || 'ubuntu-latest' }}
     env:
       RUST_BACKTRACE: 1
-      EDGE_FEATURES: ${{ matrix.storage == 'edge' && 'edge' || '' }}
     strategy:
       matrix:
         type: [ethereum]
-        storage: [stable, edge]
         include:
           - type: ethereum
             features: asm-keccak ethereum
@@ -50,14 +48,14 @@ jobs:
         run: |
           cargo nextest run \
             --no-fail-fast \
-            --features "${{ matrix.features }} $EDGE_FEATURES" --locked \
+            --features "${{ matrix.features }}" --locked \
             ${{ matrix.exclude_args }} --workspace \
             --exclude ef-tests --no-tests=warn \
             -E "!kind(test) and not binary(e2e_testsuite)"
 
   state:
     name: Ethereum state tests
-    runs-on: ${{ github.repository == 'paradigmxyz/reth' && 'depot-ubuntu-latest-4' || 'ubuntu-latest' }}
+    runs-on: ${{ github.repository == 'paradigmxyz/reth' && 'depot-ubuntu-latest-8' || 'ubuntu-latest' }}
     env:
       RUST_LOG: info,sync=error
       RUST_BACKTRACE: 1

AGENTS.md

@@ -0,0 +1,549 @@
+# Reth Development Guide for AI Agents
+
+This guide provides comprehensive instructions for AI agents working on the Reth codebase. It covers the architecture, development workflows, and critical guidelines for effective contributions.
+
+## Project Overview
+
+Reth is a high-performance Ethereum execution client written in Rust, focusing on modularity, performance, and contributor-friendliness. The codebase is organized into well-defined crates with clear boundaries and responsibilities.
+
+## Architecture Overview
+
+### Core Components
+
+1. **Consensus (`crates/consensus/`)**: Validates blocks according to Ethereum consensus rules
+2. **Storage (`crates/storage/`)**: Hybrid database using MDBX + static files for optimal performance
+3. **Networking (`crates/net/`)**: P2P networking stack with discovery, sync, and transaction propagation
+4. **RPC (`crates/rpc/`)**: JSON-RPC server supporting all standard Ethereum APIs
+5. **Execution (`crates/evm/`, `crates/ethereum/`)**: Transaction execution and state transitions
+6. **Pipeline (`crates/stages/`)**: Staged sync architecture for blockchain synchronization
+7. **Trie (`crates/trie/`)**: Merkle Patricia Trie implementation with parallel state root computation
+8. **Node Builder (`crates/node/`)**: High-level node orchestration and configuration
+9. **The Consensus Engine (`crates/engine/`)**: Handles processing blocks received from the consensus layer with the Engine API (newPayload, forkchoiceUpdated)
+
+### Key Design Principles
+
+- **Modularity**: Each crate can be used as a standalone library
+- **Performance**: Extensive use of parallelism, memory-mapped I/O, and optimized data structures
+- **Extensibility**: Traits and generic types allow for different chain implementations
+- **Type Safety**: Strong typing throughout with minimal use of dynamic dispatch
+
+## Development Workflow
+
+### Code Style and Standards
+
+1. **Formatting**: Always use nightly rustfmt
+   ```bash
+   cargo +nightly fmt --all
+   ```
+
+2. **Linting**: Run clippy with all features
+   ```bash
+   cargo +nightly clippy --workspace --lib --examples --tests --benches --all-features 
+   ```
+
+3. **Testing**: Use nextest for faster test execution
+   ```bash
+   cargo nextest run --workspace
+   ```
+
+### Common Contribution Types
+
+Based on actual recent PRs, here are typical contribution patterns:
+
+#### 1. Small Bug Fixes (1-10 lines)
+Real example: Fixing beacon block root handling ([#16767](https://github.com/paradigmxyz/reth/pull/16767))
+```rust
+// Changed a single line to fix logic error
+- parent_beacon_block_root: parent.parent_beacon_block_root(),
++ parent_beacon_block_root: parent.parent_beacon_block_root().map(|_| B256::ZERO),
+```
+
+#### 2. Integration with Upstream Changes
+Real example: Integrating revm updates ([#16752](https://github.com/paradigmxyz/reth/pull/16752))
+```rust
+// Update code to use new APIs from dependencies
+- if self.fork_tracker.is_shanghai_activated() {
+-     if let Err(err) = transaction.ensure_max_init_code_size(MAX_INIT_CODE_BYTE_SIZE) {
++ if let Some(init_code_size_limit) = self.fork_tracker.max_initcode_size() {
++     if let Err(err) = transaction.ensure_max_init_code_size(init_code_size_limit) {
+```
+
+#### 3. Adding Comprehensive Tests
+Real example: ETH69 protocol tests ([#16759](https://github.com/paradigmxyz/reth/pull/16759))
+```rust
+#[tokio::test(flavor = "multi_thread")]
+async fn test_eth69_peers_can_connect() {
+    // Create test network with specific protocol versions
+    let p0 = PeerConfig::with_protocols(NoopProvider::default(), Some(EthVersion::Eth69.into()));
+    // Test connection and version negotiation
+}
+```
+
+#### 4. Making Components Generic
+Real example: Making EthEvmConfig generic over chainspec ([#16758](https://github.com/paradigmxyz/reth/pull/16758))
+```rust
+// Before: Hardcoded to ChainSpec
+- pub struct EthEvmConfig<EvmFactory = EthEvmFactory> {
+-     pub executor_factory: EthBlockExecutorFactory<RethReceiptBuilder, Arc<ChainSpec>, EvmFactory>,
+
+// After: Generic over any chain spec type
++ pub struct EthEvmConfig<C = ChainSpec, EvmFactory = EthEvmFactory>
++ where
++     C: EthereumHardforks,
++ {
++     pub executor_factory: EthBlockExecutorFactory<RethReceiptBuilder, Arc<C>, EvmFactory>,
+```
+
+#### 5. Resource Management Improvements
+Real example: ETL directory cleanup ([#16770](https://github.com/paradigmxyz/reth/pull/16770))
+```rust
+// Add cleanup logic on startup
++ if let Err(err) = fs::remove_dir_all(&etl_path) {
++     warn!(target: "reth::cli", ?etl_path, %err, "Failed to remove ETL path on launch");
++ }
+```
+
+#### 6. Feature Additions
+Real example: Sharded mempool support ([#16756](https://github.com/paradigmxyz/reth/pull/16756))
+```rust
+// Add new filtering policies for transaction announcements
+pub struct ShardedMempoolAnnouncementFilter<T> {
+    pub inner: T,
+    pub shard_bits: u8,
+    pub node_id: Option<B256>,
+}
+```
+
+### Testing Guidelines
+
+1. **Unit Tests**: Test individual functions and components
+2. **Integration Tests**: Test interactions between components
+3. **Benchmarks**: For performance-critical code
+4. **Fuzz Tests**: For parsing and serialization code
+5. **Property Tests**: For checking component correctness on a wide variety of inputs
+
+Example test structure:
+```rust
+#[cfg(test)]
+mod tests {
+    use super::*;
+    
+    #[test]
+    fn test_component_behavior() {
+        // Arrange
+        let component = Component::new();
+        
+        // Act
+        let result = component.operation();
+        
+        // Assert
+        assert_eq!(result, expected);
+    }
+}
+```
+
+### Performance Considerations
+
+1. **Avoid Allocations in Hot Paths**: Use references and borrowing
+2. **Parallel Processing**: Use rayon for CPU-bound parallel work
+3. **Async/Await**: Use tokio for I/O-bound operations
+4. **File Operations**: Use `reth_fs_util` instead of `std::fs` for better error handling
+
+### Common Pitfalls
+
+1. **Don't Block Async Tasks**: Use `spawn_blocking` for CPU-intensive work or work with lots of blocking I/O
+2. **Handle Errors Properly**: Use `?` operator and proper error types
+
+### What to Avoid
+
+Based on PR patterns, avoid:
+
+1. **Large, sweeping changes**: Keep PRs focused and reviewable
+2. **Mixing unrelated changes**: One logical change per PR
+3. **Ignoring CI failures**: All checks must pass
+4. **Incomplete implementations**: Finish features before submitting
+5. **Modifying libmdbx sources**: Never modify files in `crates/storage/libmdbx-rs/mdbx-sys/libmdbx/` - this is vendored third-party code
+
+### CI Requirements
+
+Before submitting changes, ensure:
+
+1. **Format Check**: `cargo +nightly fmt --all --check`
+2. **Clippy**: No warnings
+3. **Tests Pass**: All unit and integration tests
+4. **Documentation**: Update relevant docs and add doc comments with `cargo docs --document-private-items`
+5. **CLI Docs** (if CLI changed): Run `make update-book-cli` (see below)
+6. **Commit Messages**: Follow conventional format (feat:, fix:, chore:, etc.)
+
+### CLI Reference Docs (`book` CI Job)
+
+The CLI reference pages under `docs/vocs/docs/pages/cli/` are **auto-generated** from the `reth` binary's `--help` output. **Do not edit these files manually** — any hand edits will be overwritten and CI will fail regardless.
+
+When you add, remove, or modify CLI commands, subcommands, or flags, regenerate the CLI docs by running:
+
+```bash
+make update-book-cli
+```
+
+This builds `reth` in debug mode and runs `docs/cli/update.sh` to regenerate all CLI pages. Commit the resulting changes.
+
+The `book` CI job (`.github/workflows/lint.yml`) enforces this by regenerating the docs and running `git diff --exit-code`. If the committed docs don't match the generated output, CI fails. Manually editing these pages is never productive — always use `make update-book-cli`.
+
+### Opening PRs against <https://github.com/paradigmxyz/reth>
+
+#### Titles
+
+Use [Conventional Commits](https://www.conventionalcommits.org/) with an optional scope:
+
+```
+<type>(<scope>): <short description>
+```
+
+**Types**: `feat`, `fix`, `perf`, `refactor`, `docs`, `test`, `chore`
+
+**Scope** (optional): crate or area, e.g. `evm`, `trie`, `rpc`, `engine`, `net`
+
+Examples:
+- `fix(rpc): correct gas estimation for ERC-20 transfers`
+- `perf: batch trie updates to reduce cursor overhead`
+- `feat(engine): add new_payload_interval metric`
+
+#### Descriptions
+
+Keep it short. Say what changed and why — nothing more.
+
+**Do:**
+- Write 1–3 sentences summarizing the change
+- Explain _why_ if the diff doesn't make it obvious
+- Link related issues or EIPs
+- Include benchmark numbers for perf changes
+
+**Don't:**
+- List every file changed — that's what the diff is for
+- Repeat the title in the body
+- Add "Files changed" or "Changes" sections
+- Write walls of text that go stale when the diff is updated
+- Use filler like "This PR introduces...", "comprehensive", "robust", "enhance", "leverage"
+
+**Template:**
+
+```
+Closes #<issue>
+
+<what changed, 1-3 sentences>
+
+<why, if not obvious from the diff>
+```
+
+**Good example:**
+
+```
+Closes #16800
+
+Adds fallback for external IP resolution so node startup doesn't fail
+when STUN is unreachable. Falls back to the configured default.
+```
+
+**Bad example:**
+
+```
+## Summary
+This PR introduces comprehensive improvements to the IP resolution system.
+
+## Changes
+- Modified `crates/net/discv4/src/lib.rs` to add fallback
+- Modified `crates/net/discv4/src/config.rs` to add default IP
+- Added tests in `crates/net/discv4/src/tests/ip.rs`
+
+## Files Changed
+- crates/net/discv4/src/lib.rs
+- crates/net/discv4/src/config.rs
+- crates/net/discv4/src/tests/ip.rs
+```
+
+#### Labels and CI
+
+Label PRs appropriately, first check the available labels and then apply the relevant ones:
+* when changes are RPC related, add A-rpc label
+* when changes are docs related, add C-docs label
+* ... and so on, check the available labels for more options.
+* if being tasked to open a pr, ensure that all changes are properly formatted: `cargo +nightly fmt --all`
+
+If changes in reth include changes to dependencies, run commands `zepter` and `make lint-toml` before finalizing the pr. Assume `zepter` binary is installed.
+
+### Debugging Tips
+
+1. **Logging**: Use `tracing` crate with appropriate levels
+   ```rust
+   tracing::debug!(target: "reth::component", ?value, "description");
+   ```
+
+2. **Metrics**: Add metrics for monitoring
+   ```rust
+   metrics::counter!("reth_component_operations").increment(1);
+   ```
+
+3. **Test Isolation**: Use separate test databases/directories
+
+### Finding Where to Contribute
+
+1. **Check Issues**: Look for issues labeled `good-first-issue` or `help-wanted`
+2. **Review TODOs**: Search for `TODO` comments in the codebase
+3. **Improve Tests**: Areas with low test coverage are good targets
+4. **Documentation**: Improve code comments and documentation
+5. **Performance**: Profile and optimize hot paths (with benchmarks)
+
+### Common PR Patterns
+
+#### Small, Focused Changes
+Most PRs change only 1-5 files. Examples:
+- Single-line bug fixes
+- Adding a missing trait implementation
+- Updating error messages
+- Adding test cases for edge conditions
+
+#### Integration Work
+When dependencies update (especially revm), code needs updating:
+- Check for breaking API changes
+- Update to use new features (like EIP implementations)
+- Ensure compatibility with new versions
+
+#### Test Improvements
+Tests often need expansion for:
+- New protocol versions (ETH68, ETH69)
+- Edge cases in state transitions
+- Network behavior under specific conditions
+- Concurrent operations
+
+#### Making Code More Generic
+Common refactoring pattern:
+- Replace concrete types with generics
+- Add trait bounds for flexibility
+- Enable reuse across different chain types
+
+#### When to Comment
+
+Write comments that remain valuable after the PR is merged. Future readers won't have PR context - they only see the current code.
+
+##### ✅ DO: Add Value
+
+**Explain WHY and non-obvious behavior:**
+```rust
+// Process must handle allocations atomically to prevent race conditions
+// between dealloc on drop and concurrent limit checks
+unsafe impl GlobalAlloc for LimitedAllocator { ... }
+
+// Binary search requires sorted input. Panics on unsorted slices.
+fn find_index(items: &[Item], target: &Item) -> Option<usize>
+
+// Timeout set to 5s to match EVM block processing limits
+const TRACER_TIMEOUT: Duration = Duration::from_secs(5);
+```
+
+**Document constraints and assumptions:**
+```rust
+/// Returns heap size estimate.
+/// 
+/// Note: May undercount shared references (Rc/Arc). For precise
+/// accounting, combine with an allocator-based approach.
+fn deep_size_of(&self) -> usize
+```
+
+**Explain complex logic:**
+```rust
+// We reset limits at task start because tokio reuses threads in
+// spawn_blocking pool. Without reset, second task inherits first
+// task's allocation count and immediately hits limit.
+THREAD_ALLOCATED.with(|allocated| allocated.set(0));
+```
+
+##### ❌ DON'T: Describe Changes
+```rust
+// ❌ BAD - Describes the change, not the code
+// Changed from Vec to HashMap for O(1) lookups
+
+// ✅ GOOD - Explains the decision
+// HashMap provides O(1) symbol lookups during trace replay
+```
+```rust
+// ❌ BAD - PR-specific context
+// Fix for issue #234 where memory wasn't freed
+
+// ✅ GOOD - Documents the actual behavior
+// Explicitly drop allocations before limit check to ensure
+// accurate accounting
+```
+```rust
+// ❌ BAD - States the obvious
+// Increment counter
+counter += 1;
+
+// ✅ GOOD - Explains non-obvious purpose
+// Track allocations across all threads for global limit enforcement
+GLOBAL_COUNTER.fetch_add(1, Ordering::SeqCst);
+```
+
+✅ **Comment when:**
+- Non-obvious behavior or edge cases
+- Performance trade-offs
+- Safety requirements (unsafe blocks must always be documented)
+- Limitations or gotchas
+- Why simpler alternatives don't work
+
+❌ **Don't comment when:**
+- Code is self-explanatory
+- Just restating the code in English
+- Describing what changed in this PR
+
+##### The Test: "Will this make sense in 6 months?"
+
+Before adding a comment, ask: Would someone reading just the current code (no PR, no history) find this helpful?
+
+
+#### Rust Style Guides
+
+##### Type Ordering in Files
+
+When defining structs, traits, and functions in a file, follow this ordering convention. The file's primary type (matching the file name) comes first, followed by supporting public types, then private types and helpers.
+
+```rust
+use ...;
+
+/// The primary type of this file (matches filename).
+pub struct PayloadProcessor { ... }
+
+impl PayloadProcessor { ... }
+
+// Followed by public auxiliary types that support the primary type
+
+/// Configuration for the processor.
+pub struct PayloadProcessorConfig { ... }
+
+/// Result type returned by processor operations.
+pub struct ProcessorResult { ... }
+
+// Followed by public traits related to the primary type
+
+pub trait ProcessorExt { ... }
+
+// Followed by private helper types
+
+struct InternalState { ... }
+
+// Followed by private helper functions
+
+fn validate_input() { ... }
+```
+
+❌ **Bad**: Adding new traits and auxiliary types **above** the file's primary type (see [#22133](https://github.com/paradigmxyz/reth/pull/22133)):
+
+```rust
+use ...;
+
+// ❌ BAD - new auxiliary struct added before the file's main type
+pub struct CacheWaitDurations { ... }
+
+// ❌ BAD - new trait added before the file's main type  
+pub trait WaitForCaches { ... }
+
+// The file's primary type is buried below unrelated additions
+pub struct PayloadProcessor { ... }
+```
+
+✅ **Good**: New types go **after** the primary type:
+
+```rust
+use ...;
+
+// ✅ The file's primary type stays at the top
+pub struct PayloadProcessor { ... }
+
+impl PayloadProcessor { ... }
+
+// ✅ Auxiliary types follow the primary type
+pub struct CacheWaitDurations { ... }
+
+pub trait WaitForCaches { ... }
+
+impl WaitForCaches for PayloadProcessor { ... }
+```
+
+### Example Contribution Workflow
+
+Let's say you want to fix a bug where external IP resolution fails on startup:
+
+1. **Create a branch**:
+   ```bash
+   git checkout -b fix-external-ip-resolution
+   ```
+
+2. **Find the relevant code**:
+   ```bash
+   # Search for IP resolution code
+   rg "external.*ip" --type rust
+   ```
+
+3. **Reason about the problem, when the problem is identified, make the fix**:
+   ```rust
+   // In crates/net/discv4/src/lib.rs
+   pub fn resolve_external_ip() -> Option<IpAddr> {
+       // Add fallback mechanism
+       nat::external_ip()
+           .or_else(|| nat::external_ip_from_stun())
+           .or_else(|| Some(DEFAULT_IP))
+   }
+   ```
+
+4. **Add a test**:
+   ```rust
+   #[test]
+   fn test_external_ip_fallback() {
+       // Test that resolution has proper fallbacks
+   }
+   ```
+
+5. **Run checks** (IMPORTANT!):
+   ```bash
+   cargo +nightly fmt --all
+   cargo clippy --workspace --all-features # Make sure WHOLE WORKSPACE compiles!
+   cargo nextest run -p reth-discv4
+   ```
+
+6. **Commit with clear message**:
+   ```bash
+   git commit -m "fix: add fallback for external IP resolution
+
+   Previously, node startup could fail if external IP resolution
+   failed. This adds fallback mechanisms to ensure the node can
+   always start with a reasonable default."
+   ```
+
+## Quick Reference
+
+### Essential Commands
+
+```bash
+# Format code
+cargo +nightly fmt --all
+
+# Run lints
+cargo +nightly clippy --workspace --all-features
+
+# Run tests
+cargo nextest run --workspace
+
+# Run specific benchmark
+cargo bench --bench bench_name
+
+# Build optimized binary
+cargo build --release
+
+# Check compilation for all features
+cargo check --workspace --all-features
+
+# Check documentation
+cargo docs --document-private-items
+
+# Regenerate CLI reference docs (after CLI changes)
+make update-book-cli
+```

CLAUDE.md

@@ -1,477 +0,0 @@
-# Reth Development Guide for AI Agents
-
-This guide provides comprehensive instructions for AI agents working on the Reth codebase. It covers the architecture, development workflows, and critical guidelines for effective contributions.
-
-## Project Overview
-
-Reth is a high-performance Ethereum execution client written in Rust, focusing on modularity, performance, and contributor-friendliness. The codebase is organized into well-defined crates with clear boundaries and responsibilities.
-
-## Architecture Overview
-
-### Core Components
-
-1. **Consensus (`crates/consensus/`)**: Validates blocks according to Ethereum consensus rules
-2. **Storage (`crates/storage/`)**: Hybrid database using MDBX + static files for optimal performance
-3. **Networking (`crates/net/`)**: P2P networking stack with discovery, sync, and transaction propagation
-4. **RPC (`crates/rpc/`)**: JSON-RPC server supporting all standard Ethereum APIs
-5. **Execution (`crates/evm/`, `crates/ethereum/`)**: Transaction execution and state transitions
-6. **Pipeline (`crates/stages/`)**: Staged sync architecture for blockchain synchronization
-7. **Trie (`crates/trie/`)**: Merkle Patricia Trie implementation with parallel state root computation
-8. **Node Builder (`crates/node/`)**: High-level node orchestration and configuration
-9. **The Consensus Engine (`crates/engine/`)**: Handles processing blocks received from the consensus layer with the Engine API (newPayload, forkchoiceUpdated)
-
-### Key Design Principles
-
-- **Modularity**: Each crate can be used as a standalone library
-- **Performance**: Extensive use of parallelism, memory-mapped I/O, and optimized data structures
-- **Extensibility**: Traits and generic types allow for different chain implementations
-- **Type Safety**: Strong typing throughout with minimal use of dynamic dispatch
-
-## Development Workflow
-
-### Code Style and Standards
-
-1. **Formatting**: Always use nightly rustfmt
-   ```bash
-   cargo +nightly fmt --all
-   ```
-
-2. **Linting**: Run clippy with all features
-   ```bash
-   cargo +nightly clippy --workspace --lib --examples --tests --benches --all-features 
-   ```
-
-3. **Testing**: Use nextest for faster test execution
-   ```bash
-   cargo nextest run --workspace
-   ```
-
-### Common Contribution Types
-
-Based on actual recent PRs, here are typical contribution patterns:
-
-#### 1. Small Bug Fixes (1-10 lines)
-Real example: Fixing beacon block root handling ([#16767](https://github.com/paradigmxyz/reth/pull/16767))
-```rust
-// Changed a single line to fix logic error
-- parent_beacon_block_root: parent.parent_beacon_block_root(),
-+ parent_beacon_block_root: parent.parent_beacon_block_root().map(|_| B256::ZERO),
-```
-
-#### 2. Integration with Upstream Changes
-Real example: Integrating revm updates ([#16752](https://github.com/paradigmxyz/reth/pull/16752))
-```rust
-// Update code to use new APIs from dependencies
-- if self.fork_tracker.is_shanghai_activated() {
--     if let Err(err) = transaction.ensure_max_init_code_size(MAX_INIT_CODE_BYTE_SIZE) {
-+ if let Some(init_code_size_limit) = self.fork_tracker.max_initcode_size() {
-+     if let Err(err) = transaction.ensure_max_init_code_size(init_code_size_limit) {
-```
-
-#### 3. Adding Comprehensive Tests
-Real example: ETH69 protocol tests ([#16759](https://github.com/paradigmxyz/reth/pull/16759))
-```rust
-#[tokio::test(flavor = "multi_thread")]
-async fn test_eth69_peers_can_connect() {
-    // Create test network with specific protocol versions
-    let p0 = PeerConfig::with_protocols(NoopProvider::default(), Some(EthVersion::Eth69.into()));
-    // Test connection and version negotiation
-}
-```
-
-#### 4. Making Components Generic
-Real example: Making EthEvmConfig generic over chainspec ([#16758](https://github.com/paradigmxyz/reth/pull/16758))
-```rust
-// Before: Hardcoded to ChainSpec
-- pub struct EthEvmConfig<EvmFactory = EthEvmFactory> {
--     pub executor_factory: EthBlockExecutorFactory<RethReceiptBuilder, Arc<ChainSpec>, EvmFactory>,
-
-// After: Generic over any chain spec type
-+ pub struct EthEvmConfig<C = ChainSpec, EvmFactory = EthEvmFactory>
-+ where
-+     C: EthereumHardforks,
-+ {
-+     pub executor_factory: EthBlockExecutorFactory<RethReceiptBuilder, Arc<C>, EvmFactory>,
-```
-
-#### 5. Resource Management Improvements
-Real example: ETL directory cleanup ([#16770](https://github.com/paradigmxyz/reth/pull/16770))
-```rust
-// Add cleanup logic on startup
-+ if let Err(err) = fs::remove_dir_all(&etl_path) {
-+     warn!(target: "reth::cli", ?etl_path, %err, "Failed to remove ETL path on launch");
-+ }
-```
-
-#### 6. Feature Additions
-Real example: Sharded mempool support ([#16756](https://github.com/paradigmxyz/reth/pull/16756))
-```rust
-// Add new filtering policies for transaction announcements
-pub struct ShardedMempoolAnnouncementFilter<T> {
-    pub inner: T,
-    pub shard_bits: u8,
-    pub node_id: Option<B256>,
-}
-```
-
-### Testing Guidelines
-
-1. **Unit Tests**: Test individual functions and components
-2. **Integration Tests**: Test interactions between components
-3. **Benchmarks**: For performance-critical code
-4. **Fuzz Tests**: For parsing and serialization code
-5. **Property Tests**: For checking component correctness on a wide variety of inputs
-
-Example test structure:
-```rust
-#[cfg(test)]
-mod tests {
-    use super::*;
-    
-    #[test]
-    fn test_component_behavior() {
-        // Arrange
-        let component = Component::new();
-        
-        // Act
-        let result = component.operation();
-        
-        // Assert
-        assert_eq!(result, expected);
-    }
-}
-```
-
-### Performance Considerations
-
-1. **Avoid Allocations in Hot Paths**: Use references and borrowing
-2. **Parallel Processing**: Use rayon for CPU-bound parallel work
-3. **Async/Await**: Use tokio for I/O-bound operations
-4. **File Operations**: Use `reth_fs_util` instead of `std::fs` for better error handling
-
-### Common Pitfalls
-
-1. **Don't Block Async Tasks**: Use `spawn_blocking` for CPU-intensive work or work with lots of blocking I/O
-2. **Handle Errors Properly**: Use `?` operator and proper error types
-
-### What to Avoid
-
-Based on PR patterns, avoid:
-
-1. **Large, sweeping changes**: Keep PRs focused and reviewable
-2. **Mixing unrelated changes**: One logical change per PR
-3. **Ignoring CI failures**: All checks must pass
-4. **Incomplete implementations**: Finish features before submitting
-5. **Modifying libmdbx sources**: Never modify files in `crates/storage/libmdbx-rs/mdbx-sys/libmdbx/` - this is vendored third-party code
-
-### CI Requirements
-
-Before submitting changes, ensure:
-
-1. **Format Check**: `cargo +nightly fmt --all --check`
-2. **Clippy**: No warnings
-3. **Tests Pass**: All unit and integration tests
-4. **Documentation**: Update relevant docs and add doc comments with `cargo docs --document-private-items`
-5. **CLI Docs** (if CLI changed): Run `make update-book-cli` (see below)
-6. **Commit Messages**: Follow conventional format (feat:, fix:, chore:, etc.)
-
-### CLI Reference Docs (`book` CI Job)
-
-The CLI reference pages under `docs/vocs/docs/pages/cli/` are **auto-generated** from the `reth` binary's `--help` output. **Do not edit these files manually** — any hand edits will be overwritten and CI will fail regardless.
-
-When you add, remove, or modify CLI commands, subcommands, or flags, regenerate the CLI docs by running:
-
-```bash
-make update-book-cli
-```
-
-This builds `reth` in debug mode and runs `docs/cli/update.sh` to regenerate all CLI pages. Commit the resulting changes.
-
-The `book` CI job (`.github/workflows/lint.yml`) enforces this by regenerating the docs and running `git diff --exit-code`. If the committed docs don't match the generated output, CI fails. Manually editing these pages is never productive — always use `make update-book-cli`.
-
-### Opening PRs against <https://github.com/paradigmxyz/reth>
-
-Label PRs appropriately, first check the available labels and then apply the relevant ones:
-* when changes are RPC related, add A-rpc label
-* when changes are docs related, add C-docs label
-* ... and so on, check the available labels for more options.
-* if being tasked to open a pr, ensure that all changes are properly formatted: `cargo +nightly fmt --all`
-
-If changes in reth include changes to dependencies, run commands `zepter` and `make lint-toml` before finalizing the pr. Assume `zepter` binary is installed.
-
-### Debugging Tips
-
-1. **Logging**: Use `tracing` crate with appropriate levels
-   ```rust
-   tracing::debug!(target: "reth::component", ?value, "description");
-   ```
-
-2. **Metrics**: Add metrics for monitoring
-   ```rust
-   metrics::counter!("reth_component_operations").increment(1);
-   ```
-
-3. **Test Isolation**: Use separate test databases/directories
-
-### Finding Where to Contribute
-
-1. **Check Issues**: Look for issues labeled `good-first-issue` or `help-wanted`
-2. **Review TODOs**: Search for `TODO` comments in the codebase
-3. **Improve Tests**: Areas with low test coverage are good targets
-4. **Documentation**: Improve code comments and documentation
-5. **Performance**: Profile and optimize hot paths (with benchmarks)
-
-### Common PR Patterns
-
-#### Small, Focused Changes
-Most PRs change only 1-5 files. Examples:
-- Single-line bug fixes
-- Adding a missing trait implementation
-- Updating error messages
-- Adding test cases for edge conditions
-
-#### Integration Work
-When dependencies update (especially revm), code needs updating:
-- Check for breaking API changes
-- Update to use new features (like EIP implementations)
-- Ensure compatibility with new versions
-
-#### Test Improvements
-Tests often need expansion for:
-- New protocol versions (ETH68, ETH69)
-- Edge cases in state transitions
-- Network behavior under specific conditions
-- Concurrent operations
-
-#### Making Code More Generic
-Common refactoring pattern:
-- Replace concrete types with generics
-- Add trait bounds for flexibility
-- Enable reuse across different chain types
-
-#### When to Comment
-
-Write comments that remain valuable after the PR is merged. Future readers won't have PR context - they only see the current code.
-
-##### ✅ DO: Add Value
-
-**Explain WHY and non-obvious behavior:**
-```rust
-// Process must handle allocations atomically to prevent race conditions
-// between dealloc on drop and concurrent limit checks
-unsafe impl GlobalAlloc for LimitedAllocator { ... }
-
-// Binary search requires sorted input. Panics on unsorted slices.
-fn find_index(items: &[Item], target: &Item) -> Option<usize>
-
-// Timeout set to 5s to match EVM block processing limits
-const TRACER_TIMEOUT: Duration = Duration::from_secs(5);
-```
-
-**Document constraints and assumptions:**
-```rust
-/// Returns heap size estimate.
-/// 
-/// Note: May undercount shared references (Rc/Arc). For precise
-/// accounting, combine with an allocator-based approach.
-fn deep_size_of(&self) -> usize
-```
-
-**Explain complex logic:**
-```rust
-// We reset limits at task start because tokio reuses threads in
-// spawn_blocking pool. Without reset, second task inherits first
-// task's allocation count and immediately hits limit.
-THREAD_ALLOCATED.with(|allocated| allocated.set(0));
-```
-
-##### ❌ DON'T: Describe Changes
-```rust
-// ❌ BAD - Describes the change, not the code
-// Changed from Vec to HashMap for O(1) lookups
-
-// ✅ GOOD - Explains the decision
-// HashMap provides O(1) symbol lookups during trace replay
-```
-```rust
-// ❌ BAD - PR-specific context
-// Fix for issue #234 where memory wasn't freed
-
-// ✅ GOOD - Documents the actual behavior
-// Explicitly drop allocations before limit check to ensure
-// accurate accounting
-```
-```rust
-// ❌ BAD - States the obvious
-// Increment counter
-counter += 1;
-
-// ✅ GOOD - Explains non-obvious purpose
-// Track allocations across all threads for global limit enforcement
-GLOBAL_COUNTER.fetch_add(1, Ordering::SeqCst);
-```
-
-✅ **Comment when:**
-- Non-obvious behavior or edge cases
-- Performance trade-offs
-- Safety requirements (unsafe blocks must always be documented)
-- Limitations or gotchas
-- Why simpler alternatives don't work
-
-❌ **Don't comment when:**
-- Code is self-explanatory
-- Just restating the code in English
-- Describing what changed in this PR
-
-##### The Test: "Will this make sense in 6 months?"
-
-Before adding a comment, ask: Would someone reading just the current code (no PR, no history) find this helpful?
-
-
-#### Rust Style Guides
-
-##### Type Ordering in Files
-
-When defining structs, traits, and functions in a file, follow this ordering convention. The file's primary type (matching the file name) comes first, followed by supporting public types, then private types and helpers.
-
-```rust
-use ...;
-
-/// The primary type of this file (matches filename).
-pub struct PayloadProcessor { ... }
-
-impl PayloadProcessor { ... }
-
-// Followed by public auxiliary types that support the primary type
-
-/// Configuration for the processor.
-pub struct PayloadProcessorConfig { ... }
-
-/// Result type returned by processor operations.
-pub struct ProcessorResult { ... }
-
-// Followed by public traits related to the primary type
-
-pub trait ProcessorExt { ... }
-
-// Followed by private helper types
-
-struct InternalState { ... }
-
-// Followed by private helper functions
-
-fn validate_input() { ... }
-```
-
-❌ **Bad**: Adding new traits and auxiliary types **above** the file's primary type (see [#22133](https://github.com/paradigmxyz/reth/pull/22133)):
-
-```rust
-use ...;
-
-// ❌ BAD - new auxiliary struct added before the file's main type
-pub struct CacheWaitDurations { ... }
-
-// ❌ BAD - new trait added before the file's main type  
-pub trait WaitForCaches { ... }
-
-// The file's primary type is buried below unrelated additions
-pub struct PayloadProcessor { ... }
-```
-
-✅ **Good**: New types go **after** the primary type:
-
-```rust
-use ...;
-
-// ✅ The file's primary type stays at the top
-pub struct PayloadProcessor { ... }
-
-impl PayloadProcessor { ... }
-
-// ✅ Auxiliary types follow the primary type
-pub struct CacheWaitDurations { ... }
-
-pub trait WaitForCaches { ... }
-
-impl WaitForCaches for PayloadProcessor { ... }
-```
-
-### Example Contribution Workflow
-
-Let's say you want to fix a bug where external IP resolution fails on startup:
-
-1. **Create a branch**:
-   ```bash
-   git checkout -b fix-external-ip-resolution
-   ```
-
-2. **Find the relevant code**:
-   ```bash
-   # Search for IP resolution code
-   rg "external.*ip" --type rust
-   ```
-
-3. **Reason about the problem, when the problem is identified, make the fix**:
-   ```rust
-   // In crates/net/discv4/src/lib.rs
-   pub fn resolve_external_ip() -> Option<IpAddr> {
-       // Add fallback mechanism
-       nat::external_ip()
-           .or_else(|| nat::external_ip_from_stun())
-           .or_else(|| Some(DEFAULT_IP))
-   }
-   ```
-
-4. **Add a test**:
-   ```rust
-   #[test]
-   fn test_external_ip_fallback() {
-       // Test that resolution has proper fallbacks
-   }
-   ```
-
-5. **Run checks** (IMPORTANT!):
-   ```bash
-   cargo +nightly fmt --all
-   cargo clippy --workspace --all-features # Make sure WHOLE WORKSPACE compiles!
-   cargo nextest run -p reth-discv4
-   ```
-
-6. **Commit with clear message**:
-   ```bash
-   git commit -m "fix: add fallback for external IP resolution
-
-   Previously, node startup could fail if external IP resolution
-   failed. This adds fallback mechanisms to ensure the node can
-   always start with a reasonable default."
-   ```
-
-## Quick Reference
-
-### Essential Commands
-
-```bash
-# Format code
-cargo +nightly fmt --all
-
-# Run lints
-cargo +nightly clippy --workspace --all-features
-
-# Run tests
-cargo nextest run --workspace
-
-# Run specific benchmark
-cargo bench --bench bench_name
-
-# Build optimized binary
-cargo build --release
-
-# Check compilation for all features
-cargo check --workspace --all-features
-
-# Check documentation
-cargo docs --document-private-items
-
-# Regenerate CLI reference docs (after CLI changes)
-make update-book-cli
-```

CLAUDE.md

@@ -0,0 +1 @@
+AGENTS.md

Cargo.toml

@@ -1,5 +1,5 @@
 [workspace.package]
-version = "1.11.1"
+version = "1.11.3"
 edition = "2024"
 rust-version = "1.93"
 license = "MIT OR Apache-2.0"
@@ -9,8 +9,8 @@ exclude = [".github/"]
 
 [workspace]
 members = [
+    "bin/reth-bb/",
     "bin/reth-bench/",
-    "bin/reth-bench-compare/",
     "bin/reth/",
     "crates/storage/rpc-provider/",
     "crates/chain-state/",
@@ -27,6 +27,7 @@ members = [
     "crates/engine/invalid-block-hooks/",
     "crates/engine/local",
     "crates/engine/primitives/",
+    "crates/engine/execution-cache/",
     "crates/engine/tree/",
     "crates/engine/util/",
     "crates/era",
@@ -77,8 +78,6 @@ members = [
     "crates/payload/primitives/",
     "crates/payload/validator/",
     "crates/payload/util/",
-    "crates/primitives-traits/",
-    "crates/primitives/",
     "crates/prune/db",
     "crates/prune/prune",
     "crates/prune/types",
@@ -100,8 +99,6 @@ members = [
     "crates/stages/types/",
     "crates/static-file/static-file",
     "crates/static-file/types/",
-    "crates/storage/codecs/",
-    "crates/storage/codecs/derive/",
     "crates/storage/db-api/",
     "crates/storage/db-common",
     "crates/storage/db-models/",
@@ -112,7 +109,6 @@ members = [
     "crates/storage/nippy-jar/",
     "crates/storage/provider/",
     "crates/storage/storage-api/",
-    "crates/storage/zstd-compressors/",
     "crates/tasks/",
     "crates/tokio-util/",
     "crates/tracing/",
@@ -170,6 +166,7 @@ rust.rust_2018_idioms = { level = "deny", priority = -1 }
 rust.unreachable_pub = "warn"
 rust.unused_must_use = "deny"
 rust.rust_2024_incompatible_pat = "warn"
+rust.unexpected_cfgs = { level = "warn", check-cfg = ['cfg(tokio_unstable)'] }
 rustdoc.all = "warn"
 # rust.unnameable-types = "warn"
 
@@ -323,15 +320,14 @@ reth = { path = "bin/reth" }
 reth-storage-rpc-provider = { path = "crates/storage/rpc-provider" }
 reth-basic-payload-builder = { path = "crates/payload/basic" }
 reth-bench = { path = "bin/reth-bench" }
-reth-bench-compare = { path = "bin/reth-bench-compare" }
 reth-chain-state = { path = "crates/chain-state" }
 reth-chainspec = { path = "crates/chainspec", default-features = false }
 reth-cli = { path = "crates/cli/cli" }
 reth-cli-commands = { path = "crates/cli/commands" }
 reth-cli-runner = { path = "crates/cli/runner" }
 reth-cli-util = { path = "crates/cli/util" }
-reth-codecs = { path = "crates/storage/codecs" }
-reth-codecs-derive = { path = "crates/storage/codecs/derive" }
+reth-codecs = { version = "0.1.0", default-features = false }
+reth-codecs-derive = "0.1.0"
 reth-config = { path = "crates/config", default-features = false }
 reth-consensus = { path = "crates/consensus/consensus", default-features = false }
 reth-consensus-common = { path = "crates/consensus/common", default-features = false }
@@ -347,6 +343,7 @@ reth-downloaders = { path = "crates/net/downloaders" }
 reth-e2e-test-utils = { path = "crates/e2e-test-utils" }
 reth-ecies = { path = "crates/net/ecies" }
 reth-engine-local = { path = "crates/engine/local" }
+reth-execution-cache = { path = "crates/engine/execution-cache" }
 reth-engine-primitives = { path = "crates/engine/primitives", default-features = false }
 reth-engine-tree = { path = "crates/engine/tree" }
 reth-engine-util = { path = "crates/engine/util" }
@@ -398,8 +395,7 @@ reth-payload-builder-primitives = { path = "crates/payload/builder-primitives" }
 reth-payload-primitives = { path = "crates/payload/primitives" }
 reth-payload-validator = { path = "crates/payload/validator" }
 reth-payload-util = { path = "crates/payload/util" }
-reth-primitives = { path = "crates/primitives", default-features = false, features = ["__internal"] }
-reth-primitives-traits = { path = "crates/primitives-traits", default-features = false }
+reth-primitives-traits = { version = "0.1.0", default-features = false }
 reth-provider = { path = "crates/storage/provider" }
 reth-prune = { path = "crates/prune/prune" }
 reth-prune-types = { path = "crates/prune/types", default-features = false }
@@ -415,6 +411,7 @@ reth-rpc-eth-types = { path = "crates/rpc/rpc-eth-types", default-features = fal
 reth-rpc-layer = { path = "crates/rpc/rpc-layer" }
 reth-rpc-server-types = { path = "crates/rpc/rpc-server-types" }
 reth-rpc-convert = { path = "crates/rpc/rpc-convert" }
+reth-rpc-traits = { version = "0.1.0", default-features = false }
 reth-stages = { path = "crates/stages/stages" }
 reth-stages-api = { path = "crates/stages/api" }
 reth-stages-types = { path = "crates/stages/types", default-features = false }
@@ -433,70 +430,64 @@ reth-trie-common = { path = "crates/trie/common", default-features = false }
 reth-trie-db = { path = "crates/trie/db" }
 reth-trie-parallel = { path = "crates/trie/parallel" }
 reth-trie-sparse = { path = "crates/trie/sparse", default-features = false }
-reth-zstd-compressors = { path = "crates/storage/zstd-compressors", default-features = false }
+reth-zstd-compressors = { version = "0.1.0", default-features = false }
 
 # revm
-revm = { version = "34.0.0", default-features = false }
-revm-bytecode = { version = "8.0.0", default-features = false }
-revm-database = { version = "10.0.0", default-features = false }
-revm-state = { version = "9.0.0", default-features = false }
-revm-primitives = { version = "22.0.0", default-features = false }
-revm-interpreter = { version = "32.0.0", default-features = false }
-revm-database-interface = { version = "9.0.0", default-features = false }
-op-revm = { version = "15.0.0", default-features = false }
-revm-inspectors = "0.34.2"
+revm = { version = "36.0.0", default-features = false }
+revm-bytecode = { version = "9.0.0", default-features = false }
+revm-database = { version = "12.0.0", default-features = false }
+revm-state = { version = "10.0.0", default-features = false }
+revm-primitives = { version = "22.1.0", default-features = false }
+revm-interpreter = { version = "34.0.0", default-features = false }
+revm-database-interface = { version = "10.0.0", default-features = false }
+revm-inspectors = "0.36.0"
 
 # eth
 alloy-dyn-abi = "1.5.6"
 alloy-primitives = { version = "1.5.6", default-features = false, features = ["map-foldhash"] }
 alloy-sol-types = { version = "1.5.6", default-features = false }
 
-alloy-chains = { version = "0.2.5", default-features = false }
+alloy-chains = { version = "0.2.33", default-features = false }
 alloy-eip2124 = { version = "0.2.0", default-features = false }
 alloy-eip7928 = { version = "0.3.0", default-features = false }
-alloy-evm = { version = "0.27.2", default-features = false }
+alloy-evm = { version = "0.30.0", default-features = false }
 alloy-rlp = { version = "0.3.13", default-features = false, features = ["core-net"] }
 alloy-trie = { version = "0.9.4", default-features = false }
 
 alloy-hardforks = "0.4.5"
 
-alloy-consensus = { version = "1.7.3", default-features = false }
-alloy-contract = { version = "1.7.3", default-features = false }
-alloy-eips = { version = "1.7.3", default-features = false }
-alloy-genesis = { version = "1.7.3", default-features = false }
-alloy-json-rpc = { version = "1.7.3", default-features = false }
-alloy-network = { version = "1.7.3", default-features = false }
-alloy-network-primitives = { version = "1.7.3", default-features = false }
-alloy-provider = { version = "1.7.3", features = ["reqwest", "debug-api"], default-features = false }
-alloy-pubsub = { version = "1.7.3", default-features = false }
-alloy-rpc-client = { version = "1.7.3", default-features = false }
-alloy-rpc-types = { version = "1.7.3", features = ["eth"], default-features = false }
-alloy-rpc-types-admin = { version = "1.7.3", default-features = false }
-alloy-rpc-types-anvil = { version = "1.7.3", default-features = false }
-alloy-rpc-types-beacon = { version = "1.7.3", default-features = false }
-alloy-rpc-types-debug = { version = "1.7.3", default-features = false }
-alloy-rpc-types-engine = { version = "1.7.3", default-features = false }
-alloy-rpc-types-eth = { version = "1.7.3", default-features = false }
-alloy-rpc-types-mev = { version = "1.7.3", default-features = false }
-alloy-rpc-types-trace = { version = "1.7.3", default-features = false }
-alloy-rpc-types-txpool = { version = "1.7.3", default-features = false }
-alloy-serde = { version = "1.7.3", default-features = false }
-alloy-signer = { version = "1.7.3", default-features = false }
-alloy-signer-local = { version = "1.7.3", default-features = false }
-alloy-transport = { version = "1.7.3" }
-alloy-transport-http = { version = "1.7.3", features = ["reqwest-rustls-tls"], default-features = false }
-alloy-transport-ipc = { version = "1.7.3", default-features = false }
-alloy-transport-ws = { version = "1.7.3", default-features = false }
+alloy-consensus = { version = "1.8.2", default-features = false }
+alloy-contract = { version = "1.8.2", default-features = false }
+alloy-eips = { version = "1.8.2", default-features = false }
+alloy-genesis = { version = "1.8.2", default-features = false }
+alloy-json-rpc = { version = "1.8.2", default-features = false }
+alloy-network = { version = "1.8.2", default-features = false }
+alloy-network-primitives = { version = "1.8.2", default-features = false }
+alloy-provider = { version = "1.8.2", features = ["reqwest", "debug-api"], default-features = false }
+alloy-pubsub = { version = "1.8.2", default-features = false }
+alloy-rpc-client = { version = "1.8.2", default-features = false }
+alloy-rpc-types = { version = "1.8.2", features = ["eth"], default-features = false }
+alloy-rpc-types-admin = { version = "1.8.2", default-features = false }
+alloy-rpc-types-anvil = { version = "1.8.2", default-features = false }
+alloy-rpc-types-beacon = { version = "1.8.2", default-features = false }
+alloy-rpc-types-debug = { version = "1.8.2", default-features = false }
+alloy-rpc-types-engine = { version = "1.8.2", default-features = false }
+alloy-rpc-types-eth = { version = "1.8.2", default-features = false }
+alloy-rpc-types-mev = { version = "1.8.2", default-features = false }
+alloy-rpc-types-trace = { version = "1.8.2", default-features = false }
+alloy-rpc-types-txpool = { version = "1.8.2", default-features = false }
+alloy-serde = { version = "1.8.2", default-features = false }
+alloy-signer = { version = "1.8.2", default-features = false }
+alloy-signer-local = { version = "1.8.2", default-features = false }
+alloy-transport = { version = "1.8.2" }
+alloy-transport-http = { version = "1.8.2", features = ["reqwest-rustls-tls"], default-features = false }
+alloy-transport-ipc = { version = "1.8.2", default-features = false }
+alloy-transport-ws = { version = "1.8.2", default-features = false }
 
 # op
-alloy-op-evm = { version = "0.27.2", default-features = false }
-alloy-op-hardforks = "0.4.4"
-op-alloy-rpc-types = { version = "0.23.1", default-features = false }
-op-alloy-rpc-types-engine = { version = "0.23.1", default-features = false }
-op-alloy-network = { version = "0.23.1", default-features = false }
-op-alloy-consensus = { version = "0.23.1", default-features = false }
-op-alloy-rpc-jsonrpsee = { version = "0.23.1", default-features = false }
-op-alloy-flz = { version = "0.13.1", default-features = false }
+op-alloy-rpc-types = { version = "0.24.0", default-features = false }
+op-alloy-rpc-types-engine = { version = "0.24.0", default-features = false }
+op-alloy-consensus = { version = "0.24.0", default-features = false }
 
 # misc
 either = { version = "1.15.0", default-features = false }
@@ -508,6 +499,7 @@ bincode = "1.3"
 bitflags = "2.4"
 boyer-moore-magiclen = "0.2.16"
 bytes = { version = "1.11.1", default-features = false }
+blake3 = "1.8"
 brotli = "8"
 cfg-if = "1.0"
 clap = "4"
@@ -523,6 +515,7 @@ humantime = "2.1"
 humantime-serde = "1.1"
 itertools = { version = "0.14", default-features = false }
 linked_hash_set = "0.1"
+libc = "0.2"
 lz4 = "1.28.1"
 modular-bitfield = "0.13.1"
 notify = { version = "8.0.0", default-features = false, features = ["macos_fsevent"] }
@@ -541,6 +534,7 @@ serde_json = { version = "1.0", default-features = false, features = ["alloc"] }
 serde_with = { version = "3", default-features = false, features = ["macros"] }
 sha2 = { version = "0.10", default-features = false }
 shlex = "1.3"
+slotmap = "1"
 smallvec = "1"
 strum = { version = "0.27", default-features = false }
 strum_macros = "0.27"
@@ -585,7 +579,7 @@ futures-util = { version = "0.3", default-features = false }
 hyper = "1.3"
 hyper-util = "0.1.5"
 pin-project = "1.0.12"
-reqwest = { version = "0.12", default-features = false, features = ["rustls-tls", "rustls-tls-native-roots", "stream"] }
+reqwest = { version = "0.13", default-features = false, features = ["rustls", "stream"] }
 tracing-futures = "0.2"
 tower = "0.5"
 tower-http = "0.6"
@@ -651,6 +645,7 @@ ethereum_ssz_derive = "0.10.1"
 # allocators
 jemalloc_pprof = { version = "0.8", default-features = false }
 tikv-jemalloc-ctl = "0.6"
+tikv-jemalloc-sys = "0.6"
 tikv-jemallocator = "0.6"
 tracy-client = { version = "0.18.0", features = ["demangle"] }
 snmalloc-rs = { version = "0.3.7", features = ["build_cc"] }
@@ -665,6 +660,7 @@ cipher = "0.4.3"
 comfy-table = "7.0"
 concat-kdf = "0.1.0"
 crossbeam-channel = "0.5.13"
+crossbeam-queue = "0.3"
 crossbeam-utils = "0.8"
 crossterm = "0.29.0"
 csv = "1.3.0"
@@ -684,7 +680,6 @@ memmap2 = "0.9.4"
 mev-share-sse = { version = "0.5.0", default-features = false }
 num-traits = "0.2.15"
 page_size = "0.6.0"
-parity-scale-codec = "3.2.1"
 plain_hasher = "0.2"
 pretty_assertions = "1.4"
 ratatui = { version = "0.30", default-features = false }
@@ -697,7 +692,7 @@ snap = "1.1.1"
 socket2 = { version = "0.6", default-features = false }
 sysinfo = { version = "0.38", default-features = false }
 tracing-journald = "0.3"
-tracing-logfmt = "=0.3.5"
+tracing-logfmt = "0.3.7"
 tracing-samply = "0.1"
 tracing-subscriber = { version = "0.3", default-features = false }
 tracing-tracy = "0.11"
@@ -741,10 +736,8 @@ ipnet = "2.11"
 # alloy-transport-ws = { git = "https://github.com/alloy-rs/alloy", rev = "3049f232fbb44d1909883e154eb38ec5962f53a3" }
 
 # op-alloy-consensus = { git = "https://github.com/alloy-rs/op-alloy", rev = "a79d6fc" }
-# op-alloy-network = { git = "https://github.com/alloy-rs/op-alloy", rev = "a79d6fc" }
 # op-alloy-rpc-types = { git = "https://github.com/alloy-rs/op-alloy", rev = "a79d6fc" }
 # op-alloy-rpc-types-engine = { git = "https://github.com/alloy-rs/op-alloy", rev = "a79d6fc" }
-# op-alloy-rpc-jsonrpsee = { git = "https://github.com/alloy-rs/op-alloy", rev = "a79d6fc" }
 #
 # revm-inspectors = { git = "https://github.com/paradigmxyz/revm-inspectors", rev = "1207e33" }
 #
@@ -754,10 +747,6 @@ ipnet = "2.11"
 # jsonrpsee-http-client = { git = "https://github.com/paradigmxyz/jsonrpsee", branch = "matt/make-rpc-service-pub" }
 # jsonrpsee-types = { git = "https://github.com/paradigmxyz/jsonrpsee", branch = "matt/make-rpc-service-pub" }
 
-# alloy-evm = { git = "https://github.com/alloy-rs/evm", rev = "df124c0" }
-# alloy-op-evm = { git = "https://github.com/alloy-rs/evm", rev = "df124c0" }
+# alloy-evm = { git = "https://github.com/alloy-rs/evm", rev = "9bc2dba" }
 
 # revm-inspectors = { git = "https://github.com/paradigmxyz/revm-inspectors", rev = "3020ea8" }
-
-# alloy-evm = { git = "https://github.com/alloy-rs/evm", rev = "072c248" }
-# alloy-op-evm = { git = "https://github.com/alloy-rs/evm", rev = "072c248" }

Dockerfile.depot

@@ -1,8 +1,10 @@
 # syntax=docker/dockerfile:1
 
 # Dockerfile for reth, optimized for Depot builds
+# Supports PGO+BOLT optimization for maximum performance
 # Usage:
 #   reth: --build-arg BINARY=reth
+#   PGO+BOLT: --build-arg USE_PGO_BOLT=true (Linux x86_64/aarch64 only)
 
 FROM rust:1.93 AS builder
 WORKDIR /app
@@ -43,6 +45,18 @@ ENV VERGEN_GIT_SHA=$VERGEN_GIT_SHA
 ENV VERGEN_GIT_DESCRIBE=$VERGEN_GIT_DESCRIBE
 ENV VERGEN_GIT_DIRTY=$VERGEN_GIT_DIRTY
 
+# Enable PGO+BOLT optimization (Linux only)
+ARG USE_PGO_BOLT=false
+ENV USE_PGO_BOLT=$USE_PGO_BOLT
+
+# Optional path to a pre-collected merged.profdata file in build context.
+ARG PGO_PROFDATA=""
+ENV PGO_PROFDATA=$PGO_PROFDATA
+
+# Whether to strip debug symbols from PGO-built binaries.
+ARG STRIP_SYMBOLS=true
+ENV STRIP_SYMBOLS=$STRIP_SYMBOLS
+
 # Build application
 # Platform-specific RUSTFLAGS: amd64 uses x86-64-v3 (Haswell+) with pclmulqdq for rocksdb
 ARG TARGETPLATFORM
@@ -53,12 +67,21 @@ RUN --mount=type=secret,id=DEPOT_TOKEN,env=SCCACHE_WEBDAV_TOKEN \
     --mount=type=cache,target=$SCCACHE_DIR,sharing=shared \
     export RUSTC_WRAPPER=sccache SCCACHE_WEBDAV_ENDPOINT=https://cache.depot.dev SCCACHE_DIR=/sccache && \
     sccache --start-server && \
-    if [ -n "$RUSTFLAGS" ]; then \
-        export RUSTFLAGS="$RUSTFLAGS"; \
-    elif [ "$TARGETPLATFORM" = "linux/amd64" ]; then \
-        export RUSTFLAGS="-C target-cpu=x86-64-v3 -C target-feature=+pclmulqdq"; \
+    if [ "$USE_PGO_BOLT" = "true" ] && [ "$TARGETPLATFORM" = "linux/amd64" ] && [ -n "$PGO_PROFDATA" ] && [ -f "$PGO_PROFDATA" ]; then \
+        apt-get update && apt-get install -y -qq lsb-release wget sudo && \
+        BINARY="$BINARY" PROFILE="$BUILD_PROFILE" FEATURES="$FEATURES" SKIP_BOLT=true STRIP_SYMBOLS="$STRIP_SYMBOLS" PGO_PROFDATA="$PGO_PROFDATA" \
+            ./.github/scripts/build_pgo_bolt.sh; \
+    else \
+        if [ "$USE_PGO_BOLT" = "true" ]; then \
+            echo "PGO requested but pre-collected profile missing at '${PGO_PROFDATA:-<unset>}' - falling back to non-PGO build"; \
+        fi; \
+        if [ -n "$RUSTFLAGS" ]; then \
+            export RUSTFLAGS="$RUSTFLAGS"; \
+        elif [ "$TARGETPLATFORM" = "linux/amd64" ]; then \
+            export RUSTFLAGS="-C target-cpu=x86-64-v3 -C target-feature=+pclmulqdq"; \
+        fi && \
+        cargo build --profile $BUILD_PROFILE --features "$FEATURES" --locked --bin $BINARY --manifest-path $MANIFEST_PATH/Cargo.toml; \
     fi && \
-    cargo build --profile $BUILD_PROFILE --features "$FEATURES" --locked --bin $BINARY --manifest-path $MANIFEST_PATH/Cargo.toml && \
     sccache --show-stats
 
 # Copy binary to a known location (ARG not resolved in COPY)

Makefile

@@ -17,6 +17,9 @@ FEATURES ?=
 # Cargo profile for builds. Default is for local builds, CI uses an override.
 PROFILE ?= release
 
+# Extra RUSTFLAGS to append to build targets (e.g., "-C target-cpu=x86-64-v3")
+EXTRA_RUSTFLAGS ?=
+
 # Extra flags for Cargo
 CARGO_INSTALL_EXTRA_FLAGS ?=
 
@@ -67,14 +70,14 @@ build-%-reproducible:
 	LC_ALL=C \
 	TZ=UTC \
 	JEMALLOC_OVERRIDE=/usr/lib/x86_64-linux-gnu/libjemalloc.a \
-	cargo build --bin reth --features "$(FEATURES) jemalloc-unprefixed" --profile "reproducible" --locked --target x86_64-unknown-linux-gnu
+	cargo build --bin reth --features "$(FEATURES)" --profile "reproducible" --locked --target x86_64-unknown-linux-gnu
 
 .PHONY: build-debug
 build-debug: ## Build the reth binary into `target/debug` directory.
 	cargo build --bin reth --features "$(FEATURES)"
 # Builds the reth binary natively.
 build-native-%:
-	cargo build --bin reth --target $* --features "$(FEATURES)" --profile "$(PROFILE)"
+	$(if $(EXTRA_RUSTFLAGS),RUSTFLAGS="$(EXTRA_RUSTFLAGS)") cargo build --bin reth --target $* --features "$(FEATURES)" --profile "$(PROFILE)"
 
 # The following commands use `cross` to build a cross-compile.
 #
@@ -92,11 +95,12 @@ build-native-%:
 # on other systems. JEMALLOC_SYS_WITH_LG_PAGE=16 tells jemalloc to use 64-KiB
 # pages. See: https://github.com/paradigmxyz/reth/issues/6742
 build-aarch64-unknown-linux-gnu: export JEMALLOC_SYS_WITH_LG_PAGE=16
+build-native-aarch64-unknown-linux-gnu: export JEMALLOC_SYS_WITH_LG_PAGE=16
 
 # Note: The additional rustc compiler flags are for intrinsics needed by MDBX.
 # See: https://github.com/cross-rs/cross/wiki/FAQ#undefined-reference-with-build-std
 build-%:
-	RUSTFLAGS="-C link-arg=-lgcc -Clink-arg=-static-libgcc" \
+	RUSTFLAGS="-C link-arg=-lgcc -Clink-arg=-static-libgcc $(EXTRA_RUSTFLAGS)" \
 		cross build --bin reth --target $* --features "$(FEATURES)" --profile "$(PROFILE)"
 
 # Unfortunately we can't easily use cross to build for Darwin because of licensing issues.

bin/reth-bb/Cargo.toml

@@ -0,0 +1,100 @@
+[package]
+name = "reth-bb"
+version.workspace = true
+edition.workspace = true
+rust-version.workspace = true
+license.workspace = true
+homepage.workspace = true
+repository.workspace = true
+description = "Reth node configured for big block payload execution"
+
+[lints]
+workspace = true
+
+[dependencies]
+# reth
+reth-ethereum-cli.workspace = true
+reth-chainspec.workspace = true
+reth-ethereum-primitives.workspace = true
+reth-cli-util.workspace = true
+reth-node-core.workspace = true
+reth-node-ethereum.workspace = true
+reth-node-builder.workspace = true
+reth-node-api.workspace = true
+reth-ethereum-consensus.workspace = true
+reth-engine-primitives = { workspace = true, features = ["std"] }
+reth-engine-tree.workspace = true
+reth-primitives-traits.workspace = true
+reth-payload-primitives.workspace = true
+reth-provider.workspace = true
+reth-rpc-api.workspace = true
+reth-rpc-engine-api.workspace = true
+reth-evm.workspace = true
+reth-evm-ethereum.workspace = true
+reth-ethereum-forks.workspace = true
+reth-revm.workspace = true
+reth-consensus.workspace = true
+reth-chain-state.workspace = true
+reth-errors.workspace = true
+reth-storage-errors.workspace = true
+
+# alloy
+alloy-rpc-types = { workspace = true, features = ["engine"] }
+alloy-primitives.workspace = true
+alloy-rlp.workspace = true
+alloy-consensus.workspace = true
+alloy-eips.workspace = true
+alloy-evm.workspace = true
+
+# tracing
+tracing.workspace = true
+
+# misc
+clap = { workspace = true, features = ["derive", "env"] }
+jsonrpsee = { workspace = true, features = ["server", "macros"] }
+async-trait.workspace = true
+derive_more.workspace = true
+crossbeam-channel.workspace = true
+tokio = { workspace = true, features = ["sync"] }
+revm.workspace = true
+revm-primitives.workspace = true
+alloy-hardforks.workspace = true
+metrics.workspace = true
+
+# std
+eyre.workspace = true
+
+[features]
+default = [
+    "jemalloc",
+    "reth-cli-util/jemalloc",
+    "asm-keccak",
+    "min-debug-logs",
+]
+
+jemalloc = [
+    "reth-cli-util/jemalloc",
+    "reth-node-core/jemalloc",
+    "reth-ethereum-cli/jemalloc",
+    "reth-provider/jemalloc",
+]
+
+asm-keccak = [
+    "reth-node-core/asm-keccak",
+    "reth-ethereum-cli/asm-keccak",
+    "reth-node-ethereum/asm-keccak",
+    "alloy-primitives/asm-keccak",
+    "alloy-evm/asm-keccak",
+    "revm/asm-keccak",
+    "revm-primitives/asm-keccak",
+]
+
+min-debug-logs = [
+    "tracing/release_max_level_debug",
+    "reth-ethereum-cli/min-debug-logs",
+    "reth-node-core/min-debug-logs",
+]
+
+[[bin]]
+name = "reth-bb"
+path = "src/main.rs"

bin/reth-bb/README.md

@@ -0,0 +1,67 @@
+# reth-bb
+
+A modified reth node for benchmarking **big block** execution — payloads that merge transactions from multiple consecutive blocks into a single block to simulate high-gas workloads.
+
+> **Not for production use.** reth-bb disables some consensus-related validations to allow artificially large blocks. It is intended solely for performance benchmarking.
+
+## How it works
+
+reth-bb extends the standard Ethereum node with:
+
+1. **Multi-segment execution** — a custom `reth_newPayload` handler that accepts optional `BigBlockData` alongside the payload. When present, the block is executed in multiple segments, each with its own EVM environment (matching the original blocks that were merged).
+
+2. **Relaxed consensus** — the gas-limit bound-divisor check and blob gas validation are skipped, since merged blocks exceed single-block limits.
+
+## Quick start
+
+The full workflow has four steps: **build** binaries, **generate** big blocks,  **start** reth-bb, and **replay** the payloads.
+
+### Prerequisites
+
+- A synced reth datadir for the target chain (e.g. hoodi)
+- Rust toolchain
+
+### 1. Build
+
+```bash
+cargo build --profile profiling -p reth-bb -p reth-bench
+```
+
+### 2. Generate big blocks
+
+Fetch consecutive blocks from an RPC and merge them until a target gas is reached. Use `--from-block` set to the block number following the one the node is currently synced to (i.e. the next block the node would process):
+
+```bash
+reth-bench generate-big-block \
+    --rpc-url https://rpc.hoodi.ethpandaops.io \
+    --chain hoodi \
+    --from-block 910020 \
+    --target-gas 2G \
+    --num-big-blocks 5 \
+    --output-dir /tmp/payloads
+```
+
+This produces one JSON file per big block in the output directory.
+
+### 3. Start reth-bb
+
+```bash
+reth-bb node \
+    --datadir /data/reth/hoodi \
+    --chain hoodi \
+    --http --http.api debug,eth \
+    --authrpc.jwtsecret /tmp/jwt.hex \
+    -d
+```
+
+### 4. Replay payloads
+
+```bash
+reth-bench replay-payloads \
+    --engine-rpc-url http://localhost:8551 \
+    --jwt-secret /tmp/jwt.hex \
+    --payload-dir /tmp/payloads \
+    --reth-new-payload
+```
+
+The `--reth-new-payload` flag is required for big blocks — it uses the `reth_newPayload` endpoint which carries the multi-segment execution metadata.

bin/reth-bb/src/evm.rs

@@ -0,0 +1,570 @@
+//! Big-block executor.
+//!
+//! Provides [`BbBlockExecutor`] and [`BbBlockExecutorFactory`] which handle
+//! segment boundaries within big-block payloads.
+//!
+//! [`BbBlockExecutor`] wraps [`EthBlockExecutor`] and intercepts
+//! `execute_transaction` to apply segment-boundary changes.
+
+use crate::evm_config::BigBlockSegment;
+use alloy_eips::eip7685::Requests;
+use alloy_evm::{
+    block::{
+        BlockExecutionError, BlockExecutionResult, BlockExecutor, BlockExecutorFactory,
+        BlockExecutorFor, ExecutableTx, OnStateHook, StateChangeSource, StateDB,
+    },
+    eth::{EthBlockExecutionCtx, EthBlockExecutor, EthEvmContext, EthTxResult},
+    precompiles::PrecompilesMap,
+    Database, EthEvm, EthEvmFactory, Evm, FromRecoveredTx, FromTxWithEncoded,
+};
+use alloy_primitives::B256;
+use reth_ethereum_primitives::{Receipt, TransactionSigned};
+use reth_evm_ethereum::RethReceiptBuilder;
+use revm::{
+    context::{BlockEnv, TxEnv},
+    context_interface::result::{EVMError, HaltReason},
+    handler::PrecompileProvider,
+    interpreter::InterpreterResult,
+    primitives::hardfork::SpecId,
+    Inspector,
+};
+use std::sync::{Arc, Mutex};
+use tracing::{debug, trace};
+
+// ---------------------------------------------------------------------------
+// BbEvmPlan — runtime segment tracking state
+// ---------------------------------------------------------------------------
+
+/// Runtime state for segment boundary tracking.
+pub(crate) struct BbEvmPlan {
+    /// The segment boundaries and environments.
+    pub(crate) segments: Vec<BigBlockSegment>,
+    /// Index of the next segment to switch to (starts at 1).
+    pub(crate) next_segment: usize,
+    /// Number of user transactions executed so far.
+    pub(crate) tx_counter: usize,
+    /// Block hashes to seed for inter-segment BLOCKHASH resolution.
+    /// Includes both prior block hashes and inter-segment hashes.
+    pub(crate) block_hashes_to_seed: Vec<(u64, B256)>,
+}
+
+impl BbEvmPlan {
+    /// Creates a new `BbEvmPlan` from segments and hardfork flags.
+    pub(crate) fn new(segments: Vec<BigBlockSegment>) -> Self {
+        // Pre-compute all inter-segment block hashes.
+        let mut block_hashes_to_seed = Vec::new();
+        for seg in segments.iter().skip(1) {
+            let finished_block_number = seg.evm_env.block_env.number.saturating_to::<u64>() - 1;
+            let finished_block_hash = seg.ctx.parent_hash;
+            block_hashes_to_seed.push((finished_block_number, finished_block_hash));
+        }
+
+        Self { segments, next_segment: 1, tx_counter: 0, block_hashes_to_seed }
+    }
+
+    /// Returns the 256 block hashes relevant to a segment with the given block
+    /// number. BLOCKHASH can look back 256 blocks, so we select entries in
+    /// `[block_number - 256, block_number)`.
+    pub(crate) fn hashes_for_block(&self, block_number: u64) -> Vec<(u64, B256)> {
+        let min = block_number.saturating_sub(256);
+        self.block_hashes_to_seed
+            .iter()
+            .copied()
+            .filter(|(n, _)| *n >= min && *n < block_number)
+            .collect()
+    }
+}
+
+impl std::fmt::Debug for BbEvmPlan {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+        f.debug_struct("BbEvmPlan")
+            .field("segments", &self.segments)
+            .field("next_segment", &self.next_segment)
+            .field("tx_counter", &self.tx_counter)
+            .field("block_hashes_to_seed", &self.block_hashes_to_seed)
+            .finish()
+    }
+}
+
+// ---------------------------------------------------------------------------
+// BbBlockExecutor — handles segment boundaries
+// ---------------------------------------------------------------------------
+
+/// Function pointer that seeds block hashes into the DB's block hash cache.
+///
+/// Injected from `ConfigureEvm::create_executor` where the concrete `State<DB>`
+/// type is known, allowing `BbBlockExecutor` to reseed the ring buffer at
+/// segment boundaries without requiring additional trait bounds on `DB`.
+pub(crate) type BlockHashSeeder<DB> = fn(&mut DB, &[(u64, B256)]);
+
+/// Block executor that wraps [`EthBlockExecutor`] and handles segment-boundary
+/// changes for big-block execution.
+///
+/// At segment boundaries, the inner executor is finished (applying its
+/// end-of-block logic: post-execution system calls, withdrawal balance
+/// increments) and a new one is constructed for the next segment (applying
+/// its start-of-block logic: EIP-2935/EIP-4788 system calls).
+///
+/// Gas counters reset at each boundary so that each segment's real gas limit
+/// is used (preserving correct GASLIMIT opcode behavior). Accumulated offsets
+/// are applied to receipts and totals in `finish()`.
+pub(crate) struct BbBlockExecutor<'a, DB, I, P, Spec>
+where
+    DB: Database,
+{
+    /// The inner executor. `None` transiently during `apply_segment_boundary`.
+    inner: Option<EthBlockExecutor<'a, EthEvm<DB, I, P>, Spec, RethReceiptBuilder>>,
+    plan: Option<BbEvmPlan>,
+    /// Requests accumulated from segments that have been finished at
+    /// boundaries. Merged into the final result in `finish()`.
+    accumulated_requests: Requests,
+    /// Cumulative gas used by all segments that have been finished at
+    /// boundaries. Added to receipts and the final gas total in `finish()`.
+    gas_used_offset: u64,
+    /// Cumulative blob gas used by all segments that have been finished at
+    /// boundaries.
+    blob_gas_used_offset: u64,
+    /// Shared state hook that survives inner executor finish/reconstruct
+    /// cycles at segment boundaries. Each inner executor receives a
+    /// forwarding hook that delegates to this shared instance.
+    shared_hook: Arc<Mutex<Option<Box<dyn OnStateHook>>>>,
+    /// Callback to reseed block hashes into the DB's cache at segment
+    /// boundaries. See [`BlockHashSeeder`].
+    block_hash_seeder: Option<BlockHashSeeder<DB>>,
+}
+
+impl<'a, DB, I, P, Spec> BbBlockExecutor<'a, DB, I, P, Spec>
+where
+    DB: StateDB,
+    I: Inspector<EthEvmContext<DB>>,
+    P: PrecompileProvider<EthEvmContext<DB>, Output = InterpreterResult>,
+    Spec: alloy_evm::eth::spec::EthExecutorSpec + Clone,
+    EthEvm<DB, I, P>: Evm<
+        DB = DB,
+        Tx = TxEnv,
+        HaltReason = HaltReason,
+        Error = EVMError<DB::Error>,
+        Spec = SpecId,
+        BlockEnv = BlockEnv,
+    >,
+    TxEnv: FromRecoveredTx<TransactionSigned> + FromTxWithEncoded<TransactionSigned>,
+{
+    pub(crate) fn new(
+        evm: EthEvm<DB, I, P>,
+        ctx: EthBlockExecutionCtx<'a>,
+        spec: Spec,
+        receipt_builder: RethReceiptBuilder,
+        plan: Option<BbEvmPlan>,
+        block_hash_seeder: Option<BlockHashSeeder<DB>>,
+    ) -> Self {
+        let inner = EthBlockExecutor::new(evm, ctx, spec, receipt_builder);
+        Self {
+            inner: Some(inner),
+            plan,
+            accumulated_requests: Requests::default(),
+            gas_used_offset: 0,
+            blob_gas_used_offset: 0,
+            shared_hook: Arc::new(Mutex::new(None)),
+            block_hash_seeder,
+        }
+    }
+
+    /// Creates a forwarding `OnStateHook` that delegates to the shared hook.
+    fn forwarding_hook(&self) -> Option<Box<dyn OnStateHook>> {
+        let shared = self.shared_hook.clone();
+        Some(Box::new(move |source: StateChangeSource, state: &revm::state::EvmState| {
+            if let Some(hook) = shared.lock().unwrap().as_mut() {
+                hook.on_state(source, state);
+            }
+        }))
+    }
+
+    const fn inner(&self) -> &EthBlockExecutor<'a, EthEvm<DB, I, P>, Spec, RethReceiptBuilder> {
+        self.inner.as_ref().expect("inner executor must exist")
+    }
+
+    const fn inner_mut(
+        &mut self,
+    ) -> &mut EthBlockExecutor<'a, EthEvm<DB, I, P>, Spec, RethReceiptBuilder> {
+        self.inner.as_mut().expect("inner executor must exist")
+    }
+
+    fn reseed_block_hashes_for(&mut self, block_number: u64) {
+        let Some(seeder) = self.block_hash_seeder else { return };
+        let hashes = match &self.plan {
+            Some(plan) => plan.hashes_for_block(block_number),
+            None => return,
+        };
+        seeder(self.inner_mut().evm_mut().db_mut(), &hashes);
+    }
+
+    fn maybe_apply_boundary(&mut self) -> Result<(), BlockExecutionError> {
+        loop {
+            let plan = match &self.plan {
+                Some(p) => p,
+                None => return Ok(()),
+            };
+
+            if plan.next_segment >= plan.segments.len() ||
+                plan.tx_counter != plan.segments[plan.next_segment].start_tx
+            {
+                return Ok(());
+            }
+
+            self.apply_segment_boundary()?;
+        }
+    }
+
+    fn apply_segment_boundary(&mut self) -> Result<(), BlockExecutionError> {
+        let plan = self.plan.as_mut().expect("plan must exist");
+        let seg_idx = plan.next_segment;
+        let prev_seg_idx = seg_idx - 1;
+
+        debug!(
+            target: "engine::bb::evm",
+            seg_idx,
+            tx_counter = plan.tx_counter,
+            "Applying segment boundary"
+        );
+
+        // Swap the inner executor's ctx to the finished segment's values so
+        // that finish() applies the correct withdrawals and post-execution
+        // system calls for that segment.
+        let prev_segment = &plan.segments[prev_seg_idx];
+        let prev_ctx = EthBlockExecutionCtx {
+            parent_hash: prev_segment.ctx.parent_hash,
+            parent_beacon_block_root: prev_segment.ctx.parent_beacon_block_root,
+            ommers: prev_segment.ctx.ommers,
+            withdrawals: prev_segment.ctx.withdrawals.clone(),
+            extra_data: prev_segment.ctx.extra_data.clone(),
+            tx_count_hint: prev_segment.ctx.tx_count_hint,
+        };
+
+        // Clone the next segment's data before we consume inner.
+        let new_segment = &plan.segments[seg_idx];
+        let new_block_env = new_segment.evm_env.block_env.clone();
+        let mut new_cfg_env = new_segment.evm_env.cfg_env.clone();
+        new_cfg_env.disable_base_fee = true;
+        let new_ctx = EthBlockExecutionCtx {
+            parent_hash: new_segment.ctx.parent_hash,
+            parent_beacon_block_root: new_segment.ctx.parent_beacon_block_root,
+            ommers: new_segment.ctx.ommers,
+            withdrawals: new_segment.ctx.withdrawals.clone(),
+            extra_data: new_segment.ctx.extra_data.clone(),
+            tx_count_hint: new_segment.ctx.tx_count_hint,
+        };
+
+        plan.next_segment += 1;
+
+        // Finish the inner executor for the completed segment. This applies
+        // post-execution system calls (EIP-7002/7251) and withdrawal balance
+        // increments via EthBlockExecutor::finish().
+        let mut inner = self.inner.take().expect("inner executor must exist");
+        inner.ctx = prev_ctx;
+        let spec = inner.spec.clone();
+        let receipt_builder = inner.receipt_builder;
+        let (mut evm, result) = inner.finish()?;
+
+        // Receipts already have globally-correct cumulative_gas_used (fixed
+        // up in commit_transaction). Update the offset with this segment's
+        // gas so that subsequent segments' receipts are adjusted correctly.
+        self.gas_used_offset += result.gas_used;
+        self.blob_gas_used_offset += result.blob_gas_used;
+        self.accumulated_requests.extend(result.requests);
+
+        let last_receipt_cumulative =
+            result.receipts.last().map(|r| r.cumulative_gas_used).unwrap_or(0);
+        let seg_block_number = prev_segment.evm_env.block_env.number.saturating_to::<u64>();
+        debug!(
+            target: "engine::bb::evm",
+            prev_seg_idx,
+            seg_block_number,
+            segment_gas_used = result.gas_used,
+            gas_used_offset = self.gas_used_offset,
+            last_receipt_cumulative,
+            receipt_count = result.receipts.len(),
+            "Finished segment"
+        );
+
+        // Swap EVM env to the next segment's values (using real gas_limit).
+        let ctx = evm.ctx_mut();
+        ctx.block = new_block_env;
+        ctx.cfg = new_cfg_env;
+
+        // Build a new inner executor for the next segment. gas_used starts
+        // at 0 so the per-transaction gas check uses this segment's real
+        // gas_limit correctly.
+        let mut new_inner = EthBlockExecutor::new(evm, new_ctx, spec, receipt_builder);
+
+        // Carry forward receipts from prior segments.
+        new_inner.receipts = result.receipts;
+
+        // Re-install the forwarding state hook so the parallel state root
+        // task continues to receive state changes.
+        if self.shared_hook.lock().unwrap().is_some() {
+            new_inner.set_state_hook(self.forwarding_hook());
+        }
+
+        self.inner = Some(new_inner);
+
+        // Reseed the block hash cache for the new segment's 256-block window
+        // before applying pre-execution changes (which may use BLOCKHASH).
+        let new_block_number = self.plan.as_ref().unwrap().segments[seg_idx]
+            .evm_env
+            .block_env
+            .number
+            .saturating_to::<u64>();
+        self.reseed_block_hashes_for(new_block_number);
+
+        // Apply pre-execution changes for the new segment (EIP-2935, EIP-4788).
+        self.inner_mut().apply_pre_execution_changes()?;
+
+        trace!(target: "engine::bb::evm", "Started segment {seg_idx}");
+
+        Ok(())
+    }
+}
+
+impl<'a, DB, I, P, Spec> BlockExecutor for BbBlockExecutor<'a, DB, I, P, Spec>
+where
+    DB: StateDB,
+    I: Inspector<EthEvmContext<DB>>,
+    P: PrecompileProvider<EthEvmContext<DB>, Output = InterpreterResult>,
+    Spec: alloy_evm::eth::spec::EthExecutorSpec + Clone,
+    EthEvm<DB, I, P>: Evm<
+        DB = DB,
+        Tx = TxEnv,
+        HaltReason = HaltReason,
+        Error = EVMError<DB::Error>,
+        Spec = SpecId,
+        BlockEnv = BlockEnv,
+    >,
+    TxEnv: FromRecoveredTx<TransactionSigned> + FromTxWithEncoded<TransactionSigned>,
+{
+    type Transaction = TransactionSigned;
+    type Receipt = Receipt;
+    type Evm = EthEvm<DB, I, P>;
+    type Result = EthTxResult<HaltReason, alloy_consensus::TxType>;
+
+    fn apply_pre_execution_changes(&mut self) -> Result<(), BlockExecutionError> {
+        // Swap the EVM's block_env and executor ctx to the first segment's
+        // values so that the initial EIP-2935/EIP-4788 system calls use the
+        // correct block number and parent hash. Without this, the outer big
+        // block header's block_number (which is synthetic) would be used,
+        // writing to wrong EIP-2935 slots and corrupting state.
+        if let Some(seg0) = self.plan.as_ref().map(|p| &p.segments[0]) {
+            let block_env = seg0.evm_env.block_env.clone();
+            let block_number = block_env.number.saturating_to::<u64>();
+            let mut cfg_env = seg0.evm_env.cfg_env.clone();
+            cfg_env.disable_base_fee = true;
+            let seg0_ctx = EthBlockExecutionCtx {
+                parent_hash: seg0.ctx.parent_hash,
+                parent_beacon_block_root: seg0.ctx.parent_beacon_block_root,
+                ommers: seg0.ctx.ommers,
+                withdrawals: seg0.ctx.withdrawals.clone(),
+                extra_data: seg0.ctx.extra_data.clone(),
+                tx_count_hint: seg0.ctx.tx_count_hint,
+            };
+
+            let inner = self.inner_mut();
+            let evm_ctx = inner.evm.ctx_mut();
+            evm_ctx.block = block_env;
+            evm_ctx.cfg = cfg_env;
+            inner.ctx = seg0_ctx;
+
+            self.reseed_block_hashes_for(block_number);
+        }
+
+        self.inner_mut().apply_pre_execution_changes()
+    }
+
+    fn execute_transaction_without_commit(
+        &mut self,
+        tx: impl ExecutableTx<Self>,
+    ) -> Result<Self::Result, BlockExecutionError> {
+        self.maybe_apply_boundary()?;
+        self.inner_mut().execute_transaction_without_commit(tx)
+    }
+
+    fn commit_transaction(&mut self, output: Self::Result) -> Result<u64, BlockExecutionError> {
+        let gas_used = self.inner_mut().commit_transaction(output)?;
+
+        // Fix up cumulative_gas_used on the just-committed receipt so that
+        // the receipt root task (which reads receipts incrementally) sees
+        // globally-correct values across all segments.
+        let offset = self.gas_used_offset;
+        if offset > 0 &&
+            let Some(receipt) = self.inner_mut().receipts.last_mut()
+        {
+            receipt.cumulative_gas_used += offset;
+        }
+
+        if let Some(plan) = &mut self.plan {
+            plan.tx_counter += 1;
+        }
+        Ok(gas_used)
+    }
+
+    fn finish(
+        mut self,
+    ) -> Result<(Self::Evm, BlockExecutionResult<Self::Receipt>), BlockExecutionError> {
+        // Swap the inner executor's ctx to the last segment's ctx so that
+        // EthBlockExecutor::finish() applies the correct withdrawal balance
+        // increments and post-execution system calls.
+        if let Some(last_seg) = self.plan.as_ref().map(|p| p.segments.last().unwrap()) {
+            let last_ctx = EthBlockExecutionCtx {
+                parent_hash: last_seg.ctx.parent_hash,
+                parent_beacon_block_root: last_seg.ctx.parent_beacon_block_root,
+                ommers: last_seg.ctx.ommers,
+                withdrawals: last_seg.ctx.withdrawals.clone(),
+                extra_data: last_seg.ctx.extra_data.clone(),
+                tx_count_hint: last_seg.ctx.tx_count_hint,
+            };
+            self.inner_mut().ctx = last_ctx;
+        }
+        let inner = self.inner.take().expect("inner executor must exist");
+        let (evm, mut result) = inner.finish()?;
+
+        // Receipts already have globally-correct cumulative_gas_used (fixed
+        // up in commit_transaction). Add the offset to the totals so they
+        // reflect gas across all segments.
+        let last_segment_gas = result.gas_used;
+        result.gas_used += self.gas_used_offset;
+        result.blob_gas_used += self.blob_gas_used_offset;
+
+        let last_receipt_cumulative =
+            result.receipts.last().map(|r| r.cumulative_gas_used).unwrap_or(0);
+        debug!(
+            target: "engine::bb::evm",
+            last_segment_gas,
+            gas_used_offset = self.gas_used_offset,
+            total_gas_used = result.gas_used,
+            last_receipt_cumulative,
+            receipt_count = result.receipts.len(),
+            "Finished final segment"
+        );
+
+        // Merge requests accumulated from earlier segment boundaries into
+        // the final result.
+        if !self.accumulated_requests.is_empty() {
+            let mut merged = self.accumulated_requests;
+            merged.extend(result.requests);
+            result.requests = merged;
+        }
+
+        Ok((evm, result))
+    }
+
+    fn set_state_hook(&mut self, hook: Option<Box<dyn OnStateHook>>) {
+        if self.plan.is_some() {
+            // Store the real hook in the shared slot and give the inner
+            // executor a lightweight forwarder. This way the hook survives
+            // inner executor finish/reconstruct cycles at segment boundaries.
+            *self.shared_hook.lock().unwrap() = hook;
+            let fwd = self.forwarding_hook();
+            self.inner_mut().set_state_hook(fwd);
+        } else {
+            self.inner_mut().set_state_hook(hook);
+        }
+    }
+
+    fn evm_mut(&mut self) -> &mut Self::Evm {
+        self.inner_mut().evm_mut()
+    }
+
+    fn evm(&self) -> &Self::Evm {
+        self.inner().evm()
+    }
+
+    fn receipts(&self) -> &[Self::Receipt] {
+        self.inner().receipts()
+    }
+}
+
+// ---------------------------------------------------------------------------
+// BbBlockExecutorFactory
+// ---------------------------------------------------------------------------
+
+/// Block executor factory that produces [`BbBlockExecutor`] for
+/// boundary-aware big-block execution.
+#[derive(Debug, Clone)]
+pub struct BbBlockExecutorFactory<Spec> {
+    receipt_builder: RethReceiptBuilder,
+    spec: Spec,
+    evm_factory: EthEvmFactory,
+    /// Staged plan consumed by the next [`BbBlockExecutor`].
+    pub(crate) staged_plan: Arc<Mutex<Option<BbEvmPlan>>>,
+}
+
+impl<Spec> BbBlockExecutorFactory<Spec> {
+    pub fn new(
+        receipt_builder: RethReceiptBuilder,
+        spec: Spec,
+        evm_factory: EthEvmFactory,
+    ) -> Self {
+        Self { receipt_builder, spec, evm_factory, staged_plan: Arc::new(Mutex::new(None)) }
+    }
+
+    pub const fn evm_factory(&self) -> &EthEvmFactory {
+        &self.evm_factory
+    }
+
+    pub const fn spec(&self) -> &Spec {
+        &self.spec
+    }
+
+    pub const fn receipt_builder(&self) -> &RethReceiptBuilder {
+        &self.receipt_builder
+    }
+
+    pub(crate) fn stage_plan(&self, plan: BbEvmPlan) {
+        *self.staged_plan.lock().unwrap() = Some(plan);
+    }
+
+    fn take_plan(&self) -> Option<BbEvmPlan> {
+        self.staged_plan.lock().unwrap().take()
+    }
+
+    pub(crate) fn create_executor_with_seeder<'a, DB, I>(
+        &'a self,
+        evm: EthEvm<DB, I, PrecompilesMap>,
+        ctx: EthBlockExecutionCtx<'a>,
+        block_hash_seeder: Option<BlockHashSeeder<DB>>,
+    ) -> BbBlockExecutor<'a, DB, I, PrecompilesMap, &'a Spec>
+    where
+        Spec: alloy_evm::eth::spec::EthExecutorSpec,
+        DB: StateDB + 'a,
+        I: Inspector<EthEvmContext<DB>> + 'a,
+    {
+        let plan = self.take_plan();
+        BbBlockExecutor::new(evm, ctx, &self.spec, self.receipt_builder, plan, block_hash_seeder)
+    }
+}
+
+impl<Spec> BlockExecutorFactory for BbBlockExecutorFactory<Spec>
+where
+    Spec: alloy_evm::eth::spec::EthExecutorSpec + 'static,
+    TxEnv: FromRecoveredTx<TransactionSigned> + FromTxWithEncoded<TransactionSigned>,
+{
+    type EvmFactory = EthEvmFactory;
+    type ExecutionCtx<'a> = EthBlockExecutionCtx<'a>;
+    type Transaction = TransactionSigned;
+    type Receipt = Receipt;
+
+    fn evm_factory(&self) -> &Self::EvmFactory {
+        &self.evm_factory
+    }
+
+    fn create_executor<'a, DB, I>(
+        &'a self,
+        evm: EthEvm<DB, I, PrecompilesMap>,
+        ctx: EthBlockExecutionCtx<'a>,
+    ) -> impl BlockExecutorFor<'a, Self, DB, I>
+    where
+        DB: StateDB + 'a,
+        I: Inspector<EthEvmContext<DB>> + 'a,
+    {
+        let plan = self.take_plan();
+        BbBlockExecutor::new(evm, ctx, &self.spec, self.receipt_builder, plan, None)
+    }
+}

bin/reth-bb/src/evm_config.rs

@@ -0,0 +1,291 @@
+//! Big-block EVM configuration.
+//!
+//! Wraps [`EthEvmConfig`] to create executors that handle multi-segment
+//! big-block execution internally. At transaction boundaries defined by
+//! [`BigBlockData`], the executor swaps the EVM environment (block env,
+//! cfg env) and applies pre/post execution changes for each segment.
+
+pub(crate) use reth_engine_primitives::BigBlockData;
+
+use crate::{
+    evm::{BbBlockExecutorFactory, BbEvmPlan},
+    BigBlockMap,
+};
+use alloy_consensus::Header;
+use alloy_evm::eth::EthBlockExecutionCtx;
+use alloy_primitives::B256;
+use alloy_rpc_types::engine::ExecutionData;
+use core::convert::Infallible;
+use reth_chainspec::{ChainSpec, EthChainSpec};
+use reth_ethereum_forks::Hardforks;
+use reth_ethereum_primitives::EthPrimitives;
+use reth_evm::{
+    ConfigureEngineEvm, ConfigureEvm, Database, EvmEnv, ExecutableTxIterator,
+    NextBlockEnvAttributes,
+};
+use reth_evm_ethereum::{EthBlockAssembler, EthEvmConfig, RethReceiptBuilder};
+use reth_primitives_traits::{SealedBlock, SealedHeader};
+use revm::primitives::hardfork::SpecId;
+use std::sync::Arc;
+use tracing::debug;
+
+use alloy_evm::{eth::spec::EthExecutorSpec, EthEvmFactory};
+use reth_evm::{EvmEnvFor, ExecutionCtxFor};
+
+// ---------------------------------------------------------------------------
+// Execution plan types
+// ---------------------------------------------------------------------------
+
+/// A single execution segment within a big block.
+#[derive(Debug, Clone)]
+pub(crate) struct BigBlockSegment {
+    /// Transaction index at which this segment starts.
+    pub start_tx: usize,
+    /// The EVM environment for this segment.
+    pub evm_env: EvmEnv,
+    /// The execution context for this segment.
+    pub ctx: EthBlockExecutionCtx<'static>,
+}
+
+// ---------------------------------------------------------------------------
+// BbEvmConfig
+// ---------------------------------------------------------------------------
+
+/// EVM configuration for big-block execution.
+///
+/// Wraps [`EthEvmConfig`] and a shared [`BigBlockMap`]. When a big-block
+/// payload is received, the plan is staged on the [`BbBlockExecutorFactory`]
+/// and consumed when the executor is created. Block hashes for inter-segment
+/// BLOCKHASH resolution are reseeded into `State::block_hashes` at each
+/// segment boundary via a [`BlockHashSeeder`](crate::evm::BlockHashSeeder)
+/// callback injected in [`ConfigureEvm::create_executor`].
+#[derive(Debug, Clone)]
+pub struct BbEvmConfig<C = ChainSpec> {
+    /// The inner Ethereum EVM configuration (used for env computation).
+    pub inner: EthEvmConfig<C>,
+    /// Shared map of pending big-block metadata.
+    pub pending: BigBlockMap,
+    /// Block executor factory for big-block execution.
+    executor_factory: BbBlockExecutorFactory<Arc<C>>,
+    /// Block assembler.
+    block_assembler: EthBlockAssembler<C>,
+}
+
+impl<C> BbEvmConfig<C> {
+    /// Creates a new big-block EVM configuration.
+    pub fn new(inner: EthEvmConfig<C>, pending: BigBlockMap) -> Self
+    where
+        C: Clone,
+    {
+        let chain_spec = inner.chain_spec().clone();
+        let executor_factory = BbBlockExecutorFactory::new(
+            RethReceiptBuilder::default(),
+            chain_spec,
+            EthEvmFactory::default(),
+        );
+        let block_assembler = inner.block_assembler.clone();
+
+        Self { inner, pending, executor_factory, block_assembler }
+    }
+}
+
+// ---------------------------------------------------------------------------
+// Block hash seeder for State<DB>
+// ---------------------------------------------------------------------------
+
+/// Reseeds `State::block_hashes` with the given hashes.
+///
+/// This is used as a [`BlockHashSeeder`](crate::evm::BlockHashSeeder) callback,
+/// injected into [`BbBlockExecutor`](crate::evm::BbBlockExecutor) from
+/// `ConfigureEvm::create_executor` where the concrete `State<DB>` type is known.
+/// At each segment boundary the executor calls this to populate the ring buffer
+/// with the 256 block hashes relevant to the new segment's block number window.
+fn seed_state_block_hashes<DB>(state: &mut &mut revm::database::State<DB>, hashes: &[(u64, B256)]) {
+    for &(number, hash) in hashes {
+        state.block_hashes.insert(number, hash);
+    }
+}
+
+// ---------------------------------------------------------------------------
+// ConfigureEvm
+// ---------------------------------------------------------------------------
+
+impl<C> ConfigureEvm for BbEvmConfig<C>
+where
+    C: EthExecutorSpec + EthChainSpec<Header = Header> + Hardforks + 'static,
+{
+    type Primitives = EthPrimitives;
+    type Error = Infallible;
+    type NextBlockEnvCtx = NextBlockEnvAttributes;
+    type BlockExecutorFactory = BbBlockExecutorFactory<Arc<C>>;
+    type BlockAssembler = EthBlockAssembler<C>;
+
+    fn block_executor_factory(&self) -> &Self::BlockExecutorFactory {
+        &self.executor_factory
+    }
+
+    fn block_assembler(&self) -> &Self::BlockAssembler {
+        &self.block_assembler
+    }
+
+    fn evm_env(&self, header: &Header) -> Result<EvmEnv<SpecId>, Self::Error> {
+        self.inner.evm_env(header)
+    }
+
+    fn next_evm_env(
+        &self,
+        parent: &Header,
+        attributes: &NextBlockEnvAttributes,
+    ) -> Result<EvmEnv, Self::Error> {
+        self.inner.next_evm_env(parent, attributes)
+    }
+
+    fn context_for_block<'a>(
+        &self,
+        block: &'a SealedBlock<reth_ethereum_primitives::Block>,
+    ) -> Result<EthBlockExecutionCtx<'a>, Self::Error> {
+        self.inner.context_for_block(block)
+    }
+
+    fn context_for_next_block(
+        &self,
+        parent: &SealedHeader,
+        attributes: Self::NextBlockEnvCtx,
+    ) -> Result<EthBlockExecutionCtx<'_>, Self::Error> {
+        self.inner.context_for_next_block(parent, attributes)
+    }
+
+    fn create_executor<'a, DB, I>(
+        &'a self,
+        evm: reth_evm::EvmFor<Self, &'a mut revm::database::State<DB>, I>,
+        ctx: EthBlockExecutionCtx<'a>,
+    ) -> impl alloy_evm::block::BlockExecutorFor<
+        'a,
+        Self::BlockExecutorFactory,
+        &'a mut revm::database::State<DB>,
+        I,
+    >
+    where
+        DB: Database,
+        I: reth_evm::InspectorFor<Self, &'a mut revm::database::State<DB>> + 'a,
+    {
+        // Use create_executor_with_seeder to inject a concrete seeder that
+        // can reseed State::block_hashes at segment boundaries. The seeder
+        // is a function pointer that knows the concrete State<DB> type,
+        // allowing the generic BbBlockExecutor to reseed without additional
+        // trait bounds on DB.
+        self.executor_factory.create_executor_with_seeder(
+            evm,
+            ctx,
+            Some(seed_state_block_hashes::<DB>),
+        )
+    }
+}
+
+// ---------------------------------------------------------------------------
+// ConfigureEngineEvm — intercepts payload methods for big blocks
+// ---------------------------------------------------------------------------
+
+impl<C> ConfigureEngineEvm<ExecutionData> for BbEvmConfig<C>
+where
+    C: EthExecutorSpec + EthChainSpec<Header = Header> + Hardforks + 'static,
+{
+    fn evm_env_for_payload(&self, payload: &ExecutionData) -> Result<EvmEnvFor<Self>, Self::Error> {
+        let payload_hash = payload.block_hash();
+        let has_plan = self.pending.lock().unwrap().contains_key(&payload_hash);
+
+        if has_plan {
+            // Compute the env from the first segment BEFORE removing the
+            // entry (stage_plan_for_payload removes it).
+            let first_exec_data = {
+                let pending = self.pending.lock().unwrap();
+                let bb_data = pending.get(&payload_hash).unwrap();
+                bb_data.env_switches[0].1.clone()
+            };
+            let mut env = self.inner.evm_env_for_payload(&first_exec_data)?;
+
+            // Disable basefee validation: transactions from different
+            // original blocks may have gas prices below the big block's
+            // effective basefee.
+            env.cfg_env.disable_base_fee = true;
+
+            // Now stage the plan on the factory (removes the entry).
+            self.stage_plan_for_payload(&payload_hash);
+
+            Ok(env)
+        } else {
+            self.inner.evm_env_for_payload(payload)
+        }
+    }
+
+    fn context_for_payload<'a>(
+        &self,
+        payload: &'a ExecutionData,
+    ) -> Result<ExecutionCtxFor<'a, Self>, Self::Error> {
+        self.inner.context_for_payload(payload)
+    }
+
+    fn tx_iterator_for_payload(
+        &self,
+        payload: &ExecutionData,
+    ) -> Result<impl ExecutableTxIterator<Self>, Self::Error> {
+        self.inner.tx_iterator_for_payload(payload)
+    }
+}
+
+// ---------------------------------------------------------------------------
+// Plan construction and staging
+// ---------------------------------------------------------------------------
+
+impl<C> BbEvmConfig<C>
+where
+    C: EthExecutorSpec + EthChainSpec<Header = Header> + Hardforks + 'static,
+{
+    /// Takes the big-block plan for a payload hash, builds a [`BbEvmPlan`],
+    /// and stages it on the factory.
+    ///
+    /// Must be called before `evm_with_env` is invoked for this payload.
+    /// In practice, this is called from `evm_env_for_payload` in the
+    /// engine pipeline.
+    pub fn stage_plan_for_payload(&self, payload_hash: &B256) {
+        let bb = match self.pending.lock().unwrap().remove(payload_hash) {
+            Some(bb) => bb,
+            None => return,
+        };
+
+        let segments: Vec<_> = bb
+            .env_switches
+            .into_iter()
+            .map(|(start_tx, exec_data)| {
+                let evm_env = self.inner.evm_env_for_payload(&exec_data).unwrap();
+                let ctx = self.inner.context_for_payload(&exec_data).unwrap();
+                let ctx = EthBlockExecutionCtx {
+                    tx_count_hint: ctx.tx_count_hint,
+                    parent_hash: ctx.parent_hash,
+                    parent_beacon_block_root: ctx.parent_beacon_block_root,
+                    ommers: &[],
+                    withdrawals: ctx.withdrawals.map(|w| std::borrow::Cow::Owned(w.into_owned())),
+                    extra_data: ctx.extra_data,
+                };
+                BigBlockSegment { start_tx, evm_env, ctx }
+            })
+            .collect();
+
+        debug!(
+            target: "engine::bb",
+            ?payload_hash,
+            segments = segments.len(),
+            seed_hashes = bb.prior_block_hashes.len(),
+            "Staging multi-segment plan"
+        );
+
+        let mut plan = BbEvmPlan::new(segments);
+
+        // Add prior block hashes to the seeding list.
+        plan.block_hashes_to_seed.extend(bb.prior_block_hashes);
+
+        plan.block_hashes_to_seed.sort_unstable_by_key(|(n, _)| *n);
+
+        self.executor_factory.stage_plan(plan);
+    }
+}

bin/reth-bb/src/main.rs

@@ -0,0 +1,374 @@
+//! reth-bb: a modified reth node for benchmarking big block execution.
+#![allow(missing_docs)]
+
+#[global_allocator]
+static ALLOC: reth_cli_util::allocator::Allocator = reth_cli_util::allocator::new_allocator();
+
+mod evm;
+mod evm_config;
+
+use alloy_primitives::B256;
+
+use alloy_rpc_types::engine::{ExecutionData, ForkchoiceState, ForkchoiceUpdated};
+use async_trait::async_trait;
+use clap::Parser;
+use evm_config::{BbEvmConfig, BigBlockData};
+use jsonrpsee::core::RpcResult;
+use reth_chainspec::{ChainSpec, EthChainSpec, EthereumHardforks, Hardforks};
+use reth_engine_primitives::ConsensusEngineHandle;
+use reth_ethereum_cli::{chainspec::EthereumChainSpecParser, interface::Cli};
+use reth_ethereum_consensus::EthBeaconConsensus;
+use reth_ethereum_primitives::EthPrimitives;
+use reth_evm_ethereum::EthEvmConfig;
+use reth_node_api::{AddOnsContext, FullNodeComponents, NodeTypes, PayloadTypes};
+use reth_node_builder::{
+    components::{
+        BasicPayloadServiceBuilder, ComponentsBuilder, ConsensusBuilder, ExecutorBuilder,
+    },
+    node::FullNodeTypes,
+    rpc::{
+        BasicEngineApiBuilder, BasicEngineValidatorBuilder, EngineApiBuilder, EngineValidatorAddOn,
+        EngineValidatorBuilder, PayloadValidatorBuilder, RethRpcAddOns, RpcAddOns, RpcHandle,
+        RpcHooks,
+    },
+    BuilderContext, Node,
+};
+use reth_node_ethereum::{
+    EthEngineTypes, EthereumEngineValidatorBuilder, EthereumEthApiBuilder, EthereumNetworkBuilder,
+    EthereumNode, EthereumPayloadBuilder, EthereumPoolBuilder,
+};
+use reth_payload_primitives::ExecutionPayload;
+use reth_primitives_traits::SealedBlock;
+use reth_provider::EthStorage;
+use reth_rpc_api::{RethNewPayloadInput, RethPayloadStatus};
+use reth_rpc_engine_api::EngineApiError;
+use std::{
+    collections::HashMap,
+    sync::{Arc, Mutex},
+};
+use tracing::{info, trace};
+
+/// Shared map for big block data, keyed by payload hash.
+pub type BigBlockMap = Arc<Mutex<HashMap<B256, BigBlockData<ExecutionData>>>>;
+
+// ---------------------------------------------------------------------------
+// Custom RPC trait for big-block payloads
+// ---------------------------------------------------------------------------
+
+/// Big-block extension of the `reth_` engine API.
+#[jsonrpsee::proc_macros::rpc(server, namespace = "reth")]
+pub trait BbRethEngineApi {
+    /// `reth_newPayload` with optional big-block data.
+    #[method(name = "newPayload")]
+    async fn reth_new_payload(
+        &self,
+        payload: RethNewPayloadInput<ExecutionData>,
+        wait_for_persistence: Option<bool>,
+        wait_for_caches: Option<bool>,
+        big_block_data: Option<BigBlockData<ExecutionData>>,
+    ) -> RpcResult<RethPayloadStatus>;
+
+    /// `reth_forkchoiceUpdated` – pass-through.
+    #[method(name = "forkchoiceUpdated")]
+    async fn reth_forkchoice_updated(
+        &self,
+        forkchoice_state: ForkchoiceState,
+    ) -> RpcResult<ForkchoiceUpdated>;
+}
+
+/// Server-side implementation of `BbRethEngineApi`.
+#[derive(Debug)]
+struct BbRethEngineApiHandler {
+    pending: BigBlockMap,
+    engine: ConsensusEngineHandle<EthEngineTypes>,
+}
+
+#[async_trait]
+impl BbRethEngineApiServer for BbRethEngineApiHandler {
+    async fn reth_new_payload(
+        &self,
+        input: RethNewPayloadInput<ExecutionData>,
+        wait_for_persistence: Option<bool>,
+        wait_for_caches: Option<bool>,
+        big_block_data: Option<BigBlockData<ExecutionData>>,
+    ) -> RpcResult<RethPayloadStatus> {
+        let wait_for_persistence = wait_for_persistence.unwrap_or(true);
+        let wait_for_caches = wait_for_caches.unwrap_or(true);
+        trace!(
+            target: "rpc::engine",
+            wait_for_persistence,
+            wait_for_caches,
+            has_big_block_data = big_block_data.is_some(),
+            "Serving bb reth_newPayload"
+        );
+
+        let payload = match input {
+            RethNewPayloadInput::ExecutionData(data) => data,
+            RethNewPayloadInput::BlockRlp(rlp) => {
+                let block = alloy_rlp::Decodable::decode(&mut rlp.as_ref())
+                    .map_err(|err| EngineApiError::Internal(Box::new(err)))?;
+                <EthEngineTypes as PayloadTypes>::block_to_payload(SealedBlock::new_unhashed(block))
+            }
+        };
+
+        if let Some(data) = big_block_data {
+            let hash = ExecutionPayload::block_hash(&payload);
+            self.pending.lock().unwrap().insert(hash, data);
+        }
+
+        let (status, timings) = self
+            .engine
+            .reth_new_payload(payload, wait_for_persistence, wait_for_caches)
+            .await
+            .map_err(EngineApiError::from)?;
+
+        Ok(RethPayloadStatus {
+            status,
+            latency_us: timings.latency.as_micros() as u64,
+            persistence_wait_us: timings.persistence_wait.map(|d| d.as_micros() as u64),
+            execution_cache_wait_us: timings.execution_cache_wait.map(|d| d.as_micros() as u64),
+            sparse_trie_wait_us: timings.sparse_trie_wait.map(|d| d.as_micros() as u64),
+        })
+    }
+
+    async fn reth_forkchoice_updated(
+        &self,
+        forkchoice_state: ForkchoiceState,
+    ) -> RpcResult<ForkchoiceUpdated> {
+        trace!(target: "rpc::engine", "Serving reth_forkchoiceUpdated");
+        self.engine
+            .fork_choice_updated(forkchoice_state, None)
+            .await
+            .map_err(|e| EngineApiError::from(e).into())
+    }
+}
+
+// ---------------------------------------------------------------------------
+// Node add-ons wrapper
+// ---------------------------------------------------------------------------
+
+/// Add-ons for the big-block node.
+#[derive(Debug)]
+pub struct BbAddOns {
+    pending: BigBlockMap,
+}
+
+impl BbAddOns {
+    const fn new(pending: BigBlockMap) -> Self {
+        Self { pending }
+    }
+
+    fn make_rpc_add_ons<N: FullNodeComponents>(
+        &self,
+    ) -> RpcAddOns<
+        N,
+        EthereumEthApiBuilder,
+        EthereumEngineValidatorBuilder,
+        BasicEngineApiBuilder<EthereumEngineValidatorBuilder>,
+        BasicEngineValidatorBuilder<EthereumEngineValidatorBuilder>,
+    >
+    where
+        EthereumEthApiBuilder: reth_node_builder::rpc::EthApiBuilder<N>,
+    {
+        RpcAddOns::new(
+            EthereumEthApiBuilder::default(),
+            EthereumEngineValidatorBuilder::default(),
+            BasicEngineApiBuilder::default(),
+            BasicEngineValidatorBuilder::default(),
+            Default::default(),
+        )
+    }
+}
+
+impl<N> reth_node_api::NodeAddOns<N> for BbAddOns
+where
+    N: FullNodeComponents<
+        Types: NodeTypes<
+            ChainSpec: EthereumHardforks + Hardforks + Clone + 'static,
+            Payload = EthEngineTypes,
+            Primitives = EthPrimitives,
+        >,
+    >,
+    EthereumEthApiBuilder: reth_node_builder::rpc::EthApiBuilder<N>,
+    EthereumEngineValidatorBuilder: PayloadValidatorBuilder<N>,
+    BasicEngineApiBuilder<EthereumEngineValidatorBuilder>: EngineApiBuilder<N>,
+    BasicEngineValidatorBuilder<EthereumEngineValidatorBuilder>: EngineValidatorBuilder<N>,
+{
+    type Handle =
+        RpcHandle<N, <EthereumEthApiBuilder as reth_node_builder::rpc::EthApiBuilder<N>>::EthApi>;
+
+    async fn launch_add_ons(self, ctx: AddOnsContext<'_, N>) -> eyre::Result<Self::Handle> {
+        let engine_handle = ctx.beacon_engine_handle.clone();
+        let pending = self.pending.clone();
+        let rpc_add_ons = self.make_rpc_add_ons::<N>();
+
+        rpc_add_ons
+            .launch_add_ons_with(ctx, move |container| {
+                let handler = BbRethEngineApiHandler { pending, engine: engine_handle };
+                let bb_module = BbRethEngineApiServer::into_rpc(handler);
+                container.auth_module.replace_auth_methods(bb_module.remove_context())?;
+                Ok(())
+            })
+            .await
+    }
+}
+
+impl<N> RethRpcAddOns<N> for BbAddOns
+where
+    N: FullNodeComponents<
+        Types: NodeTypes<
+            ChainSpec: EthereumHardforks + Hardforks + Clone + 'static,
+            Payload = EthEngineTypes,
+            Primitives = EthPrimitives,
+        >,
+    >,
+    EthereumEthApiBuilder: reth_node_builder::rpc::EthApiBuilder<N>,
+    EthereumEngineValidatorBuilder: PayloadValidatorBuilder<N>,
+    BasicEngineApiBuilder<EthereumEngineValidatorBuilder>: EngineApiBuilder<N>,
+    BasicEngineValidatorBuilder<EthereumEngineValidatorBuilder>: EngineValidatorBuilder<N>,
+{
+    type EthApi = <EthereumEthApiBuilder as reth_node_builder::rpc::EthApiBuilder<N>>::EthApi;
+
+    fn hooks_mut(&mut self) -> &mut RpcHooks<N, Self::EthApi> {
+        unimplemented!("BbAddOns does not support dynamic hook mutation")
+    }
+}
+
+impl<N> EngineValidatorAddOn<N> for BbAddOns
+where
+    N: FullNodeComponents,
+    BasicEngineValidatorBuilder<EthereumEngineValidatorBuilder>: EngineValidatorBuilder<N>,
+{
+    type ValidatorBuilder = BasicEngineValidatorBuilder<EthereumEngineValidatorBuilder>;
+
+    fn engine_validator_builder(&self) -> Self::ValidatorBuilder {
+        BasicEngineValidatorBuilder::default()
+    }
+}
+
+// ---------------------------------------------------------------------------
+// Custom executor builder
+// ---------------------------------------------------------------------------
+
+/// Executor builder that creates a [`BbEvmConfig`].
+#[derive(Debug)]
+pub struct BbExecutorBuilder {
+    pending: BigBlockMap,
+}
+
+impl<Node> ExecutorBuilder<Node> for BbExecutorBuilder
+where
+    Node: FullNodeTypes<
+        Types: NodeTypes<
+            ChainSpec: reth_ethereum_forks::Hardforks
+                           + alloy_evm::eth::spec::EthExecutorSpec
+                           + EthereumHardforks,
+            Primitives = EthPrimitives,
+        >,
+    >,
+{
+    type EVM = BbEvmConfig<<Node::Types as NodeTypes>::ChainSpec>;
+
+    async fn build_evm(self, ctx: &BuilderContext<Node>) -> eyre::Result<Self::EVM> {
+        Ok(BbEvmConfig::new(EthEvmConfig::new(ctx.chain_spec()), self.pending))
+    }
+}
+
+// ---------------------------------------------------------------------------
+// Node type
+// ---------------------------------------------------------------------------
+
+/// Node type for big block execution.
+#[derive(Debug, Clone)]
+pub struct BbNode {
+    pending: BigBlockMap,
+}
+
+impl BbNode {
+    const fn new(pending: BigBlockMap) -> Self {
+        Self { pending }
+    }
+}
+
+impl NodeTypes for BbNode {
+    type Primitives = EthPrimitives;
+    type ChainSpec = ChainSpec;
+    type Storage = EthStorage;
+    type Payload = EthEngineTypes;
+}
+
+impl<N> Node<N> for BbNode
+where
+    N: FullNodeTypes<Types = Self>,
+{
+    type ComponentsBuilder = ComponentsBuilder<
+        N,
+        EthereumPoolBuilder,
+        BasicPayloadServiceBuilder<EthereumPayloadBuilder>,
+        EthereumNetworkBuilder,
+        BbExecutorBuilder,
+        BbConsensusBuilder,
+    >;
+
+    type AddOns = BbAddOns;
+
+    fn components_builder(&self) -> Self::ComponentsBuilder {
+        EthereumNode::components()
+            .executor(BbExecutorBuilder { pending: self.pending.clone() })
+            .consensus(BbConsensusBuilder)
+    }
+
+    fn add_ons(&self) -> Self::AddOns {
+        BbAddOns::new(self.pending.clone())
+    }
+}
+
+// ---------------------------------------------------------------------------
+// Consensus builder
+// ---------------------------------------------------------------------------
+
+/// Consensus builder for big block execution.
+#[derive(Debug, Default, Clone, Copy)]
+pub struct BbConsensusBuilder;
+
+impl<Node> ConsensusBuilder<Node> for BbConsensusBuilder
+where
+    Node: FullNodeTypes<
+        Types: NodeTypes<ChainSpec: EthChainSpec + EthereumHardforks, Primitives = EthPrimitives>,
+    >,
+{
+    type Consensus = Arc<EthBeaconConsensus<<Node::Types as NodeTypes>::ChainSpec>>;
+
+    async fn build_consensus(self, ctx: &BuilderContext<Node>) -> eyre::Result<Self::Consensus> {
+        Ok(Arc::new(
+            EthBeaconConsensus::new(ctx.chain_spec())
+                .with_skip_gas_limit_ramp_check(true)
+                .with_skip_blob_gas_used_check(true)
+                .with_skip_requests_hash_check(true),
+        ))
+    }
+}
+
+// ---------------------------------------------------------------------------
+// Main
+// ---------------------------------------------------------------------------
+
+fn main() {
+    reth_cli_util::sigsegv_handler::install();
+
+    if std::env::var_os("RUST_BACKTRACE").is_none() {
+        unsafe { std::env::set_var("RUST_BACKTRACE", "1") };
+    }
+
+    let pending: BigBlockMap = Arc::new(Mutex::new(HashMap::new()));
+
+    if let Err(err) = Cli::<EthereumChainSpecParser>::parse().run(async move |builder, _| {
+        info!(target: "reth::cli", "Launching big block node");
+        let handle = builder.launch_node(BbNode::new(pending.clone())).await?;
+
+        handle.wait_for_node_exit().await
+    }) {
+        eprintln!("Error: {err:?}");
+        std::process::exit(1);
+    }
+}

bin/reth-bench-compare/Cargo.toml

@@ -1,99 +0,0 @@
-[package]
-name = "reth-bench-compare"
-version.workspace = true
-edition.workspace = true
-rust-version.workspace = true
-license.workspace = true
-homepage.workspace = true
-repository.workspace = true
-description = "Automated reth benchmark comparison between git references"
-
-[lints]
-workspace = true
-
-[[bin]]
-name = "reth-bench-compare"
-path = "src/main.rs"
-
-[dependencies]
-# reth
-reth-cli-runner.workspace = true
-reth-cli-util.workspace = true
-reth-node-core.workspace = true
-reth-tracing.workspace = true
-reth-chainspec.workspace = true
-
-# alloy
-alloy-provider = { workspace = true, features = ["reqwest-rustls-tls"], default-features = false }
-alloy-rpc-client = { workspace = true, features = ["pubsub"] }
-alloy-rpc-types-eth.workspace = true
-alloy-transport-ws.workspace = true
-alloy-primitives.workspace = true
-
-# CLI and argument parsing
-clap = { workspace = true, features = ["derive", "env"] }
-eyre.workspace = true
-
-# Async runtime
-tokio = { workspace = true, features = ["full"] }
-tracing.workspace = true
-
-# Serialization
-serde = { workspace = true, features = ["derive"] }
-serde_json.workspace = true
-
-# Time handling
-chrono = { workspace = true, features = ["serde"] }
-
-# CSV handling
-csv.workspace = true
-
-# Process management
-ctrlc.workspace = true
-shlex.workspace = true
-
-[target.'cfg(unix)'.dependencies]
-nix = { version = "0.31", features = ["signal", "process"] }
-
-[features]
-default = ["jemalloc"]
-
-asm-keccak = [
-    "reth-node-core/asm-keccak",
-    "alloy-primitives/asm-keccak",
-]
-
-jemalloc = [
-    "reth-cli-util/jemalloc",
-    "reth-node-core/jemalloc",
-]
-jemalloc-prof = ["reth-cli-util/jemalloc-prof"]
-tracy-allocator = ["reth-cli-util/tracy-allocator", "tracy"]
-tracy = [
-    "reth-node-core/tracy",
-    "reth-tracing/tracy",
-]
-
-min-error-logs = [
-    "tracing/release_max_level_error",
-    "reth-node-core/min-error-logs",
-]
-min-warn-logs = [
-    "tracing/release_max_level_warn",
-    "reth-node-core/min-warn-logs",
-]
-min-info-logs = [
-    "tracing/release_max_level_info",
-    "reth-node-core/min-info-logs",
-]
-min-debug-logs = [
-    "tracing/release_max_level_debug",
-    "reth-node-core/min-debug-logs",
-]
-min-trace-logs = [
-    "tracing/release_max_level_trace",
-    "reth-node-core/min-trace-logs",
-]
-
-# no-op feature flag for CI matrices
-ethereum = []

bin/reth-bench-compare/README.md

@@ -1,50 +0,0 @@
-# reth-bench-compare
-
-Compare reth performance between two git references.
-
-## Usage
-
-```bash
-reth-bench-compare \
-  --baseline-ref main \
-  --feature-ref my-feature \
-  --blocks 100 \
-  --wait-for-persistence
-```
-
-## Arguments
-
-| Argument | Description | Default | Required |
-|----------|-------------|---------|----------|
-| `--baseline-ref <REF>` | Git reference for baseline | - | Yes |
-| `--feature-ref <REF>` | Git reference to compare | - | Yes |
-| `--blocks <N>` | Number of blocks to benchmark | `100` | No |
-| `--chain <CHAIN>` | Chain to benchmark | `mainnet` | No |
-| `--datadir <PATH>` | Data directory path | OS-specific | No |
-| `--rpc-url <URL>` | RPC endpoint for block data | Chain default | No |
-| `--output-dir <PATH>` | Output directory | `./reth-bench-compare` | No |
-| `--wait-for-persistence` | Wait for block persistence | `false` | No |
-| `--persistence-threshold <N>` | Wait after every N+1 blocks | `2` | No |
-| `--wait-time <DURATION>` | Fixed delay (legacy) | - | No |
-| `--warmup-blocks <N>` | Cache warmup blocks | Same as `--blocks` | No |
-| `--draw` | Generate charts (needs Python/uv) | `false` | No |
-| `--profile` | Enable CPU profiling (needs samply) | `false` | No |
-| `-vvvv` | Debug logging | Info | No |
-| `--features <FEATURES>` | Extra Rust features for both builds | - | No |
-| `--rustflags <FLAGS>` | RUSTFLAGS for both builds | `-C target-cpu=native` | No |
-| `--baseline-features <FEATURES>` | Features for baseline only | Inherits `--features` | No |
-| `--feature-features <FEATURES>` | Features for feature only | Inherits `--features` | No |
-| `--baseline-rustflags <FLAGS>` | RUSTFLAGS for baseline only | Inherits `--rustflags` | No |
-| `--feature-rustflags <FLAGS>` | RUSTFLAGS for feature only | Inherits `--rustflags` | No |
-| `--baseline-args <ARGS>` | Extra args for baseline node | - | No |
-| `--feature-args <ARGS>` | Extra args for feature node | - | No |
-| `--metrics-port <PORT>` | Metrics endpoint port | `5005` | No |
-| `--sudo` | Run with elevated privileges | `false` | No |
-
-## Output
-
-Results in `./reth-bench-compare/results/<timestamp>/`:
-- `comparison_report.json` - Metrics comparison
-- `per_block_comparison.csv` - Per-block statistics
-- `baseline/` and `feature/` - Individual run results
-- `latency_comparison.png` - Chart (if `--draw` used)

bin/reth-bench-compare/src/benchmark.rs

@@ -1,307 +0,0 @@
-//! Benchmark execution using reth-bench.
-
-use crate::cli::Args;
-use eyre::{eyre, Result, WrapErr};
-use std::{
-    path::Path,
-    sync::{Arc, Mutex},
-};
-use tokio::{
-    fs::File as AsyncFile,
-    io::{AsyncBufReadExt, AsyncWriteExt, BufReader},
-    process::Command,
-};
-use tracing::{debug, error, info, warn};
-
-/// Manages benchmark execution using reth-bench
-pub(crate) struct BenchmarkRunner {
-    rpc_url: String,
-    jwt_secret: String,
-    wait_time: Option<String>,
-    wait_for_persistence: bool,
-    persistence_threshold: Option<u64>,
-    warmup_blocks: u64,
-}
-
-impl BenchmarkRunner {
-    /// Create a new `BenchmarkRunner` from CLI arguments
-    pub(crate) fn new(args: &Args) -> Self {
-        Self {
-            rpc_url: args.get_rpc_url(),
-            jwt_secret: args.jwt_secret_path().to_string_lossy().to_string(),
-            wait_time: args.wait_time.clone(),
-            wait_for_persistence: args.wait_for_persistence,
-            persistence_threshold: args.persistence_threshold,
-            warmup_blocks: args.get_warmup_blocks(),
-        }
-    }
-
-    /// Clear filesystem caches (page cache, dentries, and inodes)
-    pub(crate) async fn clear_fs_caches() -> Result<()> {
-        info!("Clearing filesystem caches...");
-
-        // First sync to ensure all pending writes are flushed
-        let sync_output =
-            Command::new("sync").output().await.wrap_err("Failed to execute sync command")?;
-
-        if !sync_output.status.success() {
-            return Err(eyre!("sync command failed"));
-        }
-
-        // Drop caches - requires sudo/root permissions
-        // 3 = drop pagecache, dentries, and inodes
-        let drop_caches_cmd = Command::new("sudo")
-            .args(["-n", "sh", "-c", "echo 3 > /proc/sys/vm/drop_caches"])
-            .output()
-            .await;
-
-        match drop_caches_cmd {
-            Ok(output) if output.status.success() => {
-                info!("Successfully cleared filesystem caches");
-                Ok(())
-            }
-            Ok(output) => {
-                let stderr = String::from_utf8_lossy(&output.stderr);
-                if stderr.contains("sudo: a password is required") {
-                    warn!("Unable to clear filesystem caches: sudo password required");
-                    warn!(
-                        "For optimal benchmarking, configure passwordless sudo for cache clearing:"
-                    );
-                    warn!("  echo '$USER ALL=(ALL) NOPASSWD: /bin/sh -c echo\\\\ [0-9]\\\\ \\\\>\\\\ /proc/sys/vm/drop_caches' | sudo tee /etc/sudoers.d/drop_caches");
-                    Ok(())
-                } else {
-                    Err(eyre!("Failed to clear filesystem caches: {}", stderr))
-                }
-            }
-            Err(e) => {
-                warn!("Unable to clear filesystem caches: {}", e);
-                Ok(())
-            }
-        }
-    }
-
-    /// Run a warmup benchmark for cache warming
-    pub(crate) async fn run_warmup(&self, from_block: u64) -> Result<()> {
-        let to_block = from_block + self.warmup_blocks;
-        info!(
-            "Running warmup benchmark from block {} to {} ({} blocks)",
-            from_block, to_block, self.warmup_blocks
-        );
-
-        // Build the reth-bench command for warmup (no output flag)
-        let mut cmd = Command::new("reth-bench");
-        cmd.args([
-            "new-payload-fcu",
-            "--rpc-url",
-            &self.rpc_url,
-            "--jwt-secret",
-            &self.jwt_secret,
-            "--from",
-            &from_block.to_string(),
-            "--to",
-            &to_block.to_string(),
-            "--wait-time=0ms", // Warmup should avoid persistence waits.
-        ]);
-
-        cmd.env("RUST_LOG_STYLE", "never")
-            .stdout(std::process::Stdio::piped())
-            .stderr(std::process::Stdio::piped())
-            .kill_on_drop(true);
-
-        // Set process group for consistent signal handling
-        #[cfg(unix)]
-        {
-            cmd.process_group(0);
-        }
-
-        debug!("Executing warmup reth-bench command: {:?}", cmd);
-
-        // Execute the warmup benchmark
-        let mut child = cmd.spawn().wrap_err("Failed to start warmup reth-bench process")?;
-
-        // Stream output at debug level
-        if let Some(stdout) = child.stdout.take() {
-            tokio::spawn(async move {
-                let reader = BufReader::new(stdout);
-                let mut lines = reader.lines();
-                while let Ok(Some(line)) = lines.next_line().await {
-                    debug!("[WARMUP] {}", line);
-                }
-            });
-        }
-
-        if let Some(stderr) = child.stderr.take() {
-            tokio::spawn(async move {
-                let reader = BufReader::new(stderr);
-                let mut lines = reader.lines();
-                while let Ok(Some(line)) = lines.next_line().await {
-                    debug!("[WARMUP] {}", line);
-                }
-            });
-        }
-
-        let status = child.wait().await.wrap_err("Failed to wait for warmup reth-bench")?;
-
-        if !status.success() {
-            return Err(eyre!("Warmup reth-bench failed with exit code: {:?}", status.code()));
-        }
-
-        info!("Warmup completed successfully");
-        Ok(())
-    }
-
-    /// Run a benchmark for the specified block range
-    pub(crate) async fn run_benchmark(
-        &self,
-        from_block: u64,
-        to_block: u64,
-        output_dir: &Path,
-    ) -> Result<()> {
-        info!(
-            "Running benchmark from block {} to {} (output: {:?})",
-            from_block, to_block, output_dir
-        );
-
-        // Ensure output directory exists
-        std::fs::create_dir_all(output_dir)
-            .wrap_err_with(|| format!("Failed to create output directory: {output_dir:?}"))?;
-
-        // Create log file path for reth-bench output
-        let log_file_path = output_dir.join("reth_bench.log");
-        info!("reth-bench logs will be saved to: {:?}", log_file_path);
-
-        // Build the reth-bench command
-        let mut cmd = Command::new("reth-bench");
-        cmd.args([
-            "new-payload-fcu",
-            "--rpc-url",
-            &self.rpc_url,
-            "--jwt-secret",
-            &self.jwt_secret,
-            "--from",
-            &from_block.to_string(),
-            "--to",
-            &to_block.to_string(),
-            "--output",
-            &output_dir.to_string_lossy(),
-        ]);
-
-        // Configure wait mode: both can be used together
-        // When both are set: wait at least wait_time, and also wait for persistence if needed
-        if let Some(ref wait_time) = self.wait_time {
-            cmd.args(["--wait-time", wait_time]);
-        }
-        if self.wait_for_persistence {
-            cmd.arg("--wait-for-persistence");
-
-            // Add persistence threshold if specified
-            if let Some(threshold) = self.persistence_threshold {
-                cmd.args(["--persistence-threshold", &threshold.to_string()]);
-            }
-        }
-
-        cmd.env("RUST_LOG_STYLE", "never")
-            .stdout(std::process::Stdio::piped())
-            .stderr(std::process::Stdio::piped())
-            .kill_on_drop(true);
-
-        // Set process group for consistent signal handling
-        #[cfg(unix)]
-        {
-            cmd.process_group(0);
-        }
-
-        // Debug log the command
-        debug!("Executing reth-bench command: {:?}", cmd);
-
-        // Execute the benchmark
-        let mut child = cmd.spawn().wrap_err("Failed to start reth-bench process")?;
-
-        // Capture stdout and stderr for error reporting
-        let stdout_lines = Arc::new(Mutex::new(Vec::new()));
-        let stderr_lines = Arc::new(Mutex::new(Vec::new()));
-
-        // Stream stdout with prefix at debug level, capture for error reporting, and write to log
-        // file
-        if let Some(stdout) = child.stdout.take() {
-            let stdout_lines_clone = stdout_lines.clone();
-            let log_file = AsyncFile::create(&log_file_path)
-                .await
-                .wrap_err(format!("Failed to create log file: {:?}", log_file_path))?;
-            tokio::spawn(async move {
-                let reader = BufReader::new(stdout);
-                let mut lines = reader.lines();
-                let mut log_file = log_file;
-                while let Ok(Some(line)) = lines.next_line().await {
-                    debug!("[RETH-BENCH] {}", line);
-                    if let Ok(mut captured) = stdout_lines_clone.lock() {
-                        captured.push(line.clone());
-                    }
-                    // Write to log file (reth-bench output already has timestamps if needed)
-                    let log_line = format!("{}\n", line);
-                    if let Err(e) = log_file.write_all(log_line.as_bytes()).await {
-                        debug!("Failed to write to log file: {}", e);
-                    }
-                }
-            });
-        }
-
-        // Stream stderr with prefix at debug level, capture for error reporting, and write to log
-        // file
-        if let Some(stderr) = child.stderr.take() {
-            let stderr_lines_clone = stderr_lines.clone();
-            let log_file = AsyncFile::options()
-                .create(true)
-                .append(true)
-                .open(&log_file_path)
-                .await
-                .wrap_err(format!("Failed to open log file for stderr: {:?}", log_file_path))?;
-            tokio::spawn(async move {
-                let reader = BufReader::new(stderr);
-                let mut lines = reader.lines();
-                let mut log_file = log_file;
-                while let Ok(Some(line)) = lines.next_line().await {
-                    debug!("[RETH-BENCH] {}", line);
-                    if let Ok(mut captured) = stderr_lines_clone.lock() {
-                        captured.push(line.clone());
-                    }
-                    // Write to log file (reth-bench output already has timestamps if needed)
-                    let log_line = format!("{}\n", line);
-                    if let Err(e) = log_file.write_all(log_line.as_bytes()).await {
-                        debug!("Failed to write to log file: {}", e);
-                    }
-                }
-            });
-        }
-
-        let status = child.wait().await.wrap_err("Failed to wait for reth-bench")?;
-
-        if !status.success() {
-            // Print all captured output when command fails
-            error!("reth-bench failed with exit code: {:?}", status.code());
-
-            if let Ok(stdout) = stdout_lines.lock() &&
-                !stdout.is_empty()
-            {
-                error!("reth-bench stdout:");
-                for line in stdout.iter() {
-                    error!("  {}", line);
-                }
-            }
-
-            if let Ok(stderr) = stderr_lines.lock() &&
-                !stderr.is_empty()
-            {
-                error!("reth-bench stderr:");
-                for line in stderr.iter() {
-                    error!("  {}", line);
-                }
-            }
-
-            return Err(eyre!("reth-bench failed with exit code: {:?}", status.code()));
-        }
-
-        info!("Benchmark completed");
-        Ok(())
-    }
-}

bin/reth-bench-compare/src/comparison.rs

@@ -1,723 +0,0 @@
-//! Results comparison and report generation.
-
-use crate::cli::Args;
-use chrono::{DateTime, Utc};
-use csv::Reader;
-use eyre::{eyre, Result, WrapErr};
-use serde::{Deserialize, Serialize};
-use std::{
-    cmp::Ordering,
-    collections::HashMap,
-    fs,
-    path::{Path, PathBuf},
-};
-use tracing::{info, warn};
-
-/// Manages comparison between baseline and feature reference results
-pub(crate) struct ComparisonGenerator {
-    output_dir: PathBuf,
-    timestamp: String,
-    baseline_ref_name: String,
-    feature_ref_name: String,
-    baseline_results: Option<BenchmarkResults>,
-    feature_results: Option<BenchmarkResults>,
-    baseline_command: Option<String>,
-    feature_command: Option<String>,
-}
-
-/// Represents the results from a single benchmark run
-#[derive(Debug, Clone)]
-pub(crate) struct BenchmarkResults {
-    pub ref_name: String,
-    pub combined_latency_data: Vec<CombinedLatencyRow>,
-    pub summary: BenchmarkSummary,
-    pub start_timestamp: Option<DateTime<Utc>>,
-    pub end_timestamp: Option<DateTime<Utc>>,
-}
-
-/// Combined latency CSV row structure
-#[derive(Debug, Clone, Deserialize, Serialize)]
-pub(crate) struct CombinedLatencyRow {
-    pub block_number: u64,
-    #[serde(default)]
-    pub transaction_count: Option<u64>,
-    pub gas_used: u64,
-    pub new_payload_latency: u128,
-}
-
-/// Total gas CSV row structure
-#[derive(Debug, Clone, Deserialize, Serialize)]
-pub(crate) struct TotalGasRow {
-    pub block_number: u64,
-    #[serde(default)]
-    pub transaction_count: Option<u64>,
-    pub gas_used: u64,
-    pub time: u128,
-}
-
-/// Summary statistics for a benchmark run.
-///
-/// Latencies are derived from per-block `engine_newPayload` timings (converted from µs to ms):
-/// - `mean_new_payload_latency_ms`: arithmetic mean latency across blocks.
-/// - `median_new_payload_latency_ms`: p50 latency across blocks.
-/// - `p90_new_payload_latency_ms` / `p99_new_payload_latency_ms`: tail latencies across blocks.
-#[derive(Debug, Clone, Serialize)]
-pub(crate) struct BenchmarkSummary {
-    pub total_blocks: u64,
-    pub total_gas_used: u64,
-    pub total_duration_ms: u128,
-    pub mean_new_payload_latency_ms: f64,
-    pub median_new_payload_latency_ms: f64,
-    pub p90_new_payload_latency_ms: f64,
-    pub p99_new_payload_latency_ms: f64,
-    pub gas_per_second: f64,
-    pub blocks_per_second: f64,
-    pub min_block_number: u64,
-    pub max_block_number: u64,
-}
-
-/// Comparison report between two benchmark runs
-#[derive(Debug, Serialize)]
-pub(crate) struct ComparisonReport {
-    pub timestamp: String,
-    pub baseline: RefInfo,
-    pub feature: RefInfo,
-    pub comparison_summary: ComparisonSummary,
-    pub per_block_comparisons: Vec<BlockComparison>,
-}
-
-/// Information about a reference in the comparison
-#[derive(Debug, Serialize)]
-pub(crate) struct RefInfo {
-    pub ref_name: String,
-    pub summary: BenchmarkSummary,
-    pub start_timestamp: Option<DateTime<Utc>>,
-    pub end_timestamp: Option<DateTime<Utc>>,
-    pub reth_command: Option<String>,
-}
-
-/// Summary of the comparison between references.
-///
-/// Percent deltas are `(feature - baseline) / baseline * 100`:
-/// - `new_payload_latency_mean_change_percent`: percent changes of the per-block means.
-/// - `new_payload_latency_p50_change_percent` / p90 / p99: percent changes of the respective
-///   per-block percentiles.
-/// - `per_block_latency_change_mean_percent` / `per_block_latency_change_median_percent` are the
-///   mean and median of per-block percent deltas (feature vs baseline), capturing block-level
-///   drift.
-/// - `per_block_latency_change_std_dev_percent`: standard deviation of per-block percent changes,
-///   measuring consistency of performance changes across blocks.
-/// - `new_payload_total_latency_change_percent` is the percent change of the total newPayload time
-///   across the run.
-///
-/// Positive means slower/higher; negative means faster/lower.
-#[derive(Debug, Serialize)]
-pub(crate) struct ComparisonSummary {
-    pub per_block_latency_change_mean_percent: f64,
-    pub per_block_latency_change_median_percent: f64,
-    pub per_block_latency_change_std_dev_percent: f64,
-    pub new_payload_total_latency_change_percent: f64,
-    pub new_payload_latency_mean_change_percent: f64,
-    pub new_payload_latency_p50_change_percent: f64,
-    pub new_payload_latency_p90_change_percent: f64,
-    pub new_payload_latency_p99_change_percent: f64,
-    pub gas_per_second_change_percent: f64,
-    pub blocks_per_second_change_percent: f64,
-}
-
-/// Per-block comparison data
-#[derive(Debug, Serialize)]
-pub(crate) struct BlockComparison {
-    pub block_number: u64,
-    #[serde(skip_serializing_if = "Option::is_none")]
-    pub transaction_count: Option<u64>,
-    pub gas_used: u64,
-    pub baseline_new_payload_latency: u128,
-    pub feature_new_payload_latency: u128,
-    pub new_payload_latency_change_percent: f64,
-}
-
-impl ComparisonGenerator {
-    /// Create a new comparison generator
-    pub(crate) fn new(args: &Args) -> Self {
-        let now: DateTime<Utc> = Utc::now();
-        let timestamp = now.format("%Y%m%d_%H%M%S").to_string();
-
-        Self {
-            output_dir: args.output_dir_path(),
-            timestamp,
-            baseline_ref_name: args.baseline_ref.clone(),
-            feature_ref_name: args.feature_ref.clone(),
-            baseline_results: None,
-            feature_results: None,
-            baseline_command: None,
-            feature_command: None,
-        }
-    }
-
-    /// Get the output directory for a specific reference
-    pub(crate) fn get_ref_output_dir(&self, ref_type: &str) -> PathBuf {
-        self.output_dir.join("results").join(&self.timestamp).join(ref_type)
-    }
-
-    /// Get the main output directory for this comparison run
-    pub(crate) fn get_output_dir(&self) -> PathBuf {
-        self.output_dir.join("results").join(&self.timestamp)
-    }
-
-    /// Add benchmark results for a reference
-    pub(crate) fn add_ref_results(&mut self, ref_type: &str, output_path: &Path) -> Result<()> {
-        let ref_name = match ref_type {
-            "baseline" => &self.baseline_ref_name,
-            "feature" => &self.feature_ref_name,
-            _ => return Err(eyre!("Unknown reference type: {}", ref_type)),
-        };
-
-        let results = self.load_benchmark_results(ref_name, output_path)?;
-
-        match ref_type {
-            "baseline" => self.baseline_results = Some(results),
-            "feature" => self.feature_results = Some(results),
-            _ => return Err(eyre!("Unknown reference type: {}", ref_type)),
-        }
-
-        info!("Loaded benchmark results for {} reference", ref_type);
-
-        Ok(())
-    }
-
-    /// Set the benchmark run timestamps for a reference
-    pub(crate) fn set_ref_timestamps(
-        &mut self,
-        ref_type: &str,
-        start: DateTime<Utc>,
-        end: DateTime<Utc>,
-    ) -> Result<()> {
-        match ref_type {
-            "baseline" => {
-                if let Some(ref mut results) = self.baseline_results {
-                    results.start_timestamp = Some(start);
-                    results.end_timestamp = Some(end);
-                } else {
-                    return Err(eyre!("Baseline results not loaded yet"));
-                }
-            }
-            "feature" => {
-                if let Some(ref mut results) = self.feature_results {
-                    results.start_timestamp = Some(start);
-                    results.end_timestamp = Some(end);
-                } else {
-                    return Err(eyre!("Feature results not loaded yet"));
-                }
-            }
-            _ => return Err(eyre!("Unknown reference type: {}", ref_type)),
-        }
-
-        Ok(())
-    }
-
-    /// Set the reth command for a reference
-    pub(crate) fn set_ref_command(&mut self, ref_type: &str, command: String) -> Result<()> {
-        match ref_type {
-            "baseline" => {
-                self.baseline_command = Some(command);
-            }
-            "feature" => {
-                self.feature_command = Some(command);
-            }
-            _ => return Err(eyre!("Unknown reference type: {}", ref_type)),
-        }
-
-        Ok(())
-    }
-
-    /// Generate the final comparison report
-    pub(crate) async fn generate_comparison_report(&self) -> Result<()> {
-        info!("Generating comparison report...");
-
-        let baseline =
-            self.baseline_results.as_ref().ok_or_else(|| eyre!("Baseline results not loaded"))?;
-
-        let feature =
-            self.feature_results.as_ref().ok_or_else(|| eyre!("Feature results not loaded"))?;
-
-        let per_block_comparisons = self.calculate_per_block_comparisons(baseline, feature)?;
-        let comparison_summary = self.calculate_comparison_summary(
-            &baseline.summary,
-            &feature.summary,
-            &per_block_comparisons,
-        )?;
-
-        let report = ComparisonReport {
-            timestamp: self.timestamp.clone(),
-            baseline: RefInfo {
-                ref_name: baseline.ref_name.clone(),
-                summary: baseline.summary.clone(),
-                start_timestamp: baseline.start_timestamp,
-                end_timestamp: baseline.end_timestamp,
-                reth_command: self.baseline_command.clone(),
-            },
-            feature: RefInfo {
-                ref_name: feature.ref_name.clone(),
-                summary: feature.summary.clone(),
-                start_timestamp: feature.start_timestamp,
-                end_timestamp: feature.end_timestamp,
-                reth_command: self.feature_command.clone(),
-            },
-            comparison_summary,
-            per_block_comparisons,
-        };
-
-        // Write reports
-        self.write_comparison_reports(&report).await?;
-
-        // Print summary to console
-        self.print_comparison_summary(&report);
-
-        Ok(())
-    }
-
-    /// Load benchmark results from CSV files
-    fn load_benchmark_results(
-        &self,
-        ref_name: &str,
-        output_path: &Path,
-    ) -> Result<BenchmarkResults> {
-        let combined_latency_path = output_path.join("combined_latency.csv");
-        let total_gas_path = output_path.join("total_gas.csv");
-
-        let combined_latency_data = self.load_combined_latency_csv(&combined_latency_path)?;
-        let total_gas_data = self.load_total_gas_csv(&total_gas_path)?;
-
-        let summary = self.calculate_summary(&combined_latency_data, &total_gas_data)?;
-
-        Ok(BenchmarkResults {
-            ref_name: ref_name.to_string(),
-            combined_latency_data,
-            summary,
-            start_timestamp: None,
-            end_timestamp: None,
-        })
-    }
-
-    /// Load combined latency CSV data
-    fn load_combined_latency_csv(&self, path: &Path) -> Result<Vec<CombinedLatencyRow>> {
-        let mut reader = Reader::from_path(path)
-            .wrap_err_with(|| format!("Failed to open combined latency CSV: {path:?}"))?;
-
-        let mut rows = Vec::new();
-        for result in reader.deserialize() {
-            let row: CombinedLatencyRow = result
-                .wrap_err_with(|| format!("Failed to parse combined latency row in {path:?}"))?;
-            rows.push(row);
-        }
-
-        if rows.is_empty() {
-            return Err(eyre!("No data found in combined latency CSV: {:?}", path));
-        }
-
-        Ok(rows)
-    }
-
-    /// Load total gas CSV data
-    fn load_total_gas_csv(&self, path: &Path) -> Result<Vec<TotalGasRow>> {
-        let mut reader = Reader::from_path(path)
-            .wrap_err_with(|| format!("Failed to open total gas CSV: {path:?}"))?;
-
-        let mut rows = Vec::new();
-        for result in reader.deserialize() {
-            let row: TotalGasRow =
-                result.wrap_err_with(|| format!("Failed to parse total gas row in {path:?}"))?;
-            rows.push(row);
-        }
-
-        if rows.is_empty() {
-            return Err(eyre!("No data found in total gas CSV: {:?}", path));
-        }
-
-        Ok(rows)
-    }
-
-    /// Calculate summary statistics for a benchmark run.
-    ///
-    /// Computes latency statistics from per-block `new_payload_latency` values in `combined_data`
-    /// (converting from µs to ms), and throughput metrics using the total run duration from
-    /// `total_gas_data`. Percentiles (p50/p90/p99) use linear interpolation on sorted latencies.
-    fn calculate_summary(
-        &self,
-        combined_data: &[CombinedLatencyRow],
-        total_gas_data: &[TotalGasRow],
-    ) -> Result<BenchmarkSummary> {
-        if combined_data.is_empty() || total_gas_data.is_empty() {
-            return Err(eyre!("Cannot calculate summary for empty data"));
-        }
-
-        let total_blocks = combined_data.len() as u64;
-        let total_gas_used: u64 = combined_data.iter().map(|r| r.gas_used).sum();
-
-        let total_duration_ms = total_gas_data.last().unwrap().time / 1000; // Convert microseconds to milliseconds
-
-        let latencies_ms: Vec<f64> =
-            combined_data.iter().map(|r| r.new_payload_latency as f64 / 1000.0).collect();
-        let mean_new_payload_latency_ms: f64 =
-            latencies_ms.iter().sum::<f64>() / total_blocks as f64;
-
-        let mut sorted_latencies_ms = latencies_ms;
-        sorted_latencies_ms.sort_by(|a, b| a.partial_cmp(b).unwrap_or(Ordering::Equal));
-        let median_new_payload_latency_ms = percentile(&sorted_latencies_ms, 0.5);
-        let p90_new_payload_latency_ms = percentile(&sorted_latencies_ms, 0.9);
-        let p99_new_payload_latency_ms = percentile(&sorted_latencies_ms, 0.99);
-
-        let total_duration_seconds = total_duration_ms as f64 / 1000.0;
-        let gas_per_second = if total_duration_seconds > f64::EPSILON {
-            total_gas_used as f64 / total_duration_seconds
-        } else {
-            0.0
-        };
-
-        let blocks_per_second = if total_duration_seconds > f64::EPSILON {
-            total_blocks as f64 / total_duration_seconds
-        } else {
-            0.0
-        };
-
-        let min_block_number = combined_data.first().unwrap().block_number;
-        let max_block_number = combined_data.last().unwrap().block_number;
-
-        Ok(BenchmarkSummary {
-            total_blocks,
-            total_gas_used,
-            total_duration_ms,
-            mean_new_payload_latency_ms,
-            median_new_payload_latency_ms,
-            p90_new_payload_latency_ms,
-            p99_new_payload_latency_ms,
-            gas_per_second,
-            blocks_per_second,
-            min_block_number,
-            max_block_number,
-        })
-    }
-
-    /// Calculate comparison summary between baseline and feature
-    fn calculate_comparison_summary(
-        &self,
-        baseline: &BenchmarkSummary,
-        feature: &BenchmarkSummary,
-        per_block_comparisons: &[BlockComparison],
-    ) -> Result<ComparisonSummary> {
-        let calc_percent_change = |baseline: f64, feature: f64| -> f64 {
-            if baseline.abs() > f64::EPSILON {
-                ((feature - baseline) / baseline) * 100.0
-            } else {
-                0.0
-            }
-        };
-
-        // Calculate per-block statistics. "Per-block" means: for each block, compute the percent
-        // change (feature - baseline) / baseline * 100, then calculate statistics across those
-        // per-block percent changes. This captures how consistently the feature performs relative
-        // to baseline across all blocks.
-        let per_block_percent_changes: Vec<f64> =
-            per_block_comparisons.iter().map(|c| c.new_payload_latency_change_percent).collect();
-        let per_block_latency_change_mean_percent = if per_block_percent_changes.is_empty() {
-            0.0
-        } else {
-            per_block_percent_changes.iter().sum::<f64>() / per_block_percent_changes.len() as f64
-        };
-        let per_block_latency_change_median_percent = if per_block_percent_changes.is_empty() {
-            0.0
-        } else {
-            let mut sorted = per_block_percent_changes.clone();
-            sorted.sort_by(|a, b| a.partial_cmp(b).unwrap_or(Ordering::Equal));
-            percentile(&sorted, 0.5)
-        };
-        let per_block_latency_change_std_dev_percent =
-            calculate_std_dev(&per_block_percent_changes, per_block_latency_change_mean_percent);
-
-        let baseline_total_latency_ms =
-            baseline.mean_new_payload_latency_ms * baseline.total_blocks as f64;
-        let feature_total_latency_ms =
-            feature.mean_new_payload_latency_ms * feature.total_blocks as f64;
-        let new_payload_total_latency_change_percent =
-            calc_percent_change(baseline_total_latency_ms, feature_total_latency_ms);
-
-        Ok(ComparisonSummary {
-            per_block_latency_change_mean_percent,
-            per_block_latency_change_median_percent,
-            per_block_latency_change_std_dev_percent,
-            new_payload_total_latency_change_percent,
-            new_payload_latency_mean_change_percent: calc_percent_change(
-                baseline.mean_new_payload_latency_ms,
-                feature.mean_new_payload_latency_ms,
-            ),
-            new_payload_latency_p50_change_percent: calc_percent_change(
-                baseline.median_new_payload_latency_ms,
-                feature.median_new_payload_latency_ms,
-            ),
-            new_payload_latency_p90_change_percent: calc_percent_change(
-                baseline.p90_new_payload_latency_ms,
-                feature.p90_new_payload_latency_ms,
-            ),
-            new_payload_latency_p99_change_percent: calc_percent_change(
-                baseline.p99_new_payload_latency_ms,
-                feature.p99_new_payload_latency_ms,
-            ),
-            gas_per_second_change_percent: calc_percent_change(
-                baseline.gas_per_second,
-                feature.gas_per_second,
-            ),
-            blocks_per_second_change_percent: calc_percent_change(
-                baseline.blocks_per_second,
-                feature.blocks_per_second,
-            ),
-        })
-    }
-
-    /// Calculate per-block comparisons
-    fn calculate_per_block_comparisons(
-        &self,
-        baseline: &BenchmarkResults,
-        feature: &BenchmarkResults,
-    ) -> Result<Vec<BlockComparison>> {
-        let mut baseline_map: HashMap<u64, &CombinedLatencyRow> = HashMap::new();
-        for row in &baseline.combined_latency_data {
-            baseline_map.insert(row.block_number, row);
-        }
-
-        let mut comparisons = Vec::new();
-        for feature_row in &feature.combined_latency_data {
-            if let Some(baseline_row) = baseline_map.get(&feature_row.block_number) {
-                let calc_percent_change = |baseline: u128, feature: u128| -> f64 {
-                    if baseline > 0 {
-                        ((feature as f64 - baseline as f64) / baseline as f64) * 100.0
-                    } else {
-                        0.0
-                    }
-                };
-
-                let comparison = BlockComparison {
-                    block_number: feature_row.block_number,
-                    transaction_count: feature_row.transaction_count,
-                    gas_used: feature_row.gas_used,
-                    baseline_new_payload_latency: baseline_row.new_payload_latency,
-                    feature_new_payload_latency: feature_row.new_payload_latency,
-                    new_payload_latency_change_percent: calc_percent_change(
-                        baseline_row.new_payload_latency,
-                        feature_row.new_payload_latency,
-                    ),
-                };
-                comparisons.push(comparison);
-            } else {
-                warn!("Block {} not found in baseline data", feature_row.block_number);
-            }
-        }
-
-        Ok(comparisons)
-    }
-
-    /// Write comparison reports to files
-    async fn write_comparison_reports(&self, report: &ComparisonReport) -> Result<()> {
-        let report_dir = self.output_dir.join("results").join(&self.timestamp);
-        fs::create_dir_all(&report_dir)
-            .wrap_err_with(|| format!("Failed to create report directory: {report_dir:?}"))?;
-
-        // Write JSON report
-        let json_path = report_dir.join("comparison_report.json");
-        let json_content = serde_json::to_string_pretty(report)
-            .wrap_err("Failed to serialize comparison report to JSON")?;
-        fs::write(&json_path, json_content)
-            .wrap_err_with(|| format!("Failed to write JSON report: {json_path:?}"))?;
-
-        // Write CSV report for per-block comparisons
-        let csv_path = report_dir.join("per_block_comparison.csv");
-        let mut writer = csv::Writer::from_path(&csv_path)
-            .wrap_err_with(|| format!("Failed to create CSV writer: {csv_path:?}"))?;
-
-        for comparison in &report.per_block_comparisons {
-            writer.serialize(comparison).wrap_err("Failed to write comparison row to CSV")?;
-        }
-        writer.flush().wrap_err("Failed to flush CSV writer")?;
-
-        info!("Comparison reports written to: {:?}", report_dir);
-        Ok(())
-    }
-
-    /// Print comparison summary to console
-    fn print_comparison_summary(&self, report: &ComparisonReport) {
-        // Parse and format timestamp nicely
-        let formatted_timestamp = if let Ok(dt) = chrono::DateTime::parse_from_str(
-            &format!("{} +0000", report.timestamp.replace('_', " ")),
-            "%Y%m%d %H%M%S %z",
-        ) {
-            dt.format("%Y-%m-%d %H:%M:%S UTC").to_string()
-        } else {
-            // Fallback to original if parsing fails
-            report.timestamp.clone()
-        };
-
-        println!("\n=== BENCHMARK COMPARISON SUMMARY ===");
-        println!("Timestamp: {formatted_timestamp}");
-        println!("Baseline: {}", report.baseline.ref_name);
-        println!("Feature:  {}", report.feature.ref_name);
-        println!();
-
-        let summary = &report.comparison_summary;
-
-        println!("Performance Changes:");
-        println!(
-            "  NewPayload Latency per-block mean change:   {:+.2}%",
-            summary.per_block_latency_change_mean_percent
-        );
-        println!(
-            "  NewPayload Latency per-block median change: {:+.2}%",
-            summary.per_block_latency_change_median_percent
-        );
-        println!(
-            "  NewPayload Latency per-block std dev:       {:.2}%",
-            summary.per_block_latency_change_std_dev_percent
-        );
-        println!(
-            "  Total newPayload time change:                {:+.2}%",
-            summary.new_payload_total_latency_change_percent
-        );
-        println!(
-            "  NewPayload Latency mean:           {:+.2}%",
-            summary.new_payload_latency_mean_change_percent
-        );
-        println!(
-            "  NewPayload Latency p50:           {:+.2}%",
-            summary.new_payload_latency_p50_change_percent
-        );
-        println!(
-            "  NewPayload Latency p90:           {:+.2}%",
-            summary.new_payload_latency_p90_change_percent
-        );
-        println!(
-            "  NewPayload Latency p99:           {:+.2}%",
-            summary.new_payload_latency_p99_change_percent
-        );
-        println!(
-            "  Gas/Second:                      {:+.2}%",
-            summary.gas_per_second_change_percent
-        );
-        println!(
-            "  Blocks/Second:                   {:+.2}%",
-            summary.blocks_per_second_change_percent
-        );
-        println!();
-
-        println!("Baseline Summary:");
-        let baseline = &report.baseline.summary;
-        println!(
-            "  Blocks: {} (blocks {} to {}), Gas: {}, Duration: {:.2}s",
-            baseline.total_blocks,
-            baseline.min_block_number,
-            baseline.max_block_number,
-            baseline.total_gas_used,
-            baseline.total_duration_ms as f64 / 1000.0
-        );
-        println!("  NewPayload latency (ms):");
-        println!(
-            "    mean: {:.2}, p50: {:.2}, p90: {:.2}, p99: {:.2}",
-            baseline.mean_new_payload_latency_ms,
-            baseline.median_new_payload_latency_ms,
-            baseline.p90_new_payload_latency_ms,
-            baseline.p99_new_payload_latency_ms
-        );
-        if let (Some(start), Some(end)) =
-            (&report.baseline.start_timestamp, &report.baseline.end_timestamp)
-        {
-            println!(
-                "  Started: {}, Ended: {}",
-                start.format("%Y-%m-%d %H:%M:%S UTC"),
-                end.format("%Y-%m-%d %H:%M:%S UTC")
-            );
-        }
-        if let Some(ref cmd) = report.baseline.reth_command {
-            println!("  Command: {}", cmd);
-        }
-        println!();
-
-        println!("Feature Summary:");
-        let feature = &report.feature.summary;
-        println!(
-            "  Blocks: {} (blocks {} to {}), Gas: {}, Duration: {:.2}s",
-            feature.total_blocks,
-            feature.min_block_number,
-            feature.max_block_number,
-            feature.total_gas_used,
-            feature.total_duration_ms as f64 / 1000.0
-        );
-        println!("  NewPayload latency (ms):");
-        println!(
-            "    mean: {:.2}, p50: {:.2}, p90: {:.2}, p99: {:.2}",
-            feature.mean_new_payload_latency_ms,
-            feature.median_new_payload_latency_ms,
-            feature.p90_new_payload_latency_ms,
-            feature.p99_new_payload_latency_ms
-        );
-        if let (Some(start), Some(end)) =
-            (&report.feature.start_timestamp, &report.feature.end_timestamp)
-        {
-            println!(
-                "  Started: {}, Ended: {}",
-                start.format("%Y-%m-%d %H:%M:%S UTC"),
-                end.format("%Y-%m-%d %H:%M:%S UTC")
-            );
-        }
-        if let Some(ref cmd) = report.feature.reth_command {
-            println!("  Command: {}", cmd);
-        }
-        println!();
-    }
-}
-
-/// Calculate standard deviation from a set of values and their mean.
-///
-/// Computes the population standard deviation using the formula:
-/// `sqrt(sum((x - mean)²) / n)`
-///
-/// Returns 0.0 for empty input.
-fn calculate_std_dev(values: &[f64], mean: f64) -> f64 {
-    if values.is_empty() {
-        return 0.0;
-    }
-
-    let variance = values
-        .iter()
-        .map(|x| {
-            let diff = x - mean;
-            diff * diff
-        })
-        .sum::<f64>() /
-        values.len() as f64;
-
-    variance.sqrt()
-}
-
-/// Calculate percentile using linear interpolation on a sorted slice.
-///
-/// Computes `rank = percentile × (n - 1)` where n is the array length. If the rank falls
-/// between two indices, linearly interpolates between those values. For example, with 100 values,
-/// p90 computes rank = 0.9 × 99 = 89.1, then returns `values[89] × 0.9 + values[90] × 0.1`.
-///
-/// Returns 0.0 for empty input.
-fn percentile(sorted_values: &[f64], percentile: f64) -> f64 {
-    if sorted_values.is_empty() {
-        return 0.0;
-    }
-
-    let clamped = percentile.clamp(0.0, 1.0);
-    let max_index = sorted_values.len() - 1;
-    let rank = clamped * max_index as f64;
-    let lower = rank.floor() as usize;
-    let upper = rank.ceil() as usize;
-
-    if lower == upper {
-        sorted_values[lower]
-    } else {
-        let weight = rank - lower as f64;
-        sorted_values[lower].mul_add(1.0 - weight, sorted_values[upper] * weight)
-    }
-}

bin/reth-bench-compare/src/compilation.rs

@@ -1,305 +0,0 @@
-//! Compilation operations for reth and reth-bench.
-
-use crate::git::GitManager;
-use eyre::{eyre, Result, WrapErr};
-use std::{fs, path::PathBuf, process::Command};
-use tracing::{debug, error, info, warn};
-
-/// Manages compilation operations for reth components
-#[derive(Debug)]
-pub(crate) struct CompilationManager {
-    repo_root: String,
-    output_dir: PathBuf,
-    git_manager: GitManager,
-}
-
-impl CompilationManager {
-    /// Create a new `CompilationManager`
-    pub(crate) const fn new(
-        repo_root: String,
-        output_dir: PathBuf,
-        git_manager: GitManager,
-    ) -> Result<Self> {
-        Ok(Self { repo_root, output_dir, git_manager })
-    }
-
-    /// Get the path to the cached binary using explicit commit hash
-    pub(crate) fn get_cached_binary_path_for_commit(&self, commit: &str) -> PathBuf {
-        let identifier = &commit[..8]; // Use first 8 chars of commit
-        self.output_dir.join("bin").join(format!("reth_{identifier}"))
-    }
-
-    /// Compile reth using cargo build and cache the binary
-    pub(crate) fn compile_reth(&self, commit: &str, features: &str, rustflags: &str) -> Result<()> {
-        // Validate that current git commit matches the expected commit
-        let current_commit = self.git_manager.get_current_commit()?;
-        if current_commit != commit {
-            return Err(eyre!(
-                "Git commit mismatch! Expected: {}, but currently at: {}",
-                &commit[..8],
-                &current_commit[..8]
-            ));
-        }
-
-        let cached_path = self.get_cached_binary_path_for_commit(commit);
-
-        // Check if cached binary already exists (since path contains commit hash, it's valid)
-        if cached_path.exists() {
-            info!("Using cached binary (commit: {})", &commit[..8]);
-            return Ok(());
-        }
-
-        info!("No cached binary found, compiling (commit: {})...", &commit[..8]);
-
-        let binary_name = "reth";
-
-        info!(
-            "Compiling {} with profiling configuration (commit: {})...",
-            binary_name,
-            &commit[..8]
-        );
-
-        let mut cmd = Command::new("cargo");
-        cmd.arg("build").arg("--profile").arg("profiling");
-
-        cmd.arg("--features").arg(features);
-        info!("Using features: {features}");
-
-        cmd.current_dir(&self.repo_root);
-
-        // Set RUSTFLAGS
-        cmd.env("RUSTFLAGS", rustflags);
-        info!("Using RUSTFLAGS: {rustflags}");
-
-        info!("Compiling {binary_name} with {cmd:?}");
-
-        let output = cmd.output().wrap_err("Failed to execute cargo build command")?;
-
-        // Print stdout and stderr with prefixes at debug level
-        let stdout = String::from_utf8_lossy(&output.stdout);
-        let stderr = String::from_utf8_lossy(&output.stderr);
-
-        for line in stdout.lines() {
-            if !line.trim().is_empty() {
-                debug!("[CARGO] {}", line);
-            }
-        }
-
-        for line in stderr.lines() {
-            if !line.trim().is_empty() {
-                debug!("[CARGO] {}", line);
-            }
-        }
-
-        if !output.status.success() {
-            // Print all output when compilation fails
-            error!("Cargo build failed with exit code: {:?}", output.status.code());
-
-            if !stdout.trim().is_empty() {
-                error!("Cargo stdout:");
-                for line in stdout.lines() {
-                    error!("  {}", line);
-                }
-            }
-
-            if !stderr.trim().is_empty() {
-                error!("Cargo stderr:");
-                for line in stderr.lines() {
-                    error!("  {}", line);
-                }
-            }
-
-            return Err(eyre!("Compilation failed with exit code: {:?}", output.status.code()));
-        }
-
-        info!("{} compilation completed", binary_name);
-
-        // Copy the compiled binary to cache
-        let source_path =
-            PathBuf::from(&self.repo_root).join(format!("target/profiling/{}", binary_name));
-        if !source_path.exists() {
-            return Err(eyre!("Compiled binary not found at {:?}", source_path));
-        }
-
-        // Create bin directory if it doesn't exist
-        let bin_dir = self.output_dir.join("bin");
-        fs::create_dir_all(&bin_dir).wrap_err("Failed to create bin directory")?;
-
-        // Copy binary to cache
-        fs::copy(&source_path, &cached_path).wrap_err("Failed to copy binary to cache")?;
-
-        // Make the cached binary executable
-        #[cfg(unix)]
-        {
-            use std::os::unix::fs::PermissionsExt;
-            let mut perms = fs::metadata(&cached_path)?.permissions();
-            perms.set_mode(0o755);
-            fs::set_permissions(&cached_path, perms)?;
-        }
-
-        info!("Cached compiled binary at: {:?}", cached_path);
-        Ok(())
-    }
-
-    /// Check if reth-bench is available in PATH
-    pub(crate) fn is_reth_bench_available(&self) -> bool {
-        match Command::new("which").arg("reth-bench").output() {
-            Ok(output) => {
-                if output.status.success() {
-                    let path = String::from_utf8_lossy(&output.stdout);
-                    info!("Found reth-bench: {}", path.trim());
-                    true
-                } else {
-                    false
-                }
-            }
-            Err(_) => false,
-        }
-    }
-
-    /// Check if samply is available in PATH
-    pub(crate) fn is_samply_available(&self) -> bool {
-        match Command::new("which").arg("samply").output() {
-            Ok(output) => {
-                if output.status.success() {
-                    let path = String::from_utf8_lossy(&output.stdout);
-                    info!("Found samply: {}", path.trim());
-                    true
-                } else {
-                    false
-                }
-            }
-            Err(_) => false,
-        }
-    }
-
-    /// Install samply using cargo
-    pub(crate) fn install_samply(&self) -> Result<()> {
-        info!("Installing samply via cargo...");
-
-        let mut cmd = Command::new("cargo");
-        cmd.args(["install", "--locked", "samply"]);
-
-        info!("Installing samply with {cmd:?}");
-
-        let output = cmd.output().wrap_err("Failed to execute cargo install samply command")?;
-
-        // Print stdout and stderr with prefixes at debug level
-        let stdout = String::from_utf8_lossy(&output.stdout);
-        let stderr = String::from_utf8_lossy(&output.stderr);
-
-        for line in stdout.lines() {
-            if !line.trim().is_empty() {
-                debug!("[CARGO-SAMPLY] {}", line);
-            }
-        }
-
-        for line in stderr.lines() {
-            if !line.trim().is_empty() {
-                debug!("[CARGO-SAMPLY] {}", line);
-            }
-        }
-
-        if !output.status.success() {
-            // Print all output when installation fails
-            error!("Cargo install samply failed with exit code: {:?}", output.status.code());
-
-            if !stdout.trim().is_empty() {
-                error!("Cargo stdout:");
-                for line in stdout.lines() {
-                    error!("  {}", line);
-                }
-            }
-
-            if !stderr.trim().is_empty() {
-                error!("Cargo stderr:");
-                for line in stderr.lines() {
-                    error!("  {}", line);
-                }
-            }
-
-            return Err(eyre!(
-                "samply installation failed with exit code: {:?}",
-                output.status.code()
-            ));
-        }
-
-        info!("Samply installation completed");
-        Ok(())
-    }
-
-    /// Ensure samply is available, installing if necessary
-    pub(crate) fn ensure_samply_available(&self) -> Result<()> {
-        if self.is_samply_available() {
-            Ok(())
-        } else {
-            warn!("samply not found in PATH, installing...");
-            self.install_samply()
-        }
-    }
-
-    /// Ensure reth-bench is available, compiling if necessary
-    pub(crate) fn ensure_reth_bench_available(&self) -> Result<()> {
-        if self.is_reth_bench_available() {
-            Ok(())
-        } else {
-            warn!("reth-bench not found in PATH, compiling and installing...");
-            self.compile_reth_bench()
-        }
-    }
-
-    /// Compile and install reth-bench using `make install-reth-bench`
-    pub(crate) fn compile_reth_bench(&self) -> Result<()> {
-        info!("Compiling and installing reth-bench...");
-
-        let mut cmd = Command::new("make");
-        cmd.arg("install-reth-bench").current_dir(&self.repo_root);
-
-        info!("Compiling reth-bench with {cmd:?}");
-
-        let output = cmd.output().wrap_err("Failed to execute make install-reth-bench command")?;
-
-        // Print stdout and stderr with prefixes at debug level
-        let stdout = String::from_utf8_lossy(&output.stdout);
-        let stderr = String::from_utf8_lossy(&output.stderr);
-
-        for line in stdout.lines() {
-            if !line.trim().is_empty() {
-                debug!("[MAKE-BENCH] {}", line);
-            }
-        }
-
-        for line in stderr.lines() {
-            if !line.trim().is_empty() {
-                debug!("[MAKE-BENCH] {}", line);
-            }
-        }
-
-        if !output.status.success() {
-            // Print all output when compilation fails
-            error!("Make install-reth-bench failed with exit code: {:?}", output.status.code());
-
-            if !stdout.trim().is_empty() {
-                error!("Make stdout:");
-                for line in stdout.lines() {
-                    error!("  {}", line);
-                }
-            }
-
-            if !stderr.trim().is_empty() {
-                error!("Make stderr:");
-                for line in stderr.lines() {
-                    error!("  {}", line);
-                }
-            }
-
-            return Err(eyre!(
-                "reth-bench compilation failed with exit code: {:?}",
-                output.status.code()
-            ));
-        }
-
-        info!("Reth-bench compilation completed");
-        Ok(())
-    }
-}

bin/reth-bench-compare/src/git.rs

@@ -1,328 +0,0 @@
-//! Git operations for branch management.
-
-use eyre::{eyre, Result, WrapErr};
-use std::process::Command;
-use tracing::{info, warn};
-
-/// Manages git operations for branch switching
-#[derive(Debug, Clone)]
-pub(crate) struct GitManager {
-    repo_root: String,
-}
-
-impl GitManager {
-    /// Create a new `GitManager`, detecting the repository root
-    pub(crate) fn new() -> Result<Self> {
-        let output = Command::new("git")
-            .args(["rev-parse", "--show-toplevel"])
-            .output()
-            .wrap_err("Failed to execute git command - is git installed?")?;
-
-        if !output.status.success() {
-            return Err(eyre!("Not in a git repository or git command failed"));
-        }
-
-        let repo_root = String::from_utf8(output.stdout)
-            .wrap_err("Git output is not valid UTF-8")?
-            .trim()
-            .to_string();
-
-        let manager = Self { repo_root };
-        info!(
-            "Detected git repository at: {}, current reference: {}",
-            manager.repo_root(),
-            manager.get_current_ref()?
-        );
-
-        Ok(manager)
-    }
-
-    /// Get the current git branch name
-    pub(crate) fn get_current_branch(&self) -> Result<String> {
-        let output = Command::new("git")
-            .args(["branch", "--show-current"])
-            .current_dir(&self.repo_root)
-            .output()
-            .wrap_err("Failed to get current branch")?;
-
-        if !output.status.success() {
-            return Err(eyre!("Failed to determine current branch"));
-        }
-
-        let branch = String::from_utf8(output.stdout)
-            .wrap_err("Branch name is not valid UTF-8")?
-            .trim()
-            .to_string();
-
-        if branch.is_empty() {
-            return Err(eyre!("Not on a named branch (detached HEAD?)"));
-        }
-
-        Ok(branch)
-    }
-
-    /// Get the current git reference (branch name, tag, or commit hash)
-    pub(crate) fn get_current_ref(&self) -> Result<String> {
-        // First try to get branch name
-        if let Ok(branch) = self.get_current_branch() {
-            return Ok(branch);
-        }
-
-        // If not on a branch, check if we're on a tag
-        let tag_output = Command::new("git")
-            .args(["describe", "--exact-match", "--tags", "HEAD"])
-            .current_dir(&self.repo_root)
-            .output()
-            .wrap_err("Failed to check for tag")?;
-
-        if tag_output.status.success() {
-            let tag = String::from_utf8(tag_output.stdout)
-                .wrap_err("Tag name is not valid UTF-8")?
-                .trim()
-                .to_string();
-            return Ok(tag);
-        }
-
-        // If not on a branch or tag, return the commit hash
-        let commit_output = Command::new("git")
-            .args(["rev-parse", "HEAD"])
-            .current_dir(&self.repo_root)
-            .output()
-            .wrap_err("Failed to get current commit")?;
-
-        if !commit_output.status.success() {
-            return Err(eyre!("Failed to get current commit hash"));
-        }
-
-        let commit_hash = String::from_utf8(commit_output.stdout)
-            .wrap_err("Commit hash is not valid UTF-8")?
-            .trim()
-            .to_string();
-
-        Ok(commit_hash)
-    }
-
-    /// Check if the git working directory has uncommitted changes to tracked files
-    pub(crate) fn validate_clean_state(&self) -> Result<()> {
-        let output = Command::new("git")
-            .args(["status", "--porcelain"])
-            .current_dir(&self.repo_root)
-            .output()
-            .wrap_err("Failed to check git status")?;
-
-        if !output.status.success() {
-            return Err(eyre!("Git status command failed"));
-        }
-
-        let status_output =
-            String::from_utf8(output.stdout).wrap_err("Git status output is not valid UTF-8")?;
-
-        // Check for uncommitted changes to tracked files
-        // Status codes: M = modified, A = added, D = deleted, R = renamed, C = copied, U = updated
-        // ?? = untracked files (we want to ignore these)
-        let has_uncommitted_changes = status_output.lines().any(|line| {
-            if line.len() >= 2 {
-                let status = &line[0..2];
-                // Ignore untracked files (??) and ignored files (!!)
-                !matches!(status, "??" | "!!")
-            } else {
-                false
-            }
-        });
-
-        if has_uncommitted_changes {
-            warn!("Git working directory has uncommitted changes to tracked files:");
-            for line in status_output.lines() {
-                if line.len() >= 2 && !matches!(&line[0..2], "??" | "!!") {
-                    warn!("  {}", line);
-                }
-            }
-            return Err(eyre!(
-                "Git working directory has uncommitted changes to tracked files. Please commit or stash changes before running benchmark comparison."
-            ));
-        }
-
-        // Check if there are untracked files and log them as info
-        let untracked_files: Vec<&str> =
-            status_output.lines().filter(|line| line.starts_with("??")).collect();
-
-        if !untracked_files.is_empty() {
-            info!(
-                "Git working directory has {} untracked files (this is OK)",
-                untracked_files.len()
-            );
-        }
-
-        info!("Git working directory is clean (no uncommitted changes to tracked files)");
-        Ok(())
-    }
-
-    /// Fetch all refs from remote to ensure we have latest branches and tags
-    pub(crate) fn fetch_all(&self) -> Result<()> {
-        let output = Command::new("git")
-            .args(["fetch", "--all", "--tags", "--quiet", "--force"])
-            .current_dir(&self.repo_root)
-            .output()
-            .wrap_err("Failed to fetch latest refs")?;
-
-        if output.status.success() {
-            info!("Fetched latest refs");
-        } else {
-            let stderr = String::from_utf8_lossy(&output.stderr);
-            // Only warn if there's actual error content, not just fetch progress
-            if !stderr.trim().is_empty() && !stderr.contains("-> origin/") {
-                warn!("Git fetch encountered issues (continuing anyway): {}", stderr);
-            }
-        }
-
-        Ok(())
-    }
-
-    /// Validate that the specified git references exist (branches, tags, or commits)
-    pub(crate) fn validate_refs(&self, refs: &[&str]) -> Result<()> {
-        for &git_ref in refs {
-            // Try to resolve the ref similar to `git checkout` by peeling to a commit.
-            // First try the ref as-is with ^{commit}, then fall back to origin/{ref}^{commit}.
-            let as_is = format!("{git_ref}^{{commit}}");
-            let ref_check = Command::new("git")
-                .args(["rev-parse", "--verify", &as_is])
-                .current_dir(&self.repo_root)
-                .output();
-
-            let found = if let Ok(output) = ref_check &&
-                output.status.success()
-            {
-                info!("Validated reference exists: {}", git_ref);
-                true
-            } else {
-                // Try remote-only branches via origin/{ref}
-                let origin_ref = format!("origin/{git_ref}^{{commit}}");
-                let origin_check = Command::new("git")
-                    .args(["rev-parse", "--verify", &origin_ref])
-                    .current_dir(&self.repo_root)
-                    .output();
-
-                if let Ok(output) = origin_check &&
-                    output.status.success()
-                {
-                    info!("Validated remote reference exists: origin/{}", git_ref);
-                    true
-                } else {
-                    false
-                }
-            };
-
-            if !found {
-                return Err(eyre!(
-                    "Git reference '{}' does not exist as branch, tag, or commit (tried '{}' and 'origin/{}^{{commit}}')",
-                    git_ref,
-                    format!("{git_ref}^{{commit}}"),
-                    git_ref,
-                ));
-            }
-        }
-
-        Ok(())
-    }
-
-    /// Switch to the specified git reference (branch, tag, or commit)
-    pub(crate) fn switch_ref(&self, git_ref: &str) -> Result<()> {
-        // First checkout the reference
-        let output = Command::new("git")
-            .args(["checkout", git_ref])
-            .current_dir(&self.repo_root)
-            .output()
-            .wrap_err_with(|| format!("Failed to switch to reference '{git_ref}'"))?;
-
-        if !output.status.success() {
-            let stderr = String::from_utf8_lossy(&output.stderr);
-            return Err(eyre!("Failed to switch to reference '{}': {}", git_ref, stderr));
-        }
-
-        // Check if this is a branch that tracks a remote and pull latest changes
-        let is_branch = Command::new("git")
-            .args(["show-ref", "--verify", "--quiet", &format!("refs/heads/{git_ref}")])
-            .current_dir(&self.repo_root)
-            .status()
-            .map(|s| s.success())
-            .unwrap_or(false);
-
-        if is_branch {
-            // Check if the branch tracks a remote
-            let tracking_output = Command::new("git")
-                .args([
-                    "rev-parse",
-                    "--abbrev-ref",
-                    "--symbolic-full-name",
-                    &format!("{git_ref}@{{upstream}}"),
-                ])
-                .current_dir(&self.repo_root)
-                .output();
-
-            if let Ok(output) = tracking_output &&
-                output.status.success()
-            {
-                let upstream = String::from_utf8_lossy(&output.stdout).trim().to_string();
-                if !upstream.is_empty() && upstream != format!("{git_ref}@{{upstream}}") {
-                    // Branch tracks a remote, pull latest changes
-                    info!("Pulling latest changes for branch: {}", git_ref);
-
-                    let pull_output = Command::new("git")
-                        .args(["pull", "--ff-only"])
-                        .current_dir(&self.repo_root)
-                        .output()
-                        .wrap_err_with(|| {
-                            format!("Failed to pull latest changes for branch '{git_ref}'")
-                        })?;
-
-                    if pull_output.status.success() {
-                        info!("Successfully pulled latest changes for branch: {}", git_ref);
-                    } else {
-                        let stderr = String::from_utf8_lossy(&pull_output.stderr);
-                        warn!("Failed to pull latest changes for branch '{}': {}", git_ref, stderr);
-                        // Continue anyway, we'll use whatever version we have
-                    }
-                }
-            }
-        }
-
-        // Verify the checkout succeeded by checking the current commit
-        let current_commit_output = Command::new("git")
-            .args(["rev-parse", "HEAD"])
-            .current_dir(&self.repo_root)
-            .output()
-            .wrap_err("Failed to get current commit")?;
-
-        if !current_commit_output.status.success() {
-            return Err(eyre!("Failed to verify git checkout"));
-        }
-
-        info!("Switched to reference: {}", git_ref);
-        Ok(())
-    }
-
-    /// Get the current commit hash
-    pub(crate) fn get_current_commit(&self) -> Result<String> {
-        let output = Command::new("git")
-            .args(["rev-parse", "HEAD"])
-            .current_dir(&self.repo_root)
-            .output()
-            .wrap_err("Failed to get current commit")?;
-
-        if !output.status.success() {
-            return Err(eyre!("Failed to get current commit hash"));
-        }
-
-        let commit_hash = String::from_utf8(output.stdout)
-            .wrap_err("Commit hash is not valid UTF-8")?
-            .trim()
-            .to_string();
-
-        Ok(commit_hash)
-    }
-
-    /// Get the repository root path
-    pub(crate) fn repo_root(&self) -> &str {
-        &self.repo_root
-    }
-}

bin/reth-bench-compare/src/main.rs

@@ -1,47 +0,0 @@
-//! # reth-bench-compare
-//!
-//! Automated tool for comparing reth performance between two git branches.
-//! This tool automates the complete workflow of compiling, running, and benchmarking
-//! reth on different branches to provide meaningful performance comparisons.
-
-#![doc(
-    html_logo_url = "https://raw.githubusercontent.com/paradigmxyz/reth/main/assets/reth-docs.png",
-    html_favicon_url = "https://avatars0.githubusercontent.com/u/97369466?s=256",
-    issue_tracker_base_url = "https://github.com/paradigmxyz/reth/issues/"
-)]
-#![cfg_attr(not(test), warn(unused_crate_dependencies))]
-
-#[global_allocator]
-static ALLOC: reth_cli_util::allocator::Allocator = reth_cli_util::allocator::new_allocator();
-
-use alloy_primitives as _;
-
-mod benchmark;
-mod cli;
-mod comparison;
-mod compilation;
-mod git;
-mod node;
-
-use clap::Parser;
-use cli::{run_comparison, Args};
-use eyre::Result;
-use reth_cli_runner::CliRunner;
-
-fn main() -> Result<()> {
-    // Enable backtraces unless a RUST_BACKTRACE value has already been explicitly provided.
-    if std::env::var_os("RUST_BACKTRACE").is_none() {
-        unsafe {
-            std::env::set_var("RUST_BACKTRACE", "1");
-        }
-    }
-
-    let args = Args::parse();
-
-    // Initialize tracing
-    let _guard = args.init_tracing()?;
-
-    // Run until either exit or sigint or sigterm
-    let runner = CliRunner::try_default_runtime()?;
-    runner.run_command_until_exit(|ctx| run_comparison(args, ctx))
-}

bin/reth-bench-compare/src/node.rs

@@ -1,695 +0,0 @@
-//! Node management for starting, stopping, and controlling reth instances.
-
-use crate::cli::Args;
-use alloy_provider::{Provider, ProviderBuilder};
-use alloy_rpc_client::RpcClient;
-use alloy_rpc_types_eth::SyncStatus;
-use alloy_transport_ws::WsConnect;
-use eyre::{eyre, OptionExt, Result, WrapErr};
-#[cfg(unix)]
-use nix::sys::signal::{killpg, Signal};
-#[cfg(unix)]
-use nix::unistd::Pid;
-use reth_chainspec::Chain;
-use std::{fs, path::PathBuf, time::Duration};
-use tokio::{
-    fs::File as AsyncFile,
-    io::{AsyncBufReadExt, AsyncWriteExt, BufReader as AsyncBufReader},
-    process::Command,
-    time::{sleep, timeout},
-};
-use tracing::{debug, info, warn};
-
-/// Default websocket RPC port used by reth
-const DEFAULT_WS_RPC_PORT: u16 = 8546;
-
-/// Manages reth node lifecycle and operations
-pub(crate) struct NodeManager {
-    datadir: Option<String>,
-    metrics_port: u16,
-    chain: Chain,
-    use_sudo: bool,
-    binary_path: Option<std::path::PathBuf>,
-    enable_profiling: bool,
-    output_dir: PathBuf,
-    additional_reth_args: Vec<String>,
-    comparison_dir: Option<PathBuf>,
-    tracing_endpoint: Option<String>,
-    otlp_max_queue_size: usize,
-}
-
-impl NodeManager {
-    /// Create a new `NodeManager` with configuration from CLI args
-    pub(crate) fn new(args: &Args) -> Self {
-        Self {
-            datadir: Some(args.datadir_path().to_string_lossy().to_string()),
-            metrics_port: args.metrics_port,
-            chain: args.chain,
-            use_sudo: args.sudo,
-            binary_path: None,
-            enable_profiling: args.profile,
-            output_dir: args.output_dir_path(),
-            // Filter out empty strings to prevent invalid arguments being passed to reth node
-            additional_reth_args: args
-                .reth_args
-                .iter()
-                .filter(|s| !s.is_empty())
-                .cloned()
-                .collect(),
-            comparison_dir: None,
-            tracing_endpoint: args.traces.otlp.as_ref().map(|u| u.to_string()),
-            otlp_max_queue_size: args.otlp_max_queue_size,
-        }
-    }
-
-    /// Set the comparison directory path for logging
-    pub(crate) fn set_comparison_dir(&mut self, dir: PathBuf) {
-        self.comparison_dir = Some(dir);
-    }
-
-    /// Get the log file path for a given reference type
-    fn get_log_file_path(&self, ref_type: &str) -> Result<PathBuf> {
-        let comparison_dir = self
-            .comparison_dir
-            .as_ref()
-            .ok_or_eyre("Comparison directory not set. Call set_comparison_dir first.")?;
-
-        // The comparison directory already contains the full path to results/<timestamp>
-        let log_dir = comparison_dir.join(ref_type);
-
-        // Create the directory if it doesn't exist
-        fs::create_dir_all(&log_dir)
-            .wrap_err(format!("Failed to create log directory: {:?}", log_dir))?;
-
-        let log_file = log_dir.join("reth_node.log");
-        Ok(log_file)
-    }
-
-    /// Get the perf event max sample rate from the system, capped at 10000
-    fn get_perf_sample_rate(&self) -> Option<String> {
-        let perf_rate_file = "/proc/sys/kernel/perf_event_max_sample_rate";
-        if let Ok(content) = fs::read_to_string(perf_rate_file) {
-            let rate_str = content.trim();
-            if !rate_str.is_empty() {
-                if let Ok(system_rate) = rate_str.parse::<u32>() {
-                    let capped_rate = std::cmp::min(system_rate, 10000);
-                    info!(
-                        "Detected perf_event_max_sample_rate: {}, using: {}",
-                        system_rate, capped_rate
-                    );
-                    return Some(capped_rate.to_string());
-                }
-                warn!("Failed to parse perf_event_max_sample_rate: {}", rate_str);
-            }
-        }
-        None
-    }
-
-    /// Get the absolute path to samply using 'which' command
-    async fn get_samply_path(&self) -> Result<String> {
-        let output = Command::new("which")
-            .arg("samply")
-            .output()
-            .await
-            .wrap_err("Failed to execute 'which samply' command")?;
-
-        if !output.status.success() {
-            return Err(eyre!("samply not found in PATH"));
-        }
-
-        let samply_path = String::from_utf8(output.stdout)
-            .wrap_err("samply path is not valid UTF-8")?
-            .trim()
-            .to_string();
-
-        if samply_path.is_empty() {
-            return Err(eyre!("which samply returned empty path"));
-        }
-
-        Ok(samply_path)
-    }
-
-    /// Build reth arguments as a vector of strings
-    fn build_reth_args(
-        &self,
-        binary_path_str: &str,
-        additional_args: &[String],
-        ref_type: &str,
-    ) -> (Vec<String>, String) {
-        let mut reth_args = vec![binary_path_str.to_string(), "node".to_string()];
-
-        // Add chain argument (skip for mainnet as it's the default)
-        let chain_str = self.chain.to_string();
-        if chain_str != "mainnet" {
-            reth_args.extend_from_slice(&["--chain".to_string(), chain_str.clone()]);
-        }
-
-        // Add datadir if specified
-        if let Some(ref datadir) = self.datadir {
-            reth_args.extend_from_slice(&["--datadir".to_string(), datadir.clone()]);
-        }
-
-        // Add reth-specific arguments
-        let metrics_arg = format!("0.0.0.0:{}", self.metrics_port);
-        reth_args.extend_from_slice(&[
-            "--engine.accept-execution-requests-hash".to_string(),
-            "--metrics".to_string(),
-            metrics_arg,
-            "--http".to_string(),
-            "--http.api".to_string(),
-            "eth,reth".to_string(),
-            "--ws".to_string(),
-            "--ws.api".to_string(),
-            "eth,reth".to_string(),
-            "--disable-discovery".to_string(),
-            "--trusted-only".to_string(),
-            "--disable-tx-gossip".to_string(),
-        ]);
-
-        // Add tracing arguments if OTLP endpoint is configured
-        if let Some(ref endpoint) = self.tracing_endpoint {
-            info!("Enabling OTLP tracing export to: {} (service: reth-{})", endpoint, ref_type);
-            // Endpoint requires equals per clap settings in reth
-            reth_args.push(format!("--tracing-otlp={}", endpoint));
-        }
-
-        // Add any additional arguments passed via command line (common to both baseline and
-        // feature)
-        reth_args.extend_from_slice(&self.additional_reth_args);
-
-        // Add reference-specific additional arguments
-        reth_args.extend_from_slice(additional_args);
-
-        (reth_args, chain_str)
-    }
-
-    /// Create a command for profiling mode
-    async fn create_profiling_command(
-        &self,
-        ref_type: &str,
-        reth_args: &[String],
-    ) -> Result<Command> {
-        // Create profiles directory if it doesn't exist
-        let profile_dir = self.output_dir.join("profiles");
-        fs::create_dir_all(&profile_dir).wrap_err("Failed to create profiles directory")?;
-
-        let profile_path = profile_dir.join(format!("{}.json.gz", ref_type));
-        info!("Starting reth node with samply profiling...");
-        info!("Profile output: {:?}", profile_path);
-
-        // Get absolute path to samply
-        let samply_path = self.get_samply_path().await?;
-
-        let mut cmd = if self.use_sudo {
-            let mut sudo_cmd = Command::new("sudo");
-            sudo_cmd.arg(&samply_path);
-            sudo_cmd
-        } else {
-            Command::new(&samply_path)
-        };
-
-        // Add samply arguments
-        cmd.args(["record", "--save-only", "-o", &profile_path.to_string_lossy()]);
-
-        // Add rate argument if available
-        if let Some(rate) = self.get_perf_sample_rate() {
-            cmd.args(["--rate", &rate]);
-        }
-
-        // Add separator and complete reth command
-        cmd.arg("--");
-        cmd.args(reth_args);
-
-        // Enable tracing-samply
-        if supports_samply_flags(&reth_args[0]) {
-            cmd.arg("--log.samply");
-        }
-
-        // Set environment variable to disable log styling
-        cmd.env("RUST_LOG_STYLE", "never");
-
-        Ok(cmd)
-    }
-
-    /// Create a command for direct reth execution
-    fn create_direct_command(&self, reth_args: &[String]) -> Command {
-        let binary_path = &reth_args[0];
-
-        let mut cmd = if self.use_sudo {
-            info!("Starting reth node with sudo...");
-            let mut sudo_cmd = Command::new("sudo");
-            sudo_cmd.args(reth_args);
-            sudo_cmd
-        } else {
-            info!("Starting reth node...");
-            let mut reth_cmd = Command::new(binary_path);
-            reth_cmd.args(&reth_args[1..]); // Skip the binary path since it's the command
-            reth_cmd
-        };
-
-        // Set environment variable to disable log styling
-        cmd.env("RUST_LOG_STYLE", "never");
-
-        cmd
-    }
-
-    /// Start a reth node using the specified binary path and return the process handle
-    /// along with the formatted reth command string for reporting.
-    pub(crate) async fn start_node(
-        &mut self,
-        binary_path: &std::path::Path,
-        _git_ref: &str,
-        ref_type: &str,
-        additional_args: &[String],
-    ) -> Result<(tokio::process::Child, String)> {
-        // Store the binary path for later use (e.g., in unwind_to_block)
-        self.binary_path = Some(binary_path.to_path_buf());
-
-        let binary_path_str = binary_path.to_string_lossy();
-        let (reth_args, _) = self.build_reth_args(&binary_path_str, additional_args, ref_type);
-
-        // Format the reth command string for reporting
-        let reth_command = shlex::try_join(reth_args.iter().map(|s| s.as_str()))
-            .wrap_err("Failed to format reth command string")?;
-
-        // Log additional arguments if any
-        if !self.additional_reth_args.is_empty() {
-            info!("Using common additional reth arguments: {:?}", self.additional_reth_args);
-        }
-        if !additional_args.is_empty() {
-            info!("Using reference-specific additional reth arguments: {:?}", additional_args);
-        }
-
-        let mut cmd = if self.enable_profiling {
-            self.create_profiling_command(ref_type, &reth_args).await?
-        } else {
-            self.create_direct_command(&reth_args)
-        };
-
-        // Set process group for better signal handling
-        #[cfg(unix)]
-        {
-            cmd.process_group(0);
-        }
-
-        // Set high queue size to prevent trace dropping during benchmarks
-        if self.tracing_endpoint.is_some() {
-            cmd.env("OTEL_BSP_MAX_QUEUE_SIZE", self.otlp_max_queue_size.to_string()); // Traces
-            cmd.env("OTEL_BLRP_MAX_QUEUE_SIZE", "10000"); // Logs
-
-            // Set service name to differentiate baseline vs feature runs in Jaeger
-            cmd.env("OTEL_SERVICE_NAME", format!("reth-{}", ref_type));
-        }
-
-        debug!("Executing reth command: {cmd:?}");
-
-        let mut child = cmd
-            .stdout(std::process::Stdio::piped())
-            .stderr(std::process::Stdio::piped())
-            .kill_on_drop(true) // Kill on drop so that on Ctrl-C for parent process we stop all child processes
-            .spawn()
-            .wrap_err("Failed to start reth node")?;
-
-        info!(
-            "Reth node started with PID: {:?} (binary: {})",
-            child.id().ok_or_eyre("Reth node is not running")?,
-            binary_path_str
-        );
-
-        // Prepare log file path
-        let log_file_path = self.get_log_file_path(ref_type)?;
-        info!("Reth node logs will be saved to: {:?}", log_file_path);
-
-        // Stream stdout and stderr with prefixes at debug level and to log file
-        if let Some(stdout) = child.stdout.take() {
-            let log_file = AsyncFile::create(&log_file_path)
-                .await
-                .wrap_err(format!("Failed to create log file: {:?}", log_file_path))?;
-            tokio::spawn(async move {
-                let reader = AsyncBufReader::new(stdout);
-                let mut lines = reader.lines();
-                let mut log_file = log_file;
-                while let Ok(Some(line)) = lines.next_line().await {
-                    debug!("[RETH] {}", line);
-                    // Write to log file (reth already includes timestamps)
-                    let log_line = format!("{}\n", line);
-                    if let Err(e) = log_file.write_all(log_line.as_bytes()).await {
-                        debug!("Failed to write to log file: {}", e);
-                    }
-                }
-            });
-        }
-
-        if let Some(stderr) = child.stderr.take() {
-            let log_file = AsyncFile::options()
-                .create(true)
-                .append(true)
-                .open(&log_file_path)
-                .await
-                .wrap_err(format!("Failed to open log file for stderr: {:?}", log_file_path))?;
-            tokio::spawn(async move {
-                let reader = AsyncBufReader::new(stderr);
-                let mut lines = reader.lines();
-                let mut log_file = log_file;
-                while let Ok(Some(line)) = lines.next_line().await {
-                    debug!("[RETH] {}", line);
-                    // Write to log file (reth already includes timestamps)
-                    let log_line = format!("{}\n", line);
-                    if let Err(e) = log_file.write_all(log_line.as_bytes()).await {
-                        debug!("Failed to write to log file: {}", e);
-                    }
-                }
-            });
-        }
-
-        // Give the node a moment to start up
-        sleep(Duration::from_secs(5)).await;
-
-        Ok((child, reth_command))
-    }
-
-    /// Wait for the node to be ready and return its current tip.
-    ///
-    /// Fails early if the node process exits before becoming ready.
-    pub(crate) async fn wait_for_node_ready_and_get_tip(
-        &self,
-        child: &mut tokio::process::Child,
-    ) -> Result<u64> {
-        info!("Waiting for node to be ready and synced...");
-
-        let max_wait = Duration::from_secs(120); // 2 minutes to allow for sync
-        let check_interval = Duration::from_secs(2);
-        let rpc_url = "http://localhost:8545";
-
-        // Create Alloy provider
-        let url = rpc_url.parse().map_err(|e| eyre!("Invalid RPC URL '{}': {}", rpc_url, e))?;
-        let provider = ProviderBuilder::new().connect_http(url);
-
-        let start_time = tokio::time::Instant::now();
-        let mut iteration = 0;
-
-        timeout(max_wait, async {
-            loop {
-                iteration += 1;
-                debug!(
-                    "Readiness check iteration {} (elapsed: {:?})",
-                    iteration,
-                    start_time.elapsed()
-                );
-
-                // Check if the node process has exited.
-                if let Some(status) = child.try_wait()? {
-                    return Err(eyre!("Node process exited unexpectedly with {status}"));
-                }
-
-                // First check if RPC is up and node is not syncing
-                match provider.syncing().await {
-                    Ok(sync_result) => {
-                        match sync_result {
-                            SyncStatus::Info(sync_info) => {
-                                debug!("Node is still syncing {sync_info:?}, waiting...");
-                            }
-                            _ => {
-                                debug!("HTTP RPC is up and node is not syncing, checking block number...");
-                                // Node is not syncing, now get the tip
-                                match provider.get_block_number().await {
-                                    Ok(tip) => {
-                                        debug!("HTTP RPC ready at block: {}, checking WebSocket...", tip);
-
-                                        // Verify WebSocket RPC is ready (public endpoint, no JWT required)
-                                        let ws_url = format!("ws://localhost:{}", DEFAULT_WS_RPC_PORT);
-                                        debug!("Attempting WebSocket connection to {} (public endpoint)", ws_url);
-                                        let ws_connect = WsConnect::new(&ws_url);
-
-                                        match RpcClient::connect_pubsub(ws_connect).await
-                                        {
-                                            Ok(_) => {
-                                                info!(
-                                                    "Node is ready (HTTP and WebSocket) at block: {} (took {:?}, {} iterations)",
-                                                    tip, start_time.elapsed(), iteration
-                                                );
-                                                return Ok(tip);
-                                            }
-                                            Err(e) => {
-                                                debug!(
-                                                    "HTTP RPC ready but WebSocket not ready yet (iteration {}): {:?}",
-                                                    iteration, e
-                                                );
-                                                debug!("WebSocket error details: {}", e);
-                                            }
-                                        }
-                                    }
-                                    Err(e) => {
-                                        debug!("Failed to get block number (iteration {}): {:?}", iteration, e);
-                                    }
-                                }
-                            }
-                        }
-                    }
-                    Err(e) => {
-                        debug!("Node RPC not ready yet or failed to check sync status (iteration {}): {:?}", iteration, e);
-                    }
-                }
-
-                debug!("Sleeping for {:?} before next check", check_interval);
-                sleep(check_interval).await;
-            }
-        })
-        .await
-        .wrap_err("Timed out waiting for node to be ready and synced")?
-    }
-
-    /// Wait for the node RPC to be ready and return its current tip, without waiting for sync.
-    ///
-    /// This is faster than `wait_for_node_ready_and_get_tip` but may return a tip while
-    /// the node is still syncing.
-    pub(crate) async fn wait_for_rpc_and_get_tip(
-        &self,
-        child: &mut tokio::process::Child,
-    ) -> Result<u64> {
-        info!("Waiting for node RPC to be ready (skipping sync wait)...");
-
-        let max_wait = Duration::from_secs(60);
-        let check_interval = Duration::from_secs(2);
-        let rpc_url = "http://localhost:8545";
-
-        let url = rpc_url.parse().map_err(|e| eyre!("Invalid RPC URL '{}': {}", rpc_url, e))?;
-        let provider = ProviderBuilder::new().connect_http(url);
-
-        let start_time = tokio::time::Instant::now();
-        let mut iteration = 0;
-
-        timeout(max_wait, async {
-            loop {
-                iteration += 1;
-                debug!(
-                    "RPC readiness check iteration {} (elapsed: {:?})",
-                    iteration,
-                    start_time.elapsed()
-                );
-
-                if let Some(status) = child.try_wait()? {
-                    return Err(eyre!("Node process exited unexpectedly with {status}"));
-                }
-
-                match provider.get_block_number().await {
-                    Ok(tip) => {
-                        debug!("HTTP RPC ready at block: {}, checking WebSocket...", tip);
-
-                        let ws_url = format!("ws://localhost:{}", DEFAULT_WS_RPC_PORT);
-                        let ws_connect = WsConnect::new(&ws_url);
-
-                        match RpcClient::connect_pubsub(ws_connect).await {
-                            Ok(_) => {
-                                info!(
-                                    "Node RPC is ready at block: {} (took {:?}, {} iterations)",
-                                    tip,
-                                    start_time.elapsed(),
-                                    iteration
-                                );
-                                return Ok(tip);
-                            }
-                            Err(e) => {
-                                debug!(
-                                    "HTTP RPC ready but WebSocket not ready yet (iteration {}): {:?}",
-                                    iteration, e
-                                );
-                            }
-                        }
-                    }
-                    Err(e) => {
-                        debug!("RPC not ready yet (iteration {}): {:?}", iteration, e);
-                    }
-                }
-
-                sleep(check_interval).await;
-            }
-        })
-        .await
-        .wrap_err("Timed out waiting for node RPC to be ready")?
-    }
-
-    /// Stop the reth node gracefully
-    pub(crate) async fn stop_node(&self, child: &mut tokio::process::Child) -> Result<()> {
-        let pid = child.id().ok_or_eyre("Child process ID should be available")?;
-
-        // Check if the process has already exited
-        match child.try_wait() {
-            Ok(Some(status)) => {
-                info!("Reth node (PID: {}) has already exited with status: {:?}", pid, status);
-                return Ok(());
-            }
-            Ok(None) => {
-                // Process is still running, proceed to stop it
-                info!("Stopping process gracefully with SIGINT (PID: {})...", pid);
-            }
-            Err(e) => {
-                return Err(eyre!("Failed to check process status: {}", e));
-            }
-        }
-
-        #[cfg(unix)]
-        {
-            // Send SIGINT to process group to mimic Ctrl-C behavior
-            let nix_pgid = Pid::from_raw(pid as i32);
-
-            match killpg(nix_pgid, Signal::SIGINT) {
-                Ok(()) => {}
-                Err(nix::errno::Errno::ESRCH) => {
-                    info!("Process group {} has already exited", pid);
-                }
-                Err(e) => {
-                    return Err(eyre!("Failed to send SIGINT to process group {}: {}", pid, e));
-                }
-            }
-        }
-
-        #[cfg(not(unix))]
-        {
-            // On non-Unix systems, fall back to using external kill command
-            let output = Command::new("taskkill")
-                .args(["/PID", &pid.to_string(), "/F"])
-                .output()
-                .await
-                .wrap_err("Failed to execute taskkill command")?;
-
-            if !output.status.success() {
-                let stderr = String::from_utf8_lossy(&output.stderr);
-                // Check if the error is because the process doesn't exist
-                if stderr.contains("not found") || stderr.contains("not exist") {
-                    info!("Process {} has already exited", pid);
-                } else {
-                    return Err(eyre!("Failed to kill process {}: {}", pid, stderr));
-                }
-            }
-        }
-
-        // Wait for the process to exit
-        match child.wait().await {
-            Ok(status) => {
-                info!("Reth node (PID: {}) exited with status: {:?}", pid, status);
-            }
-            Err(e) => {
-                // If we get an error here, it might be because the process already exited
-                debug!("Error waiting for process exit (may have already exited): {}", e);
-            }
-        }
-
-        Ok(())
-    }
-
-    /// Unwind the node to a specific block
-    pub(crate) async fn unwind_to_block(&self, block_number: u64) -> Result<()> {
-        if self.use_sudo {
-            info!("Unwinding node to block: {} (with sudo)", block_number);
-        } else {
-            info!("Unwinding node to block: {}", block_number);
-        }
-
-        // Use the binary path from the last start_node call, or fallback to default
-        let binary_path = self
-            .binary_path
-            .as_ref()
-            .map(|p| p.to_string_lossy().to_string())
-            .unwrap_or_else(|| "./target/profiling/reth".to_string());
-
-        let mut cmd = if self.use_sudo {
-            let mut sudo_cmd = Command::new("sudo");
-            sudo_cmd.args([&binary_path, "stage", "unwind"]);
-            sudo_cmd
-        } else {
-            let mut reth_cmd = Command::new(&binary_path);
-            reth_cmd.args(["stage", "unwind"]);
-            reth_cmd
-        };
-
-        // Add chain argument (skip for mainnet as it's the default)
-        let chain_str = self.chain.to_string();
-        if chain_str != "mainnet" {
-            cmd.args(["--chain", &chain_str]);
-        }
-
-        // Add datadir if specified
-        if let Some(ref datadir) = self.datadir {
-            cmd.args(["--datadir", datadir]);
-        }
-
-        cmd.args(["to-block", &block_number.to_string()]);
-
-        // Set environment variable to disable log styling
-        cmd.env("RUST_LOG_STYLE", "never");
-
-        // Debug log the command
-        debug!("Executing reth unwind command: {:?}", cmd);
-
-        let mut child = cmd
-            .stdout(std::process::Stdio::piped())
-            .stderr(std::process::Stdio::piped())
-            .spawn()
-            .wrap_err("Failed to start unwind command")?;
-
-        // Stream stdout and stderr with prefixes in real-time
-        if let Some(stdout) = child.stdout.take() {
-            tokio::spawn(async move {
-                let reader = AsyncBufReader::new(stdout);
-                let mut lines = reader.lines();
-                while let Ok(Some(line)) = lines.next_line().await {
-                    debug!("[RETH-UNWIND] {}", line);
-                }
-            });
-        }
-
-        if let Some(stderr) = child.stderr.take() {
-            tokio::spawn(async move {
-                let reader = AsyncBufReader::new(stderr);
-                let mut lines = reader.lines();
-                while let Ok(Some(line)) = lines.next_line().await {
-                    debug!("[RETH-UNWIND] {}", line);
-                }
-            });
-        }
-
-        // Wait for the command to complete
-        let status = child.wait().await.wrap_err("Failed to wait for unwind command")?;
-
-        if !status.success() {
-            return Err(eyre!("Unwind command failed with exit code: {:?}", status.code()));
-        }
-
-        info!("Unwound to block: {}", block_number);
-        Ok(())
-    }
-}
-
-fn supports_samply_flags(bin: &str) -> bool {
-    let mut cmd = std::process::Command::new(bin);
-    // NOTE: The flag to check must come before --help.
-    // We pass --help as a shortcut to not execute any command.
-    cmd.args(["--log.samply", "--help"]);
-    debug!(?cmd, "Checking samply flags support");
-    let Ok(output) = cmd.output() else {
-        return false;
-    };
-    debug!(?output, "Samply flags support check");
-    output.status.success()
-}

bin/reth-bench/Cargo.toml

@@ -14,9 +14,12 @@ workspace = true
 
 [dependencies]
 # reth
+reth-chainspec.workspace = true
+reth-cli.workspace = true
 reth-cli-runner.workspace = true
 reth-cli-util.workspace = true
 reth-engine-primitives.workspace = true
+reth-ethereum-cli.workspace = true
 reth-ethereum-primitives.workspace = true
 reth-fs-util.workspace = true
 reth-node-api.workspace = true
@@ -25,13 +28,12 @@ reth-primitives-traits.workspace = true
 reth-rpc-api.workspace = true
 
 reth-tracing.workspace = true
-reth-chainspec.workspace = true
 
 # alloy
+alloy-consensus.workspace = true
 alloy-eips.workspace = true
 alloy-json-rpc.workspace = true
-alloy-consensus.workspace = true
-alloy-network.workspace = true
+
 alloy-primitives = { workspace = true, features = ["rand"] }
 alloy-provider = { workspace = true, features = ["engine-api", "pubsub", "reqwest-rustls-tls"], default-features = false }
 alloy-pubsub.workspace = true
@@ -82,39 +84,54 @@ default = ["jemalloc"]
 
 asm-keccak = [
     "reth-node-core/asm-keccak",
+    "reth-ethereum-cli/asm-keccak",
     "alloy-primitives/asm-keccak",
 ]
 
 jemalloc = [
     "reth-cli-util/jemalloc",
     "reth-node-core/jemalloc",
+    "reth-ethereum-cli/jemalloc",
+]
+jemalloc-prof = [
+    "reth-cli-util/jemalloc-prof",
+    "reth-ethereum-cli/jemalloc-prof",
+]
+tracy-allocator = [
+    "reth-cli-util/tracy-allocator",
+    "tracy",
+    "reth-ethereum-cli/tracy-allocator",
 ]
-jemalloc-prof = ["reth-cli-util/jemalloc-prof"]
-tracy-allocator = ["reth-cli-util/tracy-allocator", "tracy"]
 tracy = [
     "reth-node-core/tracy",
     "reth-tracing/tracy",
+    "reth-ethereum-cli/tracy",
 ]
 
 min-error-logs = [
     "tracing/release_max_level_error",
     "reth-node-core/min-error-logs",
+    "reth-ethereum-cli/min-error-logs",
 ]
 min-warn-logs = [
     "tracing/release_max_level_warn",
     "reth-node-core/min-warn-logs",
+    "reth-ethereum-cli/min-warn-logs",
 ]
 min-info-logs = [
     "tracing/release_max_level_info",
     "reth-node-core/min-info-logs",
+    "reth-ethereum-cli/min-info-logs",
 ]
 min-debug-logs = [
     "tracing/release_max_level_debug",
     "reth-node-core/min-debug-logs",
+    "reth-ethereum-cli/min-debug-logs",
 ]
 min-trace-logs = [
     "tracing/release_max_level_trace",
     "reth-node-core/min-trace-logs",
+    "reth-ethereum-cli/min-trace-logs",
 ]
 
 # no-op feature flag for CI matrices

bin/reth-bench/README.md

@@ -35,6 +35,10 @@ The `new-payload-fcu` command supports two optional waiting modes that can be us
 - `--wait-time <duration>`: Fixed sleep interval between blocks (e.g., `--wait-time 100ms` or `--wait-time 400` for 400ms)
 - `--wait-for-persistence`: Waits for blocks to be persisted using the `reth_subscribePersistedBlock` subscription
 
+Both `new-payload-fcu` and `new-payload-only` support `--rpc-block-fetch-retries <RETRIES>`
+to control how many times block fetches are retried after an RPC failure. The default is `10`.
+Use `--rpc-block-fetch-retries forever` to keep retrying indefinitely.
+
 When using `--wait-for-persistence`, the benchmark waits after every `(threshold + 1)` blocks, where the threshold defaults to the engine's persistence threshold (2). This can be customized with `--persistence-threshold <N>`.
 
 By default, the WebSocket URL for persistence subscriptions is derived from `--engine-rpc-url` (converting to ws:// on port 8546). Use `--ws-rpc-url` to override this.

bin/reth-bench/src/authenticated_transport.rs

@@ -52,8 +52,7 @@ impl InnerTransport {
         url: Url,
         jwt: JwtSecret,
     ) -> Result<(Self, Claims), AuthenticatedTransportError> {
-        let mut client_builder =
-            reqwest::Client::builder().tls_built_in_root_certs(url.scheme() == "https");
+        let mut client_builder = reqwest::Client::builder();
         let mut headers = reqwest::header::HeaderMap::new();
 
         // Add the JWT to the headers if we can decode it.

bin/reth-bench/src/bench/context.rs

@@ -7,9 +7,9 @@ use alloy_primitives::address;
 use alloy_provider::{network::AnyNetwork, Provider, RootProvider};
 use alloy_rpc_client::ClientBuilder;
 use alloy_rpc_types_engine::JwtSecret;
-use alloy_transport::layers::RetryBackoffLayer;
+use alloy_transport::layers::{RateLimitRetryPolicy, RetryBackoffLayer};
 use reqwest::Url;
-use reth_node_core::args::BenchmarkArgs;
+use reth_node_core::args::{BenchmarkArgs, WaitForPersistence};
 use tracing::info;
 
 /// This is intended to be used by benchmarks that replay blocks from an RPC.
@@ -31,6 +31,12 @@ pub(crate) struct BenchContext {
     pub(crate) is_optimism: bool,
     /// Whether to use `reth_newPayload` endpoint instead of `engine_newPayload*`.
     pub(crate) use_reth_namespace: bool,
+    /// Whether to fetch and replay RLP-encoded blocks.
+    pub(crate) rlp_blocks: bool,
+    /// Controls when `reth_newPayload` waits for persistence.
+    pub(crate) wait_for_persistence: WaitForPersistence,
+    /// Whether to skip waiting for caches (pass `wait_for_caches: false`).
+    pub(crate) no_wait_for_caches: bool,
 }
 
 impl BenchContext {
@@ -51,9 +57,16 @@ impl BenchContext {
             }
         }
 
-        // set up alloy client for blocks
+        // set up alloy client for blocks, retrying on 429/503 (default) and 502
+        let retry_policy =
+            RateLimitRetryPolicy::default().or(|err: &alloy_transport::TransportError| -> bool {
+                err.as_transport_err()
+                    .and_then(|t| t.as_http_error())
+                    .is_some_and(|e| e.status == 502)
+            });
+        let max_retries = bench_args.rpc_block_fetch_retries.as_max_retries();
         let client = ClientBuilder::default()
-            .layer(RetryBackoffLayer::new(10, 800, u64::MAX))
+            .layer(RetryBackoffLayer::new_with_policy(max_retries, 800, u64::MAX, retry_policy))
             .http(rpc_url.parse()?);
         let block_provider = RootProvider::<AnyNetwork>::new(client);
 
@@ -86,9 +99,11 @@ impl BenchContext {
 
         // Computes the block range for the benchmark.
         //
-        // - If `--advance` is provided, fetches the latest block and sets:
+        // - If `--advance` is provided, fetches the latest block from the engine and sets:
         //     - `from = head + 1`
         //     - `to = head + advance`
+        // - If only `--to` is provided, fetches the latest block from the engine and sets:
+        //     - `from = head`
         // - Otherwise, uses the values from `--from` and `--to`.
         let (from, to) = if let Some(advance) = bench_args.advance {
             if advance == 0 {
@@ -101,6 +116,14 @@ impl BenchContext {
                 .ok_or_else(|| eyre::eyre!("Failed to fetch latest block for --advance"))?;
             let head_number = head_block.header.number;
             (Some(head_number), Some(head_number + advance))
+        } else if bench_args.from.is_none() && bench_args.to.is_some() {
+            let head_block = auth_provider
+                .get_block_by_number(BlockNumberOrTag::Latest)
+                .await?
+                .ok_or_else(|| eyre::eyre!("Failed to fetch latest block from engine"))?;
+            let head_number = head_block.header.number;
+            info!(target: "reth-bench", "No --from provided, derived from engine head: {}", head_number);
+            (Some(head_number), bench_args.to)
         } else {
             (bench_args.from, bench_args.to)
         };
@@ -112,7 +135,7 @@ impl BenchContext {
             .full()
             .await?
             .ok_or_else(|| eyre::eyre!("Failed to fetch latest block from RPC"))?;
-        let mut benchmark_mode = BenchMode::new(from, to, latest_block.into_inner().number())?;
+        let mut benchmark_mode = BenchMode::new(from, to, latest_block.into_inner().number());
 
         let first_block = match benchmark_mode {
             BenchMode::Continuous(start) => {
@@ -142,7 +165,11 @@ impl BenchContext {
         };
 
         let next_block = first_block.header.number + 1;
-        let use_reth_namespace = bench_args.reth_new_payload;
+        let rlp_blocks = bench_args.rlp_blocks;
+        let wait_for_persistence =
+            bench_args.wait_for_persistence.unwrap_or(WaitForPersistence::Never);
+        let use_reth_namespace = bench_args.reth_new_payload || rlp_blocks;
+        let no_wait_for_caches = bench_args.no_wait_for_caches;
         Ok(Self {
             auth_provider,
             block_provider,
@@ -150,6 +177,9 @@ impl BenchContext {
             next_block,
             is_optimism,
             use_reth_namespace,
+            rlp_blocks,
+            wait_for_persistence,
+            no_wait_for_caches,
         })
     }
 }

bin/reth-bench/src/bench/gas_limit_ramp.rs

@@ -1,252 +0,0 @@
-//! Benchmarks empty block processing by ramping the block gas limit.
-
-use crate::{
-    authenticated_transport::AuthenticatedTransportConnect,
-    bench::{
-        helpers::{build_payload, parse_gas_limit, prepare_payload_request, rpc_block_to_header},
-        output::GasRampPayloadFile,
-    },
-    valid_payload::{
-        call_forkchoice_updated_with_reth, call_new_payload_with_reth, payload_to_new_payload,
-    },
-};
-use alloy_eips::BlockNumberOrTag;
-use alloy_provider::{network::AnyNetwork, Provider, RootProvider};
-use alloy_rpc_client::ClientBuilder;
-use alloy_rpc_types_engine::{ExecutionPayload, ForkchoiceState, JwtSecret};
-
-use clap::Parser;
-use reqwest::Url;
-use reth_chainspec::ChainSpec;
-use reth_cli_runner::CliContext;
-use reth_ethereum_primitives::TransactionSigned;
-use reth_primitives_traits::constants::{GAS_LIMIT_BOUND_DIVISOR, MAXIMUM_GAS_LIMIT_BLOCK};
-use std::{path::PathBuf, time::Instant};
-use tracing::info;
-
-/// `reth benchmark gas-limit-ramp` command.
-#[derive(Debug, Parser)]
-pub struct Command {
-    /// Number of blocks to generate. Mutually exclusive with --target-gas-limit.
-    #[arg(long, value_name = "BLOCKS", conflicts_with = "target_gas_limit")]
-    blocks: Option<u64>,
-
-    /// Target gas limit to ramp up to. The benchmark will generate blocks until the gas limit
-    /// reaches or exceeds this value. Mutually exclusive with --blocks.
-    /// Accepts short notation: K for thousand, M for million, G for billion (e.g., 2G = 2
-    /// billion).
-    #[arg(long, value_name = "TARGET_GAS_LIMIT", conflicts_with = "blocks", value_parser = parse_gas_limit)]
-    target_gas_limit: Option<u64>,
-
-    /// The Engine API RPC URL.
-    #[arg(long = "engine-rpc-url", value_name = "ENGINE_RPC_URL")]
-    engine_rpc_url: String,
-
-    /// Path to the JWT secret for Engine API authentication.
-    #[arg(long = "jwt-secret", value_name = "JWT_SECRET")]
-    jwt_secret: PathBuf,
-
-    /// Output directory for benchmark results and generated payloads.
-    #[arg(long, value_name = "OUTPUT")]
-    output: PathBuf,
-
-    /// Use `reth_newPayload` endpoint instead of `engine_newPayload*`.
-    ///
-    /// The `reth_newPayload` endpoint is a reth-specific extension that takes `ExecutionData`
-    /// directly, waits for persistence and cache updates to complete before processing,
-    /// and returns server-side timing breakdowns (latency, persistence wait, cache wait).
-    #[arg(long, default_value = "false", verbatim_doc_comment)]
-    reth_new_payload: bool,
-}
-
-/// Mode for determining when to stop ramping.
-#[derive(Debug, Clone, Copy)]
-enum RampMode {
-    /// Ramp for a fixed number of blocks.
-    Blocks(u64),
-    /// Ramp until reaching or exceeding target gas limit.
-    TargetGasLimit(u64),
-}
-
-impl Command {
-    /// Execute `benchmark gas-limit-ramp` command.
-    pub async fn execute(self, _ctx: CliContext) -> eyre::Result<()> {
-        let mode = match (self.blocks, self.target_gas_limit) {
-            (Some(blocks), None) => {
-                if blocks == 0 {
-                    return Err(eyre::eyre!("--blocks must be greater than 0"));
-                }
-                RampMode::Blocks(blocks)
-            }
-            (None, Some(target)) => {
-                if target == 0 {
-                    return Err(eyre::eyre!("--target-gas-limit must be greater than 0"));
-                }
-                RampMode::TargetGasLimit(target)
-            }
-            _ => {
-                return Err(eyre::eyre!(
-                    "Exactly one of --blocks or --target-gas-limit must be specified"
-                ));
-            }
-        };
-
-        // Ensure output directory exists
-        if self.output.is_file() {
-            return Err(eyre::eyre!("Output path must be a directory"));
-        }
-        if !self.output.exists() {
-            std::fs::create_dir_all(&self.output)?;
-            info!(target: "reth-bench", "Created output directory: {:?}", self.output);
-        }
-
-        // Set up authenticated provider (used for both Engine API and eth_ methods)
-        let jwt = std::fs::read_to_string(&self.jwt_secret)?;
-        let jwt = JwtSecret::from_hex(jwt)?;
-        let auth_url = Url::parse(&self.engine_rpc_url)?;
-
-        info!(target: "reth-bench", "Connecting to Engine RPC at {}", auth_url);
-        let auth_transport = AuthenticatedTransportConnect::new(auth_url, jwt);
-        let client = ClientBuilder::default().connect_with(auth_transport).await?;
-        let provider = RootProvider::<AnyNetwork>::new(client);
-
-        // Get chain spec - required for fork detection
-        let chain_id = provider.get_chain_id().await?;
-        let chain_spec = ChainSpec::from_chain_id(chain_id)
-            .ok_or_else(|| eyre::eyre!("Unsupported chain id: {chain_id}"))?;
-
-        // Fetch the current head block as parent
-        let parent_block = provider
-            .get_block_by_number(BlockNumberOrTag::Latest)
-            .full()
-            .await?
-            .ok_or_else(|| eyre::eyre!("Failed to fetch latest block"))?;
-
-        let (mut parent_header, mut parent_hash) = rpc_block_to_header(parent_block);
-
-        let canonical_parent = parent_header.number;
-        let start_block = canonical_parent + 1;
-
-        match mode {
-            RampMode::Blocks(blocks) => {
-                info!(
-                    target: "reth-bench",
-                    canonical_parent,
-                    start_block,
-                    end_block = start_block + blocks - 1,
-                    "Starting gas limit ramp benchmark (block count mode)"
-                );
-            }
-            RampMode::TargetGasLimit(target) => {
-                info!(
-                    target: "reth-bench",
-                    canonical_parent,
-                    start_block,
-                    current_gas_limit = parent_header.gas_limit,
-                    target_gas_limit = target,
-                    "Starting gas limit ramp benchmark (target gas limit mode)"
-                );
-            }
-        }
-        if self.reth_new_payload {
-            info!("Using reth_newPayload and reth_forkchoiceUpdated endpoints");
-        }
-
-        let mut blocks_processed = 0u64;
-        let total_benchmark_duration = Instant::now();
-
-        while !should_stop(mode, blocks_processed, parent_header.gas_limit) {
-            let timestamp = parent_header.timestamp.saturating_add(1);
-
-            let request = prepare_payload_request(&chain_spec, timestamp, parent_hash);
-            let new_payload_version = request.new_payload_version;
-
-            let (payload, sidecar) = build_payload(&provider, request).await?;
-
-            let mut block =
-                payload.clone().try_into_block_with_sidecar::<TransactionSigned>(&sidecar)?;
-
-            let max_increase = max_gas_limit_increase(parent_header.gas_limit);
-            let gas_limit =
-                parent_header.gas_limit.saturating_add(max_increase).min(MAXIMUM_GAS_LIMIT_BLOCK);
-
-            block.header.gas_limit = gas_limit;
-
-            let block_hash = block.header.hash_slow();
-            // Regenerate the payload from the modified block, but keep the original sidecar
-            // which contains the actual execution requests data (not just the hash)
-            let (payload, _) = ExecutionPayload::from_block_unchecked(block_hash, &block);
-            let (version, params, execution_data) = payload_to_new_payload(
-                payload,
-                sidecar,
-                false,
-                block.header.withdrawals_root,
-                Some(new_payload_version),
-            )?;
-
-            // Save payload to file with version info for replay
-            let payload_path =
-                self.output.join(format!("payload_block_{}.json", block.header.number));
-            let file = GasRampPayloadFile {
-                version: version as u8,
-                block_hash,
-                params: params.clone(),
-                execution_data: Some(execution_data.clone()),
-            };
-            let payload_json = serde_json::to_string_pretty(&file)?;
-            std::fs::write(&payload_path, &payload_json)?;
-            info!(target: "reth-bench", block_number = block.header.number, path = %payload_path.display(), "Saved payload");
-
-            let reth_data = self.reth_new_payload.then_some(execution_data);
-            let _ = call_new_payload_with_reth(&provider, version, params, reth_data).await?;
-
-            let forkchoice_state = ForkchoiceState {
-                head_block_hash: block_hash,
-                safe_block_hash: block_hash,
-                finalized_block_hash: block_hash,
-            };
-            call_forkchoice_updated_with_reth(
-                &provider,
-                version,
-                forkchoice_state,
-                self.reth_new_payload,
-            )
-            .await?;
-
-            parent_header = block.header;
-            parent_hash = block_hash;
-            blocks_processed += 1;
-
-            let progress = match mode {
-                RampMode::Blocks(total) => format!("{blocks_processed}/{total}"),
-                RampMode::TargetGasLimit(target) => {
-                    let pct = (parent_header.gas_limit as f64 / target as f64 * 100.0).min(100.0);
-                    format!("{pct:.1}%")
-                }
-            };
-            info!(target: "reth-bench", progress, block_number = parent_header.number, gas_limit = parent_header.gas_limit, "Block processed");
-        }
-
-        let final_gas_limit = parent_header.gas_limit;
-        info!(
-            target: "reth-bench",
-            total_duration=?total_benchmark_duration.elapsed(),
-            blocks_processed,
-            final_gas_limit,
-            "Benchmark complete"
-        );
-
-        Ok(())
-    }
-}
-
-const fn max_gas_limit_increase(parent_gas_limit: u64) -> u64 {
-    (parent_gas_limit / GAS_LIMIT_BOUND_DIVISOR).saturating_sub(1)
-}
-
-const fn should_stop(mode: RampMode, blocks_processed: u64, current_gas_limit: u64) -> bool {
-    match mode {
-        RampMode::Blocks(target_blocks) => blocks_processed >= target_blocks,
-        RampMode::TargetGasLimit(target) => current_gas_limit >= target,
-    }
-}