Compare commits

..

2 Commits

Author SHA1 Message Date
yongkangc
54d07c55e5 feat(engine): add descriptive thread names for spawn_blocking tasks
Closes #20430 by using prctl/pthread_setname_np to set thread names at runtime
inside spawn_blocking tasks, keeping Tokio's thread pooling while making tasks
identifiable in profilers like Samply.

Thread names (kept under 15-char Linux limit):
- reth-multiproof: Multi-proof computation task
- reth-sparse: Sparse trie task
- reth-prewarm: Prewarm task coordinator
- reth-prewarm-w: Prewarm worker threads
- reth-tx-conv: Transaction conversion task
- reth-tx-order: Transaction ordering task
- reth-stor-proof: Storage proof workers
- reth-acct-proof: Account proof workers

Changes:
- Add spawn_blocking_named() to WorkloadExecutor that uses set_thread_name()
- Add set_thread_name() helper using prctl (Linux) / pthread_setname_np (macOS)
- Update all engine blocking tasks to use named threads
- Update ProofWorkerHandle to set thread names inside spawn_blocking
2025-12-18 09:08:45 +00:00
yongkangc
a8d0a89a62 feat(tasks): add thread names for spawned blocking tasks
Closes #20430 by adding descriptive thread names to blocking task pools,
making them identifiable in profilers like Samply.

Thread naming convention (kept under 15-char Linux limit):
- reth-node-{idx}: Main node runtime threads
- reth-eng-{idx}: Engine WorkloadExecutor threads
- reth-blk-{idx}: BlockingTaskPool rayon threads
2025-12-18 08:41:35 +00:00
1115 changed files with 64060 additions and 61464 deletions

View File

@@ -1,4 +0,0 @@
---
---
Added site-level meta description for SEO.

View File

@@ -1,5 +0,0 @@
---
reth-transaction-pool: patch
---
Renamed and documented validation methods for clarity. `validate_one_no_state` and `validate_one_against_state` are now public methods `validate_stateless` and `validate_stateful` with improved documentation explaining their respective validation phases.

View File

@@ -1,20 +0,0 @@
# Changelogs configuration for reth
# https://github.com/wevm/changelogs
# How to bump packages that depend on changed packages
dependent_bump = "patch"
[changelog]
# Generate per-crate changelogs (vs single root changelog)
format = "per-crate"
# Fixed groups: all always share the same version
# reth binaries share version
[[fixed]]
members = ["reth"]
# Packages to ignore (internal/test-only crates)
ignore = [
"reth-testing-utils",
"reth-bench",
]

View File

@@ -1,17 +0,0 @@
---
reth: minor
reth-cli-commands: minor
reth-e2e-test-utils: minor
reth-ethereum-cli: minor
reth-node-core: minor
reth-optimism-bin: minor
reth-optimism-cli: minor
reth-prune: patch
reth-stages: patch
reth-storage-api: minor
reth-storage-db-api: minor
reth-storage-db-common: patch
reth-storage-provider: patch
---
Introduced `--storage.v2` flag to control storage mode defaults, replacing the `edge` feature flag with `rocksdb` feature. The new flag enables v2 storage settings (static files + RocksDB routing) while individual `--static-files.*` and `--rocksdb.*` flags can still override defaults. Updated feature gates from `edge` to `rocksdb` across all affected crates.

View File

@@ -1,5 +0,0 @@
---
reth: patch
---
Removed Windows platform support from the codebase, including the Windows cross-compilation Dockerfile, build targets in Cross.toml and Makefile, and Windows-specific options in the bug report template.

View File

@@ -1,5 +0,0 @@
---
reth-network: minor
---
Added reason label to backed_off_peers metric. The metric now tracks backed off peers by reason (too_many_peers, graceful_close, connection_error) to improve observability.

View File

@@ -1,5 +0,0 @@
---
reth-trie-sparse-parallel: patch
---
Fixed parallel sparse trie to skip revealing disconnected leaves by checking parent branch reachability before inserting leaf nodes.

View File

@@ -1,5 +0,0 @@
---
ef-tests: patch
---
Removed reth-stateless crate and stateless validation from ef-tests.

View File

@@ -1,6 +0,0 @@
---
reth-engine-tree: patch
reth-trie-sparse-parallel: patch
---
Added tracing spans and debug logs to sparse trie operations for better observability during parallel state root computation.

View File

@@ -1,6 +0,0 @@
---
reth-exex: patch
reth-exex-types: patch
---
Added configurable backfill thresholds to ExEx notifications stream and added regression tests for state provider parity between pipeline and backfill execution paths.

View File

@@ -1,4 +0,0 @@
---
---
Added WebSocket subscription integration tests for eth_subscribe.

View File

@@ -1,5 +0,0 @@
---
reth-engine-tree: patch
---
Reordered cache size calculations in `ExecutionCache::new` to group related operations together.

View File

@@ -1,7 +0,0 @@
---
reth: patch
reth-cli-commands: patch
reth-node-core: patch
---
Removed experimental ress protocol support for stateless Ethereum nodes.

View File

@@ -1,5 +0,0 @@
---
reth-node-builder: patch
---
Removed biased select in engine service loop to allow fair scheduling of shutdown requests alongside event processing.

View File

@@ -1,5 +0,0 @@
---
reth-transaction-pool: patch
---
Fixed swapped arguments in `blob_tx_priority` function calls, correcting the parameter order to match the function signature.

View File

@@ -1,5 +0,0 @@
---
reth: patch
---
Re-enabled changelog workflow to run automatically on pull requests.

View File

@@ -1,6 +0,0 @@
---
reth-chainspec: minor
reth-network-peers: minor
---
Removed OP stack bootnodes from default chain configurations and network peers module.

View File

@@ -1,5 +0,0 @@
---
reth-primitives: patch
---
Moved feature-referenced dependencies from dev-dependencies to optional dependencies to ensure they are available when their corresponding features are enabled.

View File

@@ -1,5 +0,0 @@
---
reth-transaction-pool: minor
---
Added `IntoIter: Send` bounds to `validate_transactions` and `validate_transactions_with_origin` in the `TransactionValidator` trait, avoiding unnecessary `Vec` collects. Simplified default `validate_transactions_with_origin` to delegate to `validate_transactions`.

View File

@@ -1,6 +0,0 @@
---
reth-static-file-types: patch
reth-provider: patch
---
Move changeset offsets from segment header to external `.csoff` sidecar file for incremental writes and crash recovery.

View File

@@ -1,5 +0,0 @@
---
reth-provider: patch
---
Removed unused staging types from ProviderFactoryBuilder.

View File

@@ -1,5 +0,0 @@
---
reth: patch
---
Added automated changelog generation infrastructure using wevm/changelogs-rs with Claude Code integration. Configured per-crate changelog format with fixed version groups for reth binaries and exclusions for internal test utilities.

View File

@@ -1,5 +0,0 @@
---
reth-trie-sparse: minor
---
Removed `SerialSparseTrie` from the workspace, consolidating on `ParallelSparseTrie` as the single sparse trie implementation in `reth-trie-sparse`.

View File

@@ -1,5 +0,0 @@
---
reth: patch
---
Updated Alloy dependencies from 1.5.2 to 1.6.1.

View File

@@ -1,4 +0,0 @@
---
---
Expanded CLI integration tests with subcommand help coverage, config TOML validation, genesis JSON validation, and send transaction round-trip test for dev mode.

View File

@@ -1,5 +0,0 @@
---
reth-network: minor
---
Added direction labels to `closed_sessions` and `pending_session_failures` metrics. Operators can now distinguish session closures and failures by direction (`active`, `incoming_pending`, `outgoing_pending` for closed sessions; `inbound`, `outbound` for pending session failures).

View File

@@ -1,4 +0,0 @@
---
---
Moved Kurtosis CI failure notifications to the hive Slack channel.

View File

@@ -1,7 +0,0 @@
---
reth-rpc-api: minor
reth-rpc-builder: patch
reth-rpc: minor
---
Added `subscribeFinalizedChainNotifications` RPC endpoint that buffers committed chain notifications and emits them once a new finalized block is received.

View File

@@ -1,6 +0,0 @@
---
reth-trie: minor
reth-trie-parallel: minor
---
Added `root_node` and `storage_root_node` methods to proof calculators for efficient root-only calculations. These methods directly return the root node without requiring dummy targets, replacing the previous workaround of passing fake targets to proof generation.

View File

@@ -12,7 +12,7 @@ workflows:
# Check that `A` activates the features of `B`.
"propagate-feature",
# These are the features to check:
"--features=std,op,dev,asm-keccak,jemalloc,jemalloc-prof,tracy-allocator,tracy,serde-bincode-compat,serde,test-utils,arbitrary,bench,alloy-compat,min-error-logs,min-warn-logs,min-info-logs,min-debug-logs,min-trace-logs,otlp,otlp-logs,js-tracer,portable,keccak-cache-global",
"--features=std,op,dev,asm-keccak,jemalloc,jemalloc-prof,tracy-allocator,serde-bincode-compat,serde,test-utils,arbitrary,bench,alloy-compat,min-error-logs,min-warn-logs,min-info-logs,min-debug-logs,min-trace-logs,otlp,js-tracer,portable,keccak-cache-global",
# Do not try to add a new section to `[features]` of `A` only because `B` exposes that feature. There are edge-cases where this is still needed, but we can add them manually.
"--left-side-feature-missing=ignore",
# Ignore the case that `A` it outside of the workspace. Otherwise it will report errors in external dependencies that we have no influence on.

42
.github/CODEOWNERS vendored
View File

@@ -1,51 +1,45 @@
* @gakonst
crates/blockchain-tree-api/ @rakita @mattsse @Rjected
crates/blockchain-tree/ @rakita @mattsse @Rjected
crates/chain-state/ @fgimenez @mattsse
crates/chainspec/ @Rjected @joshieDo @mattsse
crates/cli/ @mattsse
crates/config/ @shekhirin @mattsse @Rjected
crates/consensus/ @mattsse @Rjected
crates/e2e-test-utils/ @mattsse @Rjected @klkvr @fgimenez
crates/engine/ @mattsse @Rjected @mediocregopher @yongkangc
crates/era/ @mattsse
crates/era-downloader/ @mattsse
crates/era-utils/ @mattsse
crates/engine/ @mattsse @Rjected @fgimenez @mediocregopher @yongkangc
crates/era/ @mattsse @RomanHodulak
crates/errors/ @mattsse
crates/ethereum-forks/ @mattsse @Rjected
crates/ethereum/ @mattsse @Rjected
crates/etl/ @joshieDo @shekhirin
crates/evm/ @mattsse @Rjected @klkvr
crates/evm/ @rakita @mattsse @Rjected
crates/exex/ @shekhirin
crates/fs-util/ @mattsse
crates/metrics/ @mattsse @Rjected
crates/net/ @mattsse @Rjected
crates/net/downloaders/ @Rjected
crates/node/ @mattsse @Rjected @klkvr
crates/optimism/ @mattsse @Rjected @fgimenez
crates/payload/ @mattsse @Rjected
crates/primitives-traits/ @Rjected @mattsse @klkvr
crates/primitives-traits/ @Rjected @RomanHodulak @mattsse @klkvr
crates/primitives/ @Rjected @mattsse @klkvr
crates/prune/ @shekhirin @joshieDo
crates/ress/ @shekhirin @Rjected
crates/revm/ @mattsse
crates/rpc/ @mattsse @Rjected
crates/ress @shekhirin @Rjected
crates/revm/ @mattsse @rakita
crates/rpc/ @mattsse @Rjected @RomanHodulak
crates/stages/ @shekhirin @mediocregopher
crates/static-file/ @joshieDo @shekhirin
crates/stateless/ @mattsse
crates/storage/codecs/ @joshieDo
crates/storage/db-api/ @joshieDo
crates/storage/db-api/ @joshieDo @rakita
crates/storage/db-common/ @Rjected
crates/storage/db/ @joshieDo
crates/storage/errors/ @joshieDo
crates/storage/libmdbx-rs/ @shekhirin
crates/storage/db/ @joshieDo @rakita
crates/storage/errors/ @rakita
crates/storage/libmdbx-rs/ @rakita @shekhirin
crates/storage/nippy-jar/ @joshieDo @shekhirin
crates/storage/provider/ @joshieDo @shekhirin @yongkangc
crates/storage/provider/ @rakita @joshieDo @shekhirin
crates/storage/storage-api/ @joshieDo
crates/tasks/ @mattsse
crates/tokio-util/ @mattsse
crates/tracing/ @mattsse @shekhirin
crates/tracing-otlp/ @mattsse @Rjected
crates/tokio-util/ @fgimenez
crates/transaction-pool/ @mattsse @yongkangc
crates/trie/ @Rjected @shekhirin @mediocregopher @yongkangc
bin/reth/ @mattsse @shekhirin @Rjected
bin/reth-bench/ @mattsse @Rjected @shekhirin @yongkangc
crates/trie/ @Rjected @shekhirin @mediocregopher
bin/reth-bench-compare/ @mediocregopher @shekhirin @yongkangc
etc/ @Rjected @shekhirin
.github/ @gakonst @DaniPopes

View File

@@ -43,6 +43,7 @@ body:
- `~/.cache/reth/logs` on Linux
- `~/Library/Caches/reth/logs` on macOS
- `%localAppData%/reth/logs` on Windows
render: text
validations:
required: false
@@ -57,6 +58,8 @@ body:
- Linux (ARM)
- Mac (Intel)
- Mac (Apple Silicon)
- Windows (x86)
- Windows (ARM)
- type: dropdown
id: container_type
attributes:

88
.github/assets/check_rv32imac.sh vendored Executable file
View File

@@ -0,0 +1,88 @@
#!/usr/bin/env bash
set +e # Disable immediate exit on error
# Array of crates to check
crates_to_check=(
reth-codecs-derive
reth-primitives
reth-primitives-traits
reth-network-peers
reth-trie-common
reth-trie-sparse
reth-chainspec
reth-consensus
reth-consensus-common
reth-prune-types
reth-static-file-types
reth-storage-errors
reth-execution-errors
reth-errors
reth-execution-types
reth-db-models
reth-evm
reth-revm
reth-storage-api
## ethereum
reth-evm-ethereum
reth-ethereum-forks
reth-ethereum-primitives
reth-ethereum-consensus
reth-stateless
## optimism
reth-optimism-chainspec
reth-optimism-forks
reth-optimism-consensus
reth-optimism-primitives
reth-optimism-evm
)
# Array to hold the results
results=()
# Flag to track if any command fails
any_failed=0
for crate in "${crates_to_check[@]}"; do
cmd="cargo +stable build -p $crate --target riscv32imac-unknown-none-elf --no-default-features"
if [ -n "$CI" ]; then
echo "::group::$cmd"
else
printf "\n%s:\n %s\n" "$crate" "$cmd"
fi
set +e # Disable immediate exit on error
# Run the command and capture the return code
$cmd
ret_code=$?
set -e # Re-enable immediate exit on error
# Store the result in the dictionary
if [ $ret_code -eq 0 ]; then
results+=("1:✅:$crate")
else
results+=("2:❌:$crate")
any_failed=1
fi
if [ -n "$CI" ]; then
echo "::endgroup::"
fi
done
# Sort the results by status and then by crate name
IFS=$'\n' sorted_results=($(sort <<<"${results[*]}"))
unset IFS
# Print summary
echo -e "\nSummary of build results:"
for result in "${sorted_results[@]}"; do
status="${result#*:}"
status="${status%%:*}"
crate="${result##*:}"
echo "$status $crate"
done
# Exit with a non-zero status if any command fails
exit $any_failed

View File

@@ -1,10 +1,11 @@
#!/usr/bin/env bash
set -uo pipefail
set +e # Disable immediate exit on error
readarray -t crates < <(
cargo metadata --format-version=1 --no-deps | jq -r '.packages[].name' | grep '^reth' | sort
)
# Array of crates to compile
crates=($(cargo metadata --format-version=1 --no-deps | jq -r '.packages[].name' | grep '^reth' | sort))
# Array of crates to exclude
# Used with the `contains` function.
# shellcheck disable=SC2034
exclude_crates=(
# The following require investigation if they can be fixed
@@ -39,6 +40,12 @@ exclude_crates=(
reth-node-ethereum
reth-node-events
reth-node-metrics
reth-optimism-cli
reth-optimism-flashblocks
reth-optimism-node
reth-optimism-payload-builder
reth-optimism-rpc
reth-optimism-storage
reth-rpc
reth-rpc-api
reth-rpc-api-testing-util
@@ -63,7 +70,6 @@ exclude_crates=(
reth-provider # tokio
reth-prune # tokio
reth-prune-static-files # reth-provider
reth-tasks # tokio rt-multi-thread
reth-stages-api # reth-provider, reth-prune
reth-static-file # tokio
reth-transaction-pool # c-kzg
@@ -71,41 +77,77 @@ exclude_crates=(
reth-trie-parallel # tokio
reth-trie-sparse-parallel # rayon
reth-testing-utils
reth-optimism-txpool # reth-transaction-pool
reth-era-downloader # tokio
reth-era-utils # tokio
reth-tracing-otlp
reth-node-ethstats
)
# Array to hold the results
results=()
# Flag to track if any command fails
any_failed=0
tmpdir=$(mktemp -d 2>/dev/null || mktemp -d -t reth-check)
trap 'rm -rf -- "$tmpdir"' EXIT INT TERM
# Function to check if a value exists in an array
contains() {
local array="$1[@]"
local seeking="$2"
local element
local seeking=$2
local in=1
for element in "${!array}"; do
[[ "$element" == "$seeking" ]] && return 0
if [[ "$element" == "$seeking" ]]; then
in=0
break
fi
done
return 1
return $in
}
for crate in "${crates[@]}"; do
if contains exclude_crates "$crate"; then
echo "⏭️ $crate"
results+=("3:⏭️:$crate")
continue
fi
outfile="$tmpdir/$crate.log"
if cargo +stable build -p "$crate" --target wasm32-wasip1 --no-default-features --color never >"$outfile" 2>&1; then
echo "$crate"
cmd="cargo +stable build -p $crate --target wasm32-wasip1 --no-default-features"
if [ -n "$CI" ]; then
echo "::group::$cmd"
else
echo "$crate"
sed 's/^/ /' "$outfile"
echo ""
printf "\n%s:\n %s\n" "$crate" "$cmd"
fi
set +e # Disable immediate exit on error
# Run the command and capture the return code
$cmd
ret_code=$?
set -e # Re-enable immediate exit on error
# Store the result in the dictionary
if [ $ret_code -eq 0 ]; then
results+=("1:✅:$crate")
else
results+=("2:❌:$crate")
any_failed=1
fi
if [ -n "$CI" ]; then
echo "::endgroup::"
fi
done
# Sort the results by status and then by crate name
IFS=$'\n' sorted_results=($(sort <<<"${results[*]}"))
unset IFS
# Print summary
echo -e "\nSummary of build results:"
for result in "${sorted_results[@]}"; do
status="${result#*:}"
status="${status%%:*}"
crate="${result##*:}"
echo "$status $crate"
done
# Exit with a non-zero status if any command fails
exit $any_failed

View File

@@ -38,6 +38,6 @@ for pid in "${saving_pids[@]}"; do
done
# Make sure we don't rebuild images on the CI jobs
git apply ../.github/scripts/hive/no_sim_build.diff
git apply ../.github/assets/hive/no_sim_build.diff
go build .
mv ./hive ../hive_assets/

View File

@@ -0,0 +1,36 @@
ethereum_package:
participants:
- el_type: reth
el_extra_params:
- "--rpc.eth-proof-window=100"
cl_type: teku
network_params:
preset: minimal
genesis_delay: 5
additional_preloaded_contracts: '
{
"0x4e59b44847b379578588920cA78FbF26c0B4956C": {
"balance": "0ETH",
"code": "0x7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe03601600081602082378035828234f58015156039578182fd5b8082525050506014600cf3",
"storage": {},
"nonce": "1"
}
}'
optimism_package:
chains:
chain0:
participants:
node0:
el:
type: op-geth
cl:
type: op-node
node1:
el:
type: op-reth
image: "ghcr.io/paradigmxyz/op-reth:kurtosis-ci"
cl:
type: op-node
network_params:
holocene_time_offset: 0
isthmus_time_offset: 0

View File

@@ -4,17 +4,3 @@ updates:
directory: "/"
schedule:
interval: "weekly"
- package-ecosystem: "cargo"
directory: "/"
schedule:
interval: "weekly"
labels:
- "A-dependencies"
commit-message:
prefix: "chore(deps)"
open-pull-requests-limit: 1
groups:
cargo-weekly:
applies-to: "version-updates"
patterns: ["*"]
update-types: ["minor", "patch"]

View File

@@ -1,48 +0,0 @@
#!/usr/bin/env bash
set -uo pipefail
crates_to_check=(
reth-codecs-derive
reth-primitives
reth-primitives-traits
reth-network-peers
reth-trie-common
reth-trie-sparse
reth-chainspec
reth-consensus
reth-consensus-common
reth-prune-types
reth-static-file-types
reth-storage-errors
reth-execution-errors
reth-errors
reth-execution-types
reth-db-models
reth-evm
reth-revm
reth-storage-api
## ethereum
reth-evm-ethereum
reth-ethereum-forks
reth-ethereum-primitives
reth-ethereum-consensus
)
any_failed=0
tmpdir=$(mktemp -d 2>/dev/null || mktemp -d -t reth-check)
trap 'rm -rf -- "$tmpdir"' EXIT INT TERM
for crate in "${crates_to_check[@]}"; do
outfile="$tmpdir/$crate.log"
if cargo +stable build -p "$crate" --target riscv32imac-unknown-none-elf --no-default-features --color never >"$outfile" 2>&1; then
echo "$crate"
else
echo "$crate"
sed 's/^/ /' "$outfile"
echo ""
any_failed=1
fi
done
exit $any_failed

14
.github/scripts/codspeed-build.sh vendored Executable file
View File

@@ -0,0 +1,14 @@
#!/usr/bin/env bash
set -eo pipefail
# TODO: Benchmarks run WAY too slow due to excessive amount of iterations.
cmd=(cargo codspeed build --profile profiling)
crates=(
-p reth-primitives
-p reth-trie
-p reth-trie-common
-p reth-trie-sparse
)
"${cmd[@]}" --features test-utils "${crates[@]}"

View File

@@ -1,53 +0,0 @@
#!/usr/bin/env bash
# Verifies that Docker images have the expected architectures.
#
# Usage:
# ./verify_image_arch.sh <targets> <registry> <ethereum_tags>
#
# Environment:
# DRY_RUN=true - Skip actual verification, just print what would be checked.
set -euo pipefail
TARGETS="${1:-}"
REGISTRY="${2:-}"
ETHEREUM_TAGS="${3:-}"
DRY_RUN="${DRY_RUN:-false}"
verify_image() {
local image="$1"
shift
local expected_archs=("$@")
echo "Checking $image..."
if [[ "$DRY_RUN" == "true" ]]; then
echo " [dry-run] Would verify architectures: ${expected_archs[*]}"
return 0
fi
manifest=$(docker manifest inspect "$image" 2>/dev/null) || {
echo "::error::Failed to inspect manifest for $image"
return 1
}
for arch in "${expected_archs[@]}"; do
if ! echo "$manifest" | jq -e ".manifests[] | select(.platform.architecture == \"$arch\" and .platform.os == \"linux\")" > /dev/null; then
echo "::error::Missing architecture $arch for $image"
return 1
fi
echo " ✓ linux/$arch"
done
}
if [[ "$TARGETS" == *"nightly"* ]]; then
verify_image "${REGISTRY}/reth:nightly" amd64 arm64
verify_image "${REGISTRY}/reth:nightly-profiling" amd64
verify_image "${REGISTRY}/reth:nightly-edge-profiling" amd64
else
for tag in $(echo "$ETHEREUM_TAGS" | tr ',' ' '); do
verify_image "$tag" amd64 arm64
done
fi
echo "All image architectures verified successfully"

View File

@@ -17,16 +17,6 @@ name: bench
jobs:
codspeed:
runs-on: depot-ubuntu-latest
strategy:
matrix:
partition: [1, 2]
total_partitions: [2]
include:
- partition: 1
crates: "-p reth-primitives -p reth-trie-common -p reth-trie-sparse"
- partition: 2
crates: "-p reth-trie"
name: codspeed (${{ matrix.partition }}/${{ matrix.total_partitions }})
steps:
- uses: actions/checkout@v6
with:
@@ -42,10 +32,10 @@ jobs:
with:
tool: cargo-codspeed
- name: Build the benchmark target(s)
run: cargo codspeed build --profile profiling --features test-utils ${{ matrix.crates }}
run: ./.github/scripts/codspeed-build.sh
- name: Run the benchmarks
uses: CodSpeedHQ/action@v4
with:
run: cargo codspeed run ${{ matrix.crates }}
run: cargo codspeed run --workspace
mode: instrumentation
token: ${{ secrets.CODSPEED_TOKEN }}

View File

@@ -1,25 +0,0 @@
name: Changelog
on:
pull_request:
types: [opened, synchronize]
jobs:
changelog:
# Skip for fork PRs since they can't access secrets
if: github.event.pull_request.head.repo.full_name == github.repository
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
ref: ${{ github.event.pull_request.head.sha }}
- run: npm install -g @anthropic-ai/claude-code
- uses: wevm/changelogs/check@master
with:
ai: 'claude -p'
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}

View File

@@ -1,65 +0,0 @@
# Checks reth compilation against alloy branches to detect breaking changes.
# Run on-demand via workflow_dispatch.
name: Check Alloy Breaking Changes
on:
workflow_dispatch:
inputs:
alloy_branch:
description: 'Branch/rev for alloy-rs/alloy (leave empty to skip)'
required: false
type: string
alloy_evm_branch:
description: 'Branch/rev for alloy-rs/evm (alloy-evm, alloy-op-evm) (leave empty to skip)'
required: false
type: string
op_alloy_branch:
description: 'Branch/rev for alloy-rs/op-alloy (leave empty to skip)'
required: false
type: string
env:
CARGO_TERM_COLOR: always
jobs:
check:
name: Check compilation with patched alloy
runs-on: depot-ubuntu-latest-16
timeout-minutes: 60
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
with:
cache-on-failure: true
- name: Apply alloy patches
run: |
ARGS=""
if [ -n "${{ inputs.alloy_branch }}" ]; then
ARGS="$ARGS --alloy ${{ inputs.alloy_branch }}"
fi
if [ -n "${{ inputs.alloy_evm_branch }}" ]; then
ARGS="$ARGS --evm ${{ inputs.alloy_evm_branch }}"
fi
if [ -n "${{ inputs.op_alloy_branch }}" ]; then
ARGS="$ARGS --op ${{ inputs.op_alloy_branch }}"
fi
if [ -z "$ARGS" ]; then
echo "No branches specified, nothing to patch"
exit 1
fi
./scripts/patch-alloy.sh $ARGS
echo "=== Final patch section ==="
tail -50 Cargo.toml
- name: Check workspace
run: cargo clippy --workspace --lib --examples --tests --benches --all-features --locked
env:
RUSTFLAGS: -D warnings

View File

@@ -23,6 +23,7 @@ jobs:
matrix:
bin:
- cargo run --bin reth --features "dev"
- cargo run --bin op-reth --features "dev" --manifest-path crates/optimism/bin/Cargo.toml
steps:
- uses: rui314/setup-mold@v1
- uses: dtolnay/rust-toolchain@stable

View File

@@ -15,7 +15,6 @@ permissions:
jobs:
update:
if: github.repository == 'paradigmxyz/reth'
uses: tempoxyz/ci/.github/workflows/cargo-update-pr.yml@main
uses: ithacaxyz/ci/.github/workflows/cargo-update-pr.yml@main
secrets:
token: ${{ secrets.GITHUB_TOKEN }}

54
.github/workflows/docker-git.yml vendored Normal file
View File

@@ -0,0 +1,54 @@
# Publishes the Docker image, only to be used with `workflow_dispatch`. The
# images from this workflow will be tagged with the git sha of the branch used
# and will NOT tag it as `latest`.
name: docker-git
on:
workflow_dispatch: {}
env:
REPO_NAME: ${{ github.repository_owner }}/reth
IMAGE_NAME: ${{ github.repository_owner }}/reth
OP_IMAGE_NAME: ${{ github.repository_owner }}/op-reth
CARGO_TERM_COLOR: always
DOCKER_IMAGE_NAME: ghcr.io/${{ github.repository_owner }}/reth
OP_DOCKER_IMAGE_NAME: ghcr.io/${{ github.repository_owner }}/op-reth
DOCKER_USERNAME: ${{ github.actor }}
GIT_SHA: ${{ github.sha }}
jobs:
build:
name: build and push
runs-on: ubuntu-24.04
permissions:
packages: write
contents: read
strategy:
fail-fast: false
matrix:
build:
- name: 'Build and push the git-sha-tagged reth image'
command: 'make PROFILE=maxperf GIT_SHA=$GIT_SHA docker-build-push-git-sha'
- name: 'Build and push the git-sha-tagged op-reth image'
command: 'make IMAGE_NAME=$OP_IMAGE_NAME DOCKER_IMAGE_NAME=$OP_DOCKER_IMAGE_NAME GIT_SHA=$GIT_SHA PROFILE=maxperf op-docker-build-push-git-sha'
steps:
- uses: actions/checkout@v6
- uses: rui314/setup-mold@v1
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
with:
cache-on-failure: true
- name: Install cross main
id: cross_main
run: |
cargo install cross --git https://github.com/cross-rs/cross
- name: Log in to Docker
run: |
echo "${{ secrets.GITHUB_TOKEN }}" | docker login ghcr.io --username ${DOCKER_USERNAME} --password-stdin
- name: Set up Docker builder
run: |
docker run --privileged --rm tonistiigi/binfmt --install arm64,amd64
docker buildx create --use --name cross-builder
- name: Build and push ${{ matrix.build.name }}
run: ${{ matrix.build.command }}

61
.github/workflows/docker-nightly.yml vendored Normal file
View File

@@ -0,0 +1,61 @@
# Publishes the nightly Docker image.
name: docker-nightly
on:
workflow_dispatch:
schedule:
- cron: "0 1 * * *"
env:
REPO_NAME: ${{ github.repository_owner }}/reth
IMAGE_NAME: ${{ github.repository_owner }}/reth
OP_IMAGE_NAME: ${{ github.repository_owner }}/op-reth
CARGO_TERM_COLOR: always
DOCKER_IMAGE_NAME: ghcr.io/${{ github.repository_owner }}/reth
OP_DOCKER_IMAGE_NAME: ghcr.io/${{ github.repository_owner }}/op-reth
DOCKER_USERNAME: ${{ github.actor }}
jobs:
build:
name: build and push
runs-on: ubuntu-24.04
permissions:
packages: write
contents: read
strategy:
fail-fast: false
matrix:
build:
- name: 'Build and push the nightly reth image'
command: 'make PROFILE=maxperf docker-build-push-nightly'
- name: 'Build and push the nightly profiling reth image'
command: 'make PROFILE=profiling docker-build-push-nightly-profiling'
- name: 'Build and push the nightly op-reth image'
command: 'make IMAGE_NAME=$OP_IMAGE_NAME DOCKER_IMAGE_NAME=$OP_DOCKER_IMAGE_NAME PROFILE=maxperf op-docker-build-push-nightly'
- name: 'Build and push the nightly profiling op-reth image'
command: 'make IMAGE_NAME=$OP_IMAGE_NAME DOCKER_IMAGE_NAME=$OP_DOCKER_IMAGE_NAME PROFILE=profiling op-docker-build-push-nightly-profiling'
steps:
- uses: actions/checkout@v6
- name: Remove bloatware
uses: laverdet/remove-bloatware@v1.0.0
with:
docker: true
lang: rust
- uses: rui314/setup-mold@v1
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
with:
cache-on-failure: true
- name: Install cross main
id: cross_main
run: |
cargo install cross --git https://github.com/cross-rs/cross
- name: Log in to Docker
run: |
echo "${{ secrets.GITHUB_TOKEN }}" | docker login ghcr.io --username ${DOCKER_USERNAME} --password-stdin
- name: Set up Docker builder
run: |
docker run --privileged --rm tonistiigi/binfmt --install arm64,amd64
docker buildx create --use --name cross-builder
- name: Build and push ${{ matrix.build.name }}
run: ${{ matrix.build.command }}

View File

@@ -14,6 +14,12 @@ on:
required: false
type: boolean
default: true
tag_op_reth:
description: 'Tag op-reth image as latest'
required: false
type: boolean
default: false
env:
DOCKER_USERNAME: ${{ github.actor }}
@@ -41,3 +47,27 @@ jobs:
- name: Push reth latest tag
run: |
docker push ghcr.io/${{ github.repository_owner }}/reth:latest
tag-op-reth-latest:
name: Tag op-reth as latest
runs-on: ubuntu-24.04
if: ${{ inputs.tag_op_reth }}
permissions:
packages: write
contents: read
steps:
- name: Log in to Docker
run: |
echo "${{ secrets.GITHUB_TOKEN }}" | docker login ghcr.io --username ${DOCKER_USERNAME} --password-stdin
- name: Pull op-reth release image
run: |
docker pull ghcr.io/${{ github.repository_owner }}/op-reth:${{ inputs.version }}
- name: Tag op-reth as latest
run: |
docker tag ghcr.io/${{ github.repository_owner }}/op-reth:${{ inputs.version }} ghcr.io/${{ github.repository_owner }}/op-reth:latest
- name: Push op-reth latest tag
run: |
docker push ghcr.io/${{ github.repository_owner }}/op-reth:latest

View File

@@ -1,53 +0,0 @@
name: Build test Docker image
on:
workflow_call:
inputs:
hive_target:
required: true
type: string
description: "Docker bake target to build (e.g. hive-stable, hive-edge)"
artifact_name:
required: false
type: string
default: "artifacts"
description: "Name for the uploaded artifact"
jobs:
build:
if: github.repository == 'paradigmxyz/reth'
timeout-minutes: 45
runs-on: ubuntu-latest
permissions:
id-token: write
contents: read
steps:
- uses: actions/checkout@v6
- run: mkdir -p artifacts
- name: Get git info
id: git
run: |
echo "sha=${{ github.sha }}" >> "$GITHUB_OUTPUT"
echo "describe=$(git describe --always --tags)" >> "$GITHUB_OUTPUT"
- name: Set up Depot CLI
uses: depot/setup-action@v1
- name: Build reth image
uses: depot/bake-action@v1
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
VERGEN_GIT_SHA: ${{ steps.git.outputs.sha }}
VERGEN_GIT_DESCRIBE: ${{ steps.git.outputs.describe }}
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
files: docker-bake.hcl
targets: ${{ inputs.hive_target }}
push: false
- name: Upload reth image
uses: actions/upload-artifact@v6
with:
name: ${{ inputs.artifact_name }}
path: ./artifacts

View File

@@ -1,9 +1,4 @@
# Publishes Docker images.
#
# Triggers:
# - Push tag v*: builds release (RC or latest)
# - Schedule: builds nightly + profiling
# - Manual: builds git-sha or nightly
# Publishes the Docker image.
name: docker
@@ -11,112 +6,84 @@ on:
push:
tags:
- v*
schedule:
- cron: "0 1 * * *"
workflow_dispatch:
inputs:
build_type:
description: "Build type"
required: true
type: choice
options:
- git-sha
- nightly
default: git-sha
dry_run:
description: "Skip pushing images (dry run)"
required: false
type: boolean
default: false
env:
IMAGE_NAME: ${{ github.repository_owner }}/reth
OP_IMAGE_NAME: ${{ github.repository_owner }}/op-reth
CARGO_TERM_COLOR: always
DOCKER_IMAGE_NAME: ghcr.io/${{ github.repository_owner }}/reth
OP_DOCKER_IMAGE_NAME: ghcr.io/${{ github.repository_owner }}/op-reth
DOCKER_USERNAME: ${{ github.actor }}
jobs:
build:
if: github.repository == 'paradigmxyz/reth'
name: Build Docker images
build-rc:
if: contains(github.ref, '-rc')
name: build and push as release candidate
runs-on: ubuntu-24.04
permissions:
packages: write
contents: read
id-token: write
strategy:
fail-fast: false
matrix:
build:
- name: "Build and push reth image"
command: "make IMAGE_NAME=$IMAGE_NAME DOCKER_IMAGE_NAME=$DOCKER_IMAGE_NAME PROFILE=maxperf docker-build-push"
- name: "Build and push op-reth image"
command: "make IMAGE_NAME=$OP_IMAGE_NAME DOCKER_IMAGE_NAME=$OP_DOCKER_IMAGE_NAME PROFILE=maxperf op-docker-build-push"
steps:
- uses: actions/checkout@v6
- name: Set up Depot CLI
uses: depot/setup-action@v1
- name: Log in to GHCR
uses: docker/login-action@v3
- uses: rui314/setup-mold@v1
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Get git info for vergen
id: git
cache-on-failure: true
- name: Install cross main
id: cross_main
run: |
echo "sha=${{ github.sha }}" >> "$GITHUB_OUTPUT"
echo "describe=$(git describe --always --tags)" >> "$GITHUB_OUTPUT"
echo "dirty=false" >> "$GITHUB_OUTPUT"
- name: Determine build parameters
id: params
cargo install cross --git https://github.com/cross-rs/cross
- name: Log in to Docker
run: |
REGISTRY="ghcr.io/${{ github.repository_owner }}"
if [[ "${{ github.event_name }}" == "push" ]]; then
VERSION="${GITHUB_REF#refs/tags/}"
echo "targets=ethereum" >> "$GITHUB_OUTPUT"
# Add 'latest' tag for non-RC releases
if [[ ! "$VERSION" =~ -rc ]]; then
echo "ethereum_tags=${REGISTRY}/reth:${VERSION},${REGISTRY}/reth:latest" >> "$GITHUB_OUTPUT"
else
echo "ethereum_tags=${REGISTRY}/reth:${VERSION}" >> "$GITHUB_OUTPUT"
fi
elif [[ "${{ github.event_name }}" == "schedule" ]] || [[ "${{ inputs.build_type }}" == "nightly" ]]; then
echo "targets=nightly" >> "$GITHUB_OUTPUT"
echo "ethereum_tags=${REGISTRY}/reth:nightly" >> "$GITHUB_OUTPUT"
else
# git-sha build
echo "targets=ethereum" >> "$GITHUB_OUTPUT"
echo "ethereum_tags=${REGISTRY}/reth:${{ github.sha }}" >> "$GITHUB_OUTPUT"
fi
- name: Build and push images
uses: depot/bake-action@v1
env:
VERGEN_GIT_SHA: ${{ steps.git.outputs.sha }}
VERGEN_GIT_DESCRIBE: ${{ steps.git.outputs.describe }}
VERGEN_GIT_DIRTY: ${{ steps.git.outputs.dirty }}
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
files: docker-bake.hcl
targets: ${{ steps.params.outputs.targets }}
push: ${{ !(github.event_name == 'workflow_dispatch' && inputs.dry_run) }}
set: |
ethereum.tags=${{ steps.params.outputs.ethereum_tags }}
- name: Verify image architectures
env:
DRY_RUN: ${{ github.event_name == 'workflow_dispatch' && inputs.dry_run }}
echo "${{ secrets.GITHUB_TOKEN }}" | docker login ghcr.io --username ${DOCKER_USERNAME} --password-stdin
- name: Set up Docker builder
run: |
./.github/scripts/verify_image_arch.sh \
"${{ steps.params.outputs.targets }}" \
"ghcr.io/${{ github.repository_owner }}" \
"${{ steps.params.outputs.ethereum_tags }}"
docker run --privileged --rm tonistiigi/binfmt --install arm64,amd64
docker buildx create --use --name cross-builder
- name: Build and push ${{ matrix.build.name }}
run: ${{ matrix.build.command }}
notify:
name: Notify on failure
runs-on: ubuntu-latest
needs: build
if: failure() && github.event_name == 'schedule'
build:
if: ${{ !contains(github.ref, '-rc') }}
name: build and push as latest
runs-on: ubuntu-24.04
permissions:
packages: write
contents: read
strategy:
fail-fast: false
matrix:
build:
- name: "Build and push reth image"
command: "make IMAGE_NAME=$IMAGE_NAME DOCKER_IMAGE_NAME=$DOCKER_IMAGE_NAME PROFILE=maxperf docker-build-push-latest"
- name: "Build and push op-reth image"
command: "make IMAGE_NAME=$OP_IMAGE_NAME DOCKER_IMAGE_NAME=$OP_DOCKER_IMAGE_NAME PROFILE=maxperf op-docker-build-push-latest"
steps:
- name: Slack Webhook Action
uses: rtCamp/action-slack-notify@v2
env:
SLACK_COLOR: ${{ job.status }}
SLACK_MESSAGE: "Failed run: https://github.com/paradigmxyz/reth/actions/runs/${{ github.run_id }}"
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK_URL }}
- uses: actions/checkout@v6
- uses: rui314/setup-mold@v1
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
with:
cache-on-failure: true
- name: Install cross main
id: cross_main
run: |
cargo install cross --git https://github.com/cross-rs/cross
- name: Log in to Docker
run: |
echo "${{ secrets.GITHUB_TOKEN }}" | docker login ghcr.io --username ${DOCKER_USERNAME} --password-stdin
- name: Set up Docker builder
run: |
docker run --privileged --rm tonistiigi/binfmt --install arm64,amd64
docker buildx create --use --name cross-builder
- name: Build and push ${{ matrix.build.name }}
run: ${{ matrix.build.command }}

View File

@@ -35,34 +35,12 @@ jobs:
- name: Run e2e tests
run: |
cargo nextest run \
--no-fail-fast \
--locked --features "asm-keccak" \
--workspace \
--exclude 'example-*' \
--exclude 'exex-subscription' \
--exclude 'reth-bench' \
--exclude 'ef-tests' \
--exclude 'op-reth' \
--exclude 'reth' \
-E 'binary(e2e_testsuite)'
rocksdb:
name: e2e-rocksdb
runs-on: depot-ubuntu-latest-4
env:
RUST_BACKTRACE: 1
timeout-minutes: 60
steps:
- uses: actions/checkout@v6
- uses: dtolnay/rust-toolchain@stable
- uses: mozilla-actions/sccache-action@v0.0.9
- uses: taiki-e/install-action@nextest
- uses: Swatinem/rust-cache@v2
with:
cache-on-failure: true
- name: Run RocksDB e2e tests
run: |
cargo nextest run \
--no-fail-fast \
--locked --features "edge" \
-p reth-e2e-test-utils \
-E 'binary(rocksdb)'

View File

@@ -5,7 +5,7 @@ name: hive
on:
workflow_dispatch:
schedule:
- cron: "0 0 * * *"
- cron: "0 */6 * * *"
env:
CARGO_TERM_COLOR: always
@@ -15,24 +15,16 @@ concurrency:
cancel-in-progress: true
jobs:
build-reth-stable:
uses: ./.github/workflows/docker-test.yml
prepare-reth:
uses: ./.github/workflows/prepare-reth.yml
with:
hive_target: hive-stable
artifact_name: "reth-stable"
secrets: inherit
build-reth-edge:
uses: ./.github/workflows/docker-test.yml
with:
hive_target: hive-edge
artifact_name: "reth-edge"
secrets: inherit
image_tag: ghcr.io/paradigmxyz/reth:latest
binary_name: reth
prepare-hive:
if: github.repository == 'paradigmxyz/reth'
timeout-minutes: 45
runs-on: depot-ubuntu-latest-4
runs-on: depot-ubuntu-latest-16
steps:
- uses: actions/checkout@v6
- name: Checkout hive tests
@@ -55,11 +47,11 @@ jobs:
uses: actions/cache@v5
with:
path: ./hive_assets
key: hive-assets-${{ steps.hive-commit.outputs.hash }}-${{ hashFiles('.github/scripts/hive/build_simulators.sh') }}
key: hive-assets-${{ steps.hive-commit.outputs.hash }}-${{ hashFiles('.github/assets/hive/build_simulators.sh') }}
- name: Build hive assets
if: steps.cache-hive.outputs.cache-hit != 'true'
run: .github/scripts/hive/build_simulators.sh
run: .github/assets/hive/build_simulators.sh
- name: Load cached Docker images
if: steps.cache-hive.outputs.cache-hit == 'true'
@@ -84,7 +76,6 @@ jobs:
strategy:
fail-fast: false
matrix:
storage: [stable, edge]
# ethereum/rpc to be deprecated:
# https://github.com/ethereum/hive/pull/1117
scenario:
@@ -94,7 +85,7 @@ jobs:
- sim: devp2p
limit: discv4
# started failing after https://github.com/ethereum/go-ethereum/pull/31843, no
# action on our side, remove from here when we get unexpected passes on these tests
# action on our side, remove from here when we get unxpected passes on these tests
# - sim: devp2p
# limit: eth
# include:
@@ -184,11 +175,10 @@ jobs:
- sim: ethereum/eels/consume-rlp
limit: .*tests/paris.*
needs:
- build-reth-stable
- build-reth-edge
- prepare-reth
- prepare-hive
name: ${{ matrix.storage }} / ${{ matrix.scenario.sim }}${{ matrix.scenario.limit && format(' - {0}', matrix.scenario.limit) }}
runs-on: depot-ubuntu-latest-4
name: run ${{ matrix.scenario.sim }}${{ matrix.scenario.limit && format(' - {0}', matrix.scenario.limit) }}
runs-on: depot-ubuntu-latest-16
permissions:
issues: write
steps:
@@ -205,11 +195,11 @@ jobs:
- name: Download reth image
uses: actions/download-artifact@v7
with:
name: reth-${{ matrix.storage }}
name: artifacts
path: /tmp
- name: Load Docker images
run: .github/scripts/hive/load_images.sh
run: .github/assets/hive/load_images.sh
- name: Move hive binary
run: |
@@ -237,11 +227,11 @@ jobs:
FILTER="/"
fi
echo "filter: $FILTER"
.github/scripts/hive/run_simulator.sh "${{ matrix.scenario.sim }}" "$FILTER"
.github/assets/hive/run_simulator.sh "${{ matrix.scenario.sim }}" "$FILTER"
- name: Parse hive output
run: |
find hivetests/workspace/logs -type f -name "*.json" ! -name "hive.json" | xargs -I {} python .github/scripts/hive/parse.py {} --exclusion .github/scripts/hive/expected_failures.yaml --ignored .github/scripts/hive/ignored_tests.yaml
find hivetests/workspace/logs -type f -name "*.json" ! -name "hive.json" | xargs -I {} python .github/assets/hive/parse.py {} --exclusion .github/assets/hive/expected_failures.yaml --ignored .github/assets/hive/ignored_tests.yaml
- name: Print simulator output
if: ${{ failure() }}
@@ -262,4 +252,4 @@ jobs:
env:
SLACK_COLOR: ${{ job.status }}
SLACK_MESSAGE: "Failed run: https://github.com/paradigmxyz/reth/actions/runs/${{ github.run_id }}"
SLACK_WEBHOOK: ${{ secrets.SLACK_HIVE_WEBHOOK_URL }}
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK_URL }}

View File

@@ -22,34 +22,38 @@ concurrency:
jobs:
test:
name: test / ${{ matrix.network }} / ${{ matrix.storage }}
name: test / ${{ matrix.network }}
if: github.event_name != 'schedule'
runs-on: depot-ubuntu-latest-4
env:
RUST_BACKTRACE: 1
strategy:
matrix:
network: ["ethereum"]
storage: ["stable", "edge"]
network: ["ethereum", "optimism"]
timeout-minutes: 60
steps:
- uses: actions/checkout@v6
- uses: rui314/setup-mold@v1
- uses: dtolnay/rust-toolchain@stable
- name: Install Geth
run: .github/scripts/install_geth.sh
run: .github/assets/install_geth.sh
- uses: taiki-e/install-action@nextest
- uses: mozilla-actions/sccache-action@v0.0.9
- uses: Swatinem/rust-cache@v2
with:
cache-on-failure: true
- name: Run tests
- if: matrix.network == 'ethereum'
name: Run tests
run: |
cargo nextest run \
--no-fail-fast \
--locked --features "asm-keccak ${{ matrix.network }} ${{ matrix.storage == 'edge' && 'edge' || '' }}" \
--locked --features "asm-keccak ${{ matrix.network }}" \
--workspace --exclude ef-tests \
-E "kind(test) and not binary(e2e_testsuite)"
- if: matrix.network == 'optimism'
name: Run tests
run: |
cargo nextest run \
--locked -p reth-optimism-node
integration-success:
name: integration success
@@ -65,7 +69,7 @@ jobs:
era-files:
name: era1 file integration tests once a day
if: github.event_name == 'schedule' && github.repository == 'paradigmxyz/reth'
if: github.event_name == 'schedule'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
@@ -77,4 +81,4 @@ jobs:
with:
cache-on-failure: true
- name: run era1 files integration tests
run: cargo nextest run --no-fail-fast --release --package reth-era --test it -- --ignored
run: cargo nextest run --release --package reth-era --test it -- --ignored

95
.github/workflows/kurtosis-op.yml vendored Normal file
View File

@@ -0,0 +1,95 @@
# Runs simple OP stack setup in Kurtosis
name: kurtosis-op
on:
workflow_dispatch:
schedule:
- cron: "0 */6 * * *"
push:
tags:
- "*"
env:
CARGO_TERM_COLOR: always
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true
jobs:
prepare-reth:
uses: ./.github/workflows/prepare-reth.yml
with:
image_tag: ghcr.io/paradigmxyz/op-reth:kurtosis-ci
binary_name: op-reth
cargo_features: asm-keccak
cargo_package: crates/optimism/bin/Cargo.toml
test:
timeout-minutes: 60
strategy:
fail-fast: false
name: run kurtosis
runs-on: depot-ubuntu-latest
needs:
- prepare-reth
steps:
- uses: actions/checkout@v6
with:
fetch-depth: 0
- name: Download reth image
uses: actions/download-artifact@v7
with:
name: artifacts
path: /tmp
- name: Load Docker image
run: |
docker load -i /tmp/reth_image.tar &
wait
docker image ls -a
- name: Install Foundry
uses: foundry-rs/foundry-toolchain@v1
- name: Run kurtosis
run: |
echo "deb [trusted=yes] https://apt.fury.io/kurtosis-tech/ /" | sudo tee /etc/apt/sources.list.d/kurtosis.list
sudo apt update
sudo apt install kurtosis-cli
kurtosis engine start
kurtosis run --enclave op-devnet github.com/ethpandaops/optimism-package --args-file .github/assets/kurtosis_op_network_params.yaml
ENCLAVE_ID=$(curl http://127.0.0.1:9779/api/enclaves | jq --raw-output 'keys[0]')
GETH_PORT=$(curl "http://127.0.0.1:9779/api/enclaves/$ENCLAVE_ID/services" | jq '."op-el-2151908-node0-op-geth".public_ports.rpc.number')
RETH_PORT=$(curl "http://127.0.0.1:9779/api/enclaves/$ENCLAVE_ID/services" | jq '."op-el-2151908-node1-op-reth".public_ports.rpc.number')
echo "GETH_RPC=http://127.0.0.1:$GETH_PORT" >> $GITHUB_ENV
echo "RETH_RPC=http://127.0.0.1:$RETH_PORT" >> $GITHUB_ENV
- name: Assert that clients advance
run: |
for i in {1..100}; do
sleep 5
BLOCK_GETH=$(cast bn --rpc-url $GETH_RPC)
BLOCK_RETH=$(cast bn --rpc-url $RETH_RPC)
if [ $BLOCK_GETH -ge 100 ] && [ $BLOCK_RETH -ge 100 ] ; then exit 0; fi
echo "Waiting for clients to advance..., Reth: $BLOCK_RETH Geth: $BLOCK_GETH"
done
kurtosis service logs -a op-devnet op-el-2151908-2-op-reth-op-node-op-kurtosis
kurtosis service logs -a op-devnet op-cl-2151908-2-op-node-op-reth-op-kurtosis
exit 1
notify-on-error:
needs: test
if: failure()
runs-on: ubuntu-latest
steps:
- name: Slack Webhook Action
uses: rtCamp/action-slack-notify@v2
env:
SLACK_COLOR: ${{ job.status }}
SLACK_MESSAGE: "Failed run: https://github.com/paradigmxyz/reth/actions/runs/${{ github.run_id }}"
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK_URL }}

View File

@@ -5,7 +5,7 @@ name: kurtosis
on:
workflow_dispatch:
schedule:
- cron: "0 0 * * *"
- cron: "0 */6 * * *"
push:
tags:
@@ -19,12 +19,11 @@ concurrency:
cancel-in-progress: true
jobs:
build-reth:
if: github.repository == 'paradigmxyz/reth'
uses: ./.github/workflows/docker-test.yml
prepare-reth:
uses: ./.github/workflows/prepare-reth.yml
with:
hive_target: kurtosis
secrets: inherit
image_tag: ghcr.io/paradigmxyz/reth:kurtosis-ci
binary_name: reth
test:
timeout-minutes: 60
@@ -33,7 +32,7 @@ jobs:
name: run kurtosis
runs-on: depot-ubuntu-latest
needs:
- build-reth
- prepare-reth
steps:
- uses: actions/checkout@v6
with:
@@ -66,4 +65,4 @@ jobs:
env:
SLACK_COLOR: ${{ job.status }}
SLACK_MESSAGE: "Failed run: https://github.com/paradigmxyz/reth/actions/runs/${{ github.run_id }}"
SLACK_WEBHOOK: ${{ secrets.SLACK_HIVE_WEBHOOK_URL }}
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK_URL }}

View File

@@ -19,5 +19,5 @@ jobs:
uses: actions/github-script@v8
with:
script: |
const label_pr = require('./.github/scripts/label_pr.js')
const label_pr = require('./.github/assets/label_pr.js')
await label_pr({github, context})

View File

@@ -76,7 +76,7 @@ jobs:
- name: Run Wasm checks
run: |
sudo apt update && sudo apt install gcc-multilib
.github/scripts/check_wasm.sh
.github/assets/check_wasm.sh
riscv:
runs-on: depot-ubuntu-latest
@@ -94,7 +94,7 @@ jobs:
cache-on-failure: true
- uses: dcarbone/install-jq-action@v3
- name: Run RISC-V checks
run: .github/scripts/check_rv32imac.sh
run: .github/assets/check_rv32imac.sh
crate-checks:
name: crate-checks (${{ matrix.partition }}/${{ matrix.total_partitions }})
@@ -119,6 +119,11 @@ jobs:
name: MSRV
runs-on: depot-ubuntu-latest
timeout-minutes: 30
strategy:
matrix:
include:
- binary: reth
- binary: op-reth
steps:
- uses: actions/checkout@v6
- uses: rui314/setup-mold@v1
@@ -129,7 +134,7 @@ jobs:
- uses: Swatinem/rust-cache@v2
with:
cache-on-failure: true
- run: cargo build --bin reth --workspace
- run: cargo build --bin "${{ matrix.binary }}" --workspace
env:
RUSTFLAGS: -D warnings
@@ -193,9 +198,10 @@ jobs:
with:
cache-on-failure: true
- run: cargo build --bin reth --workspace
- run: cargo build --bin op-reth --workspace
env:
RUSTFLAGS: -D warnings
- run: ./docs/cli/update.sh target/debug/reth
- run: ./docs/cli/update.sh target/debug/reth target/debug/op-reth
- name: Check docs changes
run: git diff --exit-code
@@ -271,7 +277,7 @@ jobs:
- uses: dtolnay/rust-toolchain@stable
- uses: mozilla-actions/sccache-action@v0.0.9
- uses: rui314/setup-mold@v1
- uses: taiki-e/cache-cargo-install-action@v3
- uses: taiki-e/cache-cargo-install-action@v2
with:
tool: zepter
- name: Eagerly pull dependencies
@@ -279,7 +285,7 @@ jobs:
- run: zepter run check
deny:
uses: tempoxyz/ci/.github/workflows/deny.yml@main
uses: ithacaxyz/ci/.github/workflows/deny.yml@main
lint-success:
name: lint success

56
.github/workflows/prepare-reth.yml vendored Normal file
View File

@@ -0,0 +1,56 @@
name: Prepare Reth Image
on:
workflow_call:
inputs:
image_tag:
required: true
type: string
description: "Docker image tag to use"
binary_name:
required: false
type: string
default: "reth"
description: "Binary name to build (reth or op-reth)"
cargo_features:
required: false
type: string
default: "asm-keccak"
description: "Cargo features to enable"
cargo_package:
required: false
type: string
description: "Optional cargo package path"
jobs:
prepare-reth:
if: github.repository == 'paradigmxyz/reth'
timeout-minutes: 45
runs-on: depot-ubuntu-latest
steps:
- uses: actions/checkout@v6
- run: mkdir artifacts
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Build and export reth image
uses: docker/build-push-action@v6
with:
context: .
file: .github/assets/hive/Dockerfile
tags: ${{ inputs.image_tag }}
outputs: type=docker,dest=./artifacts/reth_image.tar
build-args: |
CARGO_BIN=${{ inputs.binary_name }}
MANIFEST_PATH=${{ inputs.cargo_package }}
FEATURES=${{ inputs.cargo_features }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Upload reth image
id: upload
uses: actions/upload-artifact@v6
with:
name: artifacts
path: ./artifacts

View File

@@ -17,9 +17,11 @@ on:
env:
REPO_NAME: ${{ github.repository_owner }}/reth
IMAGE_NAME: ${{ github.repository_owner }}/reth
OP_IMAGE_NAME: ${{ github.repository_owner }}/op-reth
REPRODUCIBLE_IMAGE_NAME: ${{ github.repository_owner }}/reth-reproducible
CARGO_TERM_COLOR: always
DOCKER_IMAGE_NAME_URL: https://ghcr.io/${{ github.repository_owner }}/reth
DOCKER_OP_IMAGE_NAME_URL: https://ghcr.io/${{ github.repository_owner }}/op-reth
RUSTC_WRAPPER: "sccache"
jobs:
@@ -85,6 +87,10 @@ jobs:
os: macos-14
profile: maxperf
allow_fail: false
- target: x86_64-pc-windows-gnu
os: ubuntu-24.04
profile: maxperf
allow_fail: false
- target: riscv64gc-unknown-linux-gnu
os: ubuntu-24.04
profile: maxperf
@@ -92,6 +98,8 @@ jobs:
build:
- command: build
binary: reth
- command: op-build
binary: op-reth
steps:
- uses: actions/checkout@v6
- uses: rui314/setup-mold@v1
@@ -118,7 +126,8 @@ jobs:
- name: Move binary
run: |
mkdir artifacts
mv "target/${{ matrix.configs.target }}/${{ matrix.configs.profile }}/${{ matrix.build.binary }}" ./artifacts
[[ "${{ matrix.configs.target }}" == *windows* ]] && ext=".exe"
mv "target/${{ matrix.configs.target }}/${{ matrix.configs.profile }}/${{ matrix.build.binary }}${ext}" ./artifacts
- name: Configure GPG and create artifacts
env:
@@ -235,9 +244,21 @@ jobs:
|:---:|:---:|:---:|:---|
| <img src="https://www.svgrepo.com/download/473700/linux.svg" width="50"/> | x86_64 | [reth-${{ env.VERSION }}-x86_64-unknown-linux-gnu.tar.gz](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/reth-${{ env.VERSION }}-x86_64-unknown-linux-gnu.tar.gz) | [PGP Signature](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/reth-${{ env.VERSION }}-x86_64-unknown-linux-gnu.tar.gz.asc) |
| <img src="https://www.svgrepo.com/download/473700/linux.svg" width="50"/> | aarch64 | [reth-${{ env.VERSION }}-aarch64-unknown-linux-gnu.tar.gz](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/reth-${{ env.VERSION }}-aarch64-unknown-linux-gnu.tar.gz) | [PGP Signature](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/reth-${{ env.VERSION }}-aarch64-unknown-linux-gnu.tar.gz.asc) |
| <img src="https://www.svgrepo.com/download/513083/windows-174.svg" width="50"/> | x86_64 | [reth-${{ env.VERSION }}-x86_64-pc-windows-gnu.tar.gz](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/reth-${{ env.VERSION }}-x86_64-pc-windows-gnu.tar.gz) | [PGP Signature](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/reth-${{ env.VERSION }}-x86_64-pc-windows-gnu.tar.gz.asc) |
| <img src="https://www.svgrepo.com/download/511330/apple-173.svg" width="50"/> | x86_64 | [reth-${{ env.VERSION }}-x86_64-apple-darwin.tar.gz](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/reth-${{ env.VERSION }}-x86_64-apple-darwin.tar.gz) | [PGP Signature](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/reth-${{ env.VERSION }}-x86_64-apple-darwin.tar.gz.asc) |
| <img src="https://www.svgrepo.com/download/511330/apple-173.svg" width="50"/> | aarch64 | [reth-${{ env.VERSION }}-aarch64-apple-darwin.tar.gz](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/reth-${{ env.VERSION }}-aarch64-apple-darwin.tar.gz) | [PGP Signature](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/reth-${{ env.VERSION }}-aarch64-apple-darwin.tar.gz.asc) |
| <img src="https://www.svgrepo.com/download/473589/docker.svg" width="50"/> | Docker | [${{ env.IMAGE_NAME }}](${{ env.DOCKER_IMAGE_NAME_URL }}) | - |
### OP-Reth
| System | Architecture | Binary | PGP Signature |
|:---:|:---:|:---:|:---|
| <img src="https://www.svgrepo.com/download/473700/linux.svg" width="50"/> | x86_64 | [op-reth-${{ env.VERSION }}-x86_64-unknown-linux-gnu.tar.gz](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/op-reth-${{ env.VERSION }}-x86_64-unknown-linux-gnu.tar.gz) | [PGP Signature](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/op-reth-${{ env.VERSION }}-x86_64-unknown-linux-gnu.tar.gz.asc) |
| <img src="https://www.svgrepo.com/download/473700/linux.svg" width="50"/> | aarch64 | [op-reth-${{ env.VERSION }}-aarch64-unknown-linux-gnu.tar.gz](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/op-reth-${{ env.VERSION }}-aarch64-unknown-linux-gnu.tar.gz) | [PGP Signature](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/op-reth-${{ env.VERSION }}-aarch64-unknown-linux-gnu.tar.gz.asc) |
| <img src="https://www.svgrepo.com/download/513083/windows-174.svg" width="50"/> | x86_64 | [op-reth-${{ env.VERSION }}-x86_64-pc-windows-gnu.tar.gz](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/op-reth-${{ env.VERSION }}-x86_64-pc-windows-gnu.tar.gz) | [PGP Signature](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/op-reth-${{ env.VERSION }}-x86_64-pc-windows-gnu.tar.gz.asc) |
| <img src="https://www.svgrepo.com/download/511330/apple-173.svg" width="50"/> | x86_64 | [op-reth-${{ env.VERSION }}-x86_64-apple-darwin.tar.gz](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/op-reth-${{ env.VERSION }}-x86_64-apple-darwin.tar.gz) | [PGP Signature](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/op-reth-${{ env.VERSION }}-x86_64-apple-darwin.tar.gz.asc) |
| <img src="https://www.svgrepo.com/download/511330/apple-173.svg" width="50"/> | aarch64 | [op-reth-${{ env.VERSION }}-aarch64-apple-darwin.tar.gz](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/op-reth-${{ env.VERSION }}-aarch64-apple-darwin.tar.gz) | [PGP Signature](https://github.com/${{ env.REPO_NAME }}/releases/download/${{ env.VERSION }}/op-reth-${{ env.VERSION }}-aarch64-apple-darwin.tar.gz.asc) |
| <img src="https://www.svgrepo.com/download/473589/docker.svg" width="50"/> | Docker | [${{ env.OP_IMAGE_NAME }}](${{ env.DOCKER_OP_IMAGE_NAME_URL }}) | - |
ENDBODY
)
assets=()

View File

@@ -7,7 +7,6 @@ on:
jobs:
build:
if: github.repository == 'paradigmxyz/reth'
name: build reproducible binaries
runs-on: ${{ matrix.runner }}
strategy:

View File

@@ -38,7 +38,7 @@ jobs:
cache-on-failure: true
- name: Build reth
run: |
cargo install --path bin/reth
cargo install --features asm-keccak,jemalloc --path bin/reth
- name: Run headers stage
run: |
reth stage run headers --from ${{ env.FROM_BLOCK }} --to ${{ env.TO_BLOCK }} --commit --checkpoints

View File

@@ -9,7 +9,6 @@ on:
jobs:
close-issues:
if: github.repository == 'paradigmxyz/reth'
runs-on: ubuntu-latest
permissions:
issues: write

View File

@@ -17,7 +17,6 @@ concurrency:
jobs:
sync:
if: github.repository == 'paradigmxyz/reth'
name: sync (${{ matrix.chain.bin }})
runs-on: depot-ubuntu-latest
env:
@@ -33,6 +32,12 @@ jobs:
tip: "0x91c90676cab257a59cd956d7cb0bceb9b1a71d79755c23c7277a0697ccfaf8c4"
block: 100000
unwind-target: "0x52e0509d33a988ef807058e2980099ee3070187f7333aae12b64d4d675f34c5a"
- build: install-op
bin: op-reth
chain: base
tip: "0xbb9b85352c7ebca6ba8efc63bd66cecd038c92ec8ebd02e153a3e0b197e672b7"
block: 10000
unwind-target: "0x118a6e922a8c6cab221fc5adfe5056d2b72d58c6580e9c5629de55299e2cf8de"
steps:
- uses: actions/checkout@v6
- uses: rui314/setup-mold@v1

View File

@@ -17,7 +17,6 @@ concurrency:
jobs:
sync:
if: github.repository == 'paradigmxyz/reth'
name: sync (${{ matrix.chain.bin }})
runs-on: depot-ubuntu-latest
env:
@@ -33,6 +32,12 @@ jobs:
tip: "0x91c90676cab257a59cd956d7cb0bceb9b1a71d79755c23c7277a0697ccfaf8c4"
block: 100000
unwind-target: "0x52e0509d33a988ef807058e2980099ee3070187f7333aae12b64d4d675f34c5a"
- build: install-op
bin: op-reth
chain: base
tip: "0xbb9b85352c7ebca6ba8efc63bd66cecd038c92ec8ebd02e153a3e0b197e672b7"
block: 10000
unwind-target: "0x118a6e922a8c6cab221fc5adfe5056d2b72d58c6580e9c5629de55299e2cf8de"
steps:
- uses: actions/checkout@v6
- uses: rui314/setup-mold@v1

View File

@@ -19,19 +19,29 @@ concurrency:
jobs:
test:
name: test / ${{ matrix.type }} / ${{ matrix.storage }}
name: test / ${{ matrix.type }} (${{ matrix.partition }}/${{ matrix.total_partitions }})
runs-on: depot-ubuntu-latest-4
env:
RUST_BACKTRACE: 1
EDGE_FEATURES: ${{ matrix.storage == 'edge' && 'edge' || '' }}
strategy:
matrix:
type: [ethereum]
storage: [stable, edge]
include:
- type: ethereum
features: asm-keccak ethereum
exclude_args: ""
args: --features "asm-keccak ethereum" --locked
partition: 1
total_partitions: 2
- type: ethereum
args: --features "asm-keccak ethereum" --locked
partition: 2
total_partitions: 2
- type: optimism
args: --features "asm-keccak" --locked --exclude reth --exclude reth-bench --exclude "example-*" --exclude "reth-ethereum-*" --exclude "*-ethereum"
partition: 1
total_partitions: 2
- type: optimism
args: --features "asm-keccak" --locked --exclude reth --exclude reth-bench --exclude "example-*" --exclude "reth-ethereum-*" --exclude "*-ethereum"
partition: 2
total_partitions: 2
timeout-minutes: 30
steps:
- uses: actions/checkout@v6
@@ -49,10 +59,9 @@ jobs:
- name: Run tests
run: |
cargo nextest run \
--no-fail-fast \
--features "${{ matrix.features }} $EDGE_FEATURES" --locked \
${{ matrix.exclude_args }} --workspace \
${{ matrix.args }} --workspace \
--exclude ef-tests --no-tests=warn \
--partition hash:${{ matrix.partition }}/2 \
-E "!kind(test) and not binary(e2e_testsuite)"
state:
@@ -88,7 +97,7 @@ jobs:
- uses: Swatinem/rust-cache@v2
with:
cache-on-failure: true
- run: cargo nextest run --no-fail-fast --cargo-profile hivetests -p ef-tests --features "asm-keccak ef-tests"
- run: cargo nextest run --release -p ef-tests --features "asm-keccak ef-tests"
doc:
name: doc tests

36
.github/workflows/update-superchain.yml vendored Normal file
View File

@@ -0,0 +1,36 @@
name: Update Superchain Config
on:
schedule:
- cron: '0 3 * * 0'
workflow_dispatch:
permissions:
contents: write
jobs:
update-superchain:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Install required tools
run: |
sudo apt-get update
sudo apt-get install -y jq zstd qpdf yq
- name: Run fetch_superchain_config.sh
run: |
chmod +x crates/optimism/chainspec/res/fetch_superchain_config.sh
cd crates/optimism/chainspec/res
./fetch_superchain_config.sh
- name: Create Pull Request
uses: peter-evans/create-pull-request@v8
with:
commit-message: "chore: update superchain config"
title: "chore: update superchain config"
body: "This PR updates the superchain configs via scheduled workflow."
branch: "ci/update-superchain-config"
delete-branch: true

54
.github/workflows/windows.yml vendored Normal file
View File

@@ -0,0 +1,54 @@
# Windows build
name: windows
on:
push:
branches: [main]
pull_request:
branches: [main]
merge_group:
env:
RUSTC_WRAPPER: "sccache"
jobs:
check-reth:
runs-on: depot-ubuntu-latest
timeout-minutes: 60
steps:
- uses: actions/checkout@v6
- uses: rui314/setup-mold@v1
- uses: dtolnay/rust-toolchain@stable
with:
target: x86_64-pc-windows-gnu
- uses: taiki-e/install-action@cross
- uses: mozilla-actions/sccache-action@v0.0.9
- uses: Swatinem/rust-cache@v2
with:
cache-on-failure: true
- name: mingw-w64
run: sudo apt-get install -y mingw-w64
- name: Check Reth
run: cargo check --target x86_64-pc-windows-gnu
check-op-reth:
runs-on: depot-ubuntu-latest
timeout-minutes: 60
steps:
- uses: actions/checkout@v6
- uses: rui314/setup-mold@v1
- uses: dtolnay/rust-toolchain@stable
with:
target: x86_64-pc-windows-gnu
- uses: taiki-e/install-action@cross
- uses: mozilla-actions/sccache-action@v0.0.9
- uses: Swatinem/rust-cache@v2
with:
cache-on-failure: true
- name: mingw-w64
run: sudo apt-get install -y mingw-w64
- name: Check OP-Reth
run: cargo check -p op-reth --target x86_64-pc-windows-gnu

3
.gitignore vendored
View File

@@ -12,9 +12,6 @@ target/
# Generated by Intellij-based IDEs.
.idea
# ck-search metadata
.ck
# Generated by MacOS
.DS_Store

View File

@@ -18,13 +18,13 @@ Reth is a high-performance Ethereum execution client written in Rust, focusing o
6. **Pipeline (`crates/stages/`)**: Staged sync architecture for blockchain synchronization
7. **Trie (`crates/trie/`)**: Merkle Patricia Trie implementation with parallel state root computation
8. **Node Builder (`crates/node/`)**: High-level node orchestration and configuration
9. **The Consensus Engine (`crates/engine/`)**: Handles processing blocks received from the consensus layer with the Engine API (newPayload, forkchoiceUpdated)
9 **The Consensus Engine (`crates/engine/`)**: Handles processing blocks received from the consensus layer with the Engine API (newPayload, forkchoiceUpdated)
### Key Design Principles
- **Modularity**: Each crate can be used as a standalone library
- **Performance**: Extensive use of parallelism, memory-mapped I/O, and optimized data structures
- **Extensibility**: Traits and generic types allow for different chain implementations
- **Extensibility**: Traits and generic types allow for different implementations (Ethereum, Optimism, etc.)
- **Type Safety**: Strong typing throughout with minimal use of dynamic dispatch
## Development Workflow
@@ -38,7 +38,7 @@ Reth is a high-performance Ethereum execution client written in Rust, focusing o
2. **Linting**: Run clippy with all features
```bash
cargo +nightly clippy --workspace --lib --examples --tests --benches --all-features
RUSTFLAGS="-D warnings" cargo +nightly clippy --workspace --lib --examples --tests --benches --all-features --locked
```
3. **Testing**: Use nextest for faster test execution
@@ -169,16 +169,18 @@ Based on PR patterns, avoid:
Before submitting changes, ensure:
1. **Format Check**: `cargo +nightly fmt --all --check`
2. **Clippy**: No warnings
2. **Clippy**: No warnings with `RUSTFLAGS="-D warnings"`
3. **Tests Pass**: All unit and integration tests
4. **Documentation**: Update relevant docs and add doc comments with `cargo docs --document-private-items`
5. **Commit Messages**: Follow conventional format (feat:, fix:, chore:, etc.)
### Opening PRs against <https://github.com/paradigmxyz/reth>
Label PRs appropriately, first check the available labels and then apply the relevant ones:
* when changes are RPC related, add A-rpc label
* when changes are docs related, add C-docs label
* when changes are optimism related (e.g. new feature or exclusive changes to crates/optimism), add A-op-reth label
* ... and so on, check the available labels for more options.
* if being tasked to open a pr, ensure that all changes are properly formatted: `cargo +nightly fmt --all`
@@ -232,7 +234,7 @@ Tests often need expansion for:
Common refactoring pattern:
- Replace concrete types with generics
- Add trait bounds for flexibility
- Enable reuse across different chain types
- Enable reuse across different chain types (Ethereum, Optimism)
#### When to Comment
@@ -247,7 +249,7 @@ Write comments that remain valuable after the PR is merged. Future readers won't
unsafe impl GlobalAlloc for LimitedAllocator { ... }
// Binary search requires sorted input. Panics on unsorted slices.
fn find_index(items: &[Item], target: &Item) -> Option<usize>
fn find_index(items: &[Item], target: &Item) -> Option
// Timeout set to 5s to match EVM block processing limits
const TRACER_TIMEOUT: Duration = Duration::from_secs(5);
@@ -347,11 +349,11 @@ Let's say you want to fix a bug where external IP resolution fails on startup:
}
```
5. **Run checks** (IMPORTANT!):
5. **Run checks**:
```bash
cargo +nightly fmt --all
cargo clippy --workspace --all-features # Make sure WHOLE WORKSPACE compiles!
cargo nextest run -p reth-discv4
cargo clippy --all-features
cargo test -p reth-discv4
```
6. **Commit with clear message**:
@@ -372,7 +374,7 @@ Let's say you want to fix a bug where external IP resolution fails on startup:
cargo +nightly fmt --all
# Run lints
cargo +nightly clippy --workspace --all-features
RUSTFLAGS="-D warnings" cargo +nightly clippy --workspace --all-features --locked
# Run tests
cargo nextest run --workspace
@@ -381,7 +383,7 @@ cargo nextest run --workspace
cargo bench --bench bench_name
# Build optimized binary
cargo build --release
cargo build --release --features "jemalloc asm-keccak"
# Check compilation for all features
cargo check --workspace --all-features

View File

@@ -51,7 +51,9 @@ elsewhere.
<!-- - **Asking in the support Telegram:** The [Foundry Support Telegram][support-tg] is a fast and easy way to ask questions. -->
<!-- - **Opening a discussion:** This repository comes with a discussions board where you can also ask for help. Click the "Discussions" tab at the top. -->
If you have reviewed existing documentation and still have questions, or you are having problems, you can get help by **opening a discussion**. This repository comes with a discussions board where you can also ask for help. Click the "Discussions" tab at the top.
If you have reviewed existing documentation and still have questions, or you are having problems, you can get help by *
*opening a discussion**. This repository comes with a discussions board where you can also ask for help. Click the "
Discussions" tab at the top.
As Reth is still in heavy development, the documentation can be a bit scattered. The [Reth Docs][reth-docs] is our
current best-effort attempt at keeping up-to-date information.

3632
Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,5 +1,5 @@
[workspace.package]
version = "1.10.2"
version = "1.9.3"
edition = "2024"
rust-version = "1.88"
license = "MIT OR Apache-2.0"
@@ -72,6 +72,20 @@ members = [
"crates/node/events/",
"crates/node/metrics",
"crates/node/types",
"crates/optimism/bin",
"crates/optimism/chainspec",
"crates/optimism/cli",
"crates/optimism/consensus",
"crates/optimism/evm/",
"crates/optimism/flashblocks/",
"crates/optimism/hardforks/",
"crates/optimism/node/",
"crates/optimism/payload/",
"crates/optimism/primitives/",
"crates/optimism/reth/",
"crates/optimism/rpc/",
"crates/optimism/storage",
"crates/optimism/txpool/",
"crates/payload/basic/",
"crates/payload/builder/",
"crates/payload/builder-primitives/",
@@ -83,6 +97,8 @@ members = [
"crates/prune/db",
"crates/prune/prune",
"crates/prune/types",
"crates/ress/protocol",
"crates/ress/provider",
"crates/revm/",
"crates/rpc/ipc/",
"crates/rpc/rpc-api/",
@@ -99,6 +115,7 @@ members = [
"crates/stages/api/",
"crates/stages/stages/",
"crates/stages/types/",
"crates/stateless",
"crates/static-file/static-file",
"crates/static-file/types/",
"crates/storage/codecs/",
@@ -122,11 +139,13 @@ members = [
"crates/trie/db",
"crates/trie/parallel/",
"crates/trie/sparse",
"crates/trie/sparse-parallel/",
"crates/trie/trie",
"examples/beacon-api-sidecar-fetcher/",
"examples/beacon-api-sse/",
"examples/bsc-p2p",
"examples/custom-dev-node/",
"examples/custom-node/",
"examples/custom-engine-types/",
"examples/custom-evm/",
"examples/custom-hardforks/",
@@ -135,7 +154,10 @@ members = [
"examples/custom-payload-builder/",
"examples/custom-rlpx-subprotocol",
"examples/custom-rpc-middleware",
"examples/custom-node",
"examples/db-access",
"examples/engine-api-access",
"examples/exex-hello-world",
"examples/exex-subscription",
"examples/exex-test",
"examples/full-contract-state",
@@ -146,6 +168,7 @@ members = [
"examples/node-builder-api/",
"examples/node-custom-rpc/",
"examples/node-event-hooks/",
"examples/op-db-access/",
"examples/polygon-p2p/",
"examples/rpc-db/",
"examples/precompile-cache/",
@@ -306,11 +329,6 @@ inherits = "release"
lto = "fat"
codegen-units = 1
[profile.maxperf-symbols]
inherits = "maxperf"
debug = "full"
strip = "none"
[profile.reproducible]
inherits = "release"
panic = "abort"
@@ -319,6 +337,7 @@ incremental = false
[workspace.dependencies]
# reth
op-reth = { path = "crates/optimism/bin" }
reth = { path = "bin/reth" }
reth-storage-rpc-provider = { path = "crates/storage/rpc-provider" }
reth-basic-payload-builder = { path = "crates/payload/basic" }
@@ -367,6 +386,7 @@ reth-ethereum = { path = "crates/ethereum/reth" }
reth-etl = { path = "crates/etl" }
reth-evm = { path = "crates/evm/evm", default-features = false }
reth-evm-ethereum = { path = "crates/ethereum/evm", default-features = false }
reth-optimism-evm = { path = "crates/optimism/evm", default-features = false }
reth-execution-errors = { path = "crates/evm/execution-errors", default-features = false }
reth-execution-types = { path = "crates/evm/execution-types", default-features = false }
reth-exex = { path = "crates/exex/exex" }
@@ -393,7 +413,18 @@ reth-node-ethereum = { path = "crates/ethereum/node" }
reth-node-ethstats = { path = "crates/node/ethstats" }
reth-node-events = { path = "crates/node/events" }
reth-node-metrics = { path = "crates/node/metrics" }
reth-optimism-node = { path = "crates/optimism/node" }
reth-node-types = { path = "crates/node/types" }
reth-op = { path = "crates/optimism/reth", default-features = false }
reth-optimism-chainspec = { path = "crates/optimism/chainspec", default-features = false }
reth-optimism-cli = { path = "crates/optimism/cli", default-features = false }
reth-optimism-consensus = { path = "crates/optimism/consensus", default-features = false }
reth-optimism-forks = { path = "crates/optimism/hardforks", default-features = false }
reth-optimism-payload-builder = { path = "crates/optimism/payload" }
reth-optimism-primitives = { path = "crates/optimism/primitives", default-features = false }
reth-optimism-rpc = { path = "crates/optimism/rpc" }
reth-optimism-storage = { path = "crates/optimism/storage" }
reth-optimism-txpool = { path = "crates/optimism/txpool" }
reth-payload-builder = { path = "crates/payload/builder" }
reth-payload-builder-primitives = { path = "crates/payload/builder-primitives" }
reth-payload-primitives = { path = "crates/payload/primitives" }
@@ -414,11 +445,13 @@ reth-rpc-engine-api = { path = "crates/rpc/rpc-engine-api" }
reth-rpc-eth-api = { path = "crates/rpc/rpc-eth-api" }
reth-rpc-eth-types = { path = "crates/rpc/rpc-eth-types", default-features = false }
reth-rpc-layer = { path = "crates/rpc/rpc-layer" }
reth-optimism-flashblocks = { path = "crates/optimism/flashblocks" }
reth-rpc-server-types = { path = "crates/rpc/rpc-server-types" }
reth-rpc-convert = { path = "crates/rpc/rpc-convert" }
reth-stages = { path = "crates/stages/stages" }
reth-stages-api = { path = "crates/stages/api" }
reth-stages-types = { path = "crates/stages/types", default-features = false }
reth-stateless = { path = "crates/stateless", default-features = false }
reth-static-file = { path = "crates/static-file/static-file" }
reth-static-file-types = { path = "crates/static-file/types", default-features = false }
reth-storage-api = { path = "crates/storage/storage-api", default-features = false }
@@ -434,63 +467,66 @@ reth-trie-common = { path = "crates/trie/common", default-features = false }
reth-trie-db = { path = "crates/trie/db" }
reth-trie-parallel = { path = "crates/trie/parallel" }
reth-trie-sparse = { path = "crates/trie/sparse", default-features = false }
reth-trie-sparse-parallel = { path = "crates/trie/sparse-parallel" }
reth-zstd-compressors = { path = "crates/storage/zstd-compressors", default-features = false }
reth-ress-protocol = { path = "crates/ress/protocol" }
reth-ress-provider = { path = "crates/ress/provider" }
# revm
revm = { version = "34.0.0", default-features = false }
revm-bytecode = { version = "8.0.0", default-features = false }
revm-database = { version = "10.0.0", default-features = false }
revm-state = { version = "9.0.0", default-features = false }
revm-primitives = { version = "22.0.0", default-features = false }
revm-interpreter = { version = "32.0.0", default-features = false }
revm-database-interface = { version = "9.0.0", default-features = false }
op-revm = { version = "15.0.0", default-features = false }
revm-inspectors = "0.34.2"
revm = { version = "33.1.0", default-features = false }
revm-bytecode = { version = "7.1.1", default-features = false }
revm-database = { version = "9.0.5", default-features = false }
revm-state = { version = "8.1.1", default-features = false }
revm-primitives = { version = "21.0.2", default-features = false }
revm-interpreter = { version = "31.1.0", default-features = false }
revm-database-interface = { version = "8.0.5", default-features = false }
op-revm = { version = "14.1.0", default-features = false }
revm-inspectors = "0.33.2"
# eth
alloy-dyn-abi = "1.5.6"
alloy-primitives = { version = "1.5.6", default-features = false, features = ["map-foldhash"] }
alloy-sol-types = { version = "1.5.6", default-features = false }
alloy-chains = { version = "0.2.5", default-features = false }
alloy-dyn-abi = "1.4.1"
alloy-eip2124 = { version = "0.2.0", default-features = false }
alloy-eip7928 = { version = "0.3.0", default-features = false }
alloy-evm = { version = "0.27.2", default-features = false }
alloy-rlp = { version = "0.3.13", default-features = false, features = ["core-net"] }
alloy-trie = { version = "0.9.4", default-features = false }
alloy-eip7928 = { version = "0.1.0" }
alloy-evm = { version = "0.25.1", default-features = false }
alloy-primitives = { version = "1.5.0", default-features = false, features = ["map-foldhash"] }
alloy-rlp = { version = "0.3.10", default-features = false, features = ["core-net"] }
alloy-sol-macro = "1.5.0"
alloy-sol-types = { version = "1.5.0", default-features = false }
alloy-trie = { version = "0.9.1", default-features = false }
alloy-hardforks = "0.4.5"
alloy-consensus = { version = "1.6.3", default-features = false }
alloy-contract = { version = "1.6.3", default-features = false }
alloy-eips = { version = "1.6.3", default-features = false }
alloy-genesis = { version = "1.6.3", default-features = false }
alloy-json-rpc = { version = "1.6.3", default-features = false }
alloy-network = { version = "1.6.3", default-features = false }
alloy-network-primitives = { version = "1.6.3", default-features = false }
alloy-provider = { version = "1.6.3", features = ["reqwest", "debug-api"], default-features = false }
alloy-pubsub = { version = "1.6.3", default-features = false }
alloy-rpc-client = { version = "1.6.3", default-features = false }
alloy-rpc-types = { version = "1.6.3", features = ["eth"], default-features = false }
alloy-rpc-types-admin = { version = "1.6.3", default-features = false }
alloy-rpc-types-anvil = { version = "1.6.3", default-features = false }
alloy-rpc-types-beacon = { version = "1.6.3", default-features = false }
alloy-rpc-types-debug = { version = "1.6.3", default-features = false }
alloy-rpc-types-engine = { version = "1.6.3", default-features = false }
alloy-rpc-types-eth = { version = "1.6.3", default-features = false }
alloy-rpc-types-mev = { version = "1.6.3", default-features = false }
alloy-rpc-types-trace = { version = "1.6.3", default-features = false }
alloy-rpc-types-txpool = { version = "1.6.3", default-features = false }
alloy-serde = { version = "1.6.3", default-features = false }
alloy-signer = { version = "1.6.3", default-features = false }
alloy-signer-local = { version = "1.6.3", default-features = false }
alloy-transport = { version = "1.6.3" }
alloy-transport-http = { version = "1.6.3", features = ["reqwest-rustls-tls"], default-features = false }
alloy-transport-ipc = { version = "1.6.3", default-features = false }
alloy-transport-ws = { version = "1.6.3", default-features = false }
alloy-consensus = { version = "1.1.3", default-features = false }
alloy-contract = { version = "1.1.3", default-features = false }
alloy-eips = { version = "1.1.3", default-features = false }
alloy-genesis = { version = "1.1.3", default-features = false }
alloy-json-rpc = { version = "1.1.3", default-features = false }
alloy-network = { version = "1.1.3", default-features = false }
alloy-network-primitives = { version = "1.1.3", default-features = false }
alloy-provider = { version = "1.1.3", features = ["reqwest", "debug-api"], default-features = false }
alloy-pubsub = { version = "1.1.3", default-features = false }
alloy-rpc-client = { version = "1.1.3", default-features = false }
alloy-rpc-types = { version = "1.1.3", features = ["eth"], default-features = false }
alloy-rpc-types-admin = { version = "1.1.3", default-features = false }
alloy-rpc-types-anvil = { version = "1.1.3", default-features = false }
alloy-rpc-types-beacon = { version = "1.1.3", default-features = false }
alloy-rpc-types-debug = { version = "1.1.3", default-features = false }
alloy-rpc-types-engine = { version = "1.1.3", default-features = false }
alloy-rpc-types-eth = { version = "1.1.3", default-features = false }
alloy-rpc-types-mev = { version = "1.1.3", default-features = false }
alloy-rpc-types-trace = { version = "1.1.3", default-features = false }
alloy-rpc-types-txpool = { version = "1.1.3", default-features = false }
alloy-serde = { version = "1.1.3", default-features = false }
alloy-signer = { version = "1.1.3", default-features = false }
alloy-signer-local = { version = "1.1.3", default-features = false }
alloy-transport = { version = "1.1.3" }
alloy-transport-http = { version = "1.1.3", features = ["reqwest-rustls-tls"], default-features = false }
alloy-transport-ipc = { version = "1.1.3", default-features = false }
alloy-transport-ws = { version = "1.1.3", default-features = false }
# op
alloy-op-evm = { version = "0.27.2", default-features = false }
alloy-op-evm = { version = "0.25.0", default-features = false }
alloy-op-hardforks = "0.4.4"
op-alloy-rpc-types = { version = "0.23.1", default-features = false }
op-alloy-rpc-types-engine = { version = "0.23.1", default-features = false }
@@ -508,26 +544,24 @@ backon = { version = "1.2", default-features = false, features = ["std-blocking-
bincode = "1.3"
bitflags = "2.4"
boyer-moore-magiclen = "0.2.16"
bytes = { version = "1.11.1", default-features = false }
bytes = { version = "1.5", default-features = false }
brotli = "8"
cfg-if = "1.0"
clap = "4"
color-eyre = "0.6"
dashmap = "6.0"
derive_more = { version = "2", default-features = false, features = ["full"] }
dirs-next = "2.0.0"
dyn-clone = "1.0.17"
eyre = "0.6"
fdlimit = "0.3.0"
fixed-map = { version = "0.9", default-features = false }
humantime = "2.1"
humantime-serde = "1.1"
itertools = { version = "0.14", default-features = false }
linked_hash_set = "0.1"
lz4 = "1.28.1"
modular-bitfield = "0.13.1"
modular-bitfield = "0.11.2"
notify = { version = "8.0.0", default-features = false, features = ["macos_fsevent"] }
nybbles = { version = "0.4.8", default-features = false }
nybbles = { version = "0.4.2", default-features = false }
once_cell = { version = "1.19", default-features = false, features = ["critical-section"] }
parking_lot = "0.12"
paste = "1.0"
@@ -547,23 +581,22 @@ strum_macros = "0.27"
syn = "2.0"
thiserror = { version = "2.0.0", default-features = false }
tar = "0.4.44"
tracing = { version = "0.1.0", default-features = false, features = ["attributes"] }
tracing = { version = "0.1.0", default-features = false }
tracing-appender = "0.2"
url = { version = "2.3", default-features = false }
zstd = "0.13"
byteorder = "1"
fixed-cache = { version = "0.1.7", features = ["stats"] }
moka = "0.12"
tar-no-std = { version = "0.4.2", default-features = false }
miniz_oxide = { version = "0.9.0", default-features = false }
mini-moka = "0.10"
tar-no-std = { version = "0.3.2", default-features = false }
miniz_oxide = { version = "0.8.4", default-features = false }
chrono = "0.4.41"
# metrics
metrics = "0.24.0"
metrics-derive = "0.1.1"
metrics-exporter-prometheus = { version = "0.18.0", default-features = false }
metrics-derive = "0.1"
metrics-exporter-prometheus = { version = "0.16.0", default-features = false }
metrics-process = "2.1.0"
metrics-util = { default-features = false, version = "0.20.0" }
metrics-util = { default-features = false, version = "0.19.0" }
# proc-macros
proc-macro2 = "1.0"
@@ -572,7 +605,7 @@ quote = "1.0"
# tokio
tokio = { version = "1.44.2", default-features = false }
tokio-stream = "0.1.11"
tokio-tungstenite = "0.28.0"
tokio-tungstenite = "0.26.2"
tokio-util = { version = "0.7.4", features = ["codec"] }
# async
@@ -585,7 +618,7 @@ futures-util = { version = "0.3", default-features = false }
hyper = "1.3"
hyper-util = "0.1.5"
pin-project = "1.0.12"
reqwest = { version = "0.12", default-features = false, features = ["rustls-tls", "rustls-tls-native-roots", "stream"] }
reqwest = { version = "0.12", default-features = false }
tracing-futures = "0.2"
tower = "0.5"
tower-http = "0.6"
@@ -605,6 +638,7 @@ jsonrpsee-types = "0.26.0"
http = "1.0"
http-body = "1.0"
http-body-util = "0.1.2"
jsonwebtoken = "9"
proptest-arbitrary-interop = "0.1.0"
# crypto
@@ -618,7 +652,7 @@ rand_08 = { package = "rand", version = "0.8" }
c-kzg = "2.1.5"
# config
toml = "0.9"
toml = "0.8"
# rocksdb
rocksdb = { version = "0.24" }
@@ -628,28 +662,26 @@ opentelemetry_sdk = "0.31"
opentelemetry = "0.31"
opentelemetry-otlp = "0.31"
opentelemetry-semantic-conventions = "0.31"
opentelemetry-appender-tracing = "0.31"
tracing-opentelemetry = "0.32"
# misc-testing
arbitrary = "1.3"
assert_matches = "1.5.0"
criterion = { package = "codspeed-criterion-compat", version = "4.3" }
criterion = { package = "codspeed-criterion-compat", version = "2.7" }
insta = "1.41"
proptest = "1.7"
proptest-derive = "0.7"
proptest-derive = "0.5"
similar-asserts = { version = "1.5.0", features = ["serde"] }
tempfile = "3.20"
test-fuzz = "7"
rstest = "0.26.1"
rstest = "0.24.0"
test-case = "3"
# ssz encoding
ethereum_ssz = "0.10.1"
ethereum_ssz_derive = "0.10.1"
ethereum_ssz = "0.9.0"
ethereum_ssz_derive = "0.9.0"
# allocators
jemalloc_pprof = { version = "0.8", default-features = false }
tikv-jemalloc-ctl = "0.6"
tikv-jemallocator = "0.6"
tracy-client = "0.18.0"
@@ -658,14 +690,14 @@ snmalloc-rs = { version = "0.3.7", features = ["build_cc"] }
aes = "0.8.1"
ahash = "0.8"
anyhow = "1.0"
bindgen = { version = "0.72", default-features = false }
block-padding = "0.3"
cc = "1.2.15"
bindgen = { version = "0.71", default-features = false }
block-padding = "0.3.2"
cc = "=1.2.15"
cipher = "0.4.3"
comfy-table = "7.0"
concat-kdf = "0.1.0"
crossbeam-channel = "0.5.13"
crossterm = "0.29.0"
crossterm = "0.28.0"
csv = "1.3.0"
ctrlc = "3.4"
ctr = "0.9.2"
@@ -678,7 +710,7 @@ hmac = "0.12.1"
human_bytes = "0.4.1"
indexmap = "2"
interprocess = "2.2.0"
lz4_flex = { version = "0.12", default-features = false }
lz4_flex = { version = "0.11", default-features = false }
memmap2 = "0.9.4"
mev-share-sse = { version = "0.5.0", default-features = false }
num-traits = "0.2.15"
@@ -686,53 +718,29 @@ page_size = "0.6.0"
parity-scale-codec = "3.2.1"
plain_hasher = "0.2"
pretty_assertions = "1.4"
ratatui = { version = "0.30", default-features = false }
ringbuffer = "0.16.0"
ratatui = { version = "0.29", default-features = false }
ringbuffer = "0.15.0"
rmp-serde = "1.3"
roaring = "0.11.3"
roaring = "0.10.2"
rolling-file = "0.2.0"
sha3 = "0.10.5"
snap = "1.1.1"
socket2 = { version = "0.6", default-features = false }
sysinfo = { version = "0.38", default-features = false }
socket2 = { version = "0.5", default-features = false }
sysinfo = { version = "0.33", default-features = false }
tracing-journald = "0.3"
tracing-logfmt = "=0.3.5"
tracing-samply = "0.1"
tracing-logfmt = "0.3.3"
tracing-subscriber = { version = "0.3", default-features = false }
tracing-tracy = "0.11"
triehash = "0.8"
typenum = "1.15.0"
vergen = "9.1.0"
vergen = "9.0.4"
visibility = "0.1.1"
walkdir = "2.3.3"
vergen-git2 = "9.1.0"
vergen-git2 = "1.0.5"
# networking
ipnet = "2.11"
[patch.crates-io]
# revm staging patches
revm = { git = "https://github.com/bluealloy/revm", rev = "33330a285e621b9170c30a21cfea9ab32e2a2169" }
op-revm = { git = "https://github.com/bluealloy/revm", rev = "33330a285e621b9170c30a21cfea9ab32e2a2169" }
revm-bytecode = { git = "https://github.com/bluealloy/revm", rev = "33330a285e621b9170c30a21cfea9ab32e2a2169" }
revm-context = { git = "https://github.com/bluealloy/revm", rev = "33330a285e621b9170c30a21cfea9ab32e2a2169" }
revm-context-interface = { git = "https://github.com/bluealloy/revm", rev = "33330a285e621b9170c30a21cfea9ab32e2a2169" }
revm-database = { git = "https://github.com/bluealloy/revm", rev = "33330a285e621b9170c30a21cfea9ab32e2a2169" }
revm-database-interface = { git = "https://github.com/bluealloy/revm", rev = "33330a285e621b9170c30a21cfea9ab32e2a2169" }
revm-handler = { git = "https://github.com/bluealloy/revm", rev = "33330a285e621b9170c30a21cfea9ab32e2a2169" }
revm-inspector = { git = "https://github.com/bluealloy/revm", rev = "33330a285e621b9170c30a21cfea9ab32e2a2169" }
revm-interpreter = { git = "https://github.com/bluealloy/revm", rev = "33330a285e621b9170c30a21cfea9ab32e2a2169" }
revm-precompile = { git = "https://github.com/bluealloy/revm", rev = "33330a285e621b9170c30a21cfea9ab32e2a2169" }
revm-primitives = { git = "https://github.com/bluealloy/revm", rev = "33330a285e621b9170c30a21cfea9ab32e2a2169" }
revm-state = { git = "https://github.com/bluealloy/revm", rev = "33330a285e621b9170c30a21cfea9ab32e2a2169" }
# revm-inspectors staging patch
revm-inspectors = { git = "https://github.com/paradigmxyz/revm-inspectors", rev = "e80e2eab72dfa18011e6a99abd37027290a46e83" }
# alloy-evm staging patches
alloy-evm = { git = "https://github.com/alloy-rs/evm", rev = "742dc14749ea0279c03ca27b1c26f26ac19fbefb" }
alloy-op-evm = { git = "https://github.com/alloy-rs/evm", rev = "742dc14749ea0279c03ca27b1c26f26ac19fbefb" }
# [patch.crates-io]
# alloy-consensus = { git = "https://github.com/alloy-rs/alloy", rev = "3049f232fbb44d1909883e154eb38ec5962f53a3" }
# alloy-contract = { git = "https://github.com/alloy-rs/alloy", rev = "3049f232fbb44d1909883e154eb38ec5962f53a3" }
# alloy-eips = { git = "https://github.com/alloy-rs/alloy", rev = "3049f232fbb44d1909883e154eb38ec5962f53a3" }
@@ -775,10 +783,5 @@ alloy-op-evm = { git = "https://github.com/alloy-rs/evm", rev = "742dc14749ea027
# jsonrpsee-http-client = { git = "https://github.com/paradigmxyz/jsonrpsee", branch = "matt/make-rpc-service-pub" }
# jsonrpsee-types = { git = "https://github.com/paradigmxyz/jsonrpsee", branch = "matt/make-rpc-service-pub" }
# alloy-evm = { git = "https://github.com/alloy-rs/evm", rev = "df124c0" }
# alloy-op-evm = { git = "https://github.com/alloy-rs/evm", rev = "df124c0" }
# revm-inspectors = { git = "https://github.com/paradigmxyz/revm-inspectors", rev = "3020ea8" }
# alloy-evm = { git = "https://github.com/alloy-rs/evm", rev = "072c248" }
# alloy-op-evm = { git = "https://github.com/alloy-rs/evm", rev = "072c248" }
# alloy-evm = { git = "https://github.com/alloy-rs/evm", rev = "a69f0b45a6b0286e16072cb8399e02ce6ceca353" }
# alloy-op-evm = { git = "https://github.com/alloy-rs/evm", rev = "a69f0b45a6b0286e16072cb8399e02ce6ceca353" }

View File

@@ -15,6 +15,15 @@ pre-build = [
"apt-get update && apt-get install --assume-yes --no-install-recommends llvm-dev libclang-dev clang",
]
[target.x86_64-pc-windows-gnu]
# Why do we need a custom Dockerfile on Windows:
# 1. `reth-libmdbx` stopped working with MinGW 9.3 that cross image comes with.
# 2. To be able to update the version of MinGW, we need to also update the Ubuntu that the image is based on.
#
# Also see https://github.com/cross-rs/cross/issues/1667
# Inspired by https://github.com/cross-rs/cross/blob/9e2298e17170655342d3248a9c8ac37ef92ba38f/docker/Dockerfile.x86_64-pc-windows-gnu#L51
dockerfile = "./Dockerfile.x86_64-pc-windows-gnu"
[target.riscv64gc-unknown-linux-gnu]
image = "ubuntu:24.04"
pre-build = [

15
Dockerfile.cross Normal file
View File

@@ -0,0 +1,15 @@
# This image is meant to enable cross-architecture builds.
# It assumes the reth binary has already been compiled for `$TARGETPLATFORM` and is
# locatable in `./dist/bin/$TARGETARCH`
FROM --platform=$TARGETPLATFORM ubuntu:22.04
LABEL org.opencontainers.image.source=https://github.com/paradigmxyz/reth
LABEL org.opencontainers.image.licenses="MIT OR Apache-2.0"
# Filled by docker buildx
ARG TARGETARCH
COPY ./dist/bin/$TARGETARCH/reth /usr/local/bin/reth
EXPOSE 30303 30303/udp 9001 8545 8546
ENTRYPOINT ["/usr/local/bin/reth"]

View File

@@ -1,90 +0,0 @@
# syntax=docker/dockerfile:1
# Dockerfile for reth, optimized for Depot builds
# Usage:
# reth: --build-arg BINARY=reth
FROM rust:1 AS builder
WORKDIR /app
LABEL org.opencontainers.image.source=https://github.com/paradigmxyz/reth
LABEL org.opencontainers.image.licenses="MIT OR Apache-2.0"
RUN apt-get update && apt-get install -y libclang-dev pkg-config
# Install sccache for compilation caching
RUN cargo install sccache --locked
ENV RUSTC_WRAPPER=sccache
ENV SCCACHE_DIR=/sccache
ENV SCCACHE_WEBDAV_ENDPOINT=https://cache.depot.dev
# Binary to build
ARG BINARY=reth
# Manifest path for the binary
ARG MANIFEST_PATH=bin/reth
# Build profile, release by default
ARG BUILD_PROFILE=release
ENV BUILD_PROFILE=$BUILD_PROFILE
# Extra Cargo flags (can be overridden, otherwise set per-platform below)
ARG RUSTFLAGS=""
# Extra Cargo features
ARG FEATURES=""
ENV FEATURES=$FEATURES
# Git info for vergen (since .git is excluded from Docker context)
ARG VERGEN_GIT_SHA=""
ARG VERGEN_GIT_DESCRIBE=""
ARG VERGEN_GIT_DIRTY="false"
ENV VERGEN_GIT_SHA=$VERGEN_GIT_SHA
ENV VERGEN_GIT_DESCRIBE=$VERGEN_GIT_DESCRIBE
ENV VERGEN_GIT_DIRTY=$VERGEN_GIT_DIRTY
# Build application
# Platform-specific RUSTFLAGS: amd64 uses x86-64-v3 (Haswell+) with pclmulqdq for rocksdb
ARG TARGETPLATFORM
COPY --exclude=.git . .
RUN --mount=type=secret,id=DEPOT_TOKEN,env=SCCACHE_WEBDAV_TOKEN \
--mount=type=cache,target=/usr/local/cargo/registry,sharing=shared \
--mount=type=cache,target=/usr/local/cargo/git,sharing=shared \
--mount=type=cache,target=$SCCACHE_DIR,sharing=shared \
export RUSTC_WRAPPER=sccache SCCACHE_WEBDAV_ENDPOINT=https://cache.depot.dev SCCACHE_DIR=/sccache && \
sccache --start-server && \
if [ -n "$RUSTFLAGS" ]; then \
export RUSTFLAGS="$RUSTFLAGS"; \
elif [ "$TARGETPLATFORM" = "linux/amd64" ]; then \
export RUSTFLAGS="-C target-cpu=x86-64-v3 -C target-feature=+pclmulqdq"; \
fi && \
cargo build --profile $BUILD_PROFILE --features "$FEATURES" --locked --bin $BINARY --manifest-path $MANIFEST_PATH/Cargo.toml && \
sccache --show-stats
# Copy binary to a known location (ARG not resolved in COPY)
# Note: Custom profiles like maxperf/profiling output to target/<profile>/, not target/release/
RUN cp /app/target/$BUILD_PROFILE/$BINARY /app/binary || \
cp /app/target/release/$BINARY /app/binary
FROM ubuntu:24.04 AS runtime
WORKDIR /app
# Binary name for entrypoint
ARG BINARY=reth
# Install runtime dependencies
RUN apt-get update && \
apt-get install -y --no-install-recommends ca-certificates && \
rm -rf /var/lib/apt/lists/*
# Copy binary from build stage and create canonical symlink for entrypoint
COPY --from=builder /app/binary /usr/local/bin/
RUN mv /usr/local/bin/binary /usr/local/bin/$BINARY && \
ln -s /usr/local/bin/$BINARY /usr/local/bin/reth-binary && \
chmod +x /usr/local/bin/$BINARY
# Copy licenses
COPY LICENSE-* ./
EXPOSE 30303 30303/udp 9001 8545 8546
ENTRYPOINT ["/usr/local/bin/reth-binary"]

View File

@@ -0,0 +1,79 @@
FROM ubuntu:24.04 AS cross-base
ENV DEBIAN_FRONTEND=noninteractive
# Use HTTPS for package sources
RUN apt-get update && apt-get install --assume-yes --no-install-recommends ca-certificates
RUN find /etc/apt/ -type f \( -name '*.list' -o -name '*.sources' \) -exec sed -i 's|http://|https://|g' {} +
# Configure APT retries and timeouts to handle network issues
RUN echo 'Acquire::Retries \"3\";' > /etc/apt/apt.conf.d/80-retries && \
echo 'Acquire::http::Timeout \"60\";' >> /etc/apt/apt.conf.d/80-retries && \
echo 'Acquire::ftp::Timeout \"60\";' >> /etc/apt/apt.conf.d/80-retries
# configure fallback mirrors
RUN sed -i 's|URIs: https://archive.ubuntu.com/ubuntu/|URIs: https://mirror.cov.ukservers.com/ubuntu/ https://archive.ubuntu.com/ubuntu/ https://mirror.ox.ac.uk/sites/archive.ubuntu.com/ubuntu/|g' /etc/apt/sources.list.d/ubuntu.sources
RUN apt-get update && apt-get install --assume-yes --no-install-recommends git
RUN git clone https://github.com/cross-rs/cross /cross
WORKDIR /cross/docker
RUN git checkout baf457efc2555225af47963475bd70e8d2f5993f
# xargo doesn't work with Rust 1.89 and higher: https://github.com/cross-rs/cross/issues/1701.
#
# When this PR https://github.com/cross-rs/cross/pull/1580 is merged,
# we can update the checkout above and remove this replacement.
RUN sed -i 's|sh rustup-init.sh -y --no-modify-path --profile minimal|sh rustup-init.sh -y --no-modify-path --profile minimal --default-toolchain=1.88.0|' xargo.sh
RUN cp common.sh lib.sh / && /common.sh
RUN cp cmake.sh / && /cmake.sh
RUN cp xargo.sh / && /xargo.sh
FROM cross-base AS build
RUN apt-get install --assume-yes --no-install-recommends libz-mingw-w64-dev g++-mingw-w64-x86-64 gfortran-mingw-w64-x86-64
# Install Wine using OpenSUSE repository because official one is often lagging behind
RUN dpkg --add-architecture i386 && \
apt-get install --assume-yes --no-install-recommends wget gpg && \
mkdir -pm755 /etc/apt/keyrings && curl -fsSL \
https://download.opensuse.org/repositories/Emulators:/Wine:/Debian/xUbuntu_24.04/Release.key \
| tee /etc/apt/keyrings/obs-winehq.key >/dev/null && \
echo "deb [arch=amd64,i386 signed-by=/etc/apt/keyrings/obs-winehq.key] \
https://download.opensuse.org/repositories/Emulators:/Wine:/Debian/xUbuntu_24.04/ ./" \
| tee /etc/apt/sources.list.d/obs-winehq.list && \
apt-get update && apt-get install --assume-yes --install-recommends winehq-stable
# run-detectors are responsible for calling the correct interpreter for exe
# files. For some reason it does not work inside a docker container (it works
# fine in the host). So we replace the usual paths of run-detectors to run wine
# directly. This only affects the guest, we are not messing up with the host.
#
# See /usr/share/doc/binfmt-support/detectors
RUN mkdir -p /usr/lib/binfmt-support/ && \
rm -f /usr/lib/binfmt-support/run-detectors /usr/bin/run-detectors && \
ln -s /usr/bin/wine /usr/lib/binfmt-support/run-detectors && \
ln -s /usr/bin/wine /usr/bin/run-detectors
RUN cp windows-entry.sh /
ENTRYPOINT ["/windows-entry.sh"]
RUN cp toolchain.cmake /opt/toolchain.cmake
# for why we always link with pthread support, see:
# https://github.com/cross-rs/cross/pull/1123#issuecomment-1312287148
ENV CROSS_TOOLCHAIN_PREFIX=x86_64-w64-mingw32-
ENV CROSS_TOOLCHAIN_SUFFIX=-posix
ENV CROSS_SYSROOT=/usr/x86_64-w64-mingw32
ENV CROSS_TARGET_RUNNER="env -u CARGO_TARGET_X86_64_PC_WINDOWS_GNU_RUNNER wine"
ENV CARGO_TARGET_X86_64_PC_WINDOWS_GNU_LINKER="$CROSS_TOOLCHAIN_PREFIX"gcc"$CROSS_TOOLCHAIN_SUFFIX" \
CARGO_TARGET_X86_64_PC_WINDOWS_GNU_RUNNER="$CROSS_TARGET_RUNNER" \
AR_x86_64_pc_windows_gnu="$CROSS_TOOLCHAIN_PREFIX"ar \
CC_x86_64_pc_windows_gnu="$CROSS_TOOLCHAIN_PREFIX"gcc"$CROSS_TOOLCHAIN_SUFFIX" \
CXX_x86_64_pc_windows_gnu="$CROSS_TOOLCHAIN_PREFIX"g++"$CROSS_TOOLCHAIN_SUFFIX" \
CMAKE_TOOLCHAIN_FILE_x86_64_pc_windows_gnu=/opt/toolchain.cmake \
BINDGEN_EXTRA_CLANG_ARGS_x86_64_pc_windows_gnu="--sysroot=$CROSS_SYSROOT -idirafter/usr/include" \
CROSS_CMAKE_SYSTEM_NAME=Windows \
CROSS_CMAKE_SYSTEM_PROCESSOR=AMD64 \
CROSS_CMAKE_CRT=gnu \
CROSS_CMAKE_OBJECT_FLAGS="-ffunction-sections -fdata-sections -m64"

46
DockerfileOp Normal file
View File

@@ -0,0 +1,46 @@
FROM lukemathwalker/cargo-chef:latest-rust-1 AS chef
WORKDIR /app
LABEL org.opencontainers.image.source=https://github.com/paradigmxyz/reth
LABEL org.opencontainers.image.licenses="MIT OR Apache-2.0"
RUN apt-get update && apt-get -y upgrade && apt-get install -y libclang-dev pkg-config
# Builds a cargo-chef plan
FROM chef AS planner
COPY . .
RUN cargo chef prepare --recipe-path recipe.json
FROM chef AS builder
COPY --from=planner /app/recipe.json recipe.json
ARG BUILD_PROFILE=maxperf
ENV BUILD_PROFILE=$BUILD_PROFILE
ARG RUSTFLAGS=""
ENV RUSTFLAGS="$RUSTFLAGS"
ARG FEATURES=""
ENV FEATURES=$FEATURES
RUN cargo chef cook --profile $BUILD_PROFILE --features "$FEATURES" --recipe-path recipe.json --manifest-path /app/crates/optimism/bin/Cargo.toml
COPY . .
RUN cargo build --profile $BUILD_PROFILE --features "$FEATURES" --bin op-reth --manifest-path /app/crates/optimism/bin/Cargo.toml
RUN ls -la /app/target/$BUILD_PROFILE/op-reth
RUN cp /app/target/$BUILD_PROFILE/op-reth /app/op-reth
FROM ubuntu AS runtime
RUN apt-get update && \
apt-get install -y ca-certificates libssl-dev pkg-config strace && \
rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY --from=builder /app/op-reth /usr/local/bin/
RUN chmod +x /usr/local/bin/op-reth
COPY LICENSE-* ./
EXPOSE 30303 30303/udp 9001 8545 8546 7545 8551
ENTRYPOINT ["/usr/local/bin/op-reth"]

15
DockerfileOp.cross Normal file
View File

@@ -0,0 +1,15 @@
# This image is meant to enable cross-architecture builds.
# It assumes the reth binary has already been compiled for `$TARGETPLATFORM` and is
# locatable in `./dist/bin/$TARGETARCH`
FROM --platform=$TARGETPLATFORM ubuntu:22.04
LABEL org.opencontainers.image.source=https://github.com/paradigmxyz/reth
LABEL org.opencontainers.image.licenses="MIT OR Apache-2.0"
# Filled by docker buildx
ARG TARGETARCH
COPY ./dist/bin/$TARGETARCH/op-reth /usr/local/bin/op-reth
EXPOSE 30303 30303/udp 9001 8545 8546
ENTRYPOINT ["/usr/local/bin/op-reth"]

View File

@@ -24,3 +24,18 @@
parameters.
- Update version specific validation checks in the `EngineValidator` trait.
## Op-Reth changes
### Updates to the engine API
Opstack tries to be as close to the L1 engine API as much as possible. Isthmus (Prague equivalent) introduced the first
deviation from the L1 engine API with an additional field in the `ExecutionPayload`. For this reason the op engine API
has its own server traits `OpEngineApi`.
Adding a new versioned endpoint requires the same changes as for L1 just for the dedicated OP types.
### Hardforks
Opstack has dedicated hardforks (e.g. Isthmus), that can be entirely opstack specific (e.g. Holocene) or can be an L1
equivalent hardfork. Since opstack sticks to the L1 header primitive, a new L1 equivalent hardfork also requires new
equivalent consensus checks. For this reason these `OpHardfork` must be mapped to L1 `EthereumHardfork`, for example:
`OpHardfork::Isthmus` corresponds to `EthereumHardfork::Prague`. These mappings must be defined in the `ChainSpec`.

View File

@@ -186,7 +186,7 @@ APPENDIX: How to apply the Apache License to your work.
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2022-2026 Reth Contributors
Copyright 2022-2025 Reth Contributors
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.

View File

@@ -1,6 +1,6 @@
The MIT License (MIT)
Copyright (c) 2022-2026 Reth Contributors
Copyright (c) 2022-2025 Reth Contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal

200
Makefile
View File

@@ -12,7 +12,12 @@ FULL_DB_TOOLS_DIR := $(shell pwd)/$(DB_TOOLS_DIR)/
CARGO_TARGET_DIR ?= target
# List of features to use when building. Can be overridden via the environment.
FEATURES ?=
# No jemalloc on Windows
ifeq ($(OS),Windows_NT)
FEATURES ?= asm-keccak min-debug-logs
else
FEATURES ?= jemalloc asm-keccak min-debug-logs
endif
# Cargo profile for builds. Default is for local builds, CI uses an override.
PROFILE ?= release
@@ -30,6 +35,9 @@ EEST_TESTS_TAG := v4.5.0
EEST_TESTS_URL := https://github.com/ethereum/execution-spec-tests/releases/download/$(EEST_TESTS_TAG)/fixtures_stable.tar.gz
EEST_TESTS_DIR := ./testing/ef-tests/execution-spec-tests
# The docker image name
DOCKER_IMAGE_NAME ?= ghcr.io/paradigmxyz/reth
##@ Help
.PHONY: help
@@ -45,6 +53,13 @@ install: ## Build and install the reth binary under `$(CARGO_HOME)/bin`.
--profile "$(PROFILE)" \
$(CARGO_INSTALL_EXTRA_FLAGS)
.PHONY: install-op
install-op: ## Build and install the op-reth binary under `$(CARGO_HOME)/bin`.
cargo install --path crates/optimism/bin --bin op-reth --force --locked \
--features "$(FEATURES)" \
--profile "$(PROFILE)" \
$(CARGO_INSTALL_EXTRA_FLAGS)
.PHONY: build
build: ## Build the reth binary into `target` directory.
cargo build --bin reth --features "$(FEATURES)" --profile "$(PROFILE)"
@@ -72,10 +87,21 @@ build-%-reproducible:
.PHONY: build-debug
build-debug: ## Build the reth binary into `target/debug` directory.
cargo build --bin reth --features "$(FEATURES)"
.PHONY: build-debug-op
build-debug-op: ## Build the op-reth binary into `target/debug` directory.
cargo build --bin op-reth --features "$(FEATURES)" --manifest-path crates/optimism/bin/Cargo.toml
.PHONY: build-op
build-op: ## Build the op-reth binary into `target` directory.
cargo build --bin op-reth --features "$(FEATURES)" --profile "$(PROFILE)" --manifest-path crates/optimism/bin/Cargo.toml
# Builds the reth binary natively.
build-native-%:
cargo build --bin reth --target $* --features "$(FEATURES)" --profile "$(PROFILE)"
op-build-native-%:
cargo build --bin op-reth --target $* --features "$(FEATURES)" --profile "$(PROFILE)" --manifest-path crates/optimism/bin/Cargo.toml
# The following commands use `cross` to build a cross-compile.
#
# These commands require that:
@@ -92,6 +118,11 @@ build-native-%:
# on other systems. JEMALLOC_SYS_WITH_LG_PAGE=16 tells jemalloc to use 64-KiB
# pages. See: https://github.com/paradigmxyz/reth/issues/6742
build-aarch64-unknown-linux-gnu: export JEMALLOC_SYS_WITH_LG_PAGE=16
op-build-aarch64-unknown-linux-gnu: export JEMALLOC_SYS_WITH_LG_PAGE=16
# No jemalloc on Windows
build-x86_64-pc-windows-gnu: FEATURES := $(filter-out jemalloc jemalloc-prof,$(FEATURES))
op-build-x86_64-pc-windows-gnu: FEATURES := $(filter-out jemalloc jemalloc-prof,$(FEATURES))
# Note: The additional rustc compiler flags are for intrinsics needed by MDBX.
# See: https://github.com/cross-rs/cross/wiki/FAQ#undefined-reference-with-build-std
@@ -99,6 +130,10 @@ build-%:
RUSTFLAGS="-C link-arg=-lgcc -Clink-arg=-static-libgcc" \
cross build --bin reth --target $* --features "$(FEATURES)" --profile "$(PROFILE)"
op-build-%:
RUSTFLAGS="-C link-arg=-lgcc -Clink-arg=-static-libgcc" \
cross build --bin op-reth --target $* --features "$(FEATURES)" --profile "$(PROFILE)" --manifest-path crates/optimism/bin/Cargo.toml
# Unfortunately we can't easily use cross to build for Darwin because of licensing issues.
# If we wanted to, we would need to build a custom Docker image with the SDK available.
#
@@ -109,6 +144,11 @@ build-x86_64-apple-darwin:
$(MAKE) build-native-x86_64-apple-darwin
build-aarch64-apple-darwin:
$(MAKE) build-native-aarch64-apple-darwin
op-build-x86_64-apple-darwin:
$(MAKE) op-build-native-x86_64-apple-darwin
op-build-aarch64-apple-darwin:
$(MAKE) op-build-native-aarch64-apple-darwin
build-deb-%:
@case "$*" in \
x86_64-unknown-linux-gnu|aarch64-unknown-linux-gnu|riscv64gc-unknown-linux-gnu) \
@@ -144,6 +184,8 @@ build-release-tarballs: ## Create a series of `.tar.gz` files in the BIN_DIR dir
$(call tarball_release_binary,"x86_64-unknown-linux-gnu","reth","")
$(MAKE) build-aarch64-unknown-linux-gnu
$(call tarball_release_binary,"aarch64-unknown-linux-gnu","reth","")
$(MAKE) build-x86_64-pc-windows-gnu
$(call tarball_release_binary,"x86_64-pc-windows-gnu","reth.exe","")
##@ Test
@@ -153,7 +195,7 @@ COV_FILE := lcov.info
.PHONY: test-unit
test-unit: ## Run unit tests.
cargo install cargo-nextest --locked
cargo nextest run --no-fail-fast $(UNIT_TEST_ARGS)
cargo nextest run $(UNIT_TEST_ARGS)
.PHONY: cov-unit
@@ -186,7 +228,7 @@ $(EEST_TESTS_DIR):
.PHONY: ef-tests
ef-tests: $(EF_TESTS_DIR) $(EEST_TESTS_DIR) ## Runs Legacy and EEST tests.
cargo nextest run --no-fail-fast -p ef-tests --release --features ef-tests
cargo nextest run -p ef-tests --release --features ef-tests
##@ reth-bench
@@ -200,6 +242,127 @@ install-reth-bench: ## Build and install the reth binary under `$(CARGO_HOME)/bi
--features "$(FEATURES)" \
--profile "$(PROFILE)"
##@ Docker
# Note: This requires a buildx builder with emulation support. For example:
#
# `docker run --privileged --rm tonistiigi/binfmt --install amd64,arm64`
# `docker buildx create --use --driver docker-container --name cross-builder`
.PHONY: docker-build-push
docker-build-push: ## Build and push a cross-arch Docker image tagged with the latest git tag.
$(call docker_build_push,$(GIT_TAG),$(GIT_TAG))
# Note: This requires a buildx builder with emulation support. For example:
#
# `docker run --privileged --rm tonistiigi/binfmt --install amd64,arm64`
# `docker buildx create --use --driver docker-container --name cross-builder`
.PHONY: docker-build-push-git-sha
docker-build-push-git-sha: ## Build and push a cross-arch Docker image tagged with the latest git sha.
$(call docker_build_push,$(GIT_SHA),$(GIT_SHA))
# Note: This requires a buildx builder with emulation support. For example:
#
# `docker run --privileged --rm tonistiigi/binfmt --install amd64,arm64`
# `docker buildx create --use --driver docker-container --name cross-builder`
.PHONY: docker-build-push-latest
docker-build-push-latest: ## Build and push a cross-arch Docker image tagged with the latest git tag and `latest`.
$(call docker_build_push,$(GIT_TAG),latest)
# Note: This requires a buildx builder with emulation support. For example:
#
# `docker run --privileged --rm tonistiigi/binfmt --install amd64,arm64`
# `docker buildx create --use --name cross-builder`
.PHONY: docker-build-push-nightly
docker-build-push-nightly: ## Build and push cross-arch Docker image tagged with the latest git tag with a `-nightly` suffix, and `latest-nightly`.
$(call docker_build_push,nightly,nightly)
# Create a cross-arch Docker image with the given tags and push it
define docker_build_push
$(MAKE) build-x86_64-unknown-linux-gnu
mkdir -p $(BIN_DIR)/amd64
cp $(CARGO_TARGET_DIR)/x86_64-unknown-linux-gnu/$(PROFILE)/reth $(BIN_DIR)/amd64/reth
$(MAKE) build-aarch64-unknown-linux-gnu
mkdir -p $(BIN_DIR)/arm64
cp $(CARGO_TARGET_DIR)/aarch64-unknown-linux-gnu/$(PROFILE)/reth $(BIN_DIR)/arm64/reth
docker buildx build --file ./Dockerfile.cross . \
--platform linux/amd64,linux/arm64 \
--tag $(DOCKER_IMAGE_NAME):$(1) \
--tag $(DOCKER_IMAGE_NAME):$(2) \
--provenance=false \
--push
endef
##@ Optimism docker
# Note: This requires a buildx builder with emulation support. For example:
#
# `docker run --privileged --rm tonistiigi/binfmt --install amd64,arm64`
# `docker buildx create --use --driver docker-container --name cross-builder`
.PHONY: op-docker-build-push
op-docker-build-push: ## Build and push a cross-arch Docker image tagged with the latest git tag.
$(call op_docker_build_push,$(GIT_TAG),$(GIT_TAG))
# Note: This requires a buildx builder with emulation support. For example:
#
# `docker run --privileged --rm tonistiigi/binfmt --install amd64,arm64`
# `docker buildx create --use --driver docker-container --name cross-builder`
.PHONY: op-docker-build-push-git-sha
op-docker-build-push-git-sha: ## Build and push a cross-arch Docker image tagged with the latest git sha.
$(call op_docker_build_push,$(GIT_SHA),$(GIT_SHA))
# Note: This requires a buildx builder with emulation support. For example:
#
# `docker run --privileged --rm tonistiigi/binfmt --install amd64,arm64`
# `docker buildx create --use --driver docker-container --name cross-builder`
.PHONY: op-docker-build-push-latest
op-docker-build-push-latest: ## Build and push a cross-arch Docker image tagged with the latest git tag and `latest`.
$(call op_docker_build_push,$(GIT_TAG),latest)
# Note: This requires a buildx builder with emulation support. For example:
#
# `docker run --privileged --rm tonistiigi/binfmt --install amd64,arm64`
# `docker buildx create --use --name cross-builder`
.PHONY: op-docker-build-push-nightly
op-docker-build-push-nightly: ## Build and push cross-arch Docker image tagged with the latest git tag with a `-nightly` suffix, and `latest-nightly`.
$(call op_docker_build_push,nightly,nightly)
# Note: This requires a buildx builder with emulation support. For example:
#
# `docker run --privileged --rm tonistiigi/binfmt --install amd64,arm64`
# `docker buildx create --use --name cross-builder`
.PHONY: docker-build-push-nightly-profiling
docker-build-push-nightly-profiling: ## Build and push cross-arch Docker image with profiling profile tagged with nightly-profiling.
$(call docker_build_push,nightly-profiling,nightly-profiling)
# Note: This requires a buildx builder with emulation support. For example:
#
# `docker run --privileged --rm tonistiigi/binfmt --install amd64,arm64`
# `docker buildx create --use --name cross-builder`
.PHONY: op-docker-build-push-nightly-profiling
op-docker-build-push-nightly-profiling: ## Build and push cross-arch Docker image tagged with the latest git tag with a `-nightly` suffix, and `latest-nightly`.
$(call op_docker_build_push,nightly-profiling,nightly-profiling)
# Create a cross-arch Docker image with the given tags and push it
define op_docker_build_push
$(MAKE) op-build-x86_64-unknown-linux-gnu
mkdir -p $(BIN_DIR)/amd64
cp $(CARGO_TARGET_DIR)/x86_64-unknown-linux-gnu/$(PROFILE)/op-reth $(BIN_DIR)/amd64/op-reth
$(MAKE) op-build-aarch64-unknown-linux-gnu
mkdir -p $(BIN_DIR)/arm64
cp $(CARGO_TARGET_DIR)/aarch64-unknown-linux-gnu/$(PROFILE)/op-reth $(BIN_DIR)/arm64/op-reth
docker buildx build --file ./DockerfileOp.cross . \
--platform linux/amd64,linux/arm64 \
--tag $(DOCKER_IMAGE_NAME):$(1) \
--tag $(DOCKER_IMAGE_NAME):$(2) \
--provenance=false \
--push
endef
##@ Other
.PHONY: clean
@@ -227,21 +390,30 @@ db-tools: ## Compile MDBX debugging tools.
@echo "Run \"$(DB_TOOLS_DIR)/mdbx_chk\" for the MDBX db file integrity check."
.PHONY: update-book-cli
update-book-cli: build-debug ## Update book cli documentation.
update-book-cli: build-debug build-debug-op## Update book cli documentation.
@echo "Updating book cli doc..."
@./docs/cli/update.sh $(CARGO_TARGET_DIR)/debug/reth
@./docs/cli/update.sh $(CARGO_TARGET_DIR)/debug/reth $(CARGO_TARGET_DIR)/debug/op-reth
.PHONY: profiling
profiling: ## Builds `reth` with optimisations, but also symbols.
RUSTFLAGS="-C target-cpu=native" cargo build --profile profiling
RUSTFLAGS="-C target-cpu=native" cargo build --profile profiling --features jemalloc,asm-keccak
.PHONY: profiling-op
profiling-op: ## Builds `op-reth` with optimisations, but also symbols.
RUSTFLAGS="-C target-cpu=native" cargo build --profile profiling --features jemalloc,asm-keccak --bin op-reth --manifest-path crates/optimism/bin/Cargo.toml
.PHONY: maxperf
maxperf: ## Builds `reth` with the most aggressive optimisations.
RUSTFLAGS="-C target-cpu=native" cargo build --profile maxperf
RUSTFLAGS="-C target-cpu=native" cargo build --profile maxperf --features jemalloc,asm-keccak
.PHONY: maxperf-op
maxperf-op: ## Builds `op-reth` with the most aggressive optimisations.
RUSTFLAGS="-C target-cpu=native" cargo build --profile maxperf --features jemalloc,asm-keccak --bin op-reth --manifest-path crates/optimism/bin/Cargo.toml
.PHONY: maxperf-no-asm
maxperf-no-asm: ## Builds `reth` with the most aggressive optimisations, minus the "asm-keccak" feature.
RUSTFLAGS="-C target-cpu=native" cargo build --profile maxperf --no-default-features --features jemalloc,min-debug-logs,otlp,otlp-logs,reth-revm/portable,js-tracer,keccak-cache-global,rocksdb
RUSTFLAGS="-C target-cpu=native" cargo build --profile maxperf --features jemalloc
fmt:
cargo +nightly fmt
@@ -256,6 +428,17 @@ clippy:
--all-features \
-- -D warnings
clippy-op-dev:
cargo +nightly clippy \
--bin op-reth \
--workspace \
--lib \
--examples \
--tests \
--benches \
--locked \
--all-features
lint-typos: ensure-typos
typos
@@ -320,6 +503,7 @@ rustdocs: ## Runs `cargo docs` to generate the Rust documents in the `target/doc
cargo-test:
cargo test \
--workspace \
--bin "op-reth" \
--lib --examples \
--tests \
--benches \

View File

@@ -18,11 +18,6 @@
[gh-lint]: https://github.com/paradigmxyz/reth/actions/workflows/lint.yml
[tg-badge]: https://img.shields.io/endpoint?color=neon&logo=telegram&label=chat&url=https%3A%2F%2Ftg.sumanjay.workers.dev%2Fparadigm%5Freth
> **Note: OP-Reth has moved**
>
> The Optimism (op-reth) crates have been moved to [ethereum-optimism/optimism](https://github.com/ethereum-optimism/optimism).
> Git contribution history has been preserved. If you are looking for op-reth, please see the new repository.
## What is Reth?
Reth (short for Rust Ethereum, [pronunciation](https://x.com/kelvinfichter/status/1597653609411268608)) is a new Ethereum full node implementation that is focused on being user-friendly, highly modular, as well as being fast and efficient. Reth is an Execution Layer (EL) and is compatible with all Ethereum Consensus Layer (CL) implementations that support the [Engine API](https://github.com/ethereum/execution-apis/tree/a0d03086564ab1838b462befbc083f873dcf0c0f/src/engine). It is originally built and driven forward by [Paradigm](https://paradigm.xyz/), and is licensed under the Apache and MIT licenses.
@@ -37,7 +32,7 @@ More concretely, our goals are:
2. **Performance**: Reth aims to be fast, so we use Rust and the [Erigon staged-sync](https://erigon.substack.com/p/erigon-stage-sync-and-control-flows) node architecture. We also use our Ethereum libraries (including [Alloy](https://github.com/alloy-rs/alloy/) and [revm](https://github.com/bluealloy/revm/)) which we've battle-tested and optimized via [Foundry](https://github.com/foundry-rs/foundry/).
3. **Free for anyone to use any way they want**: Reth is free open source software, built for the community, by the community. By licensing the software under the Apache/MIT license, we want developers to use it without being bound by business licenses, or having to think about the implications of GPL-like licenses.
4. **Client Diversity**: The Ethereum protocol becomes more antifragile when no node implementation dominates. This ensures that if there's a software bug, the network does not finalize a bad block. By building a new client, we hope to contribute to Ethereum's antifragility.
5. **Support as many EVM chains as possible**: We aspire that Reth can full-sync not only Ethereum, but also other chains like Optimism, Polygon, BNB Smart Chain, and more. If you're working on any of these projects, please reach out. Note: OP-Reth has moved to [ethereum-optimism/optimism](https://github.com/ethereum-optimism/optimism).
5. **Support as many EVM chains as possible**: We aspire that Reth can full-sync not only Ethereum, but also other chains like Optimism, Polygon, BNB Smart Chain, and more. If you're working on any of these projects, please reach out.
6. **Configurability**: We want to solve for node operators that care about fast historical queries, but also for hobbyists who cannot operate on large hardware. We also want to support teams and individuals who want both sync from genesis and via "fast sync". We envision that Reth will be configurable enough and provide configurable "profiles" for the tradeoffs that each team faces.
## Status
@@ -46,13 +41,13 @@ Reth is production ready, and suitable for usage in mission-critical environment
More historical context below:
- We released 1.0 "production-ready" stable Reth in June 2024.
- Reth completed an audit with [Sigma Prime](https://sigmaprime.io/), the developers of [Lighthouse](https://github.com/sigp/lighthouse), the Rust Consensus Layer implementation. Find it [here](./audit/sigma_prime_audit_v2.pdf).
- Revm (the EVM used in Reth) underwent an audit with [Guido Vranken](https://x.com/guidovranken) (#1 [Ethereum Bug Bounty](https://ethereum.org/en/bug-bounty)). We will publish the results soon.
- We released multiple iterative beta versions, up to [beta.9](https://github.com/paradigmxyz/reth/releases/tag/v0.2.0-beta.9) on Monday June 3, 2024, the last beta release.
- We released [beta](https://github.com/paradigmxyz/reth/releases/tag/v0.2.0-beta.1) on Monday March 4, 2024, our first breaking change to the database model, providing faster query speed, smaller database footprint, and allowing "history" to be mounted on separate drives.
- We shipped iterative improvements until the last alpha release on February 28, 2024, [0.1.0-alpha.21](https://github.com/paradigmxyz/reth/releases/tag/v0.1.0-alpha.21).
- We [initially announced](https://www.paradigm.xyz/2023/06/reth-alpha) [0.1.0-alpha.1](https://github.com/paradigmxyz/reth/releases/tag/v0.1.0-alpha.1) on June 20, 2023.
- We released 1.0 "production-ready" stable Reth in June 2024.
- Reth completed an audit with [Sigma Prime](https://sigmaprime.io/), the developers of [Lighthouse](https://github.com/sigp/lighthouse), the Rust Consensus Layer implementation. Find it [here](./audit/sigma_prime_audit_v2.pdf).
- Revm (the EVM used in Reth) underwent an audit with [Guido Vranken](https://x.com/guidovranken) (#1 [Ethereum Bug Bounty](https://ethereum.org/en/bug-bounty)). We will publish the results soon.
- We released multiple iterative beta versions, up to [beta.9](https://github.com/paradigmxyz/reth/releases/tag/v0.2.0-beta.9) on Monday June 3, 2024,the last beta release.
- We released [beta](https://github.com/paradigmxyz/reth/releases/tag/v0.2.0-beta.1) on Monday March 4, 2024, our first breaking change to the database model, providing faster query speed, smaller database footprint, and allowing "history" to be mounted on separate drives.
- We shipped iterative improvements until the last alpha release on February 28, 2024, [0.1.0-alpha.21](https://github.com/paradigmxyz/reth/releases/tag/v0.1.0-alpha.21).
- We [initially announced](https://www.paradigm.xyz/2023/06/reth-alpha) [0.1.0-alpha.1](https://github.com/paradigmxyz/reth/releases/tag/v0.1.0-alpha.1) on June 20, 2023.
### Database compatibility
@@ -82,8 +77,8 @@ For a general overview of the crates, see [Project Layout](./docs/repo/layout.md
If you want to contribute, or follow along with contributor discussion, you can use our [main telegram](https://t.me/paradigm_reth) to chat with us about the development of Reth!
- Our contributor guidelines can be found in [`CONTRIBUTING.md`](./CONTRIBUTING.md).
- See our [contributor docs](./docs) for more information on the project. A good starting point is [Project Layout](./docs/repo/layout.md).
- Our contributor guidelines can be found in [`CONTRIBUTING.md`](./CONTRIBUTING.md).
- See our [contributor docs](./docs) for more information on the project. A good starting point is [Project Layout](./docs/repo/layout.md).
### Building and testing
@@ -128,9 +123,9 @@ If you have any questions, first see if the answer to your question can be found
If the answer is not there:
- Join the [Telegram][tg-url] to get help, or
- Open a [discussion](https://github.com/paradigmxyz/reth/discussions/new) with your question, or
- Open an issue with [the bug](https://github.com/paradigmxyz/reth/issues/new?assignees=&labels=C-bug%2CS-needs-triage&projects=&template=bug.yml)
- Join the [Telegram][tg-url] to get help, or
- Open a [discussion](https://github.com/paradigmxyz/reth/discussions/new) with your question, or
- Open an issue with [the bug](https://github.com/paradigmxyz/reth/issues/new?assignees=&labels=C-bug%2CS-needs-triage&projects=&template=bug.yml)
## Security
@@ -142,9 +137,9 @@ Reth is a new implementation of the Ethereum protocol. In the process of develop
None of this would have been possible without them, so big shoutout to the teams below:
- [Geth](https://github.com/ethereum/go-ethereum/): We would like to express our heartfelt gratitude to the go-ethereum team for their outstanding contributions to Ethereum over the years. Their tireless efforts and dedication have helped to shape the Ethereum ecosystem and make it the vibrant and innovative community it is today. Thank you for your hard work and commitment to the project.
- [Erigon](https://github.com/ledgerwatch/erigon) (fka Turbo-Geth): Erigon pioneered the ["Staged Sync" architecture](https://erigon.substack.com/p/erigon-stage-sync-and-control-flows) that Reth is using, as well as [introduced MDBX](https://github.com/ledgerwatch/erigon/wiki/Choice-of-storage-engine) as the database of choice. We thank Erigon for pushing the state of the art research on the performance limits of Ethereum nodes.
- [Akula](https://github.com/akula-bft/akula/): Reth uses forks of the Apache versions of Akula's [MDBX Bindings](https://github.com/paradigmxyz/reth/pull/132), [FastRLP](https://github.com/paradigmxyz/reth/pull/63) and [ECIES](https://github.com/paradigmxyz/reth/pull/80). Given that these packages were already released under the Apache License, and they implement standardized solutions, we decided not to reimplement them to iterate faster. We thank the Akula team for their contributions to the Rust Ethereum ecosystem and for publishing these packages.
- [Geth](https://github.com/ethereum/go-ethereum/): We would like to express our heartfelt gratitude to the go-ethereum team for their outstanding contributions to Ethereum over the years. Their tireless efforts and dedication have helped to shape the Ethereum ecosystem and make it the vibrant and innovative community it is today. Thank you for your hard work and commitment to the project.
- [Erigon](https://github.com/ledgerwatch/erigon) (fka Turbo-Geth): Erigon pioneered the ["Staged Sync" architecture](https://erigon.substack.com/p/erigon-stage-sync-and-control-flows) that Reth is using, as well as [introduced MDBX](https://github.com/ledgerwatch/erigon/wiki/Choice-of-storage-engine) as the database of choice. We thank Erigon for pushing the state of the art research on the performance limits of Ethereum nodes.
- [Akula](https://github.com/akula-bft/akula/): Reth uses forks of the Apache versions of Akula's [MDBX Bindings](https://github.com/paradigmxyz/reth/pull/132), [FastRLP](https://github.com/paradigmxyz/reth/pull/63) and [ECIES](https://github.com/paradigmxyz/reth/pull/80). Given that these packages were already released under the Apache License, and they implement standardized solutions, we decided not to reimplement them to iterate faster. We thank the Akula team for their contributions to the Rust Ethereum ecosystem and for publishing these packages.
## Warning

View File

@@ -25,9 +25,7 @@ reth-chainspec.workspace = true
# alloy
alloy-provider = { workspace = true, features = ["reqwest-rustls-tls"], default-features = false }
alloy-rpc-client = { workspace = true, features = ["pubsub"] }
alloy-rpc-types-eth.workspace = true
alloy-transport-ws.workspace = true
alloy-primitives.workspace = true
# CLI and argument parsing
@@ -56,7 +54,7 @@ ctrlc.workspace = true
shlex.workspace = true
[target.'cfg(unix)'.dependencies]
nix = { version = "0.31", features = ["signal", "process"] }
nix = { version = "0.29", features = ["signal", "process"] }
[features]
default = ["jemalloc"]
@@ -71,11 +69,7 @@ jemalloc = [
"reth-node-core/jemalloc",
]
jemalloc-prof = ["reth-cli-util/jemalloc-prof"]
tracy-allocator = ["reth-cli-util/tracy-allocator", "tracy"]
tracy = [
"reth-node-core/tracy",
"reth-tracing/tracy",
]
tracy-allocator = ["reth-cli-util/tracy-allocator"]
min-error-logs = [
"tracing/release_max_level_error",
@@ -98,5 +92,5 @@ min-trace-logs = [
"reth-node-core/min-trace-logs",
]
# no-op feature flag for CI matrices
# no-op feature flag for switching between the `optimism` and default functionality in CI matrices
ethereum = []

View File

@@ -1,50 +0,0 @@
# reth-bench-compare
Compare reth performance between two git references.
## Usage
```bash
reth-bench-compare \
--baseline-ref main \
--feature-ref my-feature \
--blocks 100 \
--wait-for-persistence
```
## Arguments
| Argument | Description | Default | Required |
|----------|-------------|---------|----------|
| `--baseline-ref <REF>` | Git reference for baseline | - | Yes |
| `--feature-ref <REF>` | Git reference to compare | - | Yes |
| `--blocks <N>` | Number of blocks to benchmark | `100` | No |
| `--chain <CHAIN>` | Chain to benchmark | `mainnet` | No |
| `--datadir <PATH>` | Data directory path | OS-specific | No |
| `--rpc-url <URL>` | RPC endpoint for block data | Chain default | No |
| `--output-dir <PATH>` | Output directory | `./reth-bench-compare` | No |
| `--wait-for-persistence` | Wait for block persistence | `false` | No |
| `--persistence-threshold <N>` | Wait after every N+1 blocks | `2` | No |
| `--wait-time <DURATION>` | Fixed delay (legacy) | - | No |
| `--warmup-blocks <N>` | Cache warmup blocks | Same as `--blocks` | No |
| `--draw` | Generate charts (needs Python/uv) | `false` | No |
| `--profile` | Enable CPU profiling (needs samply) | `false` | No |
| `-vvvv` | Debug logging | Info | No |
| `--features <FEATURES>` | Extra Rust features for both builds | - | No |
| `--rustflags <FLAGS>` | RUSTFLAGS for both builds | `-C target-cpu=native` | No |
| `--baseline-features <FEATURES>` | Features for baseline only | Inherits `--features` | No |
| `--feature-features <FEATURES>` | Features for feature only | Inherits `--features` | No |
| `--baseline-rustflags <FLAGS>` | RUSTFLAGS for baseline only | Inherits `--rustflags` | No |
| `--feature-rustflags <FLAGS>` | RUSTFLAGS for feature only | Inherits `--rustflags` | No |
| `--baseline-args <ARGS>` | Extra args for baseline node | - | No |
| `--feature-args <ARGS>` | Extra args for feature node | - | No |
| `--metrics-port <PORT>` | Metrics endpoint port | `5005` | No |
| `--sudo` | Run with elevated privileges | `false` | No |
## Output
Results in `./reth-bench-compare/results/<timestamp>/`:
- `comparison_report.json` - Metrics comparison
- `per_block_comparison.csv` - Per-block statistics
- `baseline/` and `feature/` - Individual run results
- `latency_comparison.png` - Chart (if `--draw` used)

View File

@@ -18,8 +18,6 @@ pub(crate) struct BenchmarkRunner {
rpc_url: String,
jwt_secret: String,
wait_time: Option<String>,
wait_for_persistence: bool,
persistence_threshold: Option<u64>,
warmup_blocks: u64,
}
@@ -30,8 +28,6 @@ impl BenchmarkRunner {
rpc_url: args.get_rpc_url(),
jwt_secret: args.jwt_secret_path().to_string_lossy().to_string(),
wait_time: args.wait_time.clone(),
wait_for_persistence: args.wait_for_persistence,
persistence_threshold: args.persistence_threshold,
warmup_blocks: args.get_warmup_blocks(),
}
}
@@ -100,9 +96,13 @@ impl BenchmarkRunner {
&from_block.to_string(),
"--to",
&to_block.to_string(),
"--wait-time=0ms", // Warmup should avoid persistence waits.
]);
// Add wait-time argument if provided
if let Some(ref wait_time) = self.wait_time {
cmd.args(["--wait-time", wait_time]);
}
cmd.env("RUST_LOG_STYLE", "never")
.stdout(std::process::Stdio::piped())
.stderr(std::process::Stdio::piped())
@@ -186,19 +186,10 @@ impl BenchmarkRunner {
&output_dir.to_string_lossy(),
]);
// Configure wait mode: both can be used together
// When both are set: wait at least wait_time, and also wait for persistence if needed
// Add wait-time argument if provided
if let Some(ref wait_time) = self.wait_time {
cmd.args(["--wait-time", wait_time]);
}
if self.wait_for_persistence {
cmd.arg("--wait-for-persistence");
// Add persistence threshold if specified
if let Some(threshold) = self.persistence_threshold {
cmd.args(["--persistence-threshold", &threshold.to_string()]);
}
}
cmd.env("RUST_LOG_STYLE", "never")
.stdout(std::process::Stdio::piped())

View File

@@ -114,32 +114,10 @@ pub(crate) struct Args {
#[arg(long)]
pub profile: bool,
/// Optional fixed delay between engine API calls (passed to reth-bench).
///
/// Can be combined with `--wait-for-persistence`: when both are set,
/// waits at least this duration, and also waits for persistence if needed.
/// Wait time between engine API calls (passed to reth-bench)
#[arg(long, value_name = "DURATION")]
pub wait_time: Option<String>,
/// Wait for blocks to be persisted before sending the next batch (passed to reth-bench).
///
/// When enabled, waits for every Nth block to be persisted using the
/// `reth_subscribePersistedBlock` subscription. This ensures the benchmark
/// doesn't outpace persistence.
///
/// Can be combined with `--wait-time`: when both are set, waits at least
/// wait-time, and also waits for persistence if the block hasn't been persisted yet.
#[arg(long)]
pub wait_for_persistence: bool,
/// Engine persistence threshold (passed to reth-bench).
///
/// The benchmark waits after every `(threshold + 1)` blocks. By default this
/// matches the engine's default persistence threshold (2), so waits occur
/// at blocks 3, 6, 9, etc.
#[arg(long, value_name = "PERSISTENCE_THRESHOLD")]
pub persistence_threshold: Option<u64>,
/// Number of blocks to run for cache warmup after clearing caches.
/// If not specified, defaults to the same as --blocks
#[arg(long, value_name = "N")]
@@ -150,11 +128,6 @@ pub(crate) struct Args {
#[arg(long)]
pub no_clear_cache: bool,
/// Skip waiting for the node to sync before starting benchmarks.
/// When enabled, assumes the node is already synced and skips the initial tip check.
#[arg(long)]
pub skip_wait_syncing: bool,
#[command(flatten)]
pub logs: LogArgs,
@@ -191,41 +164,12 @@ pub(crate) struct Args {
#[arg(trailing_var_arg = true, allow_hyphen_values = true)]
pub reth_args: Vec<String>,
/// Comma-separated list of extra features to enable during reth compilation (applied to both
/// builds)
#[arg(long, value_name = "FEATURES", default_value = "")]
/// Comma-separated list of features to enable during reth compilation
///
/// Example: `jemalloc,asm-keccak`
#[arg(long, value_name = "FEATURES", default_value = "jemalloc,asm-keccak")]
pub features: String,
/// Comma-separated list of features to enable only for baseline build (overrides --features)
///
/// Example: `--baseline-features jemalloc`
#[arg(long, value_name = "FEATURES")]
pub baseline_features: Option<String>,
/// Comma-separated list of features to enable only for feature build (overrides --features)
///
/// Example: `--feature-features jemalloc-prof`
#[arg(long, value_name = "FEATURES")]
pub feature_features: Option<String>,
/// RUSTFLAGS to use for both baseline and feature builds
///
/// Example: `--rustflags "-C target-cpu=native"`
#[arg(long, value_name = "FLAGS", default_value = "-C target-cpu=native")]
pub rustflags: String,
/// RUSTFLAGS to use only for baseline build (overrides --rustflags)
///
/// Example: `--baseline-rustflags "-C target-cpu=native -C lto"`
#[arg(long, value_name = "FLAGS")]
pub baseline_rustflags: Option<String>,
/// RUSTFLAGS to use only for feature build (overrides --rustflags)
///
/// Example: `--feature-rustflags "-C target-cpu=native -C lto"`
#[arg(long, value_name = "FLAGS")]
pub feature_rustflags: Option<String>,
/// Disable automatic --debug.startup-sync-state-idle flag for specific runs.
/// Can be "baseline", "feature", or "all".
/// By default, the flag is passed to warmup, baseline, and feature runs.
@@ -276,8 +220,10 @@ impl Args {
/// Get the default RPC URL for a given chain
const fn get_default_rpc_url(chain: &Chain) -> &'static str {
match chain.id() {
27082 => "https://rpc.hoodi.ethpandaops.io", // hoodi
_ => "https://ethereum.reth.rs/rpc", // mainnet and fallback
8453 => "https://base-mainnet.rpc.ithaca.xyz", // base
84532 => "https://base-sepolia.rpc.ithaca.xyz", // base-sepolia
27082 => "https://rpc.hoodi.ethpandaops.io", // hoodi
_ => "https://reth-ethereum.ithaca.xyz/rpc", // mainnet and fallback
}
}
@@ -382,6 +328,7 @@ pub(crate) async fn run_comparison(args: Args, _ctx: CliContext) -> Result<()> {
git_manager.repo_root().to_string(),
output_dir.clone(),
git_manager.clone(),
args.features.clone(),
)?;
// Initialize node manager
let mut node_manager = NodeManager::new(&args);
@@ -471,6 +418,7 @@ async fn run_compilation_phase(
git_manager: &GitManager,
compilation_manager: &CompilationManager,
args: &Args,
is_optimism: bool,
) -> Result<(String, String)> {
info!("=== Running compilation phase ===");
@@ -500,18 +448,6 @@ async fn run_compilation_phase(
let ref_type = ref_types[i];
let commit = &ref_commits[git_ref];
// Get per-build features and rustflags
let features = match ref_type {
"baseline" => args.baseline_features.as_ref().unwrap_or(&args.features),
"feature" => args.feature_features.as_ref().unwrap_or(&args.features),
_ => &args.features,
};
let rustflags = match ref_type {
"baseline" => args.baseline_rustflags.as_ref().unwrap_or(&args.rustflags),
"feature" => args.feature_rustflags.as_ref().unwrap_or(&args.rustflags),
_ => &args.rustflags,
};
info!(
"Compiling {} binary for reference: {} (commit: {})",
ref_type,
@@ -523,7 +459,7 @@ async fn run_compilation_phase(
git_manager.switch_ref(git_ref)?;
// Compile reth (with caching)
compilation_manager.compile_reth(commit, features, rustflags)?;
compilation_manager.compile_reth(commit, is_optimism)?;
info!("Completed compilation for {} reference", ref_type);
}
@@ -535,7 +471,6 @@ async fn run_compilation_phase(
Ok((baseline_commit, feature_commit))
}
#[allow(clippy::too_many_arguments)]
/// Run warmup phase to warm up caches before benchmarking
async fn run_warmup_phase(
git_manager: &GitManager,
@@ -543,16 +478,11 @@ async fn run_warmup_phase(
node_manager: &mut NodeManager,
benchmark_runner: &BenchmarkRunner,
args: &Args,
is_optimism: bool,
baseline_commit: &str,
starting_tip: u64,
) -> Result<()> {
info!("=== Running warmup phase ===");
// Unwind to starting block minus warmup blocks, so we end up back at starting_tip
let warmup_blocks = args.get_warmup_blocks();
let unwind_target = starting_tip.saturating_sub(warmup_blocks);
node_manager.unwind_to_block(unwind_target).await?;
// Use baseline for warmup
let warmup_ref = &args.baseline_ref;
@@ -560,7 +490,8 @@ async fn run_warmup_phase(
git_manager.switch_ref(warmup_ref)?;
// Get the cached binary path for baseline (should already be compiled)
let binary_path = compilation_manager.get_cached_binary_path_for_commit(baseline_commit);
let binary_path =
compilation_manager.get_cached_binary_path_for_commit(baseline_commit, is_optimism);
// Verify the cached binary exists
if !binary_path.exists() {
@@ -580,13 +511,12 @@ async fn run_warmup_phase(
node_manager.start_node(&binary_path, warmup_ref, "warmup", &additional_args).await?;
// Wait for node to be ready and get its current tip
let current_tip = if args.skip_wait_syncing {
node_manager.wait_for_rpc_and_get_tip(&mut node_process).await?
} else {
node_manager.wait_for_node_ready_and_get_tip(&mut node_process).await?
};
let current_tip = node_manager.wait_for_node_ready_and_get_tip().await?;
info!("Warmup node is ready at tip: {}", current_tip);
// Store the tip we'll unwind back to
let original_tip = current_tip;
// Clear filesystem caches before warmup run only (unless disabled)
if args.no_clear_cache {
info!("Skipping filesystem cache clearing (--no-clear-cache flag set)");
@@ -597,9 +527,12 @@ async fn run_warmup_phase(
// Run warmup to warm up caches
benchmark_runner.run_warmup(current_tip).await?;
// Stop node after warmup
// Stop node before unwinding (node must be stopped to release database lock)
node_manager.stop_node(&mut node_process).await?;
// Unwind back to starting block after warmup
node_manager.unwind_to_block(original_tip).await?;
info!("Warmup phase completed");
Ok(())
}
@@ -613,33 +546,13 @@ async fn run_benchmark_workflow(
comparison_generator: &mut ComparisonGenerator,
args: &Args,
) -> Result<()> {
// Detect if this is an Optimism chain once at the beginning
let rpc_url = args.get_rpc_url();
let is_optimism = compilation_manager.detect_optimism_chain(&rpc_url).await?;
// Run compilation phase for both binaries
let (baseline_commit, feature_commit) =
run_compilation_phase(git_manager, compilation_manager, args).await?;
// Switch to baseline reference and get the starting tip
git_manager.switch_ref(&args.baseline_ref)?;
let binary_path = compilation_manager.get_cached_binary_path_for_commit(&baseline_commit);
if !binary_path.exists() {
return Err(eyre!(
"Cached baseline binary not found at {:?}. Compilation phase should have created it.",
binary_path
));
}
// Start node briefly to get the current tip, then stop it
info!("=== Determining initial block height ===");
let additional_args = args.build_additional_args("baseline", args.baseline_args.as_ref());
let (mut node_process, _) = node_manager
.start_node(&binary_path, &args.baseline_ref, "baseline", &additional_args)
.await?;
let starting_tip = if args.skip_wait_syncing {
node_manager.wait_for_rpc_and_get_tip(&mut node_process).await?
} else {
node_manager.wait_for_node_ready_and_get_tip(&mut node_process).await?
};
info!("Node starting tip: {}", starting_tip);
node_manager.stop_node(&mut node_process).await?;
run_compilation_phase(git_manager, compilation_manager, args, is_optimism).await?;
// Run warmup phase before benchmarking (skip if warmup_blocks is 0)
if args.get_warmup_blocks() > 0 {
@@ -649,8 +562,8 @@ async fn run_benchmark_workflow(
node_manager,
benchmark_runner,
args,
is_optimism,
&baseline_commit,
starting_tip,
)
.await?;
} else {
@@ -666,15 +579,12 @@ async fn run_benchmark_workflow(
let commit = commits[i];
info!("=== Processing {} reference: {} ===", ref_type, git_ref);
// Unwind to starting block minus benchmark blocks, so we end up back at starting_tip
let unwind_target = starting_tip.saturating_sub(args.blocks);
node_manager.unwind_to_block(unwind_target).await?;
// Switch to target reference
git_manager.switch_ref(git_ref)?;
// Get the cached binary path for this git reference (should already be compiled)
let binary_path = compilation_manager.get_cached_binary_path_for_commit(commit);
let binary_path =
compilation_manager.get_cached_binary_path_for_commit(commit, is_optimism);
// Verify the cached binary exists
if !binary_path.exists() {
@@ -702,18 +612,17 @@ async fn run_benchmark_workflow(
node_manager.start_node(&binary_path, git_ref, ref_type, &additional_args).await?;
// Wait for node to be ready and get its current tip (wherever it is)
let current_tip = if args.skip_wait_syncing {
node_manager.wait_for_rpc_and_get_tip(&mut node_process).await?
} else {
node_manager.wait_for_node_ready_and_get_tip(&mut node_process).await?
};
let current_tip = node_manager.wait_for_node_ready_and_get_tip().await?;
info!("Node is ready at tip: {}", current_tip);
// Store the tip we'll unwind back to
let original_tip = current_tip;
// Calculate benchmark range
// Note: reth-bench has an off-by-one error where it consumes the first block
// of the range, so we add 1 to compensate and get exactly args.blocks blocks
let from_block = current_tip;
let to_block = current_tip + args.blocks;
let from_block = original_tip;
let to_block = original_tip + args.blocks;
// Run benchmark
let output_dir = comparison_generator.get_ref_output_dir(ref_type);
@@ -730,6 +639,9 @@ async fn run_benchmark_workflow(
// Stop node
node_manager.stop_node(&mut node_process).await?;
// Unwind back to original tip
node_manager.unwind_to_block(original_tip).await?;
// Store results for comparison
comparison_generator.add_ref_results(ref_type, &output_dir)?;

View File

@@ -39,8 +39,7 @@ pub(crate) struct BenchmarkResults {
#[derive(Debug, Clone, Deserialize, Serialize)]
pub(crate) struct CombinedLatencyRow {
pub block_number: u64,
#[serde(default)]
pub transaction_count: Option<u64>,
pub transaction_count: u64,
pub gas_used: u64,
pub new_payload_latency: u128,
}
@@ -49,8 +48,7 @@ pub(crate) struct CombinedLatencyRow {
#[derive(Debug, Clone, Deserialize, Serialize)]
pub(crate) struct TotalGasRow {
pub block_number: u64,
#[serde(default)]
pub transaction_count: Option<u64>,
pub transaction_count: u64,
pub gas_used: u64,
pub time: u128,
}
@@ -99,7 +97,6 @@ pub(crate) struct RefInfo {
/// Summary of the comparison between references.
///
/// Percent deltas are `(feature - baseline) / baseline * 100`:
/// - `new_payload_latency_mean_change_percent`: percent changes of the per-block means.
/// - `new_payload_latency_p50_change_percent` / p90 / p99: percent changes of the respective
/// per-block percentiles.
/// - `per_block_latency_change_mean_percent` / `per_block_latency_change_median_percent` are the
@@ -117,7 +114,6 @@ pub(crate) struct ComparisonSummary {
pub per_block_latency_change_median_percent: f64,
pub per_block_latency_change_std_dev_percent: f64,
pub new_payload_total_latency_change_percent: f64,
pub new_payload_latency_mean_change_percent: f64,
pub new_payload_latency_p50_change_percent: f64,
pub new_payload_latency_p90_change_percent: f64,
pub new_payload_latency_p99_change_percent: f64,
@@ -129,8 +125,7 @@ pub(crate) struct ComparisonSummary {
#[derive(Debug, Serialize)]
pub(crate) struct BlockComparison {
pub block_number: u64,
#[serde(skip_serializing_if = "Option::is_none")]
pub transaction_count: Option<u64>,
pub transaction_count: u64,
pub gas_used: u64,
pub baseline_new_payload_latency: u128,
pub feature_new_payload_latency: u128,
@@ -447,10 +442,6 @@ impl ComparisonGenerator {
per_block_latency_change_median_percent,
per_block_latency_change_std_dev_percent,
new_payload_total_latency_change_percent,
new_payload_latency_mean_change_percent: calc_percent_change(
baseline.mean_new_payload_latency_ms,
feature.mean_new_payload_latency_ms,
),
new_payload_latency_p50_change_percent: calc_percent_change(
baseline.median_new_payload_latency_ms,
feature.median_new_payload_latency_ms,
@@ -581,10 +572,6 @@ impl ComparisonGenerator {
" Total newPayload time change: {:+.2}%",
summary.new_payload_total_latency_change_percent
);
println!(
" NewPayload Latency mean: {:+.2}%",
summary.new_payload_latency_mean_change_percent
);
println!(
" NewPayload Latency p50: {:+.2}%",
summary.new_payload_latency_p50_change_percent

View File

@@ -1,6 +1,8 @@
//! Compilation operations for reth and reth-bench.
use crate::git::GitManager;
use alloy_primitives::address;
use alloy_provider::{Provider, ProviderBuilder};
use eyre::{eyre, Result, WrapErr};
use std::{fs, path::PathBuf, process::Command};
use tracing::{debug, error, info, warn};
@@ -11,6 +13,7 @@ pub(crate) struct CompilationManager {
repo_root: String,
output_dir: PathBuf,
git_manager: GitManager,
features: String,
}
impl CompilationManager {
@@ -19,18 +22,53 @@ impl CompilationManager {
repo_root: String,
output_dir: PathBuf,
git_manager: GitManager,
features: String,
) -> Result<Self> {
Ok(Self { repo_root, output_dir, git_manager })
Ok(Self { repo_root, output_dir, git_manager, features })
}
/// Detect if the RPC endpoint is an Optimism chain
pub(crate) async fn detect_optimism_chain(&self, rpc_url: &str) -> Result<bool> {
info!("Detecting chain type from RPC endpoint...");
// Create Alloy provider
let url = rpc_url.parse().map_err(|e| eyre!("Invalid RPC URL '{}': {}", rpc_url, e))?;
let provider = ProviderBuilder::new().connect_http(url);
// Check for Optimism predeploy at address 0x420000000000000000000000000000000000000F
let is_optimism = !provider
.get_code_at(address!("0x420000000000000000000000000000000000000F"))
.await?
.is_empty();
if is_optimism {
info!("Detected Optimism chain");
} else {
info!("Detected Ethereum chain");
}
Ok(is_optimism)
}
/// Get the path to the cached binary using explicit commit hash
pub(crate) fn get_cached_binary_path_for_commit(&self, commit: &str) -> PathBuf {
pub(crate) fn get_cached_binary_path_for_commit(
&self,
commit: &str,
is_optimism: bool,
) -> PathBuf {
let identifier = &commit[..8]; // Use first 8 chars of commit
self.output_dir.join("bin").join(format!("reth_{identifier}"))
let binary_name = if is_optimism {
format!("op-reth_{}", identifier)
} else {
format!("reth_{}", identifier)
};
self.output_dir.join("bin").join(binary_name)
}
/// Compile reth using cargo build and cache the binary
pub(crate) fn compile_reth(&self, commit: &str, features: &str, rustflags: &str) -> Result<()> {
pub(crate) fn compile_reth(&self, commit: &str, is_optimism: bool) -> Result<()> {
// Validate that current git commit matches the expected commit
let current_commit = self.git_manager.get_current_commit()?;
if current_commit != commit {
@@ -41,7 +79,7 @@ impl CompilationManager {
));
}
let cached_path = self.get_cached_binary_path_for_commit(commit);
let cached_path = self.get_cached_binary_path_for_commit(commit, is_optimism);
// Check if cached binary already exists (since path contains commit hash, it's valid)
if cached_path.exists() {
@@ -51,7 +89,7 @@ impl CompilationManager {
info!("No cached binary found, compiling (commit: {})...", &commit[..8]);
let binary_name = "reth";
let binary_name = if is_optimism { "op-reth" } else { "reth" };
info!(
"Compiling {} with profiling configuration (commit: {})...",
@@ -62,16 +100,25 @@ impl CompilationManager {
let mut cmd = Command::new("cargo");
cmd.arg("build").arg("--profile").arg("profiling");
cmd.arg("--features").arg(features);
info!("Using features: {features}");
// Add features
cmd.arg("--features").arg(&self.features);
info!("Using features: {}", self.features);
// Add bin-specific arguments for optimism
if is_optimism {
cmd.arg("--bin")
.arg("op-reth")
.arg("--manifest-path")
.arg("crates/optimism/bin/Cargo.toml");
}
cmd.current_dir(&self.repo_root);
// Set RUSTFLAGS
cmd.env("RUSTFLAGS", rustflags);
info!("Using RUSTFLAGS: {rustflags}");
// Set RUSTFLAGS for native CPU optimization
cmd.env("RUSTFLAGS", "-C target-cpu=native");
info!("Compiling {binary_name} with {cmd:?}");
// Debug log the command
debug!("Executing cargo command: {:?}", cmd);
let output = cmd.output().wrap_err("Failed to execute cargo build command")?;
@@ -180,7 +227,8 @@ impl CompilationManager {
let mut cmd = Command::new("cargo");
cmd.args(["install", "--locked", "samply"]);
info!("Installing samply with {cmd:?}");
// Debug log the command
debug!("Executing cargo command: {:?}", cmd);
let output = cmd.output().wrap_err("Failed to execute cargo install samply command")?;
@@ -255,7 +303,8 @@ impl CompilationManager {
let mut cmd = Command::new("make");
cmd.arg("install-reth-bench").current_dir(&self.repo_root);
info!("Compiling reth-bench with {cmd:?}");
// Debug log the command
debug!("Executing make command: {:?}", cmd);
let output = cmd.output().wrap_err("Failed to execute make install-reth-bench command")?;

View File

@@ -14,8 +14,6 @@
#[global_allocator]
static ALLOC: reth_cli_util::allocator::Allocator = reth_cli_util::allocator::new_allocator();
use alloy_primitives as _;
mod benchmark;
mod cli;
mod comparison;

View File

@@ -2,9 +2,7 @@
use crate::cli::Args;
use alloy_provider::{Provider, ProviderBuilder};
use alloy_rpc_client::RpcClient;
use alloy_rpc_types_eth::SyncStatus;
use alloy_transport_ws::WsConnect;
use eyre::{eyre, OptionExt, Result, WrapErr};
#[cfg(unix)]
use nix::sys::signal::{killpg, Signal};
@@ -20,9 +18,6 @@ use tokio::{
};
use tracing::{debug, info, warn};
/// Default websocket RPC port used by reth
const DEFAULT_WS_RPC_PORT: u16 = 8546;
/// Manages reth node lifecycle and operations
pub(crate) struct NodeManager {
datadir: Option<String>,
@@ -157,13 +152,9 @@ impl NodeManager {
metrics_arg,
"--http".to_string(),
"--http.api".to_string(),
"eth,reth".to_string(),
"--ws".to_string(),
"--ws.api".to_string(),
"eth,reth".to_string(),
"eth".to_string(),
"--disable-discovery".to_string(),
"--trusted-only".to_string(),
"--disable-tx-gossip".to_string(),
]);
// Add tracing arguments if OTLP endpoint is configured
@@ -220,11 +211,6 @@ impl NodeManager {
cmd.arg("--");
cmd.args(reth_args);
// Enable tracing-samply
if supports_samply_flags(&reth_args[0]) {
cmd.arg("--log.samply");
}
// Set environment variable to disable log styling
cmd.env("RUST_LOG_STYLE", "never");
@@ -368,13 +354,8 @@ impl NodeManager {
Ok((child, reth_command))
}
/// Wait for the node to be ready and return its current tip.
///
/// Fails early if the node process exits before becoming ready.
pub(crate) async fn wait_for_node_ready_and_get_tip(
&self,
child: &mut tokio::process::Child,
) -> Result<u64> {
/// Wait for the node to be ready and return its current tip
pub(crate) async fn wait_for_node_ready_and_get_tip(&self) -> Result<u64> {
info!("Waiting for node to be ready and synced...");
let max_wait = Duration::from_secs(120); // 2 minutes to allow for sync
@@ -385,23 +366,8 @@ impl NodeManager {
let url = rpc_url.parse().map_err(|e| eyre!("Invalid RPC URL '{}': {}", rpc_url, e))?;
let provider = ProviderBuilder::new().connect_http(url);
let start_time = tokio::time::Instant::now();
let mut iteration = 0;
timeout(max_wait, async {
loop {
iteration += 1;
debug!(
"Readiness check iteration {} (elapsed: {:?})",
iteration,
start_time.elapsed()
);
// Check if the node process has exited.
if let Some(status) = child.try_wait()? {
return Err(eyre!("Node process exited unexpectedly with {status}"));
}
// First check if RPC is up and node is not syncing
match provider.syncing().await {
Ok(sync_result) => {
@@ -410,48 +376,24 @@ impl NodeManager {
debug!("Node is still syncing {sync_info:?}, waiting...");
}
_ => {
debug!("HTTP RPC is up and node is not syncing, checking block number...");
// Node is not syncing, now get the tip
match provider.get_block_number().await {
Ok(tip) => {
debug!("HTTP RPC ready at block: {}, checking WebSocket...", tip);
// Verify WebSocket RPC is ready (public endpoint, no JWT required)
let ws_url = format!("ws://localhost:{}", DEFAULT_WS_RPC_PORT);
debug!("Attempting WebSocket connection to {} (public endpoint)", ws_url);
let ws_connect = WsConnect::new(&ws_url);
match RpcClient::connect_pubsub(ws_connect).await
{
Ok(_) => {
info!(
"Node is ready (HTTP and WebSocket) at block: {} (took {:?}, {} iterations)",
tip, start_time.elapsed(), iteration
);
return Ok(tip);
}
Err(e) => {
debug!(
"HTTP RPC ready but WebSocket not ready yet (iteration {}): {:?}",
iteration, e
);
debug!("WebSocket error details: {}", e);
}
}
info!("Node is ready and not syncing at block: {}", tip);
return Ok(tip);
}
Err(e) => {
debug!("Failed to get block number (iteration {}): {:?}", iteration, e);
debug!("Failed to get block number: {}", e);
}
}
}
}
}
Err(e) => {
debug!("Node RPC not ready yet or failed to check sync status (iteration {}): {:?}", iteration, e);
debug!("Node RPC not ready yet or failed to check sync status: {}", e);
}
}
debug!("Sleeping for {:?} before next check", check_interval);
sleep(check_interval).await;
}
})
@@ -459,79 +401,9 @@ impl NodeManager {
.wrap_err("Timed out waiting for node to be ready and synced")?
}
/// Wait for the node RPC to be ready and return its current tip, without waiting for sync.
///
/// This is faster than `wait_for_node_ready_and_get_tip` but may return a tip while
/// the node is still syncing.
pub(crate) async fn wait_for_rpc_and_get_tip(
&self,
child: &mut tokio::process::Child,
) -> Result<u64> {
info!("Waiting for node RPC to be ready (skipping sync wait)...");
let max_wait = Duration::from_secs(60);
let check_interval = Duration::from_secs(2);
let rpc_url = "http://localhost:8545";
let url = rpc_url.parse().map_err(|e| eyre!("Invalid RPC URL '{}': {}", rpc_url, e))?;
let provider = ProviderBuilder::new().connect_http(url);
let start_time = tokio::time::Instant::now();
let mut iteration = 0;
timeout(max_wait, async {
loop {
iteration += 1;
debug!(
"RPC readiness check iteration {} (elapsed: {:?})",
iteration,
start_time.elapsed()
);
if let Some(status) = child.try_wait()? {
return Err(eyre!("Node process exited unexpectedly with {status}"));
}
match provider.get_block_number().await {
Ok(tip) => {
debug!("HTTP RPC ready at block: {}, checking WebSocket...", tip);
let ws_url = format!("ws://localhost:{}", DEFAULT_WS_RPC_PORT);
let ws_connect = WsConnect::new(&ws_url);
match RpcClient::connect_pubsub(ws_connect).await {
Ok(_) => {
info!(
"Node RPC is ready at block: {} (took {:?}, {} iterations)",
tip,
start_time.elapsed(),
iteration
);
return Ok(tip);
}
Err(e) => {
debug!(
"HTTP RPC ready but WebSocket not ready yet (iteration {}): {:?}",
iteration, e
);
}
}
}
Err(e) => {
debug!("RPC not ready yet (iteration {}): {:?}", iteration, e);
}
}
sleep(check_interval).await;
}
})
.await
.wrap_err("Timed out waiting for node RPC to be ready")?
}
/// Stop the reth node gracefully
pub(crate) async fn stop_node(&self, child: &mut tokio::process::Child) -> Result<()> {
let pid = child.id().ok_or_eyre("Child process ID should be available")?;
let pid = child.id().expect("Child process ID should be available");
// Check if the process has already exited
match child.try_wait() {
@@ -680,16 +552,3 @@ impl NodeManager {
Ok(())
}
}
fn supports_samply_flags(bin: &str) -> bool {
let mut cmd = std::process::Command::new(bin);
// NOTE: The flag to check must come before --help.
// We pass --help as a shortcut to not execute any command.
cmd.args(["--log.samply", "--help"]);
debug!(?cmd, "Checking samply flags support");
let Ok(output) = cmd.output() else {
return false;
};
debug!(?output, "Samply flags support check");
output.status.success()
}

Some files were not shown because too many files have changed in this diff Show More