Commit Graph

11927 Commits

Author SHA1 Message Date
Yong Kang
1cc535a7d4 update values to overlay to be tgt 2025-10-30 23:50:52 +08:00
Yong Kang
861864732d added panels:
- Panels Added:

  1. Overlay State Provider - Total Duration (Panel 303)
    - Metric: reth_storage_overlay_state_provider_total_database_provider_ro_duration
    - Shows complete database_provider_ro() execution time
    - Thresholds: Green (0s), Yellow (0.1s), Red (0.5s)
  2. Overlay State Provider - Base Provider Creation (Panel 304)
    - Metric: reth_storage_overlay_state_provider_base_provider_creation_duration
    - Time to create base DB provider
    - Thresholds: Green (0s), Yellow (0.01s), Red (0.05s)
  3. Overlay State Provider - Block Hash Lookup (Panel 305)
    - Metric: reth_storage_overlay_state_provider_block_hash_lookup_duration
    - Time to convert block hash to block number
    - Thresholds: Green (0s), Yellow (0.01s), Red (0.05s)
  4. Overlay State Provider - Trie Reverts Fetch (Panel 306)
    - Metric: reth_storage_overlay_state_provider_trie_reverts_duration
    - Time to fetch trie reverts from database
    - Thresholds: Green (0s), Yellow (0.1s), Red (0.5s)
  5. Overlay State Provider - State Reverts Fetch (Panel 307)
    - Metric: reth_storage_overlay_state_provider_state_reverts_duration
    - Time to fetch state reverts from database
    - Thresholds: Green (0s), Yellow (0.1s), Red (0.5s)
2025-10-30 23:08:33 +08:00
Yong Kang
fa65504d21 add dashboard too why not right 2025-10-30 23:04:46 +08:00
Yong Kang
56a423ebe9 Implement metrics for overlay state provider operations 2025-10-30 23:02:28 +08:00
Yong Kang
d96ba9ec21 Refactor proof worker initialization for lazy DB provider creation
- Changed the proof worker to lazily initialize the database provider and ProofTaskTx on the first job message instead of during worker spawn.
- This improves the worker's availability time by allowing it to mark itself as available immediately, reducing the startup overhead.
- Updated metrics to capture the duration of DB provider creation and proof transaction initialization only when they are actually created, enhancing performance tracking.
2025-10-30 21:56:50 +08:00
Yong Kang
d80d52c785 Instrumentation now captures how long worker startup spends waiting on DB state and logs a
bit of context around pool creation, so you can see whether those steps are dragging.

  - Added per-worker timers around database_provider_ro() and ProofTaskTx::new() for both
    storage and account pools, feeding new histograms in ProofTaskTrieMetrics (crates/trie/
    parallel/src/proof_task.rs:728, crates/trie/parallel/src/proof_task_metrics.rs:18).
  - Introduced debug logs when a payload spawns and finishes building the proof worker pools
    to make block-level spikes line up with worker lifecycle events (crates/engine/tree/src/
    tree/payload_processor/mod.rs:231).
2025-10-30 20:45:28 +08:00
Yong Kang
7185fab5a1 feat(metrics): add worker spawn-to-available duration histograms
Add metrics to track time from worker spawn start until each proof
worker marks itself as available (after initialization, before
processing first job).

Changes:
- Add storage_worker_spawn_to_available_duration histogram
- Add account_worker_spawn_to_available_duration histogram
- Capture timestamp before spawning workers
- Pass timestamp to worker constructors
- Record elapsed time when workers mark themselves available

These metrics help identify worker initialization bottlenecks and
understand parallel proof computation startup performance.
2025-10-30 18:45:11 +08:00
nobody
f9cfde1f62 Merge remote-tracking branch 'origin/mediocregopher/overlay-state-provider-perf-fix' into mediocregopher/main-syncing-fix 2025-10-29 12:56:45 +00:00
nobody
def9cddae6 Reapply "chore(engine): Remove ConsistentDbView (#19188)"
This reverts commit 4fecad0f5a.
2025-10-29 12:56:19 +00:00
Brian Picciano
2d2ac682ca fix: Don't always clone in-memory overlays in OverlayStateProviderFactory
If there are no reverts then there's no need to call `extend_ref`, which
clones the full overlay value; we can just use the overlays directly.
2025-10-29 13:43:58 +01:00
Matthias Seitz
caaedfadcb chore: bump 1.8.3 (#19379) 2025-10-29 13:07:30 +01:00
joshieDo
644be05659 feat: add pruning of transactions from static-files (#19241) 2025-10-29 11:50:51 +00:00
nobody
4fecad0f5a Revert "chore(engine): Remove ConsistentDbView (#19188)"
This reverts commit ffeaa4772d.
2025-10-29 11:32:03 +00:00
Brian Picciano
527c24df6d fix(trie): use block hash in OverlayStateProviderFactory (#19353) 2025-10-29 10:34:51 +00:00
Đạt Nguyễn
17a984929b feat: impl a function to create new instance of TransactionEvents (#19375)
Co-authored-by: Neo Krypt <neo@canxium.org>
Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>
2025-10-29 10:00:37 +00:00
Karl Yu
3827e5cb1d perf: wrap tx with Arc to avoid deep cloning (#19350) 2025-10-29 08:30:29 +00:00
YK
10d9a7e3c6 refactor(trie): restructure proof task workers into structs (#19344) 2025-10-29 08:09:39 +00:00
theo
77ef028aca fix(op-reth/consensus): fixes header validation for jovian. decouple excess blob gas and blob gas used (#19338) 2025-10-29 07:39:29 +00:00
Matthias Seitz
ff46daddb6 feat: insert at timestamp (#19365) 2025-10-28 20:29:26 +00:00
Mablr
adb4f48471 feat(reth-optimism-node): Add OP E2E mineblock test with isthmus activated at genesis (#19305) 2025-10-28 20:13:44 +00:00
Matthias Seitz
6651ae7852 chore: add ChainHardforks::extend (#19332) 2025-10-28 20:36:23 +01:00
joshieDo
ac4f80ded3 chore: dont write receipts to both storages on archive node (#19361) 2025-10-28 17:22:20 +00:00
Matthias Seitz
3ce6e87ab9 chore: update docs for expected test failure (#19343) 2025-10-28 16:07:39 +00:00
nobody
d178a114ea Disable trie overlay changeset range validation 2025-10-28 15:52:42 +00:00
joshieDo
020eb6ad7e fix(pipeline): ensure we dont pass an outdated target to header stage (#19351) 2025-10-28 15:02:19 +00:00
YK
5091482dec refactor(trie): reorder proof_task.rs for better code organization (#19342) 2025-10-28 11:14:08 +00:00
Avory
0f3e0eee63 refactor: make DatabaseProof trait stateful (#18753) 2025-10-28 10:14:07 +00:00
nobody
ab12c15ef5 Merge remote-tracking branch 'origin/main' into mediocregopher/main-syncing-fix 2025-10-28 10:03:26 +00:00
Francis Li
5e2ed163f3 fix(engine): Eliminates spurious warning logs in prewarm task (#19133)
Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>
2025-10-28 09:27:33 +00:00
dependabot[bot]
0da38b9732 chore(deps): bump actions/download-artifact from 5 to 6 (#19336)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-10-28 09:23:53 +00:00
dependabot[bot]
e547c027f3 chore(deps): bump actions/upload-artifact from 4 to 5 (#19335)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-10-28 09:22:33 +00:00
YK
7e6f676d16 feat(metrics): improve multiproof worker metrics (#19337) 2025-10-28 09:00:22 +00:00
YK
e2b5c7367c chore: update Grafana dashboard with split pending multiproof metrics (#19339) 2025-10-28 06:44:19 +00:00
josé v
50e88c29be chore: replace CacheDB with State<DB> in RPC crate (#19330)
Co-authored-by: Arsenii Kulikov <klkvrr@gmail.com>
2025-10-27 22:00:58 +00:00
Jennifer
1581aaa615 fix: update section name in expected failures, add more concise comments (#19328) 2025-10-27 21:46:29 +00:00
Brian Picciano
ffeaa4772d chore(engine): Remove ConsistentDbView (#19188)
Co-authored-by: Alexey Shekhirin <5773434+shekhirin@users.noreply.github.com>
2025-10-27 19:09:21 +00:00
Mablr
a264ccbbc2 feat(metrics): add push gateway support for Prometheus metrics (#19243) 2025-10-27 17:11:23 +00:00
Dan Cline
b1dfbc7e88 chore: remove trie capacity metrics (#19327) 2025-10-27 17:07:37 +00:00
Gengar
0569e884c4 docs: improve documentation for mock database and transactions (#19302) 2025-10-27 15:59:48 +00:00
theo
f9c89a9bc9 feat(jovian/block-validation): fix block validation for jovian (#19304) 2025-10-27 15:59:04 +00:00
Brian Picciano
7e59141c4b fix(trie): Rewrite InMemoryTrieOverlay (with proptests!) (#19277) 2025-10-27 15:18:48 +00:00
Alexey Shekhirin
080cf72464 chore(trie): reduce sparse trie tracing (#19321) 2025-10-27 15:16:56 +00:00
phrwlk
a6fe713a6c chore: remove dead OpL1BlockInfo.number field and writes (#19325) 2025-10-27 14:42:55 +00:00
YK
eed0d9686c refactor(trie): Unify proof return types (#19311) 2025-10-27 13:58:28 +00:00
Brian Picciano
fa1f86cb91 fix(prune): Add unused variants back to PruneSegment enum (#19318) 2025-10-27 13:12:22 +00:00
nobody
d720e5b196 Don't record trie capacity metrics 2025-10-27 12:35:37 +00:00
radik878
f088ec09cb docs(eth-wire): update docs to reflect eth-wire-types, alloy_rlp, version-aware decoding, and RLPx multiplexing (#19319) 2025-10-27 11:58:55 +00:00
Matthias Seitz
106ffefc0f chore: use hex bytes type (#19317) 2025-10-27 12:57:44 +01:00
Gengar
bb73d794fd docs: populate modify-node section with node-custom-rpc implementation guide (#18672) 2025-10-27 12:57:27 +01:00
nobody
a3a95efb89 Merge remote-tracking branch 'origin/mediocregopher/prune-segment-enum-fix' into mediocregopher/main-syncing-fix 2025-10-27 11:42:09 +00:00