Roman Krasiuk
ec88f1deef
feat(sync): standardize stage control flow checks ( #681 )
...
* feat(sync): standardize stage control flow checks
* fix input parameter for test_eth_chain
* clean up & put behind macro
* Update crates/stages/src/stage.rs
Co-authored-by: Bjerg <onbjerg@users.noreply.github.com >
* address comments
Co-authored-by: Bjerg <onbjerg@users.noreply.github.com >
2023-01-04 11:03:36 -08:00
Georgios Konstantopoulos
9d6be78f4b
chore: Box<dyn Error> -> StageError ( #721 )
...
* chore(stages): replace Box<dyn Error> with StageError
* chore: fix tests
* chore: fmt
2023-01-04 20:45:35 +02:00
Roman Krasiuk
c60495df7e
feat(sync): update sync state within pipeline ( #697 )
...
* feat(sync): update sync state within pipeline
* address pr comments
* fix comment
2023-01-04 11:25:40 +02:00
joshieDo
6407b5087e
chore: use B160, B256 and U256 coming from revm and ruint ( #676 )
...
* use B256, B160 and U256 from revm
* fix U256 from_str
* use U256::ZERO
* use temporary commit for revm and interpreter
* more U256::ZERO
* more changes for revm/ruint types
* clippy
* change revm and revm-interpreter repo
* remove H160 wrap
* minor cleanup
* remove unused
* fix MIN_PROTOCOL_BASE_FEE
2023-01-04 11:11:48 +02:00
Matthias Seitz
13441ff299
style: rm redundant Pipeline::new ( #682 )
2023-01-02 14:58:31 +01:00
Roman Krasiuk
cb6ddfcc43
feat(sync): total difficulty stage ( #665 )
...
* feat(sync): total difficulty stage
* linter
* rm commented log
* patch current td in headers tests
2023-01-01 19:03:25 +02:00
Sanket Shanbhag
24d5612027
rename StateProvider structs ( #663 )
2022-12-30 13:58:44 +02:00
Roman Krasiuk
a7d8059357
test(engine): consensus engine tests ( #648 )
2022-12-30 13:17:27 +02:00
LambdaClass
6efd50b25f
bug(execution stage): Fix ommers error ( #661 )
...
* Start loosing ommers checks
* Replace ommers database integrity error in execution stage for unwrap_or_else so that node does not stop when block has no ommers
* Replace ommers database integrity error in execution stage for unwrap_or_else so that node does not stop when block has no ommers
2022-12-29 22:31:08 +01:00
Sanket Shanbhag
28669d4aa8
fix(sync): return control from HeaderStage back to pipeline ( #609 )
...
* Headerstage now gives control back to pipeline after commit_threshold blocks have been synced
* Updated tests
* Only take required number of headers from the stream
* Simplified stage complete check
2022-12-29 00:57:49 -08:00
Andrew Kirillov
d4d8a8c882
Moves code walkthrough book chapters to docs ( #629 )
...
* replaced template blocks with code blocks in stages chapter
* replaced template blocks with code blocks in network chapter
* moved book sections to docs
* fix indentation in recover_signer codeblock
* remove unnecessary TODO comment in network.md
2022-12-28 10:24:39 +01:00
Roman Krasiuk
5bb14ecb7c
fix(metrics): metrics derive separator ( #625 )
...
* fix(metrics): metrics derive separator
* fix existing metrics
* rm log
* static regex
2022-12-27 03:16:41 -08:00
Roman Krasiuk
663efa8d2a
feat(metrics): Metrics derive macro ( #592 )
...
* feat(metrics): `Metrics` derive macro
* rename metrics to metrics-derive
* use fully qualified fmt path
* metric vec with capacity
* favor metrics over simple scope attr, simplify code
* pull metric description from doc comment
* rm debug log
* add more docs and sample usage
* link to metrics macros
* add tests for metrics derive macro
* clippy
* simplify path comparison
* cleanup
* remove unused dep
* rename stage metrics
2022-12-26 14:27:24 -08:00
rakita
d2b4081849
feat: Enable ExecutionStage ( #622 )
2022-12-27 00:09:04 +02:00
rakita
f21a4020b1
feat(execution): Add ommers reward ( #611 )
...
* feat(execution): Add ommers reward
* Refactor and add comments
2022-12-26 15:44:51 +01:00
rakita
19a618e3a4
fix(db): Dont compress DupSort SubKey ( #594 )
...
* bug(db): Dont compress DupSort SubKey
* unwrap or default config, Added notes
* fmt empty lines
2022-12-25 01:13:03 +01:00
rakita
284391c181
test(execution): chain test state check, execution bug fixes/upgrades ( #472 )
...
* temp: header stage backoff stand-in
* feat(execution): Check chain post state, fix StateProviderLatest and evm return
* Disable receipt merkle tree check
* update and merge
* Fix storage double values in dup table
* fmt
* Update bin/reth/src/test_eth_chain/runner.rs
Co-authored-by: Georgios Konstantopoulos <me@gakonst.com >
* Enable receipt root check after byzantium
* Receipt inner rlp without header for proof root
* some cleanup nits
* nit
Co-authored-by: Oliver Nordbjerg <hi@notbjerg.me >
Co-authored-by: Georgios Konstantopoulos <me@gakonst.com >
2022-12-23 18:42:19 +01:00
Bjerg
f8d1521c50
fix: set stage progress metric to absolute value ( #593 )
2022-12-23 18:14:53 +01:00
Bjerg
562e895f21
ci: unused dependencies sanity check ( #590 )
...
* ci: add unused deps sanity check
* ci: fix unused deps issue template title
* ci: rename udeps job
* chore: formatting
* chore: make `cargo-udeps` ignore `aquamarine`
* deps: resolve unused deps
* ci: check every feature and target in udeps job
2022-12-23 13:19:50 +01:00
Sanket Shanbhag
c9958a4d13
Remove unwind priority. ( #577 )
...
* Remove unwind_priority from QueuedStage
* Remove unnecessary test for unwind_priority
* Update pipeline.rs based on code review
2022-12-22 07:15:07 -08:00
Mariano A. Nicolini
b12939db47
feat: add stagedsync headers metrics ( #498 )
...
* add dockerfile for reth
* Add docker compose for prometheus
* Add some metrics
* Add p2p connection errors metric
* Add dependency caching for dockerfile
This reduces image build times by ~50% after the first one.
Uses cargo-chef inside the image.
More info in: https://morioh.com/p/987a2bda4526
* Add --metrics flag to docker-compose config file
* add Cargo.lock
* Move docker-compose.yml to docker directory
* Apply formatting
* Remove docker folder
* Remove .dockerignore file
* Add 'reth' prefix to metric names
* Add headers errors and request time metrics
* Modularize metrics exporter functionality and describe metrics
* Format files
* Add metrics documentation in metrics.md
* Fix metrics doc title
* Commit changes after rebase
* Solve conflict
* Modularize metrics describers
* Add stages_metrics_describer
* Rearrange header error metrics
* Add update_headers_metrics function
* Add one-line docs to describers
* Remove commented line
* Refactor metrics describer
* Update metrics doc
* Fix import
* Add header metrics struct
* add new metrics format in the headers execute method
* Add default implementation for HeaderMetrics
* Fix typo
* Fix another typo
* Fix more typos
* Move new HeaderMetrics meathod to default
* Solve conflicts
* Fix test
Co-authored-by: Tomás <tomas.gruner@lambdaclass.com >
2022-12-22 15:45:57 +01:00
Genysys
0b1b5368e6
chore: Rename BlockLocked to SealedBlock ( #573 )
...
* chore: Rename BlockLocked to SealedBlock
* chore: fmt
Co-authored-by: Genysys <112424909+samtvlabs@users.noreply.github.com >
Co-authored-by: Georgios Konstantopoulos <me@gakonst.com >
2022-12-22 14:45:36 +02:00
Roman Krasiuk
276be27310
chore(sync): rename senders stage ( #554 )
2022-12-21 05:34:18 -08:00
Roman Krasiuk
0995ac2777
meta(sync): tracing ( #455 )
...
* meta(sync): tracing
* Apply suggestions from code review
Co-authored-by: Bjerg <onbjerg@users.noreply.github.com >
* apply comments
* upd discv5 dep
Co-authored-by: Bjerg <onbjerg@users.noreply.github.com >
2022-12-20 03:00:59 -08:00
Bjerg
82cd84eca7
test: fix emits_bodies_in_order test ( #534 )
...
* feat(test-utils): make body tx count configurable
* test: fix `emits_bodies_in_order` test
2022-12-20 01:29:04 +01:00
Bjerg
f65562e2e4
refactor(sync): remove require_tip ( #528 )
...
`require_tip` could only be determined by the headers stage,
and it signalled that we have all of the headers to sync all
the way to the chain tip. Some stages may wait to execute
until the tip is reached, e.g. the stage that checks the
stage root, but there are a few problems:
- On initial sync, `reached_tip` would be `true`, but by
the time we reach the hashing stage, this would actually
no longer be the case: the other stages have spent
enough time for us to be "out of sync". This means
that the optimization here is lost, and the additional
logic is added for nothing.
- When we are not doing our initial sync, `reached_tip` would
always be `true` for each subsequent block we sync.
The same logic applies as above, i.e. the extra logic
is there for nothing.
In other words, `reached_tip` would *always* be `true` once
we leave the header stage, making the extra logic entirely
redundant.
2022-12-19 22:04:42 +01:00
Bjerg
13bb41517d
feat: config ( #496 )
...
* feat: default reth dirs
* docs: clarify txpool config docs
* feat: `serde` feature for `reth-consensus`
* feat: add missing hardforks
* feat: full chain specification
* feat: `DbPath` type
* feat: custom chain specs
* feat: `--debug.tip` option
* feat: basic stage config
* fix: resize db geometry
* refactor: don't commit in headers stage
* chore: remove headers dl timeout
* feat: loading of config
* refactor(config): `stage` -> `stages`
2022-12-19 21:27:26 +01:00
Roman Krasiuk
1dca4b58df
test(sync): sender recovery single tx ( #516 )
2022-12-19 16:08:07 +01:00
0xYami
0a4bf67c32
chore: complete StageDB -> Transaction renaming ( #497 )
2022-12-16 23:16:31 +01:00
Roman Krasiuk
daaf039fbf
feat(sync): state transition indexes ( #449 )
...
* introduce state transitions and revert/modify block bodies table
* init refactor
* revamp transaction iteration based on bodies and add state transition mappings
* change expected return on empty db execution
* interim commit
* fix body downloader & stage
* refactor(bodies/dl): make fetch bodies fn more clear
* chore: disable unused vars/fns temporarily until exec is back
* chore: fmt
* test: fix tests
* use transitions in execution stage
* clarify empty unwind test
* remove last_tx_index fn
* rename fn and var names
* fix full block response comment
* rename fetcher`s get_block_body to get_block_bodies
* Update crates/stages/src/db.rs
Co-authored-by: Georgios Konstantopoulos <me@gakonst.com >
* fmt
* fix index overlap check error
* uncomment eth chain command
* fix doc comment
* typos
* cleanup
* any_last_tx_index -> last_tx_index
Co-authored-by: Georgios Konstantopoulos <me@gakonst.com >
Co-authored-by: Oliver Nordbjerg <hi@notbjerg.me >
2022-12-16 08:37:49 -08:00
Bjerg
8e4a35ae31
chore: clean up traces ( #481 )
...
* chore: clean up traces
* fix: typo in error
2022-12-16 11:24:46 +02:00
Roman Krasiuk
e9ec6c2847
fix(sync): remove redundant error ( #476 )
2022-12-16 10:08:49 +02:00
Roman Krasiuk
f89fb4b453
fix(sync): database integrity header error message ( #475 )
2022-12-15 23:00:56 +01:00
Roman Krasiuk
2b0f5316f9
fix(sync): headers stage progress ( #467 )
...
* fix(sync): headers stage progress
* add tests to head/tip and split suite
* fix(sync): headers stage progress
* add tests to head/tip and split suite
* fix local tip and address comments
* rename error
2022-12-15 10:41:32 -08:00
Georgios Konstantopoulos
9cd5824f35
chore: rename StageDB to Transaction ( #470 )
...
* chore: rename StageDB to Transaction
* chore: cargo fmt
* chore: add note about potentially removing this in the future
2022-12-15 18:14:14 +02:00
Roman Krasiuk
9208f2fd9b
test(execution): execution test runner ( #426 )
...
* wip run chain test
* dump pre test state
* bug: fix casting to revm spec
* wip move execution to std thread
* wip scoped thread execution
* fmt clippy
* Update crates/stages/src/stages/execution.rs
* Update bin/reth/Cargo.toml
Co-authored-by: Georgios Konstantopoulos <me@gakonst.com >
* Update bin/reth/src/test_eth_chain/models.rs
Co-authored-by: Georgios Konstantopoulos <me@gakonst.com >
* Correct models for chain tests
Co-authored-by: rakita <dragan0rakita@gmail.com >
Co-authored-by: rakita <rakita@users.noreply.github.com >
Co-authored-by: Georgios Konstantopoulos <me@gakonst.com >
2022-12-15 13:39:55 +01:00
Oliver Nordbjerg
48fadbdf2e
fix: missing import
2022-12-15 13:53:20 +02:00
Bjerg
f8b4251886
feat: don't stop the pipeline on internal stage errs ( #453 )
...
* feat: don't stop the pipeline on internal stage errs
* test: add tests for pipeline err handling
* chore: few notes
* refactor: stage error fatal/recoverable variants
* refactor: use recoverable errors in headers stage
* test: adjust tests
* chore: nits
Co-authored-by: Georgios Konstantopoulos <me@gakonst.com >
Co-authored-by: Georgios Konstantopoulos <me@gakonst.com >
2022-12-15 12:48:41 +01:00
Roman Krasiuk
22dc50e5f6
feat(sync): download peer penalization ( #427 )
...
* feat(sync): download peer penalization
* peer penalization
* add tracing on penalization
* add trace on request
* rename consensus back
* clippy
* fix tests
* nit: download result
* nit: fix comment
* rename penalize() to report_bad_message() and move DownloadError
Co-authored-by: Georgios Konstantopoulos <me@gakonst.com >
2022-12-15 10:42:18 +02:00
mempirate
c23c65fc3b
feat(net): draft for sending status updates through NetworkHandle ( #436 )
...
* feat(net): draft for sending status updates through `NetworkHandle`
* feat(net): draft for sending status updates through `NetworkHandle`
* fix(net): remove unused import
* feat(net): implement getters for status update sender half
* docs(net): document methods
* chore: cargo fmt
* feat(net): move status updating logic to NetworkManager and NetworkHandle
* feat(net): move status updating logic to NetworkManager and NetworkHandle
* fix(net): fix headers stage testing
* fix: derive default
* fix: remove StatusUpdate struct
Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de >
2022-12-15 08:59:28 +01:00
joshieDo
7b6bf0820e
fix(cli): remove usage of StageDB on DbTool ( #448 )
...
* use view and update instead of StageDB
* change DbTool docs
* clippy
2022-12-15 08:33:49 +01:00
Bjerg
ae2fc494c8
test: fix header dl error test ( #441 )
2022-12-14 17:32:26 +02:00
Georgios Konstantopoulos
d5d876b286
fix(headers-stage): do not exit stage on downloader error ( #439 )
...
* fix: return ctrl to pipeline on header dl error
* chore: adjust log levels
* ecies: improve tag check error granularity
* chore: instrument read_ack/read_auth
Co-authored-by: Oliver Nordbjerg <hi@notbjerg.me >
2022-12-14 16:25:54 +02:00
Bjerg
601bdc5022
feat: wire components together in reth node ( #416 )
...
* feat: naively wire up components
* chore: clippy lints
Co-authored-by: Georgios Konstantopoulos <me@gakonst.com >
2022-12-14 12:51:56 +02:00
0xKitsune
df9d14127d
Reth Book - Stages framework/draft ( #360 )
...
* Added framework and start to draft for stages, stopping here to get feedback on approach before pushing forward
* Update README.md
Fixing some wording / grammar.
* Fixing grammar/wording.
* Added note about non-exhaustive stages list, fixed grammar, fixed State exectuion function name, updated language to reflect that the stream yields a SealedHeader and added language to describe that the initial header validation is only a basic validation.
* updated stages chapter, added bodies, senders, execution, next chapter prelude
* typo
* Added line numbers to code snippets for the stages chapter of the reth book
* address reverse header download + other nits
* add note about book hosting
* tweaked wording, formatting
* Address typo "staring"
* Address typo "HeadderDownloader"
* consolidated book.toml
* updating snippets in stages chapter to ignore errors
* template & removed empty fields from book.toml
* addressed build issues, added templating for source code
* only deploy on push to main
* using single quotes in github action if expression
Co-authored-by: Andrew Kirillov <andrew@paradigm.xyz >
Co-authored-by: Andrew Kirillov <20803092+akirillo@users.noreply.github.com >
2022-12-14 10:30:45 +02:00
Roman Krasiuk
2ace52a8dd
feat(sync): add tx hash number mapping ( #417 )
2022-12-13 19:53:16 +02:00
rakita
0fa7d5c29b
bug: Correct nonce validation ( #409 )
...
* feat: Correct nonce validation
* clenup
* Update crates/consensus/src/verification.rs
Co-authored-by: Georgios Konstantopoulos <me@gakonst.com >
2022-12-13 19:28:05 +02:00
Roman Krasiuk
c6d38f0f6a
fix(sync): commit headers upon threshold ( #406 )
2022-12-13 19:11:05 +02:00
Georgios Konstantopoulos
dff3936b29
feat(cli): db cmd scaffold ( #405 )
...
* feat(cli): db cmd scaffold
* feat(cli): add methods for seeding/listing tables
* feat(cli): add reth db stats
* chore: docs / cleanup
* chore: remove ethers
Co-authored-by: Oliver Nordbjerg <hi@notbjerg.me >
2022-12-13 18:57:37 +02:00
Roman Krasiuk
5057e8ec0a
feat(p2p): refactor downloaders and add peer id to the result ( #410 )
...
* feat(p2p): refactor downloaders and add peer id to the result
* rm unused import
* fix tests
* clean up deps
* Update crates/interfaces/src/p2p/error.rs
Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de >
* add split fn
Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de >
2022-12-13 17:14:45 +01:00
