rakita
e014eaad33
feat(db): Refactor storage libraries ( #371 )
2022-12-09 23:14:07 -08:00
rakita
4fb3626546
chore: Last PR cleanup, nits ( #353 )
...
* chore: cleanup, tests and nits
* fmt
2022-12-08 11:16:01 +01:00
rakita
d66138e143
feat(execution): Account NotExisting and block reward changesets ( #349 )
2022-12-07 21:10:06 -08:00
Georgios Konstantopoulos
4d718bbf33
s/foundry-rs/paradigmxyz
2022-12-07 00:00:45 -08:00
Roman Krasiuk
449321ddc4
refactor(db): separate transaction indexes from block body ( #326 )
2022-12-06 18:02:04 -08:00
Roman Krasiuk
9628d03871
feat(sync): headers commit threshold ( #296 )
...
* headers stream init
* fix tests
* return header if available regardless of control flow
* proper stream termination & docs
* upd headers stage to consume stream
* adjust response validation for stream
* use cursor.insert for headers
* wrap poll_next in a loop to bypass poking waker
* fix typo
* fix last td lookup
* Apply suggestions from code review
Co-authored-by: Georgios Konstantopoulos <me@gakonst.com >
* misc
* remove waker ref
* dedup response handling logic
* clippy
* add docs to poll
Co-authored-by: Georgios Konstantopoulos <me@gakonst.com >
2022-12-06 18:02:04 -08:00
rakita
a68cb3f357
feat(stage): Add execution stage ( #287 )
...
* Execution stage
* StorageProviderRef. cleanup, fmt/clippy
* block inclusion
* insert blocks for execution stage
* Sanity test for execution stage
* nits
* Update crates/stages/src/stages/execution.rs
Co-authored-by: Georgios Konstantopoulos <me@gakonst.com >
* Update crates/stages/Cargo.toml
Co-authored-by: Georgios Konstantopoulos <me@gakonst.com >
* Update crates/stages/src/stages/execution.rs
Co-authored-by: Georgios Konstantopoulos <me@gakonst.com >
* Update crates/stages/src/stages/execution.rs
Co-authored-by: Georgios Konstantopoulos <me@gakonst.com >
* feat(execution) Execution change diff. (#323 )
* Insert to execution changediffs
* progress on exec stage unwind
* wip
* Remove tables:BlockBody from execution stage
* unwind passing
* Nits cleanup and refactoring
* DatabaseIntegrity errors
* KECCAK_EMPTY acc to None
* Update crates/interfaces/src/db/tables.rs
Co-authored-by: Georgios Konstantopoulos <me@gakonst.com >
* Update crates/interfaces/src/provider/block.rs
Co-authored-by: Georgios Konstantopoulos <me@gakonst.com >
* Update crates/stages/src/stages/execution.rs
Co-authored-by: Georgios Konstantopoulos <me@gakonst.com >
* nits
Co-authored-by: Georgios Konstantopoulos <me@gakonst.com >
Co-authored-by: Georgios Konstantopoulos <me@gakonst.com >
2022-12-06 18:02:04 -08:00
Roman Krasiuk
3ac52bae84
chore(sync): unwind tx by latest available ( #280 )
2022-11-28 14:49:54 -08:00
Roman Krasiuk
4d708ce8af
chore(sync): basic header response validation ( #276 )
2022-11-28 09:32:46 -08:00
joshieDo
3efc6ee67e
feat(db): Make TransactionSigned compactable ( #252 )
2022-11-27 23:29:30 -08:00
Matthias Seitz
fd840e1c66
refactor: bodies client API ( #257 )
...
* refactor: bodies client API
* chore: fix errors
2022-11-26 10:04:42 +01:00
Roman Krasiuk
5d5b83d575
fix(sync): download error handling ( #260 )
2022-11-25 22:27:53 -08:00
Roman Krasiuk
6e7928ab84
feat(sync): stage db helper ( #240 )
...
* feat(sync): stage db helper
* stagedb cont
* merge stage db & db container
* rename test stage db accessor methods
* clippy
* remove legacy test
2022-11-25 07:12:13 -08:00
Matthias Seitz
fb2861f112
feat(net): integrate HeadersClient ( #251 )
...
* refactor: headers client
* chore: rustfmt
* chore(clippy): make clippy happy
* feat(net): integrate HeadersClient
* fix: parse td as U256 with from_little_endian
* conflicts
Co-authored-by: Georgios Konstantopoulos <me@gakonst.com >
2022-11-25 14:26:43 +01:00
Matthias Seitz
dda8df7341
refactor: headers client ( #249 )
...
* refactor: headers client
* chore: rustfmt
* chore(clippy): make clippy happy
* address comments
* use Error instead
* fix(sync): headers test client & stage tests (#255 )
* headers test client & stage tests
* fix timeout test
* fix import
Co-authored-by: Roman Krasiuk <rokrassyuk@gmail.com >
2022-11-25 14:00:06 +01:00
Roman Krasiuk
82b37b9bfb
feat(sync): sender recovery stage ( #181 )
...
* feat(sync): sender recovery stage
* execute tests
* more tests & cleanup
* senders cont
* clean up & comments
* clippy
* cleanup
* comments
* put back arbitrary dep
2022-11-23 00:22:17 -08:00
Bjerg
a523cb7024
feat: finish concurrent body downloader ( #220 )
...
* refactor: remove timeout config from downloader
The timeout should be controlled by the client
implementation.
* feat: downloader request retries
* test: add concurrent body downloader tests
* chore: fmt
Co-authored-by: Georgios Konstantopoulos <me@gakonst.com >
2022-11-22 21:06:49 +01:00
Roman Krasiuk
4936d467c9
test(sync): stage test suite ( #204 )
...
* test(sync): stage test suite
* cleanup txindex tests
* nit
* start revamping bodies testing
* revamp body testing
* add comments to suite tests
* fmt
* cleanup dup code
* cleanup insert_headers helper fn
* fix tests
* linter
* switch mutex to atomic
* cleanup
* revert
* test: make unwind runner return value instead of channel
* test: make execute runner return value instead of channel
* Revert "test: make execute runner return value instead of channel"
This reverts commit f8608654f2me@gakonst.com >
2022-11-18 17:57:29 -08:00
rakita
09c5c3449e
feat(executor): transaction and receipts verification ( #195 )
...
* transaction verification
* Add providers to interfaces
* add receipt check in executor verify
* Removed receipts, added bloom
* post merge header checks
* fmt clippy
* gas price type removal
* rm leftover
* move consts to contstant.rs
2022-11-16 11:10:36 -08:00
Bjerg
92a7818512
feat: bodies stage ( #190 )
...
* chore: clean up `.gitignore`
* fix: make RO cursors `Send + Sync`
* feat(wip): bodies stage
* driveby: improve docs
* chore: don't panic if we're the first stage
* chore: use `Vec` for ommers
* feat: error handling in bodies downloader
* chore: remove stale comment
* chore: pascal-case stage id
* refactor: remove unused new fns
* refactor: distinguish downloaders with prefix
* refactor: move downloader errs to own module
* refactor: `stream_bodies` -> `bodies_stream`
* test: fix borked imports in header stage
* test: clean up header tests
* test: add basic body stage tests
* test: add 2 more body stage test skeletons
* test: move generator test utils to own module
* refactor: move proof functions to primitives crate
* feat: add block generator test utils
* test: more body stage tests
* chore: fix typo (`Cannonical*` -> `Canonical`)
* docs: document `bodies_to_download`
* test: more body stage tests
* test: more body stage tests
* refactor: clean up body stage tests a bit
* test: fix broken tests
* refactor: clean up body stage tests
* test: more body stage tests
2022-11-15 08:49:11 +01:00
rakita
7ecbe01741
feat(provider): Add StorageProvider impl, table changes ( #172 )
...
* feat(provider): Add StorageProvider impl, table changes
* unwind header numbers by walker (#174 )
* readme, fmt
* fix tests
* Update crates/interfaces/src/provider/db_provider/storage.rs
Co-authored-by: Roman Krasiuk <rokrassyuk@gmail.com >
* Update crates/interfaces/src/provider/db_provider/storage.rs
Co-authored-by: Roman Krasiuk <rokrassyuk@gmail.com >
Co-authored-by: Roman Krasiuk <rokrassyuk@gmail.com >
2022-11-08 15:55:45 +01:00
rakita
86ffb4756d
feat(bin): simple reth node command ( #158 )
...
* feat(bin): simple reth node command
* use silent flag
* small chore
2022-11-07 12:10:09 -08:00
Matthias Seitz
caad026c70
refactor: rename HeaderLocked to SealedHeader ( #173 )
2022-11-07 21:06:53 +01:00
Roman Krasiuk
9c40f8265b
test(sync): headers stage test runner ( #163 )
...
* refactor headers stage tests
* inline unwind util functions
2022-11-04 08:15:34 -07:00
Roman Krasiuk
f6d3a49d28
feat(sync): stage tx index ( #149 )
...
* init
* feat(sync): tx index stage
* create test runner
* finish tests
* clippy
* cleanup & docs
* add more comments
* revert headers test refactor
* rm unused function
2022-11-04 03:28:18 -07:00
joshieDo
becceb2802
fix(db): add traits Encode/Decode for keys and Compress/Uncompress for values ( #151 )
...
* add UncompressedUint
* add more UncompressedUint keys
* add docs
* split key/value to encode/decode and compress/uncompress traits
* reveert into
* clippy
* rm whitespaces
* remove TODO
* Remove scale encode/decode traits
* decompress
* clippy
Co-authored-by: rakita <dragan0rakita@gmail.com >
2022-11-03 00:02:19 -07:00
Roman Krasiuk
c232a72338
test(stage): headers linear downloader ( #143 )
...
* headers stage linear test
* cleanup unwind
2022-11-02 17:28:44 -07:00
Roman Krasiuk
580c8eb1e4
chore(sync): db integrity error ( #148 )
...
* db integrity error
* rm unused import
2022-10-28 01:29:57 -07:00
Roman Krasiuk
61b8829bdf
feat(sync): headers stage ( #126 )
...
* port headers stage
* clean up, utils & tests
* more tests
* unwind cleanup & tests
* remove unused rand crate
2022-10-25 16:50:01 -07:00
Bjerg
65247ef203
feat: metrics ( #92 )
...
* feat(stages): `stage.progress` metric
* docs: metrics
2022-10-24 15:16:45 +02:00
Georgios Konstantopoulos
a4e505132c
headers(part1) feat(interfaces): introduce implicit trait bounds ( #117 )
...
* feat(ifaces): modify DB GAT to have implicit bounds
https://sabrinajewson.org/blog/the-better-alternative-to-lifetime-gats\#the-better-gats
* test(ifaces): impl new trait for mock
* feat(ifaces): impl db container
we use that downstream to avoid consuming the db transactions
when committing a stage
* fix(test): explicitly define lifetime
* test: ensure can spawn
* test: pls review
* chore: lints/warnings
* feat(db): impl the new gats
* test(db): try to make real db work with gats like in stages
* test(db): make Stage trait take DBContainer
otherwise we cannot call db.commit()
gst
* feat(stages): impl new traits
* chore(db): cleanup tests
* chore: remove unused imports
2022-10-23 23:10:59 -07:00
rakita
483bcdf9ab
feat(db): Database Abstraction ( #101 )
...
* database tx traits
* wip build passes
* Db and tx results
* nightly and db GAT
* Impl tx, wip cursor
* Move Decode to Table Key, working cursor trait
* wip dupsort
* build all Cursor abstraction
* cleanup
* wip cleanup
* old stages
* codecs moved o interface,stages wip
* resolve db ref, it builds
* Add tx commit after execution
* fmt
* Remove sync send restriction
* Add missing rw cursor functions
* Cleanup, added missing cursor fn. rust toolchain
* fmt
* add nightly to ci
* deny dead_code, remove unwrap
* rm printfn, stages fix, bench fix
2022-10-20 08:32:13 -07:00
Bjerg
4790256507
feat: skip/error pipeline events ( #70 )
2022-10-14 14:08:41 +02:00
Bjerg
0d97014f4c
docs: mermaid diagram for pipeline ( #67 )
2022-10-14 13:23:13 +02:00
Bjerg
111c13f12f
test: more staged sync tests ( #49 )
2022-10-11 23:04:07 +02:00
Roman Krasiuk
bbb46216ea
chore: test utils ( #48 )
2022-10-11 20:09:18 +02:00
Roman Krasiuk
5ebfc01338
chore: migrate stages to new db abstractions ( #43 )
...
* migrate
* set block as u64
* linter
* BNum -> BlockNumber
* stage id alias
2022-10-11 19:54:05 +02:00
Bjerg
a8c1eaf140
feat: simplified unwind priority ( #45 )
2022-10-11 18:52:05 +02:00
Georgios Konstantopoulos
bce14b22ce
chore: staged sync cleanup ( #34 )
...
* docs: expand on unwinding
* chore: commit inside run_loop / cleanup run
* chore: instrument unwind via span to make more readable
* chore: address review comments
2022-10-10 19:48:20 -07:00
Bjerg
a759201b40
refactor(stages): split Pipeline::run ( #30 )
2022-10-10 15:52:23 +02:00
Bjerg
72968d758b
refactor: pipeline refactor ( #23 )
...
* refactor: split stage crate into modules
* refactor: clean up pipeline events
* refactor: internal fn for executing single stage
* refactor: pipeline error
* refactor: move errors to own module
* refactor: misc cleanup
* test: add tests for `opt::max` and `opt::min`
* chore: db errors (#26 )
* refactor: replace ext trait with wrapper type
Co-authored-by: Roman Krasiuk <rokrassyuk@gmail.com >
2022-10-10 05:49:45 +02:00
Bjerg
8317cd2b63
chore: use primitives::BlockNumber ( #17 )
2022-10-06 19:30:32 +02:00
Bjerg
c749658fd4
feat: pipeline loop ( #10 )
...
* feat: block number in validation error
* feat: wip pipeline execution and unwind
* chore: clippy
* feat: remove dummy db tx in stages crate
* feat: stage progress saving and loading
* test: wip tests
* feat: basic tracing in pipeline
* feat: pipeline event channel
* test: wip more tests
* test: test unwind during run
* chore: readability shuffle
* chore: incl block number in validation error
2022-10-06 18:24:43 +02:00
Bjerg
6868dda544
feat: basic staged sync crate ( #6 )
...
* feat: basic staged sync crate
* refactor: rename to `reth-stages`
* feat: remove eyre
* feat: impl `Error` for `StageError`
* chore: bump cargo lock
* chore: nits
* docs: clarify unwind
Co-authored-by: Georgios Konstantopoulos <me@gakonst.com >
2022-10-03 14:40:50 +02:00
