github/reth
1
1
Fork 0
mirror of https://github.com/paradigmxyz/reth.git synced 2026-02-16 01:46:50 -05:00
Code Issues Packages Projects Releases Wiki Activity
350 Commits 1,254 Branches 97 Tags
ae2fc494c8d8d863f484ab53d92ce6a04bfdf1f4
Commit Graph

54 Commits

Author SHA1 Message Date
Bjerg
ae2fc494c8 test: fix header dl error test (#441) 2022-12-14 17:32:26 +02:00
Georgios Konstantopoulos
d5d876b286 fix(headers-stage): do not exit stage on downloader error (#439) 
* fix: return ctrl to pipeline on header dl error

* chore: adjust log levels

* ecies: improve tag check error granularity

* chore: instrument read_ack/read_auth

Co-authored-by: Oliver Nordbjerg <hi@notbjerg.me>
 2022-12-14 16:25:54 +02:00
Bjerg
601bdc5022 feat: wire components together in reth node (#416) 
* feat: naively wire up components

* chore: clippy lints

Co-authored-by: Georgios Konstantopoulos <me@gakonst.com>
 2022-12-14 12:51:56 +02:00
0xKitsune
df9d14127d Reth Book - Stages framework/draft (#360) 
* Added framework and start to draft for stages, stopping here to get feedback on approach before pushing forward

* Update README.md

Fixing some wording / grammar.

* Fixing grammar/wording.

* Added note about non-exhaustive stages list, fixed grammar, fixed State exectuion function name, updated language to reflect that the stream yields a SealedHeader and added language to describe that the initial header validation is only a basic validation.

* updated stages chapter, added bodies, senders, execution, next chapter prelude

* typo

* Added line numbers to code snippets for the stages chapter of the reth book

* address reverse header download + other nits

* add note about book hosting

* tweaked wording, formatting

* Address typo "staring"

* Address typo "HeadderDownloader"

* consolidated book.toml

* updating snippets in stages chapter to ignore errors

* template & removed empty fields from book.toml

* addressed build issues, added templating for source code

* only deploy on push to main

* using single quotes in github action if expression

Co-authored-by: Andrew Kirillov <andrew@paradigm.xyz>
Co-authored-by: Andrew Kirillov <20803092+akirillo@users.noreply.github.com>
 2022-12-14 10:30:45 +02:00
Roman Krasiuk
2ace52a8dd feat(sync): add tx hash number mapping (#417) 2022-12-13 19:53:16 +02:00
rakita
0fa7d5c29b bug: Correct nonce validation (#409) 
* feat: Correct nonce validation

* clenup

* Update crates/consensus/src/verification.rs

Co-authored-by: Georgios Konstantopoulos <me@gakonst.com>
 2022-12-13 19:28:05 +02:00
Roman Krasiuk
c6d38f0f6a fix(sync): commit headers upon threshold (#406) 2022-12-13 19:11:05 +02:00
Georgios Konstantopoulos
dff3936b29 feat(cli): db cmd scaffold (#405) 
* feat(cli): db cmd scaffold

* feat(cli): add methods for seeding/listing tables

* feat(cli): add reth db stats

* chore: docs / cleanup

* chore: remove ethers

Co-authored-by: Oliver Nordbjerg <hi@notbjerg.me>
 2022-12-13 18:57:37 +02:00
Roman Krasiuk
5057e8ec0a feat(p2p): refactor downloaders and add peer id to the result (#410) 
* feat(p2p): refactor downloaders and add peer id to the result

* rm unused import

* fix tests

* clean up deps

* Update crates/interfaces/src/p2p/error.rs

Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>

* add split fn

Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>
 2022-12-13 17:14:45 +01:00
Roman Krasiuk
0a6ef3b5af feat(sync): senders commit threshold (#401) 2022-12-13 11:23:40 +02:00
rakita
e014eaad33 feat(db): Refactor storage libraries (#371) 2022-12-09 23:14:07 -08:00
rakita
4fb3626546 chore: Last PR cleanup, nits (#353) 
* chore: cleanup, tests and nits

* fmt
 2022-12-08 11:16:01 +01:00
rakita
d66138e143 feat(execution): Account NotExisting and block reward changesets (#349) 2022-12-07 21:10:06 -08:00
Georgios Konstantopoulos
4d718bbf33 s/foundry-rs/paradigmxyz 2022-12-07 00:00:45 -08:00
Roman Krasiuk
449321ddc4 refactor(db): separate transaction indexes from block body (#326) 2022-12-06 18:02:04 -08:00
Roman Krasiuk
9628d03871 feat(sync): headers commit threshold (#296) 
* headers stream init

* fix tests

* return header if available regardless of control flow

* proper stream termination & docs

* upd headers stage to consume stream

* adjust response validation for stream

* use cursor.insert for headers

* wrap poll_next in a loop to bypass poking waker

* fix typo

* fix last td lookup

* Apply suggestions from code review

Co-authored-by: Georgios Konstantopoulos <me@gakonst.com>

* misc

* remove waker ref

* dedup response handling logic

* clippy

* add docs to poll

Co-authored-by: Georgios Konstantopoulos <me@gakonst.com>
 2022-12-06 18:02:04 -08:00
rakita
a68cb3f357 feat(stage): Add execution stage (#287) 
* Execution stage

* StorageProviderRef. cleanup, fmt/clippy

* block inclusion

* insert blocks for execution stage

* Sanity test for execution stage

* nits

* Update crates/stages/src/stages/execution.rs

Co-authored-by: Georgios Konstantopoulos <me@gakonst.com>

* Update crates/stages/Cargo.toml

Co-authored-by: Georgios Konstantopoulos <me@gakonst.com>

* Update crates/stages/src/stages/execution.rs

Co-authored-by: Georgios Konstantopoulos <me@gakonst.com>

* Update crates/stages/src/stages/execution.rs

Co-authored-by: Georgios Konstantopoulos <me@gakonst.com>

* feat(execution) Execution change diff. (#323)

* Insert to execution changediffs

* progress on exec stage unwind

* wip

* Remove tables:BlockBody from execution stage

* unwind passing

* Nits cleanup and refactoring

* DatabaseIntegrity errors

* KECCAK_EMPTY acc to None

* Update crates/interfaces/src/db/tables.rs

Co-authored-by: Georgios Konstantopoulos <me@gakonst.com>

* Update crates/interfaces/src/provider/block.rs

Co-authored-by: Georgios Konstantopoulos <me@gakonst.com>

* Update crates/stages/src/stages/execution.rs

Co-authored-by: Georgios Konstantopoulos <me@gakonst.com>

* nits

Co-authored-by: Georgios Konstantopoulos <me@gakonst.com>

Co-authored-by: Georgios Konstantopoulos <me@gakonst.com>
 2022-12-06 18:02:04 -08:00
Roman Krasiuk
3ac52bae84 chore(sync): unwind tx by latest available (#280) 2022-11-28 14:49:54 -08:00
Roman Krasiuk
4d708ce8af chore(sync): basic header response validation (#276) 2022-11-28 09:32:46 -08:00
joshieDo
3efc6ee67e feat(db): Make TransactionSigned compactable (#252) 2022-11-27 23:29:30 -08:00
Matthias Seitz
fd840e1c66 refactor: bodies client API (#257) 
* refactor: bodies client API

* chore: fix errors
 2022-11-26 10:04:42 +01:00
Roman Krasiuk
5d5b83d575 fix(sync): download error handling (#260) 2022-11-25 22:27:53 -08:00
Roman Krasiuk
6e7928ab84 feat(sync): stage db helper (#240) 
* feat(sync): stage db helper

* stagedb cont

* merge stage db & db container

* rename test stage db accessor methods

* clippy

* remove legacy test
 2022-11-25 07:12:13 -08:00
Matthias Seitz
fb2861f112 feat(net): integrate HeadersClient (#251) 
* refactor: headers client

* chore: rustfmt

* chore(clippy): make clippy happy

* feat(net): integrate HeadersClient

* fix: parse td as U256 with from_little_endian

* conflicts

Co-authored-by: Georgios Konstantopoulos <me@gakonst.com>
 2022-11-25 14:26:43 +01:00
Matthias Seitz
dda8df7341 refactor: headers client (#249) 
* refactor: headers client

* chore: rustfmt

* chore(clippy): make clippy happy

* address comments

* use Error instead

* fix(sync): headers test client & stage tests (#255)

* headers test client & stage tests

* fix timeout test

* fix import

Co-authored-by: Roman Krasiuk <rokrassyuk@gmail.com>
 2022-11-25 14:00:06 +01:00
Roman Krasiuk
82b37b9bfb feat(sync): sender recovery stage (#181) 
* feat(sync): sender recovery stage

* execute tests

* more tests & cleanup

* senders cont

* clean up & comments

* clippy

* cleanup

* comments

* put back arbitrary dep
 2022-11-23 00:22:17 -08:00
Bjerg
a523cb7024 feat: finish concurrent body downloader (#220) 
* refactor: remove timeout config from downloader

The timeout should be controlled by the client
implementation.

* feat: downloader request retries

* test: add concurrent body downloader tests

* chore: fmt

Co-authored-by: Georgios Konstantopoulos <me@gakonst.com>
 2022-11-22 21:06:49 +01:00
Roman Krasiuk
4936d467c9 test(sync): stage test suite (#204) 
* test(sync): stage test suite

* cleanup txindex tests

* nit

* start revamping bodies testing

* revamp body testing

* add comments to suite tests

* fmt

* cleanup dup code

* cleanup insert_headers helper fn

* fix tests

* linter

* switch mutex to atomic

* cleanup

* revert

* test: make unwind runner return value instead of channel

* test: make execute runner return value instead of channel

* Revert "test: make execute runner return value instead of channel"

This reverts commit f8608654f2.

Co-authored-by: Georgios Konstantopoulos <me@gakonst.com>
 2022-11-18 17:57:29 -08:00
rakita
09c5c3449e feat(executor): transaction and receipts verification (#195) 
* transaction verification

* Add providers to interfaces

* add receipt check in executor verify

* Removed receipts, added bloom

* post merge header checks

* fmt clippy

* gas price type removal

* rm leftover

* move consts to contstant.rs
 2022-11-16 11:10:36 -08:00
Bjerg
92a7818512 feat: bodies stage (#190) 
* chore: clean up `.gitignore`

* fix: make RO cursors `Send + Sync`

* feat(wip): bodies stage

* driveby: improve docs

* chore: don't panic if we're the first stage

* chore: use `Vec` for ommers

* feat: error handling in bodies downloader

* chore: remove stale comment

* chore: pascal-case stage id

* refactor: remove unused new fns

* refactor: distinguish downloaders with prefix

* refactor: move downloader errs to own module

* refactor: `stream_bodies` -> `bodies_stream`

* test: fix borked imports in header stage

* test: clean up header tests

* test: add basic body stage tests

* test: add 2 more body stage test skeletons

* test: move generator test utils to own module

* refactor: move proof functions to primitives crate

* feat: add block generator test utils

* test: more body stage tests

* chore: fix typo (`Cannonical*` -> `Canonical`)

* docs: document `bodies_to_download`

* test: more body stage tests

* test: more body stage tests

* refactor: clean up body stage tests a bit

* test: fix broken tests

* refactor: clean up body stage tests

* test: more body stage tests
 2022-11-15 08:49:11 +01:00
rakita
7ecbe01741 feat(provider): Add StorageProvider impl, table changes (#172) 
* feat(provider): Add StorageProvider impl, table changes

* unwind header numbers by walker (#174)

* readme, fmt

* fix tests

* Update crates/interfaces/src/provider/db_provider/storage.rs

Co-authored-by: Roman Krasiuk <rokrassyuk@gmail.com>

* Update crates/interfaces/src/provider/db_provider/storage.rs

Co-authored-by: Roman Krasiuk <rokrassyuk@gmail.com>

Co-authored-by: Roman Krasiuk <rokrassyuk@gmail.com>
 2022-11-08 15:55:45 +01:00
rakita
86ffb4756d feat(bin): simple reth node command (#158) 
* feat(bin): simple reth node command

* use silent flag

* small chore
 2022-11-07 12:10:09 -08:00
Matthias Seitz
caad026c70 refactor: rename HeaderLocked to SealedHeader (#173) 2022-11-07 21:06:53 +01:00
Roman Krasiuk
9c40f8265b test(sync): headers stage test runner (#163) 
* refactor headers stage tests

* inline unwind util functions
 2022-11-04 08:15:34 -07:00
Roman Krasiuk
f6d3a49d28 feat(sync): stage tx index (#149) 
* init

* feat(sync): tx index stage

* create test runner

* finish tests

* clippy

* cleanup & docs

* add more comments

* revert headers test refactor

* rm unused function
 2022-11-04 03:28:18 -07:00
joshieDo
becceb2802 fix(db): add traits Encode/Decode for keys and Compress/Uncompress for values (#151) 
* add UncompressedUint

* add more UncompressedUint keys

* add docs

* split key/value to encode/decode and compress/uncompress traits

* reveert into

* clippy

* rm whitespaces

* remove TODO

* Remove scale encode/decode traits

* decompress

* clippy

Co-authored-by: rakita <dragan0rakita@gmail.com>
 2022-11-03 00:02:19 -07:00
Roman Krasiuk
c232a72338 test(stage): headers linear downloader (#143) 
* headers stage linear test

* cleanup unwind
 2022-11-02 17:28:44 -07:00
Roman Krasiuk
580c8eb1e4 chore(sync): db integrity error (#148) 
* db integrity error

* rm unused import
 2022-10-28 01:29:57 -07:00
Roman Krasiuk
61b8829bdf feat(sync): headers stage (#126) 
* port headers stage

* clean up, utils & tests

* more tests

* unwind cleanup & tests

* remove unused rand crate
 2022-10-25 16:50:01 -07:00
Bjerg
65247ef203 feat: metrics (#92) 
* feat(stages): `stage.progress` metric

* docs: metrics
 2022-10-24 15:16:45 +02:00
Georgios Konstantopoulos
a4e505132c headers(part1) feat(interfaces): introduce implicit trait bounds (#117) 
* feat(ifaces): modify DB GAT to have implicit bounds

https://sabrinajewson.org/blog/the-better-alternative-to-lifetime-gats\#the-better-gats

* test(ifaces): impl new trait for mock

* feat(ifaces): impl db container

we use that downstream to avoid consuming the db transactions
when committing a stage

* fix(test): explicitly define lifetime

* test: ensure can spawn

* test: pls review

* chore: lints/warnings

* feat(db): impl the new gats

* test(db): try to make real db work with gats like in stages

* test(db): make Stage trait take DBContainer

otherwise we cannot call db.commit()
gst

* feat(stages): impl new traits

* chore(db): cleanup tests

* chore: remove unused imports
 2022-10-23 23:10:59 -07:00
rakita
483bcdf9ab feat(db): Database Abstraction (#101) 
* database tx traits

* wip build passes

* Db and tx results

* nightly and db GAT

* Impl tx, wip cursor

* Move Decode to Table Key, working cursor trait

* wip dupsort

* build all Cursor abstraction

* cleanup

* wip cleanup

* old stages

* codecs moved o interface,stages wip

* resolve db ref, it builds

* Add tx commit after execution

* fmt

* Remove sync send restriction

* Add missing rw cursor functions

* Cleanup, added missing cursor fn. rust toolchain

* fmt

* add nightly to ci

* deny dead_code, remove unwrap

* rm printfn, stages fix, bench fix
 2022-10-20 08:32:13 -07:00
Bjerg
4790256507 feat: skip/error pipeline events (#70) 2022-10-14 14:08:41 +02:00
Bjerg
0d97014f4c docs: mermaid diagram for pipeline (#67) 2022-10-14 13:23:13 +02:00
Bjerg
111c13f12f test: more staged sync tests (#49) 2022-10-11 23:04:07 +02:00
Roman Krasiuk
bbb46216ea chore: test utils (#48) 2022-10-11 20:09:18 +02:00
Roman Krasiuk
5ebfc01338 chore: migrate stages to new db abstractions (#43) 
* migrate

* set block as u64

* linter

* BNum -> BlockNumber

* stage id alias
 2022-10-11 19:54:05 +02:00
Bjerg
a8c1eaf140 feat: simplified unwind priority (#45) 2022-10-11 18:52:05 +02:00
Georgios Konstantopoulos
bce14b22ce chore: staged sync cleanup (#34) 
* docs: expand on unwinding

* chore: commit inside run_loop / cleanup run

* chore: instrument unwind via span to make more readable

* chore: address review comments
 2022-10-10 19:48:20 -07:00
Bjerg
a759201b40 refactor(stages): split Pipeline::run (#30) 2022-10-10 15:52:23 +02:00