Commit Graph

45 Commits

Author SHA1 Message Date
Bjerg
65247ef203 feat: metrics (#92)
* feat(stages): `stage.progress` metric

* docs: metrics
2022-10-24 15:16:45 +02:00
joshieDo
8bc8bd6820 feat: add IntegerList type Elias-Fano (#111)
* add elias

* docs: should not be empty

* change models doc

* add arbitrary feature

* add simple test

* add test-utils to primitives

* cfg arbitrary
2022-10-24 06:14:59 -07:00
Georgios Konstantopoulos
15bd88e637 headers(part2) - feat: add Downloader trait and test utils (#118)
* feat(interfaces): implement header client traits

* feat: add downloader trait implementer

* feat: use explicit error type instead of ok(false)

* feat: add constructor to HeaderLocked

* test: scaffold mock consensus, downloader and headersclient helpers

* test: implement test consensus

* test: implement test headers client

* refactor: cleanup download headers

* chore: fix lint

* s/test_utils/test_helpers

* headers(part 3) feat: implement Linear downloader (#119)

* feat: add headers downloaders crate

* feat: more scaffolding

* interfaces: generalize retryable erros

* feat: implement linear downloader

* fix linear downloader tests & add builder

* extend & reverse

* feat: linear downloader generics behind arc and reversed return order (#120)

* put client & consensus behind arc and return headers in rev

* cleanup

Co-authored-by: Roman Krasiuk <rokrassyuk@gmail.com>

* extract test_utils

* cargo fmt

Co-authored-by: Roman Krasiuk <rokrassyuk@gmail.com>
2022-10-24 04:56:43 -07:00
Georgios Konstantopoulos
a4e505132c headers(part1) feat(interfaces): introduce implicit trait bounds (#117)
* feat(ifaces): modify DB GAT to have implicit bounds

https://sabrinajewson.org/blog/the-better-alternative-to-lifetime-gats\#the-better-gats

* test(ifaces): impl new trait for mock

* feat(ifaces): impl db container

we use that downstream to avoid consuming the db transactions
when committing a stage

* fix(test): explicitly define lifetime

* test: ensure can spawn

* test: pls review

* chore: lints/warnings

* feat(db): impl the new gats

* test(db): try to make real db work with gats like in stages

* test(db): make Stage trait take DBContainer

otherwise we cannot call db.commit()
gst

* feat(stages): impl new traits

* chore(db): cleanup tests

* chore: remove unused imports
2022-10-23 23:10:59 -07:00
Matthias Seitz
8af8c89018 feat: support finalized and safe variants (#115) 2022-10-21 20:02:06 +02:00
rakita
483bcdf9ab feat(db): Database Abstraction (#101)
* database tx traits

* wip build passes

* Db and tx results

* nightly and db GAT

* Impl tx, wip cursor

* Move Decode to Table Key, working cursor trait

* wip dupsort

* build all Cursor abstraction

* cleanup

* wip cleanup

* old stages

* codecs moved o interface,stages wip

* resolve db ref, it builds

* Add tx commit after execution

* fmt

* Remove sync send restriction

* Add missing rw cursor functions

* Cleanup, added missing cursor fn. rust toolchain

* fmt

* add nightly to ci

* deny dead_code, remove unwrap

* rm printfn, stages fix, bench fix
2022-10-20 08:32:13 -07:00
Matthias Seitz
e9945b565a chore: reexport sha3 from primitives (#109)
* chore: reexport sha3 from primitives

* feat: add keccak256

* replace sha3 with tinykeccak
2022-10-20 16:43:29 +02:00
Matthias Seitz
4298e3aa87 feat(rpc): more rpc scaffolding (#99)
* feat(rpc): more rpc scaffolding

* use Into<String>

* rm unused error
2022-10-19 22:12:53 +02:00
joshieDo
630baf5d70 feat(db): fuzzing & benchmark (#86)
* fuzz with test-fuzz

* move fuzzing to db/codecs

* add criterion & iai

* print encoded size sum on criterion benchmark

* fix BlockNumHash encode

* add gh action for benchmarks

* don't sum results

* test ci values

* Revert "test ci values"

This reverts commit cc47fd8053.

* specify criterion version

* add docs

* remove benchmark job
2022-10-19 13:08:07 -07:00
Matthias Seitz
2e19f94048 feat: impl rlp for receipt (#83)
* feat: impl rlp for receipt

* fix: change to bloom

* chore: rustfmt
2022-10-17 07:13:02 -07:00
joshieDo
063b444792 feat(db): codec encoding/decoding (#51)
* wip

* add table macro

* add simple put get test with Address

* add Env.view and Env.update

* docs

* slightly change the test

* add initial table initialization and placeholders

* lint & some

* replace String with str

* add error.rs

* add docs to encode

* add docs

* clamp

* add source on libmdbx_max_page_size

* add BlockNumer_BlockHash

* add scale

* set header filed to bytes Bytes

* remove unwrap

* restrict scale to chosen types

* into bytes

* add postcard

* changed to BlockNumHash

* add proc_macro_attribute codecs

* fix feature flagging

* set a version for postcard

* cleanup

* seal ScaleOnly

* remove unnecessary dependencies

* properly encode/decode blocknumhash

* change Account codec to scale

* add missing feature to scale

* add codec to a couple more types

* silence clippy

* add docs about table encoding

* move and add reth-codecs

* clippy

* make proc-macro visible

* add README.md
2022-10-17 01:04:57 -07:00
Georgios Konstantopoulos
8009d997c0 Implement ETH P2P (#81)
* refactor: move things to types

* feat(ethwire): bring in message type from ethp2p

30c11138d5/src/message.rs

* feat(ethwire): add eth-stream with Stream/Sink impls

* feat(ethwire): make Sink error an EthStreamError

* feat(ecies): expose util module

* add more deps

* feat: add broadcast newblockhashes

* fix: rlp serialize with message-id first

* chore: cleanup doc

* wip: test eth connection

* bump cargo lock

* feat: add rlp codec and get stream tests working

* fix: convert RlpCodec to PassthroughCodec

we were rlp encoding twice

* Revert "fix: convert RlpCodec to PassthroughCodec"

This reverts commit 6e6e0a5811.

This does not handle framing, which would fail decoding if a partial
write happened to the socket.

* add tracing

* refactor: add handshake error

* feat(ethwire): add status handshake

* test(ethwire): handshake works

* refactor: expose EthStream::new

* chore: clippy lints

* fix: get rid of rlp codec

we can instead use LengthLimitedCodec from Tokio IO, which we re-export
as PassthroughCodec

* fix(eth): add handshake + msg error checks

1. 10MB message lengths
2. Same Genesis, Version, Chain on Status Handshake

* chore: ignore result_large_err

Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>
2022-10-16 20:10:25 -07:00
Dan Cline
5dfe5ac29b feat(net): add Framed ECIES implementation (#80)
* feat(net): add ecies crate

* cargo fmt

* add hex-literal and proptest to dev-dependencies

 * adds std feature to reth-rlp

* document util

* document more

* chore: allow missing docs in ecies/algorith.rs

* feat(ecies): improve error handling

remove anyhow and strictly type all errors

* refactor(ecies): movem ingress/egress to lib.rs

* chore(ecies): allow missing docs in mac

* chore(ecies): cleanup utils

* refactor(ecies): move ECIES Codec to separate file

* refactor(ecies): rename proto to stream

* add test scaffold

* implement server/client read/write test

* chore: clippy / fmt

Co-authored-by: Georgios Konstantopoulos <me@gakonst.com>
2022-10-15 18:20:17 -07:00
Dan Cline
07a83b163f feat(primitives): add H128 (#79)
* feat(primitives): add H128

 * uses patch for now

* cargo fmt
2022-10-14 20:06:26 -07:00
Matthias Seitz
533c3ebe98 feat(rpc): add EthApiServer impl skeleton (#75) 2022-10-14 12:01:00 -07:00
rakita
a644318c32 feat(p2p): Include forkid file from apache licenced repo (#68)
* Start using reth-rlp
* Include apache licenced forkid file in primitives
* Move forkid to eth-wire
2022-10-14 16:52:59 +02:00
Bjerg
0d97014f4c docs: mermaid diagram for pipeline (#67) 2022-10-14 13:23:13 +02:00
rakita
92e64ed71f include apache licenced fastrlp as reth-rlp (#63)
* include apache licenced fastrlp as reth-rlp
* fmt
* Include LICENCE and README for reth-rlp
2022-10-14 12:43:21 +02:00
Matthias Seitz
84ec30db5b feat(txpool): add support for mock testing (#55)
* chore: some cleanup

* refactor(txpool): simplify layers and add docs

* refactor: more cleanup

* refactor: cleanup and simplifications

* feat(txpool): mock test support

* feat(txpool): more mock testing

* chore: rustfmt

* set basefee correctly
2022-10-13 10:44:31 -07:00
Matthias Seitz
6d4e39deef refactor(txpool): cleanup pass (#53)
* chore: some cleanup

* refactor(txpool): simplify layers and add docs

* refactor: more cleanup

* refactor: cleanup and simplifications
2022-10-12 18:23:08 +02:00
rakita
ae5935e6b2 feat: reth binary and parsing of eth chain tests (#38)
* Scaffolding for reth test binary. wip for blockchain tests

* wip models for chain json tests

* reth binary and chain test

* fmt,clippy

* Update bin/reth/src/lib.rs

Co-authored-by: Bjerg <onbjerg@users.noreply.github.com>

* Update bin/reth/src/lib.rs

Co-authored-by: Bjerg <onbjerg@users.noreply.github.com>

* Update bin/reth/src/test_eth_chain/mod.rs

Co-authored-by: Bjerg <onbjerg@users.noreply.github.com>

* Update bin/reth/src/test_eth_chain/mod.rs

Co-authored-by: Bjerg <onbjerg@users.noreply.github.com>

* Update bin/reth/src/test_eth_chain/mod.rs

Co-authored-by: Bjerg <onbjerg@users.noreply.github.com>

* Update bin/reth/src/test_eth_chain/models.rs

Co-authored-by: Bjerg <onbjerg@users.noreply.github.com>

* Update bin/reth/src/test_eth_chain/models.rs

Co-authored-by: Bjerg <onbjerg@users.noreply.github.com>

* Update bin/reth/src/test_eth_chain/mod.rs

Co-authored-by: Bjerg <onbjerg@users.noreply.github.com>

* Update bin/reth/src/test_eth_chain/runner.rs

Co-authored-by: Bjerg <onbjerg@users.noreply.github.com>

* Move JsonU256 to primitives

* fmt

* Use eyre

* nits

Co-authored-by: Bjerg <onbjerg@users.noreply.github.com>
2022-10-12 17:43:01 +02:00
Matthias Seitz
0b1c86df30 docs(txpool): add mermaid diagram (#46) 2022-10-11 19:07:16 +02:00
Matthias Seitz
3fed7cfe21 feat(txpool): new txpool design (#22) 2022-10-11 17:10:02 +02:00
Matthias Seitz
8eb2ea4152 feat(rpc): add rpc-api client feature (#33)
* feat(rpc): add rpc-api client feature

* refactor: combine proc macro

* feat: add missing deserialize functions

* add missing derive
2022-10-11 11:56:12 +02:00
joshieDo
60d3c64410 feat(db): mdbx integration & table skeletons (#15)
* wip

* add table macro

* add simple put get test with Address

* add Env.view and Env.update

* docs

* slightly change the test

* add initial table initialization and placeholders

* lint & some

* replace String with str

* add error.rs

* add docs to encode

* add docs

* clamp

* add source on libmdbx_max_page_size
2022-10-10 12:35:35 -07:00
Dan Cline
230e9ef179 feat(p2p): add anchor file for discovery state (#11)
* feat(p2p): add anchor file for discovery state

* move rustdoc and improve error messages

Co-authored-by: Bjerg <onbjerg@users.noreply.github.com>

* add temp file tests and log drop error

 * fix error due to lack of read option
 * fix empty and nonexistent file error

* remove redundant new

* replace println with tracing

* show underlying error in custom error message

* chore: cargo fmt

* change AsRef<Path> to &Path

* remove ineffective dedups

* chore: cargo fmt

* switch out Vec<Enr<K>> for HashSet<Enr<K>>

* cargo fmt

* use tempdir instead of of std::env::temp_dir

* refactor anchor to contain &Path instead of File

 * change new_from_file to explicitly include logic for opening existing
   files, rather than calling out to from_toml
 * remove from_toml because new_from_file handles existing files
   properly with only a path. It is not possible to obtain a Path from a
   File anyways, its only purpose was to accept a File type

* use PathBuf instead of Path

Co-authored-by: Bjerg <onbjerg@users.noreply.github.com>
2022-10-10 13:02:54 -04:00
Dan Cline
d7c8b70cc3 feat: add eth-wire (#20) 2022-10-10 09:46:02 -07:00
Roman Krasiuk
573d343b66 chore: extend consensus with fork choice state (#27)
* consensus fork choice

* fork_choice -> fork_choice_state
2022-10-09 19:31:57 +02:00
Matthias Seitz
791ee2839f feat(txpool): initial sketch (#7) 2022-10-07 22:00:19 +02:00
Dan Cline
a3a7f1317b feat(primitives): add Chain (#18)
* this is basically the same as in foundry-config
2022-10-06 20:52:19 +02:00
Bjerg
c749658fd4 feat: pipeline loop (#10)
* feat: block number in validation error

* feat: wip pipeline execution and unwind

* chore: clippy

* feat: remove dummy db tx in stages crate

* feat: stage progress saving and loading

* test: wip tests

* feat: basic tracing in pipeline

* feat: pipeline event channel

* test: wip more tests

* test: test unwind during run

* chore: readability shuffle

* chore: incl block number in validation error
2022-10-06 18:24:43 +02:00
rakita
074f452275 Small skeleton for executor abstraction (#14) 2022-10-06 16:01:23 +02:00
rakita
bc30cbce61 Skeleton primitives and interface crate (#13)
* wip interface primitives

* wip

* Integrate it inside rpc- crates

* fmt

* move tx to mod.rs

* Add interfaces, executor to toml

* Added nits, comments fix
2022-10-06 14:48:57 +02:00
Bjerg
849e8ae518 feat: add db crate (#9) 2022-10-03 16:13:42 +02:00
Matthias Seitz
eeda03a3f3 feat: add missing eth_ functions 2022-10-03 15:06:58 +02:00
Matthias Seitz
7c35e68f8e Merge branch 'master' into matt/rpc-support 2022-10-03 14:56:38 +02:00
Bjerg
6868dda544 feat: basic staged sync crate (#6)
* feat: basic staged sync crate

* refactor: rename to `reth-stages`

* feat: remove eyre

* feat: impl `Error` for `StageError`

* chore: bump cargo lock

* chore: nits

* docs: clarify unwind

Co-authored-by: Georgios Konstantopoulos <me@gakonst.com>
2022-10-03 14:40:50 +02:00
Matthias Seitz
ac1ec25435 feat: add rpc types and interface definiton 2022-10-01 19:32:04 +02:00
Matthias Seitz
841c87830c feat: add transaction request 2022-10-01 14:15:43 +02:00
Matthias Seitz
2717dd643a feat(rpc): add rpc crates 2022-09-30 20:20:07 +02:00
Dan Cline
6ff7cf5a74 feat: add p2p crate (#4)
* feat: add p2p crate

* move p2p to net subfolder

* add lib.rs and Cargo.toml to net

* remove net crate, use explicit names in workspace
2022-09-30 09:27:52 -07:00
Georgios Konstantopoulos
f3b26fd91b fix typo 2022-09-29 17:58:22 -07:00
Matthias Seitz
4dc32e9f76 feat: add core crate (#3)
* feat: add core crate

* rename primitives

* Update crates/primitives/Cargo.toml

-.-

Co-authored-by: rakita <rakita@users.noreply.github.com>

Co-authored-by: rakita <rakita@users.noreply.github.com>
2022-09-27 10:19:01 -07:00
Matthias Seitz
57a2b2d1d4 feat: add crate template 2022-09-22 20:16:15 +02:00
Oliver Nordbjerg
978527367f chore: init 2022-09-19 19:25:12 +02:00