Commit Graph

487 Commits

Author SHA1 Message Date
Roman Krasiuk
565a0aa90b feat(engine): new payload execution (#631)
* feat(engine): new payload execution

* address PR comments

* rm unused dev deps

* add comment about lru

* remove par_iter
2022-12-29 02:36:56 -08:00
Roman Krasiuk
47d044942b patch nightly toolchain (#642) 2022-12-29 11:16:35 +01:00
Matthias Seitz
15e6a359b4 fix: enable missing feature (#644) 2022-12-29 11:07:49 +01:00
Sanket Shanbhag
28669d4aa8 fix(sync): return control from HeaderStage back to pipeline (#609)
* Headerstage now gives control back to pipeline after commit_threshold blocks have been synced

* Updated tests

* Only take required number of headers from the stream

* Simplified stage complete check
2022-12-29 00:57:49 -08:00
Tomás
76e76bb651 feat: add trusted nodes configuration (#569)
* Add preferred nodes to config

* Add preferred nodes on boot

* Add flag in config for trusted only mode

* Add preferred nodes configuration to config

* Fix comment

* Add preferred nodes to config file

* Rename preferred_nodes to trusted_nodes

* Change preferred to trusted

I renamed preferred_nodes to trusted_nodes in various places.
Seems I missed quite a bit of them.

* Pull trusted_only from config

* Rename DiscoveryConfig to PeersConfig

* Fix last commit

Now actually renamed DiscoveryConfig

* Rename trusted_only to connect_trusted_nodes_only

* Add helper function

* Use HashSet for boot_nodes and trusted_nodes

* Change trusted nodes functions in ConfigBuilder

* Move trusted peers from discv4 to network config

* Add trusted nodes to peers on Manager creation

* Use NodeRecord in trusted_nodes config

* Fix comment

* Move trusted_nodes config to PeersConfig

* Add trusted nodes directly to peers

* Move network_config to Config impl

* Move start_network to NetworkConfig impl
2022-12-28 21:48:11 +01:00
Matthias Seitz
d9d0ba14c4 docs: add outgoing request diagram (#630) 2022-12-28 20:36:53 +01:00
Matthias Seitz
525f28a67d feat(net): integrate external public ip auto discovery (#632)
* feat(net): integrate external public ip auto discovery

* Update crates/net/discv4/src/config.rs

Co-authored-by: Georgios Konstantopoulos <me@gakonst.com>

* rename var

Co-authored-by: Georgios Konstantopoulos <me@gakonst.com>
2022-12-28 16:14:07 +01:00
Matthias Seitz
c3432bc284 feat(rpc): add rpc builder support (#635) 2022-12-28 15:55:30 +02:00
Matthias Seitz
1ba81191f6 perf(net): backoff on IO error (#633) 2022-12-28 12:43:02 +01:00
Andrew Kirillov
d4d8a8c882 Moves code walkthrough book chapters to docs (#629)
* replaced template blocks with code blocks in stages chapter

* replaced template blocks with code blocks in network chapter

* moved book sections to docs

* fix indentation in recover_signer codeblock

* remove unnecessary TODO comment in network.md
2022-12-28 10:24:39 +01:00
Andrew Kirillov
a51fa4fd63 docs(book): start book chapter for network crate (#445)
* WIP FOR DRAFT PR

* WIP draft, walking through usage in pipeline

* diving deeper into instantiation of NetworkHandle & FetchClient

* WIP network management task & eth requests task done (ish)

* address pr comments: networkmanager as standalone task, networkconfig as main entrypoint for setup

* complete draft of the  crate docs

* restructured, starts w/ expl of

* added more snippets/anchors

* using ignore flag on code blocks, brief explanation of  struct

* typos addressed
2022-12-27 14:53:15 -08:00
Tomás
dcd3923d19 Add serde support for NodeRecord primitive type (#617)
* Add serde support to NodeRecord

* Move NodeRecord to primitives

along with NodeKey and Octets

* Reexport NodeRecord from discv4

* Move NodeKey and kad_key back to discv4::node

Also, move NodeRecord::key functionality to a helper function: discv4::node::record_key.
This avoids the discv5 dependency in the primitives crate.

* Fix NodeRecord (de)serializing

The default derive macros work with a dictionary like display.
Changed that to serde_with macros, that use Display and FromStr traits.

* Add some tests for NodeRecord (de)serializing

* Hide NodeKey struct

* Move Octets after NodeRecord

* Replace record_key with From trait

* Fix clippy error

unnecessary into()
2022-12-27 18:03:54 +01:00
Mariano A. Nicolini
c2b19cecef feat: add txpool metrics (#584)
* feat(metrics): `Metrics` derive macro

* rename metrics to metrics-derive

* use fully qualified fmt path

* metric vec with capacity

* favor metrics over simple scope attr, simplify code

* pull metric description from doc comment

* rm debug log

* add more docs and sample usage

* link to metrics macros

* add tests for metrics derive macro

* clippy

* Fix doc of headers_unexpected_errors field of HeaderMetrics

* Add tx pool metrics struct and metrics updates in code

* Format file

* Update metrics doc

* Add some comments

* Format file

* Refactor metrics describers for consistency

* Format files

* Fix broken import

* Apply metrics macro to TxPoolMetrics

* Remove unused imports

* Remove unused commented code

* Remove files

Co-authored-by: Roman Krasiuk <rokrassyuk@gmail.com>
2022-12-27 16:44:41 +01:00
Matthias Seitz
1c3d704244 docs: fix indentation (#626) 2022-12-27 06:43:26 -08:00
Roman Krasiuk
5bb14ecb7c fix(metrics): metrics derive separator (#625)
* fix(metrics): metrics derive separator

* fix existing metrics

* rm log

* static regex
2022-12-27 03:16:41 -08:00
Matthias Seitz
a2c1cdb399 refactor(net): deduplicate error variants (#619) 2022-12-26 23:29:26 +01:00
Roman Krasiuk
663efa8d2a feat(metrics): Metrics derive macro (#592)
* feat(metrics): `Metrics` derive macro

* rename metrics to metrics-derive

* use fully qualified fmt path

* metric vec with capacity

* favor metrics over simple scope attr, simplify code

* pull metric description from doc comment

* rm debug log

* add more docs and sample usage

* link to metrics macros

* add tests for metrics derive macro

* clippy

* simplify path comparison

* cleanup

* remove unused dep

* rename stage metrics
2022-12-26 14:27:24 -08:00
rakita
d2b4081849 feat: Enable ExecutionStage (#622) 2022-12-27 00:09:04 +02:00
rakita
bfe2fd6221 ci: add eth chain test. Set prevrandao after merge (#621)
* Small changes

* feat: Add eth chain test to ci. Set prevrandao after merge

* Update .github/workflows/ci.yml

Co-authored-by: Bjerg <onbjerg@users.noreply.github.com>

* move env

Co-authored-by: Bjerg <onbjerg@users.noreply.github.com>
2022-12-26 22:55:46 +01:00
Matthias Seitz
410a1f9cfe perf: allocate responses capacity (#618) 2022-12-26 21:38:00 +02:00
Georgios Konstantopoulos
daa65e7f58 docs: expose db on book summary 2022-12-26 18:27:53 +02:00
0xKitsune
0744799515 docs(book): db chapter. (#444)
* Added framework and start to draft for stages, stopping here to get feedback on approach before pushing forward

* Update README.md

Fixing some wording / grammar.

* Fixing grammar/wording.

* Added note about non-exhaustive stages list, fixed grammar, fixed State exectuion function name, updated language to reflect that the stream yields a SealedHeader and added language to describe that the initial header validation is only a basic validation.

* updated stages chapter, added bodies, senders, execution, next chapter prelude

* typo

* Added line numbers to code snippets for the stages chapter of the reth book

* address reverse header download + other nits

* add note about book hosting

* tweaked wording, formatting

* Address typo "staring"

* Address typo "HeadderDownloader"

* consolidated book.toml

* updating snippets in stages chapter to ignore errors

* reviewing db crate, starting draft

* Added framework and draft for db chapter

* reformatting to be more readable

* added DbTx and DbTxMut

* updating language, structure of chapter

* restructured chapter to be a more logical read, covered the nested traits throughout the Database trait, added section about cursors.

* wording, header sizes

* updated spacing between headers and sections

* wording

* updated db chapter

* added links to code snippets

* added additional links to code snippets

* removed dup table in list of tables

* updated code snippet and link in codebase

* Update book/db/README.md

Co-authored-by: Andrew Kirillov <20803092+akirillo@users.noreply.github.com>

* updated chapter to mention how Transactions deref trait works to access DbTx::get()

* wording

* fixed typo

* Added a note about the encode and decode traits

* Added a note about the compress and decompress traits

* fixing issues with merge

* removed repetitive example

* a word

Co-authored-by: Andrew Kirillov <andrew@paradigm.xyz>
Co-authored-by: Andrew Kirillov <20803092+akirillo@users.noreply.github.com>
2022-12-26 18:23:40 +02:00
rakita
f21a4020b1 feat(execution): Add ommers reward (#611)
* feat(execution): Add ommers reward

* Refactor and add comments
2022-12-26 15:44:51 +01:00
Matthias Seitz
ebd686a407 refactor(net): rename and restructure wire error types (#614) 2022-12-26 16:33:59 +02:00
Matthias Seitz
8779000460 test(net): add test for dropped pending no response (#613) 2022-12-26 15:51:14 +02:00
Matthias Seitz
f1c1fb85ff fix(net): add no response to backoff check (#610) 2022-12-26 14:00:01 +02:00
Namra Patel
bb10305d09 Fix typo and rename remaining ExecutorDb to StateProvider (#608)
* Fix typo

* ExecuterDb -> StateProvider
2022-12-26 13:41:29 +02:00
Bjerg
ba585cf7d9 fix: do not lowercase table arg in db list (#606) 2022-12-25 22:52:39 +02:00
Matthias Seitz
3a07eb930f feat(net): add nat external ip crate (#605) 2022-12-25 18:22:31 +02:00
Sanket Shanbhag
73e12341c4 feat(net): Make PeerManager emit peerevents for admin rpc (#602)
* Make PeerManager emit peerevents for admin rpc

* Fix tests to include new peer events
2022-12-25 10:00:10 +01:00
rakita
19a618e3a4 fix(db): Dont compress DupSort SubKey (#594)
* bug(db): Dont compress DupSort SubKey

* unwrap or default config, Added notes

* fmt empty lines
2022-12-25 01:13:03 +01:00
Matthias Seitz
5affa67805 refactor(net): unify closed incoming session handling (#600) 2022-12-24 12:38:21 +01:00
Sanket Shanbhag
43008b7b07 feat: Add PeerEvent to NetworkEvent enum (#574)
* Add peerAdded and peerRemoved events to NetworkEvent enum

* Remove unnecessary todo macros
2022-12-24 10:25:09 +01:00
Matthias Seitz
7c80dde0ec fix(net): check for handshake disconnect (#596) 2022-12-24 00:58:43 +01:00
Matthias Seitz
4e677b5993 refactor(net): unify dropped connection handling (#595) 2022-12-23 21:46:44 +01:00
rakita
284391c181 test(execution): chain test state check, execution bug fixes/upgrades (#472)
* temp: header stage backoff stand-in

* feat(execution): Check chain post state, fix StateProviderLatest and evm return

* Disable receipt merkle tree check

* update and merge

* Fix storage double values in dup table

* fmt

* Update bin/reth/src/test_eth_chain/runner.rs

Co-authored-by: Georgios Konstantopoulos <me@gakonst.com>

* Enable receipt root check after byzantium

* Receipt inner rlp without header for proof root

* some cleanup nits

* nit

Co-authored-by: Oliver Nordbjerg <hi@notbjerg.me>
Co-authored-by: Georgios Konstantopoulos <me@gakonst.com>
2022-12-23 18:42:19 +01:00
Bjerg
f8d1521c50 fix: set stage progress metric to absolute value (#593) 2022-12-23 18:14:53 +01:00
Bjerg
562e895f21 ci: unused dependencies sanity check (#590)
* ci: add unused deps sanity check

* ci: fix unused deps issue template title

* ci: rename udeps job

* chore: formatting

* chore: make `cargo-udeps` ignore `aquamarine`

* deps: resolve unused deps

* ci: check every feature and target in udeps job
2022-12-23 13:19:50 +01:00
Matthias Seitz
9d439d6fc8 chore(deps): bump ethers (#591) 2022-12-23 13:04:35 +01:00
Oliver Nordbjerg
2f81137d66 ci: fix sanity check issue template frontmatter 2022-12-23 12:42:49 +01:00
Oliver Nordbjerg
affead471d ci: fix sanity job 2022-12-23 12:30:06 +01:00
Bjerg
9e2ef7e48c ci: add sanity check workflow (#588)
* ci: add sanity check workflow

Adds a workflow that performs on-demand and scheduled
checks to ensure reth crates are consumeable.

* chore: adjust issue template

* ci: rename sanity check job
2022-12-23 12:27:19 +01:00
Matthias Seitz
cb58759e66 feat(net): validate discovered enr forkid (#586) 2022-12-23 12:15:07 +01:00
mempirate
e549b29cce feat(net): increase session command buffer (#585)
* refactor: unnecessary return

* feat: increase session command buffer to 32
2022-12-23 11:24:04 +01:00
Matthias Seitz
ecbbb8835c docs(txpool): clarify validate_transaction (#582) 2022-12-23 05:43:13 +01:00
Sanket Shanbhag
c9958a4d13 Remove unwind priority. (#577)
* Remove unwind_priority from QueuedStage

* Remove unnecessary test for unwind_priority

* Update pipeline.rs based on code review
2022-12-22 07:15:07 -08:00
Mariano A. Nicolini
b12939db47 feat: add stagedsync headers metrics (#498)
* add dockerfile for reth

* Add docker compose for prometheus

* Add some metrics

* Add p2p connection errors metric

* Add dependency caching for dockerfile

This reduces image build times by ~50% after the first one.

Uses cargo-chef inside the image.
More info in: https://morioh.com/p/987a2bda4526

* Add --metrics flag to docker-compose config file

* add Cargo.lock

* Move docker-compose.yml to docker directory

* Apply formatting

* Remove docker folder

* Remove .dockerignore file

* Add 'reth' prefix to metric names

* Add headers errors and request time metrics

* Modularize metrics exporter functionality and describe metrics

* Format files

* Add metrics documentation in metrics.md

* Fix metrics doc title

* Commit changes after rebase

* Solve conflict

* Modularize metrics describers

* Add stages_metrics_describer

* Rearrange header error metrics

* Add update_headers_metrics function

* Add one-line docs to describers

* Remove commented line

* Refactor metrics describer

* Update metrics doc

* Fix import

* Add header metrics struct

* add new metrics format in the headers execute method

* Add default implementation for HeaderMetrics

* Fix typo

* Fix another typo

* Fix more typos

* Move new HeaderMetrics meathod to default

* Solve conflicts

* Fix test

Co-authored-by: Tomás <tomas.gruner@lambdaclass.com>
2022-12-22 15:45:57 +01:00
Matthias Seitz
7ce22fd186 fix: SealedBlock rename (#583) 2022-12-22 15:19:40 +01:00
Satoshi Okamoto
4b18eb8467 Update the document by adding information on how to install Geth and Lighthouse. (#568)
* Update the document by adding information on how to install Geth and Lighthouse.

* Include links to the installation pages for Geth and Lighthouse in the document.

* Change the link to the installation documentation to point to the Geth installation documentation at this link: https://geth.ethereum.org/docs/install-and-build/installing-geth.

Co-authored-by: Satoshi Okamoto <satoshi@butcher73.com>
2022-12-22 14:49:48 +02:00
Genysys
0b1b5368e6 chore: Rename BlockLocked to SealedBlock (#573)
* chore: Rename BlockLocked to SealedBlock

* chore: fmt

Co-authored-by: Genysys <112424909+samtvlabs@users.noreply.github.com>
Co-authored-by: Georgios Konstantopoulos <me@gakonst.com>
2022-12-22 14:45:36 +02:00