* 7311: Add PeerTask system for use in future PRs Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Clean up some warnings Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Add feature toggle for enabling use of the peertask system where available Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Remove log used for testing, apply spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Add private constructor to PeerTaskFeatureToggle to prevent instantiation Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Switch to logging a warning instead of throwing an exception when initializing PeerTaskFeatureToggle multiple times Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Update javadoc to match previous commit Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix broken BesuCommandTest Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: add class Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Move PeerTaskFeatureToggle to more appropriate location Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: add X prefix to peertask-system-enabled Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Move --Xpeertask-system-enabled out of BesuCommand and make hidden Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Add GetReceiptsFromPeerTask Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Move isPeerTaskSystemEnabled to SynchronizerOptions Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix javadoc issue Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix javadoc issue Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Move PeerTaskFeatureToggleTestHelper to TestUtil and fix RunnerTest Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Remove PeerTaskFeatureToggle in favor of including isPeerTaskSystemEnabled in SynchronizerConfiguration Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Adjust to the removal of PeerTaskFeatureToggle and use SynchronizerConfiguration to get the toggle instead Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Reduce timeout in PeerTaskRequestSender to 5s Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Refactor PeerManager to be an interface Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix up compile errors after merge Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix MetricsAcceptanceTest Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix MetricsAcceptanceTest Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix DownloadReceiptsStep when using peer task system Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Rename PeerManager to PeerSelector Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Reword PeerSelector javadoc to avoid implementation details Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use ConcurrentHashMap in DefaultPeerSelector Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Reword trace log in DefaultPeerSelector Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Remove unused imports Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use a 1 second delay between retries in PeerTaskExecutor to match old implementation Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Add testGetPeerButNoPeerMatchesFilter to DefaultPeerSelectorTest Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Add testGetPeerButNoPeerMatchesFilter to DefaultPeerSelectorTest Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix MetricsAcceptanceTest Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix MetricsAcceptanceTest Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Modify PeerTaskExecutor metric to include response time from peer Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use SubProtocol instead of subprotocol name string in PeerTask Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: rename timing context to ignored to prevent intellij warnings Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use constants for number of retries Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Convert PeerTaskExecutorResult to a record Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Rename PeerTaskBehavior to PeerTaskRetryBehavior Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Move peer selection logic to PeerSelector Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix up everything broken after merge Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Attempt to improve performance of peer task system in pipeline Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: fix compile check Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix broken workflow Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Reduce logging in JsonRpcExecutor to trace level Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: More changes in DownloadReceiptsStep Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Rework DownloadReceiptsStep Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Make changes as discussed in walkthrough meeting Remove DefaultPeerSelector, make EthPeers implement PeerSelector interface, and add PeerTask.getPeerRequirementFilter Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Update after merge and make discussed changes from walkthrough discussion Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Change to regular HashMap Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Remove runtime exception again Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Rename getPeerTaskBehavior to getPeerTaskRetryBehavior Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Rename getPeerTaskBehavior to getPeerTaskRetryBehavior Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Rework PeerTaskExecutor retry system to be 0-based Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix up compile errors after merge Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix broken DownloadReceiptsStepTest test Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Move GetReceipts to services worker for parallelism Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Refactor peer task system usage in DownloadReceiptsStep to better match old system Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Remove unused async methods in PeerTaskExecutor Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Return Optional<EthPeer> in PeerSelector.getPeer and utilise existing peer selection behavior in EthPeers Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Update after merge Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Redo getPeer again to include hasAvailableRequestCapacity check Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Add protocol spec supplier to GetReceiptsFromPeerTask Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Rework getPeer again to use LEAST_TO_MOST_BUSY comparator Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Import PeerNotConnected class instead of using fully qualified class name Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Change to specifying retry counts in PeerTask instead of behavior enums Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: clean up after merge Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: clean up after merge Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix up javadoc Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Add additional metrics to PeerTaskExecutor Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Add Predicate to PeerTask to check for partial success Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix incorrect name on isPartialSuccessTest Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Implement isPartialSuccess and add unit tests Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Add partialSuccessCounter and inflightRequestGauge in PeerTaskExecutor Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Also filter by whether a peer is fully validated Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Remove unneeded throws in RunnerTest Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix up inflight requests gauge in PeerTaskExecutor Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Update plugin api hash Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Update plugin api hash Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Add javadoc to LabelledGauge.isLabelsObserved Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Update plugin-api hash Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Update changelog Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Handle headers with no receipts as a special case in DownloadReceiptsStep Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Complete merge Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use taskName instead of className for labelNames Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use snake_case for metric names Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use _total metric name suffix Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: rework partial success handling Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Update GetReceiptsFromPeerTask with partialSuccess changes Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Add default implementation to LabelledGauge.isLabelsObserved Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix broken unit test Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Rename parseResponse to processResponse Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * add possibility to use the new peer task system when downloading the bodies Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net> * fix loop Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net> * 7311: Wrap peer task system usage in ethScheduler call to match other usages Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * small fixes Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net> * update API change Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net> * spotless Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net> * 7311: apply spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Move check for empty trie hash into GetReceiptsFromPeerTask and update unit test to test for this functionality Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix compile issue after merge Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Remove BodyValidator and update code and test to match Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix up pre-fill and add test to test failure scenario Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Use ProtocolSchedule.anyMatch to find if any ProtocolSpecs are PoS, remove new usages of currentProtocolSpecSupplier Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Only attempt to remove headers on successful requests Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7311: Fix broken stuff after merge Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * spotless Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net> * Fix up compile errors after merge Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * Add PeerTaskExecutor usage for GetBodies in DownloadHeaderSequenceTask Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * Add PeerTaskExecutor usage for GetBodies in ForwardSyncStep and apply spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * Allow custom retries against other peers in GetBodiesFromPeerTask Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * Fix infinite loop in CheckPointSyncChainDownloaderTest Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * spotless Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * Update CompleteBlocksWithPeerTask.getBlocks to retrieveBlocksFromPeers and add javadoc Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * Add javadoc to GetBodiesFromPeerTask Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> * 7582: Simplify withdrawals validation Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> --------- Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net> Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net> Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com> Co-authored-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
Besu Ethereum Client
Besu is an Apache 2.0 licensed, MainNet compatible, Ethereum client written in Java.
Useful Links
Issues
Besu issues are tracked in the github issues tab. See our guidelines for more details on searching and creating issues.
If you have any questions, queries or comments, Besu channel on Discord is the place to find us.
Besu Users
To install the Besu binary, follow these instructions.
Besu Developers
- Contributing Guidelines
- Coding Conventions
- Command Line Interface (CLI) Style Guide
- Besu User Documentation for running and using Besu
Development
Instructions for how to get started with developing on the Besu codebase. Please also read the wiki for more details on how to submit a pull request (PR).
Release Notes
Special thanks
YourKit for providing us with a free profiler open source license.
YourKit supports open source projects with innovative and intelligent tools for monitoring and profiling Java and .NET applications. YourKit is the creator of YourKit Java Profiler, YourKit .NET Profiler, and YourKit YouMonitor.
