Matilda-Clerke 4ae3be55ad Add GetBodiesFromPeerTask (#8040)
* 7311: Add PeerTask system for use in future PRs

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Clean up some warnings

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Add feature toggle for enabling use of the peertask system where available

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Remove log used for testing, apply spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Add private constructor to PeerTaskFeatureToggle to prevent instantiation

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Switch to logging a warning instead of throwing an exception when initializing PeerTaskFeatureToggle multiple times

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Update javadoc to match previous commit

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix broken BesuCommandTest

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: add class

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Move PeerTaskFeatureToggle to more appropriate location

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: add X prefix to peertask-system-enabled

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Move --Xpeertask-system-enabled out of BesuCommand and make hidden

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Add GetReceiptsFromPeerTask

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Move isPeerTaskSystemEnabled to SynchronizerOptions

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix javadoc issue

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix javadoc issue

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Move PeerTaskFeatureToggleTestHelper to TestUtil and fix RunnerTest

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Remove PeerTaskFeatureToggle in favor of including isPeerTaskSystemEnabled in SynchronizerConfiguration

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Adjust to the removal of PeerTaskFeatureToggle and use SynchronizerConfiguration to get the toggle instead

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Reduce timeout in PeerTaskRequestSender to 5s

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Refactor PeerManager to be an interface

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix up compile errors after merge

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix MetricsAcceptanceTest

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix MetricsAcceptanceTest

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix DownloadReceiptsStep when using peer task system

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Rename PeerManager to PeerSelector

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Reword PeerSelector javadoc to avoid implementation details

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use ConcurrentHashMap in DefaultPeerSelector

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Reword trace log in DefaultPeerSelector

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Remove unused imports

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use a 1 second delay between retries in PeerTaskExecutor to match old implementation

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Add testGetPeerButNoPeerMatchesFilter to DefaultPeerSelectorTest

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Add testGetPeerButNoPeerMatchesFilter to DefaultPeerSelectorTest

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix MetricsAcceptanceTest

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix MetricsAcceptanceTest

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Modify PeerTaskExecutor metric to include response time from peer

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use SubProtocol instead of subprotocol name string in PeerTask

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: rename timing context to ignored to prevent intellij warnings

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use constants for number of retries

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Convert PeerTaskExecutorResult to a record

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Rename PeerTaskBehavior to PeerTaskRetryBehavior

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Move peer selection logic to PeerSelector

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix up everything broken after merge

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Attempt to improve performance of peer task system in pipeline

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: fix compile check

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix broken workflow

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Reduce logging in JsonRpcExecutor to trace level

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: More changes in DownloadReceiptsStep

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Rework DownloadReceiptsStep

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Make changes as discussed in walkthrough meeting

Remove DefaultPeerSelector, make EthPeers implement PeerSelector interface, and add PeerTask.getPeerRequirementFilter

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Update after merge and make discussed changes from walkthrough discussion

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Change to regular HashMap

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Remove runtime exception again

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Rename getPeerTaskBehavior to getPeerTaskRetryBehavior

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Rename getPeerTaskBehavior to getPeerTaskRetryBehavior

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Rework PeerTaskExecutor retry system to be 0-based

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix up compile errors after merge

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix broken DownloadReceiptsStepTest test

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Move GetReceipts to services worker for parallelism

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Refactor peer task system usage in DownloadReceiptsStep to better match old system

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Remove unused async methods in PeerTaskExecutor

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Return Optional<EthPeer> in PeerSelector.getPeer and utilise existing peer selection behavior in EthPeers

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Update after merge

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Redo getPeer again to include hasAvailableRequestCapacity check

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Add protocol spec supplier to GetReceiptsFromPeerTask

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Rework getPeer again to use LEAST_TO_MOST_BUSY comparator

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Import PeerNotConnected class instead of using fully qualified class name

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Change to specifying retry counts in PeerTask instead of behavior enums

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: clean up after merge

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: clean up after merge

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix up javadoc

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Add additional metrics to PeerTaskExecutor

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Add Predicate to PeerTask to check for partial success

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix incorrect name on isPartialSuccessTest

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Implement isPartialSuccess and add unit tests

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Add partialSuccessCounter and inflightRequestGauge in PeerTaskExecutor

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Also filter by whether a peer is fully validated

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Remove unneeded throws in RunnerTest

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix up inflight requests gauge in PeerTaskExecutor

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Update plugin api hash

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Update plugin api hash

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Add javadoc to LabelledGauge.isLabelsObserved

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Update plugin-api hash

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Update changelog

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Handle headers with no receipts as a special case in DownloadReceiptsStep

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Complete merge

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use taskName instead of className for labelNames

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use snake_case for metric names

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use _total metric name suffix

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: rework partial success handling

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Update GetReceiptsFromPeerTask with partialSuccess changes

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Add default implementation to LabelledGauge.isLabelsObserved

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix broken unit test

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Rename parseResponse to processResponse

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* add possibility to use the new peer task system when downloading the bodies

Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>

* fix loop

Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>

* 7311: Wrap peer task system usage in ethScheduler call to match other usages

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* small fixes

Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>

* update API change

Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>

* spotless

Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>

* 7311: apply spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Move check for empty trie hash into GetReceiptsFromPeerTask and update unit test to test for this functionality

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix compile issue after merge

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Remove BodyValidator and update code and test to match

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix up pre-fill and add test to test failure scenario

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use ProtocolSchedule.anyMatch to find if any ProtocolSpecs are PoS, remove new usages of currentProtocolSpecSupplier

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Only attempt to remove headers on successful requests

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix broken stuff after merge

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* spotless

Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>

* Fix up compile errors after merge

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* Add PeerTaskExecutor usage for GetBodies in DownloadHeaderSequenceTask

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* Add PeerTaskExecutor usage for GetBodies in ForwardSyncStep and apply spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* Allow custom retries against other peers in GetBodiesFromPeerTask

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* Fix infinite loop in CheckPointSyncChainDownloaderTest

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* Update CompleteBlocksWithPeerTask.getBlocks to retrieveBlocksFromPeers and add javadoc

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* Add javadoc to GetBodiesFromPeerTask

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7582: Simplify withdrawals validation

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

---------

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>
Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Co-authored-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
2025-01-09 00:06:06 +00:00
2025-01-07 16:25:02 +10:00
2025-01-07 01:19:26 +00:00
2025-01-07 01:19:26 +00:00
2024-10-15 12:47:41 +10:00
2024-10-21 09:49:06 +00:00
2025-01-07 01:19:26 +00:00
2024-10-15 12:47:41 +10:00
2024-10-15 12:47:41 +10:00
2022-04-08 13:16:42 +10:00
2021-02-01 16:00:16 -07:00
2024-10-31 02:54:11 +00:00
2023-10-13 15:08:04 +10:00
2024-06-05 10:21:02 +10:00
2024-10-31 08:09:22 +00:00
2024-10-31 08:09:22 +00:00

Besu Ethereum Client

CircleCI Documentation CII Best Practices License Discord Twitter Follow

Download

Besu is an Apache 2.0 licensed, MainNet compatible, Ethereum client written in Java.

Issues

Besu issues are tracked in the github issues tab. See our guidelines for more details on searching and creating issues.

If you have any questions, queries or comments, Besu channel on Discord is the place to find us.

Besu Users

To install the Besu binary, follow these instructions.

Besu Developers

Development

Instructions for how to get started with developing on the Besu codebase. Please also read the wiki for more details on how to submit a pull request (PR).

Release Notes

Release Notes

Special thanks

YourKit for providing us with a free profiler open source license.

YourKit supports open source projects with innovative and intelligent tools for monitoring and profiling Java and .NET applications. YourKit is the creator of YourKit Java Profiler, YourKit .NET Profiler, and YourKit YouMonitor.

YourKit Logo

Description
No description provided
Readme Apache-2.0 96 MiB
Languages
Java 99.7%
Shell 0.2%