Matilda-Clerke 657effffd4 7311 add get headers from peer task (#7781)
* 7311: spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix broken BesuCommandTest

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: add class

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Move PeerTaskFeatureToggle to more appropriate location

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: add X prefix to peertask-system-enabled

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Move --Xpeertask-system-enabled out of BesuCommand and make hidden

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Add GetReceiptsFromPeerTask

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Move isPeerTaskSystemEnabled to SynchronizerOptions

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix javadoc issue

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix javadoc issue

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Move PeerTaskFeatureToggleTestHelper to TestUtil and fix RunnerTest

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Remove PeerTaskFeatureToggle in favor of including isPeerTaskSystemEnabled in SynchronizerConfiguration

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Adjust to the removal of PeerTaskFeatureToggle and use SynchronizerConfiguration to get the toggle instead

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Reduce timeout in PeerTaskRequestSender to 5s

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Refactor PeerManager to be an interface

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix up compile errors after merge

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix MetricsAcceptanceTest

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix MetricsAcceptanceTest

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix DownloadReceiptsStep when using peer task system

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Rename PeerManager to PeerSelector

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Reword PeerSelector javadoc to avoid implementation details

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use ConcurrentHashMap in DefaultPeerSelector

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Reword trace log in DefaultPeerSelector

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Remove unused imports

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use a 1 second delay between retries in PeerTaskExecutor to match old implementation

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Add testGetPeerButNoPeerMatchesFilter to DefaultPeerSelectorTest

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Add testGetPeerButNoPeerMatchesFilter to DefaultPeerSelectorTest

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix MetricsAcceptanceTest

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix MetricsAcceptanceTest

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Modify PeerTaskExecutor metric to include response time from peer

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use SubProtocol instead of subprotocol name string in PeerTask

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: rename timing context to ignored to prevent intellij warnings

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use constants for number of retries

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Convert PeerTaskExecutorResult to a record

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Rename PeerTaskBehavior to PeerTaskRetryBehavior

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Move peer selection logic to PeerSelector

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix up everything broken after merge

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Attempt to improve performance of peer task system in pipeline

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: fix compile check

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix broken workflow

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Reduce logging in JsonRpcExecutor to trace level

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: More changes in DownloadReceiptsStep

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Rework DownloadReceiptsStep

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Make changes as discussed in walkthrough meeting

Remove DefaultPeerSelector, make EthPeers implement PeerSelector interface, and add PeerTask.getPeerRequirementFilter

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Update after merge and make discussed changes from walkthrough discussion

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Change to regular HashMap

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Remove runtime exception again

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Rename getPeerTaskBehavior to getPeerTaskRetryBehavior

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Rename getPeerTaskBehavior to getPeerTaskRetryBehavior

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Rework PeerTaskExecutor retry system to be 0-based

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix up compile errors after merge

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix broken DownloadReceiptsStepTest test

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Move GetReceipts to services worker for parallelism

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Refactor peer task system usage in DownloadReceiptsStep to better match old system

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Remove unused async methods in PeerTaskExecutor

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Return Optional<EthPeer> in PeerSelector.getPeer and utilise existing peer selection behavior in EthPeers

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Update after merge

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Redo getPeer again to include hasAvailableRequestCapacity check

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Add protocol spec supplier to GetReceiptsFromPeerTask

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Rework getPeer again to use LEAST_TO_MOST_BUSY comparator

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Import PeerNotConnected class instead of using fully qualified class name

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Change to specifying retry counts in PeerTask instead of behavior enums

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: clean up after merge

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: clean up after merge

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix up javadoc

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Add additional metrics to PeerTaskExecutor

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Add Predicate to PeerTask to check for partial success

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix incorrect name on isPartialSuccessTest

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Implement isPartialSuccess and add unit tests

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Add partialSuccessCounter and inflightRequestGauge in PeerTaskExecutor

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Also filter by whether a peer is fully validated

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Remove unneeded throws in RunnerTest

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix up inflight requests gauge in PeerTaskExecutor

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Update plugin api hash

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Update plugin api hash

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Add javadoc to LabelledGauge.isLabelsObserved

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Update plugin-api hash

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Update changelog

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Implement GetHeadersFromPeerTask and use in DetermineCommonAncestorTask

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Handle headers with no receipts as a special case in DownloadReceiptsStep

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Complete merge

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Get DetermineCommonAncestorTask working with peer task system

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use taskName instead of className for labelNames

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use snake_case for metric names

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use _total metric name suffix

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: rework partial success handling

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Update GetReceiptsFromPeerTask with partialSuccess changes

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Update GetHeadersFromPeerTask with partialSuccess changes

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Add default implementation to LabelledGauge.isLabelsObserved

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use Peer task systems GetHeadersFromPeerTask in GetBlockFromPeerTask

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix broken unit test

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Remove unused constructor from AbstractPeerBlockValidator

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use GetHeadersFromPeerTask in AbstractPeerBlockValidator

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use peer task executor in SyncTargetManager

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix javadoc on BesuControllerBuilder

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Remove logs used to confirm operation

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Implement GetHeadersFromPeerTask in FastSyncActions and PivotBlockConfirmer

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Rename parseResponse to processResponse

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Wrap peer task system usage in ethScheduler call to match other usages

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: apply spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Move check for empty trie hash into GetReceiptsFromPeerTask and update unit test to test for this functionality

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix compile issue after merge

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix compile issue after merge

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Remove BodyValidator and update code and test to match

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Implement GetHeadersForPeerTask usage in DownloadHeadersStep

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: remove unneeded logs

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix up pre-fill and add test to test failure scenario

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use ProtocolSchedule.anyMatch to find if any ProtocolSpecs are PoS, remove new usages of currentProtocolSpecSupplier

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Only attempt to remove headers on successful requests

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: clean up after merge

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: clean up after merge

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use peer task system in RangeHeadersFetcher

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use peer task system in DownloadHeaderSequenceTask

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix GetHeadersFromPeerTask mocking in CheckPointSyncChainDownloaderTest

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Extract peer task executor answer for getHeaders to separate class for reuse in tests

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Implement peer task system usage in BackwardSyncStep

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Implement peer task system usage in ChainHeadTracker

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Implement peer task system usage in PivotSelectorFromSafeBlock and improve logging

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Implement unit test for GetHeadersFromPeerTask

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix up merge compile error

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Ensure FastSyncActions and PivotSelectorFromSafeBlock retry getting headers for all peers, matching RetryingGetHeaderFromPeerByHashTask

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Change PeerTaskExecutorResult.ethPeer to an Optional

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use CancellationException instead of InterruptedException in PivotBlockConfirmer

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use PivotBlockRetriever.MAX_QUERY_RETRIES_PER_PEER to set retries for GetHeadersFromPeerTask

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Add PeerTask.shouldDisconnectPeer and ensure functionality matches old code

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Remove old info logs

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix broken test by correctly including peer in PeerTaskExecutorResults in test classes

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix incorrect equality tests

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix broken test

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Move PeerTaskExecutor into EthContext to reduce plumbing changes

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Remove protocol check from GetHeadersFromPeerTask.getPeerRequirementFilter

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix broken test

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix broken integration test

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Refactor peer task validation

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Refactor peer task validation

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use peer count for retry count when getting headers in BackwardSyncStep, FastSyncActions, and PivotSelectorFromSafeBlock

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Move chainstate update into GetHeadersFromPeerTask.postProcessResult

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix compile errors

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Update after merge

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

---------

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>
Signed-off-by: Matilda-Clerke <matilda.clerke@consensys.net>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
2024-12-11 05:03:31 +00:00
2021-09-27 09:49:34 +10:00
2024-12-03 10:51:43 +00:00
2024-10-15 12:47:41 +10:00
2024-12-09 07:33:18 +10:00
2024-10-21 09:49:06 +00:00
2024-10-15 12:47:41 +10:00
2024-10-15 12:47:41 +10:00
2022-04-08 13:16:42 +10:00
2021-02-01 16:00:16 -07:00
2024-10-31 02:54:11 +00:00
2023-10-13 15:08:04 +10:00
2024-06-05 10:21:02 +10:00
2024-10-31 08:09:22 +00:00
2024-10-31 08:09:22 +00:00

Besu Ethereum Client

CircleCI Documentation CII Best Practices License Discord Twitter Follow

Download

Besu is an Apache 2.0 licensed, MainNet compatible, Ethereum client written in Java.

Issues

Besu issues are tracked in the github issues tab. See our guidelines for more details on searching and creating issues.

If you have any questions, queries or comments, Besu channel on Discord is the place to find us.

Besu Users

To install the Besu binary, follow these instructions.

Besu Developers

Development

Instructions for how to get started with developing on the Besu codebase. Please also read the wiki for more details on how to submit a pull request (PR).

Release Notes

Release Notes

Special thanks

YourKit for providing us with a free profiler open source license.

YourKit supports open source projects with innovative and intelligent tools for monitoring and profiling Java and .NET applications. YourKit is the creator of YourKit Java Profiler, YourKit .NET Profiler, and YourKit YouMonitor.

YourKit Logo

Description
No description provided
Readme Apache-2.0 96 MiB
Languages
Java 99.7%
Shell 0.2%