Files
status-linea-besu/consensus
Matilda-Clerke 657effffd4 7311 add get headers from peer task (#7781)
* 7311: spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix broken BesuCommandTest

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: add class

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Move PeerTaskFeatureToggle to more appropriate location

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: add X prefix to peertask-system-enabled

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Move --Xpeertask-system-enabled out of BesuCommand and make hidden

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Add GetReceiptsFromPeerTask

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Move isPeerTaskSystemEnabled to SynchronizerOptions

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix javadoc issue

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix javadoc issue

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Move PeerTaskFeatureToggleTestHelper to TestUtil and fix RunnerTest

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Remove PeerTaskFeatureToggle in favor of including isPeerTaskSystemEnabled in SynchronizerConfiguration

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Adjust to the removal of PeerTaskFeatureToggle and use SynchronizerConfiguration to get the toggle instead

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Reduce timeout in PeerTaskRequestSender to 5s

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Refactor PeerManager to be an interface

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix up compile errors after merge

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix MetricsAcceptanceTest

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix MetricsAcceptanceTest

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix DownloadReceiptsStep when using peer task system

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Rename PeerManager to PeerSelector

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Reword PeerSelector javadoc to avoid implementation details

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use ConcurrentHashMap in DefaultPeerSelector

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Reword trace log in DefaultPeerSelector

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Remove unused imports

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use a 1 second delay between retries in PeerTaskExecutor to match old implementation

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Add testGetPeerButNoPeerMatchesFilter to DefaultPeerSelectorTest

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Add testGetPeerButNoPeerMatchesFilter to DefaultPeerSelectorTest

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix MetricsAcceptanceTest

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix MetricsAcceptanceTest

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Modify PeerTaskExecutor metric to include response time from peer

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use SubProtocol instead of subprotocol name string in PeerTask

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: rename timing context to ignored to prevent intellij warnings

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use constants for number of retries

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Convert PeerTaskExecutorResult to a record

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Rename PeerTaskBehavior to PeerTaskRetryBehavior

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Move peer selection logic to PeerSelector

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix up everything broken after merge

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Attempt to improve performance of peer task system in pipeline

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: fix compile check

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix broken workflow

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Reduce logging in JsonRpcExecutor to trace level

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: More changes in DownloadReceiptsStep

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Rework DownloadReceiptsStep

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Make changes as discussed in walkthrough meeting

Remove DefaultPeerSelector, make EthPeers implement PeerSelector interface, and add PeerTask.getPeerRequirementFilter

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Update after merge and make discussed changes from walkthrough discussion

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Change to regular HashMap

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Remove runtime exception again

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Rename getPeerTaskBehavior to getPeerTaskRetryBehavior

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Rename getPeerTaskBehavior to getPeerTaskRetryBehavior

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Rework PeerTaskExecutor retry system to be 0-based

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix up compile errors after merge

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix broken DownloadReceiptsStepTest test

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Move GetReceipts to services worker for parallelism

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Refactor peer task system usage in DownloadReceiptsStep to better match old system

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Remove unused async methods in PeerTaskExecutor

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Return Optional<EthPeer> in PeerSelector.getPeer and utilise existing peer selection behavior in EthPeers

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Update after merge

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Redo getPeer again to include hasAvailableRequestCapacity check

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Add protocol spec supplier to GetReceiptsFromPeerTask

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Rework getPeer again to use LEAST_TO_MOST_BUSY comparator

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Import PeerNotConnected class instead of using fully qualified class name

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Change to specifying retry counts in PeerTask instead of behavior enums

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: clean up after merge

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: clean up after merge

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix up javadoc

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Add additional metrics to PeerTaskExecutor

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Add Predicate to PeerTask to check for partial success

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix incorrect name on isPartialSuccessTest

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Implement isPartialSuccess and add unit tests

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Add partialSuccessCounter and inflightRequestGauge in PeerTaskExecutor

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Also filter by whether a peer is fully validated

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Remove unneeded throws in RunnerTest

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix up inflight requests gauge in PeerTaskExecutor

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Update plugin api hash

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Update plugin api hash

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Add javadoc to LabelledGauge.isLabelsObserved

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Update plugin-api hash

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Update changelog

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Implement GetHeadersFromPeerTask and use in DetermineCommonAncestorTask

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Handle headers with no receipts as a special case in DownloadReceiptsStep

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Complete merge

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Get DetermineCommonAncestorTask working with peer task system

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use taskName instead of className for labelNames

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use snake_case for metric names

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use _total metric name suffix

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: rework partial success handling

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Update GetReceiptsFromPeerTask with partialSuccess changes

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Update GetHeadersFromPeerTask with partialSuccess changes

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Add default implementation to LabelledGauge.isLabelsObserved

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use Peer task systems GetHeadersFromPeerTask in GetBlockFromPeerTask

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix broken unit test

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Remove unused constructor from AbstractPeerBlockValidator

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use GetHeadersFromPeerTask in AbstractPeerBlockValidator

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use peer task executor in SyncTargetManager

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix javadoc on BesuControllerBuilder

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Remove logs used to confirm operation

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Implement GetHeadersFromPeerTask in FastSyncActions and PivotBlockConfirmer

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Rename parseResponse to processResponse

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Wrap peer task system usage in ethScheduler call to match other usages

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: apply spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Move check for empty trie hash into GetReceiptsFromPeerTask and update unit test to test for this functionality

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix compile issue after merge

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix compile issue after merge

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Remove BodyValidator and update code and test to match

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Implement GetHeadersForPeerTask usage in DownloadHeadersStep

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: remove unneeded logs

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix up pre-fill and add test to test failure scenario

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use ProtocolSchedule.anyMatch to find if any ProtocolSpecs are PoS, remove new usages of currentProtocolSpecSupplier

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Only attempt to remove headers on successful requests

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: clean up after merge

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: clean up after merge

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use peer task system in RangeHeadersFetcher

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use peer task system in DownloadHeaderSequenceTask

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix GetHeadersFromPeerTask mocking in CheckPointSyncChainDownloaderTest

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Extract peer task executor answer for getHeaders to separate class for reuse in tests

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Implement peer task system usage in BackwardSyncStep

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Implement peer task system usage in ChainHeadTracker

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Implement peer task system usage in PivotSelectorFromSafeBlock and improve logging

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Implement unit test for GetHeadersFromPeerTask

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix up merge compile error

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Ensure FastSyncActions and PivotSelectorFromSafeBlock retry getting headers for all peers, matching RetryingGetHeaderFromPeerByHashTask

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Change PeerTaskExecutorResult.ethPeer to an Optional

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use CancellationException instead of InterruptedException in PivotBlockConfirmer

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use PivotBlockRetriever.MAX_QUERY_RETRIES_PER_PEER to set retries for GetHeadersFromPeerTask

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Add PeerTask.shouldDisconnectPeer and ensure functionality matches old code

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Remove old info logs

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix broken test by correctly including peer in PeerTaskExecutorResults in test classes

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix incorrect equality tests

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix broken test

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Move PeerTaskExecutor into EthContext to reduce plumbing changes

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Remove protocol check from GetHeadersFromPeerTask.getPeerRequirementFilter

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix broken test

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix broken integration test

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Refactor peer task validation

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Refactor peer task validation

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Use peer count for retry count when getting headers in BackwardSyncStep, FastSyncActions, and PivotSelectorFromSafeBlock

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: spotless

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Move chainstate update into GetHeadersFromPeerTask.postProcessResult

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Fix compile errors

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 7311: Update after merge

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

---------

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>
Signed-off-by: Matilda-Clerke <matilda.clerke@consensys.net>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
2024-12-11 05:03:31 +00:00
..
2024-11-04 09:12:00 +10:00