Files
status-linea-besu/.gitignore
Fabio Di Fabio 423fe1d481 Layered Transaction Pool (#5290)
* Introduce experimental layered transaction pool

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* new Xlayered-tx-pool flag to enabled the new tx pool

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Move pending transaction sorter tests in the sorter folder

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Unit tests for new and old transaction pool

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix: do not decrease size when promoting ready txs

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix: remove tx from orderByFee when replaced

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix: decrease size when removing confirmed txs

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix: always recreate orderByFee for London fee market

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix: transaction removal counter when txs added to block

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix: update expected nonce when demoting a prioritized transaction

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix: correctly remove expected nonce entry when removing the last prioritized transaction for the sender
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix NullPointerException when the replaced tx is not prioritized

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Replace postponed with spare transactions

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* WIP

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* WIP

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix merge from main

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fixed most tests

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix more tests

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Rename and reorg some classes

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* More renaming and code clean up

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Refactor transaction pool metrics

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Stats log refined

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Cleanup unit tests

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Improve stats log

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Remove unnecessary test parameters

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix unit test

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

# Conflicts:
#	ethereum/core/src/main/java/org/hyperledger/besu/ethereum/core/Transaction.java
#	ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/MainnetTransactionValidator.java

* Cancel older block creation tasks upon receiving a new one

Signed-off-by: Simon Dudley <simon.dudley@consensys.net>

* Fixes to expected next nonce for sender

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix promotion filter and use synchronized methods instead of blocks

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix metrics concurrent access issue

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fixes

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Configuration options for the layered txpool

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Use long instead of Instant for PendingTransaction add time

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fixes

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Move layered txpool clasess in a dedicated package

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fixes

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* WIP

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Remove confirmed transaction from sparse set too

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* WIP

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* WIP

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fill gap on added tx

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix eviction on sparse layer

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix remove from ready layer

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix remove from sparse layer

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix for block added and confirmed txs

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fixes to sparse layer

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix the filling of the gap when adding transactions

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Layered pending transactions test and fixes

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Distinguish between layer and comulative space used

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* unit tests for expected next nonce for sender

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Adding test for transaction selection

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Re-enable prioritized transaction tests and more fixes

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* log stats

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* syncronized some methods, metrics update and dump transactions for replay

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Test that replay tx and fix for tx replacement across layers

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Add missing copyright and fix replay test

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Add consistency check asserts

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix ready internalRemove

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Metrics tests improvements

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* WIP: Transaction memory size estimation

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Complete pending transaction memory used computation

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Improve metrics

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Rename to specify that the limit is per layer

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update metric names in tests

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Adjust tx layer max capacity according to new tx memory size calculation

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix legacy transaction expiration tests

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix IndexOutOfBoundsException in sparse layer

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Unique senders metric

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Ignore ReplayTest by default, fix logging of stats

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Log for replay renamings

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Document howto generate txpool replay

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Reduce max layer capacity

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* exclude transaction replay resource

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Improve compareByFee when effectivePriorityFee is 0

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* More debug logs during transaction selection for a block

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Use only one thread for building blocks so there is no risk of overlapping

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Improve transaction trace log making wei human readable

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Use List instead of Set when getting all pending transactions

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* More detailed log on adding remote txs

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Execute transaction broadcast aysnc after adding them to the txpool

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Log time taken to add remote txs before their broadcast

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix test

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Add missing header

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix unit tests

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Add CHANGELOG entry

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Delete unneeded file

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Rename some layered txpool metrics to avoid conflict with existing metrics
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix pushing to next layers txs following an invalid one

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* In case of an unexpected error, log more data and do a consistency check

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix null check on wrong var

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix some codeql alerts

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix sparse gap calculation when invalidating

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Apply suggestions from doce review

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Only trigger consistency check if trace log is enable in case of unexpected error

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix replay of blocks with no transactions

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix for negative gap when there is a reorg

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Implement code review suggestions

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix for a case when deleting tx with zero gap in sparse layer

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Delete redoundant tests

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update CHANGELOG.md

Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/TransactionPool.java

Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/TransactionPoolMetrics.java

Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/ReadyTransactions.java

Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/layered/AbstractPrioritizedTransactionsTestBase.java

Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/SparseTransactions.java

Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update besu/src/main/java/org/hyperledger/besu/cli/options/unstable/TransactionPoolOptions.java

Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update besu/src/main/java/org/hyperledger/besu/cli/options/unstable/TransactionPoolOptions.java

Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Address code review suggestions

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Improve logSender when there are no sparse txs

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix off by one error

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/TransactionsLayer.java

Co-authored-by: Simon Dudley <simon.dudley@consensys.net>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Address code review suggestions

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Rename fix

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Simplify the way reorgs are handled, by detecting a negative gap and deleting and readding all the txs for that sender

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Do not run consistency check on internal error since it is too expensive,
instead force a reorg of the sender txs

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Remove invalid txs after the selection is complete to avoid ConcurrentModificationException

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update txpool defaults

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Tune default

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix merge

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

---------

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Co-authored-by: Simon Dudley <simon.dudley@consensys.net>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Co-authored-by: garyschulte <garyschulte@gmail.com>
2023-05-09 14:38:56 +02:00

33 lines
361 B
Plaintext

*.bak
*.swp
*.tmp
*~.nib
*.iml
*.launch
*.log
.classpath
.DS_Store
.externalToolBuilders/
.gradle/
.idea/
.loadpath
.metadata
.prefs
.project
.recommenders/
.settings
.springBeans
.vertx
bin/
local.properties
target/
tmp/
build/
out/
site/
/kubernetes/reports/
/kubernetes/besu-*.tar.gz
**/src/*/generated
jitpack.yml
/ethereum/eth/src/test/resources/tx.csv.gz