Merge branch 'main' into zkbesu

CHANGELOG.md

@@ -1,35 +1,56 @@
 # Changelog
 
-## 24.1.1-SNAPSHOT
+## 24.1.2-SNAPSHOT
+
+### Breaking Changes
+
+### Deprecations
+
+### Additions and Improvements
+
+### Bug fixes
+- Fix the way an advertised host configured with `--p2p-host` is treated when communicating with the originator of a PING packet [#6225](https://github.com/hyperledger/besu/pull/6225)
+
+### Download Links
+
+## 24.1.1
 
 ### Breaking Changes
 - New `EXECUTION_HALTED` error returned if there is an error executing or simulating a transaction, with the reason for execution being halted. Replaces the generic `INTERNAL_ERROR` return code in certain cases which some applications may be checking for [#6343](https://github.com/hyperledger/besu/pull/6343)
 - The Besu Docker images with `openjdk-latest` tags since 23.10.3 were incorrectly using UID 1001 instead of 1000 for the container's `besu` user. The user now uses 1000 again. Containers created from or migrated to images using UID 1001 will need to chown their persistent database files to UID 1000 [#6360](https://github.com/hyperledger/besu/pull/6360)
+- The deprecated `--privacy-onchain-groups-enabled` option has now been removed. Use the `--privacy-flexible-groups-enabled` option instead. [#6411](https://github.com/hyperledger/besu/pull/6411)
+- Requesting the Ethereum Node Record (ENR) to acquire the fork id from bonded peers is now enabled by default, so the following change has been made [#5628](https://github.com/hyperledger/besu/pull/5628):
+  - `--Xfilter-on-enr-fork-id` has been removed. To disable the feature use `--filter-on-enr-fork-id=false`. 
+- The time that can be spent selecting transactions during block creation is not capped at 5 seconds for PoS and PoW networks, and for PoA networks, at 75% of the block period specified in the genesis, this to prevent possible DoS in case a single transaction is taking too long to execute, and to have a stable block production rate, but it could be a breaking change if an existing network used to have transactions that takes more time to executed that the newly introduced limit, if it is mandatory for these network to keep processing these long processing transaction, then the default value of `block-txs-selection-max-time` or `poa-block-txs-selection-max-time` needs to be tuned accordingly.
 
 ### Deprecations
 
 ### Additions and Improvements
 - Optimize RocksDB WAL files, allows for faster restart and a more linear disk space utilization [#6328](https://github.com/hyperledger/besu/pull/6328)
 - Disable transaction handling when the node is not in sync, to avoid unnecessary transaction validation work [#6302](https://github.com/hyperledger/besu/pull/6302)
-- Introduce TransactionEvaluationContext to pass data between transaction selectors and plugin, during block creation [#6381](https://github.com/hyperledger/besu/pull/6381) 
+- Introduce TransactionEvaluationContext to pass data between transaction selectors and plugin, during block creation [#6381](https://github.com/hyperledger/besu/pull/6381)
 - Upgrade dependencies [#6377](https://github.com/hyperledger/besu/pull/6377)
-- Upgrade `com.fasterxml.jackson` dependencies [#6378](https://github.com/hyperledger/besu/pull/6378) 
+- Upgrade `com.fasterxml.jackson` dependencies [#6378](https://github.com/hyperledger/besu/pull/6378)
+- Upgrade Guava dependency [#6396](https://github.com/hyperledger/besu/pull/6396)
+- Upgrade Mockito [#6397](https://github.com/hyperledger/besu/pull/6397)
+- Upgrade `tech.pegasys.discovery:discovery` [#6414](https://github.com/hyperledger/besu/pull/6414)
+- Options to tune the max allowed time that can be spent selecting transactions during block creation are now stable [#6423](https://github.com/hyperledger/besu/pull/6423)
 
 ### Bug fixes
 - INTERNAL_ERROR from `eth_estimateGas` JSON/RPC calls [#6344](https://github.com/hyperledger/besu/issues/6344)
 - Fix Besu Docker images with `openjdk-latest` tags since 23.10.3 using UID 1001 instead of 1000 for the `besu` user [#6360](https://github.com/hyperledger/besu/pull/6360)
 - Fluent EVM API definition for Tangerine Whistle had incorrect code size validation configured [#6382](https://github.com/hyperledger/besu/pull/6382)
 - Correct mining beneficiary for Clique networks in TraceServiceImpl [#6390](https://github.com/hyperledger/besu/pull/6390)
+- Fix to gas limit delta calculations used in block production. Besu should now increment or decrement the block gas limit towards its target correctly (thanks @arbora) #6425
 
 ### Download Links
 
-
 ## 24.1.0
 
 ### Breaking Changes
 
 ### Deprecations
-- Forest pruning (`pruning-enabled` options) is deprecated and will be removed soon. To save disk space consider switching to Bonsai data storage format [#6230](https://github.com/hyperledger/besu/pull/6230)
+- Forest pruning (`pruning-enabled` option) is deprecated and will be removed soon. To save disk space consider switching to Bonsai data storage format [#6230](https://github.com/hyperledger/besu/pull/6230)
 
 ### Additions and Improvements
 - Add error messages on authentication failures with username and password [#6212](https://github.com/hyperledger/besu/pull/6212)

SUPPORT.md

@@ -7,7 +7,7 @@ Welcome to the Besu repository! The following links are a set of guidelines for
 Having Github, Discord, and Linux Foundation accounts is necessary for obtaining support for Besu through the community channels, wiki and issue management.  
 * If you want to raise an issue, you can do so [on the github issue tab](https://github.com/hyperledger/besu/issues).   
 * Hyperledger Discord requires a [Discord account].
-* The Hyperlegder wiki also requires a [Linux Foundation (LF) account] in order to edit pages.
+* The Hyperledger wiki also requires a [Linux Foundation (LF) account] in order to edit pages.
 
 ### Useful support links
 

acceptance-tests/tests/src/test/resources/jsonrpc/engine/cancun/test-cases/block-production/12_cancun_get_built_block.json

@@ -10,7 +10,7 @@
         "stateRoot" : "0x8d9115d9211932d4a3a1f068fb8fe262b0b2ab0bfd74eaece1a572efe6336677",
         "logsBloom" : "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
         "prevRandao" : "0xc13da06dc53836ca0766057413b9683eb9a8773bbb8fcc5691e41c25b56dda1d",
-        "gasLimit" : "0x2ff3d8",
+        "gasLimit" : "0x2ffbd2",
         "gasUsed" : "0xf618",
         "timestamp" : "0x1236",
         "extraData" : "0x",
@@ -70,7 +70,7 @@
           "amount" : "0x64"
         } ],
         "blockNumber" : "0x1",
-        "blockHash" : "0xf1e35607932349e87f29e1053a4fb2666782e09fde21ded74c1f7e4a57d3fa2b",
+        "blockHash" : "0x736bdddc2eca36fe8ed4ed515e5d295a08d7eaddc0d0fda2a35408127eb890d0",
         "receiptsRoot" : "0x9af165447e5b3193e9ac8389418648ee6d6cb1d37459fe65cfc245fc358721bd",
         "blobGasUsed" : "0x60000"
       },

besu/src/main/java/org/hyperledger/besu/cli/BesuCommand.java

@@ -22,7 +22,6 @@ import static java.util.Collections.singletonList;
 import static org.hyperledger.besu.cli.DefaultCommandValues.getDefaultBesuDataPath;
 import static org.hyperledger.besu.cli.config.NetworkName.MAINNET;
 import static org.hyperledger.besu.cli.util.CommandLineUtils.DEPENDENCY_WARNING_MSG;
-import static org.hyperledger.besu.cli.util.CommandLineUtils.DEPRECATION_WARNING_MSG;
 import static org.hyperledger.besu.cli.util.CommandLineUtils.isOptionSet;
 import static org.hyperledger.besu.controller.BesuController.DATABASE_PATH;
 import static org.hyperledger.besu.ethereum.api.graphql.GraphQLConfiguration.DEFAULT_GRAPHQL_HTTP_PORT;
@@ -148,6 +147,7 @@ import org.hyperledger.besu.ethereum.storage.keyvalue.KeyValueSegmentIdentifier;
 import org.hyperledger.besu.ethereum.storage.keyvalue.KeyValueStorageProvider;
 import org.hyperledger.besu.ethereum.storage.keyvalue.KeyValueStorageProviderBuilder;
 import org.hyperledger.besu.ethereum.trie.forest.pruner.PrunerConfiguration;
+import org.hyperledger.besu.ethereum.worldstate.DataStorageFormat;
 import org.hyperledger.besu.evm.precompile.AbstractAltBnPrecompiledContract;
 import org.hyperledger.besu.evm.precompile.BigIntegerModularExponentiationPrecompiledContract;
 import org.hyperledger.besu.evm.precompile.KZGPointEvalPrecompiledContract;
@@ -953,13 +953,6 @@ public class BesuCommand implements DefaultCommandValues, Runnable {
         names = {"--privacy-flexible-groups-enabled"},
         description = "Enable flexible privacy groups (default: ${DEFAULT-VALUE})")
     private final Boolean isFlexiblePrivacyGroupsEnabled = false;
-
-    @Option(
-        hidden = true,
-        names = {"--privacy-onchain-groups-enabled"},
-        description =
-            "!!DEPRECATED!! Use `--privacy-flexible-groups-enabled` instead. Enable flexible (onchain) privacy groups (default: ${DEFAULT-VALUE})")
-    private final Boolean isOnchainPrivacyGroupsEnabled = false;
   }
 
   // Metrics Option Group
@@ -1716,8 +1709,7 @@ public class BesuCommand implements DefaultCommandValues, Runnable {
       }
 
       if (unstablePrivacyPluginOptions.isPrivacyPluginEnabled()
-          && (privacyOptionGroup.isFlexiblePrivacyGroupsEnabled
+          && privacyOptionGroup.isFlexiblePrivacyGroupsEnabled) {
-              || privacyOptionGroup.isOnchainPrivacyGroupsEnabled)) {
         throw new ParameterException(
             commandLine, "Privacy Plugin can not be used with flexible privacy groups");
       }
@@ -2056,16 +2048,16 @@ public class BesuCommand implements DefaultCommandValues, Runnable {
           "--security-module=" + DEFAULT_SECURITY_MODULE);
     }
 
-    if (Boolean.TRUE.equals(privacyOptionGroup.isOnchainPrivacyGroupsEnabled)) {
-      logger.warn(
-          DEPRECATION_WARNING_MSG,
-          "--privacy-onchain-groups-enabled",
-          "--privacy-flexible-groups-enabled");
-    }
-
     if (isPruningEnabled()) {
-      logger.warn(
+      if (dataStorageOptions
-          "Forest pruning is deprecated and will be removed soon. To save disk space consider switching to Bonsai data storage format.");
+          .toDomainObject()
+          .getDataStorageFormat()
+          .equals(DataStorageFormat.BONSAI)) {
+        logger.warn("Forest pruning is ignored with Bonsai data storage format.");
+      } else {
+        logger.warn(
+            "Forest pruning is deprecated and will be removed soon. To save disk space consider switching to Bonsai data storage format.");
+      }
     }
   }
 
@@ -2743,8 +2735,7 @@ public class BesuCommand implements DefaultCommandValues, Runnable {
       privacyParametersBuilder.setMultiTenancyEnabled(
           privacyOptionGroup.isPrivacyMultiTenancyEnabled);
       privacyParametersBuilder.setFlexiblePrivacyGroupsEnabled(
-          privacyOptionGroup.isFlexiblePrivacyGroupsEnabled
+          privacyOptionGroup.isFlexiblePrivacyGroupsEnabled);
-              || privacyOptionGroup.isOnchainPrivacyGroupsEnabled);
       privacyParametersBuilder.setPrivacyPluginEnabled(
           unstablePrivacyPluginOptions.isPrivacyPluginEnabled());
 
@@ -2917,17 +2908,15 @@ public class BesuCommand implements DefaultCommandValues, Runnable {
           ImmutableMiningParameters.builder().from(miningOptions.toDomainObject());
       final var actualGenesisOptions = getActualGenesisConfigOptions();
       if (actualGenesisOptions.isPoa()) {
-        miningParametersBuilder.unstable(
+        miningParametersBuilder.genesisBlockPeriodSeconds(
-            ImmutableMiningParameters.Unstable.builder()
+            getGenesisBlockPeriodSeconds(actualGenesisOptions));
-                .minBlockTime(getMinBlockTime(actualGenesisOptions))
-                .build());
       }
       miningParameters = miningParametersBuilder.build();
     }
     return miningParameters;
   }
 
-  private int getMinBlockTime(final GenesisConfigOptions genesisConfigOptions) {
+  private int getGenesisBlockPeriodSeconds(final GenesisConfigOptions genesisConfigOptions) {
     if (genesisConfigOptions.isClique()) {
       return genesisConfigOptions.getCliqueConfigOptions().getBlockPeriodSeconds();
     }

besu/src/main/java/org/hyperledger/besu/cli/converter/PositiveNumberConverter.java

@@ -0,0 +1,33 @@
+/*
+ * Copyright Hyperledger Besu Contributors.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+ * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations under the License.
+ *
+ * SPDX-License-Identifier: Apache-2.0
+ */
+package org.hyperledger.besu.cli.converter;
+
+import org.hyperledger.besu.cli.converter.exception.PercentageConversionException;
+import org.hyperledger.besu.util.number.PositiveNumber;
+
+import picocli.CommandLine;
+
+/** The PositiveNumber Cli type converter. */
+public class PositiveNumberConverter implements CommandLine.ITypeConverter<PositiveNumber> {
+
+  @Override
+  public PositiveNumber convert(final String value) throws PercentageConversionException {
+    try {
+      return PositiveNumber.fromString(value);
+    } catch (NullPointerException | IllegalArgumentException e) {
+      throw new PercentageConversionException(value);
+    }
+  }
+}

besu/src/main/java/org/hyperledger/besu/cli/converter/exception/PositiveNumberConversionException.java

@@ -0,0 +1,30 @@
+/*
+ * Copyright Hyperledger Besu Contributors.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+ * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations under the License.
+ *
+ * SPDX-License-Identifier: Apache-2.0
+ */
+package org.hyperledger.besu.cli.converter.exception;
+
+import static java.lang.String.format;
+
+/** The custom PositiveNumber conversion exception. */
+public final class PositiveNumberConversionException extends Exception {
+
+  /**
+   * Instantiates a new PositiveNumber conversion exception.
+   *
+   * @param value the invalid value to add in exception message
+   */
+  public PositiveNumberConversionException(final String value) {
+    super(format("Invalid value: %s, should be a positive number >0.", value));
+  }
+}

besu/src/main/java/org/hyperledger/besu/cli/options/MiningOptions.java

@@ -16,20 +16,20 @@ package org.hyperledger.besu.cli.options;
 
 import static java.util.Arrays.asList;
 import static java.util.Collections.singletonList;
+import static org.hyperledger.besu.ethereum.core.MiningParameters.DEFAULT_NON_POA_BLOCK_TXS_SELECTION_MAX_TIME;
+import static org.hyperledger.besu.ethereum.core.MiningParameters.DEFAULT_POA_BLOCK_TXS_SELECTION_MAX_TIME;
 import static org.hyperledger.besu.ethereum.core.MiningParameters.MutableInitValues.DEFAULT_EXTRA_DATA;
 import static org.hyperledger.besu.ethereum.core.MiningParameters.MutableInitValues.DEFAULT_MIN_BLOCK_OCCUPANCY_RATIO;
 import static org.hyperledger.besu.ethereum.core.MiningParameters.MutableInitValues.DEFAULT_MIN_PRIORITY_FEE_PER_GAS;
 import static org.hyperledger.besu.ethereum.core.MiningParameters.MutableInitValues.DEFAULT_MIN_TRANSACTION_GAS_PRICE;
 import static org.hyperledger.besu.ethereum.core.MiningParameters.Unstable.DEFAULT_MAX_OMMERS_DEPTH;
-import static org.hyperledger.besu.ethereum.core.MiningParameters.Unstable.DEFAULT_NON_POA_BLOCK_TXS_SELECTION_MAX_TIME;
-import static org.hyperledger.besu.ethereum.core.MiningParameters.Unstable.DEFAULT_POA_BLOCK_TXS_SELECTION_MAX_TIME;
 import static org.hyperledger.besu.ethereum.core.MiningParameters.Unstable.DEFAULT_POS_BLOCK_CREATION_MAX_TIME;
 import static org.hyperledger.besu.ethereum.core.MiningParameters.Unstable.DEFAULT_POS_BLOCK_CREATION_REPETITION_MIN_DURATION;
 import static org.hyperledger.besu.ethereum.core.MiningParameters.Unstable.DEFAULT_POW_JOB_TTL;
 import static org.hyperledger.besu.ethereum.core.MiningParameters.Unstable.DEFAULT_REMOTE_SEALERS_LIMIT;
 import static org.hyperledger.besu.ethereum.core.MiningParameters.Unstable.DEFAULT_REMOTE_SEALERS_TTL;
 
-import org.hyperledger.besu.cli.converter.PercentageConverter;
+import org.hyperledger.besu.cli.converter.PositiveNumberConverter;
 import org.hyperledger.besu.cli.util.CommandLineUtils;
 import org.hyperledger.besu.config.GenesisConfigOptions;
 import org.hyperledger.besu.datatypes.Address;
@@ -37,7 +37,7 @@ import org.hyperledger.besu.datatypes.Wei;
 import org.hyperledger.besu.ethereum.core.ImmutableMiningParameters;
 import org.hyperledger.besu.ethereum.core.ImmutableMiningParameters.MutableInitValues;
 import org.hyperledger.besu.ethereum.core.MiningParameters;
-import org.hyperledger.besu.util.number.Percentage;
+import org.hyperledger.besu.util.number.PositiveNumber;
 
 import java.util.List;
 
@@ -115,6 +115,24 @@ public class MiningOptions implements CLIOptions<MiningParameters> {
               + " If set, each block's gas limit will approach this setting over time.")
   private Long targetGasLimit = null;
 
+  @Option(
+      names = {"--block-txs-selection-max-time"},
+      converter = PositiveNumberConverter.class,
+      description =
+          "Specifies the maximum time, in milliseconds, that could be spent selecting transactions to be included in the block."
+              + " Not compatible with PoA networks, see poa-block-txs-selection-max-time. (default: ${DEFAULT-VALUE})")
+  private PositiveNumber nonPoaBlockTxsSelectionMaxTime =
+      DEFAULT_NON_POA_BLOCK_TXS_SELECTION_MAX_TIME;
+
+  @Option(
+      names = {"--poa-block-txs-selection-max-time"},
+      converter = PositiveNumberConverter.class,
+      description =
+          "Specifies the maximum time that could be spent selecting transactions to be included in the block, as a percentage of the fixed block time of the PoA network."
+              + " To be only used on PoA networks, for other networks see block-txs-selection-max-time."
+              + " (default: ${DEFAULT-VALUE})")
+  private PositiveNumber poaBlockTxsSelectionMaxTime = DEFAULT_POA_BLOCK_TXS_SELECTION_MAX_TIME;
+
   @CommandLine.ArgGroup(validate = false)
   private final Unstable unstableOptions = new Unstable();
 
@@ -168,25 +186,6 @@ public class MiningOptions implements CLIOptions<MiningParameters> {
                 + " then it waits before next repetition. Must be positive and ≤ 2000 (default: ${DEFAULT-VALUE} milliseconds)")
     private Long posBlockCreationRepetitionMinDuration =
         DEFAULT_POS_BLOCK_CREATION_REPETITION_MIN_DURATION;
-
-    @CommandLine.Option(
-        hidden = true,
-        names = {"--Xblock-txs-selection-max-time"},
-        description =
-            "Specifies the maximum time, in milliseconds, that could be spent selecting transactions to be included in the block."
-                + " Not compatible with PoA networks, see Xpoa-block-txs-selection-max-time."
-                + " Must be positive and ≤ (default: ${DEFAULT-VALUE})")
-    private Long nonPoaBlockTxsSelectionMaxTime = DEFAULT_NON_POA_BLOCK_TXS_SELECTION_MAX_TIME;
-
-    @CommandLine.Option(
-        hidden = true,
-        names = {"--Xpoa-block-txs-selection-max-time"},
-        converter = PercentageConverter.class,
-        description =
-            "Specifies the maximum time that could be spent selecting transactions to be included in the block, as a percentage of the fixed block time of the PoA network."
-                + " To be only used on PoA networks, for other networks see Xblock-txs-selection-max-time."
-                + " (default: ${DEFAULT-VALUE})")
-    private Percentage poaBlockTxsSelectionMaxTime = DEFAULT_POA_BLOCK_TXS_SELECTION_MAX_TIME;
   }
 
   private MiningOptions() {}
@@ -270,26 +269,17 @@ public class MiningOptions implements CLIOptions<MiningParameters> {
     if (genesisConfigOptions.isPoa()) {
       CommandLineUtils.failIfOptionDoesntMeetRequirement(
           commandLine,
-          "--Xblock-txs-selection-max-time can't be used with PoA networks,"
+          "--block-txs-selection-max-time can't be used with PoA networks,"
-              + " see Xpoa-block-txs-selection-max-time instead",
+              + " see poa-block-txs-selection-max-time instead",
           false,
-          singletonList("--Xblock-txs-selection-max-time"));
+          singletonList("--block-txs-selection-max-time"));
     } else {
       CommandLineUtils.failIfOptionDoesntMeetRequirement(
           commandLine,
-          "--Xpoa-block-txs-selection-max-time can be only used with PoA networks,"
+          "--poa-block-txs-selection-max-time can be only used with PoA networks,"
-              + " see --Xblock-txs-selection-max-time instead",
+              + " see --block-txs-selection-max-time instead",
           false,
-          singletonList("--Xpoa-block-txs-selection-max-time"));
+          singletonList("--poa-block-txs-selection-max-time"));
-
-      if (unstableOptions.nonPoaBlockTxsSelectionMaxTime <= 0
-          || unstableOptions.nonPoaBlockTxsSelectionMaxTime
-              > DEFAULT_NON_POA_BLOCK_TXS_SELECTION_MAX_TIME) {
-        throw new ParameterException(
-            commandLine,
-            "--Xblock-txs-selection-max-time must be positive and ≤ "
-                + DEFAULT_NON_POA_BLOCK_TXS_SELECTION_MAX_TIME);
-      }
     }
   }
 
@@ -303,6 +293,10 @@ public class MiningOptions implements CLIOptions<MiningParameters> {
     miningOptions.minTransactionGasPrice = miningParameters.getMinTransactionGasPrice();
     miningOptions.minPriorityFeePerGas = miningParameters.getMinPriorityFeePerGas();
     miningOptions.minBlockOccupancyRatio = miningParameters.getMinBlockOccupancyRatio();
+    miningOptions.nonPoaBlockTxsSelectionMaxTime =
+        miningParameters.getNonPoaBlockTxsSelectionMaxTime();
+    miningOptions.poaBlockTxsSelectionMaxTime = miningParameters.getPoaBlockTxsSelectionMaxTime();
+
     miningOptions.unstableOptions.remoteSealersLimit =
         miningParameters.getUnstable().getRemoteSealersLimit();
     miningOptions.unstableOptions.remoteSealersTimeToLive =
@@ -317,10 +311,6 @@ public class MiningOptions implements CLIOptions<MiningParameters> {
         miningParameters.getUnstable().getPosBlockCreationMaxTime();
     miningOptions.unstableOptions.posBlockCreationRepetitionMinDuration =
         miningParameters.getUnstable().getPosBlockCreationRepetitionMinDuration();
-    miningOptions.unstableOptions.nonPoaBlockTxsSelectionMaxTime =
-        miningParameters.getUnstable().getBlockTxsSelectionMaxTime();
-    miningOptions.unstableOptions.poaBlockTxsSelectionMaxTime =
-        miningParameters.getUnstable().getPoaBlockTxsSelectionMaxTime();
 
     miningParameters.getCoinbase().ifPresent(coinbase -> miningOptions.coinbase = coinbase);
     miningParameters.getTargetGasLimit().ifPresent(tgl -> miningOptions.targetGasLimit = tgl);
@@ -350,6 +340,8 @@ public class MiningOptions implements CLIOptions<MiningParameters> {
             .isStratumMiningEnabled(iStratumMiningEnabled)
             .stratumNetworkInterface(stratumNetworkInterface)
             .stratumPort(stratumPort)
+            .nonPoaBlockTxsSelectionMaxTime(nonPoaBlockTxsSelectionMaxTime)
+            .poaBlockTxsSelectionMaxTime(poaBlockTxsSelectionMaxTime)
             .unstable(
                 ImmutableMiningParameters.Unstable.builder()
                     .remoteSealersLimit(unstableOptions.remoteSealersLimit)
@@ -360,8 +352,6 @@ public class MiningOptions implements CLIOptions<MiningParameters> {
                     .posBlockCreationMaxTime(unstableOptions.posBlockCreationMaxTime)
                     .posBlockCreationRepetitionMinDuration(
                         unstableOptions.posBlockCreationRepetitionMinDuration)
-                    .nonPoaBlockTxsSelectionMaxTime(unstableOptions.nonPoaBlockTxsSelectionMaxTime)
-                    .poaBlockTxsSelectionMaxTime(unstableOptions.poaBlockTxsSelectionMaxTime)
                     .build());
 
     return miningParametersBuilder.build();

besu/src/main/java/org/hyperledger/besu/cli/options/stable/DataStorageOptions.java

@@ -62,23 +62,28 @@ public class DataStorageOptions implements CLIOptions<DataStorageConfiguration>
   private final DataStorageOptions.Unstable unstableOptions = new Unstable();
 
   static class Unstable {
+    private static final String BONSAI_LIMIT_TRIE_LOGS_ENABLED =
+        "--Xbonsai-limit-trie-logs-enabled";
+    private static final String BONSAI_TRIE_LOGS_RETENTION_THRESHOLD =
+        "--Xbonsai-trie-logs-retention-threshold";
+    private static final String BONSAI_TRIE_LOG_PRUNING_LIMIT = "--Xbonsai-trie-logs-pruning-limit";
 
     @CommandLine.Option(
         hidden = true,
-        names = {"--Xbonsai-trie-log-pruning-enabled"},
+        names = {BONSAI_LIMIT_TRIE_LOGS_ENABLED},
         description = "Enable trie log pruning. (default: ${DEFAULT-VALUE})")
     private boolean bonsaiTrieLogPruningEnabled = DEFAULT_BONSAI_TRIE_LOG_PRUNING_ENABLED;
 
     @CommandLine.Option(
         hidden = true,
-        names = {"--Xbonsai-trie-log-retention-threshold"},
+        names = {BONSAI_TRIE_LOGS_RETENTION_THRESHOLD},
         description =
             "The number of blocks for which to retain trie logs. (default: ${DEFAULT-VALUE})")
     private long bonsaiTrieLogRetentionThreshold = DEFAULT_BONSAI_TRIE_LOG_RETENTION_THRESHOLD;
 
     @CommandLine.Option(
         hidden = true,
-        names = {"--Xbonsai-trie-log-pruning-limit"},
+        names = {BONSAI_TRIE_LOG_PRUNING_LIMIT},
         description =
             "The max number of blocks to load and prune trie logs for at startup. (default: ${DEFAULT-VALUE})")
     private int bonsaiTrieLogPruningLimit = DEFAULT_BONSAI_TRIE_LOG_PRUNING_LIMIT;

besu/src/main/java/org/hyperledger/besu/cli/options/unstable/NetworkingOptions.java

@@ -37,7 +37,7 @@ public class NetworkingOptions implements CLIOptions<NetworkingConfiguration> {
   private final String DNS_DISCOVERY_SERVER_OVERRIDE_FLAG = "--Xp2p-dns-discovery-server";
   private final String DISCOVERY_PROTOCOL_V5_ENABLED = "--Xv5-discovery-enabled";
   /** The constant FILTER_ON_ENR_FORK_ID. */
-  public static final String FILTER_ON_ENR_FORK_ID = "--Xfilter-on-enr-fork-id";
+  public static final String FILTER_ON_ENR_FORK_ID = "--filter-on-enr-fork-id";
 
   @CommandLine.Option(
       names = INITIATE_CONNECTIONS_FREQUENCY_FLAG,
@@ -76,9 +76,9 @@ public class NetworkingOptions implements CLIOptions<NetworkingConfiguration> {
   @CommandLine.Option(
       names = FILTER_ON_ENR_FORK_ID,
       hidden = true,
-      defaultValue = "false",
+      defaultValue = "true",
       description = "Whether to enable filtering of peers based on the ENR field ForkId)")
-  private final Boolean filterOnEnrForkId = false;
+  private final Boolean filterOnEnrForkId = NetworkingConfiguration.DEFAULT_FILTER_ON_ENR_FORK_ID;
 
   @CommandLine.Option(
       hidden = true,

besu/src/main/java/org/hyperledger/besu/cli/subcommands/storage/TrieLogHelper.java

@@ -22,7 +22,11 @@ import org.hyperledger.besu.datatypes.Hash;
 import org.hyperledger.besu.ethereum.chain.Blockchain;
 import org.hyperledger.besu.ethereum.chain.MutableBlockchain;
 import org.hyperledger.besu.ethereum.core.BlockHeader;
+import org.hyperledger.besu.ethereum.rlp.BytesValueRLPInput;
+import org.hyperledger.besu.ethereum.rlp.RLP;
 import org.hyperledger.besu.ethereum.trie.bonsai.storage.BonsaiWorldStateKeyValueStorage;
+import org.hyperledger.besu.ethereum.trie.bonsai.trielog.TrieLogFactoryImpl;
+import org.hyperledger.besu.ethereum.trie.bonsai.trielog.TrieLogLayer;
 import org.hyperledger.besu.ethereum.worldstate.DataStorageConfiguration;
 
 import java.io.File;
@@ -32,6 +36,7 @@ import java.io.IOException;
 import java.io.ObjectInputStream;
 import java.io.ObjectOutputStream;
 import java.io.PrintWriter;
+import java.nio.file.Files;
 import java.nio.file.Path;
 import java.util.ArrayList;
 import java.util.IdentityHashMap;
@@ -39,6 +44,7 @@ import java.util.List;
 import java.util.Optional;
 import java.util.concurrent.atomic.AtomicInteger;
 
+import org.apache.tuweni.bytes.Bytes;
 import org.apache.tuweni.bytes.Bytes32;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -97,16 +103,15 @@ public class TrieLogHelper {
       final String batchFileNameBase) {
 
     for (long batchNumber = 1; batchNumber <= numberOfBatches; batchNumber++) {
-
+      final String batchFileName = batchFileNameBase + "-" + batchNumber;
       final long firstBlockOfBatch = chainHeight - ((batchNumber - 1) * BATCH_SIZE);
-
       final long lastBlockOfBatch =
           Math.max(chainHeight - (batchNumber * BATCH_SIZE), lastBlockNumberToRetainTrieLogsFor);
-
       final List<Hash> trieLogKeys =
           getTrieLogKeysForBlocks(blockchain, firstBlockOfBatch, lastBlockOfBatch);
 
-      saveTrieLogBatches(batchFileNameBase, rootWorldStateStorage, batchNumber, trieLogKeys);
+      LOG.info("Saving trie logs to retain in file (batch {})...", batchNumber);
+      saveTrieLogBatches(batchFileName, rootWorldStateStorage, trieLogKeys);
     }
 
     LOG.info("Clear trie logs...");
@@ -118,15 +123,12 @@ public class TrieLogHelper {
   }
 
   private static void saveTrieLogBatches(
-      final String batchFileNameBase,
+      final String batchFileName,
       final BonsaiWorldStateKeyValueStorage rootWorldStateStorage,
-      final long batchNumber,
       final List<Hash> trieLogKeys) {
 
-    LOG.info("Saving trie logs to retain in file (batch {})...", batchNumber);
-
     try {
-      saveTrieLogsInFile(trieLogKeys, rootWorldStateStorage, batchNumber, batchFileNameBase);
+      saveTrieLogsInFile(trieLogKeys, rootWorldStateStorage, batchFileName);
     } catch (IOException e) {
       LOG.error("Error saving trie logs to file: {}", e.getMessage());
       throw new RuntimeException(e);
@@ -210,9 +212,8 @@ public class TrieLogHelper {
       final String batchFileNameBase)
       throws IOException {
     // process in chunk to avoid OOM
-
+    final String batchFileName = batchFileNameBase + "-" + batchNumber;
-    IdentityHashMap<byte[], byte[]> trieLogsToRetain =
+    IdentityHashMap<byte[], byte[]> trieLogsToRetain = readTrieLogsFromFile(batchFileName);
-        readTrieLogsFromFile(batchFileNameBase, batchNumber);
     final int chunkSize = ROCKSDB_MAX_INSERTS_PER_TRANSACTION;
     List<byte[]> keys = new ArrayList<>(trieLogsToRetain.keySet());
 
@@ -265,11 +266,10 @@ public class TrieLogHelper {
   private static void saveTrieLogsInFile(
       final List<Hash> trieLogsKeys,
       final BonsaiWorldStateKeyValueStorage rootWorldStateStorage,
-      final long batchNumber,
+      final String batchFileName)
-      final String batchFileNameBase)
       throws IOException {
 
-    File file = new File(batchFileNameBase + "-" + batchNumber);
+    File file = new File(batchFileName);
     if (file.exists()) {
       LOG.error("File already exists, skipping file creation");
       return;
@@ -285,17 +285,14 @@ public class TrieLogHelper {
   }
 
   @SuppressWarnings("unchecked")
-  private static IdentityHashMap<byte[], byte[]> readTrieLogsFromFile(
+  static IdentityHashMap<byte[], byte[]> readTrieLogsFromFile(final String batchFileName) {
-      final String batchFileNameBase, final long batchNumber) {
 
     IdentityHashMap<byte[], byte[]> trieLogs;
-    try (FileInputStream fis = new FileInputStream(batchFileNameBase + "-" + batchNumber);
+    try (FileInputStream fis = new FileInputStream(batchFileName);
         ObjectInputStream ois = new ObjectInputStream(fis)) {
 
       trieLogs = (IdentityHashMap<byte[], byte[]>) ois.readObject();
-
     } catch (IOException | ClassNotFoundException e) {
-
       LOG.error(e.getMessage());
       throw new RuntimeException(e);
     }
@@ -303,6 +300,52 @@ public class TrieLogHelper {
     return trieLogs;
   }
 
+  private static void saveTrieLogsAsRlpInFile(
+      final List<Hash> trieLogsKeys,
+      final BonsaiWorldStateKeyValueStorage rootWorldStateStorage,
+      final String batchFileName) {
+    File file = new File(batchFileName);
+    if (file.exists()) {
+      LOG.error("File already exists, skipping file creation");
+      return;
+    }
+
+    final IdentityHashMap<byte[], byte[]> trieLogs =
+        getTrieLogs(trieLogsKeys, rootWorldStateStorage);
+    final Bytes rlp =
+        RLP.encode(
+            o ->
+                o.writeList(
+                    trieLogs.entrySet(), (val, out) -> out.writeRaw(Bytes.wrap(val.getValue()))));
+    try {
+      Files.write(file.toPath(), rlp.toArrayUnsafe());
+    } catch (IOException e) {
+      LOG.error(e.getMessage());
+      throw new RuntimeException(e);
+    }
+  }
+
+  static IdentityHashMap<byte[], byte[]> readTrieLogsAsRlpFromFile(final String batchFileName) {
+    try {
+      final Bytes file = Bytes.wrap(Files.readAllBytes(Path.of(batchFileName)));
+      final BytesValueRLPInput input = new BytesValueRLPInput(file, false);
+
+      input.enterList();
+      final IdentityHashMap<byte[], byte[]> trieLogs = new IdentityHashMap<>();
+      while (!input.isEndOfCurrentList()) {
+        final Bytes trieLogBytes = input.currentListAsBytes();
+        TrieLogLayer trieLogLayer =
+            TrieLogFactoryImpl.readFrom(new BytesValueRLPInput(Bytes.wrap(trieLogBytes), false));
+        trieLogs.put(trieLogLayer.getBlockHash().toArrayUnsafe(), trieLogBytes.toArrayUnsafe());
+      }
+      input.leaveList();
+
+      return trieLogs;
+    } catch (IOException e) {
+      throw new RuntimeException(e);
+    }
+  }
+
   private static IdentityHashMap<byte[], byte[]> getTrieLogs(
       final List<Hash> trieLogKeys, final BonsaiWorldStateKeyValueStorage rootWorldStateStorage) {
     IdentityHashMap<byte[], byte[]> trieLogsToRetain = new IdentityHashMap<>();
@@ -357,5 +400,25 @@ public class TrieLogHelper {
         count.total, count.canonicalCount, count.forkCount, count.orphanCount);
   }
 
+  static void importTrieLog(
+      final BonsaiWorldStateKeyValueStorage rootWorldStateStorage, final Path trieLogFilePath) {
+
+    var trieLog = readTrieLogsAsRlpFromFile(trieLogFilePath.toString());
+
+    var updater = rootWorldStateStorage.updater();
+    trieLog.forEach((key, value) -> updater.getTrieLogStorageTransaction().put(key, value));
+    updater.getTrieLogStorageTransaction().commit();
+  }
+
+  static void exportTrieLog(
+      final BonsaiWorldStateKeyValueStorage rootWorldStateStorage,
+      final List<Hash> trieLogHash,
+      final Path directoryPath)
+      throws IOException {
+    final String trieLogFile = directoryPath.toString();
+
+    saveTrieLogsAsRlpInFile(trieLogHash, rootWorldStateStorage, trieLogFile);
+  }
+
   record TrieLogCount(int total, int canonicalCount, int forkCount, int orphanCount) {}
 }

besu/src/main/java/org/hyperledger/besu/cli/subcommands/storage/TrieLogSubCommand.java

@@ -19,6 +19,7 @@ import static com.google.common.base.Preconditions.checkNotNull;
 
 import org.hyperledger.besu.cli.util.VersionProvider;
 import org.hyperledger.besu.controller.BesuController;
+import org.hyperledger.besu.datatypes.Hash;
 import org.hyperledger.besu.ethereum.chain.MutableBlockchain;
 import org.hyperledger.besu.ethereum.storage.StorageProvider;
 import org.hyperledger.besu.ethereum.trie.bonsai.storage.BonsaiWorldStateKeyValueStorage;
@@ -26,9 +27,11 @@ import org.hyperledger.besu.ethereum.trie.bonsai.trielog.TrieLogPruner;
 import org.hyperledger.besu.ethereum.worldstate.DataStorageConfiguration;
 import org.hyperledger.besu.ethereum.worldstate.DataStorageFormat;
 
+import java.io.IOException;
 import java.io.PrintWriter;
 import java.nio.file.Path;
 import java.nio.file.Paths;
+import java.util.List;
 
 import org.apache.logging.log4j.Level;
 import org.apache.logging.log4j.core.config.Configurator;
@@ -43,7 +46,12 @@ import picocli.CommandLine.ParentCommand;
     description = "Manipulate trie logs",
     mixinStandardHelpOptions = true,
     versionProvider = VersionProvider.class,
-    subcommands = {TrieLogSubCommand.CountTrieLog.class, TrieLogSubCommand.PruneTrieLog.class})
+    subcommands = {
+      TrieLogSubCommand.CountTrieLog.class,
+      TrieLogSubCommand.PruneTrieLog.class,
+      TrieLogSubCommand.ExportTrieLog.class,
+      TrieLogSubCommand.ImportTrieLog.class
+    })
 public class TrieLogSubCommand implements Runnable {
 
   @SuppressWarnings("UnusedVariable")
@@ -123,6 +131,102 @@ public class TrieLogSubCommand implements Runnable {
     }
   }
 
+  @Command(
+      name = "export",
+      description = "This command exports the trie log of a determined block to a binary file",
+      mixinStandardHelpOptions = true,
+      versionProvider = VersionProvider.class)
+  static class ExportTrieLog implements Runnable {
+
+    @SuppressWarnings("unused")
+    @ParentCommand
+    private TrieLogSubCommand parentCommand;
+
+    @SuppressWarnings("unused")
+    @CommandLine.Spec
+    private CommandLine.Model.CommandSpec spec; // Picocli injects reference to command spec
+
+    @CommandLine.Option(
+        names = "--trie-log-block-hash",
+        description =
+            "Comma separated list of hashes from the blocks you want to export the trie logs of",
+        split = " {0,1}, {0,1}",
+        arity = "1..*")
+    private List<String> trieLogBlockHashList;
+
+    @CommandLine.Option(
+        names = "--trie-log-file-path",
+        description = "The file you want to export the trie logs to",
+        arity = "1..1")
+    private Path trieLogFilePath = null;
+
+    @Override
+    public void run() {
+      if (trieLogFilePath == null) {
+        trieLogFilePath =
+            Paths.get(
+                TrieLogSubCommand.parentCommand
+                    .parentCommand
+                    .dataDir()
+                    .resolve("trie-logs.bin")
+                    .toAbsolutePath()
+                    .toString());
+      }
+
+      TrieLogContext context = getTrieLogContext();
+
+      final List<Hash> listOfBlockHashes =
+          trieLogBlockHashList.stream().map(Hash::fromHexString).toList();
+
+      try {
+        TrieLogHelper.exportTrieLog(
+            context.rootWorldStateStorage(), listOfBlockHashes, trieLogFilePath);
+      } catch (IOException e) {
+        throw new RuntimeException(e);
+      }
+    }
+  }
+
+  @Command(
+      name = "import",
+      description = "This command imports a trie log exported by another besu node",
+      mixinStandardHelpOptions = true,
+      versionProvider = VersionProvider.class)
+  static class ImportTrieLog implements Runnable {
+
+    @SuppressWarnings("unused")
+    @ParentCommand
+    private TrieLogSubCommand parentCommand;
+
+    @SuppressWarnings("unused")
+    @CommandLine.Spec
+    private CommandLine.Model.CommandSpec spec; // Picocli injects reference to command spec
+
+    @CommandLine.Option(
+        names = "--trie-log-file-path",
+        description = "The file you want to import the trie logs from",
+        arity = "1..1")
+    private Path trieLogFilePath = null;
+
+    @Override
+    public void run() {
+      if (trieLogFilePath == null) {
+        trieLogFilePath =
+            Paths.get(
+                TrieLogSubCommand.parentCommand
+                    .parentCommand
+                    .dataDir()
+                    .resolve("trie-logs.bin")
+                    .toAbsolutePath()
+                    .toString());
+      }
+
+      TrieLogContext context = getTrieLogContext();
+
+      TrieLogHelper.importTrieLog(context.rootWorldStateStorage(), trieLogFilePath);
+    }
+  }
+
   record TrieLogContext(
       DataStorageConfiguration config,
       BonsaiWorldStateKeyValueStorage rootWorldStateStorage,
@@ -139,8 +243,7 @@ public class TrieLogSubCommand implements Runnable {
 
     final StorageProvider storageProvider = besuController.getStorageProvider();
     final BonsaiWorldStateKeyValueStorage rootWorldStateStorage =
-        (BonsaiWorldStateKeyValueStorage)
+        (BonsaiWorldStateKeyValueStorage) storageProvider.createWorldStateStorage(config);
-            storageProvider.createWorldStateStorage(DataStorageFormat.BONSAI);
     final MutableBlockchain blockchain = besuController.getProtocolContext().getBlockchain();
     return new TrieLogContext(config, rootWorldStateStorage, blockchain);
   }

besu/src/main/java/org/hyperledger/besu/cli/util/ConfigOptionSearchAndRunHandler.java

@@ -55,7 +55,6 @@ public class ConfigOptionSearchAndRunHandler extends CommandLine.RunLast {
   public List<Object> handle(final ParseResult parseResult) throws ParameterException {
     final CommandLine commandLine = parseResult.commandSpec().commandLine();
     final Optional<File> configFile = findConfigFile(parseResult, commandLine);
-    validatePrivacyOptions(parseResult, commandLine);
     commandLine.setDefaultValueProvider(createDefaultValueProvider(commandLine, configFile));
     commandLine.setExecutionStrategy(resultHandler);
     commandLine.setParameterExceptionHandler(parameterExceptionHandler);
@@ -64,16 +63,6 @@ public class ConfigOptionSearchAndRunHandler extends CommandLine.RunLast {
     return new ArrayList<>();
   }
 
-  private void validatePrivacyOptions(
-      final ParseResult parseResult, final CommandLine commandLine) {
-    if (parseResult.hasMatchedOption("--privacy-onchain-groups-enabled")
-        && parseResult.hasMatchedOption("--privacy-flexible-groups-enabled")) {
-      throw new ParameterException(
-          commandLine,
-          "The `--privacy-onchain-groups-enabled` option is deprecated and you should only use `--privacy-flexible-groups-enabled`");
-    }
-  }
-
   private Optional<File> findConfigFile(
       final ParseResult parseResult, final CommandLine commandLine) {
     if (parseResult.hasMatchedOption("--config-file")

besu/src/main/java/org/hyperledger/besu/controller/BesuControllerBuilder.java

@@ -591,12 +591,14 @@ public abstract class BesuControllerBuilder implements MiningParameterOverrides
     prepForBuild();
 
     final ProtocolSchedule protocolSchedule = createProtocolSchedule();
-    final GenesisState genesisState = GenesisState.fromConfig(genesisConfig, protocolSchedule);
+    final GenesisState genesisState =
+        GenesisState.fromConfig(
+            dataStorageConfiguration.getDataStorageFormat(), genesisConfig, protocolSchedule);
 
     final VariablesStorage variablesStorage = storageProvider.createVariablesStorage();
 
     final WorldStateStorage worldStateStorage =
-        storageProvider.createWorldStateStorage(dataStorageConfiguration.getDataStorageFormat());
+        storageProvider.createWorldStateStorage(dataStorageConfiguration);
 
     final BlockchainStorage blockchainStorage =
         storageProvider.createBlockchainStorage(protocolSchedule, variablesStorage);
@@ -1086,7 +1088,6 @@ public abstract class BesuControllerBuilder implements MiningParameterOverrides
             blockchain,
             Optional.of(dataStorageConfiguration.getBonsaiMaxLayersToLoad()),
             cachedMerkleTrieLoader,
-            metricsSystem,
             besuComponent.map(BesuComponent::getBesuPluginContext).orElse(null),
             evmConfiguration,
             trieLogPruner);

besu/src/test/java/org/hyperledger/besu/cli/BesuCommandTest.java

@@ -29,7 +29,6 @@ import static org.hyperledger.besu.cli.config.NetworkName.MAINNET;
 import static org.hyperledger.besu.cli.config.NetworkName.MORDOR;
 import static org.hyperledger.besu.cli.config.NetworkName.SEPOLIA;
 import static org.hyperledger.besu.cli.util.CommandLineUtils.DEPENDENCY_WARNING_MSG;
-import static org.hyperledger.besu.cli.util.CommandLineUtils.DEPRECATION_WARNING_MSG;
 import static org.hyperledger.besu.ethereum.api.jsonrpc.RpcApis.ENGINE;
 import static org.hyperledger.besu.ethereum.api.jsonrpc.RpcApis.ETH;
 import static org.hyperledger.besu.ethereum.api.jsonrpc.RpcApis.NET;
@@ -96,6 +95,7 @@ import org.hyperledger.besu.plugin.services.privacy.PrivateMarkerTransactionFact
 import org.hyperledger.besu.plugin.services.rpc.PluginRpcRequest;
 import org.hyperledger.besu.util.number.Fraction;
 import org.hyperledger.besu.util.number.Percentage;
+import org.hyperledger.besu.util.number.PositiveNumber;
 import org.hyperledger.besu.util.platform.PlatformDetector;
 
 import java.io.File;
@@ -847,6 +847,8 @@ public class BesuCommandTest extends CommandTestAbstract {
         tomlResult.getDouble(tomlKey);
       } else if (Percentage.class.isAssignableFrom(optionSpec.type())) {
         tomlResult.getLong(tomlKey);
+      } else if (PositiveNumber.class.isAssignableFrom(optionSpec.type())) {
+        tomlResult.getLong(tomlKey);
       } else {
         tomlResult.getString(tomlKey);
       }
@@ -1977,16 +1979,6 @@ public class BesuCommandTest extends CommandTestAbstract {
             "The `--ethstats-contact` requires ethstats server URL to be provided. Either remove --ethstats-contact or provide a URL (via --ethstats=nodename:secret@host:port)");
   }
 
-  @Test
-  public void privacyOnchainGroupsEnabledCannotBeUsedWithPrivacyFlexibleGroupsEnabled() {
-    parseCommand("--privacy-onchain-groups-enabled", "--privacy-flexible-groups-enabled");
-    Mockito.verifyNoInteractions(mockRunnerBuilder);
-    assertThat(commandOutput.toString(UTF_8)).isEmpty();
-    assertThat(commandErrorOutput.toString(UTF_8))
-        .contains(
-            "The `--privacy-onchain-groups-enabled` option is deprecated and you should only use `--privacy-flexible-groups-enabled`");
-  }
-
   @Test
   public void parsesValidBonsaiTrieLimitBackLayersOption() {
     parseCommand("--data-storage-format", "BONSAI", "--bonsai-historical-block-limit", "11");
@@ -3840,8 +3832,8 @@ public class BesuCommandTest extends CommandTestAbstract {
   }
 
   @Test
-  public void pruningLogsDeprecationWarning() {
+  public void pruningLogsDeprecationWarningWithForest() {
-    parseCommand("--pruning-enabled");
+    parseCommand("--pruning-enabled", "--data-storage-format=FOREST");
 
     verify(mockControllerBuilder).isPruningEnabled(true);
 
@@ -3854,6 +3846,17 @@ public class BesuCommandTest extends CommandTestAbstract {
                     + " To save disk space consider switching to Bonsai data storage format."));
   }
 
+  @Test
+  public void pruningLogsIgnoredWarningWithBonsai() {
+    parseCommand("--pruning-enabled", "--data-storage-format=BONSAI");
+
+    verify(mockControllerBuilder).isPruningEnabled(true);
+
+    assertThat(commandOutput.toString(UTF_8)).isEmpty();
+    assertThat(commandErrorOutput.toString(UTF_8)).isEmpty();
+    verify(mockLogger).warn(contains("Forest pruning is ignored with Bonsai data storage format."));
+  }
+
   @Test
   public void devModeOptionMustBeUsed() throws Exception {
     parseCommand("--network", "dev");
@@ -4192,46 +4195,6 @@ public class BesuCommandTest extends CommandTestAbstract {
     assertThat(privacyParameters.isFlexiblePrivacyGroupsEnabled()).isEqualTo(false);
   }
 
-  @Test
-  public void onchainPrivacyGroupEnabledFlagValueIsSet() {
-    parseCommand(
-        "--privacy-enabled",
-        "--privacy-public-key-file",
-        ENCLAVE_PUBLIC_KEY_PATH,
-        "--privacy-onchain-groups-enabled",
-        "--min-gas-price",
-        "0");
-
-    final ArgumentCaptor<PrivacyParameters> privacyParametersArgumentCaptor =
-        ArgumentCaptor.forClass(PrivacyParameters.class);
-
-    verify(mockControllerBuilder).privacyParameters(privacyParametersArgumentCaptor.capture());
-    verify(mockControllerBuilder).build();
-
-    assertThat(commandOutput.toString(UTF_8)).isEmpty();
-    assertThat(commandErrorOutput.toString(UTF_8)).isEmpty();
-
-    final PrivacyParameters privacyParameters = privacyParametersArgumentCaptor.getValue();
-    assertThat(privacyParameters.isFlexiblePrivacyGroupsEnabled()).isEqualTo(true);
-  }
-
-  @Test
-  public void onchainPrivacyGroupEnabledOptionIsDeprecated() {
-    parseCommand(
-        "--privacy-enabled",
-        "--privacy-public-key-file",
-        ENCLAVE_PUBLIC_KEY_PATH,
-        "--privacy-onchain-groups-enabled",
-        "--min-gas-price",
-        "0");
-
-    verify(mockLogger)
-        .warn(
-            DEPRECATION_WARNING_MSG,
-            "--privacy-onchain-groups-enabled",
-            "--privacy-flexible-groups-enabled");
-  }
-
   @Test
   public void flexiblePrivacyGroupEnabledFlagValueIsSet() {
     parseCommand(

besu/src/test/java/org/hyperledger/besu/cli/options/MiningOptionsTest.java

@@ -15,8 +15,8 @@
 package org.hyperledger.besu.cli.options;
 
 import static org.assertj.core.api.Assertions.assertThat;
-import static org.hyperledger.besu.ethereum.core.MiningParameters.Unstable.DEFAULT_NON_POA_BLOCK_TXS_SELECTION_MAX_TIME;
+import static org.hyperledger.besu.ethereum.core.MiningParameters.DEFAULT_NON_POA_BLOCK_TXS_SELECTION_MAX_TIME;
-import static org.hyperledger.besu.ethereum.core.MiningParameters.Unstable.DEFAULT_POA_BLOCK_TXS_SELECTION_MAX_TIME;
+import static org.hyperledger.besu.ethereum.core.MiningParameters.DEFAULT_POA_BLOCK_TXS_SELECTION_MAX_TIME;
 import static org.hyperledger.besu.ethereum.core.MiningParameters.Unstable.DEFAULT_POS_BLOCK_CREATION_MAX_TIME;
 import static org.mockito.Mockito.atMost;
 import static org.mockito.Mockito.verify;
@@ -28,7 +28,7 @@ import org.hyperledger.besu.ethereum.core.ImmutableMiningParameters;
 import org.hyperledger.besu.ethereum.core.ImmutableMiningParameters.MutableInitValues;
 import org.hyperledger.besu.ethereum.core.ImmutableMiningParameters.Unstable;
 import org.hyperledger.besu.ethereum.core.MiningParameters;
-import org.hyperledger.besu.util.number.Percentage;
+import org.hyperledger.besu.util.number.PositiveNumber;
 
 import java.io.IOException;
 import java.nio.file.Path;
@@ -315,35 +315,26 @@ public class MiningOptionsTest extends AbstractCLIOptionsTest<MiningParameters,
   public void blockTxsSelectionMaxTimeDefaultValue() {
     internalTestSuccess(
         miningParams ->
-            assertThat(miningParams.getUnstable().getBlockTxsSelectionMaxTime())
+            assertThat(miningParams.getNonPoaBlockTxsSelectionMaxTime())
                 .isEqualTo(DEFAULT_NON_POA_BLOCK_TXS_SELECTION_MAX_TIME));
   }
 
   @Test
   public void blockTxsSelectionMaxTimeOption() {
     internalTestSuccess(
-        miningParams ->
+        miningParams -> assertThat(miningParams.getBlockTxsSelectionMaxTime()).isEqualTo(1700L),
-            assertThat(miningParams.getUnstable().getBlockTxsSelectionMaxTime()).isEqualTo(1700L),
+        "--block-txs-selection-max-time",
-        "--Xblock-txs-selection-max-time",
         "1700");
   }
 
-  @Test
-  public void blockTxsSelectionMaxTimeOutOfAllowedRange() {
-    internalTestFailure(
-        "--Xblock-txs-selection-max-time must be positive and ≤ 5000",
-        "--Xblock-txs-selection-max-time",
-        "6000");
-  }
-
   @Test
   public void blockTxsSelectionMaxTimeIncompatibleWithPoaNetworks() throws IOException {
     final Path genesisFileIBFT2 = createFakeGenesisFile(VALID_GENESIS_IBFT2_POST_LONDON);
     internalTestFailure(
-        "--Xblock-txs-selection-max-time can't be used with PoA networks, see Xpoa-block-txs-selection-max-time instead",
+        "--block-txs-selection-max-time can't be used with PoA networks, see poa-block-txs-selection-max-time instead",
         "--genesis-file",
         genesisFileIBFT2.toString(),
-        "--Xblock-txs-selection-max-time",
+        "--block-txs-selection-max-time",
         "2");
   }
 
@@ -351,7 +342,7 @@ public class MiningOptionsTest extends AbstractCLIOptionsTest<MiningParameters,
   public void poaBlockTxsSelectionMaxTimeDefaultValue() {
     internalTestSuccess(
         miningParams ->
-            assertThat(miningParams.getUnstable().getPoaBlockTxsSelectionMaxTime())
+            assertThat(miningParams.getPoaBlockTxsSelectionMaxTime())
                 .isEqualTo(DEFAULT_POA_BLOCK_TXS_SELECTION_MAX_TIME));
   }
 
@@ -360,27 +351,32 @@ public class MiningOptionsTest extends AbstractCLIOptionsTest<MiningParameters,
     final Path genesisFileIBFT2 = createFakeGenesisFile(VALID_GENESIS_IBFT2_POST_LONDON);
     internalTestSuccess(
         miningParams ->
-            assertThat(miningParams.getUnstable().getPoaBlockTxsSelectionMaxTime())
+            assertThat(miningParams.getPoaBlockTxsSelectionMaxTime())
-                .isEqualTo(Percentage.fromInt(80)),
+                .isEqualTo(PositiveNumber.fromInt(80)),
         "--genesis-file",
         genesisFileIBFT2.toString(),
-        "--Xpoa-block-txs-selection-max-time",
+        "--poa-block-txs-selection-max-time",
         "80");
   }
 
   @Test
-  public void poaBlockTxsSelectionMaxTimeOutOfAllowedRange() {
+  public void poaBlockTxsSelectionMaxTimeOptionOver100Percent() throws IOException {
-    internalTestFailure(
+    final Path genesisFileIBFT2 = createFakeGenesisFile(VALID_GENESIS_IBFT2_POST_LONDON);
-        "Invalid value for option '--Xpoa-block-txs-selection-max-time': cannot convert '110' to Percentage",
+    internalTestSuccess(
-        "--Xpoa-block-txs-selection-max-time",
+        miningParams ->
-        "110");
+            assertThat(miningParams.getPoaBlockTxsSelectionMaxTime())
+                .isEqualTo(PositiveNumber.fromInt(200)),
+        "--genesis-file",
+        genesisFileIBFT2.toString(),
+        "--poa-block-txs-selection-max-time",
+        "200");
   }
 
   @Test
   public void poaBlockTxsSelectionMaxTimeOnlyCompatibleWithPoaNetworks() {
     internalTestFailure(
-        "--Xpoa-block-txs-selection-max-time can be only used with PoA networks, see --Xblock-txs-selection-max-time instead",
+        "--poa-block-txs-selection-max-time can be only used with PoA networks, see --block-txs-selection-max-time instead",
-        "--Xpoa-block-txs-selection-max-time",
+        "--poa-block-txs-selection-max-time",
         "90");
   }
 

besu/src/test/java/org/hyperledger/besu/cli/options/NetworkingOptionsTest.java

@@ -134,7 +134,7 @@ public class NetworkingOptionsTest
 
     final NetworkingOptions options = cmd.getNetworkingOptions();
     final NetworkingConfiguration networkingConfig = options.toDomainObject();
-    assertThat(networkingConfig.getDiscovery().isFilterOnEnrForkIdEnabled()).isEqualTo(false);
+    assertThat(networkingConfig.getDiscovery().isFilterOnEnrForkIdEnabled()).isEqualTo(true);
 
     assertThat(commandErrorOutput.toString(UTF_8)).isEmpty();
     assertThat(commandOutput.toString(UTF_8)).isEmpty();

besu/src/test/java/org/hyperledger/besu/cli/options/stable/DataStorageOptionsTest.java

@@ -34,8 +34,8 @@ public class DataStorageOptionsTest
         dataStorageConfiguration ->
             assertThat(dataStorageConfiguration.getUnstable().getBonsaiTrieLogPruningLimit())
                 .isEqualTo(1),
-        "--Xbonsai-trie-log-pruning-enabled",
+        "--Xbonsai-limit-trie-logs-enabled",
-        "--Xbonsai-trie-log-pruning-limit",
+        "--Xbonsai-trie-logs-pruning-limit",
         "1");
   }
 
@@ -43,8 +43,8 @@ public class DataStorageOptionsTest
   public void bonsaiTrieLogPruningLimitShouldBePositive() {
     internalTestFailure(
         "--Xbonsai-trie-log-pruning-limit=0 must be greater than 0",
-        "--Xbonsai-trie-log-pruning-enabled",
+        "--Xbonsai-limit-trie-logs-enabled",
-        "--Xbonsai-trie-log-pruning-limit",
+        "--Xbonsai-trie-logs-pruning-limit",
         "0");
   }
 
@@ -54,8 +54,8 @@ public class DataStorageOptionsTest
         dataStorageConfiguration ->
             assertThat(dataStorageConfiguration.getUnstable().getBonsaiTrieLogRetentionThreshold())
                 .isEqualTo(MINIMUM_BONSAI_TRIE_LOG_RETENTION_THRESHOLD + 1),
-        "--Xbonsai-trie-log-pruning-enabled",
+        "--Xbonsai-limit-trie-logs-enabled",
-        "--Xbonsai-trie-log-retention-threshold",
+        "--Xbonsai-trie-logs-retention-threshold",
         "513");
   }
 
@@ -65,8 +65,8 @@ public class DataStorageOptionsTest
         dataStorageConfiguration ->
             assertThat(dataStorageConfiguration.getUnstable().getBonsaiTrieLogRetentionThreshold())
                 .isEqualTo(MINIMUM_BONSAI_TRIE_LOG_RETENTION_THRESHOLD),
-        "--Xbonsai-trie-log-pruning-enabled",
+        "--Xbonsai-limit-trie-logs-enabled",
-        "--Xbonsai-trie-log-retention-threshold",
+        "--Xbonsai-trie-logs-retention-threshold",
         "512");
   }
 
@@ -74,8 +74,8 @@ public class DataStorageOptionsTest
   public void bonsaiTrieLogRetentionThresholdShouldBeAboveMinimum() {
     internalTestFailure(
         "--Xbonsai-trie-log-retention-threshold minimum value is 512",
-        "--Xbonsai-trie-log-pruning-enabled",
+        "--Xbonsai-limit-trie-logs-enabled",
-        "--Xbonsai-trie-log-retention-threshold",
+        "--Xbonsai-trie-logs-retention-threshold",
         "511");
   }
 

besu/src/test/java/org/hyperledger/besu/cli/subcommands/storage/TrieLogHelperTest.java

@@ -15,6 +15,7 @@
 
 package org.hyperledger.besu.cli.subcommands.storage;
 
+import static java.util.Collections.singletonList;
 import static org.hyperledger.besu.ethereum.worldstate.DataStorageFormat.BONSAI;
 import static org.junit.jupiter.api.Assertions.assertArrayEquals;
 import static org.junit.jupiter.api.Assertions.assertEquals;
@@ -27,8 +28,11 @@ import org.hyperledger.besu.ethereum.chain.MutableBlockchain;
 import org.hyperledger.besu.ethereum.core.BlockHeader;
 import org.hyperledger.besu.ethereum.core.BlockHeaderTestFixture;
 import org.hyperledger.besu.ethereum.core.InMemoryKeyValueStorageProvider;
+import org.hyperledger.besu.ethereum.rlp.BytesValueRLPOutput;
 import org.hyperledger.besu.ethereum.storage.StorageProvider;
 import org.hyperledger.besu.ethereum.trie.bonsai.storage.BonsaiWorldStateKeyValueStorage;
+import org.hyperledger.besu.ethereum.trie.bonsai.trielog.TrieLogFactoryImpl;
+import org.hyperledger.besu.ethereum.trie.bonsai.trielog.TrieLogLayer;
 import org.hyperledger.besu.ethereum.worldstate.DataStorageConfiguration;
 import org.hyperledger.besu.ethereum.worldstate.ImmutableDataStorageConfiguration;
 import org.hyperledger.besu.metrics.noop.NoOpMetricsSystem;
@@ -36,11 +40,12 @@ import org.hyperledger.besu.metrics.noop.NoOpMetricsSystem;
 import java.io.IOException;
 import java.nio.file.Files;
 import java.nio.file.Path;
+import java.util.List;
+import java.util.Map;
 import java.util.Optional;
+import java.util.stream.Collectors;
 
 import org.apache.tuweni.bytes.Bytes;
-import org.junit.jupiter.api.AfterEach;
-import org.junit.jupiter.api.BeforeAll;
 import org.junit.jupiter.api.BeforeEach;
 import org.junit.jupiter.api.Test;
 import org.junit.jupiter.api.extension.ExtendWith;
@@ -56,17 +61,14 @@ class TrieLogHelperTest {
 
   @Mock private MutableBlockchain blockchain;
 
-  @TempDir static Path dataDir;
-
-  Path test;
   static BlockHeader blockHeader1;
   static BlockHeader blockHeader2;
   static BlockHeader blockHeader3;
   static BlockHeader blockHeader4;
   static BlockHeader blockHeader5;
 
-  @BeforeAll
+  @BeforeEach
-  public static void setup() throws IOException {
+  public void setup() throws IOException {
 
     blockHeader1 = new BlockHeaderTestFixture().number(1).buildHeader();
     blockHeader2 = new BlockHeaderTestFixture().number(2).buildHeader();
@@ -75,35 +77,36 @@ class TrieLogHelperTest {
     blockHeader5 = new BlockHeaderTestFixture().number(5).buildHeader();
 
     inMemoryWorldState =
-        new BonsaiWorldStateKeyValueStorage(storageProvider, new NoOpMetricsSystem());
+        new BonsaiWorldStateKeyValueStorage(
+            storageProvider, new NoOpMetricsSystem(), DataStorageConfiguration.DEFAULT_CONFIG);
+
+    createTrieLog(blockHeader1);
 
     var updater = inMemoryWorldState.updater();
     updater
         .getTrieLogStorageTransaction()
-        .put(blockHeader1.getHash().toArrayUnsafe(), Bytes.fromHexString("0x01").toArrayUnsafe());
+        .put(blockHeader1.getHash().toArrayUnsafe(), createTrieLog(blockHeader1));
     updater
         .getTrieLogStorageTransaction()
-        .put(blockHeader2.getHash().toArrayUnsafe(), Bytes.fromHexString("0x02").toArrayUnsafe());
+        .put(blockHeader2.getHash().toArrayUnsafe(), createTrieLog(blockHeader2));
     updater
         .getTrieLogStorageTransaction()
-        .put(blockHeader3.getHash().toArrayUnsafe(), Bytes.fromHexString("0x03").toArrayUnsafe());
+        .put(blockHeader3.getHash().toArrayUnsafe(), createTrieLog(blockHeader3));
     updater
         .getTrieLogStorageTransaction()
-        .put(blockHeader4.getHash().toArrayUnsafe(), Bytes.fromHexString("0x04").toArrayUnsafe());
+        .put(blockHeader4.getHash().toArrayUnsafe(), createTrieLog(blockHeader4));
     updater
         .getTrieLogStorageTransaction()
-        .put(blockHeader5.getHash().toArrayUnsafe(), Bytes.fromHexString("0x05").toArrayUnsafe());
+        .put(blockHeader5.getHash().toArrayUnsafe(), createTrieLog(blockHeader5));
     updater.getTrieLogStorageTransaction().commit();
   }
 
-  @BeforeEach
+  private static byte[] createTrieLog(final BlockHeader blockHeader) {
-  void createDirectory() throws IOException {
+    TrieLogLayer trieLogLayer = new TrieLogLayer();
-    Files.createDirectories(dataDir.resolve("database"));
+    trieLogLayer.setBlockHash(blockHeader.getBlockHash());
-  }
+    final BytesValueRLPOutput rlpLog = new BytesValueRLPOutput();
-
+    TrieLogFactoryImpl.writeTo(trieLogLayer, rlpLog);
-  @AfterEach
+    return rlpLog.encoded().toArrayUnsafe();
-  void deleteDirectory() throws IOException {
-    Files.deleteIfExists(dataDir.resolve("database"));
   }
 
   void mockBlockchainBase() {
@@ -113,7 +116,8 @@ class TrieLogHelperTest {
   }
 
   @Test
-  public void prune() {
+  public void prune(final @TempDir Path dataDir) throws IOException {
+    Files.createDirectories(dataDir.resolve("database"));
 
     DataStorageConfiguration dataStorageConfiguration =
         ImmutableDataStorageConfiguration.builder()
@@ -133,14 +137,11 @@ class TrieLogHelperTest {
 
     // assert trie logs that will be pruned exist before prune call
     assertArrayEquals(
-        inMemoryWorldState.getTrieLog(blockHeader1.getHash()).get(),
+        inMemoryWorldState.getTrieLog(blockHeader1.getHash()).get(), createTrieLog(blockHeader1));
-        Bytes.fromHexString("0x01").toArrayUnsafe());
     assertArrayEquals(
-        inMemoryWorldState.getTrieLog(blockHeader2.getHash()).get(),
+        inMemoryWorldState.getTrieLog(blockHeader2.getHash()).get(), createTrieLog(blockHeader2));
-        Bytes.fromHexString("0x02").toArrayUnsafe());
     assertArrayEquals(
-        inMemoryWorldState.getTrieLog(blockHeader3.getHash()).get(),
+        inMemoryWorldState.getTrieLog(blockHeader3.getHash()).get(), createTrieLog(blockHeader3));
-        Bytes.fromHexString("0x03").toArrayUnsafe());
 
     TrieLogHelper.prune(dataStorageConfiguration, inMemoryWorldState, blockchain, dataDir);
 
@@ -150,18 +151,15 @@ class TrieLogHelperTest {
 
     // assert retained trie logs are in the DB
     assertArrayEquals(
-        inMemoryWorldState.getTrieLog(blockHeader3.getHash()).get(),
+        inMemoryWorldState.getTrieLog(blockHeader3.getHash()).get(), createTrieLog(blockHeader3));
-        Bytes.fromHexString("0x03").toArrayUnsafe());
     assertArrayEquals(
-        inMemoryWorldState.getTrieLog(blockHeader4.getHash()).get(),
+        inMemoryWorldState.getTrieLog(blockHeader4.getHash()).get(), createTrieLog(blockHeader4));
-        Bytes.fromHexString("0x04").toArrayUnsafe());
     assertArrayEquals(
-        inMemoryWorldState.getTrieLog(blockHeader5.getHash()).get(),
+        inMemoryWorldState.getTrieLog(blockHeader5.getHash()).get(), createTrieLog(blockHeader5));
-        Bytes.fromHexString("0x05").toArrayUnsafe());
   }
 
   @Test
-  public void cantPruneIfNoFinalizedIsFound() {
+  public void cantPruneIfNoFinalizedIsFound(final @TempDir Path dataDir) {
     DataStorageConfiguration dataStorageConfiguration =
         ImmutableDataStorageConfiguration.builder()
             .dataStorageFormat(BONSAI)
@@ -183,7 +181,7 @@ class TrieLogHelperTest {
   }
 
   @Test
-  public void cantPruneIfUserRetainsMoreLayerThanExistingChainLength() {
+  public void cantPruneIfUserRetainsMoreLayerThanExistingChainLength(final @TempDir Path dataDir) {
     DataStorageConfiguration dataStorageConfiguration =
         ImmutableDataStorageConfiguration.builder()
             .dataStorageFormat(BONSAI)
@@ -204,7 +202,7 @@ class TrieLogHelperTest {
   }
 
   @Test
-  public void cantPruneIfUserRequiredFurtherThanFinalized() {
+  public void cantPruneIfUserRequiredFurtherThanFinalized(final @TempDir Path dataDir) {
 
     DataStorageConfiguration dataStorageConfiguration =
         ImmutableDataStorageConfiguration.builder()
@@ -226,8 +224,7 @@ class TrieLogHelperTest {
   }
 
   @Test
-  public void exceptionWhileSavingFileStopsPruneProcess() throws IOException {
+  public void exceptionWhileSavingFileStopsPruneProcess(final @TempDir Path dataDir) {
-    Files.delete(dataDir.resolve("database"));
 
     DataStorageConfiguration dataStorageConfiguration =
         ImmutableDataStorageConfiguration.builder()
@@ -243,23 +240,121 @@ class TrieLogHelperTest {
     assertThrows(
         RuntimeException.class,
         () ->
-            TrieLogHelper.prune(dataStorageConfiguration, inMemoryWorldState, blockchain, dataDir));
+            TrieLogHelper.prune(
+                dataStorageConfiguration,
+                inMemoryWorldState,
+                blockchain,
+                dataDir.resolve("unknownPath")));
 
     // assert all trie logs are still in the DB
     assertArrayEquals(
-        inMemoryWorldState.getTrieLog(blockHeader1.getHash()).get(),
+        inMemoryWorldState.getTrieLog(blockHeader1.getHash()).get(), createTrieLog(blockHeader1));
-        Bytes.fromHexString("0x01").toArrayUnsafe());
     assertArrayEquals(
-        inMemoryWorldState.getTrieLog(blockHeader2.getHash()).get(),
+        inMemoryWorldState.getTrieLog(blockHeader2.getHash()).get(), createTrieLog(blockHeader2));
-        Bytes.fromHexString("0x02").toArrayUnsafe());
     assertArrayEquals(
-        inMemoryWorldState.getTrieLog(blockHeader3.getHash()).get(),
+        inMemoryWorldState.getTrieLog(blockHeader3.getHash()).get(), createTrieLog(blockHeader3));
-        Bytes.fromHexString("0x03").toArrayUnsafe());
     assertArrayEquals(
-        inMemoryWorldState.getTrieLog(blockHeader4.getHash()).get(),
+        inMemoryWorldState.getTrieLog(blockHeader4.getHash()).get(), createTrieLog(blockHeader4));
-        Bytes.fromHexString("0x04").toArrayUnsafe());
     assertArrayEquals(
-        inMemoryWorldState.getTrieLog(blockHeader5.getHash()).get(),
+        inMemoryWorldState.getTrieLog(blockHeader5.getHash()).get(), createTrieLog(blockHeader5));
-        Bytes.fromHexString("0x05").toArrayUnsafe());
+  }
+
+  @Test
+  public void exportedTrieMatchesDbTrieLog(final @TempDir Path dataDir) throws IOException {
+    TrieLogHelper.exportTrieLog(
+        inMemoryWorldState,
+        singletonList(blockHeader1.getHash()),
+        dataDir.resolve("trie-log-dump"));
+
+    var trieLog =
+        TrieLogHelper.readTrieLogsAsRlpFromFile(dataDir.resolve("trie-log-dump").toString())
+            .entrySet()
+            .stream()
+            .findFirst()
+            .get();
+
+    assertArrayEquals(trieLog.getKey(), blockHeader1.getHash().toArrayUnsafe());
+    assertArrayEquals(
+        trieLog.getValue(), inMemoryWorldState.getTrieLog(blockHeader1.getHash()).get());
+  }
+
+  @Test
+  public void exportedMultipleTriesMatchDbTrieLogs(final @TempDir Path dataDir) throws IOException {
+    TrieLogHelper.exportTrieLog(
+        inMemoryWorldState,
+        List.of(blockHeader1.getHash(), blockHeader2.getHash(), blockHeader3.getHash()),
+        dataDir.resolve("trie-log-dump"));
+
+    var trieLogs =
+        TrieLogHelper.readTrieLogsAsRlpFromFile(dataDir.resolve("trie-log-dump").toString())
+            .entrySet()
+            .stream()
+            .collect(Collectors.toMap(e -> Bytes.wrap(e.getKey()), Map.Entry::getValue));
+
+    assertArrayEquals(
+        trieLogs.get(blockHeader1.getHash()),
+        inMemoryWorldState.getTrieLog(blockHeader1.getHash()).get());
+    assertArrayEquals(
+        trieLogs.get(blockHeader2.getHash()),
+        inMemoryWorldState.getTrieLog(blockHeader2.getHash()).get());
+    assertArrayEquals(
+        trieLogs.get(blockHeader3.getHash()),
+        inMemoryWorldState.getTrieLog(blockHeader3.getHash()).get());
+  }
+
+  @Test
+  public void importedTrieLogMatchesDbTrieLog(final @TempDir Path dataDir) throws IOException {
+    StorageProvider tempStorageProvider = new InMemoryKeyValueStorageProvider();
+    BonsaiWorldStateKeyValueStorage inMemoryWorldState2 =
+        new BonsaiWorldStateKeyValueStorage(
+            tempStorageProvider, new NoOpMetricsSystem(), DataStorageConfiguration.DEFAULT_CONFIG);
+
+    TrieLogHelper.exportTrieLog(
+        inMemoryWorldState,
+        singletonList(blockHeader1.getHash()),
+        dataDir.resolve("trie-log-dump"));
+
+    var trieLog =
+        TrieLogHelper.readTrieLogsAsRlpFromFile(dataDir.resolve("trie-log-dump").toString());
+    var updater = inMemoryWorldState2.updater();
+
+    trieLog.forEach((k, v) -> updater.getTrieLogStorageTransaction().put(k, v));
+
+    updater.getTrieLogStorageTransaction().commit();
+
+    assertArrayEquals(
+        inMemoryWorldState2.getTrieLog(blockHeader1.getHash()).get(),
+        inMemoryWorldState.getTrieLog(blockHeader1.getHash()).get());
+  }
+
+  @Test
+  public void importedMultipleTriesMatchDbTrieLogs(final @TempDir Path dataDir) throws IOException {
+    StorageProvider tempStorageProvider = new InMemoryKeyValueStorageProvider();
+    BonsaiWorldStateKeyValueStorage inMemoryWorldState2 =
+        new BonsaiWorldStateKeyValueStorage(
+            tempStorageProvider, new NoOpMetricsSystem(), DataStorageConfiguration.DEFAULT_CONFIG);
+
+    TrieLogHelper.exportTrieLog(
+        inMemoryWorldState,
+        List.of(blockHeader1.getHash(), blockHeader2.getHash(), blockHeader3.getHash()),
+        dataDir.resolve("trie-log-dump"));
+
+    var trieLog =
+        TrieLogHelper.readTrieLogsAsRlpFromFile(dataDir.resolve("trie-log-dump").toString());
+    var updater = inMemoryWorldState2.updater();
+
+    trieLog.forEach((k, v) -> updater.getTrieLogStorageTransaction().put(k, v));
+
+    updater.getTrieLogStorageTransaction().commit();
+
+    assertArrayEquals(
+        inMemoryWorldState2.getTrieLog(blockHeader1.getHash()).get(),
+        inMemoryWorldState.getTrieLog(blockHeader1.getHash()).get());
+    assertArrayEquals(
+        inMemoryWorldState2.getTrieLog(blockHeader2.getHash()).get(),
+        inMemoryWorldState.getTrieLog(blockHeader2.getHash()).get());
+    assertArrayEquals(
+        inMemoryWorldState2.getTrieLog(blockHeader3.getHash()).get(),
+        inMemoryWorldState.getTrieLog(blockHeader3.getHash()).get());
   }
 }

besu/src/test/java/org/hyperledger/besu/controller/BesuControllerBuilderTest.java

@@ -131,7 +131,7 @@ public class BesuControllerBuilderTest {
     when(synchronizerConfiguration.getBlockPropagationRange()).thenReturn(Range.closed(1L, 2L));
 
     lenient()
-        .when(storageProvider.createWorldStateStorage(DataStorageFormat.FOREST))
+        .when(storageProvider.createWorldStateStorage(DataStorageConfiguration.DEFAULT_CONFIG))
         .thenReturn(worldStateStorage);
     lenient()
         .when(storageProvider.createWorldStatePreimageStorage())
@@ -166,6 +166,11 @@ public class BesuControllerBuilderTest {
 
   @Test
   public void shouldDisablePruningIfBonsaiIsEnabled() {
+    DataStorageConfiguration dataStorageConfiguration =
+        ImmutableDataStorageConfiguration.builder()
+            .dataStorageFormat(DataStorageFormat.BONSAI)
+            .bonsaiMaxLayersToLoad(DataStorageConfiguration.DEFAULT_BONSAI_MAX_LAYERS_TO_LOAD)
+            .build();
     BonsaiWorldState mockWorldState = mock(BonsaiWorldState.class, Answers.RETURNS_DEEP_STUBS);
     doReturn(worldStateArchive)
         .when(besuControllerBuilder)
@@ -173,15 +178,9 @@ public class BesuControllerBuilderTest {
             any(WorldStateStorage.class), any(Blockchain.class), any(CachedMerkleTrieLoader.class));
     doReturn(mockWorldState).when(worldStateArchive).getMutable();
 
-    when(storageProvider.createWorldStateStorage(DataStorageFormat.BONSAI))
+    when(storageProvider.createWorldStateStorage(dataStorageConfiguration))
         .thenReturn(bonsaiWorldStateStorage);
-    besuControllerBuilder
+    besuControllerBuilder.isPruningEnabled(true).dataStorageConfiguration(dataStorageConfiguration);
-        .isPruningEnabled(true)
-        .dataStorageConfiguration(
-            ImmutableDataStorageConfiguration.builder()
-                .dataStorageFormat(DataStorageFormat.BONSAI)
-                .bonsaiMaxLayersToLoad(DataStorageConfiguration.DEFAULT_BONSAI_MAX_LAYERS_TO_LOAD)
-                .build());
     besuControllerBuilder.build();
 
     verify(storageProvider, never())

besu/src/test/java/org/hyperledger/besu/controller/MergeBesuControllerBuilderTest.java

@@ -52,7 +52,7 @@ import org.hyperledger.besu.ethereum.p2p.config.NetworkingConfiguration;
 import org.hyperledger.besu.ethereum.storage.StorageProvider;
 import org.hyperledger.besu.ethereum.storage.keyvalue.KeyValueStoragePrefixedKeyBlockchainStorage;
 import org.hyperledger.besu.ethereum.storage.keyvalue.VariablesKeyValueStorage;
-import org.hyperledger.besu.ethereum.worldstate.DataStorageFormat;
+import org.hyperledger.besu.ethereum.worldstate.DataStorageConfiguration;
 import org.hyperledger.besu.ethereum.worldstate.WorldStateArchive;
 import org.hyperledger.besu.ethereum.worldstate.WorldStatePreimageStorage;
 import org.hyperledger.besu.ethereum.worldstate.WorldStateStorage;
@@ -145,7 +145,7 @@ public class MergeBesuControllerBuilderTest {
         .thenReturn(Range.closed(1L, 2L));
 
     lenient()
-        .when(storageProvider.createWorldStateStorage(DataStorageFormat.FOREST))
+        .when(storageProvider.createWorldStateStorage(DataStorageConfiguration.DEFAULT_CONFIG))
         .thenReturn(worldStateStorage);
     lenient()
         .when(storageProvider.createWorldStatePreimageStorage())

besu/src/test/java/org/hyperledger/besu/controller/QbftBesuControllerBuilderTest.java

@@ -48,7 +48,7 @@ import org.hyperledger.besu.ethereum.p2p.config.NetworkingConfiguration;
 import org.hyperledger.besu.ethereum.storage.StorageProvider;
 import org.hyperledger.besu.ethereum.storage.keyvalue.KeyValueStoragePrefixedKeyBlockchainStorage;
 import org.hyperledger.besu.ethereum.storage.keyvalue.VariablesKeyValueStorage;
-import org.hyperledger.besu.ethereum.worldstate.DataStorageFormat;
+import org.hyperledger.besu.ethereum.worldstate.DataStorageConfiguration;
 import org.hyperledger.besu.ethereum.worldstate.WorldStatePreimageStorage;
 import org.hyperledger.besu.ethereum.worldstate.WorldStateStorage;
 import org.hyperledger.besu.evm.internal.EvmConfiguration;
@@ -114,7 +114,7 @@ public class QbftBesuControllerBuilderTest {
                 new VariablesKeyValueStorage(new InMemoryKeyValueStorage()),
                 new MainnetBlockHeaderFunctions()));
     lenient()
-        .when(storageProvider.createWorldStateStorage(DataStorageFormat.FOREST))
+        .when(storageProvider.createWorldStateStorage(DataStorageConfiguration.DEFAULT_CONFIG))
         .thenReturn(worldStateStorage);
     lenient().when(worldStateStorage.isWorldStateAvailable(any(), any())).thenReturn(true);
     lenient().when(worldStateStorage.updater()).thenReturn(mock(WorldStateStorage.Updater.class));

besu/src/test/resources/everything_config.toml

@@ -142,6 +142,8 @@ min-priority-fee=0
 min-block-occupancy-ratio=0.7
 miner-stratum-host="0.0.0.0"
 miner-stratum-port=8008
+block-txs-selection-max-time=5000
+poa-block-txs-selection-max-time=75
 Xminer-remote-sealers-limit=1000
 Xminer-remote-sealers-hashrate-ttl=10
 Xpos-block-creation-max-time=5
@@ -169,7 +171,6 @@ privacy-enabled=false
 privacy-multi-tenancy-enabled=true
 privacy-marker-transaction-signing-key-file="./signerKey"
 privacy-enable-database-migration=false
-privacy-onchain-groups-enabled=false
 privacy-flexible-groups-enabled=false
 
 # Transaction Pool

ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/handlers/HandlerFactory.java

@@ -22,6 +22,7 @@ import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.JsonRpcMethod;
 import java.util.Collection;
 import java.util.Map;
 import java.util.Optional;
+import java.util.function.Function;
 import java.util.stream.Collectors;
 
 import io.opentelemetry.api.trace.Tracer;
@@ -35,7 +36,8 @@ public class HandlerFactory {
     assert methods != null && globalOptions != null;
     return TimeoutHandler.handler(
         Optional.of(globalOptions),
-        methods.keySet().stream().collect(Collectors.toMap(String::new, ignored -> globalOptions)));
+        methods.keySet().stream()
+            .collect(Collectors.toMap(Function.identity(), ignored -> globalOptions)));
   }
 
   public static Handler<RoutingContext> authentication(

ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/DebugTraceBlock.java

@@ -46,15 +46,15 @@ public class DebugTraceBlock implements JsonRpcMethod {
   private static final Logger LOG = LoggerFactory.getLogger(DebugTraceBlock.class);
   private final Supplier<BlockTracer> blockTracerSupplier;
   private final BlockHeaderFunctions blockHeaderFunctions;
-  private final BlockchainQueries blockchain;
+  private final BlockchainQueries blockchainQueries;
 
   public DebugTraceBlock(
       final Supplier<BlockTracer> blockTracerSupplier,
       final BlockHeaderFunctions blockHeaderFunctions,
-      final BlockchainQueries blockchain) {
+      final BlockchainQueries blockchainQueries) {
     this.blockTracerSupplier = blockTracerSupplier;
     this.blockHeaderFunctions = blockHeaderFunctions;
-    this.blockchain = blockchain;
+    this.blockchainQueries = blockchainQueries;
   }
 
   @Override
@@ -79,18 +79,17 @@ public class DebugTraceBlock implements JsonRpcMethod {
             .map(TransactionTraceParams::traceOptions)
             .orElse(TraceOptions.DEFAULT);
 
-    if (this.blockchain.blockByHash(block.getHeader().getParentHash()).isPresent()) {
+    if (this.blockchainQueries.blockByHash(block.getHeader().getParentHash()).isPresent()) {
       final Collection<DebugTraceTransactionResult> results =
           Tracer.processTracing(
-                  blockchain,
+                  blockchainQueries,
                   Optional.of(block.getHeader()),
-                  mutableWorldState -> {
+                  mutableWorldState ->
-                    return blockTracerSupplier
+                      blockTracerSupplier
-                        .get()
+                          .get()
-                        .trace(mutableWorldState, block, new DebugOperationTracer(traceOptions))
+                          .trace(mutableWorldState, block, new DebugOperationTracer(traceOptions))
-                        .map(BlockTrace::getTransactionTraces)
+                          .map(BlockTrace::getTransactionTraces)
-                        .map(DebugTraceTransactionResult::of);
+                          .map(DebugTraceTransactionResult::of))
-                  })
               .orElse(null);
       return new JsonRpcSuccessResponse(requestContext.getRequest().getId(), results);
     } else {

ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/jsonrpc/JsonRpcHttpServiceHostAllowlistTest.java

@@ -17,7 +17,6 @@ package org.hyperledger.besu.ethereum.api.jsonrpc;
 import static org.assertj.core.api.Assertions.assertThat;
 import static org.hyperledger.besu.ethereum.api.jsonrpc.RpcApis.DEFAULT_RPC_APIS;
 import static org.mockito.Mockito.mock;
-import static org.mockito.Mockito.spy;
 
 import org.hyperledger.besu.config.StubGenesisConfigOptions;
 import org.hyperledger.besu.ethereum.ProtocolContext;
@@ -98,38 +97,37 @@ public class JsonRpcHttpServiceHostAllowlistTest {
     supportedCapabilities.add(EthProtocol.ETH63);
 
     rpcMethods =
-        spy(
+        new JsonRpcMethodsFactory()
-            new JsonRpcMethodsFactory()
+            .methods(
-                .methods(
+                CLIENT_VERSION,
-                    CLIENT_VERSION,
+                CHAIN_ID,
-                    CHAIN_ID,
+                new StubGenesisConfigOptions(),
-                    new StubGenesisConfigOptions(),
+                peerDiscoveryMock,
-                    peerDiscoveryMock,
+                blockchainQueries,
-                    blockchainQueries,
+                synchronizer,
-                    synchronizer,
+                MainnetProtocolSchedule.fromConfig(
-                    MainnetProtocolSchedule.fromConfig(
+                    new StubGenesisConfigOptions().constantinopleBlock(0).chainId(CHAIN_ID)),
-                        new StubGenesisConfigOptions().constantinopleBlock(0).chainId(CHAIN_ID)),
+                mock(ProtocolContext.class),
-                    mock(ProtocolContext.class),
+                mock(FilterManager.class),
-                    mock(FilterManager.class),
+                mock(TransactionPool.class),
-                    mock(TransactionPool.class),
+                mock(MiningParameters.class),
-                    mock(MiningParameters.class),
+                mock(PoWMiningCoordinator.class),
-                    mock(PoWMiningCoordinator.class),
+                new NoOpMetricsSystem(),
-                    new NoOpMetricsSystem(),
+                supportedCapabilities,
-                    supportedCapabilities,
+                Optional.of(mock(AccountLocalConfigPermissioningController.class)),
-                    Optional.of(mock(AccountLocalConfigPermissioningController.class)),
+                Optional.of(mock(NodeLocalConfigPermissioningController.class)),
-                    Optional.of(mock(NodeLocalConfigPermissioningController.class)),
+                DEFAULT_RPC_APIS,
-                    DEFAULT_RPC_APIS,
+                mock(PrivacyParameters.class),
-                    mock(PrivacyParameters.class),
+                mock(JsonRpcConfiguration.class),
-                    mock(JsonRpcConfiguration.class),
+                mock(WebSocketConfiguration.class),
-                    mock(WebSocketConfiguration.class),
+                mock(MetricsConfiguration.class),
-                    mock(MetricsConfiguration.class),
+                natService,
-                    natService,
+                new HashMap<>(),
-                    new HashMap<>(),
+                folder,
-                    folder,
+                mock(EthPeers.class),
-                    mock(EthPeers.class),
+                vertx,
-                    vertx,
+                mock(ApiConfiguration.class),
-                    mock(ApiConfiguration.class),
+                Optional.empty());
-                    Optional.empty()));
     service = createJsonRpcHttpService();
     service.start().join();
 

ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/jsonrpc/JsonRpcHttpServiceLoginTest.java

@@ -19,7 +19,6 @@ import static java.util.concurrent.TimeUnit.MINUTES;
 import static org.assertj.core.api.Assertions.assertThat;
 import static org.assertj.core.util.Lists.list;
 import static org.mockito.Mockito.mock;
-import static org.mockito.Mockito.spy;
 
 import org.hyperledger.besu.config.StubGenesisConfigOptions;
 import org.hyperledger.besu.ethereum.ProtocolContext;
@@ -129,37 +128,36 @@ public class JsonRpcHttpServiceLoginTest {
         new StubGenesisConfigOptions().constantinopleBlock(0).chainId(CHAIN_ID);
 
     rpcMethods =
-        spy(
+        new JsonRpcMethodsFactory()
-            new JsonRpcMethodsFactory()
+            .methods(
-                .methods(
+                CLIENT_VERSION,
-                    CLIENT_VERSION,
+                CHAIN_ID,
-                    CHAIN_ID,
+                genesisConfigOptions,
-                    genesisConfigOptions,
+                peerDiscoveryMock,
-                    peerDiscoveryMock,
+                blockchainQueries,
-                    blockchainQueries,
+                synchronizer,
-                    synchronizer,
+                MainnetProtocolSchedule.fromConfig(genesisConfigOptions),
-                    MainnetProtocolSchedule.fromConfig(genesisConfigOptions),
+                mock(ProtocolContext.class),
-                    mock(ProtocolContext.class),
+                mock(FilterManager.class),
-                    mock(FilterManager.class),
+                mock(TransactionPool.class),
-                    mock(TransactionPool.class),
+                mock(MiningParameters.class),
-                    mock(MiningParameters.class),
+                mock(PoWMiningCoordinator.class),
-                    mock(PoWMiningCoordinator.class),
+                new NoOpMetricsSystem(),
-                    new NoOpMetricsSystem(),
+                supportedCapabilities,
-                    supportedCapabilities,
+                Optional.empty(),
-                    Optional.empty(),
+                Optional.empty(),
-                    Optional.empty(),
+                JSON_RPC_APIS,
-                    JSON_RPC_APIS,
+                mock(PrivacyParameters.class),
-                    mock(PrivacyParameters.class),
+                mock(JsonRpcConfiguration.class),
-                    mock(JsonRpcConfiguration.class),
+                mock(WebSocketConfiguration.class),
-                    mock(WebSocketConfiguration.class),
+                mock(MetricsConfiguration.class),
-                    mock(MetricsConfiguration.class),
+                natService,
-                    natService,
+                new HashMap<>(),
-                    new HashMap<>(),
+                folder,
-                    folder,
+                mock(EthPeers.class),
-                    mock(EthPeers.class),
+                vertx,
-                    vertx,
+                mock(ApiConfiguration.class),
-                    mock(ApiConfiguration.class),
+                Optional.empty());
-                    Optional.empty()));
     service = createJsonRpcHttpService();
     jwtAuth = service.authenticationService.get().getJwtAuthProvider();
     service.start().join();

ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/jsonrpc/JsonRpcHttpServiceRpcApisTest.java

@@ -17,7 +17,6 @@ package org.hyperledger.besu.ethereum.api.jsonrpc;
 import static java.util.Collections.singletonList;
 import static org.assertj.core.api.Assertions.assertThat;
 import static org.mockito.Mockito.mock;
-import static org.mockito.Mockito.spy;
 import static org.mockito.Mockito.when;
 
 import org.hyperledger.besu.config.StubGenesisConfigOptions;
@@ -201,37 +200,36 @@ public class JsonRpcHttpServiceRpcApisTest {
     supportedCapabilities.add(EthProtocol.ETH63);
 
     final Map<String, JsonRpcMethod> rpcMethods =
-        spy(
+        new JsonRpcMethodsFactory()
-            new JsonRpcMethodsFactory()
+            .methods(
-                .methods(
+                CLIENT_VERSION,
-                    CLIENT_VERSION,
+                NETWORK_ID,
-                    NETWORK_ID,
+                new StubGenesisConfigOptions(),
-                    new StubGenesisConfigOptions(),
+                mock(P2PNetwork.class),
-                    mock(P2PNetwork.class),
+                blockchainQueries,
-                    blockchainQueries,
+                mock(Synchronizer.class),
-                    mock(Synchronizer.class),
+                ProtocolScheduleFixture.MAINNET,
-                    ProtocolScheduleFixture.MAINNET,
+                mock(ProtocolContext.class),
-                    mock(ProtocolContext.class),
+                mock(FilterManager.class),
-                    mock(FilterManager.class),
+                mock(TransactionPool.class),
-                    mock(TransactionPool.class),
+                mock(MiningParameters.class),
-                    mock(MiningParameters.class),
+                mock(PoWMiningCoordinator.class),
-                    mock(PoWMiningCoordinator.class),
+                new NoOpMetricsSystem(),
-                    new NoOpMetricsSystem(),
+                supportedCapabilities,
-                    supportedCapabilities,
+                Optional.of(mock(AccountLocalConfigPermissioningController.class)),
-                    Optional.of(mock(AccountLocalConfigPermissioningController.class)),
+                Optional.of(mock(NodeLocalConfigPermissioningController.class)),
-                    Optional.of(mock(NodeLocalConfigPermissioningController.class)),
+                config.getRpcApis(),
-                    config.getRpcApis(),
+                mock(PrivacyParameters.class),
-                    mock(PrivacyParameters.class),
+                mock(JsonRpcConfiguration.class),
-                    mock(JsonRpcConfiguration.class),
+                mock(WebSocketConfiguration.class),
-                    mock(WebSocketConfiguration.class),
+                mock(MetricsConfiguration.class),
-                    mock(MetricsConfiguration.class),
+                natService,
-                    natService,
+                new HashMap<>(),
-                    new HashMap<>(),
+                folder,
-                    folder,
+                mock(EthPeers.class),
-                    mock(EthPeers.class),
+                vertx,
-                    vertx,
+                mock(ApiConfiguration.class),
-                    mock(ApiConfiguration.class),
+                Optional.empty());
-                    Optional.empty()));
     final JsonRpcHttpService jsonRpcHttpService =
         new JsonRpcHttpService(
             vertx,
@@ -302,8 +300,7 @@ public class JsonRpcHttpServiceRpcApisTest {
       final WebSocketConfiguration webSocketConfiguration,
       final P2PNetwork p2pNetwork,
       final MetricsConfiguration metricsConfiguration,
-      final NatService natService)
+      final NatService natService) {
-      throws Exception {
     final Set<Capability> supportedCapabilities = new HashSet<>();
     supportedCapabilities.add(EthProtocol.ETH62);
     supportedCapabilities.add(EthProtocol.ETH63);
@@ -311,37 +308,36 @@ public class JsonRpcHttpServiceRpcApisTest {
     webSocketConfiguration.setPort(0);
 
     final Map<String, JsonRpcMethod> rpcMethods =
-        spy(
+        new JsonRpcMethodsFactory()
-            new JsonRpcMethodsFactory()
+            .methods(
-                .methods(
+                CLIENT_VERSION,
-                    CLIENT_VERSION,
+                NETWORK_ID,
-                    NETWORK_ID,
+                new StubGenesisConfigOptions(),
-                    new StubGenesisConfigOptions(),
+                p2pNetwork,
-                    p2pNetwork,
+                blockchainQueries,
-                    blockchainQueries,
+                mock(Synchronizer.class),
-                    mock(Synchronizer.class),
+                ProtocolScheduleFixture.MAINNET,
-                    ProtocolScheduleFixture.MAINNET,
+                mock(ProtocolContext.class),
-                    mock(ProtocolContext.class),
+                mock(FilterManager.class),
-                    mock(FilterManager.class),
+                mock(TransactionPool.class),
-                    mock(TransactionPool.class),
+                mock(MiningParameters.class),
-                    mock(MiningParameters.class),
+                mock(PoWMiningCoordinator.class),
-                    mock(PoWMiningCoordinator.class),
+                new NoOpMetricsSystem(),
-                    new NoOpMetricsSystem(),
+                supportedCapabilities,
-                    supportedCapabilities,
+                Optional.of(mock(AccountLocalConfigPermissioningController.class)),
-                    Optional.of(mock(AccountLocalConfigPermissioningController.class)),
+                Optional.of(mock(NodeLocalConfigPermissioningController.class)),
-                    Optional.of(mock(NodeLocalConfigPermissioningController.class)),
+                jsonRpcConfiguration.getRpcApis(),
-                    jsonRpcConfiguration.getRpcApis(),
+                mock(PrivacyParameters.class),
-                    mock(PrivacyParameters.class),
+                jsonRpcConfiguration,
-                    jsonRpcConfiguration,
+                webSocketConfiguration,
-                    webSocketConfiguration,
+                metricsConfiguration,
-                    metricsConfiguration,
+                natService,
-                    natService,
+                new HashMap<>(),
-                    new HashMap<>(),
+                folder,
-                    folder,
+                mock(EthPeers.class),
-                    mock(EthPeers.class),
+                vertx,
-                    vertx,
+                mock(ApiConfiguration.class),
-                    mock(ApiConfiguration.class),
+                Optional.empty());
-                    Optional.empty()));
     final JsonRpcHttpService jsonRpcHttpService =
         new JsonRpcHttpService(
             vertx,
@@ -425,8 +421,7 @@ public class JsonRpcHttpServiceRpcApisTest {
         "{\"jsonrpc\":\"2.0\",\"id\":" + Json.encode(id) + ",\"method\":\"net_services\"}", JSON);
   }
 
-  public JsonRpcHttpService getJsonRpcHttpService(final boolean[] enabledNetServices)
+  public JsonRpcHttpService getJsonRpcHttpService(final boolean[] enabledNetServices) {
-      throws Exception {
 
     JsonRpcConfiguration jsonRpcConfiguration = JsonRpcConfiguration.createDefault();
     WebSocketConfiguration webSocketConfiguration = WebSocketConfiguration.createDefault();

ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/jsonrpc/JsonRpcHttpServiceTest.java

@@ -17,10 +17,7 @@ package org.hyperledger.besu.ethereum.api.jsonrpc;
 import static org.assertj.core.api.Assertions.assertThat;
 import static org.mockito.ArgumentMatchers.any;
 import static org.mockito.ArgumentMatchers.eq;
-import static org.mockito.Mockito.doReturn;
 import static org.mockito.Mockito.mock;
-import static org.mockito.Mockito.reset;
-import static org.mockito.Mockito.verify;
 import static org.mockito.Mockito.when;
 
 import org.hyperledger.besu.datatypes.Address;
@@ -1389,65 +1386,68 @@ public class JsonRpcHttpServiceTest extends JsonRpcHttpServiceTestBase {
                 + "\"}",
             JSON);
 
-    when(rpcMethods.get(any(String.class))).thenReturn(null);
+    try (var unused = disableRpcMethod(methodName)) {
-    when(rpcMethods.containsKey(any(String.class))).thenReturn(false);
 
-    try (final Response resp = client.newCall(buildPostRequest(body)).execute()) {
+      try (final Response resp = client.newCall(buildPostRequest(body)).execute()) {
-      assertThat(resp.code()).isEqualTo(200);
+        assertThat(resp.code()).isEqualTo(200);
-      final JsonObject json = new JsonObject(resp.body().string());
+        final JsonObject json = new JsonObject(resp.body().string());
-      final RpcErrorType expectedError = RpcErrorType.METHOD_NOT_ENABLED;
+        final RpcErrorType expectedError = RpcErrorType.METHOD_NOT_ENABLED;
-      testHelper.assertValidJsonRpcError(
+        testHelper.assertValidJsonRpcError(
-          json, id, expectedError.getCode(), expectedError.getMessage());
+            json, id, expectedError.getCode(), expectedError.getMessage());
+      }
     }
-
-    verify(rpcMethods).containsKey(methodName);
-    verify(rpcMethods).get(methodName);
-
-    reset(rpcMethods);
   }
 
   @Test
   public void exceptionallyHandleJsonSingleRequest() throws Exception {
+    final String methodName = "foo";
     final JsonRpcMethod jsonRpcMethod = mock(JsonRpcMethod.class);
-    when(jsonRpcMethod.getName()).thenReturn("foo");
+    when(jsonRpcMethod.getName()).thenReturn(methodName);
     when(jsonRpcMethod.response(any())).thenThrow(new RuntimeException("test exception"));
 
-    doReturn(jsonRpcMethod).when(rpcMethods).get("foo");
+    try (var unused = addRpcMethod(methodName, jsonRpcMethod)) {
 
-    final RequestBody body =
+      final RequestBody body =
-        RequestBody.create("{\"jsonrpc\":\"2.0\",\"id\":\"666\",\"method\":\"foo\"}", JSON);
+          RequestBody.create(
+              "{\"jsonrpc\":\"2.0\",\"id\":\"666\",\"method\":\"" + methodName + "\"}", JSON);
 
-    try (final Response resp = client.newCall(buildPostRequest(body)).execute()) {
+      try (final Response resp = client.newCall(buildPostRequest(body)).execute()) {
-      assertThat(resp.code()).isEqualTo(200);
+        assertThat(resp.code()).isEqualTo(200);
-      final JsonObject json = new JsonObject(resp.body().string());
+        final JsonObject json = new JsonObject(resp.body().string());
-      final RpcErrorType expectedError = RpcErrorType.INTERNAL_ERROR;
+        final RpcErrorType expectedError = RpcErrorType.INTERNAL_ERROR;
-      testHelper.assertValidJsonRpcError(
+        testHelper.assertValidJsonRpcError(
-          json, "666", expectedError.getCode(), expectedError.getMessage());
+            json, "666", expectedError.getCode(), expectedError.getMessage());
+      }
     }
   }
 
   @Test
   public void exceptionallyHandleJsonBatchRequest() throws Exception {
+    final String methodName = "foo";
     final JsonRpcMethod jsonRpcMethod = mock(JsonRpcMethod.class);
-    when(jsonRpcMethod.getName()).thenReturn("foo");
+    when(jsonRpcMethod.getName()).thenReturn(methodName);
     when(jsonRpcMethod.response(any())).thenThrow(new RuntimeException("test exception"));
-    doReturn(jsonRpcMethod).when(rpcMethods).get("foo");
 
-    final RequestBody body =
+    try (var unused = addRpcMethod(methodName, jsonRpcMethod)) {
-        RequestBody.create(
-            "[{\"jsonrpc\":\"2.0\",\"id\":\"000\",\"method\":\"web3_clientVersion\"},"
-                + "{\"jsonrpc\":\"2.0\",\"id\":\"111\",\"method\":\"foo\"},"
-                + "{\"jsonrpc\":\"2.0\",\"id\":\"222\",\"method\":\"net_version\"}]",
-            JSON);
 
-    try (final Response resp = client.newCall(buildPostRequest(body)).execute()) {
+      final RequestBody body =
-      assertThat(resp.code()).isEqualTo(200);
+          RequestBody.create(
-      final JsonArray array = new JsonArray(resp.body().string());
+              "[{\"jsonrpc\":\"2.0\",\"id\":\"000\",\"method\":\"web3_clientVersion\"},"
-      testHelper.assertValidJsonRpcResult(array.getJsonObject(0), "000");
+                  + "{\"jsonrpc\":\"2.0\",\"id\":\"111\",\"method\":\""
-      final RpcErrorType expectedError = RpcErrorType.INTERNAL_ERROR;
+                  + methodName
-      testHelper.assertValidJsonRpcError(
+                  + "\"},"
-          array.getJsonObject(1), "111", expectedError.getCode(), expectedError.getMessage());
+                  + "{\"jsonrpc\":\"2.0\",\"id\":\"222\",\"method\":\"net_version\"}]",
-      testHelper.assertValidJsonRpcResult(array.getJsonObject(2), "222");
+              JSON);
+
+      try (final Response resp = client.newCall(buildPostRequest(body)).execute()) {
+        assertThat(resp.code()).isEqualTo(200);
+        final JsonArray array = new JsonArray(resp.body().string());
+        testHelper.assertValidJsonRpcResult(array.getJsonObject(0), "000");
+        final RpcErrorType expectedError = RpcErrorType.INTERNAL_ERROR;
+        testHelper.assertValidJsonRpcError(
+            array.getJsonObject(1), "111", expectedError.getCode(), expectedError.getMessage());
+        testHelper.assertValidJsonRpcResult(array.getJsonObject(2), "222");
+      }
     }
   }
 

ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/jsonrpc/JsonRpcHttpServiceTestBase.java

@@ -16,7 +16,6 @@
 package org.hyperledger.besu.ethereum.api.jsonrpc;
 
 import static org.mockito.Mockito.mock;
-import static org.mockito.Mockito.spy;
 
 import org.hyperledger.besu.config.StubGenesisConfigOptions;
 import org.hyperledger.besu.ethereum.ProtocolContext;
@@ -72,8 +71,9 @@ public class JsonRpcHttpServiceTestBase {
   protected final JsonRpcTestHelper testHelper = new JsonRpcTestHelper();
 
   private static final Vertx vertx = Vertx.vertx();
-
   protected static Map<String, JsonRpcMethod> rpcMethods;
+  private static Map<String, JsonRpcMethod> disabledRpcMethods;
+  private static Set<String> addedRpcMethods;
   protected static JsonRpcHttpService service;
   protected static OkHttpClient client;
   protected static String baseUrl;
@@ -106,39 +106,41 @@ public class JsonRpcHttpServiceTestBase {
     supportedCapabilities.add(EthProtocol.ETH63);
 
     rpcMethods =
-        spy(
+        new JsonRpcMethodsFactory()
-            new JsonRpcMethodsFactory()
+            .methods(
-                .methods(
+                CLIENT_VERSION,
-                    CLIENT_VERSION,
+                CHAIN_ID,
-                    CHAIN_ID,
+                new StubGenesisConfigOptions(),
-                    new StubGenesisConfigOptions(),
+                peerDiscoveryMock,
-                    peerDiscoveryMock,
+                blockchainQueries,
-                    blockchainQueries,
+                synchronizer,
-                    synchronizer,
+                MainnetProtocolSchedule.fromConfig(
-                    MainnetProtocolSchedule.fromConfig(
+                    new StubGenesisConfigOptions().constantinopleBlock(0).chainId(CHAIN_ID),
-                        new StubGenesisConfigOptions().constantinopleBlock(0).chainId(CHAIN_ID),
+                    EvmConfiguration.DEFAULT),
-                        EvmConfiguration.DEFAULT),
+                mock(ProtocolContext.class),
-                    mock(ProtocolContext.class),
+                mock(FilterManager.class),
-                    mock(FilterManager.class),
+                mock(TransactionPool.class),
-                    mock(TransactionPool.class),
+                mock(MiningParameters.class),
-                    mock(MiningParameters.class),
+                mock(PoWMiningCoordinator.class),
-                    mock(PoWMiningCoordinator.class),
+                new NoOpMetricsSystem(),
-                    new NoOpMetricsSystem(),
+                supportedCapabilities,
-                    supportedCapabilities,
+                Optional.of(mock(AccountLocalConfigPermissioningController.class)),
-                    Optional.of(mock(AccountLocalConfigPermissioningController.class)),
+                Optional.of(mock(NodeLocalConfigPermissioningController.class)),
-                    Optional.of(mock(NodeLocalConfigPermissioningController.class)),
+                JSON_RPC_APIS,
-                    JSON_RPC_APIS,
+                mock(PrivacyParameters.class),
-                    mock(PrivacyParameters.class),
+                mock(JsonRpcConfiguration.class),
-                    mock(JsonRpcConfiguration.class),
+                mock(WebSocketConfiguration.class),
-                    mock(WebSocketConfiguration.class),
+                mock(MetricsConfiguration.class),
-                    mock(MetricsConfiguration.class),
+                natService,
-                    natService,
+                new HashMap<>(),
-                    new HashMap<>(),
+                folder,
-                    folder,
+                ethPeersMock,
-                    ethPeersMock,
+                vertx,
-                    vertx,
+                mock(ApiConfiguration.class),
-                    mock(ApiConfiguration.class),
+                Optional.empty());
-                    Optional.empty()));
+    disabledRpcMethods = new HashMap<>();
+    addedRpcMethods = new HashSet<>();
+
     service = createJsonRpcHttpService(createLimitedJsonRpcConfig());
     service.start().join();
 
@@ -189,6 +191,22 @@ public class JsonRpcHttpServiceTestBase {
     return new Request.Builder().get().url(baseUrl + path).build();
   }
 
+  protected AutoCloseable disableRpcMethod(final String methodName) {
+    disabledRpcMethods.put(methodName, rpcMethods.remove(methodName));
+    return () -> resetRpcMethods();
+  }
+
+  protected AutoCloseable addRpcMethod(final String methodName, final JsonRpcMethod method) {
+    rpcMethods.put(methodName, method);
+    addedRpcMethods.add(methodName);
+    return () -> resetRpcMethods();
+  }
+
+  protected void resetRpcMethods() {
+    disabledRpcMethods.forEach(rpcMethods::put);
+    addedRpcMethods.forEach(rpcMethods::remove);
+  }
+
   /** Tears down the HTTP server. */
   @AfterAll
   public static void shutdownServer() {

ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/jsonrpc/JsonRpcHttpServiceTlsClientAuthTest.java

@@ -21,7 +21,6 @@ import static org.hyperledger.besu.ethereum.api.tls.KnownClientFileUtil.writeToK
 import static org.hyperledger.besu.ethereum.api.tls.TlsClientAuthConfiguration.Builder.aTlsClientAuthConfiguration;
 import static org.hyperledger.besu.ethereum.api.tls.TlsConfiguration.Builder.aTlsConfiguration;
 import static org.mockito.Mockito.mock;
-import static org.mockito.Mockito.spy;
 
 import org.hyperledger.besu.config.StubGenesisConfigOptions;
 import org.hyperledger.besu.ethereum.ProtocolContext;
@@ -112,38 +111,37 @@ public class JsonRpcHttpServiceTlsClientAuthTest {
     supportedCapabilities.add(EthProtocol.ETH63);
 
     rpcMethods =
-        spy(
+        new JsonRpcMethodsFactory()
-            new JsonRpcMethodsFactory()
+            .methods(
-                .methods(
+                CLIENT_VERSION,
-                    CLIENT_VERSION,
+                CHAIN_ID,
-                    CHAIN_ID,
+                new StubGenesisConfigOptions(),
-                    new StubGenesisConfigOptions(),
+                peerDiscoveryMock,
-                    peerDiscoveryMock,
+                blockchainQueries,
-                    blockchainQueries,
+                synchronizer,
-                    synchronizer,
+                MainnetProtocolSchedule.fromConfig(
-                    MainnetProtocolSchedule.fromConfig(
+                    new StubGenesisConfigOptions().constantinopleBlock(0).chainId(CHAIN_ID)),
-                        new StubGenesisConfigOptions().constantinopleBlock(0).chainId(CHAIN_ID)),
+                mock(ProtocolContext.class),
-                    mock(ProtocolContext.class),
+                mock(FilterManager.class),
-                    mock(FilterManager.class),
+                mock(TransactionPool.class),
-                    mock(TransactionPool.class),
+                mock(MiningParameters.class),
-                    mock(MiningParameters.class),
+                mock(PoWMiningCoordinator.class),
-                    mock(PoWMiningCoordinator.class),
+                new NoOpMetricsSystem(),
-                    new NoOpMetricsSystem(),
+                supportedCapabilities,
-                    supportedCapabilities,
+                Optional.of(mock(AccountLocalConfigPermissioningController.class)),
-                    Optional.of(mock(AccountLocalConfigPermissioningController.class)),
+                Optional.of(mock(NodeLocalConfigPermissioningController.class)),
-                    Optional.of(mock(NodeLocalConfigPermissioningController.class)),
+                DEFAULT_RPC_APIS,
-                    DEFAULT_RPC_APIS,
+                mock(PrivacyParameters.class),
-                    mock(PrivacyParameters.class),
+                mock(JsonRpcConfiguration.class),
-                    mock(JsonRpcConfiguration.class),
+                mock(WebSocketConfiguration.class),
-                    mock(WebSocketConfiguration.class),
+                mock(MetricsConfiguration.class),
-                    mock(MetricsConfiguration.class),
+                natService,
-                    natService,
+                Collections.emptyMap(),
-                    Collections.emptyMap(),
+                folder,
-                    folder,
+                mock(EthPeers.class),
-                    mock(EthPeers.class),
+                vertx,
-                    vertx,
+                mock(ApiConfiguration.class),
-                    mock(ApiConfiguration.class),
+                Optional.empty());
-                    Optional.empty()));
 
     System.setProperty("javax.net.ssl.trustStore", CLIENT_AS_CA_CERT.getKeyStoreFile().toString());
     System.setProperty(

ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/jsonrpc/JsonRpcHttpServiceTlsMisconfigurationTest.java

@@ -20,7 +20,6 @@ import static org.hyperledger.besu.ethereum.api.tls.KnownClientFileUtil.writeToK
 import static org.hyperledger.besu.ethereum.api.tls.TlsClientAuthConfiguration.Builder.aTlsClientAuthConfiguration;
 import static org.hyperledger.besu.ethereum.api.tls.TlsConfiguration.Builder.aTlsConfiguration;
 import static org.mockito.Mockito.mock;
-import static org.mockito.Mockito.spy;
 
 import org.hyperledger.besu.config.StubGenesisConfigOptions;
 import org.hyperledger.besu.ethereum.ProtocolContext;
@@ -100,38 +99,37 @@ class JsonRpcHttpServiceTlsMisconfigurationTest {
     supportedCapabilities.add(EthProtocol.ETH63);
 
     rpcMethods =
-        spy(
+        new JsonRpcMethodsFactory()
-            new JsonRpcMethodsFactory()
+            .methods(
-                .methods(
+                CLIENT_VERSION,
-                    CLIENT_VERSION,
+                CHAIN_ID,
-                    CHAIN_ID,
+                new StubGenesisConfigOptions(),
-                    new StubGenesisConfigOptions(),
+                peerDiscoveryMock,
-                    peerDiscoveryMock,
+                blockchainQueries,
-                    blockchainQueries,
+                synchronizer,
-                    synchronizer,
+                MainnetProtocolSchedule.fromConfig(
-                    MainnetProtocolSchedule.fromConfig(
+                    new StubGenesisConfigOptions().constantinopleBlock(0).chainId(CHAIN_ID)),
-                        new StubGenesisConfigOptions().constantinopleBlock(0).chainId(CHAIN_ID)),
+                mock(ProtocolContext.class),
-                    mock(ProtocolContext.class),
+                mock(FilterManager.class),
-                    mock(FilterManager.class),
+                mock(TransactionPool.class),
-                    mock(TransactionPool.class),
+                mock(MiningParameters.class),
-                    mock(MiningParameters.class),
+                mock(PoWMiningCoordinator.class),
-                    mock(PoWMiningCoordinator.class),
+                new NoOpMetricsSystem(),
-                    new NoOpMetricsSystem(),
+                supportedCapabilities,
-                    supportedCapabilities,
+                Optional.of(mock(AccountLocalConfigPermissioningController.class)),
-                    Optional.of(mock(AccountLocalConfigPermissioningController.class)),
+                Optional.of(mock(NodeLocalConfigPermissioningController.class)),
-                    Optional.of(mock(NodeLocalConfigPermissioningController.class)),
+                DEFAULT_RPC_APIS,
-                    DEFAULT_RPC_APIS,
+                mock(PrivacyParameters.class),
-                    mock(PrivacyParameters.class),
+                mock(JsonRpcConfiguration.class),
-                    mock(JsonRpcConfiguration.class),
+                mock(WebSocketConfiguration.class),
-                    mock(WebSocketConfiguration.class),
+                mock(MetricsConfiguration.class),
-                    mock(MetricsConfiguration.class),
+                natService,
-                    natService,
+                Collections.emptyMap(),
-                    Collections.emptyMap(),
+                tempDir.getRoot(),
-                    tempDir.getRoot(),
+                mock(EthPeers.class),
-                    mock(EthPeers.class),
+                vertx,
-                    vertx,
+                mock(ApiConfiguration.class),
-                    mock(ApiConfiguration.class),
+                Optional.empty());
-                    Optional.empty()));
   }
 
   @AfterEach

ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/jsonrpc/JsonRpcHttpServiceTlsTest.java

@@ -20,7 +20,6 @@ import static org.assertj.core.api.Assertions.assertThat;
 import static org.hyperledger.besu.ethereum.api.jsonrpc.RpcApis.DEFAULT_RPC_APIS;
 import static org.hyperledger.besu.ethereum.api.tls.TlsConfiguration.Builder.aTlsConfiguration;
 import static org.mockito.Mockito.mock;
-import static org.mockito.Mockito.spy;
 
 import org.hyperledger.besu.config.StubGenesisConfigOptions;
 import org.hyperledger.besu.ethereum.ProtocolContext;
@@ -101,38 +100,37 @@ public class JsonRpcHttpServiceTlsTest {
     supportedCapabilities.add(EthProtocol.ETH63);
 
     rpcMethods =
-        spy(
+        new JsonRpcMethodsFactory()
-            new JsonRpcMethodsFactory()
+            .methods(
-                .methods(
+                CLIENT_VERSION,
-                    CLIENT_VERSION,
+                CHAIN_ID,
-                    CHAIN_ID,
+                new StubGenesisConfigOptions(),
-                    new StubGenesisConfigOptions(),
+                peerDiscoveryMock,
-                    peerDiscoveryMock,
+                blockchainQueries,
-                    blockchainQueries,
+                synchronizer,
-                    synchronizer,
+                MainnetProtocolSchedule.fromConfig(
-                    MainnetProtocolSchedule.fromConfig(
+                    new StubGenesisConfigOptions().constantinopleBlock(0).chainId(CHAIN_ID)),
-                        new StubGenesisConfigOptions().constantinopleBlock(0).chainId(CHAIN_ID)),
+                mock(ProtocolContext.class),
-                    mock(ProtocolContext.class),
+                mock(FilterManager.class),
-                    mock(FilterManager.class),
+                mock(TransactionPool.class),
-                    mock(TransactionPool.class),
+                mock(MiningParameters.class),
-                    mock(MiningParameters.class),
+                mock(PoWMiningCoordinator.class),
-                    mock(PoWMiningCoordinator.class),
+                new NoOpMetricsSystem(),
-                    new NoOpMetricsSystem(),
+                supportedCapabilities,
-                    supportedCapabilities,
+                Optional.of(mock(AccountLocalConfigPermissioningController.class)),
-                    Optional.of(mock(AccountLocalConfigPermissioningController.class)),
+                Optional.of(mock(NodeLocalConfigPermissioningController.class)),
-                    Optional.of(mock(NodeLocalConfigPermissioningController.class)),
+                DEFAULT_RPC_APIS,
-                    DEFAULT_RPC_APIS,
+                mock(PrivacyParameters.class),
-                    mock(PrivacyParameters.class),
+                mock(JsonRpcConfiguration.class),
-                    mock(JsonRpcConfiguration.class),
+                mock(WebSocketConfiguration.class),
-                    mock(WebSocketConfiguration.class),
+                mock(MetricsConfiguration.class),
-                    mock(MetricsConfiguration.class),
+                natService,
-                    natService,
+                Collections.emptyMap(),
-                    Collections.emptyMap(),
+                folder,
-                    folder,
+                mock(EthPeers.class),
-                    mock(EthPeers.class),
+                vertx,
-                    vertx,
+                mock(ApiConfiguration.class),
-                    mock(ApiConfiguration.class),
+                Optional.empty());
-                    Optional.empty()));
     service = createJsonRpcHttpService(createJsonRpcConfig());
     service.start().join();
     baseUrl = service.url();

ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/DebugStandardTraceBlockToFileTest.java

@@ -18,11 +18,12 @@ import static org.assertj.core.api.Assertions.assertThat;
 import static org.mockito.ArgumentMatchers.any;
 import static org.mockito.ArgumentMatchers.eq;
 import static org.mockito.Mockito.mock;
-import static org.mockito.Mockito.spy;
 import static org.mockito.Mockito.when;
 
 import org.hyperledger.besu.ethereum.api.jsonrpc.internal.JsonRpcRequest;
 import org.hyperledger.besu.ethereum.api.jsonrpc.internal.JsonRpcRequestContext;
+import org.hyperledger.besu.ethereum.api.jsonrpc.internal.processor.BlockTracer;
+import org.hyperledger.besu.ethereum.api.jsonrpc.internal.processor.Tracer;
 import org.hyperledger.besu.ethereum.api.jsonrpc.internal.processor.TransactionTracer;
 import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcSuccessResponse;
 import org.hyperledger.besu.ethereum.api.query.BlockchainQueries;
@@ -30,28 +31,23 @@ import org.hyperledger.besu.ethereum.chain.Blockchain;
 import org.hyperledger.besu.ethereum.core.Block;
 import org.hyperledger.besu.ethereum.core.BlockDataGenerator;
 import org.hyperledger.besu.ethereum.core.MutableWorldState;
-import org.hyperledger.besu.ethereum.worldstate.WorldStateArchive;
 
 import java.nio.file.Path;
-import java.util.ArrayList;
 import java.util.HashMap;
 import java.util.List;
 import java.util.Optional;
+import java.util.function.Function;
 
 import org.junit.jupiter.api.Test;
 import org.junit.jupiter.api.io.TempDir;
-import org.mockito.Answers;
 
 public class DebugStandardTraceBlockToFileTest {
 
   // this tempDir is deliberately static
   @TempDir private static Path folder;
 
-  private final WorldStateArchive archive =
-      mock(WorldStateArchive.class, Answers.RETURNS_DEEP_STUBS);
   private final Blockchain blockchain = mock(Blockchain.class);
-  private final BlockchainQueries blockchainQueries =
+  private final BlockchainQueries blockchainQueries = mock(BlockchainQueries.class);
-      spy(new BlockchainQueries(blockchain, archive));
   private final TransactionTracer transactionTracer = mock(TransactionTracer.class);
   private final DebugStandardTraceBlockToFile debugStandardTraceBlockToFile =
       new DebugStandardTraceBlockToFile(() -> transactionTracer, blockchainQueries, folder);
@@ -76,20 +72,26 @@ public class DebugStandardTraceBlockToFileTest {
         new JsonRpcRequestContext(
             new JsonRpcRequest("2.0", "debug_standardTraceBlockToFile", params));
 
-    final List<String> paths = new ArrayList<>();
+    final List<String> paths = List.of("path-1");
-    paths.add("path-1");
-
-    when(blockchainQueries.getBlockchain()).thenReturn(blockchain);
 
     when(blockchain.getBlockByHash(block.getHash())).thenReturn(Optional.of(block));
     when(blockchain.getBlockHeader(genesis.getHash())).thenReturn(Optional.of(genesis.getHeader()));
+    when(blockchainQueries.getBlockchain()).thenReturn(blockchain);
+
+    when(blockchainQueries.getAndMapWorldState(any(), any()))
+        .thenAnswer(
+            invocationOnMock -> {
+              Function<Tracer.TraceableState, ? extends Optional<BlockTracer>> mapper =
+                  invocationOnMock.getArgument(1);
+              return mapper.apply(mock(Tracer.TraceableState.class));
+            });
 
     when(transactionTracer.traceTransactionToFile(
             any(MutableWorldState.class), eq(block.getHash()), any(), any()))
         .thenReturn(paths);
     final JsonRpcSuccessResponse response =
         (JsonRpcSuccessResponse) debugStandardTraceBlockToFile.response(request);
-    final List result = (ArrayList) response.getResult();
+    final List result = (List) response.getResult();
 
     assertThat(result.size()).isEqualTo(1);
   }

ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/DebugTraceBlockTest.java

@@ -18,9 +18,8 @@ import static java.util.Arrays.asList;
 import static java.util.Collections.singletonList;
 import static org.assertj.core.api.Assertions.assertThat;
 import static org.mockito.ArgumentMatchers.any;
-import static org.mockito.Mockito.doAnswer;
+import static org.mockito.ArgumentMatchers.eq;
 import static org.mockito.Mockito.mock;
-import static org.mockito.Mockito.spy;
 import static org.mockito.Mockito.when;
 
 import org.hyperledger.besu.datatypes.Wei;
@@ -35,32 +34,25 @@ import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcSucces
 import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.RpcErrorType;
 import org.hyperledger.besu.ethereum.api.query.BlockWithMetadata;
 import org.hyperledger.besu.ethereum.api.query.BlockchainQueries;
-import org.hyperledger.besu.ethereum.chain.Blockchain;
 import org.hyperledger.besu.ethereum.core.Block;
 import org.hyperledger.besu.ethereum.core.BlockDataGenerator;
 import org.hyperledger.besu.ethereum.debug.TraceFrame;
 import org.hyperledger.besu.ethereum.mainnet.MainnetBlockHeaderFunctions;
 import org.hyperledger.besu.ethereum.processing.TransactionProcessingResult;
-import org.hyperledger.besu.ethereum.worldstate.WorldStateArchive;
 
 import java.util.Collection;
 import java.util.Collections;
 import java.util.Optional;
 import java.util.OptionalLong;
+import java.util.function.Function;
 
 import org.apache.tuweni.bytes.Bytes;
 import org.junit.jupiter.api.Test;
-import org.mockito.Answers;
-import org.mockito.Mockito;
 
 public class DebugTraceBlockTest {
 
   private final BlockTracer blockTracer = mock(BlockTracer.class);
-  private final WorldStateArchive archive =
+  private final BlockchainQueries blockchainQueries = mock(BlockchainQueries.class);
-      mock(WorldStateArchive.class, Answers.RETURNS_DEEP_STUBS);
-  private final Blockchain blockchain = mock(Blockchain.class);
-  private final BlockchainQueries blockchainQueries =
-      spy(new BlockchainQueries(blockchain, archive));
   private final DebugTraceBlock debugTraceBlock =
       new DebugTraceBlock(() -> blockTracer, new MainnetBlockHeaderFunctions(), blockchainQueries);
 
@@ -127,22 +119,25 @@ public class DebugTraceBlockTest {
     when(transaction2Trace.getResult()).thenReturn(transaction2Result);
     when(transaction1Result.getOutput()).thenReturn(Bytes.fromHexString("1234"));
     when(transaction2Result.getOutput()).thenReturn(Bytes.fromHexString("1234"));
-    when(blockTracer.trace(any(Tracer.TraceableState.class), Mockito.eq(block), any()))
+    when(blockTracer.trace(any(Tracer.TraceableState.class), eq(block), any()))
         .thenReturn(Optional.of(blockTrace));
 
-    when(blockchain.getBlockHeader(parentBlock.getHash()))
+    when(blockchainQueries.blockByHash(parentBlock.getHash()))
-        .thenReturn(Optional.of(parentBlock.getHeader()));
+        .thenReturn(
-    doAnswer(
+            Optional.of(
-            invocation ->
+                new BlockWithMetadata<>(
-                Optional.of(
+                    parentBlock.getHeader(),
-                    new BlockWithMetadata<>(
+                    Collections.emptyList(),
-                        parentBlock.getHeader(),
+                    Collections.emptyList(),
-                        Collections.emptyList(),
+                    parentBlock.getHeader().getDifficulty(),
-                        Collections.emptyList(),
+                    parentBlock.calculateSize())));
-                        parentBlock.getHeader().getDifficulty(),
+    when(blockchainQueries.getAndMapWorldState(eq(parentBlock.getHash()), any()))
-                        parentBlock.calculateSize())))
+        .thenAnswer(
-        .when(blockchainQueries)
+            invocationOnMock -> {
-        .blockByHash(parentBlock.getHash());
+              Function<Tracer.TraceableState, ? extends Optional<BlockTracer>> mapper =
+                  invocationOnMock.getArgument(1);
+              return mapper.apply(mock(Tracer.TraceableState.class));
+            });
 
     final JsonRpcSuccessResponse response =
         (JsonRpcSuccessResponse) debugTraceBlock.response(request);

ethereum/blockcreation/src/main/java/org/hyperledger/besu/ethereum/blockcreation/txselection/BlockTransactionSelector.java

@@ -136,7 +136,7 @@ public class BlockTransactionSelector {
     this.pluginTransactionSelector = pluginTransactionSelector;
     this.pluginOperationTracer = pluginTransactionSelector.getOperationTracer();
     blockWorldStateUpdater = worldState.updater();
-    blockTxsSelectionMaxTime = miningParameters.getUnstable().getBlockTxsSelectionMaxTime();
+    blockTxsSelectionMaxTime = miningParameters.getBlockTxsSelectionMaxTime();
   }
 
   private List<AbstractTransactionSelector> createTransactionSelectors(

ethereum/blockcreation/src/test/java/org/hyperledger/besu/ethereum/blockcreation/AbstractBlockTransactionSelectorTest.java

@@ -17,7 +17,7 @@ package org.hyperledger.besu.ethereum.blockcreation;
 import static org.assertj.core.api.Assertions.assertThat;
 import static org.assertj.core.api.Assertions.entry;
 import static org.awaitility.Awaitility.await;
-import static org.hyperledger.besu.ethereum.core.MiningParameters.Unstable.DEFAULT_NON_POA_BLOCK_TXS_SELECTION_MAX_TIME;
+import static org.hyperledger.besu.ethereum.core.MiningParameters.DEFAULT_NON_POA_BLOCK_TXS_SELECTION_MAX_TIME;
 import static org.hyperledger.besu.plugin.data.TransactionSelectionResult.BLOCK_SELECTION_TIMEOUT;
 import static org.hyperledger.besu.plugin.data.TransactionSelectionResult.PRIORITY_FEE_PER_GAS_BELOW_CURRENT_MIN;
 import static org.hyperledger.besu.plugin.data.TransactionSelectionResult.SELECTED;
@@ -54,7 +54,6 @@ import org.hyperledger.besu.ethereum.core.BlockHeaderTestFixture;
 import org.hyperledger.besu.ethereum.core.Difficulty;
 import org.hyperledger.besu.ethereum.core.ImmutableMiningParameters;
 import org.hyperledger.besu.ethereum.core.ImmutableMiningParameters.MutableInitValues;
-import org.hyperledger.besu.ethereum.core.ImmutableMiningParameters.Unstable;
 import org.hyperledger.besu.ethereum.core.InMemoryKeyValueStorageProvider;
 import org.hyperledger.besu.ethereum.core.MiningParameters;
 import org.hyperledger.besu.ethereum.core.MutableWorldState;
@@ -85,7 +84,7 @@ import org.hyperledger.besu.plugin.services.txselection.PluginTransactionSelecto
 import org.hyperledger.besu.plugin.services.txselection.PluginTransactionSelectorFactory;
 import org.hyperledger.besu.plugin.services.txselection.TransactionEvaluationContext;
 import org.hyperledger.besu.services.kvstore.InMemoryKeyValueStorage;
-import org.hyperledger.besu.util.number.Percentage;
+import org.hyperledger.besu.util.number.PositiveNumber;
 
 import java.math.BigInteger;
 import java.time.Instant;
@@ -960,8 +959,8 @@ public abstract class AbstractBlockTransactionSelectorTest {
 
     final ProcessableBlockHeader blockHeader = createBlock(301_000);
     final Address miningBeneficiary = AddressHelpers.ofValue(1);
-    final int poaMinBlockTime = 1;
+    final int poaGenesisBlockPeriod = 1;
-    final long blockTxsSelectionMaxTime = 750;
+    final int blockTxsSelectionMaxTime = 750;
 
     final List<Transaction> transactionsToInject = new ArrayList<>(3);
     for (int i = 0; i < 2; i++) {
@@ -991,9 +990,14 @@ public abstract class AbstractBlockTransactionSelectorTest {
         createBlockSelectorAndSetupTxPool(
             isPoa
                 ? createMiningParameters(
-                    Wei.ZERO, MIN_OCCUPANCY_100_PERCENT, poaMinBlockTime, Percentage.fromInt(75))
+                    Wei.ZERO,
+                    MIN_OCCUPANCY_100_PERCENT,
+                    poaGenesisBlockPeriod,
+                    PositiveNumber.fromInt(75))
                 : createMiningParameters(
-                    Wei.ZERO, MIN_OCCUPANCY_100_PERCENT, blockTxsSelectionMaxTime),
+                    Wei.ZERO,
+                    MIN_OCCUPANCY_100_PERCENT,
+                    PositiveNumber.fromInt(blockTxsSelectionMaxTime)),
             transactionProcessor,
             blockHeader,
             miningBeneficiary,
@@ -1180,33 +1184,32 @@ public abstract class AbstractBlockTransactionSelectorTest {
   }
 
   protected MiningParameters createMiningParameters(
-      final Wei minGasPrice, final double minBlockOccupancyRatio, final long txsSelectionMaxTime) {
+      final Wei minGasPrice,
+      final double minBlockOccupancyRatio,
+      final PositiveNumber txsSelectionMaxTime) {
     return ImmutableMiningParameters.builder()
         .mutableInitValues(
             MutableInitValues.builder()
                 .minTransactionGasPrice(minGasPrice)
                 .minBlockOccupancyRatio(minBlockOccupancyRatio)
                 .build())
-        .unstable(Unstable.builder().nonPoaBlockTxsSelectionMaxTime(txsSelectionMaxTime).build())
+        .nonPoaBlockTxsSelectionMaxTime(txsSelectionMaxTime)
         .build();
   }
 
   protected MiningParameters createMiningParameters(
       final Wei minGasPrice,
       final double minBlockOccupancyRatio,
-      final int minBlockTime,
+      final int genesisBlockPeriodSeconds,
-      final Percentage minBlockTimePercentage) {
+      final PositiveNumber minBlockTimePercentage) {
     return ImmutableMiningParameters.builder()
         .mutableInitValues(
             MutableInitValues.builder()
                 .minTransactionGasPrice(minGasPrice)
                 .minBlockOccupancyRatio(minBlockOccupancyRatio)
                 .build())
-        .unstable(
+        .genesisBlockPeriodSeconds(genesisBlockPeriodSeconds)
-            Unstable.builder()
+        .poaBlockTxsSelectionMaxTime(minBlockTimePercentage)
-                .minBlockTime(minBlockTime)
-                .poaBlockTxsSelectionMaxTime(minBlockTimePercentage)
-                .build())
         .build();
   }
 

ethereum/blockcreation/src/test/java/org/hyperledger/besu/ethereum/blockcreation/LondonFeeMarketBlockTransactionSelectorTest.java

@@ -16,7 +16,7 @@ package org.hyperledger.besu.ethereum.blockcreation;
 
 import static org.assertj.core.api.Assertions.assertThat;
 import static org.assertj.core.api.Assertions.entry;
-import static org.hyperledger.besu.ethereum.core.MiningParameters.Unstable.DEFAULT_NON_POA_BLOCK_TXS_SELECTION_MAX_TIME;
+import static org.hyperledger.besu.ethereum.core.MiningParameters.DEFAULT_NON_POA_BLOCK_TXS_SELECTION_MAX_TIME;
 import static org.mockito.Mockito.mock;
 
 import org.hyperledger.besu.config.GenesisConfigFile;

ethereum/core/src/main/java/org/hyperledger/besu/ethereum/chain/GenesisState.java

@@ -15,6 +15,7 @@
 package org.hyperledger.besu.ethereum.chain;
 
 import static java.util.Collections.emptyList;
+import static org.hyperledger.besu.ethereum.trie.common.GenesisWorldStateProvider.createGenesisWorldState;
 
 import org.hyperledger.besu.config.GenesisAllocation;
 import org.hyperledger.besu.config.GenesisConfigFile;
@@ -32,14 +33,10 @@ import org.hyperledger.besu.ethereum.core.MutableWorldState;
 import org.hyperledger.besu.ethereum.core.Withdrawal;
 import org.hyperledger.besu.ethereum.mainnet.ProtocolSchedule;
 import org.hyperledger.besu.ethereum.mainnet.ScheduleBasedBlockHeaderFunctions;
-import org.hyperledger.besu.ethereum.storage.keyvalue.WorldStatePreimageKeyValueStorage;
+import org.hyperledger.besu.ethereum.worldstate.DataStorageFormat;
-import org.hyperledger.besu.ethereum.trie.forest.storage.ForestWorldStateKeyValueStorage;
-import org.hyperledger.besu.ethereum.trie.forest.worldview.ForestMutableWorldState;
 import org.hyperledger.besu.evm.account.MutableAccount;
-import org.hyperledger.besu.evm.internal.EvmConfiguration;
 import org.hyperledger.besu.evm.log.LogsBloomFilter;
 import org.hyperledger.besu.evm.worldstate.WorldUpdater;
-import org.hyperledger.besu.services.kvstore.InMemoryKeyValueStorage;
 
 import java.math.BigInteger;
 import java.util.HashMap;
@@ -77,6 +74,21 @@ public final class GenesisState {
     return fromConfig(GenesisConfigFile.fromConfig(json), protocolSchedule);
   }
 
+  /**
+   * Construct a {@link GenesisState} from a JSON string.
+   *
+   * @param dataStorageFormat A {@link DataStorageFormat} describing the storage format to use
+   * @param json A JSON string describing the genesis block
+   * @param protocolSchedule A protocol Schedule associated with
+   * @return A new {@link GenesisState}.
+   */
+  public static GenesisState fromJson(
+      final DataStorageFormat dataStorageFormat,
+      final String json,
+      final ProtocolSchedule protocolSchedule) {
+    return fromConfig(dataStorageFormat, GenesisConfigFile.fromConfig(json), protocolSchedule);
+  }
+
   /**
    * Construct a {@link GenesisState} from a JSON object.
    *
@@ -86,10 +98,28 @@ public final class GenesisState {
    */
   public static GenesisState fromConfig(
       final GenesisConfigFile config, final ProtocolSchedule protocolSchedule) {
+    return fromConfig(DataStorageFormat.FOREST, config, protocolSchedule);
+  }
+
+  /**
+   * Construct a {@link GenesisState} from a JSON object.
+   *
+   * @param dataStorageFormat A {@link DataStorageFormat} describing the storage format to use
+   * @param config A {@link GenesisConfigFile} describing the genesis block.
+   * @param protocolSchedule A protocol Schedule associated with
+   * @return A new {@link GenesisState}.
+   */
+  public static GenesisState fromConfig(
+      final DataStorageFormat dataStorageFormat,
+      final GenesisConfigFile config,
+      final ProtocolSchedule protocolSchedule) {
     final List<GenesisAccount> genesisAccounts = parseAllocations(config).toList();
     final Block block =
         new Block(
-            buildHeader(config, calculateGenesisStateHash(genesisAccounts), protocolSchedule),
+            buildHeader(
+                config,
+                calculateGenesisStateHash(dataStorageFormat, genesisAccounts),
+                protocolSchedule),
             buildBody(config));
     return new GenesisState(block, genesisAccounts);
   }
@@ -133,15 +163,14 @@ public final class GenesisState {
     target.persist(rootHeader);
   }
 
-  private static Hash calculateGenesisStateHash(final List<GenesisAccount> genesisAccounts) {
+  private static Hash calculateGenesisStateHash(
-    final ForestWorldStateKeyValueStorage stateStorage =
+      final DataStorageFormat dataStorageFormat, final List<GenesisAccount> genesisAccounts) {
-        new ForestWorldStateKeyValueStorage(new InMemoryKeyValueStorage());
+    try (var worldState = createGenesisWorldState(dataStorageFormat)) {
-    final WorldStatePreimageKeyValueStorage preimageStorage =
+      writeAccountsTo(worldState, genesisAccounts, null);
-        new WorldStatePreimageKeyValueStorage(new InMemoryKeyValueStorage());
+      return worldState.rootHash();
-    final MutableWorldState worldState =
+    } catch (Exception e) {
-        new ForestMutableWorldState(stateStorage, preimageStorage, EvmConfiguration.DEFAULT);
+      throw new RuntimeException(e);
-    writeAccountsTo(worldState, genesisAccounts, null);
+    }
-    return worldState.rootHash();
   }
 
   private static BlockHeader buildHeader(

ethereum/core/src/main/java/org/hyperledger/besu/ethereum/core/MiningParameters.java

@@ -16,7 +16,7 @@ package org.hyperledger.besu.ethereum.core;
 
 import org.hyperledger.besu.datatypes.Address;
 import org.hyperledger.besu.datatypes.Wei;
-import org.hyperledger.besu.util.number.Percentage;
+import org.hyperledger.besu.util.number.PositiveNumber;
 
 import java.time.Duration;
 import java.util.Objects;
@@ -32,6 +32,10 @@ import org.immutables.value.Value;
 @Value.Immutable
 @Value.Enclosing
 public abstract class MiningParameters {
+  public static final PositiveNumber DEFAULT_NON_POA_BLOCK_TXS_SELECTION_MAX_TIME =
+      PositiveNumber.fromInt((int) Duration.ofSeconds(5).toMillis());
+  public static final PositiveNumber DEFAULT_POA_BLOCK_TXS_SELECTION_MAX_TIME =
+      PositiveNumber.fromInt(75);
   public static final MiningParameters MINING_DISABLED =
       ImmutableMiningParameters.builder()
           .mutableInitValues(
@@ -130,6 +134,28 @@ public abstract class MiningParameters {
     return 8008;
   }
 
+  @Value.Default
+  public PositiveNumber getNonPoaBlockTxsSelectionMaxTime() {
+    return DEFAULT_NON_POA_BLOCK_TXS_SELECTION_MAX_TIME;
+  }
+
+  @Value.Default
+  public PositiveNumber getPoaBlockTxsSelectionMaxTime() {
+    return DEFAULT_POA_BLOCK_TXS_SELECTION_MAX_TIME;
+  }
+
+  public abstract OptionalInt getGenesisBlockPeriodSeconds();
+
+  @Value.Derived
+  public long getBlockTxsSelectionMaxTime() {
+    if (getGenesisBlockPeriodSeconds().isPresent()) {
+      return (TimeUnit.SECONDS.toMillis(getGenesisBlockPeriodSeconds().getAsInt())
+              * getPoaBlockTxsSelectionMaxTime().getValue())
+          / 100;
+    }
+    return getNonPoaBlockTxsSelectionMaxTime().getValue();
+  }
+
   @Value.Default
   protected MutableRuntimeValues getMutableRuntimeValues() {
     return new MutableRuntimeValues(getMutableInitValues());
@@ -266,8 +292,6 @@ public abstract class MiningParameters {
     int DEFAULT_MAX_OMMERS_DEPTH = 8;
     long DEFAULT_POS_BLOCK_CREATION_MAX_TIME = Duration.ofSeconds(12).toMillis();
     long DEFAULT_POS_BLOCK_CREATION_REPETITION_MIN_DURATION = Duration.ofMillis(500).toMillis();
-    long DEFAULT_NON_POA_BLOCK_TXS_SELECTION_MAX_TIME = Duration.ofSeconds(5).toMillis();
-    Percentage DEFAULT_POA_BLOCK_TXS_SELECTION_MAX_TIME = Percentage.fromInt(75);
 
     MiningParameters.Unstable DEFAULT = ImmutableMiningParameters.Unstable.builder().build();
 
@@ -305,27 +329,5 @@ public abstract class MiningParameters {
     default String getStratumExtranonce() {
       return "080c";
     }
-
-    @Value.Default
-    default long getNonPoaBlockTxsSelectionMaxTime() {
-      return DEFAULT_NON_POA_BLOCK_TXS_SELECTION_MAX_TIME;
-    }
-
-    @Value.Default
-    default Percentage getPoaBlockTxsSelectionMaxTime() {
-      return DEFAULT_POA_BLOCK_TXS_SELECTION_MAX_TIME;
-    }
-
-    OptionalInt getMinBlockTime();
-
-    @Value.Derived
-    default long getBlockTxsSelectionMaxTime() {
-      if (getMinBlockTime().isPresent()) {
-        return (TimeUnit.SECONDS.toMillis(getMinBlockTime().getAsInt())
-                * getPoaBlockTxsSelectionMaxTime().getValue())
-            / 100;
-      }
-      return getNonPoaBlockTxsSelectionMaxTime();
-    }
   }
 }

ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/AbstractGasLimitSpecification.java

@@ -19,7 +19,6 @@ import static com.google.common.base.Preconditions.checkArgument;
 /** Specification for the block gasLimit. */
 public abstract class AbstractGasLimitSpecification {
 
-  public static final long DEFAULT_MAX_CONSTANT_ADMUSTMENT_INCREMENT = 1024L;
   public static final long DEFAULT_MIN_GAS_LIMIT = 5000L;
   public static final long DEFAULT_MAX_GAS_LIMIT = Long.MAX_VALUE;
 

ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/FrontierTargetingGasLimitCalculator.java

@@ -23,16 +23,13 @@ public class FrontierTargetingGasLimitCalculator extends AbstractGasLimitSpecifi
     implements GasLimitCalculator {
   private static final Logger LOG =
       LoggerFactory.getLogger(FrontierTargetingGasLimitCalculator.class);
-  private final long maxConstantAdjustmentIncrement;
 
   public FrontierTargetingGasLimitCalculator() {
-    this(DEFAULT_MAX_CONSTANT_ADMUSTMENT_INCREMENT, DEFAULT_MIN_GAS_LIMIT, DEFAULT_MAX_GAS_LIMIT);
+    this(DEFAULT_MIN_GAS_LIMIT, DEFAULT_MAX_GAS_LIMIT);
   }
 
-  public FrontierTargetingGasLimitCalculator(
+  public FrontierTargetingGasLimitCalculator(final long minGasLimit, final long maxGasLimit) {
-      final long maxConstantAdjustmentIncrement, final long minGasLimit, final long maxGasLimit) {
     super(minGasLimit, maxGasLimit);
-    this.maxConstantAdjustmentIncrement = maxConstantAdjustmentIncrement;
   }
 
   @Override
@@ -55,8 +52,7 @@ public class FrontierTargetingGasLimitCalculator extends AbstractGasLimitSpecifi
   }
 
   private long adjustAmount(final long currentGasLimit) {
-    final long maxProportionalAdjustmentLimit = Math.max(deltaBound(currentGasLimit) - 1, 0);
+    return Math.max(deltaBound(currentGasLimit) - 1, 0);
-    return Math.min(maxConstantAdjustmentIncrement, maxProportionalAdjustmentLimit);
   }
 
   protected long safeAddAtMost(final long gasLimit) {

ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/LondonTargetingGasLimitCalculator.java

@@ -27,21 +27,15 @@ public class LondonTargetingGasLimitCalculator extends FrontierTargetingGasLimit
 
   public LondonTargetingGasLimitCalculator(
       final long londonForkBlock, final BaseFeeMarket feeMarket) {
-    this(
+    this(DEFAULT_MIN_GAS_LIMIT, DEFAULT_MAX_GAS_LIMIT, londonForkBlock, feeMarket);
-        DEFAULT_MAX_CONSTANT_ADMUSTMENT_INCREMENT,
-        DEFAULT_MIN_GAS_LIMIT,
-        DEFAULT_MAX_GAS_LIMIT,
-        londonForkBlock,
-        feeMarket);
   }
 
   public LondonTargetingGasLimitCalculator(
-      final long maxConstantAdjustmentIncrement,
       final long minGasLimit,
       final long maxGasLimit,
       final long londonForkBlock,
       final BaseFeeMarket feeMarket) {
-    super(maxConstantAdjustmentIncrement, minGasLimit, maxGasLimit);
+    super(minGasLimit, maxGasLimit);
     this.londonForkBlock = londonForkBlock;
     this.feeMarket = feeMarket;
   }

ethereum/core/src/main/java/org/hyperledger/besu/ethereum/storage/StorageProvider.java

@@ -17,7 +17,7 @@ package org.hyperledger.besu.ethereum.storage;
 import org.hyperledger.besu.ethereum.chain.BlockchainStorage;
 import org.hyperledger.besu.ethereum.chain.VariablesStorage;
 import org.hyperledger.besu.ethereum.mainnet.ProtocolSchedule;
-import org.hyperledger.besu.ethereum.worldstate.DataStorageFormat;
+import org.hyperledger.besu.ethereum.worldstate.DataStorageConfiguration;
 import org.hyperledger.besu.ethereum.worldstate.WorldStatePreimageStorage;
 import org.hyperledger.besu.ethereum.worldstate.WorldStateStorage;
 import org.hyperledger.besu.plugin.services.storage.KeyValueStorage;
@@ -34,7 +34,7 @@ public interface StorageProvider extends Closeable {
   BlockchainStorage createBlockchainStorage(
       ProtocolSchedule protocolSchedule, VariablesStorage variablesStorage);
 
-  WorldStateStorage createWorldStateStorage(DataStorageFormat dataStorageFormat);
+  WorldStateStorage createWorldStateStorage(DataStorageConfiguration dataStorageFormat);
 
   WorldStatePreimageStorage createWorldStatePreimageStorage();
 

ethereum/core/src/main/java/org/hyperledger/besu/ethereum/storage/keyvalue/KeyValueStorageProvider.java

@@ -21,6 +21,7 @@ import org.hyperledger.besu.ethereum.mainnet.ScheduleBasedBlockHeaderFunctions;
 import org.hyperledger.besu.ethereum.storage.StorageProvider;
 import org.hyperledger.besu.ethereum.trie.bonsai.storage.BonsaiWorldStateKeyValueStorage;
 import org.hyperledger.besu.ethereum.trie.forest.storage.ForestWorldStateKeyValueStorage;
+import org.hyperledger.besu.ethereum.worldstate.DataStorageConfiguration;
 import org.hyperledger.besu.ethereum.worldstate.DataStorageFormat;
 import org.hyperledger.besu.ethereum.worldstate.WorldStatePreimageStorage;
 import org.hyperledger.besu.ethereum.worldstate.WorldStateStorage;
@@ -75,9 +76,10 @@ public class KeyValueStorageProvider implements StorageProvider {
   }
 
   @Override
-  public WorldStateStorage createWorldStateStorage(final DataStorageFormat dataStorageFormat) {
+  public WorldStateStorage createWorldStateStorage(
-    if (dataStorageFormat.equals(DataStorageFormat.BONSAI)) {
+      final DataStorageConfiguration dataStorageConfiguration) {
-      return new BonsaiWorldStateKeyValueStorage(this, metricsSystem);
+    if (dataStorageConfiguration.getDataStorageFormat().equals(DataStorageFormat.BONSAI)) {
+      return new BonsaiWorldStateKeyValueStorage(this, metricsSystem, dataStorageConfiguration);
     } else {
       return new ForestWorldStateKeyValueStorage(
           getStorageBySegmentIdentifier(KeyValueSegmentIdentifier.WORLD_STATE));

ethereum/core/src/main/java/org/hyperledger/besu/ethereum/trie/bonsai/BonsaiWorldStateProvider.java

@@ -39,7 +39,6 @@ import org.hyperledger.besu.ethereum.worldstate.StateTrieAccountValue;
 import org.hyperledger.besu.ethereum.worldstate.WorldStateArchive;
 import org.hyperledger.besu.evm.internal.EvmConfiguration;
 import org.hyperledger.besu.evm.worldstate.WorldState;
-import org.hyperledger.besu.metrics.ObservableMetricsSystem;
 import org.hyperledger.besu.plugin.BesuContext;
 import org.hyperledger.besu.plugin.services.trielogs.TrieLog;
 
@@ -73,13 +72,11 @@ public class BonsaiWorldStateProvider implements WorldStateArchive {
       final Blockchain blockchain,
       final Optional<Long> maxLayersToLoad,
       final CachedMerkleTrieLoader cachedMerkleTrieLoader,
-      final ObservableMetricsSystem metricsSystem,
       final BesuContext pluginContext,
       final EvmConfiguration evmConfiguration,
       final TrieLogPruner trieLogPruner) {
 
-    this.cachedWorldStorageManager =
+    this.cachedWorldStorageManager = new CachedWorldStorageManager(this, worldStateStorage);
-        new CachedWorldStorageManager(this, worldStateStorage, metricsSystem);
     // TODO: de-dup constructors
     this.trieLogManager =
         new TrieLogManager(

ethereum/core/src/main/java/org/hyperledger/besu/ethereum/trie/bonsai/cache/CachedWorldStorageManager.java

@@ -22,7 +22,6 @@ import org.hyperledger.besu.ethereum.trie.bonsai.storage.BonsaiWorldStateKeyValu
 import org.hyperledger.besu.ethereum.trie.bonsai.storage.BonsaiWorldStateLayerStorage;
 import org.hyperledger.besu.ethereum.trie.bonsai.worldview.BonsaiWorldState;
 import org.hyperledger.besu.evm.internal.EvmConfiguration;
-import org.hyperledger.besu.metrics.ObservableMetricsSystem;
 
 import java.util.ArrayList;
 import java.util.Comparator;
@@ -41,7 +40,6 @@ public class CachedWorldStorageManager
   public static final long RETAINED_LAYERS = 512; // at least 256 + typical rollbacks
   private static final Logger LOG = LoggerFactory.getLogger(CachedWorldStorageManager.class);
   private final BonsaiWorldStateProvider archive;
-  private final ObservableMetricsSystem metricsSystem;
   private final EvmConfiguration evmConfiguration;
 
   private final BonsaiWorldStateKeyValueStorage rootWorldStateStorage;
@@ -51,26 +49,18 @@ public class CachedWorldStorageManager
       final BonsaiWorldStateProvider archive,
       final BonsaiWorldStateKeyValueStorage worldStateStorage,
       final Map<Bytes32, CachedBonsaiWorldView> cachedWorldStatesByHash,
-      final ObservableMetricsSystem metricsSystem,
       final EvmConfiguration evmConfiguration) {
     worldStateStorage.subscribe(this);
     this.rootWorldStateStorage = worldStateStorage;
     this.cachedWorldStatesByHash = cachedWorldStatesByHash;
     this.archive = archive;
-    this.metricsSystem = metricsSystem;
     this.evmConfiguration = evmConfiguration;
   }
 
   public CachedWorldStorageManager(
       final BonsaiWorldStateProvider archive,
-      final BonsaiWorldStateKeyValueStorage worldStateStorage,
+      final BonsaiWorldStateKeyValueStorage worldStateStorage) {
-      final ObservableMetricsSystem metricsSystem) {
+    this(archive, worldStateStorage, new ConcurrentHashMap<>(), EvmConfiguration.DEFAULT);
-    this(
-        archive,
-        worldStateStorage,
-        new ConcurrentHashMap<>(),
-        metricsSystem,
-        EvmConfiguration.DEFAULT);
   }
 
   public synchronized void addCachedLayer(
@@ -92,8 +82,7 @@ public class CachedWorldStorageManager
         cachedBonsaiWorldView
             .get()
             .updateWorldStateStorage(
-                new BonsaiSnapshotWorldStateKeyValueStorage(
+                new BonsaiSnapshotWorldStateKeyValueStorage(forWorldState.getWorldStateStorage()));
-                    forWorldState.getWorldStateStorage(), metricsSystem));
       }
     } else {
       LOG.atDebug()
@@ -106,8 +95,7 @@ public class CachedWorldStorageManager
             blockHeader.getHash(),
             new CachedBonsaiWorldView(
                 blockHeader,
-                new BonsaiSnapshotWorldStateKeyValueStorage(
+                new BonsaiSnapshotWorldStateKeyValueStorage(forWorldState.getWorldStateStorage())));
-                    forWorldState.getWorldStateStorage(), metricsSystem)));
       } else {
         // otherwise, add the layer to the cache
         cachedWorldStatesByHash.put(

ethereum/core/src/main/java/org/hyperledger/besu/ethereum/trie/bonsai/cache/NoOpCachedWorldStorageManager.java

@@ -0,0 +1,65 @@
+/*
+ * Copyright Hyperledger Besu Contributors.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+ * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations under the License.
+ *
+ * SPDX-License-Identifier: Apache-2.0
+ */
+package org.hyperledger.besu.ethereum.trie.bonsai.cache;
+
+import org.hyperledger.besu.datatypes.Hash;
+import org.hyperledger.besu.ethereum.core.BlockHeader;
+import org.hyperledger.besu.ethereum.trie.bonsai.storage.BonsaiWorldStateKeyValueStorage;
+import org.hyperledger.besu.ethereum.trie.bonsai.worldview.BonsaiWorldState;
+
+import java.util.Optional;
+import java.util.function.Function;
+
+public class NoOpCachedWorldStorageManager extends CachedWorldStorageManager {
+
+  public NoOpCachedWorldStorageManager(
+      final BonsaiWorldStateKeyValueStorage bonsaiWorldStateKeyValueStorage) {
+    super(null, bonsaiWorldStateKeyValueStorage);
+  }
+
+  @Override
+  public synchronized void addCachedLayer(
+      final BlockHeader blockHeader,
+      final Hash worldStateRootHash,
+      final BonsaiWorldState forWorldState) {
+    // no cache
+  }
+
+  @Override
+  public boolean containWorldStateStorage(final Hash blockHash) {
+    return false;
+  }
+
+  @Override
+  public Optional<BonsaiWorldState> getWorldState(final Hash blockHash) {
+    return Optional.empty();
+  }
+
+  @Override
+  public Optional<BonsaiWorldState> getNearestWorldState(final BlockHeader blockHeader) {
+    return Optional.empty();
+  }
+
+  @Override
+  public Optional<BonsaiWorldState> getHeadWorldState(
+      final Function<Hash, Optional<BlockHeader>> hashBlockHeaderFunction) {
+    return Optional.empty();
+  }
+
+  @Override
+  public void reset() {
+    // world states are not re-used
+  }
+}

ethereum/core/src/main/java/org/hyperledger/besu/ethereum/trie/bonsai/storage/BonsaiSnapshotWorldStateKeyValueStorage.java

@@ -18,7 +18,6 @@ package org.hyperledger.besu.ethereum.trie.bonsai.storage;
 import org.hyperledger.besu.datatypes.Hash;
 import org.hyperledger.besu.datatypes.StorageSlotKey;
 import org.hyperledger.besu.ethereum.trie.bonsai.storage.BonsaiWorldStateKeyValueStorage.BonsaiStorageSubscriber;
-import org.hyperledger.besu.metrics.ObservableMetricsSystem;
 import org.hyperledger.besu.plugin.services.exception.StorageException;
 import org.hyperledger.besu.plugin.services.storage.KeyValueStorage;
 import org.hyperledger.besu.plugin.services.storage.SnappableKeyValueStorage;
@@ -43,26 +42,19 @@ public class BonsaiSnapshotWorldStateKeyValueStorage extends BonsaiWorldStateKey
   public BonsaiSnapshotWorldStateKeyValueStorage(
       final BonsaiWorldStateKeyValueStorage parentWorldStateStorage,
       final SnappedKeyValueStorage segmentedWorldStateStorage,
-      final KeyValueStorage trieLogStorage,
+      final KeyValueStorage trieLogStorage) {
-      final ObservableMetricsSystem metricsSystem) {
     super(
-        parentWorldStateStorage.flatDbMode,
+        parentWorldStateStorage.flatDbStrategyProvider, segmentedWorldStateStorage, trieLogStorage);
-        parentWorldStateStorage.flatDbStrategy,
-        segmentedWorldStateStorage,
-        trieLogStorage,
-        metricsSystem);
     this.parentWorldStateStorage = parentWorldStateStorage;
     this.subscribeParentId = parentWorldStateStorage.subscribe(this);
   }
 
   public BonsaiSnapshotWorldStateKeyValueStorage(
-      final BonsaiWorldStateKeyValueStorage worldStateStorage,
+      final BonsaiWorldStateKeyValueStorage worldStateStorage) {
-      final ObservableMetricsSystem metricsSystem) {
     this(
         worldStateStorage,
         ((SnappableKeyValueStorage) worldStateStorage.composedWorldStateStorage).takeSnapshot(),
-        worldStateStorage.trieLogStorage,
+        worldStateStorage.trieLogStorage);
-        metricsSystem);
   }
 
   private boolean isClosedGet() {
@@ -78,7 +70,7 @@ public class BonsaiSnapshotWorldStateKeyValueStorage extends BonsaiWorldStateKey
     return new Updater(
         ((SnappedKeyValueStorage) composedWorldStateStorage).getSnapshotTransaction(),
         trieLogStorage.startTransaction(),
-        flatDbStrategy);
+        getFlatDbStrategy());
   }
 
   @Override

ethereum/core/src/main/java/org/hyperledger/besu/ethereum/trie/bonsai/storage/BonsaiWorldStateKeyValueStorage.java

@@ -25,14 +25,14 @@ import org.hyperledger.besu.ethereum.storage.StorageProvider;
 import org.hyperledger.besu.ethereum.storage.keyvalue.KeyValueSegmentIdentifier;
 import org.hyperledger.besu.ethereum.trie.MerkleTrie;
 import org.hyperledger.besu.ethereum.trie.bonsai.storage.flat.FlatDbStrategy;
-import org.hyperledger.besu.ethereum.trie.bonsai.storage.flat.FullFlatDbStrategy;
+import org.hyperledger.besu.ethereum.trie.bonsai.storage.flat.FlatDbStrategyProvider;
-import org.hyperledger.besu.ethereum.trie.bonsai.storage.flat.PartialFlatDbStrategy;
+import org.hyperledger.besu.ethereum.worldstate.DataStorageConfiguration;
 import org.hyperledger.besu.ethereum.worldstate.DataStorageFormat;
 import org.hyperledger.besu.ethereum.worldstate.FlatDbMode;
 import org.hyperledger.besu.ethereum.worldstate.StateTrieAccountValue;
 import org.hyperledger.besu.ethereum.worldstate.WorldStateStorage;
 import org.hyperledger.besu.evm.account.AccountStorageEntry;
-import org.hyperledger.besu.metrics.ObservableMetricsSystem;
+import org.hyperledger.besu.plugin.services.MetricsSystem;
 import org.hyperledger.besu.plugin.services.storage.KeyValueStorage;
 import org.hyperledger.besu.plugin.services.storage.KeyValueStorageTransaction;
 import org.hyperledger.besu.plugin.services.storage.SegmentedKeyValueStorage;
@@ -64,17 +64,11 @@ public class BonsaiWorldStateKeyValueStorage implements WorldStateStorage, AutoC
   public static final byte[] WORLD_BLOCK_HASH_KEY =
       "worldBlockHash".getBytes(StandardCharsets.UTF_8);
 
-  // 0x666C61744462537461747573
+  protected final FlatDbStrategyProvider flatDbStrategyProvider;
-  public static final byte[] FLAT_DB_MODE = "flatDbStatus".getBytes(StandardCharsets.UTF_8);
-
-  protected FlatDbMode flatDbMode;
-  protected FlatDbStrategy flatDbStrategy;
 
   protected final SegmentedKeyValueStorage composedWorldStateStorage;
   protected final KeyValueStorage trieLogStorage;
 
-  protected final ObservableMetricsSystem metricsSystem;
-
   private final AtomicBoolean shouldClose = new AtomicBoolean(false);
 
   protected final AtomicBoolean isClosed = new AtomicBoolean(false);
@@ -82,62 +76,27 @@ public class BonsaiWorldStateKeyValueStorage implements WorldStateStorage, AutoC
   protected final Subscribers<BonsaiStorageSubscriber> subscribers = Subscribers.create();
 
   public BonsaiWorldStateKeyValueStorage(
-      final StorageProvider provider, final ObservableMetricsSystem metricsSystem) {
+      final StorageProvider provider,
+      final MetricsSystem metricsSystem,
+      final DataStorageConfiguration dataStorageConfiguration) {
     this.composedWorldStateStorage =
         provider.getStorageBySegmentIdentifiers(
             List.of(
                 ACCOUNT_INFO_STATE, CODE_STORAGE, ACCOUNT_STORAGE_STORAGE, TRIE_BRANCH_STORAGE));
     this.trieLogStorage =
         provider.getStorageBySegmentIdentifier(KeyValueSegmentIdentifier.TRIE_LOG_STORAGE);
-    this.metricsSystem = metricsSystem;
+    this.flatDbStrategyProvider =
-    loadFlatDbStrategy();
+        new FlatDbStrategyProvider(metricsSystem, dataStorageConfiguration);
+    flatDbStrategyProvider.loadFlatDbStrategy(composedWorldStateStorage);
   }
 
   public BonsaiWorldStateKeyValueStorage(
-      final FlatDbMode flatDbMode,
+      final FlatDbStrategyProvider flatDbStrategyProvider,
-      final FlatDbStrategy flatDbStrategy,
       final SegmentedKeyValueStorage composedWorldStateStorage,
-      final KeyValueStorage trieLogStorage,
+      final KeyValueStorage trieLogStorage) {
-      final ObservableMetricsSystem metricsSystem) {
+    this.flatDbStrategyProvider = flatDbStrategyProvider;
-    this.flatDbMode = flatDbMode;
-    this.flatDbStrategy = flatDbStrategy;
     this.composedWorldStateStorage = composedWorldStateStorage;
     this.trieLogStorage = trieLogStorage;
-    this.metricsSystem = metricsSystem;
-  }
-
-  private void loadFlatDbStrategy() {
-    // derive our flatdb strategy from db or default:
-    var newFlatDbMode = deriveFlatDbStrategy();
-
-    // if  flatDbMode is not loaded or has changed, reload flatDbStrategy
-    if (this.flatDbMode == null || !this.flatDbMode.equals(newFlatDbMode)) {
-      this.flatDbMode = newFlatDbMode;
-      if (flatDbMode == FlatDbMode.FULL) {
-        this.flatDbStrategy = new FullFlatDbStrategy(metricsSystem);
-      } else {
-        this.flatDbStrategy = new PartialFlatDbStrategy(metricsSystem);
-      }
-    }
-  }
-
-  public FlatDbMode deriveFlatDbStrategy() {
-    var flatDbMode =
-        FlatDbMode.fromVersion(
-            composedWorldStateStorage
-                .get(TRIE_BRANCH_STORAGE, FLAT_DB_MODE)
-                .map(Bytes::wrap)
-                .orElse(FlatDbMode.PARTIAL.getVersion()));
-    LOG.info("Bonsai flat db mode found {}", flatDbMode);
-
-    return flatDbMode;
-  }
-
-  public FlatDbStrategy getFlatDbStrategy() {
-    if (flatDbStrategy == null) {
-      loadFlatDbStrategy();
-    }
-    return flatDbStrategy;
   }
 
   @Override
@@ -147,7 +106,7 @@ public class BonsaiWorldStateKeyValueStorage implements WorldStateStorage, AutoC
 
   @Override
   public FlatDbMode getFlatDbMode() {
-    return flatDbMode;
+    return flatDbStrategyProvider.getFlatDbMode();
   }
 
   @Override
@@ -155,12 +114,15 @@ public class BonsaiWorldStateKeyValueStorage implements WorldStateStorage, AutoC
     if (codeHash.equals(Hash.EMPTY)) {
       return Optional.of(Bytes.EMPTY);
     } else {
-      return getFlatDbStrategy().getFlatCode(codeHash, accountHash, composedWorldStateStorage);
+      return flatDbStrategyProvider
+          .getFlatDbStrategy(composedWorldStateStorage)
+          .getFlatCode(codeHash, accountHash, composedWorldStateStorage);
     }
   }
 
   public Optional<Bytes> getAccount(final Hash accountHash) {
-    return getFlatDbStrategy()
+    return flatDbStrategyProvider
+        .getFlatDbStrategy(composedWorldStateStorage)
         .getFlatAccount(
             this::getWorldStateRootHash,
             this::getAccountStateTrieNode,
@@ -243,7 +205,8 @@ public class BonsaiWorldStateKeyValueStorage implements WorldStateStorage, AutoC
       final Supplier<Optional<Hash>> storageRootSupplier,
       final Hash accountHash,
       final StorageSlotKey storageSlotKey) {
-    return getFlatDbStrategy()
+    return flatDbStrategyProvider
+        .getFlatDbStrategy(composedWorldStateStorage)
         .getFlatStorageValueByStorageSlotKey(
             this::getWorldStateRootHash,
             storageRootSupplier,
@@ -256,14 +219,16 @@ public class BonsaiWorldStateKeyValueStorage implements WorldStateStorage, AutoC
   @Override
   public Map<Bytes32, Bytes> streamFlatAccounts(
       final Bytes startKeyHash, final Bytes32 endKeyHash, final long max) {
-    return getFlatDbStrategy()
+    return flatDbStrategyProvider
+        .getFlatDbStrategy(composedWorldStateStorage)
         .streamAccountFlatDatabase(composedWorldStateStorage, startKeyHash, endKeyHash, max);
   }
 
   @Override
   public Map<Bytes32, Bytes> streamFlatStorages(
       final Hash accountHash, final Bytes startKeyHash, final Bytes32 endKeyHash, final long max) {
-    return getFlatDbStrategy()
+    return flatDbStrategyProvider
+        .getFlatDbStrategy(composedWorldStateStorage)
         .streamStorageFlatDatabase(
             composedWorldStateStorage, accountHash, startKeyHash, endKeyHash, max);
   }
@@ -288,31 +253,23 @@ public class BonsaiWorldStateKeyValueStorage implements WorldStateStorage, AutoC
   }
 
   public void upgradeToFullFlatDbMode() {
-    final SegmentedKeyValueStorageTransaction transaction =
+    flatDbStrategyProvider.upgradeToFullFlatDbMode(composedWorldStateStorage);
-        composedWorldStateStorage.startTransaction();
-    // TODO: consider ARCHIVE mode
-    transaction.put(
-        TRIE_BRANCH_STORAGE, FLAT_DB_MODE, FlatDbMode.FULL.getVersion().toArrayUnsafe());
-    transaction.commit();
-    loadFlatDbStrategy(); // force reload of flat db reader strategy
   }
 
   public void downgradeToPartialFlatDbMode() {
-    final SegmentedKeyValueStorageTransaction transaction =
+    flatDbStrategyProvider.downgradeToPartialFlatDbMode(composedWorldStateStorage);
-        composedWorldStateStorage.startTransaction();
-    transaction.put(
-        TRIE_BRANCH_STORAGE, FLAT_DB_MODE, FlatDbMode.PARTIAL.getVersion().toArrayUnsafe());
-    transaction.commit();
-    loadFlatDbStrategy(); // force reload of flat db reader strategy
   }
 
   @Override
   public void clear() {
     subscribers.forEach(BonsaiStorageSubscriber::onClearStorage);
-    getFlatDbStrategy().clearAll(composedWorldStateStorage);
+    flatDbStrategyProvider
+        .getFlatDbStrategy(composedWorldStateStorage)
+        .clearAll(composedWorldStateStorage);
     composedWorldStateStorage.clear(TRIE_BRANCH_STORAGE);
     trieLogStorage.clear();
-    loadFlatDbStrategy(); // force reload of flat db reader strategy
+    flatDbStrategyProvider.loadFlatDbStrategy(
+        composedWorldStateStorage); // force reload of flat db reader strategy
   }
 
   @Override
@@ -324,7 +281,9 @@ public class BonsaiWorldStateKeyValueStorage implements WorldStateStorage, AutoC
   @Override
   public void clearFlatDatabase() {
     subscribers.forEach(BonsaiStorageSubscriber::onClearFlatDatabaseStorage);
-    getFlatDbStrategy().resetOnResync(composedWorldStateStorage);
+    flatDbStrategyProvider
+        .getFlatDbStrategy(composedWorldStateStorage)
+        .resetOnResync(composedWorldStateStorage);
   }
 
   @Override
@@ -332,7 +291,7 @@ public class BonsaiWorldStateKeyValueStorage implements WorldStateStorage, AutoC
     return new Updater(
         composedWorldStateStorage.startTransaction(),
         trieLogStorage.startTransaction(),
-        flatDbStrategy);
+        flatDbStrategyProvider.getFlatDbStrategy(composedWorldStateStorage));
   }
 
   @Override
@@ -359,6 +318,10 @@ public class BonsaiWorldStateKeyValueStorage implements WorldStateStorage, AutoC
     throw new RuntimeException("removeNodeAddedListener not available");
   }
 
+  public FlatDbStrategy getFlatDbStrategy() {
+    return flatDbStrategyProvider.getFlatDbStrategy(composedWorldStateStorage);
+  }
+
   public interface BonsaiUpdater extends WorldStateStorage.Updater {
     BonsaiUpdater removeCode(final Hash accountHash);
 

ethereum/core/src/main/java/org/hyperledger/besu/ethereum/trie/bonsai/storage/BonsaiWorldStateLayerStorage.java

@@ -17,7 +17,6 @@ package org.hyperledger.besu.ethereum.trie.bonsai.storage;
 
 import org.hyperledger.besu.ethereum.trie.bonsai.storage.BonsaiWorldStateKeyValueStorage.BonsaiStorageSubscriber;
 import org.hyperledger.besu.ethereum.worldstate.FlatDbMode;
-import org.hyperledger.besu.metrics.ObservableMetricsSystem;
 import org.hyperledger.besu.plugin.services.storage.KeyValueStorage;
 import org.hyperledger.besu.plugin.services.storage.SnappedKeyValueStorage;
 import org.hyperledger.besu.services.kvstore.LayeredKeyValueStorage;
@@ -29,16 +28,14 @@ public class BonsaiWorldStateLayerStorage extends BonsaiSnapshotWorldStateKeyVal
     this(
         new LayeredKeyValueStorage(parent.composedWorldStateStorage),
         parent.trieLogStorage,
-        parent,
+        parent);
-        parent.metricsSystem);
   }
 
   public BonsaiWorldStateLayerStorage(
       final SnappedKeyValueStorage composedWorldStateStorage,
       final KeyValueStorage trieLogStorage,
-      final BonsaiWorldStateKeyValueStorage parent,
+      final BonsaiWorldStateKeyValueStorage parent) {
-      final ObservableMetricsSystem metricsSystem) {
+    super(parent, composedWorldStateStorage, trieLogStorage);
-    super(parent, composedWorldStateStorage, trieLogStorage, metricsSystem);
   }
 
   @Override
@@ -51,7 +48,6 @@ public class BonsaiWorldStateLayerStorage extends BonsaiSnapshotWorldStateKeyVal
     return new BonsaiWorldStateLayerStorage(
         ((LayeredKeyValueStorage) composedWorldStateStorage).clone(),
         trieLogStorage,
-        parentWorldStateStorage,
+        parentWorldStateStorage);
-        metricsSystem);
   }
 }

ethereum/core/src/main/java/org/hyperledger/besu/ethereum/trie/bonsai/storage/flat/FlatDbStrategyProvider.java

@@ -0,0 +1,105 @@
+/*
+ * Copyright Hyperledger Besu Contributors.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+ * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations under the License.
+ *
+ * SPDX-License-Identifier: Apache-2.0
+ */
+
+package org.hyperledger.besu.ethereum.trie.bonsai.storage.flat;
+
+import static org.hyperledger.besu.ethereum.storage.keyvalue.KeyValueSegmentIdentifier.TRIE_BRANCH_STORAGE;
+
+import org.hyperledger.besu.ethereum.worldstate.DataStorageConfiguration;
+import org.hyperledger.besu.ethereum.worldstate.FlatDbMode;
+import org.hyperledger.besu.plugin.services.MetricsSystem;
+import org.hyperledger.besu.plugin.services.storage.SegmentedKeyValueStorage;
+import org.hyperledger.besu.plugin.services.storage.SegmentedKeyValueStorageTransaction;
+
+import java.nio.charset.StandardCharsets;
+
+import org.apache.tuweni.bytes.Bytes;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class FlatDbStrategyProvider {
+  private static final Logger LOG = LoggerFactory.getLogger(FlatDbStrategyProvider.class);
+
+  // 0x666C61744462537461747573
+  public static final byte[] FLAT_DB_MODE = "flatDbStatus".getBytes(StandardCharsets.UTF_8);
+  private final MetricsSystem metricsSystem;
+  protected FlatDbMode flatDbMode;
+  protected FlatDbStrategy flatDbStrategy;
+
+  public FlatDbStrategyProvider(
+      final MetricsSystem metricsSystem, final DataStorageConfiguration dataStorageConfiguration) {
+    this.metricsSystem = metricsSystem;
+  }
+
+  public void loadFlatDbStrategy(final SegmentedKeyValueStorage composedWorldStateStorage) {
+    // derive our flatdb strategy from db or default:
+    var newFlatDbMode = deriveFlatDbStrategy(composedWorldStateStorage);
+
+    // if  flatDbMode is not loaded or has changed, reload flatDbStrategy
+    if (this.flatDbMode == null || !this.flatDbMode.equals(newFlatDbMode)) {
+      this.flatDbMode = newFlatDbMode;
+      if (flatDbMode == FlatDbMode.FULL) {
+        this.flatDbStrategy = new FullFlatDbStrategy(metricsSystem);
+      } else {
+        this.flatDbStrategy = new PartialFlatDbStrategy(metricsSystem);
+      }
+    }
+  }
+
+  private FlatDbMode deriveFlatDbStrategy(
+      final SegmentedKeyValueStorage composedWorldStateStorage) {
+    var flatDbMode =
+        FlatDbMode.fromVersion(
+            composedWorldStateStorage
+                .get(TRIE_BRANCH_STORAGE, FLAT_DB_MODE)
+                .map(Bytes::wrap)
+                .orElse(FlatDbMode.PARTIAL.getVersion()));
+    LOG.info("Bonsai flat db mode found {}", flatDbMode);
+
+    return flatDbMode;
+  }
+
+  public FlatDbStrategy getFlatDbStrategy(
+      final SegmentedKeyValueStorage composedWorldStateStorage) {
+    if (flatDbStrategy == null) {
+      loadFlatDbStrategy(composedWorldStateStorage);
+    }
+    return flatDbStrategy;
+  }
+
+  public void upgradeToFullFlatDbMode(final SegmentedKeyValueStorage composedWorldStateStorage) {
+    final SegmentedKeyValueStorageTransaction transaction =
+        composedWorldStateStorage.startTransaction();
+    // TODO: consider ARCHIVE mode
+    transaction.put(
+        TRIE_BRANCH_STORAGE, FLAT_DB_MODE, FlatDbMode.FULL.getVersion().toArrayUnsafe());
+    transaction.commit();
+    loadFlatDbStrategy(composedWorldStateStorage); // force reload of flat db reader strategy
+  }
+
+  public void downgradeToPartialFlatDbMode(
+      final SegmentedKeyValueStorage composedWorldStateStorage) {
+    final SegmentedKeyValueStorageTransaction transaction =
+        composedWorldStateStorage.startTransaction();
+    transaction.put(
+        TRIE_BRANCH_STORAGE, FLAT_DB_MODE, FlatDbMode.PARTIAL.getVersion().toArrayUnsafe());
+    transaction.commit();
+    loadFlatDbStrategy(composedWorldStateStorage); // force reload of flat db reader strategy
+  }
+
+  public FlatDbMode getFlatDbMode() {
+    return flatDbMode;
+  }
+}

ethereum/core/src/main/java/org/hyperledger/besu/ethereum/trie/bonsai/trielog/NoOpTrieLogManager.java

@@ -0,0 +1,51 @@
+/*
+ * Copyright Hyperledger Besu Contributors.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+ * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations under the License.
+ *
+ * SPDX-License-Identifier: Apache-2.0
+ */
+package org.hyperledger.besu.ethereum.trie.bonsai.trielog;
+
+import org.hyperledger.besu.datatypes.Hash;
+import org.hyperledger.besu.ethereum.core.BlockHeader;
+import org.hyperledger.besu.ethereum.trie.bonsai.worldview.BonsaiWorldState;
+import org.hyperledger.besu.ethereum.trie.bonsai.worldview.BonsaiWorldStateUpdateAccumulator;
+import org.hyperledger.besu.plugin.services.trielogs.TrieLog;
+
+import java.util.Optional;
+
+public class NoOpTrieLogManager extends TrieLogManager {
+
+  public NoOpTrieLogManager() {
+    super(null, null, 0, null, TrieLogPruner.noOpTrieLogPruner());
+  }
+
+  @Override
+  public synchronized void saveTrieLog(
+      final BonsaiWorldStateUpdateAccumulator localUpdater,
+      final Hash forWorldStateRootHash,
+      final BlockHeader forBlockHeader,
+      final BonsaiWorldState forWorldState) {
+    // notify trie log added observers, synchronously
+    TrieLog trieLog = trieLogFactory.create(localUpdater, forBlockHeader);
+    trieLogObservers.forEach(o -> o.onTrieLogAdded(new TrieLogAddedEvent(trieLog)));
+  }
+
+  @Override
+  public long getMaxLayersToLoad() {
+    return 0;
+  }
+
+  @Override
+  public Optional<TrieLog> getTrieLogLayer(final Hash blockHash) {
+    return Optional.empty();
+  }
+}

ethereum/core/src/main/java/org/hyperledger/besu/ethereum/trie/bonsai/worldview/BonsaiWorldState.java

@@ -92,7 +92,7 @@ public class BonsaiWorldState
         evmConfiguration);
   }
 
-  protected BonsaiWorldState(
+  public BonsaiWorldState(
       final BonsaiWorldStateKeyValueStorage worldStateStorage,
       final CachedMerkleTrieLoader cachedMerkleTrieLoader,
       final CachedWorldStorageManager cachedWorldStorageManager,

ethereum/core/src/main/java/org/hyperledger/besu/ethereum/trie/common/GenesisWorldStateProvider.java

@@ -0,0 +1,91 @@
+/*
+ * Copyright ConsenSys AG.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+ * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations under the License.
+ *
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ */
+
+package org.hyperledger.besu.ethereum.trie.common;
+
+import org.hyperledger.besu.ethereum.core.MutableWorldState;
+import org.hyperledger.besu.ethereum.storage.keyvalue.KeyValueStorageProvider;
+import org.hyperledger.besu.ethereum.storage.keyvalue.WorldStatePreimageKeyValueStorage;
+import org.hyperledger.besu.ethereum.trie.bonsai.cache.CachedMerkleTrieLoader;
+import org.hyperledger.besu.ethereum.trie.bonsai.cache.NoOpCachedWorldStorageManager;
+import org.hyperledger.besu.ethereum.trie.bonsai.storage.BonsaiWorldStateKeyValueStorage;
+import org.hyperledger.besu.ethereum.trie.bonsai.trielog.NoOpTrieLogManager;
+import org.hyperledger.besu.ethereum.trie.bonsai.worldview.BonsaiWorldState;
+import org.hyperledger.besu.ethereum.trie.forest.storage.ForestWorldStateKeyValueStorage;
+import org.hyperledger.besu.ethereum.trie.forest.worldview.ForestMutableWorldState;
+import org.hyperledger.besu.ethereum.worldstate.DataStorageConfiguration;
+import org.hyperledger.besu.ethereum.worldstate.DataStorageFormat;
+import org.hyperledger.besu.evm.internal.EvmConfiguration;
+import org.hyperledger.besu.metrics.noop.NoOpMetricsSystem;
+import org.hyperledger.besu.services.kvstore.InMemoryKeyValueStorage;
+import org.hyperledger.besu.services.kvstore.SegmentedInMemoryKeyValueStorage;
+
+import java.util.Objects;
+
+public class GenesisWorldStateProvider {
+
+  /**
+   * Creates a Genesis world state based on the provided data storage format.
+   *
+   * @param dataStorageFormat the data storage format to use
+   * @return a mutable world state for the Genesis block
+   */
+  public static MutableWorldState createGenesisWorldState(
+      final DataStorageFormat dataStorageFormat) {
+    if (Objects.requireNonNull(dataStorageFormat) == DataStorageFormat.BONSAI) {
+      return createGenesisBonsaiWorldState();
+    } else {
+      return createGenesisForestWorldState();
+    }
+  }
+
+  /**
+   * Creates a Genesis world state using the Bonsai data storage format.
+   *
+   * @return a mutable world state for the Genesis block
+   */
+  private static MutableWorldState createGenesisBonsaiWorldState() {
+    final CachedMerkleTrieLoader cachedMerkleTrieLoader =
+        new CachedMerkleTrieLoader(new NoOpMetricsSystem());
+    final BonsaiWorldStateKeyValueStorage bonsaiWorldStateKeyValueStorage =
+        new BonsaiWorldStateKeyValueStorage(
+            new KeyValueStorageProvider(
+                segmentIdentifiers -> new SegmentedInMemoryKeyValueStorage(),
+                new InMemoryKeyValueStorage(),
+                new NoOpMetricsSystem()),
+            new NoOpMetricsSystem(),
+            DataStorageConfiguration.DEFAULT_CONFIG);
+    return new BonsaiWorldState(
+        bonsaiWorldStateKeyValueStorage,
+        cachedMerkleTrieLoader,
+        new NoOpCachedWorldStorageManager(bonsaiWorldStateKeyValueStorage),
+        new NoOpTrieLogManager(),
+        EvmConfiguration.DEFAULT);
+  }
+
+  /**
+   * Creates a Genesis world state using the Forest data storage format.
+   *
+   * @return a mutable world state for the Genesis block
+   */
+  private static MutableWorldState createGenesisForestWorldState() {
+    final ForestWorldStateKeyValueStorage stateStorage =
+        new ForestWorldStateKeyValueStorage(new InMemoryKeyValueStorage());
+    final WorldStatePreimageKeyValueStorage preimageStorage =
+        new WorldStatePreimageKeyValueStorage(new InMemoryKeyValueStorage());
+    return new ForestMutableWorldState(stateStorage, preimageStorage, EvmConfiguration.DEFAULT);
+  }
+}

ethereum/core/src/test-support/java/org/hyperledger/besu/ethereum/core/InMemoryKeyValueStorageProvider.java

@@ -14,6 +14,8 @@
  */
 package org.hyperledger.besu.ethereum.core;
 
+import static org.hyperledger.besu.ethereum.worldstate.DataStorageConfiguration.DEFAULT_BONSAI_MAX_LAYERS_TO_LOAD;
+
 import org.hyperledger.besu.ethereum.chain.Blockchain;
 import org.hyperledger.besu.ethereum.chain.DefaultBlockchain;
 import org.hyperledger.besu.ethereum.chain.MutableBlockchain;
@@ -32,7 +34,9 @@ import org.hyperledger.besu.ethereum.trie.bonsai.trielog.TrieLogPruner;
 import org.hyperledger.besu.ethereum.trie.forest.ForestWorldStateArchive;
 import org.hyperledger.besu.ethereum.trie.forest.storage.ForestWorldStateKeyValueStorage;
 import org.hyperledger.besu.ethereum.trie.forest.worldview.ForestMutableWorldState;
+import org.hyperledger.besu.ethereum.worldstate.DataStorageConfiguration;
 import org.hyperledger.besu.ethereum.worldstate.DataStorageFormat;
+import org.hyperledger.besu.ethereum.worldstate.ImmutableDataStorageConfiguration;
 import org.hyperledger.besu.evm.internal.EvmConfiguration;
 import org.hyperledger.besu.metrics.noop.NoOpMetricsSystem;
 import org.hyperledger.besu.services.kvstore.InMemoryKeyValueStorage;
@@ -96,13 +100,18 @@ public class InMemoryKeyValueStorageProvider extends KeyValueStorageProvider {
         new InMemoryKeyValueStorageProvider();
     final CachedMerkleTrieLoader cachedMerkleTrieLoader =
         new CachedMerkleTrieLoader(new NoOpMetricsSystem());
+    final DataStorageConfiguration bonsaiDataStorageConfig =
+        ImmutableDataStorageConfiguration.builder()
+            .dataStorageFormat(DataStorageFormat.BONSAI)
+            .bonsaiMaxLayersToLoad(DEFAULT_BONSAI_MAX_LAYERS_TO_LOAD)
+            .unstable(DataStorageConfiguration.Unstable.DEFAULT)
+            .build();
     return new BonsaiWorldStateProvider(
         (BonsaiWorldStateKeyValueStorage)
-            inMemoryKeyValueStorageProvider.createWorldStateStorage(DataStorageFormat.BONSAI),
+            inMemoryKeyValueStorageProvider.createWorldStateStorage(bonsaiDataStorageConfig),
         blockchain,
         Optional.empty(),
         cachedMerkleTrieLoader,
-        new NoOpMetricsSystem(),
         null,
         evmConfiguration,
         TrieLogPruner.noOpTrieLogPruner());
@@ -111,7 +120,7 @@ public class InMemoryKeyValueStorageProvider extends KeyValueStorageProvider {
   public static MutableWorldState createInMemoryWorldState() {
     final InMemoryKeyValueStorageProvider provider = new InMemoryKeyValueStorageProvider();
     return new ForestMutableWorldState(
-        provider.createWorldStateStorage(DataStorageFormat.FOREST),
+        provider.createWorldStateStorage(DataStorageConfiguration.DEFAULT_CONFIG),
         provider.createWorldStatePreimageStorage(),
         EvmConfiguration.DEFAULT);
   }

ethereum/core/src/test/java/org/hyperledger/besu/ethereum/BlockImportExceptionHandlingTest.java

@@ -44,6 +44,7 @@ import org.hyperledger.besu.ethereum.storage.StorageProvider;
 import org.hyperledger.besu.ethereum.trie.bonsai.BonsaiWorldStateProvider;
 import org.hyperledger.besu.ethereum.trie.bonsai.storage.BonsaiWorldStateKeyValueStorage;
 import org.hyperledger.besu.ethereum.trie.bonsai.worldview.BonsaiWorldState;
+import org.hyperledger.besu.ethereum.worldstate.DataStorageConfiguration;
 import org.hyperledger.besu.ethereum.worldstate.WorldStateArchive;
 import org.hyperledger.besu.ethereum.worldstate.WorldStateStorage;
 import org.hyperledger.besu.evm.internal.EvmConfiguration;
@@ -80,7 +81,8 @@ class BlockImportExceptionHandlingTest {
   private final StorageProvider storageProvider = new InMemoryKeyValueStorageProvider();
 
   private final WorldStateStorage worldStateStorage =
-      new BonsaiWorldStateKeyValueStorage(storageProvider, new NoOpMetricsSystem());
+      new BonsaiWorldStateKeyValueStorage(
+          storageProvider, new NoOpMetricsSystem(), DataStorageConfiguration.DEFAULT_CONFIG);
 
   private final WorldStateArchive worldStateArchive =
       // contains a BonsaiWorldState which we need to spy on.

ethereum/core/src/test/java/org/hyperledger/besu/ethereum/mainnet/TargetingGasLimitCalculatorTest.java

@@ -27,22 +27,6 @@ import org.junit.jupiter.api.Test;
 public class TargetingGasLimitCalculatorTest {
   private static final long ADJUSTMENT_FACTOR = 1024L;
 
-  @Test
-  public void verifyGasLimitIsIncreasedWithinLimits() {
-    FrontierTargetingGasLimitCalculator targetingGasLimitCalculator =
-        new FrontierTargetingGasLimitCalculator();
-    assertThat(targetingGasLimitCalculator.nextGasLimit(8_000_000L, 10_000_000L, 1L))
-        .isEqualTo(8_000_000L + ADJUSTMENT_FACTOR);
-  }
-
-  @Test
-  public void verifyGasLimitIsDecreasedWithinLimits() {
-    FrontierTargetingGasLimitCalculator targetingGasLimitCalculator =
-        new FrontierTargetingGasLimitCalculator();
-    assertThat(targetingGasLimitCalculator.nextGasLimit(12_000_000L, 10_000_000L, 1L))
-        .isEqualTo(12_000_000L - ADJUSTMENT_FACTOR);
-  }
-
   @Test
   public void verifyGasLimitReachesTarget() {
     final long target = 10_000_000L;
@@ -55,6 +39,33 @@ public class TargetingGasLimitCalculatorTest {
         .isEqualTo(target);
   }
 
+  @Test
+  public void verifyAdjustmentDeltas() {
+    assertDeltas(20000000L, 20019530L, 19980470L);
+    assertDeltas(40000000L, 40039061L, 39960939L);
+  }
+
+  private void assertDeltas(
+      final long gasLimit, final long expectedIncrease, final long expectedDecrease) {
+    FrontierTargetingGasLimitCalculator targetingGasLimitCalculator =
+        new FrontierTargetingGasLimitCalculator();
+    // increase
+    assertThat(targetingGasLimitCalculator.nextGasLimit(gasLimit, gasLimit * 2, 1L))
+        .isEqualTo(expectedIncrease);
+    // decrease
+    assertThat(targetingGasLimitCalculator.nextGasLimit(gasLimit, 0, 1L))
+        .isEqualTo(expectedDecrease);
+    // small decrease
+    assertThat(targetingGasLimitCalculator.nextGasLimit(gasLimit, gasLimit - 1, 1L))
+        .isEqualTo(gasLimit - 1);
+    // small increase
+    assertThat(targetingGasLimitCalculator.nextGasLimit(gasLimit, gasLimit + 1, 1L))
+        .isEqualTo(gasLimit + 1);
+    // no change
+    assertThat(targetingGasLimitCalculator.nextGasLimit(gasLimit, gasLimit, 1L))
+        .isEqualTo(gasLimit);
+  }
+
   @Test
   public void verifyMinGasLimit() {
     assertThat(AbstractGasLimitSpecification.isValidTargetGasLimit(DEFAULT_MIN_GAS_LIMIT - 1))

ethereum/core/src/test/java/org/hyperledger/besu/ethereum/trie/bonsai/AbstractIsolationTests.java

@@ -68,7 +68,9 @@ import org.hyperledger.besu.ethereum.storage.keyvalue.KeyValueStorageProviderBui
 import org.hyperledger.besu.ethereum.trie.bonsai.cache.CachedMerkleTrieLoader;
 import org.hyperledger.besu.ethereum.trie.bonsai.storage.BonsaiWorldStateKeyValueStorage;
 import org.hyperledger.besu.ethereum.trie.bonsai.trielog.TrieLogPruner;
+import org.hyperledger.besu.ethereum.worldstate.DataStorageConfiguration;
 import org.hyperledger.besu.ethereum.worldstate.DataStorageFormat;
+import org.hyperledger.besu.ethereum.worldstate.ImmutableDataStorageConfiguration;
 import org.hyperledger.besu.evm.internal.EvmConfiguration;
 import org.hyperledger.besu.metrics.noop.NoOpMetricsSystem;
 import org.hyperledger.besu.plugin.services.BesuConfiguration;
@@ -147,14 +149,19 @@ public abstract class AbstractIsolationTests {
   public void createStorage() {
     bonsaiWorldStateStorage =
         (BonsaiWorldStateKeyValueStorage)
-            createKeyValueStorageProvider().createWorldStateStorage(DataStorageFormat.BONSAI);
+            createKeyValueStorageProvider()
+                .createWorldStateStorage(
+                    ImmutableDataStorageConfiguration.builder()
+                        .dataStorageFormat(DataStorageFormat.BONSAI)
+                        .bonsaiMaxLayersToLoad(
+                            DataStorageConfiguration.DEFAULT_BONSAI_MAX_LAYERS_TO_LOAD)
+                        .build());
     archive =
         new BonsaiWorldStateProvider(
             bonsaiWorldStateStorage,
             blockchain,
             Optional.of(16L),
             new CachedMerkleTrieLoader(new NoOpMetricsSystem()),
-            new NoOpMetricsSystem(),
             null,
             EvmConfiguration.DEFAULT,
             TrieLogPruner.noOpTrieLogPruner());

ethereum/core/src/test/java/org/hyperledger/besu/ethereum/trie/bonsai/BonsaiWorldStateArchiveTest.java

@@ -44,6 +44,7 @@ import org.hyperledger.besu.ethereum.trie.bonsai.trielog.TrieLogLayer;
 import org.hyperledger.besu.ethereum.trie.bonsai.trielog.TrieLogManager;
 import org.hyperledger.besu.ethereum.trie.bonsai.trielog.TrieLogPruner;
 import org.hyperledger.besu.ethereum.trie.bonsai.worldview.BonsaiWorldState;
+import org.hyperledger.besu.ethereum.worldstate.DataStorageConfiguration;
 import org.hyperledger.besu.evm.internal.EvmConfiguration;
 import org.hyperledger.besu.metrics.noop.NoOpMetricsSystem;
 import org.hyperledger.besu.plugin.services.storage.KeyValueStorage;
@@ -106,7 +107,8 @@ class BonsaiWorldStateArchiveTest {
         new BonsaiWorldStateProvider(
             cachedWorldStorageManager,
             trieLogManager,
-            new BonsaiWorldStateKeyValueStorage(storageProvider, new NoOpMetricsSystem()),
+            new BonsaiWorldStateKeyValueStorage(
+                storageProvider, new NoOpMetricsSystem(), DataStorageConfiguration.DEFAULT_CONFIG),
             blockchain,
             new CachedMerkleTrieLoader(new NoOpMetricsSystem()),
             EvmConfiguration.DEFAULT);
@@ -119,11 +121,11 @@ class BonsaiWorldStateArchiveTest {
   void testGetMutableReturnEmptyWhenLoadMoreThanLimitLayersBack() {
     bonsaiWorldStateArchive =
         new BonsaiWorldStateProvider(
-            new BonsaiWorldStateKeyValueStorage(storageProvider, new NoOpMetricsSystem()),
+            new BonsaiWorldStateKeyValueStorage(
+                storageProvider, new NoOpMetricsSystem(), DataStorageConfiguration.DEFAULT_CONFIG),
             blockchain,
             Optional.of(512L),
             new CachedMerkleTrieLoader(new NoOpMetricsSystem()),
-            new NoOpMetricsSystem(),
             null,
             EvmConfiguration.DEFAULT,
             TrieLogPruner.noOpTrieLogPruner());
@@ -141,7 +143,8 @@ class BonsaiWorldStateArchiveTest {
         new BonsaiWorldStateProvider(
             cachedWorldStorageManager,
             trieLogManager,
-            new BonsaiWorldStateKeyValueStorage(storageProvider, new NoOpMetricsSystem()),
+            new BonsaiWorldStateKeyValueStorage(
+                storageProvider, new NoOpMetricsSystem(), DataStorageConfiguration.DEFAULT_CONFIG),
             blockchain,
             new CachedMerkleTrieLoader(new NoOpMetricsSystem()),
             EvmConfiguration.DEFAULT);
@@ -167,7 +170,8 @@ class BonsaiWorldStateArchiveTest {
         .getTrieLogLayer(any(Hash.class));
 
     var worldStateStorage =
-        new BonsaiWorldStateKeyValueStorage(storageProvider, new NoOpMetricsSystem());
+        new BonsaiWorldStateKeyValueStorage(
+            storageProvider, new NoOpMetricsSystem(), DataStorageConfiguration.DEFAULT_CONFIG);
     bonsaiWorldStateArchive =
         spy(
             new BonsaiWorldStateProvider(
@@ -193,7 +197,8 @@ class BonsaiWorldStateArchiveTest {
   void testGetMutableWithStorageConsistencyNotRollbackTheState() {
 
     var worldStateStorage =
-        new BonsaiWorldStateKeyValueStorage(storageProvider, new NoOpMetricsSystem());
+        new BonsaiWorldStateKeyValueStorage(
+            storageProvider, new NoOpMetricsSystem(), DataStorageConfiguration.DEFAULT_CONFIG);
     bonsaiWorldStateArchive =
         spy(
             new BonsaiWorldStateProvider(
@@ -229,7 +234,8 @@ class BonsaiWorldStateArchiveTest {
         .getTrieLogLayer(any(Hash.class));
 
     var worldStateStorage =
-        new BonsaiWorldStateKeyValueStorage(storageProvider, new NoOpMetricsSystem());
+        new BonsaiWorldStateKeyValueStorage(
+            storageProvider, new NoOpMetricsSystem(), DataStorageConfiguration.DEFAULT_CONFIG);
 
     bonsaiWorldStateArchive =
         spy(
@@ -276,7 +282,10 @@ class BonsaiWorldStateArchiveTest {
             new BonsaiWorldStateProvider(
                 cachedWorldStorageManager,
                 trieLogManager,
-                new BonsaiWorldStateKeyValueStorage(storageProvider, new NoOpMetricsSystem()),
+                new BonsaiWorldStateKeyValueStorage(
+                    storageProvider,
+                    new NoOpMetricsSystem(),
+                    DataStorageConfiguration.DEFAULT_CONFIG),
                 blockchain,
                 new CachedMerkleTrieLoader(new NoOpMetricsSystem()),
                 EvmConfiguration.DEFAULT));

ethereum/core/src/test/java/org/hyperledger/besu/ethereum/trie/bonsai/CachedMerkleTrieLoaderTest.java

@@ -29,6 +29,7 @@ import org.hyperledger.besu.ethereum.trie.TrieIterator;
 import org.hyperledger.besu.ethereum.trie.bonsai.cache.CachedMerkleTrieLoader;
 import org.hyperledger.besu.ethereum.trie.bonsai.storage.BonsaiWorldStateKeyValueStorage;
 import org.hyperledger.besu.ethereum.trie.patricia.StoredMerklePatriciaTrie;
+import org.hyperledger.besu.ethereum.worldstate.DataStorageConfiguration;
 import org.hyperledger.besu.ethereum.worldstate.StateTrieAccountValue;
 import org.hyperledger.besu.metrics.noop.NoOpMetricsSystem;
 
@@ -48,7 +49,9 @@ class CachedMerkleTrieLoaderTest {
   private CachedMerkleTrieLoader merkleTrieLoader;
   private final StorageProvider storageProvider = new InMemoryKeyValueStorageProvider();
   private final BonsaiWorldStateKeyValueStorage inMemoryWorldState =
-      Mockito.spy(new BonsaiWorldStateKeyValueStorage(storageProvider, new NoOpMetricsSystem()));
+      Mockito.spy(
+          new BonsaiWorldStateKeyValueStorage(
+              storageProvider, new NoOpMetricsSystem(), DataStorageConfiguration.DEFAULT_CONFIG));
 
   final List<Address> accounts =
       List.of(Address.fromHexString("0xdeadbeef"), Address.fromHexString("0xdeadbeee"));
@@ -71,7 +74,9 @@ class CachedMerkleTrieLoaderTest {
 
     final BonsaiWorldStateKeyValueStorage emptyStorage =
         new BonsaiWorldStateKeyValueStorage(
-            new InMemoryKeyValueStorageProvider(), new NoOpMetricsSystem());
+            new InMemoryKeyValueStorageProvider(),
+            new NoOpMetricsSystem(),
+            DataStorageConfiguration.DEFAULT_CONFIG);
     StoredMerklePatriciaTrie<Bytes, Bytes> cachedTrie =
         new StoredMerklePatriciaTrie<>(
             (location, hash) ->
@@ -110,7 +115,9 @@ class CachedMerkleTrieLoaderTest {
     final List<Bytes> cachedSlots = new ArrayList<>();
     final BonsaiWorldStateKeyValueStorage emptyStorage =
         new BonsaiWorldStateKeyValueStorage(
-            new InMemoryKeyValueStorageProvider(), new NoOpMetricsSystem());
+            new InMemoryKeyValueStorageProvider(),
+            new NoOpMetricsSystem(),
+            DataStorageConfiguration.DEFAULT_CONFIG);
     final StoredMerklePatriciaTrie<Bytes, Bytes> cachedTrie =
         new StoredMerklePatriciaTrie<>(
             (location, hash) ->

ethereum/core/src/test/java/org/hyperledger/besu/ethereum/trie/bonsai/LogRollingTests.java

@@ -34,6 +34,7 @@ import org.hyperledger.besu.ethereum.trie.bonsai.trielog.TrieLogFactoryImpl;
 import org.hyperledger.besu.ethereum.trie.bonsai.trielog.TrieLogLayer;
 import org.hyperledger.besu.ethereum.trie.bonsai.worldview.BonsaiWorldState;
 import org.hyperledger.besu.ethereum.trie.bonsai.worldview.BonsaiWorldStateUpdateAccumulator;
+import org.hyperledger.besu.ethereum.worldstate.DataStorageConfiguration;
 import org.hyperledger.besu.evm.account.MutableAccount;
 import org.hyperledger.besu.evm.internal.EvmConfiguration;
 import org.hyperledger.besu.evm.log.LogsBloomFilter;
@@ -161,7 +162,8 @@ class LogRollingTests {
     final BonsaiWorldState worldState =
         new BonsaiWorldState(
             archive,
-            new BonsaiWorldStateKeyValueStorage(provider, new NoOpMetricsSystem()),
+            new BonsaiWorldStateKeyValueStorage(
+                provider, new NoOpMetricsSystem(), DataStorageConfiguration.DEFAULT_CONFIG),
             EvmConfiguration.DEFAULT);
     final WorldUpdater updater = worldState.updater();
 
@@ -174,7 +176,8 @@ class LogRollingTests {
     final BonsaiWorldState secondWorldState =
         new BonsaiWorldState(
             secondArchive,
-            new BonsaiWorldStateKeyValueStorage(secondProvider, new NoOpMetricsSystem()),
+            new BonsaiWorldStateKeyValueStorage(
+                secondProvider, new NoOpMetricsSystem(), DataStorageConfiguration.DEFAULT_CONFIG),
             EvmConfiguration.DEFAULT);
     final BonsaiWorldStateUpdateAccumulator secondUpdater =
         (BonsaiWorldStateUpdateAccumulator) secondWorldState.updater();
@@ -205,7 +208,8 @@ class LogRollingTests {
     final BonsaiWorldState worldState =
         new BonsaiWorldState(
             archive,
-            new BonsaiWorldStateKeyValueStorage(provider, new NoOpMetricsSystem()),
+            new BonsaiWorldStateKeyValueStorage(
+                provider, new NoOpMetricsSystem(), DataStorageConfiguration.DEFAULT_CONFIG),
             EvmConfiguration.DEFAULT);
 
     final WorldUpdater updater = worldState.updater();
@@ -226,7 +230,8 @@ class LogRollingTests {
     final BonsaiWorldState secondWorldState =
         new BonsaiWorldState(
             secondArchive,
-            new BonsaiWorldStateKeyValueStorage(secondProvider, new NoOpMetricsSystem()),
+            new BonsaiWorldStateKeyValueStorage(
+                secondProvider, new NoOpMetricsSystem(), DataStorageConfiguration.DEFAULT_CONFIG),
             EvmConfiguration.DEFAULT);
     final BonsaiWorldStateUpdateAccumulator secondUpdater =
         (BonsaiWorldStateUpdateAccumulator) secondWorldState.updater();
@@ -258,7 +263,8 @@ class LogRollingTests {
     final BonsaiWorldState worldState =
         new BonsaiWorldState(
             archive,
-            new BonsaiWorldStateKeyValueStorage(provider, new NoOpMetricsSystem()),
+            new BonsaiWorldStateKeyValueStorage(
+                provider, new NoOpMetricsSystem(), DataStorageConfiguration.DEFAULT_CONFIG),
             EvmConfiguration.DEFAULT);
 
     final WorldUpdater updater = worldState.updater();
@@ -286,7 +292,8 @@ class LogRollingTests {
     final BonsaiWorldState secondWorldState =
         new BonsaiWorldState(
             secondArchive,
-            new BonsaiWorldStateKeyValueStorage(secondProvider, new NoOpMetricsSystem()),
+            new BonsaiWorldStateKeyValueStorage(
+                secondProvider, new NoOpMetricsSystem(), DataStorageConfiguration.DEFAULT_CONFIG),
             EvmConfiguration.DEFAULT);
 
     final WorldUpdater secondUpdater = secondWorldState.updater();

ethereum/core/src/test/java/org/hyperledger/besu/ethereum/trie/bonsai/RollingImport.java

@@ -30,6 +30,7 @@ import org.hyperledger.besu.ethereum.trie.bonsai.trielog.TrieLogFactoryImpl;
 import org.hyperledger.besu.ethereum.trie.bonsai.trielog.TrieLogLayer;
 import org.hyperledger.besu.ethereum.trie.bonsai.worldview.BonsaiWorldState;
 import org.hyperledger.besu.ethereum.trie.bonsai.worldview.BonsaiWorldStateUpdateAccumulator;
+import org.hyperledger.besu.ethereum.worldstate.DataStorageConfiguration;
 import org.hyperledger.besu.evm.internal.EvmConfiguration;
 import org.hyperledger.besu.metrics.noop.NoOpMetricsSystem;
 import org.hyperledger.besu.services.kvstore.InMemoryKeyValueStorage;
@@ -56,7 +57,8 @@ public class RollingImport {
     final BonsaiWorldState bonsaiState =
         new BonsaiWorldState(
             archive,
-            new BonsaiWorldStateKeyValueStorage(provider, new NoOpMetricsSystem()),
+            new BonsaiWorldStateKeyValueStorage(
+                provider, new NoOpMetricsSystem(), DataStorageConfiguration.DEFAULT_CONFIG),
             EvmConfiguration.DEFAULT);
     final SegmentedInMemoryKeyValueStorage worldStateStorage =
         (SegmentedInMemoryKeyValueStorage)

ethereum/core/src/test/java/org/hyperledger/besu/ethereum/trie/bonsai/storage/BonsaiWorldStateKeyValueStorageTest.java

@@ -36,6 +36,7 @@ import org.hyperledger.besu.ethereum.storage.keyvalue.KeyValueSegmentIdentifier;
 import org.hyperledger.besu.ethereum.trie.MerkleTrie;
 import org.hyperledger.besu.ethereum.trie.StorageEntriesCollector;
 import org.hyperledger.besu.ethereum.trie.patricia.StoredMerklePatriciaTrie;
+import org.hyperledger.besu.ethereum.worldstate.DataStorageConfiguration;
 import org.hyperledger.besu.ethereum.worldstate.FlatDbMode;
 import org.hyperledger.besu.ethereum.worldstate.StateTrieAccountValue;
 import org.hyperledger.besu.metrics.noop.NoOpMetricsSystem;
@@ -452,7 +453,9 @@ public class BonsaiWorldStateKeyValueStorageTest {
 
   private BonsaiWorldStateKeyValueStorage emptyStorage() {
     return new BonsaiWorldStateKeyValueStorage(
-        new InMemoryKeyValueStorageProvider(), new NoOpMetricsSystem());
+        new InMemoryKeyValueStorageProvider(),
+        new NoOpMetricsSystem(),
+        DataStorageConfiguration.DEFAULT_CONFIG);
   }
 
   @Test
@@ -487,6 +490,7 @@ public class BonsaiWorldStateKeyValueStorageTest {
         .thenReturn(mockTrieLogStorage);
     when(mockStorageProvider.getStorageBySegmentIdentifiers(any()))
         .thenReturn(mock(SegmentedKeyValueStorage.class));
-    return new BonsaiWorldStateKeyValueStorage(mockStorageProvider, new NoOpMetricsSystem());
+    return new BonsaiWorldStateKeyValueStorage(
+        mockStorageProvider, new NoOpMetricsSystem(), DataStorageConfiguration.DEFAULT_CONFIG);
   }
 }

ethereum/core/src/test/java/org/hyperledger/besu/ethereum/trie/bonsai/storage/flat/FlatDbStrategyProviderTest.java

@@ -0,0 +1,89 @@
+/*
+ * Copyright Hyperledger Besu Contributors.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+ * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations under the License.
+ *
+ * SPDX-License-Identifier: Apache-2.0
+ */
+
+package org.hyperledger.besu.ethereum.trie.bonsai.storage.flat;
+
+import static org.assertj.core.api.Assertions.assertThat;
+
+import org.hyperledger.besu.ethereum.storage.keyvalue.KeyValueSegmentIdentifier;
+import org.hyperledger.besu.ethereum.worldstate.DataStorageConfiguration;
+import org.hyperledger.besu.ethereum.worldstate.FlatDbMode;
+import org.hyperledger.besu.metrics.noop.NoOpMetricsSystem;
+import org.hyperledger.besu.plugin.services.storage.SegmentedKeyValueStorage;
+import org.hyperledger.besu.plugin.services.storage.SegmentedKeyValueStorageTransaction;
+import org.hyperledger.besu.services.kvstore.SegmentedInMemoryKeyValueStorage;
+
+import java.util.List;
+
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.extension.ExtendWith;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.EnumSource;
+import org.mockito.junit.jupiter.MockitoExtension;
+
+@ExtendWith(MockitoExtension.class)
+class FlatDbStrategyProviderTest {
+  private final FlatDbStrategyProvider flatDbStrategyProvider =
+      new FlatDbStrategyProvider(new NoOpMetricsSystem(), DataStorageConfiguration.DEFAULT_CONFIG);
+  private final SegmentedKeyValueStorage composedWorldStateStorage =
+      new SegmentedInMemoryKeyValueStorage(List.of(KeyValueSegmentIdentifier.TRIE_BRANCH_STORAGE));
+
+  @ParameterizedTest
+  @EnumSource(FlatDbMode.class)
+  void loadsFlatDbStrategyForStoredFlatDbMode(final FlatDbMode flatDbMode) {
+    updateFlatDbMode(flatDbMode);
+
+    flatDbStrategyProvider.loadFlatDbStrategy(composedWorldStateStorage);
+    assertThat(flatDbStrategyProvider.getFlatDbMode()).isEqualTo(flatDbMode);
+  }
+
+  @Test
+  void loadsPartialFlatDbStrategyWhenNoFlatDbModeStored() {
+    flatDbStrategyProvider.loadFlatDbStrategy(composedWorldStateStorage);
+    assertThat(flatDbStrategyProvider.getFlatDbMode()).isEqualTo(FlatDbMode.PARTIAL);
+  }
+
+  @Test
+  void upgradesFlatDbStrategyToFullFlatDbMode() {
+    updateFlatDbMode(FlatDbMode.PARTIAL);
+
+    flatDbStrategyProvider.upgradeToFullFlatDbMode(composedWorldStateStorage);
+    assertThat(flatDbStrategyProvider.flatDbMode).isEqualTo(FlatDbMode.FULL);
+    assertThat(flatDbStrategyProvider.flatDbStrategy).isNotNull();
+    assertThat(flatDbStrategyProvider.getFlatDbStrategy(composedWorldStateStorage))
+        .isInstanceOf(FullFlatDbStrategy.class);
+  }
+
+  @Test
+  void downgradesFlatDbStrategyToPartiallyFlatDbMode() {
+    updateFlatDbMode(FlatDbMode.FULL);
+
+    flatDbStrategyProvider.downgradeToPartialFlatDbMode(composedWorldStateStorage);
+    assertThat(flatDbStrategyProvider.flatDbMode).isEqualTo(FlatDbMode.PARTIAL);
+    assertThat(flatDbStrategyProvider.flatDbStrategy).isNotNull();
+    assertThat(flatDbStrategyProvider.getFlatDbStrategy(composedWorldStateStorage))
+        .isInstanceOf(PartialFlatDbStrategy.class);
+  }
+
+  private void updateFlatDbMode(final FlatDbMode flatDbMode) {
+    final SegmentedKeyValueStorageTransaction transaction =
+        composedWorldStateStorage.startTransaction();
+    transaction.put(
+        KeyValueSegmentIdentifier.TRIE_BRANCH_STORAGE,
+        FlatDbStrategyProvider.FLAT_DB_MODE,
+        flatDbMode.getVersion().toArrayUnsafe());
+    transaction.commit();
+  }
+}

ethereum/eth/src/jmh/java/org/hyperledger/besu/ethereum/eth/sync/worldstate/WorldStateDownloaderBenchmark.java

@@ -38,7 +38,7 @@ import org.hyperledger.besu.ethereum.eth.sync.fastsync.worldstate.NodeDataReques
 import org.hyperledger.besu.ethereum.storage.StorageProvider;
 import org.hyperledger.besu.ethereum.storage.keyvalue.KeyValueSegmentIdentifier;
 import org.hyperledger.besu.ethereum.storage.keyvalue.KeyValueStorageProviderBuilder;
-import org.hyperledger.besu.ethereum.worldstate.DataStorageFormat;
+import org.hyperledger.besu.ethereum.worldstate.DataStorageConfiguration;
 import org.hyperledger.besu.ethereum.worldstate.WorldStateArchive;
 import org.hyperledger.besu.ethereum.worldstate.WorldStateStorage;
 import org.hyperledger.besu.metrics.ObservableMetricsSystem;
@@ -105,7 +105,8 @@ public class WorldStateDownloaderBenchmark {
 
     final StorageProvider storageProvider =
         createKeyValueStorageProvider(tempDir, tempDir.resolve("database"));
-    worldStateStorage = storageProvider.createWorldStateStorage(DataStorageFormat.FOREST);
+    worldStateStorage =
+        storageProvider.createWorldStateStorage(DataStorageConfiguration.DEFAULT_CONFIG);
 
     pendingRequests = new InMemoryTasksPriorityQueues<>();
     worldStateDownloader =

ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/manager/EthPeers.java

@@ -139,6 +139,7 @@ public class EthPeers {
         "peer_limit",
         "The maximum number of peers this node allows to connect",
         () -> peerUpperBound);
+
     connectedPeersCounter =
         metricsSystem.createCounter(
             BesuMetricCategory.PEERS, "connected_total", "Total number of peers connected");

ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/manager/EthProtocolManager.java

@@ -110,7 +110,7 @@ public class EthProtocolManager implements ProtocolManager, MinedBlockObserver {
 
     this.blockBroadcaster = new BlockBroadcaster(ethContext);
 
-    supportedCapabilities =
+    this.supportedCapabilities =
         calculateCapabilities(synchronizerConfiguration, ethereumWireProtocolConfiguration);
 
     // Run validators
@@ -252,11 +252,14 @@ public class EthProtocolManager implements ProtocolManager, MinedBlockObserver {
   @Override
   public void stop() {
     if (stopped.compareAndSet(false, true)) {
-      LOG.info("Stopping {} Subprotocol.", getSupportedProtocol());
+      LOG.atInfo().setMessage("Stopping {} Subprotocol.").addArgument(getSupportedProtocol()).log();
       scheduler.stop();
       shutdown.countDown();
     } else {
-      LOG.error("Attempted to stop already stopped {} Subprotocol.", getSupportedProtocol());
+      LOG.atInfo()
+          .setMessage("Attempted to stop already stopped {} Subprotocol.")
+          .addArgument(this::getSupportedProtocol)
+          .log();
     }
   }
 
@@ -264,7 +267,10 @@ public class EthProtocolManager implements ProtocolManager, MinedBlockObserver {
   public void awaitStop() throws InterruptedException {
     shutdown.await();
     scheduler.awaitStop();
-    LOG.info("{} Subprotocol stopped.", getSupportedProtocol());
+    LOG.atInfo()
+        .setMessage("{} Subprotocol stopped.")
+        .addArgument(this::getSupportedProtocol)
+        .log();
   }
 
   @Override
@@ -277,8 +283,10 @@ public class EthProtocolManager implements ProtocolManager, MinedBlockObserver {
     EthProtocolLogger.logProcessMessage(cap, code);
     final EthPeer ethPeer = ethPeers.peer(message.getConnection());
     if (ethPeer == null) {
-      LOG.debug(
+      LOG.atDebug()
-          "Ignoring message received from unknown peer connection: {}", message.getConnection());
+          .setMessage("Ignoring message received from unknown peer connection: {}")
+          .addArgument(message::getConnection)
+          .log();
       return;
     }
 
@@ -288,19 +296,24 @@ public class EthProtocolManager implements ProtocolManager, MinedBlockObserver {
       return;
     } else if (!ethPeer.statusHasBeenReceived()) {
       // Peers are required to send status messages before any other message type
-      LOG.debug(
+      LOG.atDebug()
-          "{} requires a Status ({}) message to be sent first.  Instead, received message {} (BREACH_OF_PROTOCOL).  Disconnecting from {}.",
+          .setMessage(
-          this.getClass().getSimpleName(),
+              "{} requires a Status ({}) message to be sent first.  Instead, received message {} (BREACH_OF_PROTOCOL).  Disconnecting from {}.")
-          EthPV62.STATUS,
+          .addArgument(() -> this.getClass().getSimpleName())
-          code,
+          .addArgument(EthPV62.STATUS)
-          ethPeer);
+          .addArgument(code)
+          .addArgument(ethPeer::toString)
+          .log();
       ethPeer.disconnect(DisconnectReason.BREACH_OF_PROTOCOL);
       return;
     }
 
     if (this.mergePeerFilter.isPresent()) {
       if (this.mergePeerFilter.get().disconnectIfGossipingBlocks(message, ethPeer)) {
-        LOG.debug("Post-merge disconnect: peer still gossiping blocks {}", ethPeer);
+        LOG.atDebug()
+            .setMessage("Post-merge disconnect: peer still gossiping blocks {}")
+            .addArgument(ethPeer::toString)
+            .log();
         handleDisconnect(ethPeer.getConnection(), DisconnectReason.SUBPROTOCOL_TRIGGERED, false);
         return;
       }
@@ -333,11 +346,12 @@ public class EthProtocolManager implements ProtocolManager, MinedBlockObserver {
         maybeResponseData = ethMessages.dispatch(ethMessage);
       }
     } catch (final RLPException e) {
-      LOG.debug(
+      LOG.atDebug()
-          "Received malformed message {} (BREACH_OF_PROTOCOL), disconnecting: {}",
+          .setMessage("Received malformed message {} (BREACH_OF_PROTOCOL), disconnecting: {}, {}")
-          messageData.getData(),
+          .addArgument(messageData::getData)
-          ethPeer,
+          .addArgument(ethPeer::toString)
-          e);
+          .addArgument(e::toString)
+          .log();
 
       ethPeer.disconnect(DisconnectMessage.DisconnectReason.BREACH_OF_PROTOCOL);
     }
@@ -368,23 +382,31 @@ public class EthProtocolManager implements ProtocolManager, MinedBlockObserver {
             genesisHash,
             latestForkId);
     try {
-      LOG.trace("Sending status message to {} for connection {}.", peer.getId(), connection);
+      LOG.atTrace()
+          .setMessage("Sending status message to {} for connection {}.")
+          .addArgument(peer::getId)
+          .addArgument(connection::toString)
+          .log();
       peer.send(status, getSupportedProtocol(), connection);
       peer.registerStatusSent(connection);
     } catch (final PeerNotConnected peerNotConnected) {
       // Nothing to do.
     }
-    LOG.trace("{}", ethPeers);
+    LOG.atTrace().setMessage("{}").addArgument(ethPeers::toString).log();
   }
 
   @Override
   public boolean shouldConnect(final Peer peer, final boolean incoming) {
-    if (peer.getForkId().map(forkId -> forkIdManager.peerCheck(forkId)).orElse(true)) {
+    if (peer.getForkId().map(forkIdManager::peerCheck).orElse(true)) {
-      LOG.trace("ForkId OK or not available");
+      LOG.atDebug()
+          .setMessage("ForkId OK or not available for peer {}")
+          .addArgument(peer::getId)
+          .log();
       if (ethPeers.shouldConnect(peer, incoming)) {
         return true;
       }
     }
+    LOG.atDebug().setMessage("ForkId check failed for peer {}").addArgument(peer::getId).log();
     return false;
   }
 
@@ -397,11 +419,11 @@ public class EthProtocolManager implements ProtocolManager, MinedBlockObserver {
       LOG.atDebug()
           .setMessage("Disconnect - {} - {} - {}... - {} peers left")
           .addArgument(initiatedByPeer ? "Inbound" : "Outbound")
-          .addArgument(reason)
+          .addArgument(reason::toString)
-          .addArgument(connection.getPeer().getId().slice(0, 8))
+          .addArgument(() -> connection.getPeer().getId().slice(0, 8))
-          .addArgument(ethPeers.peerCount())
+          .addArgument(ethPeers::peerCount)
           .log();
-      LOG.trace("{}", ethPeers);
+      LOG.atTrace().setMessage("{}").addArgument(ethPeers::toString).log();
     }
   }
 
@@ -412,43 +434,41 @@ public class EthProtocolManager implements ProtocolManager, MinedBlockObserver {
     try {
       if (!status.networkId().equals(networkId)) {
         LOG.atDebug()
-            .setMessage("Mismatched network id: {}, EthPeer {}...")
+            .setMessage("Mismatched network id: {}, peer {}")
-            .addArgument(status.networkId())
+            .addArgument(status::networkId)
-            .addArgument(peer.getShortNodeId())
+            .addArgument(() -> getPeerOrPeerId(peer))
-            .log();
-        LOG.atTrace()
-            .setMessage("Mismatched network id: {}, EthPeer {}")
-            .addArgument(status.networkId())
-            .addArgument(peer)
             .log();
         peer.disconnect(DisconnectReason.SUBPROTOCOL_TRIGGERED);
       } else if (!forkIdManager.peerCheck(forkId) && status.protocolVersion() > 63) {
-        LOG.debug(
+        LOG.atDebug()
-            "{} has matching network id ({}), but non-matching fork id: {}",
+            .setMessage("{} has matching network id ({}), but non-matching fork id: {}")
-            peer,
+            .addArgument(() -> getPeerOrPeerId(peer))
-            networkId,
+            .addArgument(networkId::toString)
-            forkId);
+            .addArgument(forkId)
+            .log();
         peer.disconnect(DisconnectReason.SUBPROTOCOL_TRIGGERED);
       } else if (forkIdManager.peerCheck(status.genesisHash())) {
-        LOG.debug(
+        LOG.atDebug()
-            "{} has matching network id ({}), but non-matching genesis hash: {}",
+            .setMessage("{} has matching network id ({}), but non-matching genesis hash: {}")
-            peer,
+            .addArgument(() -> getPeerOrPeerId(peer))
-            networkId,
+            .addArgument(networkId::toString)
-            status.genesisHash());
+            .addArgument(status::genesisHash)
+            .log();
         peer.disconnect(DisconnectReason.SUBPROTOCOL_TRIGGERED);
       } else if (mergePeerFilter.isPresent()
           && mergePeerFilter.get().disconnectIfPoW(status, peer)) {
         LOG.atDebug()
             .setMessage("Post-merge disconnect: peer still PoW {}")
-            .addArgument(peer.getShortNodeId())
+            .addArgument(() -> getPeerOrPeerId(peer))
             .log();
         handleDisconnect(peer.getConnection(), DisconnectReason.SUBPROTOCOL_TRIGGERED, false);
       } else {
-        LOG.debug(
+        LOG.atDebug()
-            "Received status message from {}: {} with connection {}",
+            .setMessage("Received status message from {}: {} with connection {}")
-            peer,
+            .addArgument(peer::toString)
-            status,
+            .addArgument(status::toString)
-            message.getConnection());
+            .addArgument(message::getConnection)
+            .log();
         peer.registerStatusReceived(
             status.bestHash(),
             status.totalDifficulty(),
@@ -467,6 +487,10 @@ public class EthProtocolManager implements ProtocolManager, MinedBlockObserver {
     }
   }
 
+  private Object getPeerOrPeerId(final EthPeer peer) {
+    return LOG.isTraceEnabled() ? peer : peer.getShortNodeId();
+  }
+
   @Override
   public void blockMined(final Block block) {
     // This assumes the block has already been included in the chain

ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/backwardsync/BackwardSyncAlgSpec.java

@@ -165,7 +165,7 @@ public class BackwardSyncAlgSpec {
 
     ttdCaptor.getValue().onTTDReached(true);
 
-    voidCompletableFuture.get(100, TimeUnit.MILLISECONDS);
+    voidCompletableFuture.get(200, TimeUnit.MILLISECONDS);
     assertThat(voidCompletableFuture).isCompleted();
 
     verify(context.getSyncState()).unsubscribeTTDReached(88L);
@@ -192,7 +192,7 @@ public class BackwardSyncAlgSpec {
 
     completionCaptor.getValue().onInitialSyncCompleted();
 
-    voidCompletableFuture.get(100, TimeUnit.MILLISECONDS);
+    voidCompletableFuture.get(200, TimeUnit.MILLISECONDS);
     assertThat(voidCompletableFuture).isCompleted();
 
     verify(context.getSyncState()).unsubscribeTTDReached(88L);

ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/fastsync/worldstate/FastWorldDownloadStateTest.java

@@ -28,6 +28,7 @@ import org.hyperledger.besu.ethereum.eth.sync.worldstate.StalledDownloadExceptio
 import org.hyperledger.besu.ethereum.eth.sync.worldstate.WorldStateDownloadProcess;
 import org.hyperledger.besu.ethereum.trie.bonsai.storage.BonsaiWorldStateKeyValueStorage;
 import org.hyperledger.besu.ethereum.trie.forest.storage.ForestWorldStateKeyValueStorage;
+import org.hyperledger.besu.ethereum.worldstate.DataStorageConfiguration;
 import org.hyperledger.besu.ethereum.worldstate.DataStorageFormat;
 import org.hyperledger.besu.ethereum.worldstate.WorldStateStorage;
 import org.hyperledger.besu.metrics.noop.NoOpMetricsSystem;
@@ -80,7 +81,9 @@ public class FastWorldDownloadStateTest {
     if (storageFormat == DataStorageFormat.BONSAI) {
       worldStateStorage =
           new BonsaiWorldStateKeyValueStorage(
-              new InMemoryKeyValueStorageProvider(), new NoOpMetricsSystem());
+              new InMemoryKeyValueStorageProvider(),
+              new NoOpMetricsSystem(),
+              DataStorageConfiguration.DEFAULT_CONFIG);
     } else {
       worldStateStorage = new ForestWorldStateKeyValueStorage(new InMemoryKeyValueStorage());
     }

ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/fastsync/worldstate/PersistDataStepTest.java

@@ -26,7 +26,7 @@ import org.hyperledger.besu.ethereum.core.BlockHeaderTestFixture;
 import org.hyperledger.besu.ethereum.core.InMemoryKeyValueStorageProvider;
 import org.hyperledger.besu.ethereum.trie.MerkleTrie;
 import org.hyperledger.besu.ethereum.trie.patricia.SimpleMerklePatriciaTrie;
-import org.hyperledger.besu.ethereum.worldstate.DataStorageFormat;
+import org.hyperledger.besu.ethereum.worldstate.DataStorageConfiguration;
 import org.hyperledger.besu.ethereum.worldstate.WorldStateStorage;
 import org.hyperledger.besu.services.tasks.Task;
 
@@ -40,7 +40,8 @@ import org.junit.jupiter.api.Test;
 public class PersistDataStepTest {
 
   private final WorldStateStorage worldStateStorage =
-      new InMemoryKeyValueStorageProvider().createWorldStateStorage(DataStorageFormat.FOREST);
+      new InMemoryKeyValueStorageProvider()
+          .createWorldStateStorage(DataStorageConfiguration.DEFAULT_CONFIG);
   private final FastWorldDownloadState downloadState = mock(FastWorldDownloadState.class);
 
   private final Bytes rootNodeData = Bytes.of(1, 1, 1, 1);

ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/snapsync/AccountHealingTrackingTest.java

@@ -34,6 +34,7 @@ import org.hyperledger.besu.ethereum.trie.TrieIterator;
 import org.hyperledger.besu.ethereum.trie.bonsai.storage.BonsaiWorldStateKeyValueStorage;
 import org.hyperledger.besu.ethereum.trie.patricia.StoredMerklePatriciaTrie;
 import org.hyperledger.besu.ethereum.trie.patricia.StoredNodeFactory;
+import org.hyperledger.besu.ethereum.worldstate.DataStorageConfiguration;
 import org.hyperledger.besu.ethereum.worldstate.StateTrieAccountValue;
 import org.hyperledger.besu.ethereum.worldstate.WorldStateStorage;
 import org.hyperledger.besu.metrics.noop.NoOpMetricsSystem;
@@ -58,7 +59,9 @@ public class AccountHealingTrackingTest {
   private final List<Address> accounts = List.of(Address.fromHexString("0xdeadbeef"));
   private final WorldStateStorage worldStateStorage =
       new BonsaiWorldStateKeyValueStorage(
-          new InMemoryKeyValueStorageProvider(), new NoOpMetricsSystem());
+          new InMemoryKeyValueStorageProvider(),
+          new NoOpMetricsSystem(),
+          DataStorageConfiguration.DEFAULT_CONFIG);
 
   private WorldStateProofProvider worldStateProofProvider;
 

ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/snapsync/PersistDataStepTest.java

@@ -26,7 +26,7 @@ import org.hyperledger.besu.ethereum.eth.sync.snapsync.request.BytecodeRequest;
 import org.hyperledger.besu.ethereum.eth.sync.snapsync.request.SnapDataRequest;
 import org.hyperledger.besu.ethereum.eth.sync.snapsync.request.StorageRangeDataRequest;
 import org.hyperledger.besu.ethereum.trie.patricia.StoredMerklePatriciaTrie;
-import org.hyperledger.besu.ethereum.worldstate.DataStorageFormat;
+import org.hyperledger.besu.ethereum.worldstate.DataStorageConfiguration;
 import org.hyperledger.besu.ethereum.worldstate.WorldStateStorage;
 import org.hyperledger.besu.services.tasks.Task;
 
@@ -39,7 +39,8 @@ import org.junit.jupiter.api.Test;
 public class PersistDataStepTest {
 
   private final WorldStateStorage worldStateStorage =
-      new InMemoryKeyValueStorageProvider().createWorldStateStorage(DataStorageFormat.FOREST);
+      new InMemoryKeyValueStorageProvider()
+          .createWorldStateStorage(DataStorageConfiguration.DEFAULT_CONFIG);
   private final SnapSyncProcessState snapSyncState = mock(SnapSyncProcessState.class);
   private final SnapWorldDownloadState downloadState = mock(SnapWorldDownloadState.class);
 

ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/snapsync/SnapWorldDownloadStateTest.java

@@ -40,6 +40,7 @@ import org.hyperledger.besu.ethereum.eth.sync.snapsync.request.SnapDataRequest;
 import org.hyperledger.besu.ethereum.eth.sync.worldstate.WorldStateDownloadProcess;
 import org.hyperledger.besu.ethereum.trie.bonsai.storage.BonsaiWorldStateKeyValueStorage;
 import org.hyperledger.besu.ethereum.trie.forest.storage.ForestWorldStateKeyValueStorage;
+import org.hyperledger.besu.ethereum.worldstate.DataStorageConfiguration;
 import org.hyperledger.besu.ethereum.worldstate.DataStorageFormat;
 import org.hyperledger.besu.ethereum.worldstate.WorldStateStorage;
 import org.hyperledger.besu.metrics.noop.NoOpMetricsSystem;
@@ -108,7 +109,9 @@ public class SnapWorldDownloadStateTest {
     if (storageFormat == DataStorageFormat.BONSAI) {
       worldStateStorage =
           new BonsaiWorldStateKeyValueStorage(
-              new InMemoryKeyValueStorageProvider(), new NoOpMetricsSystem());
+              new InMemoryKeyValueStorageProvider(),
+              new NoOpMetricsSystem(),
+              DataStorageConfiguration.DEFAULT_CONFIG);
     } else {
       worldStateStorage = new ForestWorldStateKeyValueStorage(new InMemoryKeyValueStorage());
     }

ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/snapsync/TaskGenerator.java

@@ -27,7 +27,7 @@ import org.hyperledger.besu.ethereum.trie.MerkleTrie;
 import org.hyperledger.besu.ethereum.trie.RangeStorageEntriesCollector;
 import org.hyperledger.besu.ethereum.trie.TrieIterator;
 import org.hyperledger.besu.ethereum.trie.patricia.StoredMerklePatriciaTrie;
-import org.hyperledger.besu.ethereum.worldstate.DataStorageFormat;
+import org.hyperledger.besu.ethereum.worldstate.DataStorageConfiguration;
 import org.hyperledger.besu.ethereum.worldstate.StateTrieAccountValue;
 import org.hyperledger.besu.ethereum.worldstate.WorldStateStorage;
 import org.hyperledger.besu.services.tasks.Task;
@@ -44,7 +44,8 @@ public class TaskGenerator {
   public static List<Task<SnapDataRequest>> createAccountRequest(final boolean withData) {
 
     final WorldStateStorage worldStateStorage =
-        new InMemoryKeyValueStorageProvider().createWorldStateStorage(DataStorageFormat.FOREST);
+        new InMemoryKeyValueStorageProvider()
+            .createWorldStateStorage(DataStorageConfiguration.DEFAULT_CONFIG);
 
     final WorldStateProofProvider worldStateProofProvider =
         new WorldStateProofProvider(worldStateStorage);

ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/snapsync/request/heal/AccountFlatDatabaseHealingRangeRequestTest.java

@@ -31,6 +31,7 @@ import org.hyperledger.besu.ethereum.trie.RangeStorageEntriesCollector;
 import org.hyperledger.besu.ethereum.trie.TrieIterator;
 import org.hyperledger.besu.ethereum.trie.bonsai.storage.BonsaiWorldStateKeyValueStorage;
 import org.hyperledger.besu.ethereum.trie.forest.storage.ForestWorldStateKeyValueStorage;
+import org.hyperledger.besu.ethereum.worldstate.DataStorageConfiguration;
 import org.hyperledger.besu.ethereum.worldstate.WorldStateStorage;
 import org.hyperledger.besu.metrics.noop.NoOpMetricsSystem;
 import org.hyperledger.besu.services.kvstore.InMemoryKeyValueStorage;
@@ -179,7 +180,8 @@ public class AccountFlatDatabaseHealingRangeRequestTest {
     final StorageProvider storageProvider = new InMemoryKeyValueStorageProvider();
 
     final WorldStateStorage worldStateStorage =
-        new BonsaiWorldStateKeyValueStorage(storageProvider, new NoOpMetricsSystem());
+        new BonsaiWorldStateKeyValueStorage(
+            storageProvider, new NoOpMetricsSystem(), DataStorageConfiguration.DEFAULT_CONFIG);
     final WorldStateProofProvider proofProvider = new WorldStateProofProvider(worldStateStorage);
     final MerkleTrie<Bytes, Bytes> accountStateTrie =
         TrieGenerator.generateTrie(worldStateStorage, 15);
@@ -233,7 +235,8 @@ public class AccountFlatDatabaseHealingRangeRequestTest {
     final StorageProvider storageProvider = new InMemoryKeyValueStorageProvider();
 
     final WorldStateStorage worldStateStorage =
-        new BonsaiWorldStateKeyValueStorage(storageProvider, new NoOpMetricsSystem());
+        new BonsaiWorldStateKeyValueStorage(
+            storageProvider, new NoOpMetricsSystem(), DataStorageConfiguration.DEFAULT_CONFIG);
     final WorldStateProofProvider proofProvider = new WorldStateProofProvider(worldStateStorage);
     final MerkleTrie<Bytes, Bytes> accountStateTrie =
         TrieGenerator.generateTrie(worldStateStorage, 15);

ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/snapsync/request/heal/StorageFlatDatabaseHealingRangeRequestTest.java

@@ -33,6 +33,7 @@ import org.hyperledger.besu.ethereum.trie.RangeStorageEntriesCollector;
 import org.hyperledger.besu.ethereum.trie.TrieIterator;
 import org.hyperledger.besu.ethereum.trie.bonsai.storage.BonsaiWorldStateKeyValueStorage;
 import org.hyperledger.besu.ethereum.trie.patricia.StoredMerklePatriciaTrie;
+import org.hyperledger.besu.ethereum.worldstate.DataStorageConfiguration;
 import org.hyperledger.besu.ethereum.worldstate.StateTrieAccountValue;
 import org.hyperledger.besu.ethereum.worldstate.WorldStateStorage;
 import org.hyperledger.besu.metrics.noop.NoOpMetricsSystem;
@@ -78,7 +79,8 @@ class StorageFlatDatabaseHealingRangeRequestTest {
   public void setup() {
     final StorageProvider storageProvider = new InMemoryKeyValueStorageProvider();
     worldStateStorage =
-        new BonsaiWorldStateKeyValueStorage(storageProvider, new NoOpMetricsSystem());
+        new BonsaiWorldStateKeyValueStorage(
+            storageProvider, new NoOpMetricsSystem(), DataStorageConfiguration.DEFAULT_CONFIG);
     proofProvider = new WorldStateProofProvider(worldStateStorage);
     trie =
         TrieGenerator.generateTrie(

ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/snapsync/request/heal/StorageTrieNodeHealingRequestTest.java

@@ -24,6 +24,7 @@ import org.hyperledger.besu.ethereum.storage.StorageProvider;
 import org.hyperledger.besu.ethereum.trie.MerkleTrie;
 import org.hyperledger.besu.ethereum.trie.bonsai.storage.BonsaiWorldStateKeyValueStorage;
 import org.hyperledger.besu.ethereum.trie.forest.storage.ForestWorldStateKeyValueStorage;
+import org.hyperledger.besu.ethereum.worldstate.DataStorageConfiguration;
 import org.hyperledger.besu.ethereum.worldstate.DataStorageFormat;
 import org.hyperledger.besu.ethereum.worldstate.StateTrieAccountValue;
 import org.hyperledger.besu.ethereum.worldstate.WorldStateStorage;
@@ -74,7 +75,8 @@ class StorageTrieNodeHealingRequestTest {
     } else {
       final StorageProvider storageProvider = new InMemoryKeyValueStorageProvider();
       worldStateStorage =
-          new BonsaiWorldStateKeyValueStorage(storageProvider, new NoOpMetricsSystem());
+          new BonsaiWorldStateKeyValueStorage(
+              storageProvider, new NoOpMetricsSystem(), DataStorageConfiguration.DEFAULT_CONFIG);
     }
     final MerkleTrie<Bytes, Bytes> trie =
         TrieGenerator.generateTrie(

ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/PendingTransactionEstimatedMemorySizeTest.java

@@ -58,7 +58,7 @@ public class PendingTransactionEstimatedMemorySizeTest extends BaseTransactionPo
   private static final Set<Class<?>> SHARED_CLASSES =
       Set.of(SignatureAlgorithm.class, TransactionType.class);
   private static final Set<String> COMMON_CONSTANT_FIELD_PATHS =
-      Set.of(".value.ctor", ".hashNoSignature");
+      Set.of(".value.ctor", ".hashNoSignature", ".signature.encoded.delegate");
   private static final Set<String> EIP1559_EIP4844_CONSTANT_FIELD_PATHS =
       Sets.union(COMMON_CONSTANT_FIELD_PATHS, Set.of(".gasPrice"));
   private static final Set<String> FRONTIER_ACCESS_LIST_CONSTANT_FIELD_PATHS =

ethereum/evmtool/src/main/java/org/hyperledger/besu/evmtool/EvmToolCommand.java

@@ -371,6 +371,9 @@ public class EvmToolCommand implements Runnable {
       long txGas = gas - intrinsicGasCost - accessListCost;
 
       final EVM evm = protocolSpec.getEvm();
+      if (codeBytes.isEmpty()) {
+        codeBytes = component.getWorldState().get(receiver).getCode();
+      }
       Code code = evm.getCode(Hash.hash(codeBytes), codeBytes);
       if (!code.isValid()) {
         out.println(((CodeInvalid) code).getInvalidReason());

ethereum/p2p/src/main/java/org/hyperledger/besu/ethereum/p2p/config/DiscoveryConfiguration.java

@@ -32,7 +32,7 @@ public class DiscoveryConfiguration {
   private List<EnodeURL> bootnodes = new ArrayList<>();
   private String dnsDiscoveryURL;
   private boolean discoveryV5Enabled = false;
-  private boolean filterOnEnrForkId = false;
+  private boolean filterOnEnrForkId = NetworkingConfiguration.DEFAULT_FILTER_ON_ENR_FORK_ID;
 
   public static DiscoveryConfiguration create() {
     return new DiscoveryConfiguration();

ethereum/p2p/src/main/java/org/hyperledger/besu/ethereum/p2p/config/NetworkingConfiguration.java

@@ -23,6 +23,7 @@ public class NetworkingConfiguration {
   public static final int DEFAULT_INITIATE_CONNECTIONS_FREQUENCY_SEC = 30;
   public static final int DEFAULT_CHECK_MAINTAINED_CONNECTIONS_FREQUENCY_SEC = 60;
   public static final int DEFAULT_PEER_LOWER_BOUND = 25;
+  public static final boolean DEFAULT_FILTER_ON_ENR_FORK_ID = true;
 
   private DiscoveryConfiguration discovery = new DiscoveryConfiguration();
   private RlpxConfiguration rlpx = new RlpxConfiguration();

ethereum/p2p/src/main/java/org/hyperledger/besu/ethereum/p2p/discovery/PeerDiscoveryAgent.java

@@ -26,6 +26,7 @@ import org.hyperledger.besu.ethereum.p2p.config.DiscoveryConfiguration;
 import org.hyperledger.besu.ethereum.p2p.discovery.internal.Packet;
 import org.hyperledger.besu.ethereum.p2p.discovery.internal.PeerDiscoveryController;
 import org.hyperledger.besu.ethereum.p2p.discovery.internal.PeerRequirement;
+import org.hyperledger.besu.ethereum.p2p.discovery.internal.PeerTable;
 import org.hyperledger.besu.ethereum.p2p.discovery.internal.PingPacketData;
 import org.hyperledger.besu.ethereum.p2p.discovery.internal.TimerUtil;
 import org.hyperledger.besu.ethereum.p2p.peers.EnodeURLImpl;
@@ -81,6 +82,7 @@ public abstract class PeerDiscoveryAgent {
   private final MetricsSystem metricsSystem;
   private final RlpxAgent rlpxAgent;
   private final ForkIdManager forkIdManager;
+  private final PeerTable peerTable;
 
   /* The peer controller, which takes care of the state machine of peers. */
   protected Optional<PeerDiscoveryController> controller = Optional.empty();
@@ -109,7 +111,8 @@ public abstract class PeerDiscoveryAgent {
       final MetricsSystem metricsSystem,
       final StorageProvider storageProvider,
       final ForkIdManager forkIdManager,
-      final RlpxAgent rlpxAgent) {
+      final RlpxAgent rlpxAgent,
+      final PeerTable peerTable) {
     this.metricsSystem = metricsSystem;
     checkArgument(nodeKey != null, "nodeKey cannot be null");
     checkArgument(config != null, "provided configuration cannot be null");
@@ -130,6 +133,7 @@ public abstract class PeerDiscoveryAgent {
     this.forkIdManager = forkIdManager;
     this.forkIdSupplier = () -> forkIdManager.getForkIdForChainHead().getForkIdAsBytesList();
     this.rlpxAgent = rlpxAgent;
+    this.peerTable = peerTable;
   }
 
   protected abstract TimerUtil createTimer();
@@ -263,9 +267,9 @@ public abstract class PeerDiscoveryAgent {
         .peerRequirement(PeerRequirement.combine(peerRequirements))
         .peerPermissions(peerPermissions)
         .metricsSystem(metricsSystem)
-        .forkIdManager(forkIdManager)
         .filterOnEnrForkId((config.isFilterOnEnrForkIdEnabled()))
         .rlpxAgent(rlpxAgent)
+        .peerTable(peerTable)
         .build();
   }
 
@@ -282,8 +286,31 @@ public abstract class PeerDiscoveryAgent {
             .flatMap(Endpoint::getTcpPort)
             .orElse(udpPort);
 
+    // If the host is present in the P2P PING packet itself, use that as the endpoint. If the P2P
+    // PING packet specifies 127.0.0.1 (the default if a custom value is not specified with
+    // --p2p-host or via a suitable --nat-method) we ignore it in favour of the UDP source address.
+    // The likelihood is that the UDP source will be 127.0.0.1 anyway, but this reduces the chance
+    // of an unexpected change in behaviour as a result of
+    // https://github.com/hyperledger/besu/issues/6224 being fixed.
+    final String host =
+        packet
+            .getPacketData(PingPacketData.class)
+            .flatMap(PingPacketData::getFrom)
+            .map(Endpoint::getHost)
+            .filter(
+                fromAddr ->
+                    (!fromAddr.equals("127.0.0.1") && InetAddresses.isInetAddress(fromAddr)))
+            .stream()
+            .peek(
+                h ->
+                    LOG.trace(
+                        "Using \"From\" endpoint {} specified in ping packet. Ignoring UDP source host {}",
+                        h,
+                        sourceEndpoint.getHost()))
+            .findFirst()
+            .orElseGet(sourceEndpoint::getHost);
+
     // Notify the peer controller.
-    final String host = sourceEndpoint.getHost();
     final DiscoveryPeer peer =
         DiscoveryPeer.fromEnode(
             EnodeURLImpl.builder()

ethereum/p2p/src/main/java/org/hyperledger/besu/ethereum/p2p/discovery/VertxPeerDiscoveryAgent.java

@@ -23,6 +23,7 @@ import org.hyperledger.besu.ethereum.p2p.config.DiscoveryConfiguration;
 import org.hyperledger.besu.ethereum.p2p.discovery.internal.Packet;
 import org.hyperledger.besu.ethereum.p2p.discovery.internal.PeerDiscoveryController;
 import org.hyperledger.besu.ethereum.p2p.discovery.internal.PeerDiscoveryController.AsyncExecutor;
+import org.hyperledger.besu.ethereum.p2p.discovery.internal.PeerTable;
 import org.hyperledger.besu.ethereum.p2p.discovery.internal.TimerUtil;
 import org.hyperledger.besu.ethereum.p2p.discovery.internal.VertxTimerUtil;
 import org.hyperledger.besu.ethereum.p2p.permissions.PeerPermissions;
@@ -73,7 +74,8 @@ public class VertxPeerDiscoveryAgent extends PeerDiscoveryAgent {
       final MetricsSystem metricsSystem,
       final StorageProvider storageProvider,
       final ForkIdManager forkIdManager,
-      final RlpxAgent rlpxAgent) {
+      final RlpxAgent rlpxAgent,
+      final PeerTable peerTable) {
     super(
         nodeKey,
         config,
@@ -82,7 +84,8 @@ public class VertxPeerDiscoveryAgent extends PeerDiscoveryAgent {
         metricsSystem,
         storageProvider,
         forkIdManager,
-        rlpxAgent);
+        rlpxAgent,
+        peerTable);
     checkArgument(vertx != null, "vertx instance cannot be null");
     this.vertx = vertx;
 

ethereum/p2p/src/main/java/org/hyperledger/besu/ethereum/p2p/discovery/internal/PeerDiscoveryController.java

@@ -21,8 +21,6 @@ import static java.util.concurrent.TimeUnit.MILLISECONDS;
 import static java.util.concurrent.TimeUnit.SECONDS;
 
 import org.hyperledger.besu.cryptoservices.NodeKey;
-import org.hyperledger.besu.ethereum.forkid.ForkId;
-import org.hyperledger.besu.ethereum.forkid.ForkIdManager;
 import org.hyperledger.besu.ethereum.p2p.discovery.DiscoveryPeer;
 import org.hyperledger.besu.ethereum.p2p.discovery.PeerDiscoveryStatus;
 import org.hyperledger.besu.ethereum.p2p.peers.Peer;
@@ -129,7 +127,6 @@ public class PeerDiscoveryController {
   private final DiscoveryProtocolLogger discoveryProtocolLogger;
   private final LabelledMetric<Counter> interactionCounter;
   private final LabelledMetric<Counter> interactionRetryCounter;
-  private final ForkIdManager forkIdManager;
   private final boolean filterOnEnrForkId;
   private final RlpxAgent rlpxAgent;
 
@@ -161,7 +158,6 @@ public class PeerDiscoveryController {
       final PeerPermissions peerPermissions,
       final MetricsSystem metricsSystem,
       final Optional<Cache<Bytes, Packet>> maybeCacheForEnrRequests,
-      final ForkIdManager forkIdManager,
       final boolean filterOnEnrForkId,
       final RlpxAgent rlpxAgent) {
     this.timerUtil = timerUtil;
@@ -197,11 +193,11 @@ public class PeerDiscoveryController {
             "discovery_interaction_retry_count",
             "Total number of interaction retries performed",
             "type");
+
     this.cachedEnrRequests =
         maybeCacheForEnrRequests.orElse(
             CacheBuilder.newBuilder().maximumSize(50).expireAfterWrite(10, SECONDS).build());
 
-    this.forkIdManager = forkIdManager;
     this.filterOnEnrForkId = filterOnEnrForkId;
   }
 
@@ -314,6 +310,7 @@ public class PeerDiscoveryController {
     }
 
     final DiscoveryPeer peer = resolvePeer(sender);
+    final Bytes peerId = peer.getId();
     switch (packet.getType()) {
       case PING:
         if (peerPermissions.allowInboundBonding(peer)) {
@@ -333,10 +330,10 @@ public class PeerDiscoveryController {
                   if (filterOnEnrForkId) {
                     requestENR(peer);
                   }
-                  bondingPeers.invalidate(peer.getId());
+                  bondingPeers.invalidate(peerId);
                   addToPeerTable(peer);
                   recursivePeerRefreshState.onBondingComplete(peer);
-                  Optional.ofNullable(cachedEnrRequests.getIfPresent(peer.getId()))
+                  Optional.ofNullable(cachedEnrRequests.getIfPresent(peerId))
                       .ifPresent(cachedEnrRequest -> processEnrRequest(peer, cachedEnrRequest));
                 });
         break;
@@ -360,12 +357,12 @@ public class PeerDiscoveryController {
         if (PeerDiscoveryStatus.BONDED.equals(peer.getStatus())) {
           processEnrRequest(peer, packet);
         } else if (PeerDiscoveryStatus.BONDING.equals(peer.getStatus())) {
-          LOG.trace("ENR_REQUEST cached for bonding peer Id: {}", peer.getId());
+          LOG.trace("ENR_REQUEST cached for bonding peer Id: {}", peerId);
           // Due to UDP, it may happen that we receive the ENR_REQUEST just before the PONG.
           // Because peers want to send the ENR_REQUEST directly after the pong.
           // If this happens we don't want to ignore the request but process when bonded.
           // this cache allows to keep the request and to respond after having processed the PONG
-          cachedEnrRequests.put(peer.getId(), packet);
+          cachedEnrRequests.put(peerId, packet);
         }
         break;
       case ENR_RESPONSE:
@@ -376,26 +373,6 @@ public class PeerDiscoveryController {
                       packet.getPacketData(ENRResponsePacketData.class);
                   final NodeRecord enr = packetData.get().getEnr();
                   peer.setNodeRecord(enr);
-
-                  final Optional<ForkId> maybeForkId = peer.getForkId();
-                  if (maybeForkId.isPresent()) {
-                    if (forkIdManager.peerCheck(maybeForkId.get())) {
-                      connectOnRlpxLayer(peer);
-                      LOG.debug(
-                          "Peer {} PASSED fork id check. ForkId received: {}",
-                          sender.getId(),
-                          maybeForkId.get());
-                    } else {
-                      LOG.debug(
-                          "Peer {} FAILED fork id check. ForkId received: {}",
-                          sender.getId(),
-                          maybeForkId.get());
-                    }
-                  } else {
-                    // if the peer hasn't sent the ForkId try to connect to it anyways
-                    connectOnRlpxLayer(peer);
-                    LOG.debug("No fork id sent by peer: {}", peer.getId());
-                  }
                 });
         break;
     }
@@ -431,9 +408,7 @@ public class PeerDiscoveryController {
 
     if (peer.getStatus() != PeerDiscoveryStatus.BONDED) {
       peer.setStatus(PeerDiscoveryStatus.BONDED);
-      if (!filterOnEnrForkId) {
+      connectOnRlpxLayer(peer);
-        connectOnRlpxLayer(peer);
-      }
     }
 
     final PeerTable.AddResult result = peerTable.tryAdd(peer);
@@ -560,8 +535,6 @@ public class PeerDiscoveryController {
    */
   @VisibleForTesting
   void requestENR(final DiscoveryPeer peer) {
-    peer.setStatus(PeerDiscoveryStatus.ENR_REQUESTED);
-
     final Consumer<PeerInteractionState> action =
         interaction -> {
           final ENRRequestPacketData data = ENRRequestPacketData.create();
@@ -838,7 +811,6 @@ public class PeerDiscoveryController {
 
     private Cache<Bytes, Packet> cachedEnrRequests =
         CacheBuilder.newBuilder().maximumSize(50).expireAfterWrite(10, SECONDS).build();
-    private ForkIdManager forkIdManager;
     private RlpxAgent rlpxAgent;
 
     private Builder() {}
@@ -846,10 +818,6 @@ public class PeerDiscoveryController {
     public PeerDiscoveryController build() {
       validate();
 
-      if (peerTable == null) {
-        peerTable = new PeerTable(this.nodeKey.getPublicKey().getEncodedBytes(), 16);
-      }
-
       return new PeerDiscoveryController(
           nodeKey,
           localPeer,
@@ -864,7 +832,6 @@ public class PeerDiscoveryController {
           peerPermissions,
           metricsSystem,
           Optional.of(cachedEnrRequests),
-          forkIdManager,
           filterOnEnrForkId,
           rlpxAgent);
     }
@@ -875,8 +842,8 @@ public class PeerDiscoveryController {
       validateRequiredDependency(timerUtil, "TimerUtil");
       validateRequiredDependency(workerExecutor, "AsyncExecutor");
       validateRequiredDependency(metricsSystem, "MetricsSystem");
-      validateRequiredDependency(forkIdManager, "ForkIdManager");
       validateRequiredDependency(rlpxAgent, "RlpxAgent");
+      validateRequiredDependency(peerTable, "PeerTable");
     }
 
     private void validateRequiredDependency(final Object object, final String name) {
@@ -970,11 +937,5 @@ public class PeerDiscoveryController {
       this.rlpxAgent = rlpxAgent;
       return this;
     }
-
-    public Builder forkIdManager(final ForkIdManager forkIdManager) {
-      checkNotNull(forkIdManager);
-      this.forkIdManager = forkIdManager;
-      return this;
-    }
   }
 }

ethereum/p2p/src/main/java/org/hyperledger/besu/ethereum/p2p/discovery/internal/PeerTable.java

@@ -56,26 +56,21 @@ public class PeerTable {
    * Builds a new peer table, where distance is calculated using the provided nodeId as a baseline.
    *
    * @param nodeId The ID of the node where this peer table is stored.
-   * @param bucketSize The maximum length of each k-bucket.
    */
-  public PeerTable(final Bytes nodeId, final int bucketSize) {
+  public PeerTable(final Bytes nodeId) {
     this.keccak256 = Hash.keccak256(nodeId);
     this.table =
         Stream.generate(() -> new Bucket(DEFAULT_BUCKET_SIZE))
             .limit(N_BUCKETS + 1)
             .toArray(Bucket[]::new);
     this.distanceCache = new ConcurrentHashMap<>();
-    this.maxEntriesCnt = N_BUCKETS * bucketSize;
+    this.maxEntriesCnt = N_BUCKETS * DEFAULT_BUCKET_SIZE;
 
     // A bloom filter with 4096 expected insertions of 64-byte keys with a 0.1% false positive
     // probability yields a memory footprint of ~7.5kb.
     buildBloomFilter();
   }
 
-  public PeerTable(final Bytes nodeId) {
-    this(nodeId, DEFAULT_BUCKET_SIZE);
-  }
-
   /**
    * Returns the table's representation of a peer, if it exists.
    *
@@ -83,11 +78,12 @@ public class PeerTable {
    * @return The stored representation.
    */
   public Optional<DiscoveryPeer> get(final PeerId peer) {
-    if (!idBloom.mightContain(peer.getId())) {
+    final Bytes peerId = peer.getId();
+    if (!idBloom.mightContain(peerId)) {
       return Optional.empty();
     }
     final int distance = distanceFrom(peer);
-    return table[distance].getAndTouch(peer.getId());
+    return table[distance].getAndTouch(peerId);
   }
 
   /**

ethereum/p2p/src/main/java/org/hyperledger/besu/ethereum/p2p/network/DefaultP2PNetwork.java

@@ -27,6 +27,7 @@ import org.hyperledger.besu.ethereum.p2p.discovery.DiscoveryPeer;
 import org.hyperledger.besu.ethereum.p2p.discovery.PeerDiscoveryAgent;
 import org.hyperledger.besu.ethereum.p2p.discovery.PeerDiscoveryStatus;
 import org.hyperledger.besu.ethereum.p2p.discovery.VertxPeerDiscoveryAgent;
+import org.hyperledger.besu.ethereum.p2p.discovery.internal.PeerTable;
 import org.hyperledger.besu.ethereum.p2p.peers.DefaultPeerPrivileges;
 import org.hyperledger.besu.ethereum.p2p.peers.EnodeURLImpl;
 import org.hyperledger.besu.ethereum.p2p.peers.LocalNode;
@@ -383,11 +384,12 @@ public class DefaultP2PNetwork implements P2PNetwork {
   @VisibleForTesting
   void attemptPeerConnections() {
     LOG.trace("Initiating connections to discovered peers.");
-    rlpxAgent.connect(
+    final Stream<DiscoveryPeer> toTry =
         streamDiscoveredPeers()
             .filter(peer -> peer.getStatus() == PeerDiscoveryStatus.BONDED)
             .filter(peerDiscoveryAgent::checkForkId)
-            .sorted(Comparator.comparing(DiscoveryPeer::getLastAttemptedConnection)));
+            .sorted(Comparator.comparing(DiscoveryPeer::getLastAttemptedConnection));
+    toTry.forEach(rlpxAgent::connect);
   }
 
   @Override
@@ -511,6 +513,7 @@ public class DefaultP2PNetwork implements P2PNetwork {
     private Supplier<Stream<PeerConnection>> allConnectionsSupplier;
     private Supplier<Stream<PeerConnection>> allActiveConnectionsSupplier;
     private int peersLowerBound;
+    private PeerTable peerTable;
 
     public P2PNetwork build() {
       validate();
@@ -528,6 +531,7 @@ public class DefaultP2PNetwork implements P2PNetwork {
       final MutableLocalNode localNode =
           MutableLocalNode.create(config.getRlpx().getClientId(), 5, supportedCapabilities);
       final PeerPrivileges peerPrivileges = new DefaultPeerPrivileges(maintainedPeers);
+      peerTable = new PeerTable(nodeKey.getPublicKey().getEncodedBytes());
       rlpxAgent = rlpxAgent == null ? createRlpxAgent(localNode, peerPrivileges) : rlpxAgent;
       peerDiscoveryAgent = peerDiscoveryAgent == null ? createDiscoveryAgent() : peerDiscoveryAgent;
 
@@ -572,7 +576,8 @@ public class DefaultP2PNetwork implements P2PNetwork {
           metricsSystem,
           storageProvider,
           forkIdManager,
-          rlpxAgent);
+          rlpxAgent,
+          peerTable);
     }
 
     private RlpxAgent createRlpxAgent(
@@ -589,6 +594,7 @@ public class DefaultP2PNetwork implements P2PNetwork {
           .allConnectionsSupplier(allConnectionsSupplier)
           .allActiveConnectionsSupplier(allActiveConnectionsSupplier)
           .peersLowerBound(peersLowerBound)
+          .peerTable(peerTable)
           .build();
     }
 

ethereum/p2p/src/main/java/org/hyperledger/besu/ethereum/p2p/rlpx/RlpxAgent.java

@@ -20,6 +20,7 @@ import static com.google.common.base.Preconditions.checkState;
 import org.hyperledger.besu.cryptoservices.NodeKey;
 import org.hyperledger.besu.ethereum.p2p.config.RlpxConfiguration;
 import org.hyperledger.besu.ethereum.p2p.discovery.DiscoveryPeer;
+import org.hyperledger.besu.ethereum.p2p.discovery.internal.PeerTable;
 import org.hyperledger.besu.ethereum.p2p.peers.LocalNode;
 import org.hyperledger.besu.ethereum.p2p.peers.Peer;
 import org.hyperledger.besu.ethereum.p2p.peers.PeerPrivileges;
@@ -162,13 +163,6 @@ public class RlpxAgent {
     }
   }
 
-  public void connect(final Stream<? extends Peer> peerStream) {
-    if (!localNode.isReady()) {
-      return;
-    }
-    peerStream.forEach(this::connect);
-  }
-
   public void disconnect(final Bytes peerId, final DisconnectReason reason) {
     try {
       allActiveConnectionsSupplier
@@ -206,6 +200,7 @@ public class RlpxAgent {
                   + this.getClass().getSimpleName()
                   + " has finished starting"));
     }
+
     // Check peer is valid
     final EnodeURL enode = peer.getEnodeURL();
     if (!enode.isListening()) {
@@ -380,6 +375,7 @@ public class RlpxAgent {
     private Supplier<Stream<PeerConnection>> allConnectionsSupplier;
     private Supplier<Stream<PeerConnection>> allActiveConnectionsSupplier;
     private int peersLowerBound;
+    private PeerTable peerTable;
 
     private Builder() {}
 
@@ -399,12 +395,13 @@ public class RlpxAgent {
                   localNode,
                   connectionEvents,
                   metricsSystem,
-                  p2pTLSConfiguration.get());
+                  p2pTLSConfiguration.get(),
+                  peerTable);
         } else {
           LOG.debug("Using default NettyConnectionInitializer");
           connectionInitializer =
               new NettyConnectionInitializer(
-                  nodeKey, config, localNode, connectionEvents, metricsSystem);
+                  nodeKey, config, localNode, connectionEvents, metricsSystem, peerTable);
         }
       }
 
@@ -499,5 +496,10 @@ public class RlpxAgent {
       this.peersLowerBound = peersLowerBound;
       return this;
     }
+
+    public Builder peerTable(final PeerTable peerTable) {
+      this.peerTable = peerTable;
+      return this;
+    }
   }
 }

ethereum/p2p/src/main/java/org/hyperledger/besu/ethereum/p2p/rlpx/connections/netty/AbstractHandshakeHandler.java

@@ -14,6 +14,7 @@
  */
 package org.hyperledger.besu.ethereum.p2p.rlpx.connections.netty;
 
+import org.hyperledger.besu.ethereum.p2p.discovery.internal.PeerTable;
 import org.hyperledger.besu.ethereum.p2p.peers.LocalNode;
 import org.hyperledger.besu.ethereum.p2p.peers.Peer;
 import org.hyperledger.besu.ethereum.p2p.rlpx.connections.PeerConnection;
@@ -60,6 +61,7 @@ abstract class AbstractHandshakeHandler extends SimpleChannelInboundHandler<Byte
 
   private final FramerProvider framerProvider;
   private final boolean inboundInitiated;
+  private final PeerTable peerTable;
 
   AbstractHandshakeHandler(
       final List<SubProtocol> subProtocols,
@@ -70,7 +72,8 @@ abstract class AbstractHandshakeHandler extends SimpleChannelInboundHandler<Byte
       final MetricsSystem metricsSystem,
       final HandshakerProvider handshakerProvider,
       final FramerProvider framerProvider,
-      final boolean inboundInitiated) {
+      final boolean inboundInitiated,
+      final PeerTable peerTable) {
     this.subProtocols = subProtocols;
     this.localNode = localNode;
     this.expectedPeer = expectedPeer;
@@ -80,6 +83,7 @@ abstract class AbstractHandshakeHandler extends SimpleChannelInboundHandler<Byte
     this.handshaker = handshakerProvider.buildInstance();
     this.framerProvider = framerProvider;
     this.inboundInitiated = inboundInitiated;
+    this.peerTable = peerTable;
   }
 
   /**
@@ -97,47 +101,48 @@ abstract class AbstractHandshakeHandler extends SimpleChannelInboundHandler<Byte
       ctx.writeAndFlush(nextMsg.get());
     } else if (handshaker.getStatus() != Handshaker.HandshakeStatus.SUCCESS) {
       LOG.debug("waiting for more bytes");
-      return;
+    } else {
+
+      final Bytes nodeId = handshaker.partyPubKey().getEncodedBytes();
+      if (!localNode.isReady()) {
+        // If we're handling a connection before the node is fully up, just disconnect
+        LOG.debug("Rejecting connection because local node is not ready {}", nodeId);
+        disconnect(ctx, DisconnectMessage.DisconnectReason.UNKNOWN);
+        return;
+      }
+
+      LOG.trace("Sending framed hello");
+
+      // Exchange keys done
+      final Framer framer = this.framerProvider.buildFramer(handshaker.secrets());
+
+      final ByteToMessageDecoder deFramer =
+          new DeFramer(
+              framer,
+              subProtocols,
+              localNode,
+              expectedPeer,
+              connectionEventDispatcher,
+              connectionFuture,
+              metricsSystem,
+              inboundInitiated,
+              peerTable);
+
+      ctx.channel()
+          .pipeline()
+          .replace(this, "DeFramer", deFramer)
+          .addBefore("DeFramer", "validate", new ValidateFirstOutboundMessage(framer));
+
+      ctx.writeAndFlush(new OutboundMessage(null, HelloMessage.create(localNode.getPeerInfo())))
+          .addListener(
+              ff -> {
+                if (ff.isSuccess()) {
+                  LOG.trace("Successfully wrote hello message");
+                }
+              });
+      msg.retain();
+      ctx.fireChannelRead(msg);
     }
-
-    final Bytes nodeId = handshaker.partyPubKey().getEncodedBytes();
-    if (!localNode.isReady()) {
-      // If we're handling a connection before the node is fully up, just disconnect
-      LOG.debug("Rejecting connection because local node is not ready {}", nodeId);
-      disconnect(ctx, DisconnectMessage.DisconnectReason.UNKNOWN);
-      return;
-    }
-
-    LOG.trace("Sending framed hello");
-
-    // Exchange keys done
-    final Framer framer = this.framerProvider.buildFramer(handshaker.secrets());
-
-    final ByteToMessageDecoder deFramer =
-        new DeFramer(
-            framer,
-            subProtocols,
-            localNode,
-            expectedPeer,
-            connectionEventDispatcher,
-            connectionFuture,
-            metricsSystem,
-            inboundInitiated);
-
-    ctx.channel()
-        .pipeline()
-        .replace(this, "DeFramer", deFramer)
-        .addBefore("DeFramer", "validate", new ValidateFirstOutboundMessage(framer));
-
-    ctx.writeAndFlush(new OutboundMessage(null, HelloMessage.create(localNode.getPeerInfo())))
-        .addListener(
-            ff -> {
-              if (ff.isSuccess()) {
-                LOG.trace("Successfully wrote hello message");
-              }
-            });
-    msg.retain();
-    ctx.fireChannelRead(msg);
   }
 
   private void disconnect(

ethereum/p2p/src/main/java/org/hyperledger/besu/ethereum/p2p/rlpx/connections/netty/DeFramer.java

@@ -14,6 +14,8 @@
  */
 package org.hyperledger.besu.ethereum.p2p.rlpx.connections.netty;
 
+import org.hyperledger.besu.ethereum.p2p.discovery.DiscoveryPeer;
+import org.hyperledger.besu.ethereum.p2p.discovery.internal.PeerTable;
 import org.hyperledger.besu.ethereum.p2p.network.exceptions.BreachOfProtocolException;
 import org.hyperledger.besu.ethereum.p2p.network.exceptions.IncompatiblePeerException;
 import org.hyperledger.besu.ethereum.p2p.network.exceptions.PeerChannelClosedException;
@@ -70,6 +72,7 @@ final class DeFramer extends ByteToMessageDecoder {
   private final Optional<Peer> expectedPeer;
   private final List<SubProtocol> subProtocols;
   private final boolean inboundInitiated;
+  private final PeerTable peerTable;
   private boolean hellosExchanged;
   private final LabelledMetric<Counter> outboundMessagesCounter;
 
@@ -81,7 +84,8 @@ final class DeFramer extends ByteToMessageDecoder {
       final PeerConnectionEventDispatcher connectionEventDispatcher,
       final CompletableFuture<PeerConnection> connectFuture,
       final MetricsSystem metricsSystem,
-      final boolean inboundInitiated) {
+      final boolean inboundInitiated,
+      final PeerTable peerTable) {
     this.framer = framer;
     this.subProtocols = subProtocols;
     this.localNode = localNode;
@@ -89,6 +93,7 @@ final class DeFramer extends ByteToMessageDecoder {
     this.connectFuture = connectFuture;
     this.connectionEventDispatcher = connectionEventDispatcher;
     this.inboundInitiated = inboundInitiated;
+    this.peerTable = peerTable;
     this.outboundMessagesCounter =
         metricsSystem.createLabelledCounter(
             BesuMetricCategory.NETWORK,
@@ -105,8 +110,11 @@ final class DeFramer extends ByteToMessageDecoder {
     while ((message = framer.deframe(in)) != null) {
 
       if (hellosExchanged) {
+
         out.add(message);
+
       } else if (message.getCode() == WireMessageCodes.HELLO) {
+
         hellosExchanged = true;
         // Decode first hello and use the payload to modify pipeline
         final PeerInfo peerInfo;
@@ -129,13 +137,27 @@ final class DeFramer extends ByteToMessageDecoder {
                 subProtocols,
                 localNode.getPeerInfo().getCapabilities(),
                 peerInfo.getCapabilities());
-        final Optional<Peer> peer = expectedPeer.or(() -> createPeer(peerInfo, ctx));
+
-        if (peer.isEmpty()) {
+        Optional<Peer> peer;
-          LOG.debug("Failed to create connection for peer {}", peerInfo);
+        if (expectedPeer.isPresent()) {
-          connectFuture.completeExceptionally(new PeerChannelClosedException(peerInfo));
+          peer = expectedPeer;
-          ctx.close();
+        } else {
-          return;
+          // This is an inbound "Hello" message. Create peer from information from the Hello message
+          peer = createPeer(peerInfo, ctx);
+          if (peer.isEmpty()) {
+            LOG.debug("Failed to create connection for peer {}", peerInfo);
+            connectFuture.completeExceptionally(new PeerChannelClosedException(peerInfo));
+            ctx.close();
+            return;
+          }
+          // If we can find the DiscoveryPeer for the peer in the PeerTable we use it, because
+          // it could contains additional information, like the fork id.
+          final Optional<DiscoveryPeer> discoveryPeer = peerTable.get(peer.get());
+          if (discoveryPeer.isPresent()) {
+            peer = Optional.of(discoveryPeer.get());
+          }
         }
+
         final PeerConnection connection =
             new NettyPeerConnection(
                 ctx,
@@ -176,7 +198,9 @@ final class DeFramer extends ByteToMessageDecoder {
                     capabilityMultiplexer, connection, connectionEventDispatcher, waitingForPong),
                 new MessageFramer(capabilityMultiplexer, framer));
         connectFuture.complete(connection);
+
       } else if (message.getCode() == WireMessageCodes.DISCONNECT) {
+
         final DisconnectMessage disconnectMessage = DisconnectMessage.readFrom(message);
         LOG.debug(
             "Peer {} disconnected before sending HELLO.  Reason: {}",
@@ -185,8 +209,10 @@ final class DeFramer extends ByteToMessageDecoder {
         ctx.close();
         connectFuture.completeExceptionally(
             new PeerDisconnectedException(disconnectMessage.getReason()));
+
       } else {
         // Unexpected message - disconnect
+
         LOG.debug(
             "Message received before HELLO's exchanged (BREACH_OF_PROTOCOL), disconnecting.  Peer: {}, Code: {}, Data: {}",
             expectedPeer.map(Peer::getEnodeURLString).orElse("unknown"),

ethereum/p2p/src/main/java/org/hyperledger/besu/ethereum/p2p/rlpx/connections/netty/HandshakeHandlerInbound.java

@@ -15,6 +15,7 @@
 package org.hyperledger.besu.ethereum.p2p.rlpx.connections.netty;
 
 import org.hyperledger.besu.cryptoservices.NodeKey;
+import org.hyperledger.besu.ethereum.p2p.discovery.internal.PeerTable;
 import org.hyperledger.besu.ethereum.p2p.peers.LocalNode;
 import org.hyperledger.besu.ethereum.p2p.rlpx.connections.PeerConnection;
 import org.hyperledger.besu.ethereum.p2p.rlpx.connections.PeerConnectionEventDispatcher;
@@ -40,7 +41,8 @@ final class HandshakeHandlerInbound extends AbstractHandshakeHandler {
       final PeerConnectionEventDispatcher connectionEventDispatcher,
       final MetricsSystem metricsSystem,
       final HandshakerProvider handshakerProvider,
-      final FramerProvider framerProvider) {
+      final FramerProvider framerProvider,
+      final PeerTable peerTable) {
     super(
         subProtocols,
         localNode,
@@ -50,7 +52,8 @@ final class HandshakeHandlerInbound extends AbstractHandshakeHandler {
         metricsSystem,
         handshakerProvider,
         framerProvider,
-        true);
+        true,
+        peerTable);
     handshaker.prepareResponder(nodeKey);
   }
 

ethereum/p2p/src/main/java/org/hyperledger/besu/ethereum/p2p/rlpx/connections/netty/HandshakeHandlerOutbound.java

@@ -16,6 +16,7 @@ package org.hyperledger.besu.ethereum.p2p.rlpx.connections.netty;
 
 import org.hyperledger.besu.crypto.SignatureAlgorithmFactory;
 import org.hyperledger.besu.cryptoservices.NodeKey;
+import org.hyperledger.besu.ethereum.p2p.discovery.internal.PeerTable;
 import org.hyperledger.besu.ethereum.p2p.peers.LocalNode;
 import org.hyperledger.besu.ethereum.p2p.peers.Peer;
 import org.hyperledger.besu.ethereum.p2p.rlpx.connections.PeerConnection;
@@ -50,7 +51,8 @@ final class HandshakeHandlerOutbound extends AbstractHandshakeHandler {
       final PeerConnectionEventDispatcher connectionEventDispatcher,
       final MetricsSystem metricsSystem,
       final HandshakerProvider handshakerProvider,
-      final FramerProvider framerProvider) {
+      final FramerProvider framerProvider,
+      final PeerTable peerTable) {
     super(
         subProtocols,
         localNode,
@@ -60,7 +62,8 @@ final class HandshakeHandlerOutbound extends AbstractHandshakeHandler {
         metricsSystem,
         handshakerProvider,
         framerProvider,
-        false);
+        false,
+        peerTable);
     handshaker.prepareInitiator(
         nodeKey, SignatureAlgorithmFactory.getInstance().createPublicKey(peer.getId()));
     this.first = handshaker.firstMessage();

ethereum/p2p/src/main/java/org/hyperledger/besu/ethereum/p2p/rlpx/connections/netty/NettyConnectionInitializer.java

@@ -17,6 +17,7 @@ package org.hyperledger.besu.ethereum.p2p.rlpx.connections.netty;
 import org.hyperledger.besu.cryptoservices.NodeKey;
 import org.hyperledger.besu.ethereum.p2p.config.RlpxConfiguration;
 import org.hyperledger.besu.ethereum.p2p.discovery.DiscoveryPeer;
+import org.hyperledger.besu.ethereum.p2p.discovery.internal.PeerTable;
 import org.hyperledger.besu.ethereum.p2p.peers.LocalNode;
 import org.hyperledger.besu.ethereum.p2p.peers.Peer;
 import org.hyperledger.besu.ethereum.p2p.rlpx.ConnectCallback;
@@ -68,6 +69,7 @@ public class NettyConnectionInitializer
   private final PeerConnectionEventDispatcher eventDispatcher;
   private final MetricsSystem metricsSystem;
   private final Subscribers<ConnectCallback> connectSubscribers = Subscribers.create();
+  private final PeerTable peerTable;
 
   private ChannelFuture server;
   private final EventLoopGroup boss = new NioEventLoopGroup(1);
@@ -80,12 +82,14 @@ public class NettyConnectionInitializer
       final RlpxConfiguration config,
       final LocalNode localNode,
       final PeerConnectionEventDispatcher eventDispatcher,
-      final MetricsSystem metricsSystem) {
+      final MetricsSystem metricsSystem,
+      final PeerTable peerTable) {
     this.nodeKey = nodeKey;
     this.config = config;
     this.localNode = localNode;
     this.eventDispatcher = eventDispatcher;
     this.metricsSystem = metricsSystem;
+    this.peerTable = peerTable;
 
     metricsSystem.createIntegerGauge(
         BesuMetricCategory.NETWORK,
@@ -244,7 +248,8 @@ public class NettyConnectionInitializer
         eventDispatcher,
         metricsSystem,
         this,
-        this);
+        this,
+        peerTable);
   }
 
   @Nonnull
@@ -259,7 +264,8 @@ public class NettyConnectionInitializer
         eventDispatcher,
         metricsSystem,
         this,
-        this);
+        this,
+        peerTable);
   }
 
   @Nonnull

ethereum/p2p/src/main/java/org/hyperledger/besu/ethereum/p2p/rlpx/connections/netty/NettyTLSConnectionInitializer.java

@@ -19,6 +19,7 @@ import static org.hyperledger.besu.ethereum.p2p.rlpx.RlpxFrameConstants.LENGTH_M
 
 import org.hyperledger.besu.cryptoservices.NodeKey;
 import org.hyperledger.besu.ethereum.p2p.config.RlpxConfiguration;
+import org.hyperledger.besu.ethereum.p2p.discovery.internal.PeerTable;
 import org.hyperledger.besu.ethereum.p2p.peers.LocalNode;
 import org.hyperledger.besu.ethereum.p2p.peers.Peer;
 import org.hyperledger.besu.ethereum.p2p.plain.PlainFramer;
@@ -55,7 +56,8 @@ public class NettyTLSConnectionInitializer extends NettyConnectionInitializer {
       final LocalNode localNode,
       final PeerConnectionEventDispatcher eventDispatcher,
       final MetricsSystem metricsSystem,
-      final TLSConfiguration p2pTLSConfiguration) {
+      final TLSConfiguration p2pTLSConfiguration,
+      final PeerTable peerTable) {
     this(
         nodeKey,
         config,
@@ -63,7 +65,8 @@ public class NettyTLSConnectionInitializer extends NettyConnectionInitializer {
         eventDispatcher,
         metricsSystem,
         defaultTlsContextFactorySupplier(p2pTLSConfiguration),
-        p2pTLSConfiguration.getClientHelloSniHeaderEnabled());
+        p2pTLSConfiguration.getClientHelloSniHeaderEnabled(),
+        peerTable);
   }
 
   @VisibleForTesting
@@ -74,8 +77,9 @@ public class NettyTLSConnectionInitializer extends NettyConnectionInitializer {
       final PeerConnectionEventDispatcher eventDispatcher,
       final MetricsSystem metricsSystem,
       final Supplier<TLSContextFactory> tlsContextFactorySupplier,
-      final Boolean clientHelloSniHeaderEnabled) {
+      final Boolean clientHelloSniHeaderEnabled,
-    super(nodeKey, config, localNode, eventDispatcher, metricsSystem);
+      final PeerTable peerTable) {
+    super(nodeKey, config, localNode, eventDispatcher, metricsSystem, peerTable);
     if (tlsContextFactorySupplier != null) {
       this.tlsContextFactorySupplier =
           Optional.of(Suppliers.memoize(tlsContextFactorySupplier::get));

ethereum/p2p/src/test/java/org/hyperledger/besu/ethereum/p2p/discovery/PeerDiscoveryAgentTest.java

@@ -244,6 +244,30 @@ public class PeerDiscoveryAgentTest {
     }
   }
 
+  @Test
+  public void endpointHonoursCustomAdvertisedAddressInPingPacket() {
+
+    // Start a peer with the default advertised host
+    final MockPeerDiscoveryAgent agent1 = helper.startDiscoveryAgent();
+
+    // Start another peer with its advertised host set to a custom value
+    final MockPeerDiscoveryAgent agent2 = helper.startDiscoveryAgent("192.168.0.1");
+
+    // Send a PING so we can exchange messages
+    Packet packet = helper.createPingPacket(agent2, agent1);
+    helper.sendMessageBetweenAgents(agent2, agent1, packet);
+
+    // Agent 1's peers should have endpoints that match the custom advertised value...
+    agent1
+        .streamDiscoveredPeers()
+        .forEach(peer -> assertThat(peer.getEndpoint().getHost()).isEqualTo("192.168.0.1"));
+
+    // ...but agent 2's peers should have endpoints that match the default
+    agent2
+        .streamDiscoveredPeers()
+        .forEach(peer -> assertThat(peer.getEndpoint().getHost()).isEqualTo("127.0.0.1"));
+  }
+
   @Test
   public void shouldEvictPeerWhenPermissionsRevoked() {
     final PeerPermissionsDenylist denylist = PeerPermissionsDenylist.create();

ethereum/p2p/src/test/java/org/hyperledger/besu/ethereum/p2p/discovery/PeerDiscoveryTestHelper.java

@@ -165,6 +165,14 @@ public class PeerDiscoveryTestHelper {
     return startDiscoveryAgent(agentBuilder);
   }
 
+  public MockPeerDiscoveryAgent startDiscoveryAgent(
+      final String advertisedHost, final DiscoveryPeer... bootstrapPeers) {
+    final AgentBuilder agentBuilder =
+        agentBuilder().bootstrapPeers(bootstrapPeers).advertisedHost(advertisedHost);
+
+    return startDiscoveryAgent(agentBuilder);
+  }
+
   /**
    * Start a single discovery agent with the provided bootstrap peers.
    *
@@ -287,6 +295,7 @@ public class PeerDiscoveryTestHelper {
       config.setAdvertisedHost(advertisedHost);
       config.setBindPort(port);
       config.setActive(active);
+      config.setFilterOnEnrForkId(false);
 
       final ForkIdManager mockForkIdManager = mock(ForkIdManager.class);
       final ForkId forkId = new ForkId(Bytes.EMPTY, Bytes.EMPTY);

ethereum/p2p/src/test/java/org/hyperledger/besu/ethereum/p2p/discovery/internal/ENRResponsePacketDataTest.java

@@ -20,6 +20,8 @@ import org.hyperledger.besu.ethereum.rlp.BytesValueRLPOutput;
 import org.hyperledger.besu.ethereum.rlp.RLP;
 
 import org.apache.tuweni.bytes.Bytes;
+import org.apache.tuweni.bytes.Bytes32;
+import org.apache.tuweni.crypto.SECP256K1;
 import org.apache.tuweni.units.bigints.UInt64;
 import org.ethereum.beacon.discovery.schema.EnrField;
 import org.ethereum.beacon.discovery.schema.IdentitySchema;
@@ -34,8 +36,10 @@ public class ENRResponsePacketDataTest {
     final Bytes requestHash = Bytes.fromHexStringLenient("0x1234");
     final Bytes nodeId =
         Bytes.fromHexString("a448f24c6d18e575453db13171562b71999873db5b286df957af199ec94617f7");
-    final Bytes privateKey =
+    final SECP256K1.SecretKey privateKey =
-        Bytes.fromHexString("b71c71a67e1177ad4e901695e1b4b9ee17ae16c6668d313eac2f96dbcda3f291");
+        SECP256K1.SecretKey.fromBytes(
+            Bytes32.fromHexString(
+                "b71c71a67e1177ad4e901695e1b4b9ee17ae16c6668d313eac2f96dbcda3f291"));
 
     NodeRecord nodeRecord =
         NodeRecordFactory.DEFAULT.createFromValues(
@@ -48,7 +52,8 @@ public class ENRResponsePacketDataTest {
             new EnrField(EnrField.TCP, 8080),
             new EnrField(EnrField.TCP_V6, 8080),
             new EnrField(
-                EnrField.PKEY_SECP256K1, Functions.derivePublicKeyFromPrivate(privateKey)));
+                EnrField.PKEY_SECP256K1,
+                Functions.deriveCompressedPublicKeyFromPrivate(privateKey)));
     nodeRecord.sign(privateKey);
 
     assertThat(nodeRecord.getNodeId()).isEqualTo(nodeId);
@@ -72,8 +77,10 @@ public class ENRResponsePacketDataTest {
     final Bytes requestHash = Bytes.fromHexStringLenient("0x1234");
     final Bytes nodeId =
         Bytes.fromHexString("a448f24c6d18e575453db13171562b71999873db5b286df957af199ec94617f7");
-    final Bytes privateKey =
+    final SECP256K1.SecretKey privateKey =
-        Bytes.fromHexString("b71c71a67e1177ad4e901695e1b4b9ee17ae16c6668d313eac2f96dbcda3f291");
+        SECP256K1.SecretKey.fromBytes(
+            Bytes32.fromHexString(
+                "b71c71a67e1177ad4e901695e1b4b9ee17ae16c6668d313eac2f96dbcda3f291"));
 
     NodeRecord nodeRecord =
         NodeRecordFactory.DEFAULT.createFromValues(
@@ -82,7 +89,8 @@ public class ENRResponsePacketDataTest {
             new EnrField(EnrField.IP_V4, Bytes.fromHexString("0x7F000001")),
             new EnrField(EnrField.UDP, 30303),
             new EnrField(
-                EnrField.PKEY_SECP256K1, Functions.derivePublicKeyFromPrivate(privateKey)));
+                EnrField.PKEY_SECP256K1,
+                Functions.deriveCompressedPublicKeyFromPrivate(privateKey)));
     nodeRecord.sign(privateKey);
 
     assertThat(nodeRecord.getNodeId()).isEqualTo(nodeId);
@@ -109,8 +117,10 @@ public class ENRResponsePacketDataTest {
     final Bytes requestHash = Bytes.fromHexStringLenient("0x1234");
     final Bytes nodeId =
         Bytes.fromHexString("a448f24c6d18e575453db13171562b71999873db5b286df957af199ec94617f7");
-    final Bytes privateKey =
+    final SECP256K1.SecretKey privateKey =
-        Bytes.fromHexString("b71c71a67e1177ad4e901695e1b4b9ee17ae16c6668d313eac2f96dbcda3f291");
+        SECP256K1.SecretKey.fromBytes(
+            Bytes32.fromHexString(
+                "b71c71a67e1177ad4e901695e1b4b9ee17ae16c6668d313eac2f96dbcda3f291"));
 
     NodeRecord nodeRecord =
         NodeRecordFactory.DEFAULT.createFromValues(
@@ -119,7 +129,8 @@ public class ENRResponsePacketDataTest {
             new EnrField(EnrField.IP_V4, Bytes.fromHexString("0x7F000001")),
             new EnrField(EnrField.UDP, 30303),
             new EnrField(
-                EnrField.PKEY_SECP256K1, Functions.derivePublicKeyFromPrivate(privateKey)));
+                EnrField.PKEY_SECP256K1,
+                Functions.deriveCompressedPublicKeyFromPrivate(privateKey)));
     nodeRecord.sign(privateKey);
 
     assertThat(nodeRecord.getNodeId()).isEqualTo(nodeId);
@@ -144,8 +155,10 @@ public class ENRResponsePacketDataTest {
     final Bytes requestHash = Bytes.fromHexStringLenient("0x1234");
     final Bytes nodeId =
         Bytes.fromHexString("a448f24c6d18e575453db13171562b71999873db5b286df957af199ec94617f7");
-    final Bytes privateKey =
+    final SECP256K1.SecretKey privateKey =
-        Bytes.fromHexString("b71c71a67e1177ad4e901695e1b4b9ee17ae16c6668d313eac2f96dbcda3f291");
+        SECP256K1.SecretKey.fromBytes(
+            Bytes32.fromHexString(
+                "b71c71a67e1177ad4e901695e1b4b9ee17ae16c6668d313eac2f96dbcda3f291"));
 
     NodeRecord nodeRecord =
         NodeRecordFactory.DEFAULT.createFromValues(
@@ -153,7 +166,9 @@ public class ENRResponsePacketDataTest {
             new EnrField(EnrField.ID, IdentitySchema.V4),
             new EnrField(EnrField.IP_V4, Bytes.fromHexString("0x7F000001")),
             new EnrField(EnrField.UDP, 30303),
-            new EnrField(EnrField.PKEY_SECP256K1, Functions.derivePublicKeyFromPrivate(privateKey)),
+            new EnrField(
+                EnrField.PKEY_SECP256K1,
+                Functions.deriveCompressedPublicKeyFromPrivate(privateKey)),
             new EnrField("foo", Bytes.fromHexString("0x1234")));
     nodeRecord.sign(privateKey);
 
@@ -181,8 +196,10 @@ public class ENRResponsePacketDataTest {
   @Test
   public void readFrom_invalidSignature() {
     final Bytes requestHash = Bytes.fromHexStringLenient("0x1234");
-    final Bytes privateKey =
+    final SECP256K1.SecretKey privateKey =
-        Bytes.fromHexString("b71c71a67e1177ad4e901695e1b4b9ee17ae16c6668d313eac2f96dbcda3f292");
+        SECP256K1.SecretKey.fromBytes(
+            Bytes32.fromHexString(
+                "b71c71a67e1177ad4e901695e1b4b9ee17ae16c6668d313eac2f96dbcda3f292"));
 
     NodeRecord nodeRecord =
         NodeRecordFactory.DEFAULT.createFromValues(
@@ -191,7 +208,8 @@ public class ENRResponsePacketDataTest {
             new EnrField(EnrField.IP_V4, Bytes.fromHexString("0x7F000001")),
             new EnrField(EnrField.UDP, 30303),
             new EnrField(
-                EnrField.PKEY_SECP256K1, Functions.derivePublicKeyFromPrivate(privateKey)));
+                EnrField.PKEY_SECP256K1,
+                Functions.deriveCompressedPublicKeyFromPrivate(privateKey)));
     nodeRecord.sign(privateKey);
     nodeRecord.set(EnrField.UDP, 1234);
 

ethereum/p2p/src/test/java/org/hyperledger/besu/ethereum/p2p/discovery/internal/MockPeerDiscoveryAgent.java

@@ -63,7 +63,8 @@ public class MockPeerDiscoveryAgent extends PeerDiscoveryAgent {
         new NoOpMetricsSystem(),
         new InMemoryKeyValueStorageProvider(),
         forkIdManager,
-        rlpxAgent);
+        rlpxAgent,
+        new PeerTable(nodeKey.getPublicKey().getEncodedBytes()));
     this.agentNetwork = agentNetwork;
   }
 

ethereum/p2p/src/test/java/org/hyperledger/besu/ethereum/p2p/discovery/internal/PeerDiscoveryControllerTest.java

@@ -35,8 +35,6 @@ import org.hyperledger.besu.crypto.Hash;
 import org.hyperledger.besu.crypto.SignatureAlgorithm;
 import org.hyperledger.besu.crypto.SignatureAlgorithmFactory;
 import org.hyperledger.besu.cryptoservices.NodeKey;
-import org.hyperledger.besu.ethereum.forkid.ForkId;
-import org.hyperledger.besu.ethereum.forkid.ForkIdManager;
 import org.hyperledger.besu.ethereum.p2p.discovery.DiscoveryPeer;
 import org.hyperledger.besu.ethereum.p2p.discovery.Endpoint;
 import org.hyperledger.besu.ethereum.p2p.discovery.PeerDiscoveryStatus;
@@ -1480,14 +1478,12 @@ public class PeerDiscoveryControllerTest {
   }
 
   @Test
-  public void shouldFiltersOnForkIdSuccess() {
+  public void forkIdShouldBeAvailableIfEnrPacketContainsForkId() {
     final List<NodeKey> nodeKeys = PeerDiscoveryTestHelper.generateNodeKeys(1);
     final List<DiscoveryPeer> peers = helper.createDiscoveryPeers(nodeKeys);
-    final ForkIdManager forkIdManager = mock(ForkIdManager.class);
     final DiscoveryPeer sender = peers.get(0);
-    final Packet enrPacket = prepareForForkIdCheck(forkIdManager, nodeKeys, sender, true);
+    final Packet enrPacket = prepareForForkIdCheck(nodeKeys, sender, true);
 
-    when(forkIdManager.peerCheck(any(ForkId.class))).thenReturn(true);
     controller.onMessage(enrPacket, sender);
 
     final Optional<DiscoveryPeer> maybePeer =
@@ -1501,35 +1497,12 @@ public class PeerDiscoveryControllerTest {
     verify(controller, times(1)).connectOnRlpxLayer(eq(maybePeer.get()));
   }
 
-  @Test
-  public void shouldFiltersOnForkIdFailure() {
-    final List<NodeKey> nodeKeys = PeerDiscoveryTestHelper.generateNodeKeys(1);
-    final List<DiscoveryPeer> peers = helper.createDiscoveryPeers(nodeKeys);
-    final ForkIdManager forkIdManager = mock(ForkIdManager.class);
-    final DiscoveryPeer sender = peers.get(0);
-    final Packet enrPacket = prepareForForkIdCheck(forkIdManager, nodeKeys, sender, true);
-
-    when(forkIdManager.peerCheck(any(ForkId.class))).thenReturn(false);
-    controller.onMessage(enrPacket, sender);
-
-    final Optional<DiscoveryPeer> maybePeer =
-        controller
-            .streamDiscoveredPeers()
-            .filter(p -> p.getId().equals(sender.getId()))
-            .findFirst();
-
-    assertThat(maybePeer.isPresent()).isTrue();
-    assertThat(maybePeer.get().getForkId().isPresent()).isTrue();
-    verify(controller, never()).connectOnRlpxLayer(eq(maybePeer.get()));
-  }
-
   @Test
   public void shouldStillCallConnectIfNoForkIdSent() {
     final List<NodeKey> nodeKeys = PeerDiscoveryTestHelper.generateNodeKeys(1);
     final List<DiscoveryPeer> peers = helper.createDiscoveryPeers(nodeKeys);
     final DiscoveryPeer sender = peers.get(0);
-    final Packet enrPacket =
+    final Packet enrPacket = prepareForForkIdCheck(nodeKeys, sender, false);
-        prepareForForkIdCheck(mock(ForkIdManager.class), nodeKeys, sender, false);
 
     controller.onMessage(enrPacket, sender);
 
@@ -1546,10 +1519,7 @@ public class PeerDiscoveryControllerTest {
 
   @NotNull
   private Packet prepareForForkIdCheck(
-      final ForkIdManager forkIdManager,
+      final List<NodeKey> nodeKeys, final DiscoveryPeer sender, final boolean sendForkId) {
-      final List<NodeKey> nodeKeys,
-      final DiscoveryPeer sender,
-      final boolean sendForkId) {
     final HashMap<PacketType, Bytes> packetTypeBytesHashMap = new HashMap<>();
     final OutboundMessageHandler outboundMessageHandler =
         (dp, pa) -> packetTypeBytesHashMap.put(pa.getType(), pa.getHash());
@@ -1573,7 +1543,6 @@ public class PeerDiscoveryControllerTest {
             .outboundMessageHandler(outboundMessageHandler)
             .enrCache(enrs)
             .filterOnForkId(true)
-            .forkIdManager(forkIdManager)
             .build();
 
     // Mock the creation of the PING packet, so that we can control the hash, which gets validated
@@ -1720,7 +1689,6 @@ public class PeerDiscoveryControllerTest {
     private Cache<Bytes, Packet> enrs =
         CacheBuilder.newBuilder().maximumSize(50).expireAfterWrite(10, TimeUnit.SECONDS).build();
     private boolean filterOnForkId = false;
-    private ForkIdManager forkIdManager;
 
     public static ControllerBuilder create() {
       return new ControllerBuilder();
@@ -1776,11 +1744,6 @@ public class PeerDiscoveryControllerTest {
       return this;
     }
 
-    public ControllerBuilder forkIdManager(final ForkIdManager forkIdManager) {
-      this.forkIdManager = forkIdManager;
-      return this;
-    }
-
     PeerDiscoveryController build() {
       checkNotNull(nodeKey);
       if (localPeer == null) {
@@ -1803,7 +1766,6 @@ public class PeerDiscoveryControllerTest {
               .peerPermissions(peerPermissions)
               .metricsSystem(new NoOpMetricsSystem())
               .cacheForEnrRequests(enrs)
-              .forkIdManager(forkIdManager == null ? mock(ForkIdManager.class) : forkIdManager)
               .filterOnEnrForkId(filterOnForkId)
               .rlpxAgent(mock(RlpxAgent.class))
               .build());

ethereum/p2p/src/test/java/org/hyperledger/besu/ethereum/p2p/discovery/internal/PeerDiscoveryTableRefreshTest.java

@@ -24,7 +24,6 @@ import static org.mockito.Mockito.spy;
 import static org.mockito.Mockito.verify;
 
 import org.hyperledger.besu.cryptoservices.NodeKey;
-import org.hyperledger.besu.ethereum.forkid.ForkIdManager;
 import org.hyperledger.besu.ethereum.p2p.discovery.DiscoveryPeer;
 import org.hyperledger.besu.ethereum.p2p.discovery.PeerDiscoveryStatus;
 import org.hyperledger.besu.ethereum.p2p.discovery.PeerDiscoveryTestHelper;
@@ -72,7 +71,6 @@ public class PeerDiscoveryTableRefreshTest {
                 .tableRefreshIntervalMs(0)
                 .metricsSystem(new NoOpMetricsSystem())
                 .rlpxAgent(mock(RlpxAgent.class))
-                .forkIdManager(mock(ForkIdManager.class))
                 .build());
     controller.start();