reth/IMPLEMENTATION_PLAN.md

Issue #20482: Add Configuration Validation for RocksDB Flags at Startup

Problem

Reth uses StorageSettings to control where data is stored (MDBX vs RocksDB vs static files). These flags are:

• receipts_in_static_files
• transaction_senders_in_static_files
• storages_history_in_rocksdb
• transaction_hash_numbers_in_rocksdb
• account_history_in_rocksdb

The bug: Users can change these flags via CLI after initial sync, but the node silently loads the persisted settings from the database, ignoring CLI flags. This creates confusion:

1. User syncs with --static-files.receipts=false (receipts in MDBX)
2. User restarts with --static-files.receipts=true (expecting receipts in static files)
3. Node loads persisted settings from DB → still uses MDBX
4. User thinks receipts are in static files, but they're not

Worse scenario: If CLI flags were honored on restart, old data would be in one location, new data in another → data corruption.

Design Philosophy

Following John Ousterhout's A Philosophy of Software Design, we chose to "Define Errors Out of Existence" rather than adding complex error handling in the storage layer.

Why Not Add Validation in ProviderFactory::new()?

The original approach was to:

1. Add expected_settings parameter to ProviderFactory::new()
2. Add StorageSettingsMismatch error variant
3. Return error if settings don't match

Problems with this approach:

• Adds complexity to a core type (ProviderFactory)
• Forces all callers to handle a new error type
• The "error" has no meaningful recovery - it's really a configuration mistake

Better Approach: Validate at CLI Layer

Instead, we:

1. Keep ProviderFactory::new() simple - it just loads settings from DB
2. Validate settings mismatch in CLI code where user intent is clear
3. Return a clear, actionable error message using eyre::eyre!

Solution Architecture

┌─────────────────────────────────────────────────────────────────────────────┐
│                              CLI LAYER                                       │
│                                                                              │
│   1. Create ProviderFactory (loads settings from DB)                         │
│   2. Compare CLI settings vs stored settings                                 │
│   3. If mismatch → return eyre::eyre! with clear message                     │
│                                                                              │
└─────────────────────────────────────────────────────────────────────────────┘
                                    │
                                    ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│                           STORAGE LAYER                                      │
│                                                                              │
│   ProviderFactory::new() - UNCHANGED                                         │
│   • Loads settings from DB                                                   │
│   • Falls back to legacy defaults if none exist                              │
│   • No validation, no new parameters                                         │
│                                                                              │
└─────────────────────────────────────────────────────────────────────────────┘

Implementation

Part 1: CLI Validation (crates/cli/commands/src/common.rs)

In create_provider_factory(), after creating the factory:

// Error if CLI settings differ from stored settings
let cli_settings = self.static_files.to_settings();
if let Some(stored_settings) = factory.storage_settings()? {
    if stored_settings != cli_settings {
        return Err(eyre::eyre!(
            "Storage settings mismatch!\n\n\
             Stored in DB: {stored_settings:?}\n\
             From CLI:     {cli_settings:?}\n\n\
             Storage flags cannot be changed after genesis.\n\
             Either remove the conflicting CLI flags, or delete the database and re-sync."
        ));
    }
}

Part 2: Node Builder Validation (crates/node/builder/src/launch/common.rs)

Same pattern in create_provider_factory():

// Error if CLI settings differ from stored settings
let cli_settings = self.node_config().static_files.to_settings();
if let Some(stored_settings) = factory.storage_settings()? {
    if stored_settings != cli_settings {
        return Err(eyre::eyre!(
            "Storage settings mismatch!\n\n\
             Stored in DB: {stored_settings:?}\n\
             From CLI:     {cli_settings:?}\n\n\
             Storage flags cannot be changed after genesis.\n\
             Either remove the conflicting CLI flags, or delete the database and re-sync."
        ));
    }
}

Files Modified

File	Change
crates/cli/commands/src/common.rs	Add settings mismatch check
crates/node/builder/src/launch/common.rs	Add settings mismatch check

Files NOT Modified (Key Insight!)

File	Why Not
crates/storage/errors/src/provider.rs	No new error type needed
crates/storage/provider/src/providers/database/mod.rs	ProviderFactory::new() stays simple
crates/storage/provider/src/providers/database/builder.rs	No new builder methods

Benefits of This Approach

1. Simpler core types - ProviderFactory doesn't need to know about CLI settings
2. No new error variants - Uses existing eyre for CLI errors
3. Clear error messages - Actionable guidance at the user-facing layer
4. Follows single responsibility - Storage layer stores, CLI layer validates user input

Testing

The validation is straightforward and tested by:

1. Starting a node with settings A
2. Restarting with settings B
3. Observing the clear error message

No unit tests needed in the storage layer since no logic was added there.

PR

• Title: feat(cli): validate storage settings match persisted settings at startup
• Closes #20482