- replace log crate with tracing,
- replace simple-log crate with tracing-subscriber and tracing-appender
- add nu-ansi-term crate as a dependency which is itself tracing-subscriber's dependency,
inorder to write colored outputs to the terminal since we override formatting provided
by tracing-subscriber
This commit updates the explorer web app to run on a Gunicorn-based WSGI server for testnet and mainnet rather than the built-in dev server. In doing so, it provides for improved scalability, reliability, integration with Nginx, and flexible run configuration.
We’ve introduced a dedicated configuration file (gunicorn_config.py) allowing control over the number of worker processes and threads. By adjusting these values, we can handle more simultaneous requests, fine-tune performance based on CPU resources, and tweak other operational parameters to suit deployment needs.
Update Summary:
- Added gunicorn_config.py to configure bind address, workers, threads, pidfile, and logs
- Updated the testnet and mainnet ports to run on 8000 instead of the 5000 dev port
- Created wsgi.py to load and expose the Flask app via `create_app()`
- Updated requirements.txt to include Gunicorn dependency
- Modified makefile start-% target to start with Gunicorn for testnet/mainnet instead of the development server
- Adjusted stop task to stop Gunicorn server when stopping the explorer
- We updated the requirements.txt install to run quietly
- Updated Makefile to export LOG_HOME so it can be accessed by gunicorn_config.py
This update introduces code to ensure blocks with PoW rewards containing coins already applied to the contract state prior to a reorg are synced. Additionally, it brings awareness to the need for rolling back the contract state to a specific height during reorgs.
This fix adjusts the range query in `reset_to_height` to start from `reset_height + 1`, ensuring transactions from `reset_height` are excluded when deleting reset block transactions.
Summary of changes:
- Updated the `get_by_range` query to exclude `reset_height` transactions
- Enhanced debug statements to validate transaction deletions and associated block heights during reorgs
This commit adds support for the `$(VERBOSE)` variable in the `start-%` targets, allowing changing verbosity levels when starting the explorer node environment. The help message has also been updated to include more details.
This commit enhances subscription initialization to handle scenarios where the explorer database is out of sync with darkfid.
Summary of changes:
- Added "Blockchain not fully synced" error check during block subscription initialization
- Implemented a retry mechanism to resync blocks and reattempt the subscription process
This commit modifies the `wait_for_darkfid_startup` target in the Makefile to search for a later log message during darkfid startup. Results from testnet testing revealed that additional synchronization occurs before darkfid starts, necessitating a longer wait time is needed before starting explorerd.
This commit updates the `calculate_tx_gas_data` function to apply state changes during contract execution, enabling contract data availability for subsequent contract calls during block sync. Additionally, `put_block` was modified to apply PoW reward transactions to the WASM runtime, ensuring their effects are visible to transaction runtimes used for fee calculations where applicable.
Minor cleanup was performed.
This commit expands the test suite by adding RPC tests that cover invalid/missing parameters, unsupported data types, unparsable inputs, and invalid numeric ranges. It also includes a test case to ensure `ping_darkfid` properly handles error when invoked under a disconnected darkfid endpoint.
New test modules include:
- **rpc/mod/tests**: Tests to ensure that `ping_darkfid` correctly handles non-empty parameters and scenarios where it is invoked with a disconnected darkfid endpoint
- **rpc/blocks/tests**: Tests verifying that invalid/missing parameters for fetching blocks produce the expected error responses
- **rpc/transaction/tests**: Tests confirming that incorrect/missing parameters for retrieving transactions are properly handled
- **rpc/contracts/tests**: Tests ensuring invalid/missing parameter handling for retrieving native contracts data
- **rpc/statistics/tests**: Tests confirming that invalid/missing parameters for retrieving statistics result in correct error handling
Other updates:
- Set the default test log level to 'Off' in test_mod.rs
- Removed the `TEST_DATA_DIR` constant until the test cases are updated to run on a copy of the test `explorerdb`
- Removed the `test_data/explorerd_0` folder until the test cases are updated to use a copy of the test `explorerdb`
Running Tests:
cargo test rpc
This commit enhances `test_utils` module by introducing a shared `Explorerd` instance and auxiliary functions for verifying invalid JSON-RPC parameter handling. By initializing the `Explorerd` once, tests can run asynchronously. The auxiliary functions cover invalid or missing parameters, ensuring that the Explorerd API correctly handles and reports invalid inputs.
Updates include:
- Defined a global `Explorerd` instance that is shared across tests, improving efficiency
- Updated the test setup so the `Explorerd` instance is initialized on the first invocation and then shared among subsequent calls
- Added `validate_invalid_rpc_parameter` to test JSON-RPC methods for invalid parameter handling, verifying correct error codes and messages
- Added `validate_empty_rpc_parameters`, which checks whether methods that expect no parameters properly reject non-empty parameters
- Added helper functions (`validate_invalid_rpc_contract_id`, `validate_invalid_rpc_header_hash`, `validate_invalid_rpc_tx_hash`, and `validate_invalid_rpc_hash_parameter`) to test invalid hash parameter handling
- Temporarily added `#[allow(dead_code)]` for `test_util` until the test code that utilizes the new utility functions is checked in
Adds an initial test data into a `explorerd` sled database with preloaded native smart contracts to the `test_utils` module. This test data will be updated over time, but currently optimizes testing by eliminating the need to load smart contracts for every test run.
Updates the service layer to map failed `header_hash` parsing to `ExplorerdError::InvalidHeaderHash`. This improves error messages by including the invalid hash value, making them more actionable.
Change the index route code in explore.py to call `get_last_n_blocks` with an integer instead of a string to accommodate the updated RPC layer, which now expects numeric types rather than strings for numbers.
Refactored the RPC layer to unify parameter extraction/validation, error handling, result construction, and request failure logging. These updates simplify the rpc layer, reduce individual method handler code by approximately 20%, enhance error context, and aim to improve the overall user and developer experience.
### Key Improvements:
Streamlined Error Handling:
- Unified error handling within the `handle_request` function enables RPC method handlers to use the `?` operator to return errors for unified processing. This streamlines error handling and provides consistent translation of `JsonError` responses.
Parameter Parsing and Validation:
- Incorporated use of new `jsonrpc` utilities for streamlined parameter extraction and validation. This eliminates parameter boilerplate extract and validation logic across handlers, reduces the risk of inconsistencies, and establishes a unified parameter processing approach for all RPC methods.
Simplified `JsonResult` Construction:
- Replaced direct `JsonResult` construction in handlers with a unified approach. Handlers now return a `JsonValue` wrapped in a `darkfi::Result`, simplifying implementation by removing the need to construct `JsonResponse`/`JsonError` within individual RPC method handlers.
Enhanced Error Context:
- Added detailed error information, including parameter names, indices, and values that caused validation failures. These enhancements make it easier to pinpoint the root causes of errors, benefiting both developers and API users.
Unified Logging:
- Added consolidated logging for RPC request failures in the `handle_request` function. Errors are logged with details like RPC method name, parameters, and the JSON-RPC error returned back to the caller, ensuring consistent and informative reporting.
### Highlight:
Cleaner and More Consistent Code:
By consolidating error handling, result construction, and parameter processing with the new `jsonrpc` utilities, RPC method handlers are now more concise. These changes reduce the code required for implementation, allowing developers to focus on core RPC logic and service integration.
### Module Update Details:
mod.rs:
- Refactored the `handle_request` method to streamline JSON-RPC request handling
- Unified error processing, logging, and result transformation using `JsonResult`
- Reorganized JSON-RPC methods in the match block into logical order (blocks, transactions, statistics, contracts, then miscellaneous)
- Added a utility function for failure logging, capturing method names, parameters, and error details
blocks.rs: contracts.rs, statistics.rs, transactions.rs:
- Updated individual modules to align with the refactored `handle_request` logic
- Updated numeric parameters to be processed as `JsonValue::Number` instead of strings
