This commit relocates the explorer code from the research directory to `bin/explorer` to include it as part of future releases.
### Summary of Updates:
#### Darkfi Project
- Updated `Cargo.toml` to include `bin/explorer/explorerd` as a workspace member
- Updated `Cargo.lock` to include the `explorerd` package
- Updated the `Makefile` to include `explorerd` in the build process
#### Explorer Daemon
- Renamed the project directory from `blockchain-explorer` to `explorer`
- Moved the explorer daemon source code to `bin/explorer/explorerd`
- Updated the cargo package name to `explorerd`
- Updated log statement targets from `blockchain-explorer::` to `explorerd::`
- Renamed the explorer configuration file to `explorerd_config.toml`
- Removed Halo2 patches as they are now included in the root package
- Changed default db_path to use explorerd instead of blockchain-explorer in the path
- Changed binary crate Arg structopt name from blockchain-explorer to explorerd
#### Explorer Site
- Moved the explorer site source code to `bin/explorer/site`
- Updated README.md to include new build instructions for explorerd
This commit integrates blockchain reorg functionality into the explorer’s block syncing and subscriptions, addressing inconsistencies and aligning the explorer state with a Darkfi node. Key improvements include reorg detection and handling in block synchronization and subscriptions, consolidated and refined logging for better observability, state reset methods to reliably handle reorg scenarios, and enhanced safeguards to ensure consistency before starting subscriptions.
Summary of Updates:
## Blocks Module
- Added `get_block_by_height` to retrieve blocks by height to support reorg handling
- Added `reset_to_height` function to reset blocks and related transactions to a specific height, enabling transactional resets. This function was introduced because the blockchain layer's equivalent function could not be reused due to the absence of `StateDiffs` on the explorer side.
- Changed the log level of `reset_blocks` to `debug` to streamline reorg-related logging
## Transaction Module
- Changed the log level of `reset_blocks` to `debug` to streamline reorg-related logging
## Binary Crate (main.rs)
- Added `reset_explorer_state` to reset the explorer state to a given height or to genesis. This ensures consistency across blocks, transactions, and metrics during resets or reorgs
- Updated logging to include transaction count during ExplorerDb initialization
## Rpc Blocks Module
- Refined variable and function names with "darkfid" for improved readability
### Block Sync Reorgs
- Implemented block sync reorg handling to detect and resolve inconsistencies between the Darkfid node and the explorer
- Introduced reorg detection logic based on mismatched block hashes and heights
- Added support for handling reorg scenarios, including genesis related cases
- Extended logging to report detected reorgs and re-alignments
- Updated the `reset` flag handling to purge and resync the explorer state from scratch when enabled
- Added `process_sync_blocks_reorg` to identify mismatched blocks and realign the explorer's blockchain state to the correct height
## Subscription Sync Reorgs
- Added reorg handling in the subscription process to detect reorgs through block height mismatches and reset the explorer state as needed
- Improved safeguards against mismatched last synced and last confirmed blocks before initiating the subscription process
These updates provide explorer nodes the ability to handle blockchain reorgs and ensure alignment with the canonical chain from Darkfi nodes.
This change introduces a reusable function to format error messages, add target-specific logging context, and encapsulate errors into a `DatabaseError`.
Changed the logging level of `darkfid_daemon_request` from `debug` to `trace` to facilitate debugging, particularly during the synchronization of large amounts of blocks.
This commit replaced the transaction link on the blocks page to use the `/tx/` path instead of `/transaction/`. This ensures the transaction URL aligns the updated transaction routing path.
This commit updates the explorer UI to show empty metrics when no metrics are available, replacing the previous behavior of displaying a "Gas consumption details are not available" message.
This commit enhances the Makefile by adding the bundle_contracts_src as a phony target and adding clean-up for bundled contract code.
Summary of Updates:
- Added `bundle_contracts_src` to `.PHONY`
- Updated the `clean` target to remove the `native_contracts_src` directory
This commit introduces functionality for displaying native contract source code in the DarkFi explorer application, enabling users to view available contracts from the home page, access associated source files, and examine native contract Rust code and zk proofs. This enhancement provides accessibility to native contract information. To support these updates, we have added new pages to the UI, updated styles for rendering source-related pages, and implemented new RPC routes to facilitate contract-related code retrieval.
Additionally, we have restructured the application into Flask blueprints to address the growing complexity of the explorer and adhere to best practices, as app.py was becoming large with diverging functionality. This proactive change prepares us for future expansions, including better organization of graph functionalities. The organization also mirrors the rest of the explorer codebase by maintaining separate pages for contracts, transactions, and blocks, as well as corresponding RPC and service rust modules.
UI Support for Native Contract Source Code
- We have added source code functionality to the UI, Flask blueprints, and RPC routes defined in rpc.py, along with updated styles to effectively render source-related pages.
Style Updates:
- Expanded the color palette for rendering source files.
- Enhanced styles for displaying contract IDs and names in the native contracts table.
- Implemented layout elements for proper display of source code lists and content.
- Added styles for source file list links, including hover effects.
RPC Routes:
- New routes have been added to facilitate requests to the DarkFi daemon for retrieving native contracts, source code paths, and contract source.
UI Enhancements:
- A native contract section has been introduced on the explorer home page to display native contracts.
- A contract source list page allows users to view and click on source code files associated with a native contract.
- A dedicated contract source page renders the source for a specific path associated with a given contract.
Introducing Flask Blueprints
To better manage the growing complexity of the application, we introduced Flask blueprints that organize routes according to their functionality. This restructuring not only enhances readability but also aligns with best practices for Flask applications, facilitating future development.
App.py Updates:
- Routes have been moved to their respective blueprints.
Added Blueprints:
- `__init__.py` file initializes the explorer blueprint package and exposes various blueprints used by the application.
- `block.py` module defines a blueprint for handling block-related functionality.
- `contract.py` module defines a blueprint for managing contract-related functionality.
- explore.py` module manages general functionality related to the home page and search features.
- `transaction.py` module handles transaction-related functionality.
Python Documentation:
- We documented the Python code and updated existing documentation to adhere to Python's best practices for using docstrings, enhancing usability and integration with documentation tools.
Testing:
- Manual testing has been conducted through the UI to verify the functionality of the new RPC methods, ensuring they correctly retrieve and display contract-related information.
This commit introduces a new RPC contract module to the Explorerd implementation. The module adds three RPC methods to enhance the functionality of the explorer service:
- `contracts.get_native_contracts`: Retrieves the native contracts deployed in the DarkFi network.
- `contracts.get_contract_source_code_paths`: Fetches the source code paths for a specified contract.
- `contracts.get_contract_source`: Retrieves the source code content for a given contract and source path.
These additions will allow users to retrieve contract-related information more through the RPC interface, enhancing the overall functionality of the explorer.
### Testing
Manual testing has been performed through the UI to verify the correct functionality of the new RPC methods, ensuring they properly retrieve and display contract-related information.
We opted for manual testing because the underlying service and data layer already have reliable unit tests, providing confidence in the core functionality. This approach sufficiently validates the functionality of the new RPC methods within the overall explorer context.
