hub-monorepo/apps/hubble

Hubble

Hubble is a Typescript implementation of a Farcaster Hub.

A Hub will download an entire copy of the network to your machine and keep it in sync. Messages can be created and uploaded to a Hub and they will propagate to all other Hubs. Running a Hub gives you a private instance that you can query as much as you like and helps decentralized the network.

If you already have the URL of a publicly hosted Hub, just follow instructions on interacting with Hubs.

💻 Getting Started Locally

Run a local Hubble instance to familiarize yourself with the basics.

1. Set up the Repository

1. Install dependencies
2. Navigate to /apps/hubble
3. Run yarn identity create to generate an identity key pair
4. Get an Ethereum Goerli node RPC URL from Alchemy or Infura

2. Connect to Testnet

Testnet is a sandboxed environment where you can read and write messages without affecting your production account. Dummy messages are broadcast every 10 seconds for testing. Minimum requirements are 4GB RAM and 5 GB of free space.

1. Run yarn start -e <eth-rpc-url> -b /dns/testnet1.farcaster.xyz/tcp/2282 -n 2

3. Verify Your Setup

Hubble will sync with the on-chain contracts, and issue thousands of messages like this:

0 | hubble | { level: 30, time: 1679703063660, pid: 3259, hostname: 'ip-10-0-0-85', component: 'EthEventsProvider', blockNumber: 8712752, msg: 'new block: 8712752 };

0 | hubble | { level: 30, time: 1679702496763, pid: 3259, hostname: 'ip-10-0-0-85', component: 'SyncEngine', total: 1, success: 1, msg: 'Merged messages', };

4. Switch to Mainnet

Mainnet is Farcaster's production environment apps use and writing a message here will make it show up in all applications. Minimum requirements are 8GB RAM and 20GB of disk space.

1. If you were on a different network, run yarn dbreset to clear the database.
2. Get your PeerId from the file /apps/hubble/.hub/<PEER_ID>_id.protobuf
3. Make a PR to add it to the allowed peers list.
4. Wait for the core team to deploy changes, usually within 24-48 hours.
5. If you were on a different network, run yarn dbreset to clear the database.
6. Run yarn start -e <node url> -b /dns/nemes.farcaster.xyz/tcp/2282 -n 1

☁️ Getting Started in the Cloud

Deploy Hubble to the cloud for long-term usage.

A server with at least 2 vCPUs, 8 GiB RAM and 20 GiB of SSD storage is recommended. The instance must have a publicly accessible IP address and ports 2282 and 2283 must be open.

1. Set up a Linux VM

Set up your Linux VM. Guides are available for some cloud providers:

• AWS EC2
• Digital Ocean

2. Set up the Environment

Next follow these instructions which should work on most Linux environments:

1. Install nvm.
2. Run . ~/.nvm/nvm.sh
3. Install node with nvm install 18.7.0
4. Install yarn and pm2 with npm install -g yarn
5. Clone the code with git clone https://github.com/farcasterxyz/hubble.git
6. Checkout the stable release cd hubble && git checkout @farcaster/hubble@latest
7. Build Hubble with yarn install && yarn build
8. Create an identity with cd apps/hubble/ && yarn identity create
9. Get an Ethereum Goerli node RPC URL from Alchemy or Infura

3. Connect to Testnet

Testnet is a sandboxed environment where you can read and write messages without affecting your production account. Dummy messages are broadcast every 10 seconds for testing. Minimum requirements are 4GB RAM and 5 GB of free space.

1. Install pm2 to mange the process with npm install -g pm2
2. Install pm2 logrotate to ensure the log files get trimmed pm2 install pm2-logrotate
3. Start Hubble with pm2 start "yarn start -e <node url> -b /dns/testnet1.farcaster.xyz/tcp/2282 -n 2" --name hubble --exp-backoff-restart-delay=100
4. Check the logs with pm2 logs

4. Verify Your Setup

Hubble will sync with the on-chain contracts, and issue thousands of messages like this:

0 | hubble | { level: 30, time: 1679703063660, pid: 3259, hostname: 'ip-10-0-0-85', component: 'EthEventsProvider', blockNumber: 8712752, msg: 'new block: 8712752 };

0 | hubble | { level: 30, time: 1679702496763, pid: 3259, hostname: 'ip-10-0-0-85', component: 'SyncEngine', total: 1, success: 1, msg: 'Merged messages', };

5. Switch to Mainnet

Mainnet is Farcaster's production environment apps use and writing a message here will make it show up in all applications. Minimum requirements are 8GB RAM and 20GB of disk space.

1. Stop running hubs with pm2 stop all
2. Navigate to apps/hubble
3. Clean the db with yarn dbreset, if you are moving over from testnet.
4. Get your PeerId from the file .hub/<PEER_ID>_id.protobuf
5. Make a PR to add it to the allowed peers list.
6. Wait for the core team to deploy changes, usually within 24-48 hours.
7. Run pm2 start "yarn start -e <node url> -b /dns/nemes.farcaster.xyz/tcp/2282 -n 1" --name hubble --exp-backoff-restart-delay=100

🔨 Interacting with Hubble

Fetching data from Hubble requires communicating with its gRPC API's and reading the Protobuf messages. There are SDK's in a few languages to make this easier:

• Javascript / Typescript
• Python

⚙️ Advanced Setup

Upgrading your Hubs

1. Stop running hubs with pm2 stop hubble (or relevant command)
2. Navigate to the repository root and run git fetch && git checkout @farcaster/hubble@latest (or your preferred version)
3. Run yarn install && yarn build
4. Navigate to apps/hubble
5. Start your hub with pm2 start hubble (or relevant command)

Authentication

Your hub is not authenticated meaning that anyone can connect to it and request messages. You can add authentication by setting up SSL and adding a username and password to your hubs (instructions to follow).

Peering

Your node will try to discover its public IP by connecting to a third party service, so that it can broadcast its contact details to peers. If you have a custom config or prefer not to use a service, specify your IP with the --announce-ip flag when calling yarn start.

Architecture

A Hub is a single-process daemon that receives data from clients, other hubs and farcaster contracts. It has three main components:

• P2P Engine - establishes a gossipsub network to exchange messages with hubs.
• Sync Engine - handles edge cases when gossip fails to deliver messages.
• Storage Engine - checks message validity, persists them to disk and emits events.

flowchart LR
subgraph Hubble

    subgraph Networking
        P2PEngine(P2PEngine)
        SyncEngine(SyncEngine)
    end

    subgraph Storage
        StorageEngine(Storage Engine)
    end

end
Node[ETH Node] -.-> |RPC| Hubble
Clients[FC Clients] & Clients & Clients -.-> |RPC| Hubble
Hubble <-.-> |RPC + Gossip|Hubs[Hubs] & Hubs & Hubs
Storage <--> Networking

Storage Engine

Messages received by Hubble are forwarded to the Storage engine which forwards them to the appropriate CRDT Set. Once validated by the CRDT Set, messages are persisted to RocksDB and events are emitted to listeners.

flowchart LR
subgraph Hubble
    subgraph Storage
        StorageEngine(Storage Engine) --> Sets
        Sets(CRDT Sets) <--> DB[(Rocks<br/>DB)]
    end
end

CRDT sets are implemented to meet the specification in the Farcaster protocol. The engine also tracks state of the Farcaster contracts, which are necessary for validating the Signer CRDT Set.

P2P Engine

Hubble connects to other peers over a GossipSub network established using LibP2P. Messages merged into the Storage Engine are immediately gossiped to all of is peers.

flowchart LR
subgraph Hubble

    subgraph Networking
        P2PEngine(P2PEngine) --> LibP2P(LibP2P)
    end
end

Hubble will only peer with trusted peers and employs a simple network topology during beta. It peers only with known instances which must be configured at startup. In later releases, the network topology will be modified to operate closer to a trustless mesh.

Sync Engine

Hubble periodically performs a diff sync with other peers to discover messages that may have been dropped during gossip. This is performed using gRPC APIs exposed by each Hub instance.

flowchart LR
subgraph Hubble
    subgraph Networking
        SyncEngine(SyncEngine) --> RPC(RPC Client)
    end
end