mirror of
https://github.com/farcasterxyz/hub-monorepo.git
synced 2026-02-12 05:55:16 -05:00
32e660a181
We had a report of one user on modern MacBook with 16GB of memory who was running into heap allocation issues. This was likely due to the excessive number of promises we were creating (as well as buffering all data for each call). Change the logic to fetch in batches of 1K records at most. This slows down the initial sync but should reduce the likelihood that someone will hit a memory limit. We also specify a custom limit in the `yarn start` command so that when we test locally we are using the same limit as everyone else.
Replicate hub data into Postgres
This example shows you how you can quickly start ingesting data from a Farcaster hub into a traditional database like Postgres.
Run on StackBlitz
Run locally
- Clone the repo locally
- Navigate to this folder with
cd packages/hub-nodejs/examples/replicate-data-postgres - Run
yarn installto install dependencies - Run
docker compose up -dto start a Postgres instance (install Docker if you do not yet have it) - Run
yarn start
To wipe your local data, run docker compose down -v from this directory.