* fix(eth): add stronger retry logic to eth events provider
* test has sporadic failure on CI, appears to be due to unresolved promise, perhaps this will fix
* alter retry provider to throw on retry limit
* more slight adjustments, makes concurrent tests stop freaking out
* add http status code to enforce retry
* slight changes to increase code coverage
* avoid clobbering globals
* await on emit
* finally hunted down RC, it's a timer, deep in subscriber
* ignoring mock utils
* exponential backoff + faster tests
We want to make it easier for developers to get started with hubs.
Add a separate optimized Docker image which we'll distribute for both
AMD and ARM architectures.
For now the publishing process is manual, but once hubs are fully open
(no peering allowlists) we'll be able to auto-publish with each hubble
release on NPM.
This dependency is only required for dev, not production. It's used by
Hubble when generating fake data, however, so move it to a production
dependency for Hubble.
With more Macs running ARM via Apple Silicon, the significantly slow
package installation was problematic. Fix by using our fork which
includes the prebuilt binary to reduce install time.
Fetch data in batches and execute jobs concurrently. This reduces the
time estimate from ~4 hours to under 1 hour on my machine.
We could probably make this even faster by implementing a
`getAllMessages` endpoint on the hubs.
Instead of requiring the user to run `yarn install` followed by `yarn
start`, we simply have them run `docker compose up` to start both the
app and Postgres together. This is much easier and reduces the need for
them to install anything else besides Docker. It also makes cleaning up
the example easier.
* feat: Suport sync status rpc call
* Add sync status hubble command
* Fix generated file
* Changeset
* Fix isSyncing check
* Rename to status and report db stats as well
* Fix error
We had a report of one user on modern MacBook with 16GB of memory who
was running into heap allocation issues. This was likely due to the
excessive number of promises we were creating (as well as buffering all
data for each call).
Change the logic to fetch in batches of 1K records at most. This slows
down the initial sync but should reduce the likelihood that someone will
hit a memory limit.
We also specify a custom limit in the `yarn start` command so that when
we test locally we are using the same limit as everyone else.
Provide a working end-to-end example of syncing data from hubs to a
Postgres database.
This should work with no additional dependencies besides what you
install with `yarn install` and Docker.
* feat: don't merge messages that would immediately be pruned
* Fix tests and minor cleanup/review comments
* Support all other stores
* Add changeset
* use prune iterator with keys available and expand cast store tests
* re-add prune iterator args
* Additional tests
* Add tests for other stores
---------
Co-authored-by: Sanjay Raveendran <sanjayprabhu@gmail.com>
* refactor: Rename sync events flag for clarity
* feat: Add sync statuts to HubInfo RPC call
* feat: Add sync stats to getInfo rpc call
* re-patch hub-web to use default export as before
* changeset
