## Why is this change needed?
Release shuttle 0.5.8 with validate message changes
## Merge Checklist
_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
<!-- start pr-codex -->
---
## PR-Codex overview
This PR updates versions in `hub-nodejs` and `shuttle` packages. Focus
is on upgrading `grpc-js` in `hub-nodejs` to 1.11 and bumping versions
for both packages.
### Detailed summary
- Updated `hub-nodejs` version to 0.11.22 with `grpc-js` upgrade
- Updated `shuttle` version to 0.5.8
- Removed message validation in `shuttle`
- Updated dependencies in `shuttle`
- Updated `hub-nodejs` dependency version in `shuttle`
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Why is this change needed?
Latest version grpc-js has some fixes that might help with the
RST_STREAM issues in hubs.
## Merge Checklist
_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
<!-- start pr-codex -->
---
## PR-Codex overview
This PR upgrades `grpc-js` to version 1.11, fixes stream handling in
`bufferedStreamWriter.ts`, and adds a `destroyStream` function in
`server.ts`.
### Detailed summary
- Upgraded `grpc-js` to version 1.11
- Improved stream handling in `bufferedStreamWriter.ts`
- Added `destroyStream` function in `server.ts`
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Why is this change needed?
This should significantly improve performance. This was originally
intended as additional protection when listening to untrusted hubs. But
given most users are listening to their own or trusted hub providers, no
need to enable this by default.
## Merge Checklist
_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
<!-- start pr-codex -->
---
## PR-Codex overview
This PR disables message validation by default in `shuttle`.
### Detailed summary
- Shuttle now does not validate messages by default
- Added environment variable to enable message validation
- Updated message processing logic in `hubEventProcessor.ts`
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Why is this change needed?
Release 1.14
## Merge Checklist
_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
<!-- start pr-codex -->
---
## PR-Codex overview
This PR updates the version of `@farcaster/hubble` to `1.14.0`,
introduces minor and patch changes, and updates the `FARCASTER_VERSION`
constant in `hubble.ts`.
### Detailed summary
- Updated `@farcaster/hubble` version to `1.14.0`
- Minor and patch changes in `CHANGELOG.md`
- Updated `FARCASTER_VERSION` constant in `hubble.ts`
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Why is this change needed?
The fname events are not coming through correctly for a few names, and
by consequence we're seeing a few old fname events being endlessly
replayed. The deletion of the fname record (when fid = 0 in proof)
completely removes the record rather than its association with a fid,
and by consequence replaying the older event re-adds the fname. This
resolves the hub side of the behavior by correctly holding on to the new
fname event, but deleting the association with a fid.
## Merge Checklist
_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [x] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
<!-- start pr-codex -->
---
## PR-Codex overview
The focus of this PR is to enhance the `hubble` app by fixing issues
related to username proof handling and syncing historical events.
### Detailed summary
- Fixed issue where deleted fname events allowed re-adding old fname
events
- Improved syncing historical events in `syncEngine.ts`
- Updated `mergeUserNameProof` functionality in tests
- Added `rsGetUserNameProof` import in `nameRegistryEvent.ts`
- Enhanced `UserDataStore` functionality in `user_data_store.rs`
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Why is this change needed?
We were checking for the message creator's fid instead of the target fid
and therefore rejecting valid messages older than the compact state
## Merge Checklist
_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
<!-- start pr-codex -->
---
## PR-Codex overview
This PR fixes an issue with target fid check for link messages older
than the compact state in the `store.rs` file and adds tests in
`linkStoreCompactState.test.ts`.
### Detailed summary
- Fixed incorrect target fid check for link messages in `store.rs`
- Added tests for merging link add messages in
`linkStoreCompactState.test.ts`
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Why is this change needed?
It was possible to trigger race conditions when calculating
earliest_ts_hash, this led to hubs have different values which meant
that stores would become out of sync. Migrate the cache to rust and
prevent race conditions.
## Merge Checklist
_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
<!-- start pr-codex -->
---
## PR-Codex overview
This PR migrates the storage cache to Rust and addresses a race
condition.
### Detailed summary
- Migrated storage cache to Rust for efficiency
- Fixed a race condition in the storage cache
- Added `PRIMARY_KEY_LENGTH` constant in `message.rs`
- Updated `storage_cache` exports in `store/mod.rs`
- Added functions to interact with storage cache in `utils.rs`
- Updated `lib.rs` to include `StorageCache`
- Updated `storeEventHandler.ts` to handle `StorageCache`
- Updated tests to use `StorageCache` in `storeEventHandler.test.ts`
- Implemented `StorageCache` in `store_event_handler.rs`
- Added Rust bindings for `StorageCache` in `rustfunctions.ts`
> The following files were skipped due to too many changes:
`apps/hubble/src/storage/stores/storageCache.ts`,
`apps/hubble/src/storage/stores/storageCache.test.ts`,
`apps/hubble/src/addon/src/store/storage_cache.rs`
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Why is this change needed?
The current processing of full state events results in an unbounded
number of deleted messages, making event consumption downstream
unpredictable and prone to getting stuck on singular large events.
Because merge events historically applied to deltas, and deltas were
generally small in effect size, this was a reasonable consideration, but
for full state (compact) messages, it is implicit on what is added
(everything in it) and what is not (everything not in it), making the
deleted messages something implicitly handled by downstream consumers.
## Merge Checklist
_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [x] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
<!-- start pr-codex -->
---
## PR-Codex overview
This PR focuses on fixing the behavior of full state events in the
`store.rs` file and test cases related to compact state messages in the
`linkStoreCompactState.test.ts`.
### Detailed summary
- Fixed full state events to exclude deleted messages in `store.rs`
- Updated test cases to reflect changes in handling deleted messages for
compact state messages in `linkStoreCompactState.test.ts`
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Why is this change needed?
- release 1.13.8
## Merge Checklist
_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
<!-- start pr-codex -->
---
## PR-Codex overview
The focus of this PR is to update version numbers and dependencies
across various packages.
### Detailed summary
- Updated version to `0.14.19` in `@farcaster/core`
- Updated version to `0.11.21` in `@farcaster/hub-nodejs`
- Updated version to `0.5.7` in `@farcaster/shuttle`
- Updated version to `1.13.8` in `@farcaster/hubble`
- Added event id functions in `@farcaster/core`
- Fixed conflict criteria issue in `@farcaster/shuttle`
- Upgraded libp2p to `0.44.0` in `@farcaster/hubble`
- Increased hub storage requirement to `200 GB` in `@farcaster/hubble`
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Why is this change needed?
Shuttle defines the messages table with two unique constraints, one for
the hash, another for the triple of hash, fid and type. Postgres does
not allow multiple ON CONFLICT criteria, and ambiguous conflicts are
essentially surfaced as uniqueness violation failures in spite of ON
CONFLICT criteria given. This change alters the merge to instead attempt
insert, doing nothing on any conflict, then resolving with an update if
necessary and the original conflict resolution criteria is satisfied.
Addresses Issue #2167
## Merge Checklist
_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [x] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
<!-- start pr-codex -->
---
## PR-Codex overview
This PR updates the conflict resolution logic in `MessageProcessor` to
fix ambiguity issues during upsert operations.
### Detailed summary
- Updated conflict resolution logic to use `doNothing()` method instead
of complex conditions
- Improved handling of conflict criteria for upsert operations
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Why is this change needed?
Upgrades libp2p one minor version up ([Release
Notes](https://github.com/libp2p/js-libp2p/releases/tag/v0.44.0) |
[Migration
Guide](https://github.com/libp2p/js-libp2p/blob/main/doc/migrations/v0.43-v0.44.md))
### Breaking Change in Autodial + Connection Manager Behavior
libp2p's ConnectionManager now has a default minConnection threshold of
50. If the number of active connections is < this number, it will
attempt to redial peers that are in the store.
Since we only disconnect peers and do not remove them from our store,
the naive upgrade causes us to reconnect immediately to peers
unexpectedly. We now set the threshold explicitly to 0 to prevent this.
We should consider also removing peers from the peer store in a future
upgrade.
## Merge Checklist
_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [ ] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
<!-- start pr-codex -->
---
## PR-Codex overview
This PR upgrades `libp2p` to version 0.44.0, updates
`@chainsafe/libp2p-gossipsub` to 6.2.0, and introduces
`@libp2p/interface-connection-gater` version 2.0.1.
### Detailed summary
- Upgraded `libp2p` to 0.44.0
- Updated `@chainsafe/libp2p-gossipsub` to 6.2.0
- Added `@libp2p/interface-connection-gater` version 2.0.1
> The following files were skipped due to too many changes: `yarn.lock`
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Why is this change needed?
Make timestamp to event id converstion function publicly accessible
## Merge Checklist
_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
<!-- start pr-codex -->
---
## PR-Codex overview
This PR adds new event ID functions and refactors timestamp handling in
the `core` and `hubble` packages.
### Detailed summary
- Added `extractEventTimestamp` and `makeEventId` functions in `core`
- Updated `console.ts` and `storeEventHandler.ts` to use the new
functions
- Refactored timestamp handling for event IDs
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Why is this change needed?
- update hub storage requirement to 200 GB
## Merge Checklist
_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
<!-- start pr-codex -->
---
## PR-Codex overview
This PR updates the storage requirement for the Hubble app to 200 GB and
adjusts related documentation to reflect the change.
### Detailed summary
- Updated Hubble app storage requirement to 200 GB
- Adjusted disk size in GCP tutorial to 200 GB
- Updated storage requirement in installation guide to 200 GB
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Why is this change needed?
- hub 1.13.7
- core 0.14.18
- hub-nodejs 0.11.20
## Merge Checklist
_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
<!-- start pr-codex -->
---
## PR-Codex overview
This PR increases version numbers for various packages and dependencies.
It also includes changes related to extending frame message URL max
length and assigning shards using a hashing algorithm.
### Detailed summary
- Increased version numbers for packages and dependencies
- Extended frame message URL max length
- Assigned shards using jump consistent hashing algorithm
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Why is this change needed?
Includes some performance improvements.
## Merge Checklist
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
<!-- start pr-codex -->
---
## PR-Codex overview
This PR updates the version of `@farcaster/shuttle` to `0.5.6` and
includes improvements related to stream trimming and recording size.
### Detailed summary
- Updated version to `0.5.6`
- Trim stream at regular interval
- Stop recording stream size
- Updated `@farcaster/hub-nodejs` to `0.11.20`
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Why is this change needed?
Performance profiling indicates that we're spending a non-trivial amount
of time (5-10% of wall clock time) trimming the stream. This doesn't
need to happen as frequently as it's currently happening, so switch to
an interval timer so that we're not slowing down general processing in
any meaningful way.
## Merge Checklist
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
<!-- start pr-codex -->
---
## PR-Codex overview
This PR optimizes the event stream processing in
`HubEventStreamConsumer` by adding a regular interval to trim old events
efficiently.
### Detailed summary
- Added `interval` property to schedule event processing at regular
intervals
- Implemented interval logic to clear old events and process stale
events
- Updated `stop` method to clear the interval
- Simplified event processing logic in the message loop
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Why is this change needed?
Looking at profiling data, this actually takes up a non-trivial amount
of wall clock time (10% of total time). Since the stream is
automatically trimmed, recording the size isn't particularly useful
except as perhaps an early warning on memory, but there are generally
other metrics one could use (number of events processed, etc.) to infer
that there is a slowdown in processing.
## Merge Checklist
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
<!-- start pr-codex -->
---
## PR-Codex overview
This PR stops recording stream size in the `eventStream.ts` file of the
`@farcaster/shuttle` package.
### Detailed summary
- Removed recording stream size in eventStream.ts
- Removed statsd gauge and timing metrics for stream size and time
calculations
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Why is this change needed?
To ensure this remains efficient at higher loads, perform before/after
hook processing in batches.
## Merge Checklist
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [ ] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
<!-- start pr-codex -->
---
## PR-Codex overview
This PR updates the version of `@farcaster/shuttle` to `0.5.5`. It
includes efficiency improvements in before/after hooks and updates to
`@farcaster/hub-nodejs`.
### Detailed summary
- Updated version to `0.5.5`
- Efficiency improvements in before/after hooks
- Updated dependency `@farcaster/hub-nodejs` to `0.11.20`
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Why is this change needed?
This will be much more efficient at high loads.
## Merge Checklist
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
<!-- start pr-codex -->
---
## PR-Codex overview
This PR optimizes event processing efficiency in `@farcaster/shuttle` by
switching before/after hooks to work with batches.
### Detailed summary
- Updated `PreProcessHandler` and `PostProcessHandler` signatures to
work with batches
- Refactored event processing logic to handle batches efficiently
- Improved event processing performance by processing events in batches
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Why is this change needed?
We want to allow users to add pre-post processing logic when processing
hub events using Shuttle.
## Merge Checklist
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [ ] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
<!-- start pr-codex -->
---
## PR-Codex overview
This PR updates the version of the `@farcaster/shuttle` package to
`0.5.4` and adds support for running before/after hooks when processing
events.
### Detailed summary
- Updated `@farcaster/shuttle` package version to `0.5.4`
- Added support for running before/after hooks when processing events
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Why is this change needed?
We want the ability to provide custom pre/post-processing logic when
processing events. Expose optional hooks that can be specified. This has
the added benefit that the `beforeProcess` hook can be used to skip
processing of an event.
## Merge Checklist
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [ ] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [x] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
<!-- start pr-codex -->
---
## PR-Codex overview
This PR enhances the `@farcaster/shuttle` package by adding support for
running before/after hooks when processing events.
### Detailed summary
- Added `beforeProcess` and `afterProcess` handlers to
`EventStreamHubSubscriber` and `HubEventStreamConsumer`
- Implemented handling of preprocess and postprocess actions for event
processing
- Introduced `EventStreamHubSubscriberOptions` and
`EventStreamConsumerOptions` interfaces
- Updated `HubEventStreamConsumer` to extend `TypedEmitter` for custom
event tracking
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Why is this change needed?
Describe why this issue should be fixed and link to any relevant design
docs, issues or other relevant items.
## Merge Checklist
_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
<!-- start pr-codex -->
---
## PR-Codex overview
This PR extends the maximum length of the frame message URL to 1024
characters for better validation accuracy.
### Detailed summary
- Increased maximum URL length validation from 256 to 1024 characters
- Updated test case with longer URL length for validation testing
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Why is this change needed?
While modulo-based arithmetic for assigning shards is simple and works
fine, it does not work well when the number of shards changes.
For example, when increasing the number of shards from 3 → 4 using
modulo-based shard assignment, ~83% of FIDs will be assigned to a
different shard. With consistent hashing, only ~25% will be assigned to
a different shard.
Jump consistent hashing is a particular approach to consistent hashing
that is stateless (requires no additional tracking of shard
assignments), which is ideal for hubs, since we don't want hubs to have
to manage per-client tracking of shards.
## Merge Checklist
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [x] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
<!-- start pr-codex -->
---
## PR-Codex overview
This PR focuses on implementing the jump consistent hashing algorithm
for shard assignment in the Hubble app.
### Detailed summary
- Implemented `jumpConsistentHash` algorithm for shard assignment based
on FID
- Updated shard assignment logic in `Server` class
- Updated tests in `eventService.test.ts` for shard-based event handling
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Why is this change needed?
Includeds some minor fixes that others are looking to use.
## Merge Checklist
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
<!-- start pr-codex -->
---
## PR-Codex overview
This PR updates the version of `@farcaster/shuttle` package to `0.5.2`
and includes enhancements for message processing and PostgreSQL schema
customization.
### Detailed summary
- Updated `@farcaster/shuttle` package version to `0.5.2`
- Added feature to handle removals for compact state explicitly and warn
on mismatching deletes
- Added feature to enable shuttle and its example-app to use a
PostgreSQL schema other than "public"
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Why is this change needed?
Follow up from review of #2124, we want to confirm first that moving the
responsibility of message processing marking messages as deleted for the
set difference from full-state-bearing messages is a no-op, and while
we're at it, might as well combine the potential thousands of SQL
updates into a single query.
## Merge Checklist
_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [x] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
<!-- start pr-codex -->
---
## PR-Codex overview
This PR introduces explicit handling of message removals for compact
state in the message processor, along with warnings for mismatching
deletes.
### Detailed summary
- Added handling for removals of compact state messages
- Warns on mismatching deletes
- Introduced new methods for processing compact state messages
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Why is this change needed?
This allows using shuttle to target e.g. multi-tenant dbs. Shouldn't be
a breaking change because new parameters on public functions are
optional.
## Merge Checklist
_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [ ] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [x] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
<!-- start pr-codex -->
---
## PR-Codex overview
This PR enables `shuttle` and its `example-app` to use a custom
PostgreSQL schema.
### Detailed summary
- Added support for custom PostgreSQL schema in shuttle and example-app
- Updated migrations to create functions with `CREATE OR REPLACE`
- Updated dependencies in `package.json`
- Added dotenv package for environment variables
- Updated README with installation instructions for node 21
- Modified database client creation to include schema parameter
- Updated table row count estimation function to include schema
parameter
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Why is this change needed?
Release 1.13.6
## Merge Checklist
_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
<!-- start pr-codex -->
---
## PR-Codex overview
This PR updates the version of `@farcaster/hubble` to `1.13.6` and
includes various patches and features related to message delays,
snapshot bandwidth, hub service agreements, peer maps, clock skew
validation, and Rust extension updates.
### Detailed summary
- Added stats for bundle message delays and stale contact info
- Increased message threshold to reduce snapshot bandwidth usage
- Fixed cli arguments order issue in docker-compose.yml
- Added hub service agreement: no rewards for running a hub
- Added unique peer map to sync engine for active peers
- Validated gossip message for clock skew
- Updated Rust extension to curve25519-dalek 4.1.3
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Why is this change needed?
⚠️ THERE WILL BE NO REWARDS FOR RUNNING A HUB
## Merge Checklist
_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
<!-- start pr-codex -->
---
## PR-Codex overview
This PR adds a hub service agreement feature to the Hubble script,
requiring hub operators to agree to terms before starting the hub.
### Detailed summary
- Added function `prompt_for_hub_operator_agreement` to prompt hub
operators to agree to terms
- Prompt displays a warning about no rewards for running the hub
- Operators must type "Yes" to proceed
- Implemented input validation for agreement
- Integration of agreement prompt before starting or upgrading the hub
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Why is this change needed?
Includes a fix allowing Redis Clusters to be used with the `RedisClient`
class.
## Merge Checklist
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
<!-- start pr-codex -->
---
## PR-Codex overview
This PR updates the version of `@farcaster/shuttle` package to `0.4.4`
and introduces a new feature allowing the Redis client to be a cluster
instance.
### Detailed summary
- Updated `@farcaster/shuttle` package version to `0.4.4`
- Added feature: Redis client can now be a cluster instance
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Why is this change needed?
We were artificially limiting the accepted type to a `Redis` instance,
when a `Cluster` instance would also work fine.
## Merge Checklist
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [ ] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
<!-- start pr-codex -->
---
## PR-Codex overview
This PR adds support for Redis cluster instances in the `shuttle`
package.
### Detailed summary
- Added support for Redis cluster instances in `shuttle/redis.ts`
- Updated `RedisClient` constructor to accept either `Redis` or
`Cluster` client
- Updated `create` method to handle Redis cluster instances
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Why is this change needed?
Releases a bug fix that is affecting Shuttle users.
## Merge Checklist
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [ ] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
<!-- start pr-codex -->
---
## PR-Codex overview
This PR updates the version of `@farcaster/shuttle` package to `0.4.3`
and includes a patch to handle "no such key" error when querying group
on first start.
### Detailed summary
- Updated package version to `0.4.3`
- Patched to handle "no such key" error when querying group on first
start
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Motivation
Fixes#2088#2117
There is likely an underlying issue with the commander configuration but
this is a quick change that gets this working for those having trouble
getting the container up and running.
## Change Summary
Reordered CLI arguments in docker-compose.yml.
## Merge Checklist
_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
- [x] All [commits have been
signed](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#22-signing-commits)
## Additional Context
If this is a relatively large or complex change, provide more details
here that will help reviewers
<!-- start pr-codex -->
---
## PR-Codex overview
This PR updates the `docker-compose.yml` file for the
`@farcaster/hubble` app to fix an issue with CLI arguments order
affecting the hub operator FID.
### Detailed summary
- Added `--hub-operator-fid ${HUB_OPERATOR_FID:-0}` to fix unset hub
operator FID in docker-compose.yml.
- Rearranged CLI arguments to ensure correct order and parameter
assignment.
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Motivation
- Current peer store keeps track of all hubs that successfully connected
- However, peers may connect and never be seen again due to churn or
change in Peer ID
- We add TTL Map of peers that only adds peers, never deletes, and
expires any peer that hasn't gossiped updates in 24 hours
## Change Summary
- add unique peer map to sync engine to represent current active peers
## Merge Checklist
_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
- [x] All [commits have been
signed](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#22-signing-commits)
## Additional Context
If this is a relatively large or complex change, provide more details
here that will help reviewers
<!-- start pr-codex -->
---
## PR-Codex overview
This PR introduces a unique peer map to track active peers in the sync
engine of the Hubble app. It also adds a high-performance TTLMap for
efficient entry expiration management.
### Detailed summary
- Added unique peer map for active peers in the sync engine
- Implemented TTLMap for automatic expiration of entries
- Updated handling of peer discovery events
- Enhanced peer contact management in the sync engine
> The following files were skipped due to too many changes:
`apps/hubble/src/utils/ttl_map.ts`
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Motivation
- Messages were found in gossip with very high timestamps that would be
very far in the future
## Change Summary
- Reject gossip messages that are more than 10 minutes in the future
## Merge Checklist
_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
- [x] All [commits have been
signed](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#22-signing-commits)
## Additional Context
If this is a relatively large or complex change, provide more details
here that will help reviewers
<!-- start pr-codex -->
---
## PR-Codex overview
This PR focuses on fixing clock skew validation for gossip messages in
the `@farcaster/hubble` module.
### Detailed summary
- Added validation for clock skew in gossip messages
- Introduced `ALLOWED_CLOCK_SKEW_SECONDS` constant
- Improved error handling for future timestamps in gossip messages
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Motivation
- Need better visibility into message delays and disambiguate between
bundle delay and message delay
- Ensure stale contacts aren't disseminating through the network
## Change Summary
- Add statsd metrics for earliest and latest message timestamp in bundle
and bundle creation timestamp. All timestamps are in farcaster time
- Add statsd metric for old contact info being gossiped
## Merge Checklist
_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
- [x] All [commits have been
signed](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#22-signing-commits)
## Additional Context
If this is a relatively large or complex change, provide more details
here that will help reviewers
<!-- start pr-codex -->
---
## PR-Codex overview
This PR focuses on enhancing message bundle submission in the Hubble
application by adding stats for message delays and contact info
staleness.
### Detailed summary
- Added stats for bundle message delays and stale contact info
- Updated `submitMessageBundle` method parameters in multiple files
- Updated `submitMessageBundle` method in `Hub` class to include
additional stats and parameters
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Motivation
- Reduce network bandwidth with gossip improvements
- Support experimental sync APIs (note: may be removed or significantly
altered in the future)
## Change Summary
- Update p2p gossip to reduce unnecessary contact updates
- Add experimental sync control APIs
## Merge Checklist
_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [x] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
- [x] All [commits have been
signed](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#22-signing-commits)
## Additional Context
If this is a relatively large or complex change, provide more details
here that will help reviewers
<!-- start pr-codex -->
---
## PR-Codex overview
This PR updates versions and adds endpoints for sync control in
`hub-web` and `hub-nodejs`, and includes various improvements and fixes
in the `hubble` app.
### Detailed summary
- Updated `hub-web` version to 0.8.12
- Updated `hub-nodejs` version to 0.11.19
- Added endpoints for sync control
- Improved gossip contact info frequency in `hubble` app
- Added experimental HTTP APIs for sync control in `hubble` app
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Motivation
Contact info gossip is causing undue burden on network
## Change Summary
fix: gossip contact info every 30 minutes instead of every minute, avoid
gossiping contact info on peer connect, skip contact info updates that
happen too frequently
## Merge Checklist
_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
- [x] All [commits have been
signed](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#22-signing-commits)
## Additional Context
If this is a relatively large or complex change, provide more details
here that will help reviewers
<!-- start pr-codex -->
---
## PR-Codex overview
This PR focuses on optimizing contact info gossiping in the Hubble app
by reducing frequency and skipping recent updates.
### Detailed summary
- Changed gossip contact info frequency to every 30 minutes
- Skipped gossiping contact info on peer connect
- Added a threshold to skip contact info updates happening too
frequently
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Motivation
We were seeing errors due to the number of message hashes in the `IN`
clause, resulting in a different prepared statement for each different
number of hashes.
## Change Summary
Switch to using the `= any([...])` form so that the same prepared
statement is always used.
## Merge Checklist
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
- [x] All [commits have been
signed](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#22-signing-commits)
<!-- start pr-codex -->
---
## PR-Codex overview
This PR focuses on optimizing the `shuttle` package by updating
dependencies and improving query efficiency.
### Detailed summary
- Updated `pg` dependency to `8.12.0`
- Switched `where` clause in `MessageReconciliation` for more efficiency
- Added `sql` import in `shuttle/db.ts`
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Motivation
Release performance fixes for gossip
## Change Summary
Describe the changes being made in 1-2 concise sentences.
## Merge Checklist
_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
- [x] All [commits have been
signed](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#22-signing-commits)
## Additional Context
If this is a relatively large or complex change, provide more details
here that will help reviewers
<!-- start pr-codex -->
---
## PR-Codex overview
This PR updates the version of `@farcaster/hubble` to `1.13.4` in the
package.json and includes patch changes related to fixing issues with
mainnet bootstrap peers and gossip worker.
### Detailed summary
- Updated version to `1.13.4`
- Removed `nemes` from mainnet bootstrap peers
- Fixed gossip worker spending too much time iterating peer store
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Motivation
Gossip metrics indicate the worker sometimes stalls for 5 mins before
receiving messages in a burst. Likely due to one or more bad peers. The
default socket timeout is 5 mins, set this to 30seconds to see if it
helps.
## Change Summary
Describe the changes being made in 1-2 concise sentences.
## Merge Checklist
_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
- [x] All [commits have been
signed](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#22-signing-commits)
## Additional Context
If this is a relatively large or complex change, provide more details
here that will help reviewers
<!-- start pr-codex -->
---
## PR-Codex overview
This PR updates the `hubble` package to use a stricter socket timeout
for gossip communication in the `gossipNodeWorker.ts` file.
### Detailed summary
- Updated the socket timeout for gossip communication to be stricter (30
seconds).
- Modified the `transports` configuration to include the new socket
timeout settings.
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
## Motivation
Parameters that contained invalid cast URLs with valid cast URL prefixes
were surfacing database error instead of not found error. The url itself
did not contain parent casts, but because it contained a valid prefix,
the database lookups were returning errors.
For example, a valid URL may be:
"https://warpcast.com/~/channel/japan777iine"
An invalid URL may be: "https://warpcast.com/~/chanel-makeup"
Since both URLs contain same prefix, _part_ of the invalid param gets
processed. That "prefix" then yields an error when entries aren't found
in the database. However, no entries found is expected, because the URL
is garbage.
Instead of surfacing the database error, we now correctly surface not
found error. This should fix#2067
- Certain queries to Hub API caused db errors to surface, e.g:
```
{
"code": 2,
"details": "db.internal_error/could not get message with key: [1, 129, 11, 134, 0, 1, 111, 110, 101, 6, 78, 209, 172, 203, 162, 11, 104, 228, 239, 164, 151, 212, 243, 3, 81, 109, 55, 199, 100, 32]",
"metadata": {
"errcode": [
null
]
}
}
```
## Change Summary
**Empty URL**
```
{
"errCode": "bad_request.invalid_param",
"presentable": false,
"name": "HubError",
"code": 3,
"details": "url < 1 byte",
"metadata": {
"errcode": [
"bad_request.invalid_param"
]
}
}
```
**Invalid URL**
```
{
"errCode": "not_found",
"presentable": false,
"name": "HubError",
"code": 2,
"details": "could not get message with key: [1, 129, 11, 134, 0, 1, 111, 110, 101, 6, 78, 209, 172, 203, 162, 11, 104, 228, 239, 164, 151, 212, 243, 3, 81, 109, 55, 199, 100, 32]",
"metadata": {
"errcode": [
"not_found"
]
}
}
```
## Merge Checklist
_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_
- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [ ] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
- [x] All [commits have been
signed](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#22-signing-commits)
## Additional Context
If this is a relatively large or complex change, provide more details
here that will help reviewers
<!-- start pr-codex -->
---
## PR-Codex overview
The focus of this PR is to fix the issue where the HTTP endpoint returns
"not found" instead of an internal database error.
### Detailed summary
- Fixed the issue in the `cast_store.rs` file related to generating
message primary key
- Updated error handling in `message.rs` and `store.rs` files
- Added a new function `not_found` in `store.rs`
- Modified error conversion logic in `server.ts` to handle Rust errors
> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`
<!-- end pr-codex -->
