mirror of
https://github.com/farcasterxyz/hub-monorepo.git
synced 2026-01-14 15:47:57 -05:00
@farcaster/shuttle@0.6.6
551 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Cassandra Heart
|
d0de3a91ff |
Release shuttle v0.6.6 (#2360)
## Why is this change needed? Releases v0.6.6 of shuttle ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR updates the version of the `@farcaster/shuttle` package from `0.6.5` to `0.6.6`, adds a new entry to the `CHANGELOG.md` for patch changes, and includes a fix for handling streaming hangups when the server unexpectedly dies. ### Detailed summary - Updated `version` in `packages/shuttle/package.json` from `0.6.5` to `0.6.6`. - Added a new section in `CHANGELOG.md` for version `0.6.6`. - Documented a fix for gracefully handling streaming hangups when the server dies unexpectedly. > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Cassandra Heart
|
386059ac25 |
fix: gracefully handle streaming hangups when the server dies unexpected (#2358)
## Why is this change needed? Fixes #2346. Under certain conditions, like a hub failure or odd connectivity quirks, the fetch can hang indefinitely, leading to a need for consumers to restart if they aren't timeout driven. This avoids the issue by setting a sensible timeout and returning failure in the event it can't complete the request. ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR focuses on improving the handling of server streaming timeouts and errors in the `shuttle` package, ensuring graceful termination of unresponsive connections and adding corresponding tests. ### Detailed summary - Added timeout handling to prevent hanging connections in `packages/shuttle/src/shuttle/messageReconciliation.ts`. - Updated promise resolution logic to handle server timeout errors. - Implemented a test case in `packages/shuttle/src/shuttle.integration.test.ts` for unresponsive server requests. > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Sanjay
|
3b9f2e806d |
chore: Release 1.15.5 (#2352)
## Why is this change needed? Release 1.15.5 ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR focuses on updating package versions and changelogs across multiple packages within the Farcaster project, including reintroducing a dependency and fixing various issues in the `shuttle` and `hubble` packages. ### Detailed summary - Deleted multiple `.changeset` markdown files. - Updated `@farcaster/core` version from `0.15.3` to `0.15.4`. - Updated `@farcaster/hub-nodejs` version from `0.12.3` to `0.12.4`. - Updated `@farcaster/shuttle` version from `0.6.4` to `0.6.5`. - Updated `@farcaster/hubble` version from `1.15.4` to `1.15.5`. - Added notable changes to changelogs for `core`, `hub-nodejs`, `shuttle`, and `hubble` packages, including bug fixes and new features. > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Cassandra Heart
|
8dfe98437f | fix: restore start/stop time for reconcileMessagesForFid (#2351) | ||
|
Aditi Srinivasan
|
48be01a654 |
feat: export makeMessage fn (#2283)
## Why is this change needed? This function will be useful in the backend for a script that generates a message from another message and publishes with a new signer. ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [ ] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR focuses on making the `makeMessage` function in `builders.ts` file exportable. ### Detailed summary - Exported the `makeMessage` function in `builders.ts` file. > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Patrice ORENES-LERMA
|
a8736e1de7 |
fix(core): add faker back as a dependency as it is used (#2201)
## Why is this change needed? Faker has been removed from the dependencies on the farcaster/core library by mistake, as it is used in factory.ts . Re-introducing it will prevent if from being bundled. Fixes #2031 ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [x] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR re-introduces the `faker` dependency in the `@farcaster/core` package. ### Detailed summary - Re-added `@faker-js/faker` dependency in `@farcaster/core` package.json - Removed `@faker-js/faker` from devDependencies > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Ty Book
|
6408496826 |
feat(shuttle): Support deleting messages on DB that are missing on hub (#2341)
There was existing support for handling messages that were present on hub and missing on DB, but not the opposite. From talking with Sanjay, it should be correct/safe to delete these messages from the DB. ## Why is this change needed? Shuttle only reconciled in one direction, even though everything appears 95% written to support both directions. This completes that. I know from adding logging to our Shuttle-powered app that that are a significant number of messages present in our DB that are missing in the hub. ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [ ] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR introduces functionality to delete messages from the database that are not present on the hub, enhancing the message handling logic in the `shuttle` package. ### Detailed summary - Updated `handleMissingMessage` method in `HubEventProcessor` to accept an optional `missingInHub` parameter. - Added logic to delete messages if `missingInHub` is true; otherwise, it merges them. - Adjusted calls to `handleMissingMessage` in `app.ts` to pass the new parameter. > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
dependabot[bot]
|
a19bab5b8c |
chore(deps): bump micromatch from 4.0.7 to 4.0.8 in /packages/hub-nodejs/examples/hello-world (#2279)
Bumps [micromatch](https://github.com/micromatch/micromatch) from 4.0.7 to 4.0.8. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/micromatch/micromatch/blob/4.0.8/CHANGELOG.md">micromatch's changelog</a>.</em></p> <blockquote> <h2>[4.0.8] - 2024-08-22</h2> <ul> <li>backported CVE-2024-4067 fix (from v4.0.6) over to 4.x branch</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
dependabot[bot]
|
e5f50885f9 |
chore(deps): bump elliptic from 6.5.5 to 6.5.7 in /packages/hub-nodejs/examples/hello-world (#2264)
Bumps [elliptic](https://github.com/indutny/elliptic) from 6.5.5 to 6.5.7. <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
dependabot[bot]
|
15fe47bef4 |
chore(deps): bump axios from 1.6.0 to 1.7.4 in /packages/hub-web/examples/events (#2322)
Bumps [axios](https://github.com/axios/axios) from 1.6.0 to 1.7.4. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/axios/axios/releases">axios's releases</a>.</em></p> <blockquote> <h2>Release v1.7.4</h2> <h2>Release notes:</h2> <h3>Bug Fixes</h3> <ul> <li><strong>sec:</strong> CVE-2024-39338 (<a href="https://redirect.github.com/axios/axios/issues/6539">#6539</a>) (<a href="https://redirect.github.com/axios/axios/issues/6543">#6543</a>) (<a href=" |
||
|
dependabot[bot]
|
f7c1a97547 |
chore(deps): bump axios from 1.6.0 to 1.7.4 in /packages/hub-nodejs/examples/hello-world (#2254)
Bumps [axios](https://github.com/axios/axios) from 1.6.0 to 1.7.4. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/axios/axios/releases">axios's releases</a>.</em></p> <blockquote> <h2>Release v1.7.4</h2> <h2>Release notes:</h2> <h3>Bug Fixes</h3> <ul> <li><strong>sec:</strong> CVE-2024-39338 (<a href="https://redirect.github.com/axios/axios/issues/6539">#6539</a>) (<a href="https://redirect.github.com/axios/axios/issues/6543">#6543</a>) (<a href=" |
||
|
dependabot[bot]
|
4909c176a7 |
chore(deps): bump axios from 1.6.0 to 1.7.4 in /packages/hub-web/examples/submit-message (#2253)
Bumps [axios](https://github.com/axios/axios) from 1.6.0 to 1.7.4. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/axios/axios/releases">axios's releases</a>.</em></p> <blockquote> <h2>Release v1.7.4</h2> <h2>Release notes:</h2> <h3>Bug Fixes</h3> <ul> <li><strong>sec:</strong> CVE-2024-39338 (<a href="https://redirect.github.com/axios/axios/issues/6539">#6539</a>) (<a href="https://redirect.github.com/axios/axios/issues/6543">#6543</a>) (<a href=" |
||
|
Davion Selever
|
21bb96a54c |
Removed the hyphen in onchain and offchain to maintain a consistent style (#2315)
## Why is this change needed? "The community has generally settled on onchain and offchain as individual words, rather than hyphenated." - @sds https://github.com/farcasterxyz/docs/pull/296#discussion_r1744299916 Removed the hyphen in onchain and offchain to maintain a consistent style <!-- start pr-codex --> --- ## PR-Codex overview This PR standardizes the term "on-chain" to "onchain" across various files and documentation. ### Detailed summary - Replaced "on-chain" with "onchain" for consistency - Updated terminology in code comments, classes, and documentation > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Cassandra Heart
|
7fe117e4ec |
chore: release shuttle v0.6.4 (#2321)
## Why is this change needed? Releases shuttle v.0.6.4 ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR updates the version of the `@farcaster/shuttle` package to `0.6.4` and aligns hub-nodejs dependency interfaces. ### Detailed summary - Updated `@farcaster/shuttle` package version to `0.6.4` - Aligned hub-nodejs dependency interfaces > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Cassandra Heart
|
2de0697ef5 |
chore: upgrade hub-nodejs dependency so interfaces align (#2320)
## Why is this change needed? Describe why this issue should be fixed and link to any relevant design docs, issues or other relevant items. ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR updates the `@farcaster/hub-nodejs` dependency in the `shuttle` package to version `0.12.3` to align with interfaces. ### Detailed summary - Updated `@farcaster/hub-nodejs` dependency in `shuttle` package to `0.12.3` - Ensured alignment with interfaces by upgrading hub-nodejs dependency > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Cassandra Heart
|
e2ce82b4f2 |
chore: bump version (#2316)
## Why is this change needed? bumps version of hubble and libraries ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [x] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR updates version numbers and dependencies across different packages. ### Detailed summary - Bumped version to `0.15.3` in `@farcaster/core` - Bumped version to `0.9.3` in `@farcaster/hub-web` - Bumped version to `0.12.3` in `@farcaster/hub-nodejs` - Bumped version to `1.15.2` in `@farcaster/hubble` - Updated dependencies and added new features in various packages > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Cassandra Heart
|
e5a8611498 |
feat: support bulk message RPC (#2313)
## Why is this change needed? Reconciliation of messages historically from off-hub sources is achieved via `submitMessage`, which incurs a performance penalty when many of these calls are made in rapid succession. This change introduces a new `submitBulkMessages` RPC which allows many of the to-be reconciled messages to be submitted at once and handle via the more efficient rust `mergeMany` underlying call. ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [x] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR adds a new gRPC method `SubmitBulkMessages` for submitting multiple messages at once and updates related proto files and service implementations. ### Detailed summary - Added `SubmitBulkMessages` gRPC method - Defined new message types: `SubmitBulkMessagesRequest`, `MessageError`, `BulkMessageResponse`, `SubmitBulkMessagesResponse` - Updated service implementations and proto files - Implemented server-side logic for `SubmitBulkMessages` - Added encoding and decoding functions for new message types > The following files were skipped due to too many changes: `packages/hub-web/src/generated/request_response.ts`, `packages/hub-nodejs/src/generated/request_response.ts`, `packages/core/src/protobufs/generated/request_response.ts` > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Aditi Srinivasan
|
f084daa184 |
feat: request missing on chain events on error (#2298)
## Why is this change needed? Our hubs regularly fail to get into sync because some of them are missing onchain events. This feature will have us request missing onchain events if there are errors related to missing on chain events when messages are submitted. Deployed to hoyt for testing: [Logs](https://app.datadoghq.com/logs?query=%22Attempting%20to%20retryEventsForFid%22%20OR%20%22Finished%20retryEventsForFid%22%20OR%20%22cacheOnChainEvent%22%20&agg_m=count&agg_m_source=base&agg_q=%40fid&agg_q_source=base&agg_t=count&cols=host%2Cservice&fromUser=true&messageDisplay=inline&refresh_mode=paused&storage=hot&stream_sort=time%2Cdesc&top_n=30&top_o=top&viz=stream&x_missing=true&from_ts=1726433975166&to_ts=1726433977504&live=false) ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview The focus of this PR is to enhance error handling and retry mechanisms for on-chain events in the Hubble application. ### Detailed summary - Added feature to request missing on-chain events on related submit message errors - Improved error code handling for validation failures and unknown signers - Implemented retry mechanism for on-chain events by fid - Enhanced error messages and error handling logic > The following files were skipped due to too many changes: `apps/hubble/src/storage/engine/index.ts`, `apps/hubble/src/eth/l2EventsProvider.ts` > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Aditi Srinivasan
|
398b66162f |
chore: release shuttle v0.6.3 (#2310)
Release shuttle v0.6.3. ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR updates the version of `@farcaster/shuttle` package to `0.6.3` and includes a new metric for events processed with event type tag. ### Detailed summary - Updated version to `0.6.3` - Added a new metric for events processed with event type tag > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Aditi Srinivasan
|
f6ba2b096a |
chore: add metric to track events received by kind (#2309)
There are regular spikes in events received that aren't reflected in the hub metrics. This metric should be useful for understanding what kind of events are causing the spikes. ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR adds a metric for events processed with event type tag in the `shuttle` package. ### Detailed summary - Added metric for events processed with event type tag in `shuttle/eventStream.ts` - Incremented attempt count for current and stale events - Added hubEventType tag to metrics - Calculated dequeue delay for events > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Aditi Srinivasan
|
4cd36222eb |
chore: release shuttle v0.6.2 (#2308)
Releasing shuttle v0.6.2. ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR updates the version of `@farcaster/shuttle` from 0.6.1 to 0.6.2. The focus is on adding more metrics to `HubSubscriber` and event processing metrics. ### Detailed summary - Updated version to 0.6.2 - Added more metrics to HubSubscriber - Added event kind and stale/not stale tags to event processing metrics > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Aditi Srinivasan
|
b7a0f402e2 |
chore: add more tags to event stream metrics (#2307)
In order to better understand how the stale event processing and the event-specific codepaths impact the message read latency, add tags to identify these attributes. ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR adds event kind and stale/not stale tags to event processing metrics in the `HubEventStreamConsumer` class. ### Detailed summary - Added `whenReceived` tag to event processing metrics - Updated metrics for different stages of event processing - Introduced distinction between current and stale events > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Aditi Srinivasan
|
4e897e9ade |
chore: add more metrics to hub stream reader (#2302)
We need some more instrumentation to understand how much of a time to read delay is accounted for by the hub stream -> redis step vs the redis -> postgres step. ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR adds more metrics to `HubSubscriber`. ### Detailed summary - Added `shardKey` property to `EventStreamHubSubscriber` - Added metrics for batch size and process time per event and per batch > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Cassandra Heart
|
bbfda86b9e |
chore: bump shuttle version (#2286)
## Why is this change needed? Bump shuttle version to release package with streaming fetch support. ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR updates the version of `@farcaster/shuttle` package to `0.6.1` and implements a feature to switch shuttle to use stream fetch. ### Detailed summary - Updated package version to `0.6.1` - Implemented feature to switch shuttle to use stream fetch > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Cassandra Heart
|
78342c3dc4 |
feat: switch shuttle to use stream fetch (#2285)
## Why is this change needed? Follow up to add stream fetch support to shuttle ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR updates the `shuttle` package to use stream fetch and adds error handling for stream establishment. ### Detailed summary - Updated `shuttle` to use stream fetch - Added error handling for stream establishment using `HubError` - Added new methods for different message types retrieval - Implemented stream fetch functionality in `MessageReconciliation` > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Cassandra Heart
|
57723586e3 |
release x.y.2 packages (#2284)
## Why is this change needed? Bumps the version for releasing packages ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview The focus of this PR is to update version numbers, add support for streaming, and update dependencies across different packages. ### Detailed summary - Updated version numbers in `hub-web` and `core` packages - Added support for streaming in `core` package - Updated dependencies in `hub-web` package > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Cassandra Heart
|
321658b780 |
Streaming support for sync and reconciliation (#2273)
## Why is this change needed? Add support for streaming interfaces for sync and reconciliation. Shuttle change to immediately follow. ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR updates dependencies, adds streaming support for sync/reconciliation, and introduces a new `useStreaming` option in the Hubble application. ### Detailed summary - Added `ClientDuplexStream` in `hub-nodejs` - Updated versions in `core` and `hub-nodejs` - Added streaming methods in `rpc.proto` - Introduced `useStreaming` option in `hubble` application > The following files were skipped due to too many changes: `packages/hub-web/src/generated/rpc.ts`, `packages/hub-nodejs/src/generated/rpc.ts`, `apps/hubble/src/network/sync/syncEnginePerf.test.ts`, `apps/hubble/src/network/utils/networkConfig.test.ts`, `apps/hubble/src/network/sync/syncEngine.ts`, `apps/hubble/src/rpc/server.ts`, `packages/hub-web/src/generated/request_response.ts`, `packages/hub-nodejs/src/generated/request_response.ts`, `packages/core/src/protobufs/generated/request_response.ts`, `apps/hubble/src/network/sync/multiPeerSyncEngine.test.ts` > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Sanjay
|
810a9cde10 |
chore: Release shuttle 0.6 (#2281)
## Why is this change needed? Release shuttle 0.6 with support for handling hub event level callbacks. ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR updates the version of `@farcaster/shuttle` to `0.6.0`, introducing support for event level callbacks and fixing reconciliation issues in hub rpcs. ### Detailed summary - Updated version to 0.6.0 - Added support for event level callbacks - Fixed reconciliation not using time bounds for hub rpcs - Updated `onHubEvent` to accept a transaction for consistency > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Sanjay
|
2bf64461c2 |
fix: Update onHubEvent to accept a txn for consistency (#2280)
## Why is this change needed? Process both the hub event and the messages in that event within the same db transaction. ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR updates the `onHubEvent` method in the `shuttle` package to accept a transaction for consistency. ### Detailed summary - Updated `onHubEvent` method in `shuttle` package to accept a transaction parameter - Updated method calls in `HubEventProcessor` to pass the transaction parameter for consistency > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Aditi Srinivasan
|
49ecf50e40 |
fix: have reconciliation actually use time bounds (#2275)
The time bounds provided for the hub rpcs weren't actually being used. ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [x] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR focuses on fixing reconciliation in the `shuttle` package by ensuring time bounds are used for hub rpcs. ### Detailed summary - Added `startTimestamp` and `stopTimestamp` parameters to `allHubMessagesOfTypeForFid` method call. - Updated reconciliation logic to use time bounds for hub rpcs. > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Sanjay
|
7eba6ee0c6 |
feat(shuttle): Support event level callback in shuttle (#2271)
## Why is this change needed? Support event level callbacks in Shuttle. This allows flexibility in skipping event processing if desired, and custom handling for onchain events and username proofs. ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR adds support for event level callbacks in the shuttle package. ### Detailed summary - Added `OnChainEventRow` type to `db.ts` - Updated `HubEventProcessor` to support event level callbacks - Added `onHubEvent` method to `MessageHandler` - Added migration script for `onchain_events` table - Updated `App` class to handle `MergeOnChainHubEvent` events > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Sanjay
|
e3422809fe |
chore: release 1.14.5 (#2270)
## Why is this change needed? Release 1.14.5 ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR bumps versions of various packages, introduces time range options for reconciliation, and moves storage unit type helper functions to the hub-nodejs package. ### Detailed summary - Bumped versions of multiple packages - Added time range option for reconciliation - Moved storage unit type helper functions to hub-nodejs package > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Aditi Srinivasan
|
70d1f830c1 |
feat: use start and stop time range for reconciliation bulk queries (#2269)
## Why is this change needed? Reconciliation takes a really long time and puts a lot of pressure on hubs from read rpcs. Specifying a start and stop time range will reduce the overhead of running reconciliation. ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [ ] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR introduces a time range option for reconciliation in the `shuttle` package. ### Detailed summary - Added `startTimestamp` and `stopTimestamp` parameters to `MessageReconciliation` class - Implemented time range filtering for message reconciliation - Updated message retrieval methods to support time range filtering > The following files were skipped due to too many changes: `packages/shuttle/src/shuttle/messageReconciliation.ts` > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Aditi Srinivasan
|
cc0d0a3ecb |
feat: add start and stop time to bulk rpcs with large volumes of data (#2255)
We want to be able to restrict the reconciliation job's queries to a time range of e.g. past 1-2 weeks to reduce the amount of read load on hubs. The most expensive part of the read operation is serializing the protobufs to send back to the callee and that's what we want to save on. Notes for reviewers: - There are ways to optimize the time query by (1) using the indexes (2) starting at the common prefix of the start and stop time rather than iterating all the results for the fid. I think it's not worth pursuing these optimizations until we have evidence that we need them. The simple solution implemented here will eliminate the protobuf serialization and significantly less risky than the more optimized solutions. - `getUserDataByFid` and `getAllUserDataMessagesByFid` have the exact same implementation. I added time ranges to the one we classified as a bulk rpc but not the other one. We should consider eliminating one of these rpcs. - I didn't add a time range for the link compaction bulk query because (1) we don't query for link compaction messages in reconciliation (2) it seems like we don't store a lot of link compaction state-- each new link compaction message for an fid replaces the older one. - I'm not sure we need the time filter for user data, but I added it anyway. Happy to remove if it seems like a bad complexity tradeoff. - The rpc protocol change is backwards compatible. I copied the format for `TimestampFidRequest` from `FidRequest` and added new optional parameters. ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR adds start/stop time filters for bulk queries and validation methods for time inputs. ### Detailed summary - Added start/stop time filters for bulk queries in various modules - Implemented validation methods for farcaster time inputs > The following files were skipped due to too many changes: `apps/hubble/src/addon/src/store/store.rs`, `apps/hubble/src/rpc/server.ts`, `packages/hub-web/src/generated/request_response.ts`, `packages/hub-nodejs/src/generated/request_response.ts`, `packages/core/src/protobufs/generated/request_response.ts`, `packages/hub-web/src/generated/rpc.ts`, `apps/hubble/src/storage/engine/index.ts`, `packages/hub-nodejs/src/generated/rpc.ts`, `apps/hubble/src/rpc/test/bulkService.test.ts` > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Sanjay
|
fbd3ba5fd9 |
fix: Move storage unit type helper functions to hub-nodejs package (#2267)
## Why is this change needed? Expose storage unit helper functions via hub-nodejs ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR focuses on moving storage unit type helper functions to the `hub-nodejs` package. ### Detailed summary - Moved storage unit type helper functions to `hub-nodejs` package - Added `LEGACY_STORAGE_UNIT_CUTOFF_TIMESTAMP` constant - Updated functions related to storage unit type and expiry calculations > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Sanjay
|
4fdfb0462e |
chore: Release 1.14.4 (#2266)
## Why is this change needed? Release 1.14.4 that supports the storage extension FIP: https://github.com/farcasterxyz/protocol/discussions/191 ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR updates multiple packages to implement a new Storage Extension FIP and upgrade dependencies, focusing on version changes and dependency updates. ### Detailed summary - Implemented Storage Extension FIP in multiple packages - Updated dependencies across different packages - Version upgrades to 0.15.0 in core packages and 0.12.0 in hub packages > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Sanjay
|
dd634c7913 |
feat: Implement storage extension FIP (#2256)
## Why is this change needed? Implements https://github.com/farcasterxyz/protocol/discussions/191 ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview The focus of this PR is to implement the Storage Extension FIP. ### Detailed summary - Added `StorageUnitDetails` message in `request_response.proto` - Updated storage-related functions in various Rust files - Modified default limits in different store types - Introduced new date calculations in `factories.ts` > The following files were skipped due to too many changes: `apps/hubble/src/storage/stores/storeEventHandler.test.ts`, `packages/core/src/limits.ts`, `apps/hubble/src/storage/stores/storageCache.test.ts`, `apps/hubble/src/storage/stores/storeEventHandler.ts`, `apps/hubble/src/rpc/test/server.test.ts`, `packages/core/src/limits.test.ts`, `apps/hubble/src/storage/engine/index.ts`, `packages/hub-web/src/generated/request_response.ts`, `packages/hub-nodejs/src/generated/request_response.ts`, `packages/core/src/protobufs/generated/request_response.ts`, `apps/hubble/src/storage/stores/storageCache.ts` > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Cassandra Heart
|
3fbfb6e4bd |
chore: revert "chore: revert "fix: Upgrade grpc-js to 1.11 (#2214)" (#2263)" (#2265)
## Why is this change needed? The grpc-js downgrade did not have a measurable impact ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [X] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR updates dependencies and refactors stream handling in the `bufferedStreamWriter` and `server` files. ### Detailed summary - Updated `@grpc/grpc-js` dependency to version `~1.11.1` - Refactored stream handling in `bufferedStreamWriter.ts` - Added `destroyStream` function in `server.ts` for handling stream errors > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Cassandra Heart
|
dac281ac03 |
chore: revert "fix: Upgrade grpc-js to 1.11 (#2214)" (#2263)
## Why is this change needed?
This reverts commit
|
||
|
Cassandra Heart
|
0cb68f4758 |
Revert "fix: adjust grpc keepalive time to 5s (#2250)" (#2262)
This reverts commit
|
||
|
Shane da Silva
|
ed11bf8c8d |
chore: Revert timeout change (#2261)
## Why is this change needed? We think this might be responsible for some instability we're seeing on our hubs. Revert it to confirm. - **Revert "chore: switch timeout to 25s (#2259)"** - **Revert "chore: tweak client-side config to set baseline timeout (#2257)"** ## Merge Checklist - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [ ] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview The focus of this PR is to streamline the gRPC client configuration by removing redundant keepalive settings. ### Detailed summary - Removed redundant `grpc.keepalive_time_ms` and `grpc.keepalive_timeout_ms` settings in multiple files. - Simplified `getHubRpcClient` calls by removing unnecessary options. > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Cassandra Heart
|
9d4f82461f |
chore: switch timeout to 25s (#2259)
## Why is this change needed? In the timeout saga, getting the timeout _above_ the NLB because it disregards everything you tell it seems to be the key. ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview The focus of this PR is to tweak the timeout settings in the `hub-nodejs` and `hubble` packages to 25 seconds for better performance. ### Detailed summary - Updated `grpc.keepalive_timeout_ms` to 25 seconds in `hub-nodejs/src/client.ts` - Updated `grpc.keepalive_timeout_ms` to 25 seconds in `hubble/src/hubble.ts` > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Sanjay
|
29886461b4 |
fix: Reduce reconciliation default pagination size (#2251)
## Why is this change needed? Reduce default pagination size to prevent hub from spending too much time serializing messages ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR updates version numbers and configurations for the `@farcaster/hub-nodejs` and `@farcaster/shuttle` packages. It also adjusts the `MAX_PAGE_SIZE` constant in `messageReconciliation.ts`. ### Detailed summary - Updated `MAX_PAGE_SIZE` constant in `messageReconciliation.ts` from 3000 to 500 - Updated `@farcaster/hub-nodejs` version to `0.11.24` - Updated `@farcaster/shuttle` version to `0.5.11` - Updated dependencies in `shuttle/package.json` and `hub-nodejs/package.json` - Added changelog entries for versions `0.11.24` and `0.5.11` > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Cassandra Heart
|
47fbd34e5c |
fix: adjust grpc keepalive time to 5s (#2250)
## Why is this change needed? gRPC keepalive heartbeat time was set to 10s, with a client-side response time of 5s. This allows servers that are unresponsive to go unrecognized for longer, causing sync time to dramatically increase with respect to hub volume. Lowering this value should be safe for a patch, but lowering the client response time will require a minor revision due to incompatibility with client-side expectations ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR focuses on adjusting the gRPC keepalive time to 5 seconds in `@farcaster/hub-nodejs` package to improve failover speed. ### Detailed summary - Adjusted gRPC keepalive time to 5 seconds in `client.ts` to encourage faster failover from uncooperative peers. > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Sanjay
|
45e16ced9c |
chore: Release 1.14.2 (#2235)
## Why is this change needed? Release 1.14.2 ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR updates dependencies and versions across different packages. ### Detailed summary - Updated `@farcaster/core` version to `0.14.20` - Updated `@farcaster/hub-nodejs` version to `0.11.23` - Upgraded `libp2p/gossipsub` and dependencies - Chore: upgraded `viem` to v2 > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Shane da Silva
|
c5e4457735 |
fix(shuttle): Correct example app to use await (#2234)
## Why is this change needed? This is leading to confusion. If you don't `await`, you can overwhelm the event loop. ## Merge Checklist - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [ ] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [ ] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR ensures the `processHubEvent` method is awaited when called in the stream consumer, preventing it from being executed void. ### Detailed summary - Changed `this.processHubEvent(event);` to `await this.processHubEvent(event);` in the stream consumer to ensure proper execution. > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Shane da Silva
|
fa5f13be35 |
chore(shuttle): Release 0.5.10 (#2233)
## Why is this change needed? Includes a hotfix. ## Merge Checklist - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR updates the version of `@farcaster/shuttle` from 0.5.9 to 0.5.10. It includes a bug fix for a missing `await` in a `sleep()` call and updates dependencies. ### Detailed summary - Updated `@farcaster/shuttle` version to 0.5.10 - Bug fix: Added missing `await` to `sleep()` call - Updated dependencies for `@farcaster/hub-nodejs` to version 0.11.23 > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
Ty Book
|
165a0aac74 |
fix(shuttle): Add missing await to per-iteration sleep() call (#2232)
## Why is this change needed? The fix for high CPU usage due to busy-waiting (release v0.5.9) wasn't effective because `sleep()` returns a Promise which wasn't being `await`ed, so the containing `while` loop was still iterating unchecked thousands of times per second. This should fix for good. ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR focuses on fixing a bug in the `shuttle` package where an `await` was missing in the `sleep()` call, potentially causing CPU thrashing. ### Detailed summary - Added missing `await` to `sleep()` call in `eventStream.ts` to prevent CPU thrashing. > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |
||
|
dependabot[bot]
|
5a03c774a1 |
chore(deps): bump openssl from 0.10.60 to 0.10.66 in /packages/hub-web/examples/rust-submitmessage (#2202)
Bumps [openssl](https://github.com/sfackler/rust-openssl) from 0.10.60 to 0.10.66. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/sfackler/rust-openssl/releases">openssl's releases</a>.</em></p> <blockquote> <h2>openssl-v0.10.66</h2> <h2>What's Changed</h2> <ul> <li>Fixed invariant violation in <code>MemBio::get_buf</code> with empty results by <a href="https://github.com/alex"><code>@alex</code></a> in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2266">sfackler/rust-openssl#2266</a></li> <li>Release openssl v0.10.66 by <a href="https://github.com/alex"><code>@alex</code></a> in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2267">sfackler/rust-openssl#2267</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/sfackler/rust-openssl/compare/openssl-v0.10.65...openssl-v0.10.66">https://github.com/sfackler/rust-openssl/compare/openssl-v0.10.65...openssl-v0.10.66</a></p> <h2>openssl-v0.10.65</h2> <h2>What's Changed</h2> <ul> <li>don't emit rerun-if-changed when vendoring by <a href="https://github.com/reaperhulk"><code>@reaperhulk</code></a> in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2177">sfackler/rust-openssl#2177</a></li> <li>Prepare for openssl-sys 0.9.101 release by <a href="https://github.com/alex"><code>@alex</code></a> in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2182">sfackler/rust-openssl#2182</a></li> <li>don't emit rerun-if-changed unless the path exists and is readable by <a href="https://github.com/reaperhulk"><code>@reaperhulk</code></a> in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2187">sfackler/rust-openssl#2187</a></li> <li>Added support for LibreSSL 3.9.0 by <a href="https://github.com/alex"><code>@alex</code></a> in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2202">sfackler/rust-openssl#2202</a></li> <li>Support stable LibreSSL 3.9.x by <a href="https://github.com/alex"><code>@alex</code></a> in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2209">sfackler/rust-openssl#2209</a></li> <li>openssl-sys 0.9.102 release by <a href="https://github.com/alex"><code>@alex</code></a> in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2210">sfackler/rust-openssl#2210</a></li> <li>Add repository field to openssl-macros crate by <a href="https://github.com/paolobarbolini"><code>@paolobarbolini</code></a> in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2211">sfackler/rust-openssl#2211</a></li> <li>Add missing openssl-sys dependency by <a href="https://github.com/pieterdd"><code>@pieterdd</code></a> in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2212">sfackler/rust-openssl#2212</a></li> <li>Test OpenSSL 3.3.0-beta1 by <a href="https://github.com/sfackler"><code>@sfackler</code></a> in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2216">sfackler/rust-openssl#2216</a></li> <li>test against 3.3.0 final by <a href="https://github.com/alex"><code>@alex</code></a> in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2218">sfackler/rust-openssl#2218</a></li> <li>fix min-versions in CI by <a href="https://github.com/alex"><code>@alex</code></a> in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2228">sfackler/rust-openssl#2228</a></li> <li>Make X509_VAL opaque for LibreSSL 4.0.0 by <a href="https://github.com/botovq"><code>@botovq</code></a> in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2227">sfackler/rust-openssl#2227</a></li> <li>Use the newer names for STACK_OF(T) functions with BoringSSL by <a href="https://github.com/davidben"><code>@davidben</code></a> in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2231">sfackler/rust-openssl#2231</a></li> <li>Only declare OpensslCallbacks in bindgen builds by <a href="https://github.com/alex"><code>@alex</code></a> in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2234">sfackler/rust-openssl#2234</a></li> <li>Fix building with latest BoringSSL by <a href="https://github.com/davidben"><code>@davidben</code></a> in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2230">sfackler/rust-openssl#2230</a></li> <li>Emit rustc-check-cfg for nightly by <a href="https://github.com/alex"><code>@alex</code></a> in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2235">sfackler/rust-openssl#2235</a></li> <li>Configure OpenSSL data dir on vendored builds. by <a href="https://github.com/DanielSidhion"><code>@DanielSidhion</code></a> in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2122">sfackler/rust-openssl#2122</a></li> <li>Add boringssl keylog callback support by <a href="https://github.com/mspublic"><code>@mspublic</code></a> in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2237">sfackler/rust-openssl#2237</a></li> <li>Correct the name of the <code>pkgconf</code> package on some distros by <a href="https://github.com/JonathanBrouwer"><code>@JonathanBrouwer</code></a> in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2253">sfackler/rust-openssl#2253</a></li> <li>Add some OpenSSL 3 QUIC raw bindings by <a href="https://github.com/bdbai"><code>@bdbai</code></a> in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2257">sfackler/rust-openssl#2257</a></li> <li>Initialize OpenSSL in MD constructors by <a href="https://github.com/sfackler"><code>@sfackler</code></a> in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2258">sfackler/rust-openssl#2258</a></li> <li>Switch Pkey::from_ to use set1 functions by <a href="https://github.com/alex"><code>@alex</code></a> in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2262">sfackler/rust-openssl#2262</a></li> <li>Release openssl v0.10.65 and openssl-sys v0.9.103 by <a href="https://github.com/alex"><code>@alex</code></a> in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2265">sfackler/rust-openssl#2265</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/paolobarbolini"><code>@paolobarbolini</code></a> made their first contribution in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2211">sfackler/rust-openssl#2211</a></li> <li><a href="https://github.com/pieterdd"><code>@pieterdd</code></a> made their first contribution in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2212">sfackler/rust-openssl#2212</a></li> <li><a href="https://github.com/DanielSidhion"><code>@DanielSidhion</code></a> made their first contribution in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2122">sfackler/rust-openssl#2122</a></li> <li><a href="https://github.com/mspublic"><code>@mspublic</code></a> made their first contribution in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2237">sfackler/rust-openssl#2237</a></li> <li><a href="https://github.com/JonathanBrouwer"><code>@JonathanBrouwer</code></a> made their first contribution in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2253">sfackler/rust-openssl#2253</a></li> <li><a href="https://github.com/bdbai"><code>@bdbai</code></a> made their first contribution in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2257">sfackler/rust-openssl#2257</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/sfackler/rust-openssl/compare/openssl-v0.10.64...openssl-v0.10.65">https://github.com/sfackler/rust-openssl/compare/openssl-v0.10.64...openssl-v0.10.65</a></p> <h2>openssl-v0.10.64</h2> <h2>What's Changed</h2> <ul> <li>Make _STACK opaque for LibreSSL >= 3.9.0 by <a href="https://github.com/botovq"><code>@botovq</code></a> in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2153">sfackler/rust-openssl#2153</a></li> <li>enable x509 verify and groups list for boringssl by <a href="https://github.com/zh-jq"><code>@zh-jq</code></a> in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2155">sfackler/rust-openssl#2155</a></li> <li>Cleanup some not-required Path::new invocations by <a href="https://github.com/alex"><code>@alex</code></a> in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2158">sfackler/rust-openssl#2158</a></li> <li>fixed a clippy (nightly) warning by <a href="https://github.com/alex"><code>@alex</code></a> in <a href="https://redirect.github.com/sfackler/rust-openssl/pull/2161">sfackler/rust-openssl#2161</a></li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
Shane da Silva
|
04b19fc387 |
chore(shuttle): Release 0.5.9 (#2230)
## Why is this change needed? Includes a fix for a recently introduced CPU throttling regression. ## Merge Checklist - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR updates the version of `@farcaster/shuttle` from 0.5.8 to 0.5.9. It includes a fix for CPU throttling regression and updates dependencies. ### Detailed summary - Updated `@farcaster/shuttle` version to 0.5.9 - Fixed CPU throttling regression introduced in 0.5.6 - Updated dependencies including `@farcaster/hub-nodejs@0.11.23` > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> |