Commit Graph

1553 Commits

Author SHA1 Message Date
Sanjay
45e16ced9c chore: Release 1.14.2 (#2235)
## Why is this change needed?

Release 1.14.2

## Merge Checklist

_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.


<!-- start pr-codex -->

---

## PR-Codex overview
This PR updates dependencies and versions across different packages. 

### Detailed summary
- Updated `@farcaster/core` version to `0.14.20`
- Updated `@farcaster/hub-nodejs` version to `0.11.23`
- Upgraded `libp2p/gossipsub` and dependencies
- Chore: upgraded `viem` to v2

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
@farcaster/hubble@1.14.2 @farcaster/hub-nodejs@0.11.23 @farcaster/core@0.14.20
2024-08-01 16:10:38 -07:00
Shane da Silva
c5e4457735 fix(shuttle): Correct example app to use await (#2234)
## Why is this change needed?

This is leading to confusion. If you don't `await`, you can overwhelm
the event loop.

## Merge Checklist

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [ ] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [ ] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.


<!-- start pr-codex -->

---

## PR-Codex overview
This PR ensures the `processHubEvent` method is awaited when called in
the stream consumer, preventing it from being executed void.

### Detailed summary
- Changed `this.processHubEvent(event);` to `await
this.processHubEvent(event);` in the stream consumer to ensure proper
execution.

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
2024-08-01 10:09:04 -07:00
Shane da Silva
fa5f13be35 chore(shuttle): Release 0.5.10 (#2233)
## Why is this change needed?

Includes a hotfix.

## Merge Checklist

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.


<!-- start pr-codex -->

---

## PR-Codex overview
This PR updates the version of `@farcaster/shuttle` from 0.5.9 to
0.5.10. It includes a bug fix for a missing `await` in a `sleep()` call
and updates dependencies.

### Detailed summary
- Updated `@farcaster/shuttle` version to 0.5.10
- Bug fix: Added missing `await` to `sleep()` call
- Updated dependencies for `@farcaster/hub-nodejs` to version 0.11.23

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
@farcaster/shuttle@0.5.10
2024-08-01 09:31:03 -07:00
Ty Book
165a0aac74 fix(shuttle): Add missing await to per-iteration sleep() call (#2232)
## Why is this change needed?

The fix for high CPU usage due to busy-waiting (release v0.5.9) wasn't
effective because `sleep()` returns a Promise which wasn't being
`await`ed, so the containing `while` loop was still iterating unchecked
thousands of times per second. This should fix for good.

## Merge Checklist

_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.


<!-- start pr-codex -->

---

## PR-Codex overview
This PR focuses on fixing a bug in the `shuttle` package where an
`await` was missing in the `sleep()` call, potentially causing CPU
thrashing.

### Detailed summary
- Added missing `await` to `sleep()` call in `eventStream.ts` to prevent
CPU thrashing.

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
2024-08-01 09:28:07 -07:00
Sanjay
fb2645ee22 fix: Include peerid in bootstrap multiaddr (#2231)
## Why is this change needed?

Newer version of libp2p requires the peer id in the bootstrap multiaddr.
Without it, the connection is denied. We suspect the existing hubs are
working because the peer id has been cached in the peer store. New hubs
do not successfully startup and peer.

## Merge Checklist

_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.

<!-- start pr-codex -->

---

## PR-Codex overview
This PR updates the `bootstrapPeers.mainnet.ts` file to include the
`peerid` in the bootstrap multiaddr for `MAINNET_BOOTSTRAP_PEERS`.

### Detailed summary
- Added `peerid` to `MAINNET_BOOTSTRAP_PEERS` multiaddrs for
`hoyt.farcaster.xyz`, `lamia.farcaster.xyz`, and `bootstrap.neynar.com`.

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
2024-07-31 17:12:03 -07:00
dependabot[bot]
5a03c774a1 chore(deps): bump openssl from 0.10.60 to 0.10.66 in /packages/hub-web/examples/rust-submitmessage (#2202)
Bumps [openssl](https://github.com/sfackler/rust-openssl) from 0.10.60
to 0.10.66.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/sfackler/rust-openssl/releases">openssl's
releases</a>.</em></p>
<blockquote>
<h2>openssl-v0.10.66</h2>
<h2>What's Changed</h2>
<ul>
<li>Fixed invariant violation in <code>MemBio::get_buf</code> with empty
results by <a href="https://github.com/alex"><code>@​alex</code></a> in
<a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2266">sfackler/rust-openssl#2266</a></li>
<li>Release openssl v0.10.66 by <a
href="https://github.com/alex"><code>@​alex</code></a> in <a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2267">sfackler/rust-openssl#2267</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/sfackler/rust-openssl/compare/openssl-v0.10.65...openssl-v0.10.66">https://github.com/sfackler/rust-openssl/compare/openssl-v0.10.65...openssl-v0.10.66</a></p>
<h2>openssl-v0.10.65</h2>
<h2>What's Changed</h2>
<ul>
<li>don't emit rerun-if-changed when vendoring by <a
href="https://github.com/reaperhulk"><code>@​reaperhulk</code></a> in <a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2177">sfackler/rust-openssl#2177</a></li>
<li>Prepare for openssl-sys 0.9.101 release by <a
href="https://github.com/alex"><code>@​alex</code></a> in <a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2182">sfackler/rust-openssl#2182</a></li>
<li>don't emit rerun-if-changed unless the path exists and is readable
by <a href="https://github.com/reaperhulk"><code>@​reaperhulk</code></a>
in <a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2187">sfackler/rust-openssl#2187</a></li>
<li>Added support for LibreSSL 3.9.0 by <a
href="https://github.com/alex"><code>@​alex</code></a> in <a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2202">sfackler/rust-openssl#2202</a></li>
<li>Support stable LibreSSL 3.9.x by <a
href="https://github.com/alex"><code>@​alex</code></a> in <a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2209">sfackler/rust-openssl#2209</a></li>
<li>openssl-sys 0.9.102 release by <a
href="https://github.com/alex"><code>@​alex</code></a> in <a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2210">sfackler/rust-openssl#2210</a></li>
<li>Add repository field to openssl-macros crate by <a
href="https://github.com/paolobarbolini"><code>@​paolobarbolini</code></a>
in <a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2211">sfackler/rust-openssl#2211</a></li>
<li>Add missing openssl-sys dependency by <a
href="https://github.com/pieterdd"><code>@​pieterdd</code></a> in <a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2212">sfackler/rust-openssl#2212</a></li>
<li>Test OpenSSL 3.3.0-beta1 by <a
href="https://github.com/sfackler"><code>@​sfackler</code></a> in <a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2216">sfackler/rust-openssl#2216</a></li>
<li>test against 3.3.0 final by <a
href="https://github.com/alex"><code>@​alex</code></a> in <a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2218">sfackler/rust-openssl#2218</a></li>
<li>fix min-versions in CI by <a
href="https://github.com/alex"><code>@​alex</code></a> in <a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2228">sfackler/rust-openssl#2228</a></li>
<li>Make X509_VAL opaque for LibreSSL 4.0.0 by <a
href="https://github.com/botovq"><code>@​botovq</code></a> in <a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2227">sfackler/rust-openssl#2227</a></li>
<li>Use the newer names for STACK_OF(T) functions with BoringSSL by <a
href="https://github.com/davidben"><code>@​davidben</code></a> in <a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2231">sfackler/rust-openssl#2231</a></li>
<li>Only declare OpensslCallbacks in bindgen builds by <a
href="https://github.com/alex"><code>@​alex</code></a> in <a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2234">sfackler/rust-openssl#2234</a></li>
<li>Fix building with latest BoringSSL by <a
href="https://github.com/davidben"><code>@​davidben</code></a> in <a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2230">sfackler/rust-openssl#2230</a></li>
<li>Emit rustc-check-cfg for nightly by <a
href="https://github.com/alex"><code>@​alex</code></a> in <a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2235">sfackler/rust-openssl#2235</a></li>
<li>Configure OpenSSL data dir on vendored builds. by <a
href="https://github.com/DanielSidhion"><code>@​DanielSidhion</code></a>
in <a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2122">sfackler/rust-openssl#2122</a></li>
<li>Add boringssl keylog callback support by <a
href="https://github.com/mspublic"><code>@​mspublic</code></a> in <a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2237">sfackler/rust-openssl#2237</a></li>
<li>Correct the name of the <code>pkgconf</code> package on some distros
by <a
href="https://github.com/JonathanBrouwer"><code>@​JonathanBrouwer</code></a>
in <a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2253">sfackler/rust-openssl#2253</a></li>
<li>Add some OpenSSL 3 QUIC raw bindings by <a
href="https://github.com/bdbai"><code>@​bdbai</code></a> in <a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2257">sfackler/rust-openssl#2257</a></li>
<li>Initialize OpenSSL in MD constructors by <a
href="https://github.com/sfackler"><code>@​sfackler</code></a> in <a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2258">sfackler/rust-openssl#2258</a></li>
<li>Switch Pkey::from_ to use set1 functions by <a
href="https://github.com/alex"><code>@​alex</code></a> in <a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2262">sfackler/rust-openssl#2262</a></li>
<li>Release openssl v0.10.65 and openssl-sys v0.9.103 by <a
href="https://github.com/alex"><code>@​alex</code></a> in <a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2265">sfackler/rust-openssl#2265</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a
href="https://github.com/paolobarbolini"><code>@​paolobarbolini</code></a>
made their first contribution in <a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2211">sfackler/rust-openssl#2211</a></li>
<li><a href="https://github.com/pieterdd"><code>@​pieterdd</code></a>
made their first contribution in <a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2212">sfackler/rust-openssl#2212</a></li>
<li><a
href="https://github.com/DanielSidhion"><code>@​DanielSidhion</code></a>
made their first contribution in <a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2122">sfackler/rust-openssl#2122</a></li>
<li><a href="https://github.com/mspublic"><code>@​mspublic</code></a>
made their first contribution in <a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2237">sfackler/rust-openssl#2237</a></li>
<li><a
href="https://github.com/JonathanBrouwer"><code>@​JonathanBrouwer</code></a>
made their first contribution in <a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2253">sfackler/rust-openssl#2253</a></li>
<li><a href="https://github.com/bdbai"><code>@​bdbai</code></a> made
their first contribution in <a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2257">sfackler/rust-openssl#2257</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/sfackler/rust-openssl/compare/openssl-v0.10.64...openssl-v0.10.65">https://github.com/sfackler/rust-openssl/compare/openssl-v0.10.64...openssl-v0.10.65</a></p>
<h2>openssl-v0.10.64</h2>
<h2>What's Changed</h2>
<ul>
<li>Make _STACK opaque for LibreSSL &gt;= 3.9.0 by <a
href="https://github.com/botovq"><code>@​botovq</code></a> in <a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2153">sfackler/rust-openssl#2153</a></li>
<li>enable x509 verify and groups list for boringssl by <a
href="https://github.com/zh-jq"><code>@​zh-jq</code></a> in <a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2155">sfackler/rust-openssl#2155</a></li>
<li>Cleanup some not-required Path::new invocations by <a
href="https://github.com/alex"><code>@​alex</code></a> in <a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2158">sfackler/rust-openssl#2158</a></li>
<li>fixed a clippy (nightly) warning by <a
href="https://github.com/alex"><code>@​alex</code></a> in <a
href="https://redirect.github.com/sfackler/rust-openssl/pull/2161">sfackler/rust-openssl#2161</a></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="ad70a0bea5"><code>ad70a0b</code></a>
Merge pull request <a
href="https://redirect.github.com/sfackler/rust-openssl/issues/2267">#2267</a>
from alex/bump-for-release</li>
<li><a
href="5ce473b4e5"><code>5ce473b</code></a>
Release openssl v0.10.66</li>
<li><a
href="aef36e0f39"><code>aef36e0</code></a>
Merge pull request <a
href="https://redirect.github.com/sfackler/rust-openssl/issues/2266">#2266</a>
from alex/mem-bio-invariant</li>
<li><a
href="142deef717"><code>142deef</code></a>
Fixed invariant violation in <code>MemBio::get_buf</code> with empty
results</li>
<li><a
href="32f150b05a"><code>32f150b</code></a>
Merge pull request <a
href="https://redirect.github.com/sfackler/rust-openssl/issues/2265">#2265</a>
from alex/bump-for-release</li>
<li><a
href="98addd271e"><code>98addd2</code></a>
Release openssl v0.10.65 and openssl-sys v0.9.103</li>
<li><a
href="7c7958d436"><code>7c7958d</code></a>
Merge pull request <a
href="https://redirect.github.com/sfackler/rust-openssl/issues/2262">#2262</a>
from alex/pkey-api</li>
<li><a
href="d7b12ccf1f"><code>d7b12cc</code></a>
Switch Pkey::from_ to use set1 functions</li>
<li><a
href="22ffa9add6"><code>22ffa9a</code></a>
Merge pull request <a
href="https://redirect.github.com/sfackler/rust-openssl/issues/2258">#2258</a>
from sfackler/init-md</li>
<li><a
href="9de3794474"><code>9de3794</code></a>
Initialize OpenSSL in MD constructors</li>
<li>Additional commits viewable in <a
href="https://github.com/sfackler/rust-openssl/compare/openssl-v0.10.60...openssl-v0.10.66">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=openssl&package-manager=cargo&previous-version=0.10.60&new-version=0.10.66)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/farcasterxyz/hub-monorepo/network/alerts).

</details>

<!-- start pr-codex -->

---

## PR-Codex overview
The focus of this PR is to update the `openssl` and `openssl-sys`
packages in the Cargo.lock file for the `rust-submitmessage` example.

### Detailed summary
- Updated `openssl` package version from 0.10.60 to 0.10.66
- Updated `openssl-sys` package version from 0.9.96 to 0.9.103
- Updated checksums for both packages

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-31 15:59:21 -07:00
Shane da Silva
04b19fc387 chore(shuttle): Release 0.5.9 (#2230)
## Why is this change needed?

Includes a fix for a recently introduced CPU throttling regression.

## Merge Checklist

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.


<!-- start pr-codex -->

---

## PR-Codex overview
This PR updates the version of `@farcaster/shuttle` from 0.5.8 to 0.5.9.
It includes a fix for CPU throttling regression and updates
dependencies.

### Detailed summary
- Updated `@farcaster/shuttle` version to 0.5.9
- Fixed CPU throttling regression introduced in 0.5.6
- Updated dependencies including `@farcaster/hub-nodejs@0.11.23`

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
@farcaster/shuttle@0.5.9
2024-07-31 13:42:07 -07:00
Shane da Silva
501ceff29b Always assign on-chain events to shard 0 for linear ordering (#2229)
## Why is this change needed?

Sharding works when the ordering of events in different shards are
independent of each other.

In the case of onchain events and username proofs, however, these events
can affect multiple FIDs at a time
(e.g. transfers) which means depending on which shard they are assigned
to, they might get processed out of order.

Fix this by always using the same shard (shard 0) for onchain events +
username proofs. This isn't perfect since it means shard 0 is a SPOF,
but that's better than a potential race condition in the processing of
events, and the volume of onchain events is much lower than regular hub
events.

## Merge Checklist

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.

<!-- start pr-codex -->

---

## PR-Codex overview
This PR focuses on ensuring onchain and username proofs are always
assigned to the same shard for linear ordering.

### Detailed summary
- Updated logic to assign onchain and username proofs to the same shard
for linear ordering
- Added a function to check if an event is ordered
- Adjusted tests to reflect the new shard assignment logic

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
2024-07-31 20:41:35 +00:00
Shane da Silva
5d5b3273e2 fix(shuttle): Don't throttle CPU when there are no events to process (#2228)
## Why is this change needed?

We accidentally introduced a change in 0.5.6 which lead to higher CPU
usage since we're no longer sleeping.

## Merge Checklist

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [ ] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.


<!-- start pr-codex -->

---

## PR-Codex overview
The focus of this PR is to fix a CPU throttling regression introduced in
version 0.5.6 of `@farcaster/shuttle`.

### Detailed summary
- Added logic to prevent CPU thrashing when no events are present in
`eventStream.ts`.
- Introduced a sleep function to pause processing when no events are
available.

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
2024-07-31 18:16:19 +00:00
Cassandra Heart
61959467c5 chore: upgrade libp2p all the way (#2224)
## Why is this change needed?

Upgrades libp2p all the way to the latest version ([Release
Notes](https://github.com/libp2p/js-libp2p/releases/tag/libp2p-v1.8.2) |
[Migration
Guide](https://github.com/libp2p/js-libp2p/blob/main/doc/migrations/v0.46-v1.0.0.md))

Upgrades GossipSub along with it.

No major interface changes were required here, just different dependency
references as the libp2p project moved some things around.

## Merge Checklist

_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.

<!-- start pr-codex -->

---

## PR-Codex overview
This PR upgrades `libp2p/gossipsub` and dependencies, aligns imports
with `@libp2p/interface`, and adjusts types in various files.

### Detailed summary
- Upgraded `libp2p/gossipsub` and dependencies
- Aligned imports with `@libp2p/interface`
- Adjusted types in multiple files for compatibility

> The following files were skipped due to too many changes:
`apps/hubble/src/hubble.ts`,
`apps/hubble/src/network/p2p/gossipNodeWorker.ts`, `yarn.lock`

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
2024-07-30 18:04:48 -05:00
Darryl Yeo
939dde846a chore: upgrade viem to v2 (#2181)
## Why is this change needed?

* Upgrades [`viem`](https://viem.sh/) from v1 to
[v2](https://viem.sh/docs/migration-guide#2xx-breaking-changes). More
context in [this cast](https://warpcast.com/v/0xffa5abfb).
* [Maximizes
tree-shaking](https://viem.sh/docs/clients/custom#tree-shaking) by
importing action functions.
* Adds type parameters to helper wrapper classes to ensure inputs align
with `viem`'s deeply-nested conditional types.

## Merge Checklist

_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_

- [ ] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [ ] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [ ] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.

<!-- start pr-codex -->

---

## PR-Codex overview
This PR upgrades `viem` to version 2 across multiple packages. It also
introduces new actions like `getChainId` and `watchBlockNumber`.

### Detailed summary
- Upgraded `viem` to version 2 in various packages
- Added new actions like `getChainId` and `watchBlockNumber`
- Updated usage of `viem` functions in different files

> The following files were skipped due to too many changes:
`packages/hub-nodejs/examples/hello-world/index.ts`,
`apps/hubble/src/eth/l2EventsProvider.test.ts`,
`packages/hub-nodejs/examples/contract-signatures/index.ts`,
`apps/hubble/src/eth/l2EventsProvider.ts`, `yarn.lock`,
`packages/hub-nodejs/examples/contract-signatures/yarn.lock`,
`packages/hub-nodejs/examples/hello-world/yarn.lock`

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->

---------

Co-authored-by: Cassandra Heart <7929478+CassOnMars@users.noreply.github.com>
2024-07-29 16:59:21 -05:00
Sanjay
047d79df39 chore: Release 1.14.1 (#2219)
## Why is this change needed?

Release 1.14.1

## Merge Checklist

_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.


<!-- start pr-codex -->

---

## PR-Codex overview
This PR updates the version of `@farcaster/hubble` to `1.14.1` with
patch changes including upgrading `grpc-js` and `libp2p`.

### Detailed summary
- Bumped `grpc-js` to version 1.11
- Updated `libp2p` to 0.45.0
- Added message merge timing at store level
- Reverted storage cache migration to rust

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
@farcaster/hubble@1.14.1
2024-07-27 10:47:44 -07:00
Sanjay
a9dd162185 Revert "[fix] Fix race condition with earliest ts hash calculation (#… (#2218)
…2204)"

This is causing huge merge latency spikes around pruning. Reverting
while we work on the fix.

This reverts commit 554d82a45b.

## Why is this change needed?

Describe why this issue should be fixed and link to any relevant design
docs, issues or other relevant items.

## Merge Checklist

_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.


<!-- start pr-codex -->

---

## PR-Codex overview
This PR removes the `storage_cache` module, updates related code, and
refactors the `StoreEventHandler` module.

### Detailed summary
- Deleted `storage_cache.rs` and related imports
- Updated `store.rs` to remove `storage_cache` related imports
- Removed `get_storage_cache` function from `utils.rs`
- Updated `message.rs` by removing `PRIMARY_KEY_LENGTH` constant and
related logic
- Updated `lib.rs` by removing `StorageCache` related exports
- Updated `storeEventHandler.ts` to remove `storageCache` parameter in
constructor
- Updated `storeEventHandler.test.ts` to remove `rsCreateStorageCache`
usage
- Updated `store_event_handler.rs` by removing `StorageCache` related
logic and imports
- Updated `rustfunctions.ts` by removing `RustStorageCache` related
functions

> The following files were skipped due to too many changes:
`apps/hubble/src/storage/stores/storageCache.ts`,
`apps/hubble/src/storage/stores/storageCache.test.ts`

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
2024-07-27 10:24:06 -07:00
Sanjay
7bee8436b9 perf: Add message merge timing at the store level (#2215)
## Why is this change needed?

Add store level message merge timing metrics

## Merge Checklist

_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.


<!-- start pr-codex -->

---

## PR-Codex overview
This PR focuses on improving performance by adding message merge timing
at the store level.

### Detailed summary
- Added message merge timing at the store level for performance tracking
- Imported `statsd` for timing storage merge messages
- Calculated duration of message merge operation
- Updated storage engine to include timing data for each store

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
2024-07-26 17:22:03 -07:00
Sanjay
f5960584bf Release shuttle 0.5.8 (#2216)
## Why is this change needed?

Release shuttle 0.5.8 with validate message changes

## Merge Checklist

_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.


<!-- start pr-codex -->

---

## PR-Codex overview
This PR updates versions in `hub-nodejs` and `shuttle` packages. Focus
is on upgrading `grpc-js` in `hub-nodejs` to 1.11 and bumping versions
for both packages.

### Detailed summary
- Updated `hub-nodejs` version to 0.11.22 with `grpc-js` upgrade
- Updated `shuttle` version to 0.5.8
- Removed message validation in `shuttle`
- Updated dependencies in `shuttle`
- Updated `hub-nodejs` dependency version in `shuttle`

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
@farcaster/hub-nodejs@0.11.22 @farcaster/shuttle@0.5.8
2024-07-27 00:19:33 +00:00
Cassandra Heart
095cca9796 chore: bump libp2p to v0.45.0 (#2209)
## Why is this change needed?

Upgrades libp2p one minor version up ([Release
Notes](https://github.com/libp2p/js-libp2p/releases/tag/v0.45.0) |
[Migration
Guide](https://github.com/libp2p/js-libp2p/blob/main/doc/migrations/v0.44-v0.45.md))

By extension, upgrades GossipSub two major versions up due to libp2p
interface compatibility breakage (does not break protocol
compatibility).

### Breaking Changes in Events + Emitters

libp2p altered the events to be more precisely aligned for types to
event names, and so our use of `peer:connect` and `peer:disconnect`
becomes `connection:open` and `connection:close`. Additionally the peer
store's address book methods have been coalesced into its parent type
with more fluent interfaces. These too have been upgraded.

## Merge Checklist

_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [x] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.

<!-- start pr-codex -->

---

## PR-Codex overview
This PR updates dependencies, refactors error handling in peer loading,
adds test cases for multiaddr formats, and enhances GossipNode event
handling.

### Detailed summary
- Updated `@libp2p/interface-mocks` to `^12.0.0`
- Refactored error handling in `store.ts`
- Added test cases for multiaddr formats in `gossipNode.test.ts`
- Enhanced event handling in `gossipNode.ts` and `gossipNodeWorker.ts`

> The following files were skipped due to too many changes:
`apps/hubble/src/network/p2p/gossipNodeWorker.ts`, `yarn.lock`,
`node_modules/@libp2p/peer-store/yarn.lock`,
`node_modules/@libp2p/peer-store/dist/index.min.js`

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
2024-07-26 18:29:50 -05:00
Sanjay
2fa29ad4cb fix: Upgrade grpc-js to 1.11 (#2214)
## Why is this change needed?

Latest version grpc-js has some fixes that might help with the
RST_STREAM issues in hubs.

## Merge Checklist

_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.

<!-- start pr-codex -->

---

## PR-Codex overview
This PR upgrades `grpc-js` to version 1.11, fixes stream handling in
`bufferedStreamWriter.ts`, and adds a `destroyStream` function in
`server.ts`.

### Detailed summary
- Upgraded `grpc-js` to version 1.11
- Improved stream handling in `bufferedStreamWriter.ts`
- Added `destroyStream` function in `server.ts`

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
2024-07-26 17:27:24 +00:00
Sanjay
4f40c19b06 fix: Do not validate messages in shuttle by default (#2213)
## Why is this change needed?

This should significantly improve performance. This was originally
intended as additional protection when listening to untrusted hubs. But
given most users are listening to their own or trusted hub providers, no
need to enable this by default.

## Merge Checklist

_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.

<!-- start pr-codex -->

---

## PR-Codex overview
This PR disables message validation by default in `shuttle`. 

### Detailed summary
- Shuttle now does not validate messages by default
- Added environment variable to enable message validation
- Updated message processing logic in `hubEventProcessor.ts`

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
2024-07-25 17:10:25 -07:00
Sanjay
f1d80deb01 chore: Release 1.14 (#2208)
## Why is this change needed?

Release 1.14

## Merge Checklist

_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.


<!-- start pr-codex -->

---

## PR-Codex overview
This PR updates the version of `@farcaster/hubble` to `1.14.0`,
introduces minor and patch changes, and updates the `FARCASTER_VERSION`
constant in `hubble.ts`.

### Detailed summary
- Updated `@farcaster/hubble` version to `1.14.0`
- Minor and patch changes in `CHANGELOG.md`
- Updated `FARCASTER_VERSION` constant in `hubble.ts`

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
@farcaster/hubble@1.14.0
2024-07-24 15:29:50 -07:00
Cassandra Heart
d1dce89fc9 fix: do not let deleted fname events allow old fname events to be re-added (#2203)
## Why is this change needed?

The fname events are not coming through correctly for a few names, and
by consequence we're seeing a few old fname events being endlessly
replayed. The deletion of the fname record (when fid = 0 in proof)
completely removes the record rather than its association with a fid,
and by consequence replaying the older event re-adds the fname. This
resolves the hub side of the behavior by correctly holding on to the new
fname event, but deleting the association with a fid.

## Merge Checklist

_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [x] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.

<!-- start pr-codex -->

---

## PR-Codex overview
The focus of this PR is to enhance the `hubble` app by fixing issues
related to username proof handling and syncing historical events.

### Detailed summary
- Fixed issue where deleted fname events allowed re-adding old fname
events
- Improved syncing historical events in `syncEngine.ts`
- Updated `mergeUserNameProof` functionality in tests
- Added `rsGetUserNameProof` import in `nameRegistryEvent.ts`
- Enhanced `UserDataStore` functionality in `user_data_store.rs`

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
2024-07-24 14:48:28 -07:00
Sanjay
a5f867fbcb fix: Fix incorrect target fid check for link messages older than the compact state (#2207)
## Why is this change needed?

We were checking for the message creator's fid instead of the target fid
and therefore rejecting valid messages older than the compact state

## Merge Checklist

_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.


<!-- start pr-codex -->

---

## PR-Codex overview
This PR fixes an issue with target fid check for link messages older
than the compact state in the `store.rs` file and adds tests in
`linkStoreCompactState.test.ts`.

### Detailed summary
- Fixed incorrect target fid check for link messages in `store.rs`
- Added tests for merging link add messages in
`linkStoreCompactState.test.ts`

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
2024-07-24 14:35:37 -07:00
Sanjay
554d82a45b [fix] Fix race condition with earliest ts hash calculation (#2204)
## Why is this change needed?

It was possible to trigger race conditions when calculating
earliest_ts_hash, this led to hubs have different values which meant
that stores would become out of sync. Migrate the cache to rust and
prevent race conditions.


## Merge Checklist

_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.

<!-- start pr-codex -->

---

## PR-Codex overview
This PR migrates the storage cache to Rust and addresses a race
condition.

### Detailed summary
- Migrated storage cache to Rust for efficiency
- Fixed a race condition in the storage cache
- Added `PRIMARY_KEY_LENGTH` constant in `message.rs`
- Updated `storage_cache` exports in `store/mod.rs`
- Added functions to interact with storage cache in `utils.rs`
- Updated `lib.rs` to include `StorageCache`
- Updated `storeEventHandler.ts` to handle `StorageCache`
- Updated tests to use `StorageCache` in `storeEventHandler.test.ts`
- Implemented `StorageCache` in `store_event_handler.rs`
- Added Rust bindings for `StorageCache` in `rustfunctions.ts`

> The following files were skipped due to too many changes:
`apps/hubble/src/storage/stores/storageCache.ts`,
`apps/hubble/src/storage/stores/storageCache.test.ts`,
`apps/hubble/src/addon/src/store/storage_cache.rs`

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
2024-07-23 20:50:30 -07:00
Cassandra Heart
be54d20352 fix: full state events should not include deletes (#2124)
## Why is this change needed?

The current processing of full state events results in an unbounded
number of deleted messages, making event consumption downstream
unpredictable and prone to getting stuck on singular large events.
Because merge events historically applied to deltas, and deltas were
generally small in effect size, this was a reasonable consideration, but
for full state (compact) messages, it is implicit on what is added
(everything in it) and what is not (everything not in it), making the
deleted messages something implicitly handled by downstream consumers.

## Merge Checklist

_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [x] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.

<!-- start pr-codex -->

---

## PR-Codex overview
This PR focuses on fixing the behavior of full state events in the
`store.rs` file and test cases related to compact state messages in the
`linkStoreCompactState.test.ts`.

### Detailed summary
- Fixed full state events to exclude deleted messages in `store.rs`
- Updated test cases to reflect changes in handling deleted messages for
compact state messages in `linkStoreCompactState.test.ts`

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
2024-07-19 12:36:25 -05:00
Wasif Iqbal
3746a4847a chore: release 1.13.8 (#2199)
## Why is this change needed?

- release 1.13.8

## Merge Checklist

_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.


<!-- start pr-codex -->

---

## PR-Codex overview
The focus of this PR is to update version numbers and dependencies
across various packages.

### Detailed summary
- Updated version to `0.14.19` in `@farcaster/core`
- Updated version to `0.11.21` in `@farcaster/hub-nodejs`
- Updated version to `0.5.7` in `@farcaster/shuttle`
- Updated version to `1.13.8` in `@farcaster/hubble`
- Added event id functions in `@farcaster/core`
- Fixed conflict criteria issue in `@farcaster/shuttle`
- Upgraded libp2p to `0.44.0` in `@farcaster/hubble`
- Increased hub storage requirement to `200 GB` in `@farcaster/hubble`

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
@farcaster/hubble@1.13.8 @farcaster/hub-nodejs@0.11.21 @farcaster/shuttle@0.5.7 @farcaster/core@0.14.19
2024-07-18 16:14:32 -05:00
Wasif Iqbal
ba79e46afc fix: ack script (#2198)
## Why is this change needed?

- fix: remove parentheses for prompt, which causes changes to occur in
sub shell and are not persisted to the environment file

## Merge Checklist

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
2024-07-18 20:43:27 +00:00
Cassandra Heart
ee0947ec4b fix: on conflict criteria is ambiguous and cannot be used for upsert (#2197)
## Why is this change needed?

Shuttle defines the messages table with two unique constraints, one for
the hash, another for the triple of hash, fid and type. Postgres does
not allow multiple ON CONFLICT criteria, and ambiguous conflicts are
essentially surfaced as uniqueness violation failures in spite of ON
CONFLICT criteria given. This change alters the merge to instead attempt
insert, doing nothing on any conflict, then resolving with an update if
necessary and the original conflict resolution criteria is satisfied.

Addresses Issue #2167 

## Merge Checklist

_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [x] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.


<!-- start pr-codex -->

---

## PR-Codex overview
This PR updates the conflict resolution logic in `MessageProcessor` to
fix ambiguity issues during upsert operations.

### Detailed summary
- Updated conflict resolution logic to use `doNothing()` method instead
of complex conditions
- Improved handling of conflict criteria for upsert operations

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
2024-07-18 12:36:56 -05:00
Varun Srinivasan
e2b1c7c681 chore: upgrade libp2p to 0.44.0 (#2191)
## Why is this change needed?

Upgrades libp2p one minor version up ([Release
Notes](https://github.com/libp2p/js-libp2p/releases/tag/v0.44.0) |
[Migration
Guide](https://github.com/libp2p/js-libp2p/blob/main/doc/migrations/v0.43-v0.44.md))

### Breaking Change in Autodial + Connection Manager Behavior

libp2p's ConnectionManager now has a default minConnection threshold of
50. If the number of active connections is < this number, it will
attempt to redial peers that are in the store.

Since we only disconnect peers and do not remove them from our store,
the naive upgrade causes us to reconnect immediately to peers
unexpectedly. We now set the threshold explicitly to 0 to prevent this.
We should consider also removing peers from the peer store in a future
upgrade.


## Merge Checklist

_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [ ] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.

<!-- start pr-codex -->

---

## PR-Codex overview
This PR upgrades `libp2p` to version 0.44.0, updates
`@chainsafe/libp2p-gossipsub` to 6.2.0, and introduces
`@libp2p/interface-connection-gater` version 2.0.1.

### Detailed summary
- Upgraded `libp2p` to 0.44.0
- Updated `@chainsafe/libp2p-gossipsub` to 6.2.0
- Added `@libp2p/interface-connection-gater` version 2.0.1

> The following files were skipped due to too many changes: `yarn.lock`

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
2024-07-18 02:42:53 +00:00
akira wu
c3ea869c18 fix(example): fix the argument for makeLinkRemove when removing a follow (#2196)
## Why is this change needed?

The change is a bug fix in the example code `write-data` in the
`hub-nodejs` package (packages/hub-nodejs/examples/write-data). When I
was trying to use this package, I was stuck with the original code for a
while and finally found it our by checking the source code. So I think
the change is needed because it can benefit others (especially new
developers) who want to build on Farcaster instead of getting stuck.

You can see here too that the test code for
`packages/core/src/builders.ts`
76ad1ac2a2/packages/core/src/builders.test.ts (L362)
is taking `'follow'` instead of `'unfollow'`

## Merge Checklist

_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [ ] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.

<!-- start pr-codex -->

---

## PR-Codex overview
The focus of this PR is to correct the type of message being created
when removing a follow in the `write-data` example.

### Detailed summary
- Updated the type of message from "unfollow" to "follow" when removing
a follow in the `write-data` example.

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
2024-07-17 17:22:57 -07:00
omar
db3b911715 chore: update github actions versions (#2190)
## Why is this change needed?

Describe why this issue should be fixed and link to any relevant design
docs, issues or other relevant items.

## Merge Checklist

_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [ ] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [ ] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.


<!-- start pr-codex -->

---

## PR-Codex overview
The focus of this PR is to update GitHub Actions and cache setup-node to
version 4 for better performance and reliability.

### Detailed summary
- Updated `actions/checkout` from v3 to v4 in
`.github/workflows/publish.yml`, `.github/workflows/release.yml`, and
`.github/workflows/release-packages.yml`
- Updated `actions/setup-node` from v3 to v4 in
`.github/workflows/release.yml` and
`.github/workflows/release-packages.yml`
- Updated `actions/cache` from v3 to v4 in
`.github/workflows/release.yml` and
`.github/workflows/release-packages.yml`

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
2024-07-17 10:24:23 -07:00
Sanjay
76ad1ac2a2 feat: Expose event id functions (#2195)
## Why is this change needed?

Make timestamp to event id converstion function publicly accessible

## Merge Checklist

_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.


<!-- start pr-codex -->

---

## PR-Codex overview
This PR adds new event ID functions and refactors timestamp handling in
the `core` and `hubble` packages.

### Detailed summary
- Added `extractEventTimestamp` and `makeEventId` functions in `core`
- Updated `console.ts` and `storeEventHandler.ts` to use the new
functions
- Refactored timestamp handling for event IDs

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
2024-07-17 02:09:34 +00:00
Wasif Iqbal
97d0a7ea3a fix: update hub storage requirement to 200 GB (#2194)
## Why is this change needed?

- update hub storage requirement to 200 GB

## Merge Checklist

_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.


<!-- start pr-codex -->

---

## PR-Codex overview
This PR updates the storage requirement for the Hubble app to 200 GB and
adjusts related documentation to reflect the change.

### Detailed summary
- Updated Hubble app storage requirement to 200 GB
- Adjusted disk size in GCP tutorial to 200 GB
- Updated storage requirement in installation guide to 200 GB

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
2024-07-16 15:47:23 -05:00
Wasif Iqbal
cf8c50c510 fix: force acknowledgement of no rewards for existing hubs (#2193)
## Why is this change needed?

Force acknowledgement of no rewards for hubs that are already running

## Merge Checklist

_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [ ] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.


<!-- start pr-codex -->

---

## PR-Codex overview
The focus of this PR is to add a prompt for hub operator agreement
before running the script and updating the environment file accordingly.

### Detailed summary
- Added a function `prompt_for_hub_operator_agreement` to prompt for
agreement
- Created a function `update_env_file` to update or create the
environment file based on agreement
- Improved user input handling and validation
- Added logic to check for existing agreement in the environment file
- Updated the script flow to handle agreement properly

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
2024-07-16 15:05:39 -05:00
Pavel Zaborskii
ee36f226d8 fix(docs): Improved GCP tutorial (#2175)
Fixes issue #2173 and some additional improvements
My Warpcast username: @pashu777

## Why is this change needed?

Users often encounter errors during the installation process due to
missing information in the documentation. This PR fixes several issues
in the Farcaster Hub installation guide for Google Cloud Platform (GCP).
The changes include adding missing steps, clarifying instructions, and
fixing grammatical errors to ensure a smooth setup experience for users.

### Change Summary

1. Enable Compute Engine API
   - Problem: After running the terraform apply command, users encounter
an error indicating that the Compute Engine API is disabled. This PR
includes a step to enable the API to prevent this error.
  
![GCP
Error](https://github.com/user-attachments/assets/28479156-fd80-4e90-929b-d606444cbadc)

- Solution: Added a step to enable the Compute Engine API before
proceeding with the installation. Included an illustrative screenshot
for guidance.

![gcp_compute_engine_api](https://github.com/user-attachments/assets/f393adcc-f74d-4a53-9fef-48f28e973e42)

2. GCP Configuration Clarity
   - Problem: When setting up the GCP virtual machine for the Farcaster
project, the configuration file ([main.tf](https://main.tf/)) includes
the placeholder $YOUR_PROJECT_ID. Users might mistakenly copy and paste
the configuration text without replacing $YOUR_PROJECT_ID with their
actual project ID, leading to errors.
   - Solution: Added text before the GCP virtual machine configuration
copy step to instruct users to replace $YOUR_PROJECT_ID with their
personal ID       

![gcp_project_id](https://github.com/user-attachments/assets/6afa67e1-4572-4da3-9e2a-f31b4edc6980)

3. Grammar Correction
   - Problem: Redundant text in the instructions: "Now follow the steps
as described on Install page page."
   - Solution: Removed the redundant word "page" for improved
readability.
![page page
](https://github.com/user-attachments/assets/782b1903-a031-4607-b40a-7261c552bcff)

## Merge Checklist

_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [ ] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [x] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.

<!-- start pr-codex -->

---

## PR-Codex overview
The focus of this PR is to update the main.tf file with instructions to
configure a GCP virtual machine and enable the Compute Engine API.

### Detailed summary
- Added configuration for GCP virtual machine with image references
- Instructed to replace "$YOUR_PROJECT_ID" with personal ID
- Provided steps to enable Compute Engine API
- Updated instructions for installing Docker
- Clarified steps to follow for setting up Hubble

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
2024-07-15 17:42:06 -07:00
Wasif Iqbal
9ad23d5074 chore: release 1.13.7 (#2188)
## Why is this change needed?

- hub 1.13.7
- core 0.14.18
- hub-nodejs 0.11.20

## Merge Checklist

_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.

<!-- start pr-codex -->

---

## PR-Codex overview
This PR increases version numbers for various packages and dependencies.
It also includes changes related to extending frame message URL max
length and assigning shards using a hashing algorithm.

### Detailed summary
- Increased version numbers for packages and dependencies
- Extended frame message URL max length
- Assigned shards using jump consistent hashing algorithm

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
@farcaster/hubble@1.13.7 @farcaster/hub-nodejs@0.11.20 @farcaster/core@0.14.18
2024-07-15 16:14:43 -05:00
Shane da Silva
6bf952601f chore(shuttle): Release 0.5.6 (#2186)
## Why is this change needed?

Includes some performance improvements.

## Merge Checklist

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.


<!-- start pr-codex -->

---

## PR-Codex overview
This PR updates the version of `@farcaster/shuttle` to `0.5.6` and
includes improvements related to stream trimming and recording size.

### Detailed summary
- Updated version to `0.5.6`
- Trim stream at regular interval
- Stop recording stream size
- Updated `@farcaster/hub-nodejs` to `0.11.20`

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
@farcaster/shuttle@0.5.6
2024-07-15 19:39:48 +00:00
Shane da Silva
e5760da0d3 perf(shuttle): Trim stream at regular intervals instead of every iteration (#2185)
## Why is this change needed?

Performance profiling indicates that we're spending a non-trivial amount
of time (5-10% of wall clock time) trimming the stream. This doesn't
need to happen as frequently as it's currently happening, so switch to
an interval timer so that we're not slowing down general processing in
any meaningful way.

## Merge Checklist

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.


<!-- start pr-codex -->

---

## PR-Codex overview
This PR optimizes the event stream processing in
`HubEventStreamConsumer` by adding a regular interval to trim old events
efficiently.

### Detailed summary
- Added `interval` property to schedule event processing at regular
intervals
- Implemented interval logic to clear old events and process stale
events
- Updated `stop` method to clear the interval
- Simplified event processing logic in the message loop

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
2024-07-15 19:28:13 +00:00
Wasif Iqbal
e0cb238cd0 Revert "fix: force acknowledgement of no rewards for existing hubs" (#2184)
Reverts farcasterxyz/hub-monorepo#2154

<!-- start pr-codex -->

---

## PR-Codex overview
The focus of this PR is to refactor the
`prompt_for_hub_operator_agreement` function in `hubble.sh` for better
user interaction.

### Detailed summary
- Refactored the `prompt_for_hub_operator_agreement` function for
improved user experience
- Removed unnecessary file operations and simplified the prompt logic
- Moved the prompt logic outside the function for better readability and
maintenance

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
2024-07-15 14:09:08 -05:00
Shane da Silva
fc67ccf043 chore(shuttle): Stop recording stream size (#2183)
## Why is this change needed?

Looking at profiling data, this actually takes up a non-trivial amount
of wall clock time (10% of total time). Since the stream is
automatically trimmed, recording the size isn't particularly useful
except as perhaps an early warning on memory, but there are generally
other metrics one could use (number of events processed, etc.) to infer
that there is a slowdown in processing.

## Merge Checklist

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.


<!-- start pr-codex -->

---

## PR-Codex overview
This PR stops recording stream size in the `eventStream.ts` file of the
`@farcaster/shuttle` package.

### Detailed summary
- Removed recording stream size in eventStream.ts
- Removed statsd gauge and timing metrics for stream size and time
calculations

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
2024-07-15 18:56:35 +00:00
Shane da Silva
63f29f1aca chore(shuttle): Release 0.5.5 (#2179)
## Why is this change needed?

To ensure this remains efficient at higher loads, perform before/after
hook processing in batches.

## Merge Checklist

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [ ] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.


<!-- start pr-codex -->

---

## PR-Codex overview
This PR updates the version of `@farcaster/shuttle` to `0.5.5`. It
includes efficiency improvements in before/after hooks and updates to
`@farcaster/hub-nodejs`.

### Detailed summary
- Updated version to `0.5.5`
- Efficiency improvements in before/after hooks
- Updated dependency `@farcaster/hub-nodejs` to `0.11.20`

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
@farcaster/shuttle@0.5.5
2024-07-12 22:34:34 -07:00
Shane da Silva
27e3c7ae0c fix(shuttle): Switch before/after processing hooks to use batches (#2178)
## Why is this change needed?

This will be much more efficient at high loads.

## Merge Checklist

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.

<!-- start pr-codex -->

---

## PR-Codex overview
This PR optimizes event processing efficiency in `@farcaster/shuttle` by
switching before/after hooks to work with batches.

### Detailed summary
- Updated `PreProcessHandler` and `PostProcessHandler` signatures to
work with batches
- Refactored event processing logic to handle batches efficiently
- Improved event processing performance by processing events in batches

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
2024-07-12 22:30:37 -07:00
Shane da Silva
b09009f508 chore: Release 0.5.4 (#2177)
## Why is this change needed?

We want to allow users to add pre-post processing logic when processing
hub events using Shuttle.

## Merge Checklist

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [ ] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.


<!-- start pr-codex -->

---

## PR-Codex overview
This PR updates the version of the `@farcaster/shuttle` package to
`0.5.4` and adds support for running before/after hooks when processing
events.

### Detailed summary
- Updated `@farcaster/shuttle` package version to `0.5.4`
- Added support for running before/after hooks when processing events

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
@farcaster/shuttle@0.5.4
2024-07-13 04:01:34 +00:00
Shane da Silva
41f32cd0fa feat(shuttle): Support running before/after hooks when processing events (#2176)
## Why is this change needed?

We want the ability to provide custom pre/post-processing logic when
processing events. Expose optional hooks that can be specified. This has
the added benefit that the `beforeProcess` hook can be used to skip
processing of an event.

## Merge Checklist

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [ ] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [x] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.


<!-- start pr-codex -->

---

## PR-Codex overview
This PR enhances the `@farcaster/shuttle` package by adding support for
running before/after hooks when processing events.

### Detailed summary
- Added `beforeProcess` and `afterProcess` handlers to
`EventStreamHubSubscriber` and `HubEventStreamConsumer`
- Implemented handling of preprocess and postprocess actions for event
processing
- Introduced `EventStreamHubSubscriberOptions` and
`EventStreamConsumerOptions` interfaces
- Updated `HubEventStreamConsumer` to extend `TypedEmitter` for custom
event tracking

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
2024-07-12 18:28:26 -07:00
horsefacts
fb0a083aef feat: increase frame message URL max length (#2174)
## Why is this change needed?

Describe why this issue should be fixed and link to any relevant design
docs, issues or other relevant items.

## Merge Checklist

_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.


<!-- start pr-codex -->

---

## PR-Codex overview
This PR extends the maximum length of the frame message URL to 1024
characters for better validation accuracy.

### Detailed summary
- Increased maximum URL length validation from 256 to 1024 characters
- Updated test case with longer URL length for validation testing

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
2024-07-12 18:26:42 -04:00
Shane da Silva
312340d210 feat: Assign shards using jump consistent hashing (#2171)
## Why is this change needed?

While modulo-based arithmetic for assigning shards is simple and works
fine, it does not work well when the number of shards changes.

For example, when increasing the number of shards from 3 → 4 using
modulo-based shard assignment, ~83% of FIDs will be assigned to a
different shard. With consistent hashing, only ~25% will be assigned to
a different shard.

Jump consistent hashing is a particular approach to consistent hashing
that is stateless (requires no additional tracking of shard
assignments), which is ideal for hubs, since we don't want hubs to have
to manage per-client tracking of shards.

## Merge Checklist

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [x] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.

<!-- start pr-codex -->

---

## PR-Codex overview
This PR focuses on implementing the jump consistent hashing algorithm
for shard assignment in the Hubble app.

### Detailed summary
- Implemented `jumpConsistentHash` algorithm for shard assignment based
on FID
- Updated shard assignment logic in `Server` class
- Updated tests in `eventService.test.ts` for shard-based event handling

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
2024-07-12 12:24:41 -07:00
Varun Srinivasan
9e699617f7 Revert "chore: upgrade turborepo to v2" (#2170)
Turborepo 2 is causing sporadic failures when running lint:ci for
reasons that I don't quite understand. Since this isn't essential, I'm
reverting the change.

Reverts farcasterxyz/hub-monorepo#2156

<!-- start pr-codex -->

---

## PR-Codex overview
This PR updates the `turbo` package version and refactors the
`turbo.json` configuration file structure.

### Detailed summary
- Updated `turbo` package version from `2.0.6` to `1.10.3`
- Refactored `turbo.json` configuration file structure
- Updated dependencies in `yarn.lock`

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
2024-07-12 11:53:53 -07:00
Varun Srinivasan
3cf7a148c5 chore: upgrade rimraf to 6 (#2163)
## Why is this change needed?

Upgraded rimraf to the latest version. 

This requires Node 22 - while I tried to upgrade it in a previous PR I
forgot do to it for the build step of the Dockerfile, which is now fixed
here.

## Merge Checklist

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [ ] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.

<!-- start pr-codex -->

---

## PR-Codex overview
This PR updates dependencies and versions in the project, focusing on
Jest, Rimraf, Minimatch, Minipass, Jackspeak, LRU-cache, and
Path-scurry.

### Detailed summary
- Updated Jest to version 29.6.1
- Updated Rimraf to version 6.0.1
- Updated Minimatch to version 10.0.1
- Updated Minipass to version 7.1.2
- Updated Jackspeak to version 4.0.1
- Updated LRU-cache to version 11.0.0
- Updated Path-scurry to version 2.0.0

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
2024-07-12 11:06:18 -07:00
Shane da Silva
eeb1504e20 fix(shuttle): Fix broken build for 0.5.2 (#2169)
## Why is this change needed?

We didn't build the package correctly.

## Merge Checklist

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [ ] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.


<!-- start pr-codex -->

---

## PR-Codex overview
The focus of this PR is to update the version of the
`@farcaster/shuttle` package from 0.5.2 to 0.5.3.

### Detailed summary
- Updated package version to 0.5.3 in `package.json`
- Added entry for version 0.5.3 in `CHANGELOG.md`

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
@farcaster/shuttle@0.5.3
2024-07-12 17:35:38 +00:00
Shane da Silva
3ac4e39c71 chore(shuttle): Release 0.5.2 (#2168)
## Why is this change needed?

Includeds some minor fixes that others are looking to use.

## Merge Checklist

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.


<!-- start pr-codex -->

---

## PR-Codex overview
This PR updates the version of `@farcaster/shuttle` package to `0.5.2`
and includes enhancements for message processing and PostgreSQL schema
customization.

### Detailed summary
- Updated `@farcaster/shuttle` package version to `0.5.2`
- Added feature to handle removals for compact state explicitly and warn
on mismatching deletes
- Added feature to enable shuttle and its example-app to use a
PostgreSQL schema other than "public"

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
@farcaster/shuttle@0.5.2
2024-07-12 17:11:21 +00:00
Varun Srinivasan
9d0f089746 chore: upgrade dockerfile to use node 22 (#2165)
## Why is this change needed?

Node 21 is not an LTS build and was used because we had issues with Node
20 on ARM.

Changing to using Node 22 will allow us to use only LTS builds, which
also solves issues with some dependencies like rimraf 6 explicitly
disallowing node 21.

## Merge Checklist

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [ ] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.

<!-- start pr-codex -->

---

## PR-Codex overview
This PR updates the Node.js version in the Dockerfile and GitHub Actions
workflow to 22.4.1, ensuring compatibility and possibly leveraging new
features.

### Detailed summary
- Updated Node.js version in Dockerfile from 21.6 to 22.4.1
- Updated Node.js version in GitHub Actions workflow from 21 to 22

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
2024-07-12 09:34:15 -07:00
Varun Srinivasan
5765fec186 chore: fix unhandled type error in tests (#2161)
## Why is this change needed?

Fixing this type error that started showing up in my local environment.
Unclear why it started triggering now, but the fix is to just cast it
and do the comparison.

There's a small edge case here that if the types were not strict
Uint8Arrays the tests might still pass. But that doesn't seem to be the
spirit of what we're testing here.

## Merge Checklist

_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [ ] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [ ] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.


<!-- start pr-codex -->

---

## PR-Codex overview
This PR updates a test file in `messageDataBytes.test.ts` to fix a
comparison issue in a test case related to decoding message data bytes.

### Detailed summary
- Fixed comparison issue in a test case by ensuring both sides are of
type Uint8Array.

>  Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
2024-07-11 21:09:49 -07:00