feat: Q4 commitments (#283)

This commit is contained in:
richΛrd
2025-10-07 08:39:30 -04:00
committed by GitHub
parent d6f863192d
commit 2db8a7c95b
8 changed files with 598 additions and 13 deletions

View File

@@ -0,0 +1,105 @@
---
title: nim-libp2p c-bindings
tags:
- "2025q4"
- "p2p"
- "ift"
draft: false
description: Provide c-bindings for nim-libp2p
---
`vac:p2p:ift:2025q4-nimlibp2p-cbindings`
Expose nim-libp2p functionality via a c static library, enabling external projects (e.g. logoscore) to integrate without depending on nim toolchains. Implementation may leverage [nim-ffi](https://github.com/waku-org/nim-ffi) and/or [nim-library-template](https://github.com/logos-co/nim-library-template/) as well as previous learnings from building libwaku to simplify binding generation and library packaging.
## Description
Deliver a minimal set of c-bindings around nim-libp2p core. The bindings should cover basic node lifecycle, peer connectivity, and stream i/o. This enables downstream projects like logoscore to embed libp2p functionality directly. Stretch goals include exposing Gossipsub and custom protocol support, which may be deferred to 2026q1.
## Task List
### Core Bindings
* fully qualified name: `vac:p2p:ift:2025q4-nimlibp2p-cbindings:core`
* owner:
* status: not started
* start-date:
* end-date:
#### description
Implement and test c-bindings for core nim-libp2p functionality. Exported functions must include:
- switch builder
- start/stop node
- access node peer's information (addresses, peerId)
- connect/disconnect peer
- dial peer on protocol
- read/write stream
- close stream
#### Deliverables
- Static c library build artifact
- Header file exposing ffi-safe api
- Integration tests demonstrating basic usage
- Documentation
---
### kademlia
* fully qualified name: `vac:p2p:ift:2025q4-nimlibp2p-cbindings:kademlia`
* owner:
* status:
* start-date:
* end-date:
#### description
Extend c-bindings to expose Kademlia functionality. This task depends on `vac:p2p:ift:2025q4-nimlibp2p-kad-dht`.
Exported functions must include:
- Bootstrap
- Find Node / Put Value / Get Value / Set Providers / Get Providers
- Discovery
#### deliverables
- Updated c header + library with kademlia api
- Tests verifying kademlia behavior via ffi
- Documentation
---
### gossipsub
* fully qualified name: `vac:p2p:ift:2025q4-nimlibp2p-cbindings:gossipsub`
* owner:
* status:
* start-date:
* end-date:
#### description
Extend c-bindings to expose Gossipsub Functionality. Exported functions must include:
- start/stop gossipsub
- publish/subscribe
- add/remove validators
#### deliverables
- Updated c header + library with gossipsub api
- Tests verifying pub/sub behavior via ffi
- Documentation
### Custom Protocols
* fully qualified name: `vac:p2p:ift:2025q4-nimlibp2p-cbindings:custom-protocols`
* owner:
* status:
* start-date:
* end-date:
#### Description
Provide FFI to register and handle custom protocols from c, enabling downstream applications to define custom behaviors.
#### Deliverables
- FFI API for custom protocol registration
- Example implementation + tests
- Documentation

View File

@@ -0,0 +1,69 @@
---
title: nim-libp2p IPv6 support
tags:
- "2025q4"
- "p2p"
- "ift"
draft: false
description: add first-class IPv6 support across transports
---
`vac:p2p:ift:2025q4-nimlibp2p-ipv6`
Enable IPv6 support
## Description
Enable IPv6 in nim-libp2p end-to-end so users of the library can run dual-stack when needed. Scope covers transports (tcp/quic), address handling (multiaddr, identify), and ci/test infra.
## Task List
### Core transports (tcp/quic)
* fully qualified name: `vac:p2p:ift:2025q4-nimlibp2p-ipv6:transport`
* owner:
* status: not started
* start-date:
* end-date:
#### Description
Add IPv6 listen/dial for tcp and quic. Might require changes in chronos
#### Deliverables
- nodes can listen on /ip6 and accept inbound requests
- dialer can dial /ip6 addresses
### addresses & identity
* fully qualified name: `vac:p2p:ift:2025q4-nimlibp2p-ipv6:addresses`
* owner:
* status: not started
* start-date:
* end-date:
#### Description
Add IPv6 support in the switch when specifying listening multiaddresses containing ip6 component.
Ensure ip6 addresses are advertised in identify.
Implement measures to handle duplicated IPs in peerstore/addresses being listened to (i.e 192.0.2.42 vs 2001:db8::1234).
#### Deliverables
- Identify advertises correct /ip6 addrs; observed addrs tracked per transport
### Testing, Interop & CI
* fully qualified name: `vac:p2p:ift:2025q4-nimlibp2p-ipv6:testing`
* owner:
* status: not started
* start-date:
* end-date:
#### Description
Ensure IPv6 support and ensure there are no regressions in IPv4 support. Add test units demonstrating the support, and check that interop against other implementations is achieved
#### Deliverables
- Test units focusing on IPv6
- Interop tests against other implementations that support IPv6

View File

@@ -0,0 +1,104 @@
---
title: Kademlia DHT
tags:
- "2025q4"
- "p2p"
- "ift"
draft: false
description: Implement Kademlia DHT
---
`vac:p2p:ift:2025q4-nimlibp2p-kad-dht`
Implement Kademlia Distributed Hash Table in nim-libp2p, based on the [libp2p specification](https://github.com/libp2p/specs/blob/master/kad-dht/) as an additional discovery mechanism.
This commitment involves two main objectives: implementing nim-kademlia-dht, integrating this in nim-libp2p and making sure there is interoperability with other libp2p implementations.
This deliverable continues the work started in 2025q2
## Task List
### Routing table and RPC Protocol
* fully qualified name: `vac:p2p:ift:2025q2-nimlibp2p-kad-dht:routing-and-rpc`
* owner:
* status: 50%
* start-date: 2025-04-08
* end-date:
#### Description
Implement Kademlias XOR-based routing table with bucket management. Define RPC messages (`PING`, `SET_VALUE`, `GET_VALUE`, `SET_PROVIDERS`, `GET_PROVIDERS`, `FIND_NODE`) for node communication.
#### Deliverables
### Bootstrapping & Lookup Mechanism
* fully qualified name: `vac:p2p:ift:2025q4-nimlibp2p-kad-dht:bootstraping-and-lookup`
* owner:
* status: 35%
* start-date: 2025-05-05
* end-date:
#### Description
Enable nodes to join the network via bootstrap peers. Implement iterative lookup for efficient key
discovery and retrieval
#### Deliverables
### Network Maintenance & Node Management
* fully qualified name: `vac:p2p:ift:2025q4-nimlibp2p-kad-dht:network-maintenance-node-management`
* owner:
* status: not started
* start-date:
* end-date:
#### Description
Handle bucket refresh, node eviction, and periodic pings to maintain an active routing table.
Implement bucket splitting and adaptive refresh strategies.
#### Deliverables
### Data Storage & Replication
* fully qualified name: `vac:p2p:ift:2025q4-nimlibp2p-kad-dht:data-storage`
* owner:
* status: started
* start-date: 2025-07-28
* end-date:
#### Description
Implement the logic for `SET_VALUE` and `SET_PROVIDERS`. Store key-value pairs with expiration and replication mechanisms.
Ensure values persist despite churn, using periodic re-publication and redundancy strategies
#### Deliverables
### Data Retrieval
* fully qualified name: `vac:p2p:ift:2025q4-nimlibp2p-kad-dht:data-retrieval`
* owner:
* status: not started
* start-date: 2025-07-28
* end-date:
#### Description
Implement the logic for `GET_VALUE` and `GET_PROVIDERS`. Implement validation and quorum mechanisms for queries
#### Deliverables
### Discovery
* fully qualified name: `vac:p2p:ift:2025q4-nimlibp2p-kad-dht:discovery`
* owner:
* status: 15%
* start-date: 2025-07-20
* end-date:
#### Description
Extend nim-libp2p discovery mechanism by providing a discovery interface that will use Kademlia DHT to advertise nodes and find peers

View File

@@ -0,0 +1,69 @@
---
title: Nim-libp2p maintenance
tags:
- "2025q4"
- "p2p"
- "ift"
draft: false
description: Maintain nim-libp2p through improvements, bug fixes, and user support
---
`vac:p2p:ift:2025q4-nimlibp2p-maintenance`
Maintain nim-libp2p through improvements, bug fixes, and user support
## Description
Continue supporting and maintaining the nim-libp2p repository through ongoing improvements, refactoring, and bug fixes.
This includes a range of ad-hoc tasks critical to sustaining code quality, overall stability, correct functionality,
and performance of the project.
Additionally, it provides a platform for addressing small developer requests,
ensuring that nim-libp2p remains functional and usable for its primary users — Nimbus and Waku — helping to meet their evolving needs."
## Task List
### Maintenance
* fully qualified name: `vac:p2p:ift:2025q4-nimlibp2p-maintenance:maintenance`
* owner: rramos
* status: not started
* start-date: 2025/10/01
* end-date: 2025/12/31
#### Description
Maintain [nim-libp2p](https://github.com/vacp2p/nim-libp2p) repository via improvements, refactors and bug fixes.
#### Deliverables
- [[Deliverable] Maintenance 2025Q4](TODO)
### Gossipsub Interop tests
* fully qualified name: `vac:p2p:ift:2025q4-nimlibp2p-maintenance:gossipsub-interop-tests`
* owner:
* status: not started
* start-date: 2025/10/01
* end-date: 2025/12/31
#### Description
Integrate libp2p test plans' [GossipSub interop tests](https://github.com/libp2p/test-plans/tree/master/gossipsub-interop) as part of the CI tests executed on each PR
#### Deliverables
- New github workflow: `Interoperability Tests / Run GossipSub interoperability tests`
### Fix flaky tests
* fully qualified name: `vac:p2p:ift:2025q4-nimlibp2p-maintenance:flaky-tests`
* owner:
* status: not started
* start-date: 2025/10/01
* end-date: 2025/12/31
#### Description
Continue to systematically track flaky tests using CI logs and failure reports,
and fix root causes where reproducible (e.g. race conditions, timeouts, dependency order)
#### Deliverables

View File

@@ -0,0 +1,73 @@
---
title: Nim-libp2p Mix
tags:
- "2025q4"
- "p2p"
- "ift"
draft: false
description: upstream specs for mix and reuse streams
---
`vac:p2p:ift:2025q4-nimlibp2p-mix`
Upstream spec for mix protocol and add stream reuse to reduce latency
## Description
The mix specification currently resides in [vacp2p#rfc-index](https://github.com/vacp2p/rfc-index/blob/main/vac/raw/mix.md). The goal is to upstream it into [libp2p#specs](https://github.com/libp2p/specs), ensuring adoption across libp2p implementations. Additionally, in late Q3 we identified an improvement: rather than opening and closing a new stream for each mix packet, streams can be reused, reducing lifecycle overhead and lowering end-to-end latency. Finally we found out that there's a possible metadata leakage when multiple Sphinx packets are broadcasted due to gragmentation of a large message by an upstram protocol sent in a burst.
## Task List
### Specs
* fully qualified name: `vac:p2p:ift:2025q4-nimlibp2p-mix:specs`
* owner:
* status: not started
* start-date:
* end-date:
#### Description
Collaborate with ACZ team into upstreaming initial mix design and rationale to
`libp2p/spec` as proposal or draft PR. This task was originally part of Q3 but moved to
Q4 as specs update was not completed
#### Deliverables
- Draft PR merged into `libp2p/specs`
### Stream Reuse
* fully qualified name: `vac:p2p:ift:2025q4-nimlibp2p-mix:stream-reuse`
* owner:
* status: not started
* start-date:
* end-date:
#### Description
Implement long-lived stream reuse between mix nodes, by establishing a persistent stream between mix peers.
Maintain a queue of outgoing Sphinx packets per peer, and dispatch them over the persistent stream. [vacp2p/mix#88](https://github.com/vacp2p/mix/issues/88)
#### Deliverables
- Persistent streams established between peers
- Packet queue mechanism implemented
- Test units
### Rate limit publishing
* fully qualified name: `vac:p2p:ift:2025q4-nimlibp2p-mix:rate-limit`
* owner:
* status: not started
* start-date:
* end-date:
#### Description
Introduce a randomized delay before the initiator sends each Sphinx packet to the first hop. This is important to avoid metadata leakage when multiple Sphinx packets (e.g., due to fragmentation of a large message by an upstream protocol) are sent in a burst.
#### Deliverables
- Sender should not be able to send a burst of mix messages but instead have them rate limited with a random delay similar to the one used for hops.

View File

@@ -0,0 +1,49 @@
---
title: Quic Stabilization and removal of runtime dependencies
tags:
- "2025q4"
- "p2p"
- "ift"
draft: false
description: Stabilize Quic so it can be used effectively by Nimbus and Waku and remove OpenSSL Support
---
`vac:p2p:ift:2025q4-nimlibp2p-quic`
Maintain nim-libp2p through improvements, bug fixes, and user support
## Description
In Q3 we discovered critical deficiencies in nim-quic, including runtime instability and excessive memory use during testing. As a result, we cannot safely deploy quic in production. in parallel, we identified OpenSSL as a long-term liability and will migrate to AWS-LC or Boringssl to reduce dependencies and improve maintainability.
## Task List
### Quic Stabilization
* fully qualified name: `vac:p2p:ift:2025q4-nimlibp2p-quic:stabilization`
* owner: rramos
* status: not started
* start-date: 2025/10/01
* end-date:
#### Description
Resolve runtime crashes, race conditions and excessive memory use in nim-quic. Target outcome is that quic can be enabled and run stably in both nimbus and waku test environments.
#### Deliverables
### Replace OpenSSL by AWS-LC / BoringSSL
* fully qualified name: `vac:p2p:ift:2025q4-nimlibp2p-quic:openssl-replacement`
* owner:
* status: not started
* start-date: 2025/10/01
* end-date:
#### Description
Replace the openssl dependency in nim-quic and related libraries with BoringSSL or AWS-LC. The goal is to reduce the need of runtime dependencies by statically linking any of these libraries
#### Deliverables

View File

@@ -0,0 +1,125 @@
---
title: nim-libp2p Webtransport
tags:
- "2025q4"
- "ift"
draft: false
description: Add webtransport support to nim-libp2p
---
`vac:p2p:ift:2025q4-nimlibp2p-webtransport`
Add webtransport support to nim-libp2p
## Description
Extend the number of transports supported by nim-libp2p via the developent of Webtransport,
which would allow browsers to establish bidirectional stream connections to nim-libp2p servers
This deliverable continues the work started in 2025q2
## Task List
### Wrapping nghttp3
* fully qualified name: `vac:p2p:ift:2025q4-nimlibp2p-webtransport:nghttp3-wrapping`
* owner: vlado
* status: not started
* start-date:
* end-date:
#### Description
Create wrapper around the [nghttp3](https://github.com/ngtcp2/nghttp3) C library for Nim.
#### Deliverables
- add nghttp3 wrapping with futhark [nim-nghttp3 #1](https://github.com/vacp2p/nim-nghttp3/pull/1)
- ci: add ci for linux and mac [nim-nghttp3 #2](https://github.com/vacp2p/nim-nghttp3/pull/2)
### HTTP/3 Client
* fully qualified name: `vac:p2p:ift:2025q4-nimlibp2p-webtransport:http3-client`
* owner: vlado
* status: not started
* start-date:
* end-date:
#### Description
Implement HTTP/3 Client following [draft-ietf-webtrans-http3-02](https://www.ietf.org/archive/id/draft-ietf-webtrans-http3-02.html) specification.
This should be done as part of the [nim-quic](https://github.com/vacp2p/nim-quic) module.
#### Deliverables
### HTTP/3 Server
* fully qualified name: `vac:p2p:ift:2025q4-nimlibp2p-webtransport:http3-server`
* owner: vlado
* status: not started
* start-date:
* end-date:
#### Description
Implement HTTP/3 Server following [draft-ietf-webtrans-http3-02](https://www.ietf.org/archive/id/draft-ietf-webtrans-http3-02.html) specification.
This should be done as part of the [nim-quic](https://github.com/vacp2p/nim-quic) module.
#### Deliverables
### HTTP/3 Maintenance
* fully qualified name: `vac:p2p:ift:2025q4-nimlibp2p-webtransport:http3-maintenance`
* owner: vlado
* status: not started
* start-date:
* end-date:
#### Description
Final improvements, code polishing and interoperability testing after Client and Server are merged.
#### Deliverables
### Certificate generation
* fully qualified name: `vac:p2p:ift:2025q4-nimlibp2p-webtransport:certificate`
* owner: vlado
* status: not started
* start-date:
* end-date:
#### Description
Certificate generation according to libp2p webtransport [specification](https://github.com/libp2p/specs/tree/master/webtransport)
#### Deliverables
### Key components implementation
* fully qualified name: `vac:p2p:ift:2025q4-nimlibp2p-webtransport:key-components`
* owner: vlado
* status: not started
* start-date:
* end-date:
#### Description
Implement webtransport key components: Client, Server, Session
#### Deliverables
### Noise handshake and certificate validation
* fully qualified name: `vac:p2p:ift:2025q4-nimlibp2p-webtransport:handshake-and-cert-validation`
* owner: vlado
* status: not started
* start-date:
* end-date:
#### Description
1.3. Noise handshake and certificate validtion according to libp2p webtransport [specification](https://github.com/libp2p/specs/tree/master/webtransport)
#### Deliverables

View File

@@ -8,31 +8,20 @@ tags:
## Roadmap Preview
### 2025Q4
- WebTransport (finalized)
- Complete implementation of WebTransport protocol
- Support WebTransport as a transport option within nim-libp2p
- Kad-DHT
- May continue depending on remaining gaps contingent on prior quarters outcomes.
- Cross-compilation for mobile
- Enable nim-libp2p builds for arm64/aarch64 targeting Android/iOS
- Needed for integration into Status App.
- C-bindings
- Expose stable C API surface for LogosCore and other external consumers.
- Connection abstraction refactor
- The current implementation of nim-libp2p includes abstractions that are difficult to understand,
which makes development and collaboration more challenging.
Refactoring the codebase would improve clarity, allowing contributors to grasp the code more easily.
Additionally, a more streamlined implementation would enhance maintainability,
leading to more efficient troubleshooting and debugging.
- IPv6 support
- Full handling of /ip6/... multiaddresses
- Mix-core in nim-libp2p as Done state
- core protocol logic stable;
- no blocking issues
- Issues (if any) are around API polish
- Maintenance
- evaluate whether to keep using nim-protobuf-serialization or migrate to minprotobuf
- (This is in addition to other maintenance tasks).
### 2026Q1 and beyond
- Unix Domain Socket transport (`/unix//tmp/foo`)
@@ -72,4 +61,6 @@ The following items are not part of our current roadmap. They haven't been reque
- wasm support
- compile nim-libp2p to a wasm library for embedding in browser environments.
- no spec, but precedent exists in js-libp2p and rust efforts.
- native nim impl of Quic
- native nim impl of Quic
- evaluate whether to keep using nim-protobuf-serialization or migrate to minprotobuf
- (This is in addition to other maintenance tasks).