AtHeartEngineer/rfc-index
1
0
Fork 0
mirror of https://github.com/vacp2p/rfc-index.git synced 2026-01-10 08:08:17 -05:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'main' into codex/raw/codex-prover-raw

Browse Source
This commit is contained in:
Cofson
2025-10-24 12:06:56 +02:00
committed by GitHub
parent f3e5928644 e4f5f28ea3
commit edd5efcace
2 changed files with 372 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

183
vac/raw/sds.md
View File

@@ -58,6 +58,9 @@ other participants using the corresponding message ID.
* **Participant ID:**
Each participant has a globally unique, immutable ID
visible to other participants in the communication.
* **Sender ID:**
The *Participant ID* of the original sender of a message,
often coupled with a *Message ID*.
## Wire protocol
@@ -75,6 +78,8 @@ syntax = "proto3";
message HistoryEntry {
string message_id = 1; // Unique identifier of the SDS message, as defined in `Message`
optional bytes retrieval_hint = 2; // Optional information to help remote parties retrieve this SDS message; For example, A Waku deterministic message hash or routing payload hash
optional string sender_id = 3; // Participant ID of original message sender. Only populated if using optional SDS Repair extension
}
message Message {
@@ -84,6 +89,9 @@ message Message {
optional uint64 lamport_timestamp = 10; // Logical timestamp for causal ordering in channel
repeated HistoryEntry causal_history = 11; // List of preceding message IDs that this message causally depends on. Generally 2 or 3 message IDs are included.
optional bytes bloom_filter = 12; // Bloom filter representing received message IDs in channel
repeated HistoryEntry repair_request = 13; // Capped list of history entries missing from sender's causal history. Only populated if using the optional SDS Repair extension.
optional bytes content = 20; // Actual content of the message
}
```
@@ -289,6 +297,181 @@ Upon reception,
ephemeral messages SHOULD be delivered immediately without buffering for causal dependencies
or including in the local log.
### SDS Repair (SDS-R)
SDS Repair (SDS-R) is an optional extension module for SDS,
allowing participants in a communication to collectively repair any gaps in causal history (missing messages)
preferably over a limited time window.
Since SDS-R acts as coordinated rebroadcasting of missing messages,
which involves all participants of the communication,
it is most appropriate in a limited use case for repairing relatively recent missed dependencies.
It is not meant to replace mechanisms for long-term consistency,
such as peer-to-peer syncing or the use of a high-availability centralised cache (Store node).
#### SDS-R message fields
SDS-R adds the following fields to SDS messages:
* `sender_id` in `HistoryEntry`:
the original message sender's participant ID.
This is used to determine the group of participants who will respond to a repair request.
* `repair_request` in `Message`:
a capped list of history entries missing for the message sender
and for which it's requesting a repair.
#### SDS-R participant state
SDS-R adds the following to each participant state:
* Outgoing **repair request buffer**:
a list of locally missing `HistoryEntry`s
each mapped to a future request timestamp, `T_req`,
after which this participant will request a repair if at that point the missing dependency has not been repaired yet.
`T_req` is computed as a pseudorandom backoff from the timestamp when the dependency was detected missing.
[Determining `T_req`](#determine-t_req) is described below.
We RECOMMEND that the outgoing repair request buffer be chronologically ordered in ascending order of `T_req`.
- Incoming **repair request buffer**:
a list of locally available `HistoryEntry`s
that were requested for repair by a remote participant
AND for which this participant might be an eligible responder,
each mapped to a future response timestamp, `T_resp`,
after which this participant will rebroadcast the corresponding requested `Message` if at that point no other participant had rebroadcast the `Message`.
`T_resp` is computed as a pseudorandom backoff from the timestamp when the repair was first requested.
[Determining `T_resp`](#determine-t_resp) is described below.
We describe below how a participant can [determine if they're an eligible responder](#determine-response-group) for a specific repair request.
- Augmented local history log:
for each message ID kept in the local log for which the participant could be a repair responder,
the full SDS `Message` must be cached rather than just the message ID,
in case this participant is called upon to rebroadcast the message.
We describe below how a participant can [determine if they're an eligible responder](#determine-response-group) for a specific message.
**_Note:_** The required state can likely be significantly reduced in future by simply requiring that a responding participant should _reconstruct_ the original `Message` when rebroadcasting, rather than the simpler, but heavier, requirement of caching the entire received `Message` content in local history.
#### SDS-R global state
For a specific channel (that is, within a specific SDS-controlled communication)
the following SDS-R configuration state SHOULD be common for all participants in the conversation:
* `T_min`: the _minimum_ time period to wait before a missing causal entry can be repaired.
We RECOMMEND a value of at least 30 seconds.
* `T_max`: the _maximum_ time period over which missing causal entries can be repaired.
We RECOMMEND a value of between 120 and 600 seconds.
Furthermore, to avoid a broadcast storm with multiple participants responding to a repair request,
participants in a single channel MAY be divided into discrete response groups.
Participants will only respond to a repair request if they are in the response group for that request.
The global `num_response_groups` variable configures the number of response groups for this communication.
Its use is described below.
A reasonable default value for `num_response_groups` is one response group for every `128` participants.
In other words, if the (roughly) expected number of participants is expressed as `num_participants`, then
`num_response_groups = num_participants div 128 + 1`.
In other words, if there are fewer than 128 participants in a communication,
they will all belong to the same response group.
We RECOMMEND that the global state variables `T_min`, `T_max` and `num_response_groups` be set _statically_ for a specific SDS-R application,
based on expected number of group participants and volume of traffic.
**_Note:_** Future versions of this protocol will recommend dynamic global SDS-R variables, based on the current number of participants.
#### SDS-R send message
SDS-R adds the following steps when sending a message:
Before broadcasting a message,
* the participant SHOULD populate the `repair_request` field in the message
with _eligible_ entries from the outgoing repair request buffer.
An entry is eligible to be included in a `repair_request`
if its corresponding request timestamp, `T_req`, has expired (in other words, `T_req <= current_time`).
The maximum number of repair request entries to include is up to the application.
We RECOMMEND that this quota be filled by the eligible entries from the outgoing repair request buffer with the lowest `T_req`.
We RECOMMEND a maximum of 3 entries.
If there are no eligible entries in the buffer, this optional field MUST be left unset.
#### SDS-R receive message
On receiving a message,
* the participant MUST remove entries matching the received message ID from its _outgoing_ repair request buffer.
This ensures that the participant does not request repairs for dependencies that have now been met.
* the participant MUST remove entries matching the received message ID from its _incoming_ repair request buffer.
This ensures that the participant does not respond to repair requests that another participant has already responded to.
* the participant SHOULD add any unmet causal dependencies to its outgoing repair request buffer against a unique `T_req` timestamp for that entry.
It MUST compute the `T_req` for each such HistoryEntry according to the steps outlined in [_Determine T_req_](#determine-t_req).
* for each item in the `repair_request` field:
- the participant MUST remove entries matching the repair message ID from its own outgoing repair request buffer.
This limits the number of participants that will request a common missing dependency.
- if the participant has the requested `Message` in its local history _and_ is an eligible responder for the repair request,
it SHOULD add the request to its incoming repair request buffer against a unique `T_resp` timestamp for that entry.
It MUST compute the `T_resp` for each such repair request according to the steps outlined in [_Determine T_resp_](#determine-t_resp).
It MUST determine if it's an eligible responder for a repair request according to the steps outlined in [_Determine response group_](#determine-response-group).
#### Determine T_req
A participant determines the repair request timestamp, `T_req`,
for a missing `HistoryEntry` as follows:
```
T_req = current_time + hash(participant_id, message_id) % (T_max - T_min) + T_min
```
where `current_time` is the current timestamp,
`participant_id` is the participant's _own_ participant ID (not the `sender_id` in the missing `HistoryEntry`),
`message_id` is the missing `HistoryEntry`'s message ID,
and `T_min` and `T_max` are as set out in [SDS-R global state](#sds-r-global-state).
This allows `T_req` to be pseudorandomly and linearly distributed as a backoff of between `T_min` and `T_max` from current time.
> **_Note:_** placing `T_req` values on an exponential backoff curve will likely be more appropriate and is left for a future improvement.
#### Determine T_resp
A participant determines the repair response timestamp, `T_resp`,
for a `HistoryEntry` that it could repair as follows:
```
distance = hash(participant_id) XOR hash(sender_id)
T_resp = current_time + distance*hash(message_id) % T_max
```
where `current_time` is the current timestamp,
`participant_id` is the participant's _own_ (local) participant ID,
`sender_id` is the requested `HistoryEntry` sender ID,
`message_id` is the requested `HistoryEntry` message ID,
and `T_max` is as set out in [SDS-R global state](#sds-r-global-state).
We first calculate the logical `distance` between the local `participant_id` and the original `sender_id`.
If this participant is the original sender, the `distance` will be `0`.
It should then be clear that the original participant will have a response backoff time of `0`, making it the most likely responder.
The `T_resp` values for other eligible participants will be pseudorandomly and linearly distributed as a backoff of up to `T_max` from current time.
> **_Note:_** placing `T_resp` values on an exponential backoff curve will likely be more appropriate and is left for a future improvement.
#### Determine response group
Given a message with `sender_id` and `message_id`,
a participant with `participant_id` is in the response group for that message if
```
hash(participant_id, message_id) % num_response_groups == hash(sender_id, message_id) % num_response_groups
```
where `num_response_groups` is as set out in [SDS-R global state](#sds-r-global-state).
This ensures that a participant will always be in the response group for its own published messages.
It also allows participants to determine immediately on first reception of a message or a history entry
if they are in the associated response group.
#### SDS-R incoming repair request buffer sweep
An SDS-R participant MUST periodically check if there are any incoming requests in the *incoming repair request buffer* that is due for a response.
For each item in the buffer,
the participant SHOULD broadcast the corresponding `Message` from local history
if its corresponding response timestamp, `T_resp`, has expired (in other words, `T_resp <= current_time`).
#### SDS-R Periodic Sync Message
If the participant is due to send a periodic sync message,
it SHOULD send the message according to [SDS-R send message](#sds-r-send-message)
if there are any eligible items in the outgoing repair request buffer,
regardless of whether other participants have also recently broadcast a Periodic Sync message.
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).

189
waku/standards/core/31/enr.md Normal file
View File

@@ -0,0 +1,189 @@
---
slug: 31
title: 31/WAKU2-ENR
name: Waku v2 usage of ENR
status: draft
tags: [waku/core-protocol]
editor: Franck Royer <franck@status.im>
contributors:
---
## Abstract
This specification describes the usage of the ENR (Ethereum Node Records)
format for [10/WAKU2](../10/waku2.md) purposes.
The ENR format is defined in [EIP-778](https://eips.ethereum.org/EIPS/eip-778) [[3]](#references).
This specification is an extension of EIP-778,
ENR used in Waku MUST adhere to both EIP-778 and 31/WAKU2-ENR.
## Motivation
EIP-1459 with the usage of ENR has been implemented [[1]](#references) [[2]](#references) as a discovery protocol for Waku.
EIP-778 specifies a number of pre-defined keys.
However, the usage of these keys alone does not allow for certain transport capabilities to be encoded,
such as Websocket.
Currently, Waku nodes running in a browser only support websocket transport protocol.
Hence, new ENR keys need to be defined to allow for the encoding of transport protocol other than raw TCP.
### Usage of Multiaddr Format Rationale
One solution would be to define new keys such as `ws` to encode the websocket port of a node.
However, we expect new transport protocols to be added overtime such as quic.
Hence, this would only provide a short term solution until another specification would need to be added.
Moreover, secure websocket involves SSL certificates.
SSL certificates are only valid for a given domain and ip,
so an ENR containing the following information:
- secure websocket port
- ipv4 fqdn
- ipv4 address
- ipv6 address
Would carry some ambiguity: Is the certificate securing the websocket port valid for the ipv4 fqdn?
the ipv4 address?
the ipv6 address?
The [10/WAKU2](../10/waku2.md) protocol family is built on the [libp2p](https://github.com/libp2p/specs) protocol stack.
Hence, it uses [multiaddr](https://github.com/multiformats/multiaddr) to format network addresses.
Directly storing one or several multiaddresses in the ENR would fix the issues listed above:
- multiaddr is self-describing and support addresses for any network protocol:
No new specification would be needed to support encoding other transport protocols in an ENR.
- multiaddr contains both the host and port information,
allowing the ambiguity previously described to be resolved.
## Wire Format
### `multiaddrs` ENR key
We define a `multiaddrs` key.
- The value MUST be a list of binary encoded multiaddr prefixed by their size.
- The size of the multiaddr MUST be encoded in a Big Endian unsigned 16-bit integer.
- The size of the multiaddr MUST be encoded in 2 bytes.
- The `secp256k1` value MUST be present on the record;
`secp256k1` is defined in [EIP-778](https://eips.ethereum.org/EIPS/eip-778) and
contains the compressed secp256k1 public key.
- The node's peer id SHOULD be deduced from the `secp256k1` value.
- The multiaddresses SHOULD NOT contain a peer id except for circuit relay addresses
- For raw TCP & UDP connections details,
[EIP-778](https://eips.ethereum.org/EIPS/eip-778) pre-defined keys SHOULD be used;
The keys `tcp`, `udp`, `ip` (and `tcp6`, `udp6`, `ip6` for IPv6)
are enough to convey all necessary information;
- To save space, `multiaddrs` key SHOULD only be used for connection details that cannot be represented using the [EIP-778](https://eips.ethereum.org/EIPS/eip-778) pre-defined keys.
- The 300 bytes size limit as defined by [EIP-778](https://eips.ethereum.org/EIPS/eip-778) still applies;
In practice, it is possible to encode 3 multiaddresses in ENR, more or
less could be encoded depending on the size of each multiaddress.
### Usage
#### Many connection types
Alice is a Waku node operator, she runs a node that supports inbound connection for the following protocols:
- TCP 10101 on `1.2.3.4`
- UDP 20202 on `1.2.3.4`
- TCP 30303 on `1234:5600:101:1::142`
- UDP 40404 on `1234:5600:101:1::142`
- Secure Websocket on `wss://example.com:443/`
- QUIC on `quic://quic.example.com:443/`
- A circuit relay address `/ip4/1.2.3.4/tcp/55555/p2p/QmRelay/p2p-circuit/p2p/QmAlice`
Alice SHOULD structure the ENR for her node as follows:
| key | value |
| ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `tcp` | `10101` |
| `udp` | `20202` |
| `tcp6` | `30303` |
| `udp6` | `40404` |
| `ip` | `1.2.3.4` |
| `ip6` | `1234:5600:101:1::142` |
| `secp256k1` | Alice's compressed secp256k1 public key, 33 bytes |
| `multiaddrs` | `len1 \| /dns4/example.com/tcp/443/wss \| len2 \| /dns4/quic.examle.com/tcp/443/quic \| len3 \| /ip4/1.2.3.4/tcp/55555/p2p/QmRelay` |
Where `multiaddrs`:
- `|` is the concatenation operator,
- `len1` is the length of `/dns4/example.com/tcp/443/wss` byte representation,
- `len2` is the length of `/dns4/quic.examle.com/tcp/443/quic` byte representation.
- `len3` is the length of `/ip4/1.2.3.4/tcp/55555/p2p/QmRelay` byte representation.
Notice that the `/p2p-circuit` component is not stored, but,
since circuit relay addresses are the only one containing a `p2p` component,
it's safe to assume that any address containing this component is a circuit relay address.
Decoding this type of multiaddresses would require appending the `/p2p-circuit` component.
#### Raw TCP only
Bob is a node operator that runs a node that supports inbound connection for the following protocols:
- TCP 10101 on `1.2.3.4`
Bob SHOULD structure the ENR for his node as follows:
| key | value |
| ----------- | ----------------------------------------------- |
| `tcp` | `10101` |
| `ip` | `1.2.3.4` |
| `secp256k1` | Bob's compressed secp256k1 public key, 33 bytes |
As Bob's node's connection details can be represented with EIP-778's pre-defined keys only,
it is not needed to use the `multiaddrs` key.
### Limitations
Supported key type is `secp256k1` only.
Support for other elliptic curve cryptography such as `ed25519` MAY be used.
### `waku2` ENR key
We define a `waku2` field key:
- The value MUST be an 8-bit flag field,
where bits set to `1` indicate `true` and
bits set to `0` indicate `false` for the relevant flags.
- The flag values already defined are set out below,
with `bit 7` the most significant bit and `bit 0` the least significant bit.
| bit 7 | bit 6 | bit 5 | bit 4 | bit 3 | bit 2 | bit 1 | bit 0 |
| ------- | ------- | ------- | ------- | ----------- | -------- | ------- | ------- |
| `undef` | `undef` | `undef` | `sync` | `lightpush` | `filter` | `store` | `relay` |
- In the scheme above, the flags `sync`, `lightpush`, `filter`, `store` and
`relay` correlates with support for protocols with the same name.
If a protocol is not supported, the corresponding field MUST be set to `false`.
Indicating positive support for any specific protocol is OPTIONAL,
though it MAY be required by the relevant application or discovery process.
- Flags marked as `undef` is not yet defined.
These SHOULD be set to `false` by default.
### Key Usage
- A Waku node MAY choose to populate the `waku2` field for enhanced discovery capabilities,
such as indicating supported protocols.
Such a node MAY indicate support for any specific protocol by setting the corresponding flag to `true`.
- Waku nodes that want to participate in [Node Discovery Protocol v5](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/33/discv5.md) [[4]](#references), however,
MUST implement the `waku2` key with at least one flag set to `true`.
- Waku nodes that discovered other participants using Discovery v5,
MUST filter out participant records that do not implement this field or
do not have at least one flag set to `true`.
- In addition, such nodes MAY choose to filter participants on specific flags
(such as supported protocols),
or further interpret the `waku2` field as required by the application.
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## References
- [1](../10/waku2.md)
- [2](https://github.com/status-im/nim-waku/pull/690)
- [3](https://github.com/vacp2p/rfc/issues/462#issuecomment-943869940)
- [4](https://eips.ethereum.org/EIPS/eip-778)
- [5](https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md)