mirror of
https://github.com/vacp2p/rfc-index.git
synced 2026-01-09 23:58:02 -05:00
Compare commits
11 Commits
codex/raw/
...
erasure-co
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
abaf357e1b | ||
|
|
4fa5cdbef7 | ||
|
|
4d4f565204 | ||
|
|
dd397adc59 | ||
|
|
cb4d0de84f | ||
|
|
69802377a8 | ||
|
|
950c39ec8b | ||
|
|
e1b28523bf | ||
|
|
e4f5f28ea3 | ||
|
|
171e934d61 | ||
|
|
c5df59a80e |
160
codex/raw/erasure-coding.md
Normal file
160
codex/raw/erasure-coding.md
Normal file
@@ -0,0 +1,160 @@
|
||||
---
|
||||
title: CODEX-ERASUE-CODING
|
||||
name: Codex Erasue Coding
|
||||
status: raw
|
||||
tags: codex
|
||||
editor:
|
||||
contributors:
|
||||
---
|
||||
|
||||
## Abstract
|
||||
|
||||
This specification describes the erasure coding technique used by Codex clients.
|
||||
A Codex client will encode a dataset before it is stored on the network.
|
||||
|
||||
## Background
|
||||
|
||||
The Codex protocol uses storage proofs to verify whether a storage provider (SP) is storing a certain dataset.
|
||||
Before a dataset is retrieved on the network,
|
||||
SPs must agree to store the dataset for a certain period of time.
|
||||
When a storage request is active,
|
||||
erasure coding helps ensure the dataset is retrievable from the network.
|
||||
This is achieved by the dataset that is chunked,
|
||||
which is restored in retrieval by erasure coding.
|
||||
When data blocks are abandoned by storage providers,
|
||||
the requester can be assured of data retrievability.
|
||||
|
||||
## Specification
|
||||
|
||||
The keywords “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”,
|
||||
“SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and
|
||||
“OPTIONAL” in this document are to be interpreted as described in [2119](https://www.ietf.org/rfc/rfc2119.txt).
|
||||
|
||||
A client SHOULD perform the erasure encoding locally before providing a dataset to the network.
|
||||
During validation, nodes will conduct error correction and decoding based on the erasure coding technique known to the network.
|
||||
Datasets using encodings not recognized by the network MAY be ignored during decoding and
|
||||
validation by other nodes in the network.
|
||||
|
||||
The dataset SHOULD be split into data chunks represented by `k`, e.g. $(k_1, k_2, k_3, \ldots, k_{n})$.
|
||||
Each chunk `k` MUST be encoded into `n` blocks, using an erasure encoding technique like the Reed Solomon algorithm.
|
||||
Including a set of parity blocks that MUST be generated,
|
||||
represented by `m`.
|
||||
All node roles on the Codex network use the [Leopard Codec](https://github.com/catid/leopard).
|
||||
|
||||
Below is the encoding process:
|
||||
|
||||
1. Prepare the dataset for the marketplace using erasure encoding.
|
||||
2. Derive a manifest CID from the root encoded blocks
|
||||
3. Error correction by validator nodes once the storage contract begins
|
||||
4. Decode data back to the original data.
|
||||
|
||||
### Encoding
|
||||
|
||||
A client MAY prepare a dataset locally before making the request to the network.
|
||||
The data chunks, `k`, MUST be the same size, if not,
|
||||
the smaller chunk MAY be padded with empty data.
|
||||
|
||||
The data blocks are encoded based on the following parameters:
|
||||
|
||||
```js
|
||||
|
||||
struct encodingParms {
|
||||
ecK: int, // Number of data blocks (K)
|
||||
ecM: int, // Number of parity blocks (M)
|
||||
rounded: int, // Dataset rounded to multiple of (K)
|
||||
steps: int, // Number of encoding iterations (steps)
|
||||
blocksCount: int, // Total blocks after encoding
|
||||
strategy: enum, // Indexing strategy used
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
After the erasure coding process,
|
||||
a protected manifest SHOULD be generated for the dataset, which would store the CID of the root Merkle tree.
|
||||
The content of the protected manifest below, see [CODEX-MANIFEST](./manifest.md) for more information:
|
||||
|
||||
```js
|
||||
|
||||
syntax = "proto3";
|
||||
|
||||
message verifiable {
|
||||
string verifyRoot = 1 // Root of verification tree with CID
|
||||
repeated string slot_roots = 2 // List Individual slot roots with CID
|
||||
uint32 cellSize = 3 // Size of verification cells
|
||||
string verifiableStrategy = 4 // Strategy for verification
|
||||
}
|
||||
|
||||
message ErasureInfo {
|
||||
optional uint32 ecK = 1; // number of encoded blocks
|
||||
optional uint32 ecM = 2; // number of parity blocks
|
||||
optional bytes originalTreeCid = 3; // cid of the original dataset
|
||||
optional uint32 originalDatasetSize = 4; // size of the original dataset
|
||||
optional VerificationInformation verification = 5; // verification information
|
||||
}
|
||||
|
||||
message Manifest {
|
||||
optional bytes treeCid = 1; // cid (root) of the tree
|
||||
optional uint32 blockSize = 2; // size of a single block
|
||||
optional uint64 datasetSize = 3; // size of the dataset
|
||||
optional codec: MultiCodec = 4; // Dataset codec
|
||||
optional hcodec: MultiCodec = 5 // Multihash codec
|
||||
optional version: CidVersion = 6; // Cid version
|
||||
optional ErasureInfo erasure = 7; // erasure coding info
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
After the encoding process,
|
||||
is ready to be stored on the network via the [CODEX-MARKETPLACE](./marketplace.md).
|
||||
The Merkle tree root SHOULD be included in the manifest so other nodes are able to locate and
|
||||
reconstruct a dataset from the erasure encoded blocks.
|
||||
|
||||
### Data Repair
|
||||
|
||||
Storage providers may have periods during a storage contract where they are not storing the data.
|
||||
A validator node MAY store the `treeCid` from the `Manifest` to locate all the data blocks and
|
||||
reconstruct the merkle tree.
|
||||
When a missing branch of the tree is not retrievable from an SP, data repair will be REQUIRED.
|
||||
The validator will open a request for a new SP to reconstruct the Merkle tree and
|
||||
store the missing data blocks.
|
||||
The validator role is described in the [CODEX-MARKETPLACE](./marketplace.md) specification.
|
||||
|
||||
### Decode Data
|
||||
|
||||
During dataset retrieval, a node will use the `treeCid` to locate the data blocks.
|
||||
The number of retrieved blocks by the node MUST be greater than `k`.
|
||||
If less than `k`, the node MAY not be able to reconstruct the dataset.
|
||||
The node SHOULD request missing data chunks from the network and
|
||||
wait until the threshold is reached.
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### Adversarial Attack
|
||||
|
||||
An adversarial storage provider can remove only the first element from more than half of the block,
|
||||
and the slot data can no longer be recovered from the data that the host stores.
|
||||
For example, with data blocks of size 1TB, erasure coded into 256 data and parity shards.
|
||||
An adversary could strategically remove 129 bytes, and
|
||||
the data can no longer be fully recovered with the erasure-coded data that is present on the host.
|
||||
|
||||
The RECOMMENDED solution should perform checks on entire shards to protect against adversarial erasure.
|
||||
In the Merkle storage proofs, the entire shard SHOULD be hashed,
|
||||
then that hash is checked against the Merkle proof.
|
||||
Effectively, the block size for Merkle proofs should equal the shard size of the erasure coding interleaving.
|
||||
Hashing large amounts of data will be expensive to perform in an SNARK, which is used to compress proofs in size in Codex.
|
||||
|
||||
### Data Encryption
|
||||
|
||||
If data is not encrypted before entering the encoding process, nodes, including storage providers,
|
||||
MAY be able to access the data.
|
||||
This may lead to privacy concerns and the misuse of data.
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
## References
|
||||
|
||||
- [Leapard Codec](https://github.com/catid/leopard)
|
||||
- [CODEX-MANIFEST](./manifest.md)
|
||||
- [CODEX-MARKETPLACE](./marketplace.md)
|
||||
@@ -40,7 +40,7 @@ Request For Comments specification process managed by the Vac service department
|
||||
|
||||
## License
|
||||
|
||||
Copyright (c) 2008-24 the Editor and Contributors.
|
||||
Copyright (c) 2008-26 the Editor and Contributors.
|
||||
|
||||
This Specification is free software;
|
||||
you can redistribute it and/or
|
||||
|
||||
205
vac/raw/sds.md
205
vac/raw/sds.md
@@ -58,6 +58,9 @@ other participants using the corresponding message ID.
|
||||
* **Participant ID:**
|
||||
Each participant has a globally unique, immutable ID
|
||||
visible to other participants in the communication.
|
||||
* **Sender ID:**
|
||||
The **Participant ID** of the original sender of a message,
|
||||
often coupled with a **Message ID**.
|
||||
|
||||
## Wire protocol
|
||||
|
||||
@@ -75,6 +78,8 @@ syntax = "proto3";
|
||||
message HistoryEntry {
|
||||
string message_id = 1; // Unique identifier of the SDS message, as defined in `Message`
|
||||
optional bytes retrieval_hint = 2; // Optional information to help remote parties retrieve this SDS message; For example, A Waku deterministic message hash or routing payload hash
|
||||
|
||||
optional string sender_id = 3; // Participant ID of original message sender. Only populated if using optional SDS Repair extension
|
||||
}
|
||||
|
||||
message Message {
|
||||
@@ -84,6 +89,9 @@ message Message {
|
||||
optional uint64 lamport_timestamp = 10; // Logical timestamp for causal ordering in channel
|
||||
repeated HistoryEntry causal_history = 11; // List of preceding message IDs that this message causally depends on. Generally 2 or 3 message IDs are included.
|
||||
optional bytes bloom_filter = 12; // Bloom filter representing received message IDs in channel
|
||||
|
||||
repeated HistoryEntry repair_request = 13; // Capped list of history entries missing from sender's causal history. Only populated if using the optional SDS Repair extension.
|
||||
|
||||
optional bytes content = 20; // Actual content of the message
|
||||
}
|
||||
```
|
||||
@@ -102,9 +110,11 @@ These fields MAY be left unset in the case of [ephemeral messages](#ephemeral-me
|
||||
The message `content` MAY be left empty for [periodic sync messages](#periodic-sync-message),
|
||||
otherwise it MUST contain the application-level content
|
||||
|
||||
> **_Note:_** Close readers may notice that, outside of filtering messages originating from the sender itself,
|
||||
> **_Note:_** Close readers may notice that,
|
||||
outside of filtering messages originating from the sender itself,
|
||||
the `sender_id` field is not used for much.
|
||||
Its importance is expected to increase once a p2p retrieval mechanism is added to SDS, as is planned for the protocol.
|
||||
Its importance is expected to increase once a p2p retrieval mechanism is added to SDS,
|
||||
as is planned for the protocol.
|
||||
|
||||
### Participant state
|
||||
|
||||
@@ -289,6 +299,197 @@ Upon reception,
|
||||
ephemeral messages SHOULD be delivered immediately without buffering for causal dependencies
|
||||
or including in the local log.
|
||||
|
||||
### SDS Repair (SDS-R)
|
||||
|
||||
SDS Repair (SDS-R) is an optional extension module for SDS,
|
||||
allowing participants in a communication to collectively repair any gaps in causal history (missing messages)
|
||||
preferably over a limited time window.
|
||||
Since SDS-R acts as coordinated rebroadcasting of missing messages,
|
||||
which involves all participants of the communication,
|
||||
it is most appropriate in a limited use case for repairing relatively recent missed dependencies.
|
||||
It is not meant to replace mechanisms for long-term consistency,
|
||||
such as peer-to-peer syncing or the use of a high-availability centralised cache (Store node).
|
||||
|
||||
#### SDS-R message fields
|
||||
|
||||
SDS-R adds the following fields to SDS messages:
|
||||
|
||||
* `sender_id` in `HistoryEntry`:
|
||||
the original message sender's participant ID.
|
||||
This is used to determine the group of participants who will respond to a repair request.
|
||||
* `repair_request` in `Message`:
|
||||
a capped list of history entries missing for the message sender
|
||||
and for which it's requesting a repair.
|
||||
|
||||
#### SDS-R participant state
|
||||
|
||||
SDS-R adds the following to each participant state:
|
||||
|
||||
* Outgoing **repair request buffer**:
|
||||
a list of locally missing `HistoryEntry`s
|
||||
each mapped to a future request timestamp, `T_req`,
|
||||
after which this participant will request a repair if at that point the missing dependency has not been repaired yet.
|
||||
`T_req` is computed as a pseudorandom backoff from the timestamp when the dependency was detected missing.
|
||||
[Determining `T_req`](#determine-t_req) is described below.
|
||||
We RECOMMEND that the outgoing repair request buffer be chronologically ordered in ascending order of `T_req`.
|
||||
|
||||
* Incoming **repair request buffer**:
|
||||
a list of locally available `HistoryEntry`s
|
||||
that were requested for repair by a remote participant
|
||||
AND for which this participant might be an eligible responder,
|
||||
each mapped to a future response timestamp, `T_resp`,
|
||||
after which this participant will rebroadcast the corresponding requested `Message` if at that point no other participant had rebroadcast the `Message`.
|
||||
`T_resp` is computed as a pseudorandom backoff from the timestamp when the repair was first requested.
|
||||
[Determining `T_resp`](#determine-t_resp) is described below.
|
||||
We describe below how a participant can [determine if they're an eligible responder](#determine-response-group) for a specific repair request.
|
||||
|
||||
* Augmented local history log:
|
||||
for each message ID kept in the local log for which the participant could be a repair responder,
|
||||
the full SDS `Message` must be cached rather than just the message ID,
|
||||
in case this participant is called upon to rebroadcast the message.
|
||||
We describe below how a participant can [determine if they're an eligible responder](#determine-response-group) for a specific message.
|
||||
|
||||
**_Note:_** The required state can likely be significantly reduced in future by simply requiring that a responding participant should _reconstruct_ the original `Message` when rebroadcasting, rather than the simpler, but heavier,
|
||||
requirement of caching the entire received `Message` content in local history.
|
||||
|
||||
#### SDS-R global state
|
||||
|
||||
For a specific channel (that is, within a specific SDS-controlled communication)
|
||||
the following SDS-R configuration state SHOULD be common for all participants in the conversation:
|
||||
|
||||
* `T_min`: the _minimum_ time period to wait before a missing causal entry can be repaired.
|
||||
We RECOMMEND a value of at least 30 seconds.
|
||||
* `T_max`: the _maximum_ time period over which missing causal entries can be repaired.
|
||||
We RECOMMEND a value of between 120 and 600 seconds.
|
||||
|
||||
Furthermore, to avoid a broadcast storm with multiple participants responding to a repair request,
|
||||
participants in a single channel MAY be divided into discrete response groups.
|
||||
Participants will only respond to a repair request if they are in the response group for that request.
|
||||
The global `num_response_groups` variable configures the number of response groups for this communication.
|
||||
Its use is described below.
|
||||
A reasonable default value for `num_response_groups` is one response group for every `128` participants.
|
||||
In other words, if the (roughly) expected number of participants is expressed as `num_participants`, then
|
||||
`num_response_groups = num_participants div 128 + 1`.
|
||||
In other words, if there are fewer than 128 participants in a communication,
|
||||
they will all belong to the same response group.
|
||||
|
||||
We RECOMMEND that the global state variables `T_min`, `T_max` and `num_response_groups`
|
||||
be set _statically_ for a specific SDS-R application,
|
||||
based on expected number of group participants and volume of traffic.
|
||||
|
||||
**_Note:_** Future versions of this protocol will recommend dynamic global SDS-R variables,
|
||||
based on the current number of participants.
|
||||
|
||||
#### SDS-R send message
|
||||
|
||||
SDS-R adds the following steps when sending a message:
|
||||
|
||||
Before broadcasting a message,
|
||||
|
||||
* the participant SHOULD populate the `repair_request` field in the message
|
||||
with _eligible_ entries from the outgoing repair request buffer.
|
||||
An entry is eligible to be included in a `repair_request`
|
||||
if its corresponding request timestamp, `T_req`, has expired (in other words,
|
||||
`T_req <= current_time`).
|
||||
The maximum number of repair request entries to include is up to the application.
|
||||
We RECOMMEND that this quota be filled by the eligible entries from the outgoing repair request buffer with the lowest `T_req`.
|
||||
We RECOMMEND a maximum of 3 entries.
|
||||
If there are no eligible entries in the buffer,
|
||||
this optional field MUST be left unset.
|
||||
|
||||
#### SDS-R receive message
|
||||
|
||||
On receiving a message,
|
||||
|
||||
* the participant MUST remove entries matching the received message ID from its _outgoing_ repair request buffer.
|
||||
This ensures that the participant does not request repairs for dependencies that have now been met.
|
||||
* the participant MUST remove entries matching the received message ID from its _incoming_ repair request buffer.
|
||||
This ensures that the participant does not respond to repair requests that another participant has already responded to.
|
||||
* the participant SHOULD add any unmet causal dependencies to its outgoing repair request buffer against a unique `T_req` timestamp for that entry.
|
||||
It MUST compute the `T_req` for each such HistoryEntry according to the steps outlined in [_Determine T_req_](#determine-t_req).
|
||||
* for each item in the `repair_request` field:
|
||||
* the participant MUST remove entries matching the repair message ID from its own outgoing repair request buffer.
|
||||
This limits the number of participants that will request a common missing dependency.
|
||||
* if the participant has the requested `Message` in its local history _and_ is an eligible responder for the repair request,
|
||||
it SHOULD add the request to its incoming repair request buffer against a unique `T_resp` timestamp for that entry.
|
||||
It MUST compute the `T_resp` for each such repair request according to the steps outlined in [_Determine T_resp_](#determine-t_resp).
|
||||
It MUST determine if it's an eligible responder for a repair request according to the steps outlined in [_Determine response group_](#determine-response-group).
|
||||
|
||||
#### Determine T_req
|
||||
|
||||
A participant determines the repair request timestamp, `T_req`,
|
||||
for a missing `HistoryEntry` as follows:
|
||||
|
||||
```text
|
||||
T_req = current_time + hash(participant_id, message_id) % (T_max - T_min) + T_min
|
||||
```
|
||||
|
||||
where `current_time` is the current timestamp,
|
||||
`participant_id` is the participant's _own_ participant ID
|
||||
(not the `sender_id` in the missing `HistoryEntry`),
|
||||
`message_id` is the missing `HistoryEntry`'s message ID,
|
||||
and `T_min` and `T_max` are as set out in [SDS-R global state](#sds-r-global-state).
|
||||
|
||||
This allows `T_req` to be pseudorandomly and linearly distributed as a backoff of between `T_min` and `T_max` from current time.
|
||||
|
||||
> **_Note:_** placing `T_req` values on an exponential backoff curve will likely be more appropriate and is left for a future improvement.
|
||||
|
||||
#### Determine T_resp
|
||||
|
||||
A participant determines the repair response timestamp, `T_resp`,
|
||||
for a `HistoryEntry` that it could repair as follows:
|
||||
|
||||
```text
|
||||
distance = hash(participant_id) XOR hash(sender_id)
|
||||
T_resp = current_time + distance*hash(message_id) % T_max
|
||||
```
|
||||
|
||||
where `current_time` is the current timestamp,
|
||||
`participant_id` is the participant's _own_ (local) participant ID,
|
||||
`sender_id` is the requested `HistoryEntry` sender ID,
|
||||
`message_id` is the requested `HistoryEntry` message ID,
|
||||
and `T_max` is as set out in [SDS-R global state](#sds-r-global-state).
|
||||
|
||||
We first calculate the logical `distance` between the local `participant_id` and
|
||||
the original `sender_id`.
|
||||
If this participant is the original sender, the `distance` will be `0`.
|
||||
It should then be clear that the original participant will have a response backoff time of `0`,
|
||||
making it the most likely responder.
|
||||
The `T_resp` values for other eligible participants will be pseudorandomly and
|
||||
linearly distributed as a backoff of up to `T_max` from current time.
|
||||
|
||||
> **_Note:_** placing `T_resp` values on an exponential backoff curve will likely be more appropriate and
|
||||
is left for a future improvement.
|
||||
|
||||
#### Determine response group
|
||||
|
||||
Given a message with `sender_id` and `message_id`,
|
||||
a participant with `participant_id` is in the response group for that message if
|
||||
|
||||
```text
|
||||
hash(participant_id, message_id) % num_response_groups == hash(sender_id, message_id) % num_response_groups
|
||||
```
|
||||
|
||||
where `num_response_groups` is as set out in [SDS-R global state](#sds-r-global-state).
|
||||
This ensures that a participant will always be in the response group for its own published messages.
|
||||
It also allows participants to determine immediately on first reception of a message or
|
||||
a history entry if they are in the associated response group.
|
||||
|
||||
#### SDS-R incoming repair request buffer sweep
|
||||
|
||||
An SDS-R participant MUST periodically check if there are any incoming requests in the **incoming** repair request buffer* that is due for a response.
|
||||
For each item in the buffer,
|
||||
the participant SHOULD broadcast the corresponding `Message` from local history
|
||||
if its corresponding response timestamp, `T_resp`, has expired
|
||||
(in other words, `T_resp <= current_time`).
|
||||
|
||||
#### SDS-R Periodic Sync Message
|
||||
|
||||
If the participant is due to send a periodic sync message,
|
||||
it SHOULD send the message according to [SDS-R send message](#sds-r-send-message)
|
||||
if there are any eligible items in the outgoing repair request buffer,
|
||||
regardless of whether other participants have also recently broadcast a Periodic Sync message.
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
slug: 21
|
||||
title: 21/WAKU2-FAULT-TOLERANT-STORE
|
||||
name: Waku v2 Fault-Tolerant Store
|
||||
status: draft
|
||||
status: deleted
|
||||
editor: Sanaz Taheri <sanaz@status.im>
|
||||
contributors:
|
||||
---
|
||||
189
waku/standards/core/31/enr.md
Normal file
189
waku/standards/core/31/enr.md
Normal file
@@ -0,0 +1,189 @@
|
||||
---
|
||||
slug: 31
|
||||
title: 31/WAKU2-ENR
|
||||
name: Waku v2 usage of ENR
|
||||
status: draft
|
||||
tags: [waku/core-protocol]
|
||||
editor: Franck Royer <franck@status.im>
|
||||
contributors:
|
||||
---
|
||||
|
||||
## Abstract
|
||||
|
||||
This specification describes the usage of the ENR (Ethereum Node Records)
|
||||
format for [10/WAKU2](../10/waku2.md) purposes.
|
||||
The ENR format is defined in [EIP-778](https://eips.ethereum.org/EIPS/eip-778) [[3]](#references).
|
||||
|
||||
This specification is an extension of EIP-778,
|
||||
ENR used in Waku MUST adhere to both EIP-778 and 31/WAKU2-ENR.
|
||||
|
||||
## Motivation
|
||||
|
||||
EIP-1459 with the usage of ENR has been implemented [[1]](#references) [[2]](#references) as a discovery protocol for Waku.
|
||||
|
||||
EIP-778 specifies a number of pre-defined keys.
|
||||
However, the usage of these keys alone does not allow for certain transport capabilities to be encoded,
|
||||
such as Websocket.
|
||||
Currently, Waku nodes running in a browser only support websocket transport protocol.
|
||||
Hence, new ENR keys need to be defined to allow for the encoding of transport protocol other than raw TCP.
|
||||
|
||||
### Usage of Multiaddr Format Rationale
|
||||
|
||||
One solution would be to define new keys such as `ws` to encode the websocket port of a node.
|
||||
However, we expect new transport protocols to be added overtime such as quic.
|
||||
Hence, this would only provide a short term solution until another specification would need to be added.
|
||||
|
||||
Moreover, secure websocket involves SSL certificates.
|
||||
SSL certificates are only valid for a given domain and ip,
|
||||
so an ENR containing the following information:
|
||||
|
||||
- secure websocket port
|
||||
- ipv4 fqdn
|
||||
- ipv4 address
|
||||
- ipv6 address
|
||||
|
||||
Would carry some ambiguity: Is the certificate securing the websocket port valid for the ipv4 fqdn?
|
||||
the ipv4 address?
|
||||
the ipv6 address?
|
||||
|
||||
The [10/WAKU2](../10/waku2.md) protocol family is built on the [libp2p](https://github.com/libp2p/specs) protocol stack.
|
||||
Hence, it uses [multiaddr](https://github.com/multiformats/multiaddr) to format network addresses.
|
||||
|
||||
Directly storing one or several multiaddresses in the ENR would fix the issues listed above:
|
||||
|
||||
- multiaddr is self-describing and support addresses for any network protocol:
|
||||
No new specification would be needed to support encoding other transport protocols in an ENR.
|
||||
- multiaddr contains both the host and port information,
|
||||
allowing the ambiguity previously described to be resolved.
|
||||
|
||||
## Wire Format
|
||||
|
||||
### `multiaddrs` ENR key
|
||||
|
||||
We define a `multiaddrs` key.
|
||||
|
||||
- The value MUST be a list of binary encoded multiaddr prefixed by their size.
|
||||
- The size of the multiaddr MUST be encoded in a Big Endian unsigned 16-bit integer.
|
||||
- The size of the multiaddr MUST be encoded in 2 bytes.
|
||||
- The `secp256k1` value MUST be present on the record;
|
||||
`secp256k1` is defined in [EIP-778](https://eips.ethereum.org/EIPS/eip-778) and
|
||||
contains the compressed secp256k1 public key.
|
||||
- The node's peer id SHOULD be deduced from the `secp256k1` value.
|
||||
- The multiaddresses SHOULD NOT contain a peer id except for circuit relay addresses
|
||||
- For raw TCP & UDP connections details,
|
||||
[EIP-778](https://eips.ethereum.org/EIPS/eip-778) pre-defined keys SHOULD be used;
|
||||
The keys `tcp`, `udp`, `ip` (and `tcp6`, `udp6`, `ip6` for IPv6)
|
||||
are enough to convey all necessary information;
|
||||
- To save space, `multiaddrs` key SHOULD only be used for connection details that cannot be represented using the [EIP-778](https://eips.ethereum.org/EIPS/eip-778) pre-defined keys.
|
||||
- The 300 bytes size limit as defined by [EIP-778](https://eips.ethereum.org/EIPS/eip-778) still applies;
|
||||
In practice, it is possible to encode 3 multiaddresses in ENR, more or
|
||||
less could be encoded depending on the size of each multiaddress.
|
||||
|
||||
### Usage
|
||||
|
||||
#### Many connection types
|
||||
|
||||
Alice is a Waku node operator, she runs a node that supports inbound connection for the following protocols:
|
||||
|
||||
- TCP 10101 on `1.2.3.4`
|
||||
- UDP 20202 on `1.2.3.4`
|
||||
- TCP 30303 on `1234:5600:101:1::142`
|
||||
- UDP 40404 on `1234:5600:101:1::142`
|
||||
- Secure Websocket on `wss://example.com:443/`
|
||||
- QUIC on `quic://quic.example.com:443/`
|
||||
- A circuit relay address `/ip4/1.2.3.4/tcp/55555/p2p/QmRelay/p2p-circuit/p2p/QmAlice`
|
||||
|
||||
Alice SHOULD structure the ENR for her node as follows:
|
||||
|
||||
| key | value |
|
||||
| ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `tcp` | `10101` |
|
||||
| `udp` | `20202` |
|
||||
| `tcp6` | `30303` |
|
||||
| `udp6` | `40404` |
|
||||
| `ip` | `1.2.3.4` |
|
||||
| `ip6` | `1234:5600:101:1::142` |
|
||||
| `secp256k1` | Alice's compressed secp256k1 public key, 33 bytes |
|
||||
| `multiaddrs` | `len1 \| /dns4/example.com/tcp/443/wss \| len2 \| /dns4/quic.examle.com/tcp/443/quic \| len3 \| /ip4/1.2.3.4/tcp/55555/p2p/QmRelay` |
|
||||
|
||||
Where `multiaddrs`:
|
||||
|
||||
- `|` is the concatenation operator,
|
||||
- `len1` is the length of `/dns4/example.com/tcp/443/wss` byte representation,
|
||||
- `len2` is the length of `/dns4/quic.examle.com/tcp/443/quic` byte representation.
|
||||
- `len3` is the length of `/ip4/1.2.3.4/tcp/55555/p2p/QmRelay` byte representation.
|
||||
Notice that the `/p2p-circuit` component is not stored, but,
|
||||
since circuit relay addresses are the only one containing a `p2p` component,
|
||||
it's safe to assume that any address containing this component is a circuit relay address.
|
||||
Decoding this type of multiaddresses would require appending the `/p2p-circuit` component.
|
||||
|
||||
#### Raw TCP only
|
||||
|
||||
Bob is a node operator that runs a node that supports inbound connection for the following protocols:
|
||||
|
||||
- TCP 10101 on `1.2.3.4`
|
||||
|
||||
Bob SHOULD structure the ENR for his node as follows:
|
||||
|
||||
| key | value |
|
||||
| ----------- | ----------------------------------------------- |
|
||||
| `tcp` | `10101` |
|
||||
| `ip` | `1.2.3.4` |
|
||||
| `secp256k1` | Bob's compressed secp256k1 public key, 33 bytes |
|
||||
|
||||
As Bob's node's connection details can be represented with EIP-778's pre-defined keys only,
|
||||
it is not needed to use the `multiaddrs` key.
|
||||
|
||||
### Limitations
|
||||
|
||||
Supported key type is `secp256k1` only.
|
||||
|
||||
Support for other elliptic curve cryptography such as `ed25519` MAY be used.
|
||||
|
||||
### `waku2` ENR key
|
||||
|
||||
We define a `waku2` field key:
|
||||
|
||||
- The value MUST be an 8-bit flag field,
|
||||
where bits set to `1` indicate `true` and
|
||||
bits set to `0` indicate `false` for the relevant flags.
|
||||
- The flag values already defined are set out below,
|
||||
with `bit 7` the most significant bit and `bit 0` the least significant bit.
|
||||
|
||||
| bit 7 | bit 6 | bit 5 | bit 4 | bit 3 | bit 2 | bit 1 | bit 0 |
|
||||
| ------- | ------- | ------- | ------- | ----------- | -------- | ------- | ------- |
|
||||
| `undef` | `undef` | `undef` | `sync` | `lightpush` | `filter` | `store` | `relay` |
|
||||
|
||||
- In the scheme above, the flags `sync`, `lightpush`, `filter`, `store` and
|
||||
`relay` correlates with support for protocols with the same name.
|
||||
If a protocol is not supported, the corresponding field MUST be set to `false`.
|
||||
Indicating positive support for any specific protocol is OPTIONAL,
|
||||
though it MAY be required by the relevant application or discovery process.
|
||||
- Flags marked as `undef` is not yet defined.
|
||||
These SHOULD be set to `false` by default.
|
||||
|
||||
### Key Usage
|
||||
|
||||
- A Waku node MAY choose to populate the `waku2` field for enhanced discovery capabilities,
|
||||
such as indicating supported protocols.
|
||||
Such a node MAY indicate support for any specific protocol by setting the corresponding flag to `true`.
|
||||
- Waku nodes that want to participate in [Node Discovery Protocol v5](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/33/discv5.md) [[4]](#references), however,
|
||||
MUST implement the `waku2` key with at least one flag set to `true`.
|
||||
- Waku nodes that discovered other participants using Discovery v5,
|
||||
MUST filter out participant records that do not implement this field or
|
||||
do not have at least one flag set to `true`.
|
||||
- In addition, such nodes MAY choose to filter participants on specific flags
|
||||
(such as supported protocols),
|
||||
or further interpret the `waku2` field as required by the application.
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
## References
|
||||
|
||||
- [1](../10/waku2.md)
|
||||
- [2](https://github.com/status-im/nim-waku/pull/690)
|
||||
- [3](https://github.com/vacp2p/rfc/issues/462#issuecomment-943869940)
|
||||
- [4](https://eips.ethereum.org/EIPS/eip-778)
|
||||
- [5](https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md)
|
||||
Reference in New Issue
Block a user